Develop Reference

icontains and getlist django python

We are trying to return a list of titles for the Django API, in which the title can have a few keywords.
So for instance, if we use the __icontains method to search for "money" and "world" (api.com/?keyworld=money&keyword=world) this will return all records that contain money, world or both.
The related SQL statement is:
select * from news
where news_source = 1 or news_source = 2
and news_title like '%money%' or news_title like '%world%'
We are trying to use this code to allow the user to have multiple keywords for the __icontains as well as multiple sources, so the end goal is:
api.com/?keyworld=money&keyword=world&source=1&source=2
Our code:
def get_queryset(self):
queryset = News.objects.all()
title = self.request.query_params.getlist('title')
source = self.request.query_params.getlist('source')
if title:
queryset = queryset.filter(news_title__icontains=title, news_source__in=source)
return queryset
The issue is that this is only returning the second keyword if a second keyword is used, and not other keywords prior to what is typed in &keyword=.

You can not perform an __icontains with a list, but you can for example design a function that, for a list constructs the logical or of these values. For example:
from django.db.models import Q
from functools import reduce
from operator import or_
def or_fold(list_of_qs):
if list_of_qs:
return reduce(or_, list_of_qs)
else:
return Q()
def unroll_lists_or(qs, **kwargs):
return qs.filter([
or_fold(Q(**{k: vi}) for vi in v)
for k, v in kwargs.items()
])
You can then call the unroll_lists_or with a queryset, and each item should be an iterable (for example a list). It will then perform or-logic between the items of the list, and and-logic between different keys. In case an iterable is empty, it is ignored.
So we can then write the check as:
unroll_lists_or(queryset, news_title__icontains=title, news_source=source)
In case title contains two items (so title == [title1, title2]), and source contains three items (so source = [source1, source2, source3]), then this will result in:
qs.filter(
Q(news_title__icontains=title1) | Q(news_title__icontains=title2),
Q(news_source=source1) | Q(news_source=source2) | Q(news_source=source3)
)
You can however combine it with an .filter(..) for the __in check. For example:
queryset = News.objects.all()
if source:
queryset = queryset.filter(news_source__in=source)
queryset = unroll_lists_or(queryset, news_title__icontains=title)

I was able to solve this by creating 2 separate functions within the get_querset() function, which is called when a GET request is made.
def get_queryset(self):
queryset = News.objects.all()
source_list = self.request.query_params.getlist('source')
keyword_list = self.request.query_params.getlist('title')
if source_list or keyword_list:
def create_q_source(*args):
list = [*args]
source = Q()
for value in list:
source.add(Q(news_source=value), Q.OR)
return source
def create_q_keyword(*args):
list = [*args]
keyword = Q()
for value in list:
keyword.add(Q(news_title__icontains=value), Q.OR)
return keyword
queryset = queryset.filter(create_q_source(*source_list),create_q_keyword(*keyword_list))
return queryset
Edit:
When you go to the api link and pass in the parameters, filtering will occur based on what is passed in:
http://127.0.0.1:8000/api/notes/?keyword=trump&keyword=beyond&keyword=money&source=1
SQL Equivalent:
select * from news where news_source = 1 AND news_title like '%beyond%' OR news_title like '%money%'

Django ORM: customize a ListView, and add more information to its queryset

I have a view to list a certain model (lets call it class A), like this:
class BaseListView(ListView, MultipleObjectMixin):
http_method_names = ['get']
order_field = None
def get_paginate_by(self, queryset):
session_manager = SessionManager(self.request.session)
return session_manager.paginate_by.get()
def get_context_data(self, **kwargs):
context = super(BaseListView, self).get_context_data(**kwargs)
session_manager = SessionManager(self.request.session)
session_manager.paginate_by.set_to(context)
return context
This view did just what was needed, till now. Now I have to compare the list of objects it retrieves with another list of objects (class B).
The objects of class A and B both have a primary key with their name.
I want to check if any of the objects from class A has the same name (primary key) as any of the objects in class B. In case there is an instance of A in B I would like to add a certain parameter or something like is_in_B=True.
I need this so that I can represent these instances of A in a different way in the template.
How could I do this?
This is what I have come up with by myself for the moment:
class AListView(BaseListView):
model = "A"
def get_queryset(self):
queryset = super(AListView, self). get_query_set()
all_objects_A = A.objects.all()
all_objects_B = B.objects.all()
# modify queryset to indicate which instances of A are present in B
# No idea how to do this
return queryset
I'm not really sure this is an appropiate approach.
Also, how am I supposed to modify the queryset returned by my class so that I can indicate which instances of class A share the same name as any of the instances of class B?

You can annotate your queryset with a conditional expression to achieve this:
from django.db.models import Case, When, Value
def get_queryset(self):
# This gives you the queryset of A objects
queryset = super(AListView, self).get_queryset()
# List of primary keys of B objects
all_objects_B = B.objects.all().values_list('pk',flat=True)
# modify queryset to indicate which instances of A are present in B
return queryset.annotate(
is_in_b=Case(When(pk__in=all_objects_B, then=Value(True)),
default=Value(False))
)
)
Your queryset objects will now have an is_in_b property.
This will work fine if your list of B objects is small. If it is large then I am not sure it is very efficient, and you may need to develop this further to see whether the check (is A in B) can be done directly in the database (possibly requiring raw SQL).

OR logic with Django-filter

I'm using Django-filter to allow the user to filter a database based on multiple choices in two fields. The filterset.py looks like this:
class TapFilter(django_filters.FilterSet):
bar__region = django_filters.MultipleChoiceFilter(choices=CHOICES, label="Regions:", widget=forms.CheckboxSelectMultiple,help_text="")
bar = django_filters.ModelMultipleChoiceFilter(queryset=Bar.objects.all(), label="Bars:", widget=forms.CheckboxSelectMultiple,help_text="")
However, this functions as an AND between the two lists. I need OR instead. That is, I need to show anything matching the selection in either category.
I have seen similar questions using normal filters, but I would prefer to keep using django-filter if possible.
The website in question is here: http://bestap.pythonanywhere.com/
Update: I've put this in my filtersets.py, but am clearly not doing things right...
class TapFilter(django_filters.FilterSet):
bar__region = django_filters.MultipleChoiceFilter(choices=CHOICES, label="Regions:", widget=forms.CheckboxSelectMultiple,help_text="")
bar = django_filters.ModelMultipleChoiceFilter(queryset=Bar.objects.all(), label="Bars:", widget=forms.CheckboxSelectMultiple,help_text="")
def qs(self):
base_qs = Bar.objects.all()
qs = Bar.objects.none()
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = qs | filter_.filter(base_qs, value)
return qs
This gives me the error 'function' object has no attribute 'count'.

You'll need to override qs on your TapFilter FilterSet subclass.
The base implementation is not that complicated; the essence of it loops over the filters applying them to the queryset.
Simplified:
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = filter_.filter(qs, value)
You need the union of the filters' QuerySets, which you can get because QuerySet implements __or__, so (again simplified) you'll need something like:
base_qs = Bar.objects.all()
qs = Bar.objects.none()
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = qs | filter_.filter(base_qs, value)
Hopefully that gets you started.

too many values to unpack [Django]

def index(request):
expiring_list = probe.objects.filter("isExpired=True")
output = ', '.join([p.serial for p in expiring_list])
return HttpResponse(output)
isExpired is a Boolean function. How should I modify the filter so that the filter does not raise a ValueError?

You are making the query in a wrong format.
Your query should be of the form:
expiring_list = probe.objects.filter(isExpired = True)
This was the query you needed to make in case isExpired was your model field. But since you say its a function, assuming that function is inside the class you need to get all the objects in the following way:
expiring_list = []
objects = probe.objects.all()
for obj in objects:
if obj.isExpired() == True:
expiring_list.append(obj)
The expiring_list will now contain all the objects of the model probe where isExpired function returns True

I think isExpired is not a field in your models, as reference to your previous question Refresh a field from another table [Django]
I think exp_date is the field which you are looking for.
Try this:
import datetime
def index(request):
expiring_list = probe.objects.filter(exp_date__lt=datetime.date.today())
output = ', '.join([p.serial for p in expiring_list])
return HttpResponse(output)

Creating a queryset which represents a union of querysets

Let's say I have the following models:
class House(models.Model):
address = models.CharField(max_length=255)
class Person(models.Model):
name = models.CharField(max_length=50)
home = models.ForeignKey(House, null=True, related_name='tenants')
class Car(models.Model):
make = models.CharField(max_length=50)
owner = models.ForeignKey(Person)
Let's say I have a need (strange one, albeit) to get:
list of people who live in a house or are named 'John'
list of cars of the above people
I would like to have two functions:
get_tenants_or_johns(house)
get_cars_of_tenants_or_johns(house)
I could define them as:
from django.db.models.query_utils import Q
def get_cars_of_tenants_or_johns(house):
is_john = Q(owner__in=Person.objects.filter(name='John'))
is_tenant = Q(owner__in=house.tenants.all())
return Car.filter(is_john | is_tenant)
def get_tenants_or_johns(house):
johns = Person.objects.filter(name='John')
tenants = house.tenants.all()
return set(johns) | set(tenants)
The problem is that the logic is repeated in the above examples. If I could get get_tenants_or_johns(house) to return a queryset I could define get_cars_of_tenants_or_johns(house) as:
def get_cars_of_tenants_or_johns(house):
return Car.objects.filter(owner__in=get_tenants_or_johns(house))
In order to do that, get_tenants_or_johns(house) would need to return a union of querysets, without turning them into other collections.
I cannot figure out how to implement get_tenants_or_johns(house) so that it would return a queryset containing a SQL UNION. Is there a way to do that? If not, is there an alternate way to achieve what I am trying to do?

The | operator on two querysets will return a new queryset representing a union.
The function will need to change to (got rid of set() wrappers):
def get_tenants_or_johns(house):
johns = Person.objects.filter(name='John')
tenants = house.tenants.all()
return johns | tenants
and everything will work exactly like needed.

You mention users who live in a house, but have no mention of your User model.
I think you really need to take a long look at the structure of your application - there are probably much easier ways to accomplish your goal.
But to answer your question let's set up three helper functions. Since, as I mentioned above, you haven't outlined what you want to do with the User class - I've assumed that the house that will be passed to these functions is an address:
helpers.py
def get_johns(house):
is_john = Person.objects.filter(name='John')
return is_john
def get_cars_of_tenants(house):
cars = Car.objects.filter(owner__home__address=house)
return cars
def get_tenants(house):
tenants = Person.objects.filter(home__address=house)
return tenants
Now you could create a view for each of your combination queries:
views.py:
import helpers.py
from itertools import chain
def get_cars_of_tenants_or_johns(request, house):
results = list(chain(get_cars_of_tenants(house), get_johns(house)))
return render_to_response('cars_or_johns.html', {"results": results,})
def get_tenants_or_johns(request, house):
results = list(chain(get_tenants(house), get_johns(house)))
return render_to_response('tenants_or_johns.html', {"results": results,})
And this can go on for all of the various combinations. What is returned is results which is a list of all of the matches that you can iterate over.