Develop Reference

Extend queryset in django python

I am looking for way how to add new objects to existing queryset, or how to implement what I want by other way.
contact = watson.filter(contacts, searchline)
This line returns queryset, which I later use to iterate.
Then I want to do this to add more objects, which watson couldn't find
contact_in_iteration = Contact.objects.get(id = fild.f_for)
contact.append(contact_in_iteration)
And sorry for my poor english
Did this
contacts = Contact.objects.filter(crm_id=request.session['crm_id'])
query = Q(contacts,searchline)
contact = watson.filter(query)
and get 'filter() missing 1 required positional argument: 'search_text'' error

You can use | and Q lookups. See the docs.
I'm not sure I've fully understood your initial query, but I think that in your case you would want to do:
query = Q(contacts='Foo', searchline='Bar')
contact = watson.filter(query)
Then later:
contact = watson.filter(query | Q(id=field.f_for))
Strictly speaking it won't append to the queryset, but will return a new queryset. But that's okay, because that's what .filter() does anyway.

You should look at a queryset as a sql query that will be executed later. When constructing a queryset and save the result in a variable, you can later filter it even more, but you can not expand it. If you need a query that has more particular rules (like, you need an OR operation) you should state that when you are constructing the query. One way of doing that is indeed using the Q object.
But it looks like you are confused about what querysets really are and how they are used. First of all:
Contact.objects.get(id = fild.f_for)
will never return a queryset, but an instance, because you use get and thus ask for a single particular record. You need to use filter() if you want to get a quersyet. So if you had an existing queryset say active_contacts and you wanted to filter it down so you only get the contacts that have a first_name of 'John' you would do:
active_contacts = Contact.objects.filter(active=True)
active_contacts_named_John = active_contacts.filter(first_name='John')
Of course you could do this in one line too, but I'm assuming you do the first queryset construction somewhere else in your code.
Second remark:
If in your example watson is a queryset, your user of filter() is unclear. This doesn't really make sense:
contact = watson.filter(contacts, searchline)
As stated earlier, filtering a queryset returns another queryset. So you should use a plurar as your variable name e.g. contacts. Then the correct use of filter would be:
contacts = watson.filter(first_name=searchline)
I'm assuming searchline here is a variable that contains a user inputted search term. So maybe here you should name your variable searchterm or similar. This will return all contacts that are filtered by whatever watson is filtering out already and whose first_name matches searchline exactly. You could also use a more liberate method and filter out results that 'contains' the searching term, like so:
contacts = watson.filter(first_name__contains=searchline)
Hope this helps you get on the right path.

Django complex ordering

I have a Django model Document, which can have Vote objects pointing on it. There's a integer field on Vote called score.
I want to order a queryset of documents according to the number of Vote objects with score=1 that are pointing at the document. i.e., the document that has the most positive votes should be the first one in the queryset.
Is it possible with Django? How?

This is a job for annotations.
from django.db.models import Count
Document.objects.filter(score=1).annotate(
positive_votes=Count('vote__count')).order_by('positive_votes')
Edit
There isn't really a way to do this without filtering, because that's the way the underlying database operations work. But one not-so-nice way would be to do a separate query for all the documents not included in the original, and chain the two querysets together:
positive_docs = <query from above>
other_docs = Document.objects.exclude(id__in=positive_docs)
all_docs = itertools.chain(positive_docs, other_docs)
This would work as long as you don't have millions of docs, but would break things like pagination.

I did this (on the QuerySet model):
def order_by_score(self):
q = django.db.models.Q(ratings__score=1)
documents_with_one_positive_rating = self.filter(q) # Annotation sees only
# the positive ratings
documents_without_one_positive_rating = self.filter(~q)
return (documents_with_one_positive_rating |
documents_without_one_positive_rating).annotate(
db_score=django.db.models.Count('ratings')
).order_by('-db_score')
Advantage is it still shows the documents without a positive rating.

Variable interpolation in python/django, django query filters [duplicate]

Given a class:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=20)
Is it possible, and if so how, to have a QuerySet that filters based on dynamic arguments? For example:
# Instead of:
Person.objects.filter(name__startswith='B')
# ... and:
Person.objects.filter(name__endswith='B')
# ... is there some way, given:
filter_by = '{0}__{1}'.format('name', 'startswith')
filter_value = 'B'
# ... that you can run the equivalent of this?
Person.objects.filter(filter_by=filter_value)
# ... which will throw an exception, since `filter_by` is not
# an attribute of `Person`.

Python's argument expansion may be used to solve this problem:
kwargs = {
'{0}__{1}'.format('name', 'startswith'): 'A',
'{0}__{1}'.format('name', 'endswith'): 'Z'
}
Person.objects.filter(**kwargs)
This is a very common and useful Python idiom.

A simplified example:
In a Django survey app, I wanted an HTML select list showing registered users. But because we have 5000 registered users, I needed a way to filter that list based on query criteria (such as just people who completed a certain workshop). In order for the survey element to be re-usable, I needed for the person creating the survey question to be able to attach those criteria to that question (don't want to hard-code the query into the app).
The solution I came up with isn't 100% user friendly (requires help from a tech person to create the query) but it does solve the problem. When creating the question, the editor can enter a dictionary into a custom field, e.g.:
{'is_staff':True,'last_name__startswith':'A',}
That string is stored in the database. In the view code, it comes back in as self.question.custom_query . The value of that is a string that looks like a dictionary. We turn it back into a real dictionary with eval() and then stuff it into the queryset with **kwargs:
kwargs = eval(self.question.custom_query)
user_list = User.objects.filter(**kwargs).order_by("last_name")

Additionally to extend on previous answer that made some requests for further code elements I am adding some working code that I am using
in my code with Q. Let's say that I in my request it is possible to have or not filter on fields like:
publisher_id
date_from
date_until
Those fields can appear in query but they may also be missed.
This is how I am building filters based on those fields on an aggregated query that cannot be further filtered after the initial queryset execution:
# prepare filters to apply to queryset
filters = {}
if publisher_id:
filters['publisher_id'] = publisher_id
if date_from:
filters['metric_date__gte'] = date_from
if date_until:
filters['metric_date__lte'] = date_until
filter_q = Q(**filters)
queryset = Something.objects.filter(filter_q)...
Hope this helps since I've spent quite some time to dig this up.
Edit:
As an additional benefit, you can use lists too. For previous example, if instead of publisher_id you have a list called publisher_ids, than you could use this piece of code:
if publisher_ids:
filters['publisher_id__in'] = publisher_ids

Django.db.models.Q is exactly what you want in a Django way.

This looks much more understandable to me:
kwargs = {
'name__startswith': 'A',
'name__endswith': 'Z',
***(Add more filters here)***
}
Person.objects.filter(**kwargs)

A really complex search forms usually indicates that a simpler model is trying to dig it's way out.
How, exactly, do you expect to get the values for the column name and operation?
Where do you get the values of 'name' an 'startswith'?
filter_by = '%s__%s' % ('name', 'startswith')
A "search" form? You're going to -- what? -- pick the name from a list of names? Pick the operation from a list of operations? While open-ended, most people find this confusing and hard-to-use.
How many columns have such filters? 6? 12? 18?
A few? A complex pick-list doesn't make sense. A few fields and a few if-statements make sense.
A large number? Your model doesn't sound right. It sounds like the "field" is actually a key to a row in another table, not a column.
Specific filter buttons. Wait... That's the way the Django admin works. Specific filters are turned into buttons. And the same analysis as above applies. A few filters make sense. A large number of filters usually means a kind of first normal form violation.
A lot of similar fields often means there should have been more rows and fewer fields.

Adding custom Field to django queryset containing source record

Is it possible to add a field to a queryset in django which would do the following:
fitting_set_items_list = FittingSetItem.objects.exclude(fitting_pack = None).exclude(fitting_pack = '').order_by('usage_type')
fitting_pack_list = FittingPack.objects.filter(fittingsetitem__in=fitting_set_items_list).add_field({'fitting_set_item': fittingsetitem})
This way i could use:
for fitting_pack_item in fitting_pack_item_list:
fitting_set_item = fitting_pack_item.fitting_set_item
and it'd have the FittingSetItem that i came from in the first place.
The overall Idea is that Many FittingSetItems can point to a FittingPack, so i want to be able to know which FittingSetItem my FittingPack was sourced from, and the easiest way i saw was to have the reverse FK on the FittingPack. This would allow me to go backwards and forwards on possibly ambiguous FK relations

I assume this is what you seek, if FittingSetItem.fitting_pack is a ForeignKey
fitting_packs = FittingPack.objects.filter(fittingsetitem__fitting_pack__isnull=False)
It gets the fitting packs linked by a set item. You might add .distinct() or replace fittingsetitem with fitting_set_item.

Filter and sort music info on Google App Engine

I've enjoyed building out a couple simple applications on the GAE, but now I'm stumped about how to architect a music collection organizer on the app engine. In brief, I can't figure out how to filter on multiple properties while sorting on another.
Let's assume the core model is an Album that contains several properties, including:
Title
Artist
Label
Publication Year
Genre
Length
List of track names
List of moods
Datetime of insertion into database
Let's also assume that I would like to filter the entire collection using those properties, and then sorting the results by one of:
Publication year
Length of album
Artist name
When the info was added into the database
I don't know how to do this without running into the exploding index conundrum. Specifically, I'd love to do something like:
Albums.all().filter('publication_year <', 1980).order('artist_name')
I know that's not possible, but what's the workaround?
This seems like a fairly general type of application. The music albums could be restaurants, bottles of wine, or hotels. I have a collection of items with descriptive properties that I'd like to filter and sort.
Is there a best practice data model design that I'm overlooking? Any advice?

There's a couple of options here: You can filter as best as possible, then sort the results in memory, as Alex suggests, or you can rework your data structures for equality filters instead of inequality filters.
For example, assuming you only want to filter by decade, you can add a field encoding the decade in which the song was recorded. To find everything before or after a decade, do an IN query for the decades you want to span. This will require one underlying query per decade included, but if the number of records is large, this can still be cheaper than fetching all the results and sorting them in memory.

Since storage is cheap, you could create your own ListProperty based indexfiles with key_names that reflect the sort criteria.
class album_pubyear_List(db.Model):
words = db.StringListProperty()
class album_length_List(db.Model):
words = db.StringListProperty()
class album_artist_List(db.Model):
words = db.StringListProperty()
class Album(db.Model):
blah...
def save(self):
super(Album, self).save()
# you could do this at save time or batch it and do
# it with a cronjob or taskqueue
words = []
for field in ["title", "artist", "label", "genre", ...]:
words.append("%s:%s" %(field, getattr(self, field)))
word_records = []
now = repr(time.time())
word_records.append(album_pubyear_List(parent=self, key_name="%s_%s" %(self.pubyear, now)), words=words)
word_records.append(album_length_List(parent=self, key_name="%s_%s" %(self.album_length, now)), words=words)
word_records.append(album_artist_List(parent=self, key_name="%s_%s" %(self.artist_name, now)), words=words)
db.put(word_records)
Now when it's time to search you create an appropriate WHERE clause and call the appropriate model
where = "WHERE words = " + "%s:%s" %(field-a, value-a) + " AND " + "%s:%s" %(field-b, value-b) etc.
aModel = "album_pubyear_List" # or anyone of the other key_name sorted wordlist models
indexes = db.GqlQuery("""SELECT __key__ from %s %s""" %(aModel, where))
keys = [k.parent() for k in indexes[offset:numresults+1]] # +1 for pagination
object_list = db.get(keys) # returns a sorted by key_name list of Albums

As you say, you can't have an inequality condition on one field and an order by another (or inequalities on two fields, etc, etc). The workaround is simply to use the "best" inequality condition to get data in memory (where "best" means the one that's expected to yield the least data) and then further refine it and order it by Python code in your application.
Python's list comprehensions (and other forms of loops &c), list's sort method and the sorted built-in function, the itertools module in the standard library, and so on, all help a lot to make these kinds of tasks quite simple to perform in Python itself.