Develop Reference

django Queryset filter(a__b__c=d): get hold of b for further filtering

I am writing a method that computes a complex Django QuerySet.
It starts like this
qs1 = A.objects.filter(b_set__c_obj__user=user)
qs1 will eventually become the result, but before it does, the method goes on with several further steps of filtering and annotation.
b_set is an 1:n relationship, but I know that at most one of the c_obj can actually match.
I need to reference this c_obj, because I need another attribute email from it for one of the filtering steps (which is against instances of another model D selected based on c_obj's email).
user can be either a User model instance or an OuterRef Django ORM expression, because the whole queryset created by the method is subsequently also to be used in a subquery.
Therefore, any solution that involves evaluating querysets is not suitable for me. I can only build a single queryset.
Hmm?

I need to reference this c_obj, because I need another attribute email from it for one of the filtering steps (which is against instances of another model D selected based on c_obj's email).
If you do all the filtering with the same .filter(…) clause then that c_obj will be the same for all conditions. In other words if you want to filter A such that it has a related B and a related C for example that has as email='foo#bar.com', and as type='user', you can filter with:
qs1 = A.objects.filter(
b_set__c_obj__email='foo#bar.com',
b_set__c_obj__type='user'
)
This is thus different from:
qs1 = A.objects.filter(
b_set__c_obj__email='foo#bar.com'
).filter(
b_set__c_obj__type='user'
)
since here it will look for an A object that has a related c_obj with email='foo#bar.com and for a c_obj (not per se the same one) with type='user'.

when exactly django query execution occures

In a technical interview a questioner asked me a weird question regarding to the execution of querysets. Suppose we have a profile model like below:
class Profile(models.Model):
user = models.OneToOneField('User').select_related(User)
surname = models.TextField(null=True)
q = Profile.object.all()
or
q = Profile.object.get(id=1)
l = q.filter(active=True)
he asked how many query execution has been happened and I replied as the python interpreter executes Profile.object.all() at the begging then one query is already done. However, he answered zero, and one if we call the query, something like this:
for a in l:
a.surname
Is his answer true in django?
another doubt was about models.OneToOneField('User'), why he didn't use django.contrib.auth.models.User and defined models.OneToOneField('User').select_related(User)

QuerySets are not evaluated until you do something that actually needs them to be evaluated. As the documentation for the class itself states a QuerySet:
Represent a lazy database lookup for a set of objects.
Emphasis on the word lazy. This is because one often needs to call or chain methods on a queryset, a good example being a group by requiring subsequent calls to .values() and .annotate(). If a queryset was evaluated directly then we would be making too many unneeded queries to the database, slowing down execution to a halt.
As to when exactly a queryset is evaluated I would list the answer in short (for the long answer refer to When QuerySets are evaluated [Django docs]):
Iterating a queryset
Slicing a queryset (with the step parameter)
Pickling/Caching a queryset
Calling repr(), len(), list(), or bool() on a queryset
Various methods like get(), first(), last(), latest(), or earliest(), etc. also make a query to the database

Django: Extracting a `Q` object from a `QuerySet`

I have a Django QuerySet, and I want to get a Q object out of it. (i.e. that holds the exact same query as that queryset.)
Is that possible? And if so, how?

No, but you could create the Q object first, and use that; alternatively, create your query as a dict, and pass that to your filter method and the Q object.

This is not exactly what you were asking for, but you can extract the sql from a query set by accessing the query member. For example:
x = somequeryset.query
Then you could use that on a new queryset object to reconstruct the original queryset. This may work better in saving stuff like "values" that are defined for a query set. The defined x is easy to store. I've used this in the past to save user constructed queries/searches that then are run daily with the results emailed to the user.

Relevant also if you wanted the Q object so you you can reconstruct a complex query by ORing another Q object to it, is that, provided two QuerySets are on the same model, you can OR the QuerySets directly for that same effect. It's worth trying that and examining the SQL before and after.
For example:
qs1 = model.objects.filter(...)
print("qs1: {}".format(qs1.query)
qs2 = model.objects.filter(...)
print("qs2: {}".format(qs1.query)
qs = q1 | q2
print("qs: {}".format(qs.query)
I certainly found your question because I wanted the Q object from the query for this very reason, and discovered on the Django Users Group:
https://groups.google.com/d/msg/django-users/2BuFFMDL0VI/dIih2WRKAgAJ
that QuerySets can be combined in much the same way as Q objects can.
That may or may not be helpful to you, depending on the reason you want that Q object of course.

I came across a tricky trouble about django Queryset

Tricky code:
user = User.objects.filter(id=123)
user[0].last_name = 'foo'
user[0].save() # Cannot be saved.
id(user[0]) # 32131
id(user[0]) # 44232 ( different )
user cannot be saved in this way.
Normal code:
user = User.objects.filter(id=123)
if user:
user[0].last_name = 'foo'
user[0].save() # Saved successfully.
id(user[0]) # 32131
id(user[0]) # 32131 ( same )
So, what is the problem?

In first variant your user queryset isn't evaluated yet. So every time you write user[0] ORM makes independent query to DB. In second variation queryset is evalutaed and acts like normal Python list.
And BTW if you want just one row, use get:
user = User.objects.get(id=123)

when you index into a queryset, django fetches the data (or looks in its cache) and creates a model instance for you. as you discovered with id(), each call creates a new instance. so while you can set the properties on these qs[0].last_name = 'foo', the subsequent call to qs[0].save() creates a new instance (with the original last_name) and saves that
i'm guessing your particular issue has to do with when django caches query results. when you are just indexing into the qs, nothing gets cached, but your call if users causes the entire (original) qs to be evaluated, and thus cached. so in that case each call to [0] retrieves the same model instance

Saving is possible, but everytime you access user[0], you actually get it from the database so it's unchanged.
Indeed, when you slice a Queryset, Django issues a SELECT ... FROM ... OFFSET ... LIMIT ... query to your database.
A Queryset is not a list, so if you want to it to behave like a list, you need to evaluate it, to do so, call list() on it.
user = list(User.objects.filter(id=123))
In your second example, calling if user will actually evaluate the queryset (get it from the database into your python program), so you then work with your Queryset's internal cache.
Alternatively, you can use u = user[0], edit that and then save, which will work.
Finally, you should actually be calling Queryset.get, not filter here, since you're using the unique key.

Django ORM: Selecting related set

Say I have 2 models:
class Poll(models.Model):
category = models.CharField(u"Category", max_length = 64)
[...]
class Choice(models.Model):
poll = models.ForeignKey(Poll)
[...]
Given a Poll object, I can query its choices with:
poll.choice_set.all()
But, is there a utility function to query all choices from a set of Poll?
Actually, I'm looking for something like the following (which is not supported, and I don't seek how it could be):
polls = Poll.objects.filter(category = 'foo').select_related('choice_set')
for poll in polls:
print poll.choice_set.all() # this shouldn't perform a SQL query at each iteration
I made an (ugly) function to help me achieve that:
def qbind(objects, target_name, model, field_name):
objects = list(objects)
objects_dict = dict([(object.id, object) for object in objects])
for foreign in model.objects.filter(**{field_name + '__in': objects_dict.keys()}):
id = getattr(foreign, field_name + '_id')
if id in objects_dict:
object = objects_dict[id]
if hasattr(object, target_name):
getattr(object, target_name).append(foreign)
else:
setattr(object, target_name, [foreign])
return objects
which is used as follow:
polls = Poll.objects.filter(category = 'foo')
polls = qbind(polls, 'choices', Choice, 'poll')
# Now, each object in polls have a 'choices' member with the list of choices.
# This was achieved with 2 SQL queries only.
Is there something easier already provided by Django? Or at least, a snippet doing the same thing in a better way.
How do you handle this problem usually?

Time has passed and this functionality is now available in Django 1.4 with the introduction of the prefetch_related() QuerySet function. This function effectively does what is performed by the suggested qbind function. ie. Two queries are performed and the join occurs in Python land, but now this is handled by the ORM.
The original query request would now become:
polls = Poll.objects.filter(category = 'foo').prefetch_related('choice_set')
As is shown in the following code sample, the polls QuerySet can be used to obtain all Choice objects per Poll without requiring any further database hits:
for poll in polls:
for choice in poll.choice_set:
print choice

Update: Since Django 1.4, this feature is built in: see prefetch_related.
First answer: don't waste time writing something like qbind until you've already written a working application, profiled it, and demonstrated that N queries is actually a performance problem for your database and load scenarios.
But maybe you've done that. So second answer: qbind() does what you'll need to do, but it would be more idiomatic if packaged in a custom QuerySet subclass, with an accompanying Manager subclass that returns instances of the custom QuerySet. Ideally you could even make them generic and reusable for any reverse relation. Then you could do something like:
Poll.objects.filter(category='foo').fetch_reverse_relations('choices_set')
For an example of the Manager/QuerySet technique, see this snippet, which solves a similar problem but for the case of Generic Foreign Keys, not reverse relations. It wouldn't be too hard to combine the guts of your qbind() function with the structure shown there to make a really nice solution to your problem.

I think what you're saying is, "I want all Choices for a set of Polls." If so, try this:
polls = Poll.objects.filter(category='foo')
choices = Choice.objects.filter(poll__in=polls)

I think what you are trying to do is the term "eager loading" of child data - meaning you are loading the child list (choice_set) for each Poll, but all in the first query to the DB, so that you don't have to make a bunch of queries later on.
If this is correct, then what you are looking for is 'select_related' - see https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related
I noticed you tried 'select_related' but it didn't work. Can you try doing the 'select_related' and then the filter. That might fix it.
UPDATE: This doesn't work, see comments below.