I have a Django QuerySet, and I want to get a Q object out of it. (i.e. that holds the exact same query as that queryset.)
Is that possible? And if so, how?
No, but you could create the Q object first, and use that; alternatively, create your query as a dict, and pass that to your filter method and the Q object.
This is not exactly what you were asking for, but you can extract the sql from a query set by accessing the query member. For example:
x = somequeryset.query
Then you could use that on a new queryset object to reconstruct the original queryset. This may work better in saving stuff like "values" that are defined for a query set. The defined x is easy to store. I've used this in the past to save user constructed queries/searches that then are run daily with the results emailed to the user.
Relevant also if you wanted the Q object so you you can reconstruct a complex query by ORing another Q object to it, is that, provided two QuerySets are on the same model, you can OR the QuerySets directly for that same effect. It's worth trying that and examining the SQL before and after.
For example:
qs1 = model.objects.filter(...)
print("qs1: {}".format(qs1.query)
qs2 = model.objects.filter(...)
print("qs2: {}".format(qs1.query)
qs = q1 | q2
print("qs: {}".format(qs.query)
I certainly found your question because I wanted the Q object from the query for this very reason, and discovered on the Django Users Group:
https://groups.google.com/d/msg/django-users/2BuFFMDL0VI/dIih2WRKAgAJ
that QuerySets can be combined in much the same way as Q objects can.
That may or may not be helpful to you, depending on the reason you want that Q object of course.
Related
I am writing a method that computes a complex Django QuerySet.
It starts like this
qs1 = A.objects.filter(b_set__c_obj__user=user)
qs1 will eventually become the result, but before it does, the method goes on with several further steps of filtering and annotation.
b_set is an 1:n relationship, but I know that at most one of the c_obj can actually match.
I need to reference this c_obj, because I need another attribute email from it for one of the filtering steps (which is against instances of another model D selected based on c_obj's email).
user can be either a User model instance or an OuterRef Django ORM expression, because the whole queryset created by the method is subsequently also to be used in a subquery.
Therefore, any solution that involves evaluating querysets is not suitable for me. I can only build a single queryset.
Hmm?
I need to reference this c_obj, because I need another attribute email from it for one of the filtering steps (which is against instances of another model D selected based on c_obj's email).
If you do all the filtering with the same .filter(…) clause then that c_obj will be the same for all conditions. In other words if you want to filter A such that it has a related B and a related C for example that has as email='foo#bar.com', and as type='user', you can filter with:
qs1 = A.objects.filter(
b_set__c_obj__email='foo#bar.com',
b_set__c_obj__type='user'
)
This is thus different from:
qs1 = A.objects.filter(
b_set__c_obj__email='foo#bar.com'
).filter(
b_set__c_obj__type='user'
)
since here it will look for an A object that has a related c_obj with email='foo#bar.com and for a c_obj (not per se the same one) with type='user'.
Is there a way I could add an attribute to all query objects using annotate? I basically just need to get a value from an m2m relationship of the object and save it as an attribute of the object.
Something like this:
query.annotate(value_to_be_added=("value_from_m2m"))
Basically I have two different queries of the same model, one query A needs to have a "value" changed or added for all of its objects (and that value comes from the m2m relationship). Query B doesn't need to have those values changed.
How would I do this?
I solved it. What needs to be used is just a simple filter F().
from django.db.models import F
query.annotate(value_to_be_added=F("value_from_m2m"))
https://docs.djangoproject.com/en/3.0/ref/models/expressions/#f-expressions
I try some code like this:
mymodels = MyModel.objects.filter(status=1)
mymodels.update(status=4)
print(mymodels)
And the result is an empty list
I know that I can use a for loop to replace the update.
But it will makes a lot of update query.
Is there anyway to continue manipulate mymodels after the bulk update?
Remember that Django's QuerySets are lazy:
QuerySets are lazy – the act of creating a QuerySet doesn’t involve any database activity. You can stack filters together all day long, and Django won’t actually run the query until the QuerySet is evaluated
but the update() method function is actually applied immediately:
The update() method is applied instantly, and the only restriction on the QuerySet that is updated is that it can only update columns in the model’s main table, not on related models.
So while - in your code - are applying the update call after your filter, in reality it is being applied beforehand and therefore your objects status is being changed before the filter is (lazily) applied, meaning there are no matching records and the result is empty.
mymodels = MyModel.objects.filter(status=1)
objs = [obj for obj in mymodels] # save the objects you are about to update
mymodels.update(status=4)
print(objs)
should work.
Explanations why had been given by Timmy O'Mahony.
I have the following annotation in a Django model manager I'd like to convert to a SQLAlchemy ORM query:
annotations = {
'review_count' : Count("cookbookreview", distinct=True),
'rating' : Avg("cookbookreview__rating")
}
return self.model.objects.annotate(**annotations)
What I essentially need is each model object in the query to have review_count and rating attached to them as part of the initial query. I believe I can use column_property, but I would like to avoid this type of "calculated property" on the object, because I don't want the property (expensive lookup) being done for each object when I access the property in a template.
What is the right way to approach this problem? Thanks in advance.
So, for the sake of completeness and usefulness for others with this issue I present the following solution (which may or may not be the optimal way to solve this)
sq_reviews = db_session.query(CookbookReview.cookbook_id,
func.avg(CookbookReview.rating).label('rating'),\
func.count('*').label('review_count')).\
group_by(CookbookReview.cookbook_id).subquery()
object_list = db_session.query(
Cookbook, sq_reviews.c.rating, sq_reviews.c.review_count).\
outerjoin(sq_reviews, Cookbook.id==sq_reviews.c.cookbook_id).\
order_by(Cookbook.name).limit(20)
The key here is the concept of SQLAlchemy subqueries. If you think of each annotation in my original Django query as a subquery, the concept is easier to understand. It's also worth noting that this query is quite speedy - many orders of magnitude swifter than it's (more concise/magical) Django counterpart. Hopefully this helps others curious about this particular Django/SQLAlchemy query analog.
Also keep in mind that you need to perform the actual annotation of the ORM objects yourself. A simple function like this called before sending the object list to your template will suffice:
def process(query):
for obj, rating, review_count in query:
obj.rating = rating
obj.review_count = review_count
yield obj
Say I have 2 models:
class Poll(models.Model):
category = models.CharField(u"Category", max_length = 64)
[...]
class Choice(models.Model):
poll = models.ForeignKey(Poll)
[...]
Given a Poll object, I can query its choices with:
poll.choice_set.all()
But, is there a utility function to query all choices from a set of Poll?
Actually, I'm looking for something like the following (which is not supported, and I don't seek how it could be):
polls = Poll.objects.filter(category = 'foo').select_related('choice_set')
for poll in polls:
print poll.choice_set.all() # this shouldn't perform a SQL query at each iteration
I made an (ugly) function to help me achieve that:
def qbind(objects, target_name, model, field_name):
objects = list(objects)
objects_dict = dict([(object.id, object) for object in objects])
for foreign in model.objects.filter(**{field_name + '__in': objects_dict.keys()}):
id = getattr(foreign, field_name + '_id')
if id in objects_dict:
object = objects_dict[id]
if hasattr(object, target_name):
getattr(object, target_name).append(foreign)
else:
setattr(object, target_name, [foreign])
return objects
which is used as follow:
polls = Poll.objects.filter(category = 'foo')
polls = qbind(polls, 'choices', Choice, 'poll')
# Now, each object in polls have a 'choices' member with the list of choices.
# This was achieved with 2 SQL queries only.
Is there something easier already provided by Django? Or at least, a snippet doing the same thing in a better way.
How do you handle this problem usually?
Time has passed and this functionality is now available in Django 1.4 with the introduction of the prefetch_related() QuerySet function. This function effectively does what is performed by the suggested qbind function. ie. Two queries are performed and the join occurs in Python land, but now this is handled by the ORM.
The original query request would now become:
polls = Poll.objects.filter(category = 'foo').prefetch_related('choice_set')
As is shown in the following code sample, the polls QuerySet can be used to obtain all Choice objects per Poll without requiring any further database hits:
for poll in polls:
for choice in poll.choice_set:
print choice
Update: Since Django 1.4, this feature is built in: see prefetch_related.
First answer: don't waste time writing something like qbind until you've already written a working application, profiled it, and demonstrated that N queries is actually a performance problem for your database and load scenarios.
But maybe you've done that. So second answer: qbind() does what you'll need to do, but it would be more idiomatic if packaged in a custom QuerySet subclass, with an accompanying Manager subclass that returns instances of the custom QuerySet. Ideally you could even make them generic and reusable for any reverse relation. Then you could do something like:
Poll.objects.filter(category='foo').fetch_reverse_relations('choices_set')
For an example of the Manager/QuerySet technique, see this snippet, which solves a similar problem but for the case of Generic Foreign Keys, not reverse relations. It wouldn't be too hard to combine the guts of your qbind() function with the structure shown there to make a really nice solution to your problem.
I think what you're saying is, "I want all Choices for a set of Polls." If so, try this:
polls = Poll.objects.filter(category='foo')
choices = Choice.objects.filter(poll__in=polls)
I think what you are trying to do is the term "eager loading" of child data - meaning you are loading the child list (choice_set) for each Poll, but all in the first query to the DB, so that you don't have to make a bunch of queries later on.
If this is correct, then what you are looking for is 'select_related' - see https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related
I noticed you tried 'select_related' but it didn't work. Can you try doing the 'select_related' and then the filter. That might fix it.
UPDATE: This doesn't work, see comments below.