I'm trying to add an extra field to the instances of my queryset and then sort the set by the new field.
But I get a field error( Cannot resolve keyword 'foo' into field. Choices are: ... )
My view(abstract):
def view(request):
instances = Model.objects.all()
for counter, instance in enumerate(instances):
instance.foo = 'bar' + str(counter)
instances.order_by('foo') #this line causes trouble
return render(request, 'template.html', {'instances': instance})
My template:
{% for instance in instances %}
{{instance.foo}}
{% endfor %}
If I leave out the order_by line the templates renders as expected, so the field seems to be there.
So why do I get a field error?
It would be awesome, if somebody could help me to understand what I'm doing wrong.
Thanks in advance.
I found a possible solution to change the template to
{% for instance in instances|dictsort:'foo' %}
and that works fine, but from what I understand there should be as little logic as possible in the view, so I figure sorting should be done in the view.
Or is this actually the right way?
The Django ORM aims to construct database queries. As a result, you can only query on what a database "knows". Methods, properties, or attributes you added yourself are unknown to the database. The .order_by thus has no effect, since you "patched" the objects in the instances queryset.
If you however call an instances.order_by you construct a new queryset. This queryset takes the context of the parent, and thus represents a (slightly) modified query, but again, a query. Whether the old queryset is already evaluated or patched, is of no importance.
Furthermore even if there was a column foo, it would not help, since the instance.order_by does not order the instance queryset, it constructs a new one, one that looks approximately like the old one, except that the rows are ordered.
You thus will have to sort in Python now. You can for example construct a list of ordered elements with sorted(..), like:
from operator import attrgetter
def view(request):
instances = Model.objects.all()
for counter, instance in enumerate(instances):
instance.foo = 'bar' + str(counter)
mydata = sorted(instances, key=attrgetter('foo'))
return render(request, 'template.html', {'instances': mydata})
So now mydata is no longer a QuerySet, but a vanilla list. Furthermore note that ordering in a database might be slightly different than ordering in Python. In Python exceptions can occur if not all elements have the same type, and furthermore it is not guaranteed that the semantics behind the order relation is exactly the same.
The new attribute in the Python Objects does not exist in the database and only in those instances. order_by changes the queryset and not your current list of objects stored in memory.
One approach would be to use the builtin python sorting functions in the view like: sorted or even list.sort().
Related
I have a Model with numerous attributes; in the template rendered by my DetailView I want to show only the attributes that are not None.
This can be easily done with an if template tag that checks if the value of the attribute is not None, tho I should add an if condition for every attribute in my model, and there's a lot of them.
I would like to iterate through all the attributes and if not None display them.
In pseudo code it would look like this:
{% for attribute in my_instance.all_attributes %}
{% if attribute.value is not None %}
{{attribute.value}}
I can get a tuple of concrete attributes both through the class or the instance:
cls._meta.concrete_fields
or
self._meta.concrete_fields
Now that I have the my_instance.all_attributes in my pseudo code example, I can iterate it but I don't know how to get the actual value of the instance's attribute.
EDIT:
.concrete_values returns an array of Field instances, it looks like this:
(<django.db.models.fields.BooleanField: gov>, <django.db.models.fields.BooleanField: in_group>, <django.db.models.fields.CharField: legal_class>,)
I can access the value of the name attribute of the Field instance using .name. Calling .name on the example above would return 'gov', 'in_group', 'legal_class'
The authors of Django went out of their way to make sure that the template language isn't used to do things like this! Their opinion as I understand it is that templates should do formatting, and Python should do program logic.
There are two ways around it. One is to create a structure that the template can iterate through, and pass it to the DetailView context. Something like
def get_context_data(self):
data = super().get_context_data(**kwargs)
fieldnames = [
x.name for x in self.object._meta.concrete_fields ]
display_fields = [
(name, getattr( self.object, name, None)) for name in fieldnames ]
display_fields = [ x for x in display_fields if x[1] is not None ]
data['display_fields'] = display_fields
return data
and in the template you can now do
{% for name,value in display_fields %}
You might prefer to code a list of names instead of using ._meta.concrete_fields because that lets you choose the order in which they appear. Or you could start with an ordered list and append anything that's in the _meta but not yet in your list (and delete anything that's in your list but not in _meta)
The other way is to use Jinja as the template engine for this view.
I read the Querysets Django docs regarding querysets being lazy, but still am a bit confused here.
So in my view, I set a queryset to a variable like so
players = Players.objects.filter(team=team)
Later on, I have a sorting mechanism that I can apply
sort = '-player_last_name'
if pts > 20:
players = players.filter(pts__gte = pts).order_by(sort)
else:
players = players.filter(pts__lte = pts).order_by(sort)
if ast < 5:
players = players.filter(asts__lte = ast).order_by(sort)
else:
players = players.filter(asts__gte = ast).order_by(sort)
context = {players: players)
return render(request, '2021-2022/allstars.html', context)
What I want to know is, when is the players queryset evaluated? is it when each page is rendered, or everytime I assign the queryset to a variable? Because if it's the former, then I can just apply the .order_by(sort) chain and the previous applications are redundant.
QuerySets are evaluated if you "consume" the queryset. You consume a queryset by enumerating over it, call .get(…), .exists(…), .aggregate(…) or .count(…), check the truthiness (for example with if myqueryset: or bool(queryset), or call len(…) over it, etc. As a rule of thumb, it gets evaluated if you perform an action on it such that the result is no longer a QuerySet.
If you enumerate over it, or you call len(…) the result is cached in the QuerySet, so calling it a second time, or enumerating over it after you have called len(…) will not make another trip to the database.
In this specific case, none of the QuerySets are evaluated before you make the call to the render(…) function. If in the template you for example use {% if players %}, or {% for players %}, {{ players|length }}, or {{ players.exists }}, then it will evaluate the queryset.
Django queries are designed to be "lazy" - that is. they don't run the database query until something requests actual data. Queries can be modified by the addition of filtering and other similar functions.
For example, the following code requests all TeamMember objects when the search string is 'all', but otherwise adds a filter to restrict names to those matching the given search.
squad_list = TeamMember.objects(state__in={"Hired", "Joined", "Staff", "Recruiting"})
if squadname != 'all':
squad_list = squad_list(squad__icontains=squadname.lower())
When the squadlist query is finally executed it will retrieve the required record. Dopes this help?
I am diving to a django rest-api framework that someone else wrote and configured, and I came across a problem I could not find any good solution for.
There is a model containing field of type "YAMLField". While trying to retrieve this field member, it is converted to OrderedDict (not quite sure how and where this conversion is happening...).
Now, I have a queryset of this model. I understand how to filter a queryset based on simple attributes, but how can I filter it based on this dictionary?
For example, each entry in this queryset (which is MyModel instance) contains:
MyModel.myDictionary == {'firstKey': 'firstVal', 'secondKey':'secondVal}
Now I want to get all the entries from this queryset where:
myDictionary = {'firstKey': 'Something'}
Meaning, my dictionary to filter by, may contain only a subset of the keys.
I could not find any solution or a straight forward way to do it, leading me to iterate the queryset and for each entry, iterate the dictionary.
This feels like too much overhead...
I think I had the same problem and someone told me an straightforward answer, which consists in adding "__{dictionary_key}" to your filtering request such as, in your case:
Model.objects.all().filter(myDictionary__firstKey="Something")
Though the asnwer is probably coming too late, I post it hoping that it can be useful for others in the future!
You need this is possible. For more information see django-rest-framework doc
class MultipleFieldLookupMixin(object):
"""
Apply this mixin to any view or viewset to get multiple field filtering
based on a `lookup_fields` attribute, instead of the default single field filtering.
"""
def get_object(self):
queryset = self.get_queryset() # Get the base queryset
queryset = self.filter_queryset(queryset) # Apply any filter backends
filter = {}
for field in self.lookup_fields:
filter[field] = self.kwargs[field]
return get_object_or_404(queryset, **filter) # Lookup the object
Just came across this in the django docs
Calling none() will create a queryset that never returns any objects
and no query will be executed when accessing the results. A qs.none()
queryset is an instance of EmptyQuerySet.
I build a lot of CRUD apps (surprise) and I can't think of a situation where I would need to use none().
Why would one want to return an EmptyQuerySet?
Usually in instances where you need to provide a QuerySet, but there isn't one to provide - such as calling a method or to give to a template.
The advantage is if you know there is going to be no result (or don't want a result) and you still need one, none() will not hit the database.
For a non-realistic example, say you have an API where you can query your permissions. If the account hasn't been confirmed, since you already have the Account object and you can see that account.is_activated is False, you could skip checking the database for permissions by just using Permission.objects.none()
In cases where you want to append to querysets but want an empty one to begin with
Similar to conditions where we instantiate an empty list to begin with but gradually keep appending meaningful values to it
example..
def get_me_queryset(conditionA, conditionB, conditionC):
queryset = Model.objects.none()
if conditionA:
queryset |= some_complex_computation(conditionA)
elif conditionB:
queryset |= some_complex_computation(conditionB)
if conditionC:
queryset |= some_simple_computation(conditionC)
return queryset
get_me_queryset should almost always return instance of django.db.models.query.QuerySet (because good programming) and not None or [], or else it will introduce headaches later..
This way even if none of the conditions come True, your code will still remain intact. No more type checking
For those who do not undestand | operator's usage here:
queryset |= queryset2
It translates to:
queryset = queryset + queryset
another use of queryset.none is when you don't know if there will be objects but do not want to raise an error.
example:
class DummyMixin(object):
def get_context_data(self,**kwargs):
""" Return all the pks of objects into the context """
context = super(DummyMixin, self).get_context_data(**kwargs)
objects_pks = context.get(
"object_list",
Mymodel.objects.none()
).values_list("pk", flat=True)
context["objects_pks"] = objects_pks
Another good use case for this is if some calling method wants to call .values_list() or similar on results. If the method returned None, you'd get an error like
AttributeError: 'list' object has no attribute 'values_list'
But if your clause returns MyModel.objects.none() instead of None, the calling code will be happy, since the returned data is an empty queryset rather than a None object.
Another way of putting it is that it allows you to not mix up return types (like "this function returns a QuerySet or None," which is messy).
It's useful to see where qs.none() is used in other examples in the Django docs. For example, when initializing a model formset using a queryset if you want the resulting formset to be empty the example given is:
formset = AuthorFormSet(queryset=Author.objects.none())
none() is used in get_queryset() to return an empty queryset depending on the state of has_view_or_change_permission() as shown below:
class BaseModelAdmin(metaclass=forms.MediaDefiningClass):
# ...
def has_view_or_change_permission(self, request, obj=None):
return self.has_view_permission(request, obj) or self.has_change_permission(
request, obj
)
# ...
class InlineModelAdmin(BaseModelAdmin):
# ...
def get_queryset(self, request):
queryset = super().get_queryset(request)
if not self.has_view_or_change_permission(request):
queryset = queryset.none() # Here
return queryset
I am displaying a modelformset and I would like the forms to be ordered by the contents of one of its fields. So I want to use the equivalent of SomeModel.objects.filter(whatever).order_by('somefield') for a (model)formset in a template.
How can I do this?
Note that can_order does not do what I want (it must be automatic, not user specified). I have also tried other things, like the dictsort filter, but that produces unpredictable output (i.e. not ordered by the specified field).
I even tried {% regroup formset by somefield as sorted_formset %}, but the resulting sorted_formset cannot be used (iterated) as a normal formset.
To complete the answers. There are two ways how you can control the order of forms in the formset: formsets respect order of given queryset (this is shown in other replies). Another way (if you want to have the order of the forms fully under control) is to define custom formset class and override __iter__ and __getitem__ methods:
from django.forms import BaseModelFormSet
class MyModelBaseFormset(BaseModelFormSet):
def __iter__(self):
"""Yields the forms in the order they should be rendered"""
return ...
def __getitem__(self, index):
"""Returns the form at the given index, based on the rendering order"""
return ...
MyModelFormset = modelformset_factory(model=MyModel, formset=MyModelBaseFormset, queryset=...)
This approach is described in the Django documentation:
Iterating over the formset will render the forms in the order they
were created. You can change this order by providing an alternate
implementation for the __iter__() method.
Formsets can also be indexed into, which returns the corresponding
form. If you override __iter__, you will need to also override
__getitem__ to have matching behavior.
https://docs.djangoproject.com/en/dev/topics/forms/formsets/#django.forms.formsets.BaseFormSet
The example of implementing these methods is for example in this SO thread: modelformset __iter__ overloading problem.
if you didn't defined Formset, this is the "inline code" version:
FS=inlineformset_factory(ParentClass,ChildClass)
formset=FS(instance=parentobject,
queryset=parentobject.childobject_set.order_by("-time_begin")
)
Thanks to #john-peters answer above, for pointing me in the right direction. But here is a better way:
MyFormset(inlineformset_factory(...)):
def get_queryset(self):
return super(MyFormset, self).get_queryset().order_by('myfieldname')
This way you do not have to copy or mess with django's code and potentially cause breakage down the road.. just take the queryset that django gives you and override the ordering. I have used this in my own code, it works.
EDIT. After working on this a bit I realize that although it appears to work fine, it somehow messes up the logic in BaseInlineFormset.get_queryset(), resulting in duplicate database queries. However, in the hopes that someone will comment on this and correct it, I will leave it here. Meanwhile, I have another solution which DOES WORK and does not result in redundant queries.. as follows:
MyFormset(inlineformset_factory(...)):
def __init__(self, *args, **kwargs):
super(MyFormset, self).__init__(*args, **kwargs)
self.queryset = self.queryset.order_by('myfieldname')
This modifies the queryset at a safe time, before anything is done with it. In my code I am also doing .select_related() here, which greatly speeds up my large modelformsets!
Thanks to rantanplan's comment I found a solution. I could not use the method as described in the link above because I don't know what the queryset is going to be (this is a complex form with nested formsets).
Anyway, I found a solution by overriding the get_queryset method from Django's BaseInlineFormSet Class.
I copied it below, including my mod, in case googlers find it helpful.
def get_queryset(self):
'''
Copied this method from Django code and modified the ordering statement
'''
if not hasattr(self, '_queryset'):
if self.queryset is not None:
qs = self.queryset
else:
qs = self.model._default_manager.get_query_set()
# If the queryset isn't already ordered we need to add an
# artificial ordering here to make sure that all formsets
# constructed from this queryset have the same form order.
if not qs.ordered:
# MY MOD IS HERE:
# qs = qs.order_by(self.model._meta.pk.name)
qs = qs.order_by('order_index')
#/MOD
# Removed queryset limiting here. As per discussion re: #13023
# on django-dev, max_num should not prevent existing
# related objects/inlines from being displayed.
self._queryset = qs
return self._queryset
A more simpler way but it might not apply in every situation but if you can afford it just define ordering in models Meta class:
class ExampleModel(models.Model):
...
class Meta:
ordering = ("name", )
Formset will respect the queryset's ordering which will respect the Meta class variable's value.