Develop Reference

Solution to filter by Cumulative sum and error: Window is disallowed in the filter clause

This is follow-up to a question to:
Filter Queries which sum of their amount field are greater or lesser than a number
which is supposed to be solved. Answer suggests using Window function with filter but this results in a error:
django.db.utils.NotSupportedError: Window is disallowed in the filter clause.
Comment from #atabak hooshangi suggests removing the Window function, but query doesn't work in intended way after that. Any ideas to solve this problem?

let's say we have these 2 models:
class Developer(models.Model):
first_name = models.CharField(max_length=40, null=False, blank=False,
unique=True)
last_name = models.CharField(max_length=40, null=False, blank=False,
unique=True)
profession = models.CharField(max_length=100, null=False)
cv = models.FileField(upload_to=upload_cv_location, null=True, blank=True)
description = models.TextField()
img = models.ImageField(upload_to=upload_location, null=True, blank=True)
class Meta:
verbose_name = 'Developer'
verbose_name_plural = 'Developers'
ordering = ('first_name',)
def __str__(self):
return f'{self.first_name} {self.last_name}'
class Skill(models.Model):
developer = models.ForeignKey(to=Developer, on_delete=models.CASCADE)
category = models.ForeignKey(to=SkillCategory, on_delete=models.SET_NULL,
null=True)
name = models.CharField(max_length=50, null=False)
priority = models.PositiveIntegerField()
class Meta:
ordering = ('-priority',)
def __str__(self):
return f'{self.name} skill of {self.developer}'
As you can see , we have a developer model which has relationship with skill. Each developer can have multiple skills.
now consider we want to get the developers whos sum of their priorities are greater than a number.
The orm query should work this way :
from django.db.models import Sum
developers = Developer.objects.annotate(tot=Sum('skill__priority')).filter(tot__gt=250).all()
The output will be the developers who has greater than 250 priority_sum .
You can filter tot which is an annotated variable in any way you want.
like .filter(tot__lte)
or
.filter(tot__lt)
I hope this is what you were looking for.

What's the most efficient way to retrieve django queryset with the hightest number of posts for a related name?

I'm currently working on a website where advertisements will be posted to display vehicles for sale and rent. I would like to retrieve a queryset that highlights only one car brand (i.e. Audi) which has the highest number of posts for the respective model. Example:
Displaying the Audi brand because it has the highest number of related posts.
My question is, what's the most efficient way of doing this? I've done some work here but I'm pretty sure this is not the most efficient way. What I have is the following:
# Algorithm that is currently retrieving the name of the brand and the number of related posts it has.
def top_brand_ads():
queryset = Advertisement.objects.filter(status__iexact="Published", owner__payment_made="True").order_by('-publish', 'name')
result = {}
for ad in queryset:
# Try to update an existing key-value pair
try:
count = result[ad.brand.name.title()]
result[ad.brand.name.title()] = count + 1
except KeyError:
# If the key doesn't exist then create it
result[ad.brand.name.title()] = 1
# Getting the brand with the highest number of posts from the result dictionary
top_brand = max(result, key=lambda x: result[x]) # Returns for i.e. (Mercedes Benz)
context = {
top_brand: result[top_brand] # Retrieving the value for the top_brand from the result dict.
}
print(context) # {'Mercedes Benz': 7} -> Mercedes Benz has seven (7) related posts.
return context
Is there a way I could return a queryset instead without doing what I did here or could this be way more efficient?
If the related models are needed, please see below:
models.py
# Brand
class Brand(models.Model):
name = models.CharField(max_length=255, unique=True)
image = models.ImageField(upload_to='brand_logos/', null=True, blank=True)
slug = models.SlugField(max_length=250, unique=True)
...
# Methods
# Owner
class Owner(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
telephone = models.CharField(max_length=30, blank=True, null=True)
alternate_telephone = models.CharField(max_length=30, blank=True, null=True)
user_type = models.CharField(max_length=50, blank=True, null=True)
payment_made = models.BooleanField(default=False)
expiring = models.DateTimeField(default=timezone.now)
...
# Methods
# Advertisement (Post)
class Advertisement(models.Model):
STATUS_CHOICES = (
('Draft', 'Draft'),
('Published', 'Published'),
)
owner = models.ForeignKey(Owner, on_delete=models.CASCADE, blank=True, null=True)
name = models.CharField(max_length=150, blank=True, null=True)
brand = models.ForeignKey(Brand, on_delete=models.CASCADE, blank=True, null=True)
publish = models.DateTimeField(default=timezone.now)
status = models.CharField(max_length=10, choices=STATUS_CHOICES, default='Draft')
...
# Other fields & methods
Any help would be greatly appreciated.

Since you need brands, let's query on Brand model:
Brand.objects.filter(advertisement__status__iexact="Published").\
filter(advertisement__owner__payment_made=True).\
annotate(published_ads=Count('advertisement__id')).\
order_by('-published_ads')
However, even in your proposed solution, you can improve a little bit:
Remove the order_by method from your queryset. It doesn't affect the final result but adds some overhead, especially if your Advertisement model is not indexed on those fields.
Every time you call ad.brand you are hitting the database. This is called the N+1 problem. You are in a loop of n, you make n extra db access. You can use select_related to avoid such problems. In your case: Advertisement.objects.select_related('brand')...

Did you try the count method?
from django.db.models import Count
Car.objects.annotate(num_views=Count('car_posts_related_name')).order_by('num_views')

Is it possible in Django to iterate the creation of a model?

I'm trying to create a model with 100 columns. Is it possible to iterate in my models.py file to create many entries at once?
This is an example of what I'm trying to accomplish.
class AgeFactor(models.Model):
'''
Used to populate a table with all the age-factor numbers
'''
gender = models.CharField(max_length=50, null=True, blank=True)
code = models.CharField(max_length=50, null=True, blank=True)
event_code = models.CharField(max_length=50, null=True, blank=True)
oc = models.DecimalField(max_digits=9, decimal_places=4)
for lp in range(101):
if lp > 4:
(f"{lp} = models.DecimalField(max_digits=9, decimal_places=4)")
Thank you.

Yes, although this is not how to do that. What you can do is use a loop mechanism with:
class AgeFactor(models.Model):
'''
Used to populate a table with all the age-factor numbers
'''
gender = models.CharField(max_length=50, null=True, blank=True)
code = models.CharField(max_length=50, null=True, blank=True)
event_code = models.CharField(max_length=50, null=True, blank=True)
oc = models.DecimalField(max_digits=9, decimal_places=4)
for lp in range(5, 101):
locals()[f'data{lp}'] = models.DecimalField(max_digits=9, decimal_places=4)
This will make a set of fields named data5, data6, data7, … data100.
That being said, it is rather "ugly", and often you can better define a many-to-one relation with a ForeignKey [Django-doc] since that makes the data more "flat".
Furthermore if I run this locally it takes some time for Django to fully evaluate the model, so I do not know how well Django scales with a large amount of fields (the for loop itself is not the problem, but Django has a lot of bookkeeping for these fields).
Finally querying can also be slow, because the database has to load all these columns, furthermore queries will be quite large, because the SELECT part will enumerate all the data… fields.

Return non null field in Many To Many intermediary class/model Django

I think this is a python question (maybe for x in list), but might have a django framework ¿query/or-the-like?
I have an intermediary class/model:
class VehicleFeatures(models.Model):
personal_equipment = models.OneToOneField(PersonalEquipment, null=True, blank=True, )
commercial_equipment = models.OneToOneField(CommercialEquipment, null=True, blank=True, )
personal_vehicle = models.OneToOneField(PersonalVehicle, null=True, blank=True, )
commercial_truck = models.OneToOneField(CommercialTruck, null=True, blank=True, )
commercial_tractor = models.OneToOneField(CommercialTractor, null=True, blank=True, )
…
def __unicode__(self):
# assume one and only one is allowed
return '{}'.format(self.whichever_of_the_fields_exists)
# Extra Credit :)
# assume any number is allowed
How do I return the Model.field that has a reference (ultimately to get that model's name, to pass back to the other side of this many-to-many)?
Notes:
trying to follow option 2 and 3 in this article
I am not worried about performance (yet) but if you are aware of a low cost option, then please teach us why one is better than another
Visual of models
121-Referent: Vehicle (abstract model) with concrete models [listed above]
M2M: VehicleFeatures (to connect to concrete models)
12M: Features (fields for many vehicle features)

One approach that is sometimes used in similar "entity-attribute-value" models is to add an additional field which OneToOneField is populated. Something like this:
class VehicleFeatures(models.Model):
personal_equipment = models.OneToOneField(PersonalEquipment, null=True, blank=True, )
commercial_equipment = models.OneToOneField(CommercialEquipment, null=True, blank=True, )
personal_vehicle = models.OneToOneField(PersonalVehicle, null=True, blank=True, )
commercial_truck = models.OneToOneField(CommercialTruck, null=True, blank=True, )
commercial_tractor = models.OneToOneField(CommercialTractor, null=True, blank=True, )
# Add a field to store the type of relation that is set for this instance
relation_type = models.CharField(max_length=50)
When the object is saved, you do the for x in list thing and determine which field is not null, and set relation_type to the name of that field, e.g.:
def save(self):
for k in ['personal_equipment', 'commercial_equipment', 'personal_vehicle' ...]:
if getattr(self, k) is not None:
self.relation_type = k
return super(VehicleFeatures, self).save()
This is significantly more efficient than doing the same loop every time you query the object.
Then your __unicode__ method would be:
def __unicode__(self):
return '{}'.format(getattr(self, self.relation_type))
This is for the case when only one field is ever set, but you could extend it quite easily to support multiple fields - e.g., by making relation_type a comma-separated string, or if your database supports it, an ArrayField.

Django with python3, results of sql query in selectbox

I have 2 models:
PR_Components (models.Model):
companyID = models.ForeignKey(PO_Company, blank=True, null=True)
comp_nr = models.CharField (max_length=5, blank=True, null=True)
def __str__(self):
return self.comp_nr
PR_ComponentsData (models.Model):
compID = models.ForeignKey (PR_Components, blank=False, null=True)
valid = models.DateField (max_length=10, blank=False, null=True)
comp_image = models.ImageField (upload_to="/images", blank=True, null=True)
comp_text = models.CharField (max_length=30, blank=False, null=True)
....
I want to show now in a selectbox the components number (PR_Components.comp_nr) and their current valid name (PR_Componentsdata.comp_text).
I added a manager to model PR_Components which executes a sql-query.
SELECT a.*, b1.* FROM pool_pr_components a
JOIN pool_pr_componentsdata b1 ON (a.id = b1.compID_id)
LEFT OUTER JOIN pool_pr_componentsdata b2
ON (a.id = b2.compID__id AND (b1.valid < b2.valid OR b1.valid = b2.valid
AND b1.id < b2.id)) WHERE b2.id IS NULL
Later I write forms dynamicly and add the sql-result to the field:
self.fields[field].queryset = sql_result
Until here, everything works fine.
My problem:
In the selectbox the result of the str-Method of model PR_Components is shown (=comp_nr), but I would like to show also the component name like "Component (001)".
How could I do this? It should be a solution which works for other models too, because a lot of my models have "historical" data.
Thanks a lot

Solution:
in model PR_Components we overwrite str with
def __str__(self):
related = PR_ComponentsData.objects.filter(id=self.id).last()
return "{} ({})".format (related.comp_text, self.comp_nr)