I have two models in our django app
class Reg(models.Model):
transactions = ManyToMany
price = IntegerField
class Transaction(models.Model)
amount = IntegerField
Now I would like to make a lookup like:
Registration.objects.filter(reg__price==transaction__amount)
Previously we used the following approach:
Registration has a property is_paid that computes wether a transaction with equal amount exists
[r for r in Registration.objects.filter(...) if r.is_paid]
This is ofc very query-consuming and inefficient.
I wonder whether there would be a better way to do this!
Any hint is appreciated :)
You can use an F expression for such a query:
from django.db.models import F
Registration.objects.filter(price=F('transactions__amount'))
This will filter all Registration instances whose price is equal to one of their transactions' amount. If you want all transactions amounts' sum to be equal or more than the registration price, you can use annotations to aggregate each registration's Sum:
paid_registrations = Registration.objects.\
annotate(ta=Sum('transactions__amount')).\ # annotate with ta sum
filter(price__lte=F('ta')) # filter those whose price is <= that sum
Related
I'll demonstrate by using an example. This is the model (the primary key is implicit):
class Item(models.Model):
sku = models.CharField(null=False)
description = models.CharField(null=True)
I have a list of skus, I need to get the latest descriptions for all skus in the filter list that are written in the table for the model Item. Latest item == greatest id.
I need a way to annotate the latest description per sku:
Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').order_by("-id")
but this won't work for various reasons (excluding the missing aggregate function).
Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').lastest("-id")
Or use this
Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').order_by("-id").reverse()[0]
I used postgres ArrayAgg aggregate function to aggregate the latest description like so:
from django.contrib.postgres.aggregates import ArrayAgg
class ArrayAggLatest(ArrayAgg):
template = "(%(function)s(%(expressions)s ORDER BY id DESC))[1]"
Item.objects.filter(sku__in=skus).values("sku").annotate(descr=ArrayAggLatest("description"))
The aggregate function aggregates all descriptions ordered by descending ID of the original table and gets the 1st element (0 element is None)
Answer from #M.J.GH.PY or #dekomote war not correct.
If you have a model:
class Item(models.Model):
sku = models.CharField(null=False)
description = models.CharField(null=True)
this model has already by default order_by= 'id',
You don't need annotate something. You can:
get the last object:
Item.objects.filter(sku__in=list_of_skus).last()
get the last value of description:
Item.objects.filter(sku__in=list_of_skus).values_list('description', flat=True).last()
Both variants give you a None if a queryset is empty.
I'm having model with two field.
product_ids_list = [1,2,3,4]
selling_prices_list = [65, 89, 93]
And length of product_ids_list and selling_prices_list are same.
I'm able to perform below ORM filter for one product_id and it's corresponding selling price like this.
product_instances = Product.objects.filter(product_id=product_ids_list[0], selling_price=selling_prices_list[0]).first()
But how to do perform ORM filter with just one DB call with product_ids_list and it's corresponding selling_prices_list.
product_instances = Product.objects.filter(product_id__in=product_ids_list, selling_price__in=selling_prices_list).first() (This isn't working in the expected way)
If product_id is ForeignKey and selling_price is IntegerField then the filter code would be:
product_instances = Product.objects.filter(product_id__id__in=product_ids_list, selling_price__in=selling_prices_list).first()
As you are supplying list of IDs, we would have to compare product IDs. If we filter with product_id__in, it compares Product object with IDs in the list.
Suggesstion: It would be better to have ForeignKey field name as product instead of product_id to avoid confusion like the above ORM filter.
let's say this is my model :
class Item(models.Model):
user = models.ForeignKey(User, on_delete=models.DO_NOTHING)
price = models.DecimalField(max_digits=23, decimal_places=8, null=True, blank=True)
amount = models.DecimalField(max_digits=23, decimal_places=8)
i'm trying to get all the records which sum of their amount will be lesser than any integer i give.
for example from 20 records that exists , it returns first 5 records which sum of their amount is 1000 . it is like having these values : 100,400,300,100,100 . the sum is 1000 so it returns them as queryset.
it is possible to implement it with loops but i'm trying to handle it with django orm .
can anyone help me with this ?
You can use Window functions to run a cumulative sum per row, ordered by the primary key like this:
from django.db.models import Sum, Window
Item.objects.annotate(
cumulative_sum=Window(
Sum('price'),
order_by=F('id').asc()
)
).filter(cumulative_sum__lte=1000)
This will then return the first few item instances with prices that add up to less than or equal to 1000.
Gets all the items,
Create empty item_list,
Loop through all items,
If item amount is under 1000
append to item_list.
items = Item.objects.all()
item_list = []
for item in items:
if item.amount < 1000:
item_list.append(item)
There is a way to do this using aggregate function Sum from Django.
from django.db.models import Sum
amount_sum = Item.objects.filter(amount__lt=1000).aggregate(amount_sum=Sum("amount"))["amount_sum"]
However, The above Django query will have a equivalent SQL query as:
SELECT SUM("app_Item"."amount") AS "amount_sum"
FROM "app_Item"
WHERE "app_Item"."amount" < 1000; args=(Decimal('1000'),)
Prologue:
This is a question arising often in SO:
Subtracting two annotated columns
Django query with simple arithmetic among model fields and comparison with field from another model
Django Aggregation: Summation of Multiplication of two fields
And can also be applied here:
Django F expression on datetime objects
I have composed an example on SO Documentation but since the Documentation will get shut down on August 8, 2017, I will follow the suggestion of this widely upvoted and discussed meta answer and transform my example to a self-answered post.
Of course, I would be more than happy to see any different approach as well!!
Question:
Assume the following model:
class MyModel(models.Model):
number_1 = models.IntegerField()
number_2 = models.IntegerField()
date_1 = models.DateTimeField()
date_2 = models.DateTimeField()
How can I execute arithmetic operations between fields of this model?
For example, how can I find:
The product of number_1 and number_2 of a MyModel object?
How to filter items where date_2 is 10 or more days older than date_1?
F() expressions can be used to execute arithmetic operations (+, -, * etc.) among model fields, in order to define an algebraic lookup/connection between them.
An F() object represents the value of a model field or annotated column. It makes it possible to refer to model field values and perform database operations using them without actually having to pull them out of the database into Python memory.
let's tackle the issues then:
The product of two fields:
result = MyModel.objects.all().annotate(prod=F('number_1') * F('number_2'))
Now every item in result has an extra column named 'prod' which contains the product of number_1 and number_2 of each item respectively.
Filter by day difference:
from datetime import timedelta
result = MyModel.objects.all().annotate(
delta=F('date_2') - F('date_1')
).filter(delta__gte=timedelta(days=10))
Now the items in result are those from MyModel whose date_2 is 10 or more days older than date_1. These items have a new column named delta with that difference.
A different case:
We can even use F() expressions to make arithmetic operations on annotated columns as follows:
result = MyModel.objects.all()
.annotate(sum_1=Sum('number_1'))
.annotate(sum_2=Sum('number_2'))
.annotate(sum_diff=F('sum_2') - F('sum_1'))
Step 1: from django.db.models import F
Step 2: from datetime import timedelta
Step 3: You may then apply the F operator in your queryset as follows:
MyModel.objects.all().annotate(
date_diff=F('date_2') - F('date_1')
).filter(date_diff__gte=timedelta(days=10)
)
If I have a model Foo that has a simple M2M field to model Bar:
class Foo(Model):
bar = ManyToManyField(Bar)
Django seems to create a table foo_bar which has the following indices:
index 1: primary, unique (id)
index 2: unique (foo_id, bar_id)
index 3: non_unique (foo_id)
index 4: non_unique (bar_id)
I recall from my basic knowledge of SQL, that if a query needs to look for conditions on foo_id, index 2 would suffice (since the left-most column can be used for lookup). index 3 seems to be redundant.
Am I correct to assume that index 3 does indeed take up index space while offering no benefit? That I'm better off using a through table and manually create a unique index on (foo_id, bar_id), and optionally, another index on (bar_id) if needed?
The key to understanding how a many-to-many association is represented in the database is to realize that each line of the junction table (in this case, foo_bar) connects one line from the left table (foo) with one line from the right table (bar). Each pk of "foo" can be copied many times to "foo_bar"; each pk of "bar" can also be copied many times to "foo_bar". But the same pair of fk's in "foo_bar" can only occur once.
So if you have only one index (pk of "foo" or "bar") in "foo_bar" it can be only one occurrence of it ... and it is not Many to many relation.
For example we have two models (e-commerce): Product, Order.
Each product can be in many orders and one order can contain many products.
class Product(models.Model):
...
class Order(models.Model):
products = ManyToManyField(Product, through='OrderedProduct')
class OrderedProduct(models.Model):
# each pair can be only one time, so in one order you can calculate price for each product (considering ordered amount of it).
# and at the same time you can get somewhere in your template|view all orders which contain same product
order = models.ForeignKey(Order)
product = models.ForeignKey(Product)
amount = models.PositiveSmallIntegerField() # amount of ordered products
price = models.IntegerField() # just int in this simple example
def save(self, *args, **kwargs):
self.price = self.product__price * self.amount
super(OrderedProduct, self).save(*args, **kwargs)
For someone else who is still wondering. This is a known issue and there is an open ticket on the django bug tracker :
https://code.djangoproject.com/ticket/22125