Develop Reference

How to get the latest (distinct) records filtered by a non unique field in Django

I'll demonstrate by using an example. This is the model (the primary key is implicit):
class Item(models.Model):
sku = models.CharField(null=False)
description = models.CharField(null=True)
I have a list of skus, I need to get the latest descriptions for all skus in the filter list that are written in the table for the model Item. Latest item == greatest id.
I need a way to annotate the latest description per sku:
Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').order_by("-id")
but this won't work for various reasons (excluding the missing aggregate function).

Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').lastest("-id")
Or use this
Item.objects.values("sku").filter(sku__in=list_of_skus).annotate(latest_descr=Latest('description').order_by("-id").reverse()[0]

I used postgres ArrayAgg aggregate function to aggregate the latest description like so:
from django.contrib.postgres.aggregates import ArrayAgg
class ArrayAggLatest(ArrayAgg):
template = "(%(function)s(%(expressions)s ORDER BY id DESC))[1]"
Item.objects.filter(sku__in=skus).values("sku").annotate(descr=ArrayAggLatest("description"))
The aggregate function aggregates all descriptions ordered by descending ID of the original table and gets the 1st element (0 element is None)

Answer from #M.J.GH.PY or #dekomote war not correct.
If you have a model:
class Item(models.Model):
sku = models.CharField(null=False)
description = models.CharField(null=True)
this model has already by default order_by= 'id',
You don't need annotate something. You can:
get the last object:
Item.objects.filter(sku__in=list_of_skus).last()
get the last value of description:
Item.objects.filter(sku__in=list_of_skus).values_list('description', flat=True).last()
Both variants give you a None if a queryset is empty.

SQLAlchemy: how to obtain all distinct values from Array field?

I have the following model of a blog post:
title = db.Column(db.String())
content = db.Column(db.String())
tags = db.Column(ARRAY(db.String))
Tags field can be an empty list.
Now I want to select all distinct tags from the database entries with max performance - excluding empty arrays.
So, say I have 3 records with the following values of the tags field:
['database', 'server', 'connection']
[]
['connection', 'security']
The result would be ['database', 'server', 'connection', 'security']
The actual order is not important.

The distinct() method should still work fine with array columns.
from sqlalchemy import func
unique_vals = BlogPost.query(func.unnest(BlogPost.tags)).distinct().all()
https://docs.sqlalchemy.org/en/13/orm/query.html?highlight=distinct#sqlalchemy.orm.query.Query.distinct
This would be identical to running a query in postgres:
SELECT DISTINCT unnest(tags) FROM blog_posts

If you can process the results after(usually you can) and don't want to use a nested query for this sort of thing, I usually resort to doing something like;
func.array_agg(func.array_to_string(BlogPost.tags, "||")).label("tag_lists")
and then split on the join string(||) after.

Dealing with Arrays in Flask-SqlAlchemy and MySQL

I have a datamodel where I store a list of values separated by comma (1,2,3,4,5...).
In my code, in order to work with arrays instead of string, I have defined the model like this one:
class MyModel(db.Model):
pk = db.Column(db.Integer, primary_key=True)
__fake_array = db.Column(db.String(500), name="fake_array")
#property
def fake_array(self):
if not self.__fake_array:
return
return self.__fake_array.split(',')
#fake_array.setter
def fake_array(self, value):
if value:
self.__fake_array = ",".join(value)
else:
self.__fake_array = None
This works perfect and from the point of view of my source code "fake_array" is an array, It's only transformed into string when it's stored in database.
The problem appears when I try to filter by that field. Expressions like this doesn't work:
MyModel.query.filter_by(fake_array="1").all()
It seems that I cant filter using the SqlAlchemy query model.
What can I do here? Is there any way to filter this kind of fields? Is there is a better pattern for the "fake_array" problem?
Thanks!

What you're trying to do should really be replaced with a pair of tables and a relationship between them.
The first table (which I'll call A) contains everything BUT the array column, and it should have a primary key of some sort. You should have another table (which I'll call B) that contains a primary key, a foreign key column to A (which I'll call a_id, and an integer field.
Using this layout, each row in the A table has its associated array in table B where B's a_id == A.id via a join. You can add or remove values from the array by manipulating the rows in table B. You can filter by using a join.
If the order of the values is needed, then create an order column in table B.

Django how to join two query sets so that they are sequentially serialized

Suppose I have a model call MyModel defined as below
class MyModel(models.Model):
fk = models.ForeignKey('AnotherModel')
rank = models.FloatField()
I wish to create a query set for serialization so that instances with instance.fk.other_fk_id in some set comes before the ones that don't and then sorted by decreasing rank.
So I wish to join
a = MyModel.objects.filter(fk__other_fk_id__in=some_set).order_by('-rank')
and
b = MyModel.objects.exclude(fk__other_fk_id__in=some_set).order_by('-rank')
sequentially so that a comes before b when it's handed to the serializers. Any idea on how to achieve this functionality efficiently? (let's say there is 50 instances in a and 200000 instances in b so I cannot directly concat those two as lists).
The end goal is to feed it into a custom serializer w/ pagination, say, to present in a webpage.

ok I figured this out. What you can do is to annotate another value, join them and sort by two fields.
a = a.annotate(rank2=Value(1, IntegerField()))
b = b.annotate(rank2=Value(0, IntegerField()))
qs = a.union(b).order_by('-rank2', '-rank')

You can use itertools.chain:
from itertools import chain
from django.core import serializers
data = serializers.serialize("xml", chain(a, b))

Use a model field to query another model field in django

I have two models in our django app
class Reg(models.Model):
transactions = ManyToMany
price = IntegerField
class Transaction(models.Model)
amount = IntegerField
Now I would like to make a lookup like:
Registration.objects.filter(reg__price==transaction__amount)
Previously we used the following approach:
Registration has a property is_paid that computes wether a transaction with equal amount exists
[r for r in Registration.objects.filter(...) if r.is_paid]
This is ofc very query-consuming and inefficient.
I wonder whether there would be a better way to do this!
Any hint is appreciated :)

You can use an F expression for such a query:
from django.db.models import F
Registration.objects.filter(price=F('transactions__amount'))
This will filter all Registration instances whose price is equal to one of their transactions' amount. If you want all transactions amounts' sum to be equal or more than the registration price, you can use annotations to aggregate each registration's Sum:
paid_registrations = Registration.objects.\
annotate(ta=Sum('transactions__amount')).\ # annotate with ta sum
filter(price__lte=F('ta')) # filter those whose price is <= that sum