Develop Reference

How can I join multiple models in Django into a virtual table?

If I have 3 models, like:
class Cow(models.Model):
name =
number_of_eyes =
number_of_feet =
color =
class Pig(models.Model):
name =
number_of_eyes =
number_of_feet =
intelligence =
class Horse(models.Model):
name =
number_of_eyes =
number_of_hooves =
weight_capacity =
speed =
And I'm interested in making a single Livestock table in my template that has instances of all 3, but I'm only interested in these columns that all 3 models have:
name
number_of_eyes
number_of_feet (number_of_hooves if Horse)
And we can ignore all other columns.
How can I join them into a single queryset?
The end goal is to get a single virtual table (queryset) that I can do a few other operations on (filter, order_by, slice), and then return the data in just those columns.
Is this possible in the Django ORM?

I think you have two options:
using itertools.chain:
from itertools import chain
cows = Cow.objects.all()
pigs = Pig.objects.all()
horses = Horse.objects.all()
livestock_list = sorted(
chain(cows, pigs, horses),
key=lambda livestock: livestock.created_at, reverse=True)
)
using contenttypes:
from django.contrib.contenttypes.models import ContentType
from django.contrib.contenttypes.fields import GenericForeignKey
class Livestock(models.Model):
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
created = models.DateTimeField(auto_now_add=True)
class Meta:
ordering = ['-created']
Now you can query Livestock model like any other model in Django, but you can have a foreign key that can refers to n models. that's what contenttypes do.
Livestock.content_object gives you what you want in your case it can be Cow, Pig or Horse.
Just remember to add objects to Livestock model after you create horse, etc instances. you need to add them in 2 models actually. you can do it with signals.
I think the second solution is better.

Apparently this can also be done using a Union, as suggested by Nick ODell:
from django.db.models import F
Cow.objects.filter(...).union(
Pig.objects.filter(...),
Horse.objects.filter(...).annotate(number_of_feet=F("number_of_hooves"))
).values('name', 'number_of_eyes', 'number_of_feet').order_by('name')[:3]
Unfortunately you can't filter on the resulting queryset after the union, so you need to filter each queryset before the union, but other than that, everything seems to work in my quick test.
From what I understand, the difference here from MojixCoder's suggestion of using ContentType is that you don't need to maintain a separate definition of this virtual table in your Django models module. In some cases, that can be an advantage, as you don't need to keep the module updated when you get new models you want to include in your query, but in other cases, it can be a disadvantage, because my way has a lot of typing every time you want to use this query, whereas in MojixCoder's example, you define it once, and your queries would be much shorter.
Edit: Using annotate and union can result in the results being out of order. Special care must be taken to ensure this doesn't happen

How to retrieve a queryset using any/exists/all logic across foreign key relationships in Django?

Assuming I have two models:
from django.db import models
class Parent(models.Model):
pass
class Child(models.Model):
parent = models.ForeignKey(Parent,
on_delete=models.CASCADE,
related_name='children')
active = models.BooleanField()
How might I go about getting a query set of parents that have at least 1 active child? In other words, how would I get a query set of parents that excludes those without any active children? If using model properties in filters was feasible, this would be a trivial task, but that is not possible. This is also a simple operation using list comprehensions, but it's important here for the queryset to be the end result.

One solution is to use aggregation: https://docs.djangoproject.com/en/2.1/topics/db/aggregation/
We can annotate a queryset of parents with the number of active children they have, and then filter against that queryset to only find parents with a number greater than or equal to 0:
from django.db.models import Count, Q
num_active_children = Count('children', filter=Q(children__active=True))
parents_with_any_active_children = (Parent.objects
.annotate(num_active_children=num_active_children)
.filter(num_active_children__gte=1)
)

Parent.objects.filter(child__active=True)
Hope this is enough for you.

How to combine django "prefetch_related" and "values" methods?

How can prefetch_related and values method be applied in combination?
Previously, I had the following code. Limiting fields in this query is required for performance optimization.
Organizations.objects.values('id','name').order_by('name')
Now, I need to prefetch its association and append it in the serializer using "prefetch_related" method.
Organizations.objects.prefetch_related('locations').order_by('name')
Here, I cannot seem to find a way to limit the fields after using "prefetch_related".
I have tried the following, but on doing so serializer does not see the associated "locations".
Organizations.objects.prefetch_related('locations').values("id", "name").order_by('name')
Model Skeleton:
class Organizations(models.Model):
name = models.CharField(max_length=40)
class Location(models.Model):
name = models.CharField(max_length=50)
organization = models.ForeignKey(Organizations, to_field="name", db_column="organization_name", related_name='locations')
class Meta:
db_table = u'locations'

Use only() to limit number of fields retrieved if you're concerned about your app performances. See reference.
In the example above, this would be:
Organizations.objects.prefetch_related('locations').only('id', 'name').order_by('name')
which would result in two queries:
SELECT id, name FROM organizations;
SELECT * from locations WHERE organization_name = <whatever is seen before>;

Django aggregation query on related one-to-many objects

Here is my simplified model:
class Item(models.Model):
pass
class TrackingPoint(models.Model):
item = models.ForeignKey(Item)
created = models.DateField()
data = models.IntegerField()
class Meta:
unique_together = ('item', 'created')
In many parts of my application I need to retrieve a set of Item's and annotate each item with data field from latest TrackingPoint from each item ordered by created field. For example, instance i1 of class Item has 3 TrackingPoint's:
tp1 = TrackingPoint(item=i1, created=date(2010,5,15), data=23)
tp2 = TrackingPoint(item=i1, created=date(2010,5,14), data=21)
tp3 = TrackingPoint(item=i1, created=date(2010,5,12), data=120)
I need a query to retrieve i1 instance annotated with tp1.data field value as tp1 is the latest tracking point ordered by created field. That query should also return Item's that don't have any TrackingPoint's at all. If possible I prefer not to use QuerySet's extra method to do this.
That's what I tried so far... and failed :(
Item.objects.annotate(max_created=Max('trackingpoint__created'),
data=Avg('trackingpoint__data')).filter(trackingpoint__created=F('max_created'))
Any ideas?

Here's a single query that will provide (TrackingPoint, Item)-pairs:
TrackingPoint.objects.annotate(max=Max('item__trackingpoint__created')).filter(max=F('created')).select_related('item').order_by('created')
You would have to query for items without TrackingPoints separately.

This isn't directly answer to your question, but in case don't need exactly what you described you might be interested in greatest-n-per-group solution. You can take a look on my answer on similar question:
Django Query That Get Most Recent Objects From Different Categories
-- this should apply directly to your case:
items = Item.objects.annotate(tracking_point_created=Max('trackingpoint__created'))
trackingpoints = TrackingPoint.objects.filter(created__in=[b.tracking_point_created for b in items])
Note that second line can produce ambiguous results if created dates repeat in TrackingPoint model.

How to do this join query in Django

In Django, I have two models:
class Product(models.Model):
name = models.CharField(max_length = 50)
categories = models.ManyToManyField(Category)
class ProductRank(models.Model):
product = models.ForeignKey(Product)
rank = models.IntegerField(default = 0)
I put the rank into a separate table because every view of a page will cause the rank to change and I was worried that all these writes would make my other (mostly read) queries slow down.
I gather a list of Products from a simple query:
cat = Category.objects.get(pk = 1)
products = Product.objects.filter(categories = cat)
I would now like to get all the ranks for these products. I would prefer to do it all in one go (using a SQL join) and was wondering how to express that using Django's query mechanism.
What is the right way to do this in Django?

This can be done in Django, but you will need to restructure your models a little bit differently:
class Product(models.Model):
name = models.CharField(max_length=50)
product_rank = models.OneToOneField('ProductRank')
class ProductRank(models.Model):
rank = models.IntegerField(default=0)
Now, when fetching Product objects, you can following the one-to-one relationship in one query using the select_related() method:
Product.objects.filter([...]).select_related()
This will produce one query that fetches product ranks using a join:
SELECT "example_product"."id", "example_product"."name", "example_product"."product_rank_id", "example_productrank"."id", "example_productrank"."rank" FROM "example_product" INNER JOIN "example_productrank" ON ("example_product"."product_rank_id" = "example_productrank"."id")
I had to move the relationship field between Product and ProductRank to the Product model because it looks like select_related() follows foreign keys in one direction only.

I haven't checked but:
products = Product.objects.filter(categories__pk=1).select_related()
Should grab every instance.

For Django 2.1
From documentation
This example retrieves all Entry objects with a Blog whose name is 'Beatles Blog':
Entry.objects.filter(blog__name='Beatles Blog')
Doc URL
https://docs.djangoproject.com/en/2.1/topics/db/queries/

Add a call to the QuerySet's select_related() method, though I'm not positive that grabs references in both directions, it is the most likely answer.