Users can upload three different types of content onto our site: image, video, audio. Here are the models for each type:
class ImageItem(models.Model):
user = models.ForeignKey(User)
upload_date = models.DateTimeField(auto_now_add=True)
image = models.ImageField(upload_to=img_get_file_path)
title = models.CharFiled(max_length=1000,
blank=True)
class VideoItem(models.Model):
user = models.ForeignKey(User)
upload_date = models.DateTimeField(auto_now_add=True)
video = models.FileField(upload_to=vid_get_file_path)
title = models.CharFiled(max_length=1000,
blank=True)
class AudioItem(models.Model):
user = models.ForeignKey(User)
upload_date = models.DateTimeField(auto_now_add=True)
audio = models.FileField(upload_to=aud_get_file_path)
title = models.CharFiled(max_length=1000,
blank=True)
I have a page called library.html, which renders all the items that a user has uploaded, in order from most recently uploaded to oldest uploads (it displays the title and upload_date of each instance, and puts a little icon on the left symbolizing what kind of item it is).
Assuming it requires three separate queries, how can I merge the three querysets? How can I make sure they are in order from most recently uploaded?
As an alternative, you can use multi-table inheritance and factor common attributes into a superclass model. Then you just order_by upload date on the superclass model. The third-party app django-model-utils provides a custom manager called Inheritance manager that lets you automatically downcast to the subclass models in your query.
from model_utils.managers import InheritanceManager
class MediaItem(models.Model):
objects = InheritanceManager()
user = models.ForeignKey(User)
upload_date = models.DateTimeField(auto_now_add=True)
title = models.CharFiled(max_length=1000,
blank=True)
class ImageItem(MediaItem):
image = models.ImageField(upload_to=img_get_file_path)
class VideoItem(MediaItem):
video = models.FileField(upload_to=vid_get_file_path)
class AudioItem(MediaItem):
audio = models.FileField(upload_to=aud_get_file_path)
Then, your query is just:
MediaItem.objects.all().order_by('upload_date').select_subclasses()
This way, you get what you want with one just query (with 3 joins). Plus, your data is better normalized, and it's fairly simple to support all sorts more complicated queries, as well as pagination.
Even if you don't go for that, I'd still use abstract base class inheritance to conceptually normalize your data model, even though you don't get the database-side and ORM benefits.
attrgetter can be used to pull out attributes of objects that you may want to key a sort by.
from operator import attrgetter
results = []
results.extend(list(AudioItem.objects.filter(...)))
results.extend(list(VideoItem.objects.filter(...)))
results.extend(list(ImageItem.objects.filter(...))
results.sort(key=attrgetter("upload_date")
Referencing the question, this is the exact solution I used. Influenced by monkut's answer and the top answer from Frantzdy's link in his comment (thanks guys).
from itertools import chain
from operator import attrgetter
images = ImageItem.objects.filter(user=user)
video = VideoItem.objects.filter(user=user)
audio = AudioItem.objects.filter(user=user)
result_list = sorted(chain(images, video, audio),
key=attrgetter('upload_date'),
reverse=True)
Related
I'm creating a blog in Django where I have a base model PostType which I then extend in to several subclasses for different types of content on the website. For example CodeSnippet and BlogPost.
The idea is that these content types are mostly the same, they all have an author, a title, a slug, etc, but they also have a few unique fields. For example a blog post has a field for the text content, while a code snippet has a related field for programming language.
Something like this:
class PostType(models.Model):
author = models.ForeignKey(
settings.AUTH_USER_MODEL,
on_delete=models.CASCADE
)
title = models.CharField(
max_length=255,
unique=True,
)
class Meta:
abstract = True
class BlogPost(PostType):
content = models.TextField(
default='',
)
class GitHubRepo(PostType):
url = models.URLField(
unique=True
)
class CodeSnippet(PostType):
language = models.ForeignKey(
to=Language,
on_delete=models.CASCADE,
)
Now what I want to know is if there's any good/prefered way to query all objects in the database that are based on the parent class PostType?
For the site's search I am currently querying each of the different content types, and then merging the result. This is the code for the search view:
class Search(View):
def get(self, request):
context = {}
try:
query = request.GET.get('s')
blog_list = models.BlogPost.objects.filter(title__icontains=query)
projects_list = models.Project.objects.filter(title__icontains=query)
code_list = models.CodeSnippet.objects.filter(title__icontains=query)
from itertools import chain
context['result_list'] = list(chain(blog_list, projects_list, code_list))
except KeyError:
query = ''
context['title'] = 'Result for "{}"'.format(query)
return render(request, 'blog/search.html', context)
This all works fine, but I would like to know if there's any way to query all children of PostType at the same time?
Is Django somehow aware of what child models exist? And can I use that somehow?
Like a PostType.child_objects.get() or something similar.
Even a way to programmatically get all the children so that I could loop through them and get all the objects would be fine too.
For the time being I just have a few models, but the number of child models were to increase, it would be great if I could be assured that all the models would be included in the site search automatically based on their relationship to their parent model.
PostType is an abstract Model (So, it does not create physical table. It's just to use inheritance feature in Django). As far as i understand you want to generate list of QuerySet's merge it in a single list and iterate over list/QuerySet later.
get_qs_list = [model.objects.filter(title__icontains=query) for model in PostType.__subclasses__()] # This contains QuerySet instances now.
for qs in get_qs_list:
# qs iterator returns QuerySet instance
for instance in qs:
# instance iterator is single Model instance from QuerySet collection
print(instance.title)
Hope, it helps you.
If PostType is not an abstract model then you should be able to query it directly to get all those subclass results
PostType.objects.filter(title__icontains=query)
Otherwise, you cannot really do this with a single query.
Even a way to programmatically get all the children so that I could
loop through them and get all the objects would be fine too.
This is possible --- to get the subclasses programmatically, you would do
PostType.__subclasses__()
I am a new user of Django, and I am trying to figure out how to created a model which can support many kind (type) of elements.
This is the plot : I want to create a Blog module on my application.
To do this, I created a model Page, which describe a Blog Page. And a model PageElement, which describe a Post on the blog. Each Page can contain many PageElement.
A PageElement can have many types, because I want my users could post like just a short text, or just a video, or just a picture. I also would like (for example) the user could just post a reference to another model (like a reference to an user). Depending of the kind of content the user posted, the HTML page will display each PageElement in a different way.
But I don't know what is the right way to declare the PageElement class in order to support all these cases :(
Here is my Page model :
class Page(models.Model):
uuid = models.UUIDField(default=uuid.uuid4, editable=False, unique=True)
# Basical informations
title = models.CharField(max_length=150)
description = models.TextField(blank=True)
# Foreign links
user = models.ForeignKey(
settings.AUTH_USER_MODEL,
on_delete=models.SET_NULL,
null=True,
related_name='pages_as_user'
)
created_at = models.DateTimeField(default=timezone.now)
# Other fields ....
class Meta:
indexes = [
models.Index(fields=['uuid']),
models.Index(fields=['user', 'artist'])
]
For now, I have two solutions, the first one use inheritance : When you create a new post on the blog, you create an Element which inherit from PageElement model. Here are my different Models for each cases :
class PageElement(models.Model):
page = models.ForeignKey(
Page,
on_delete=models.CASCADE,
related_name='%(class)s_elements'
)
updated_at = models.DateTimeField(default=timezone.now)
created_at = models.DateTimeField(default=timezone.now)
class PageImageElement(PageElement):
image = models.ImageField(null=True)
image_url = models.URLField(null=True)
class PageVideoElement(PageElement):
video = models.FileField(null=True)
video_url = models.URLField(null=True)
class PageTextElement(PageElement):
text = models.TextField(null=True)
class PageUserElement(PageElement):
user = models.ForeignKey(
'auth.User',
on_delete=models.CASCADE,
related_name='elements'
)
This solution would be the one I have choosen if I had to work with "pure" Python. Because I could stored each PageElement in a dictionnary and filter them by class. And this solution could be easily extended in the futur with new type of content.
But with Django models. It seems that is not the best solution. Because it will be really difficult to get all PageElement children from the database (I can't just write "page.elements" to get all elements of all types, I need to get all %(class)s_elements elements manually and concatenate them :/). I have thinked about a solution like below (I don't have tried it yet), but it seems overkilled for this problem (and for the database which will have to deal with a large number of request):
class Page(models.Model):
# ...
def get_elements(self):
# Retrieve all PageElements children linked to the current Page
R = []
fields = self._meta.get_fields(include_hidden=True)
for f in fields:
try:
if '_elements' in f.name:
R += getattr(self, f.name)
except TypeError as e:
continue
return R
My second "solution" use an unique class which contains all fields I need. Depending of the kind of PageElement I want to create, I would put type field to the correct value, put the values in the corresponding fields, and put to NULL all other unused fields :
class PageElement(models.Model):
page = models.OneToOneField(
Page,
on_delete=models.CASCADE,
related_name='elements'
)
updated_at = models.DateTimeField(default=timezone.now)
created_at = models.DateTimeField(default=timezone.now)
TYPES_CHOICE = (
('img', 'Image'),
('vid', 'Video'),
('txt', 'Text'),
('usr', 'User'),
)
type = models.CharField(max_length=60, choices=TYPES_CHOICE)
# For type Image
image = models.ImageField(null=True)
image_url = models.URLField(null=True)
# For type Video
video = models.FileField(null=True)
video_url = models.URLField(null=True)
# For type Text
text = models.TextField(null=True)
# For type User
user = models.ForeignKey(
'auth.User',
on_delete=models.CASCADE,
related_name='elements',
null=True
)
With this solution, I can retrieve all elements in a single request with "page.elements". But it is less extendable than the previous one (I need to modify my entire table structure to add a new field or a new kind of Element).
To be honnest, I have absolutly no idea of which solution is the best. And I am sure other (better) solutions exist, but my poor Oriented-Object skills don't give me the ability to think about them ( :( )...
I want a solution which can be easily modified in the future (if for example, I want to add a new Type "calendar" on the Blog, which reference a DateTime). And which would be easy to use in my application if I want to retrieve all Elements related to a Page...
Thanks for your attention :)
I'm not sure it fits your problem but using GenericForeignKeys/ContentType framework may be appropriate in this case. It's quite powerful when one grasps the concept.
Example construct:
class Page(models.Model):
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
page_element = GenericForeignKey('content_type', 'object_id')
...
You can now connect any model object by the GenericFK to the Page model. So adding a new type (as a new model), at a later stage, is not intrusive.
Update:
As a comment pointed out this construct doesn't support many PageElements in a good way for a Page.
To elaborate, one way to solve that problem, still taking advantage of the GenericFK...
class PageElement(models.Model):
class Meta:
unique_together=('page', 'content_type', 'object_id') # Solve the unique per page
page = models.ForeignKey(Page, related_name='page_elements')
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
A Page can have many "abstract" PageElements and content_object is the "concrete PageElement model/implementation".
Easy to retrieve all elements for a specific page and allows inspection of the ContentType to check the type of element etc.
Just one way of many to solve this particular problem.
To establish the relationship between Page and PageElement in Django you would rather use Foreign Key relationship, than inheritance.
class PageImageElement(PageElement):
page = models.ForeignKey(Page,
on_delete=models.CASCADE,
related_name='images')
image = models.ImageField(null=True)
image_url = models.URLField(null=True)
Every user's post would create an instance of Page. Every addition of image to the Page would create an instance of PageImageElement and you could query for them using the related name. This way would be really easy to access all video, image, text modules of a single Page.
On a related note, I would say that PageElement class could be abstract see the docs and if you declare fields as possibly containing null values as in video = models.FileField(null=True) then it might be worth declaring blank=True as well, otherwise there will be errors when creating the object with these fields undefined. Discussed, for example, here: differentiate null=True, blank=True in django
I can't just write "page.elements" to get all elements of all types
Well actually, you can if you use multi-table inheritance. The problem is that all records returned are instances of PageElement, meaning you lose all information of the subclass type and the additional data these child objects may hold.
There are quite a lot of packages that tackle this polymorphism problem:
django packages: Model inheritance
I'm building a simple API for an ESP8266 to connect to in an IoT application, passing a JSON string. In this application there are multiple Monitors (internet connected devices) per Site (location/address), and multiple LogEntries per Site/Monitor.
The API was originally setup with an endpoint like:
/api/logentries/
Posting a JSON string like:
{"site":"abcd","monitor":"xyz","data_point":"value"}
In the object model, Monitor is a child of Site, but for convenience of entry creation and reporting, the JSON format of the LogEntry posted by each device flattens this structure out, meaning that the LogEntry model also has a FK relationship for both Site and Monitor. In the code below, "textID" is the ID used within the context of the API for the Site/Monitor (e.g. PK values remain "hidden" for API callers).
In models.py:
class Site(models.Model):
name = models.CharField(max_length=32)
textID = models.CharField(max_length=32, blank=True, db_index=True, unique=True)
class Monitor(models.Model):
textID = models.CharField(max_length=32)
site = models.ForeignKey(Site, on_delete=models.CASCADE)
class Meta:
unique_together = ('site', 'textID')
class LogEntry(models.Model):
site = models.ForeignKey(Site, on_delete=models.CASCADE)
monitor = models.ForeignKey(Monitor, on_delete=models.CASCADE)
data_point = models.CharField(max_length=8, default='')
To get this to work on a single site, I created a custom serializer:
class LogEntrySerializer(serializers.HyperlinkedModelSerializer):
site = serializers.SlugRelatedField(slug_field='textID', queryset=Site.objects.all())
monitor = serializers.SlugRelatedField(slug_field='textID', queryset=Monitor.objects.filter())
class Meta:
model = LogEntry
fields = ('pk', 'site', 'monitor', 'data_point', )
This works for reading valid data, and saving when all monitor IDs are unique across sites.
However, if two sites have a Monitor with the same textID—e.g. "Site1/001" and "Site2/001" this breaks, as the Monitor.objects.all() results in multiple records being retrieved (which makes sense and is expected behaviour).
What I'm wanting to do is to have the second queryset (for monitor) limited to the specified site, to avoid this error.
This post almost answers my question, however it benefits from the second field value (user) being available in the request object, something that is not available in this case.
Is there a way I can retrieve the Site.pk or Site.textID for the queryset value to resolve correctly--e.g. queryset=Monitor.objects.filter(site__textID=xxx)--what would 'xxx' be? Or do I need to completely override the serializer (and not rely on SlugRelatedField)? Or some other approach that might work?
(As an aside: I recognise that this could be achieved by modifying the URL pattern to something like /api///logentries, which would then have this information available as part of the request/context and from a normalisation perspective would be better also. However this would require reflashing of a number of already deployed devices to reflect the changed API details, so I'd like to avoid such a change if possible, even though upon reflection this is probably a cleaner solution/approach long-term.)
Thanks in advance.
You'll need to write your own SlugRelatedField subclass. The unicity constraint that applies to a SlugRelatedField doesn't apply to your case.
This can be done by creating a subfield and overriding the get_value to retrieve the site/monitor tuple and to_internal_value to select the appropriate monitor.
Thanks to the pointers from Linovia, the following field class resolves the issue:
class MonitorRelatedField(serializers.Field):
def to_representation(self, obj):
return obj.textID
def get_value(self, data):
site_textID = data['site']
monitor_textID = data['monitor']
return ( site_textID, monitor_textID, )
def to_internal_value(self, data):
return Monitor.objects.get(site__textID=data[0], textID=data[1])
In a Django-based social website I maintain, users post photos. Each posted photo is a part of one photostream (i.e. a list of related photos). I calculate the 200 most recent photos via the get_queryset method of a CBV (ListView):
def get_queryset(self):
return Photo.objects.order_by('-id')[:200]
Next, for each photo, I append the count of the number of related photos exist. That I do by first checking which photostream each photo belongs to, then getting other photos from the said stream, and lastly excluding some based on freshness. In other words:
for obj in context["object_list"]:
count = Photo.objects.filter(which_stream=obj.which_stream).order_by('-upload_time').exclude(upload_time__gt=obj.upload_time).count()
The count is then paired with each obj so that I end up with a dictionary to use in populating the template. As you would have guessed, I basically use this info to show the count of related photos alongwith each listed photo.
But doing it this way is just too many DB lookups! How can I optimize this, for performance? Please advise!
Here's the photo and photostream data models with relevant fields:
class Photo(models.Model):
owner = models.ForeignKey(User)
which_stream = models.ForeignKey(PhotoStream)
image_file = models.ImageField(upload_to=upload_photo_to_location, storage=OverwriteStorage())
upload_time = models.DateTimeField(auto_now_add=True, db_index=True)
class PhotoStream(models.Model):
stream_cover = models.ForeignKey(Photo)
children_count = models.IntegerField(default=1)
creation_time = models.DateTimeField(auto_now_add=True)
Plesae check if you could use Conditional Aggregations like this:
from django.db.models import Count, Case, When, IntegerField
Photo.objects.annotate(
count=Count(Case(
When(which_stream__photo__upload_time__lte=F('upload_time')), then=1),
output_field=IntegerField(),
))
).order_by('-id')[:200]
I haven't tested this but I think you will get an idea how to use it.
okay so I have just a test blog system to practice my django skills. I have 2 apps one called article on called likes here they are:
article models:
from django.db import models
# Create your models here.
class Article(models.Model):
title = models.CharField(max_length = 200)
description = models.TextField()
pub_date = models.DateTimeField('Date Published',auto_now = True)
def __str__(self):
return self.title
and here is the likes models:
from django.db import models
from apps.article.models import Article
# Create your models here.
class Like(models.Model):
article_id = models.ForeignKey(Article)
likes = models.IntegerField(default = 0)
def __str__(self):
return self.Likes
now im rendering out the pages but I want to display how many likes each article has. How can I join both these models. Change the objects.all method to also grab the likes from the Like model
You can use FOO_set, docs about this here, basically you do this to get all likes:
article.likes_set.all()
and you can just use count() to get number
First you might want to rename article_id to article since when you use the attribute, you will actually get the article and not just the id.
In this case you seem to have a many-to-one relationship between Like and Article. That means you need to refer to the likes as 'like_set'. So if you happen to have the object stored in article, you can get all the likes with article.like_set.all() and the count with article.like_set.count().
Reference: https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_one/
If you're interested in fetching this ahead of time you can use prefetch_related to save the additional database calls:
https://docs.djangoproject.com/en/1.4/ref/models/querysets/#prefetch-related
It would like something like this:
articles = Article.objects.all().prefetch_related('like_set')