Develop Reference

Efficiently bulk updating many ManyToMany fields

A version of this question has been asked here several times but none of the answers provided solve my exact problem.
I'm trying to bulk_create a batch of objects of a model with a ManyToMany field.
In this case, the ManyToMany field refers to the same model, though I'd also be interested in the general case.
Let's say this is my model:
class Person(models.Model):
name = models.CharField(max_length=20, blank=True, null=True)
friends = models.ManyToMany("self", related_name="friends_with", null=True)
After bulk_creating a large number of Person objects, I want to add the information who's friends with whom within this group.
Is there a more efficient way to go about this than looping through each new Person and calling .set(friend_pks) or .add(*friend_pks)?
I.e., an analogue of bulk_update.
I've achieved some speed-up by wrapping the loop into with transaction.atomic() (from this answer) but it's still quite slow.

Okay, my post was premature -- it seems that this answers the question.
The key is to bulk_create the through models. In this example:
friends_relation_1 = Person.friends.through(from_person_id=1, to_person_id=2)
friends_relation_2 = Person.friends.through(from_person_id=2, to_person_id=8)
Person.friends.through.objects.bulk_create([friends_relation_1, friends_relation_2, ...])

Implement items ordering in Django

Suppose, I want to build a simple TODO-app. I want to make it possible to create todo-items, and also it should be possible to rearrange items manually.
I made following model:
class Item(models.Model):
title = models.CharField(max_length=500)
Now, I need to add a special field, let's call it order, to keep custom ordering. It should be unique, and it should be greater for any new record, so I tried to make an AutoField.
class Item(models.Model):
title = models.CharField(max_length=500)
order = models.AutoField(primary_key=False)
But it turned out that Django doesn't support several auto fields for a single model.
I guess, it should be possible to write custom raw SQL code and use Postgres sequences directly, but it would look really ugly, and I don't want to write DB-specific code for such simple functionality.
So, here is a question: what is a common way to implement items ordering in Django? It sounds like a very common requirement, and, I think, it should be a simple way to do this.

As it turns out there is no straightforward way to implement this in Django. There are packages which help you, like this one
But I would recommend just look at their model implementation and fit your needs. models.py

You could use Item.objects.count() to automatically increment your field. Plug it in the save() method of your model so that your field is calculated each time you create an instance.

What is the best way to implement persistent data model in Django?

What I need is basically a database model with version control. So that every time a record is modified/deleted, the data isn't lost, and the change can be undone.
I've been trying to implement it myself with something like this:
from django.db import models
class AbstractPersistentModel(models.Model):
time_created = models.DateTimeField(auto_now_add=True)
time_changed = models.DateTimeField(null=True, default=None)
time_deleted = models.DateTimeField(null=True, default=None)
class Meta:
abstract = True
Then every model would inherit from AbstractPersistentModel.
Problem is, if I override save() and delete() to make sure they don't actually touch the existing data, I'll still be left with the original object, and not the new version.
After trying to come up with a clean, safe and easy-to-use solution for some hours, I gave up.
Is there some way to implement this functionality that isn't overwhelming?
It seems common enough problem that I thought it would be built into Django itself, or at least there'd be a well documented package for this, but I couldn't find any.

When I hear version control for models and Django, I immediately think of django-reversion.
Then, if you want to access the versions of an instance, and not the actual instance, simply use the Version model.
from reversion.models import Version
versions = Version.objects.get_for_object(instance)

I feel you can work around your issue not by modifying your models but by modifying the logic that access them.
So, you could have two models for your same object: one that can be your staging area, in which you store values as the ones you mention, such as time_created, time_modified, and modifying_user, or others. From there, in the code for your views you go through that table and select the records you want/need according to your design and store in your definitive table.

Number of attributes in Django Models

I searched a lot and did not find what I´am looking for.
What would be the best concept for a model class in django?
To extend User, would be better to have a class with several attributes, or break this class into several classes with few attributes? I´m using the django ORM now.
Say I have a class called Person that extends User, would be better:
class Person(models.Model):
user = foreingkey(User)
attribute1 =
...
attributeN =
Or, would it be better to do this:
class PersonContac(models.Model):
user = foreingkey(User)
attribute1 =
...
attribute3 =
class PersonAddress(models.Model):
user = foreingkey(User)
attribute1 =
...
attribute3 =
class PersonHobby(models.Model):
user = foreingkey(User)
attribute1 =
...
attribute3 =
My each of my views would use the data from the smaller classes (probably).
Over time, the atrribute number can expand.
I want to do is do it once, and touch the minimum possible.
Various attributes can be unfilled by the user, they are not required.
The number of user is indefinite (can be a lot).
I´m concerned in terms of long term performance and maintaining.
If someone can explain me, what would be better for my code, and why.
And what would be better in general (less classes/more attributes, or more classes/less attributes), using the Django ORM.
It is better if my views use the data of only one model class, or it makes no (or little) difference?
Edit:
On the rush for writing I used bad names on class. None of these attributes are many-to-many fields, the User will have only one value for each attribute, or blank.
The number of atributes can expand over time, but not in a great number.

Put any data that is specific to only one User directly in the model. This would probably be things like "Name", "Birthday", etc.
Some things might be better served by a separate model, though. For example multiple people might have the same Hobby or one User might have multiple Hobby(s). Make this a separate class and use a ForeignKeyField or ManyToManyField as necessary.
Whatever you choose, the real trick is to optimize the number of database queries. The django-debug-toolbar is helpful here.
Splitting up your models would by default result in multiple database queries, so make sure to read up on select related to condense that down to one.
Also take a look at the defer method when retrieving a queryset. You can exclude some of those fields that aren't necessary if you know you won't use them in a particular view.

I think it's all up to your interface.
If you have to expose ALL data for a user in a single page and you have a single, large model you will end up with a single sql join instead of one for each smaller table.
Conversely, if you just need a few of these attributes, you might obtain a small performance gain in memory usage if you join the user table with a smaller one because you don't have to load a lot of attributes that aren't going to be used (though this might be mitigated through values (documentation here)
Also, if your attributes are not mandatory, you should at least have an idea of how many attributes are going to be filled. Having a large table of almost empty records could be a waste of space. Maybe a problem, maybe not. It depends on your hw resources.
Lastly, if you really think that your attributes can expand a lot, you could try the EAV approach.

Querying through several models

I have a django project with 5 different models in it. All of them has date field. Let's say i want to get all entries from all models with today date. Of course, i could just filter every model, and put results in one big list, but i believe it's bad. What would be efficient way to do that?

I don't think that it's a bad idea to query each model separately - indeed, from a database perspective, I can't see how you'd be able to do otherwise, as each model will need a separate SQL query. Even if, as #Nagaraj suggests, you set up a common Date model every other model references, you'd still need to query each model separately. You are probably correct, however, that putting the results into a list is bad practice, unless you actually need to load every object into memory, as explained here:
Be warned, though, that [evaluating a QuerySet as a list] could have a large memory overhead, because Django will load each element of the list into memory. In contrast, iterating over a QuerySet will take advantage of your database to load data and instantiate objects only as you need them.
It's hard to suggest other options without knowing more about your use case. However, I think I'd probably approach this by making a list or dictionary of QuerySets, which I could then use in my view, e.g.:
querysets = [cls.objects.filter(date=now) for cls in [Model1, Model2, Model3]]

Take a look at using Multiple Inheritance (docs here) to define those date fields in a class that you can subclass in the classes you want to return in the query.
For example:
class DateStuff(db.Model):
date = db.DateProperty()
class MyClass1(DateStuff):
...
class MyClass2(DateStuff):
...
I believe Django will let you query over the DateStuff class, and it'll return objects from MyClass1 and MyClass2.
Thank #nrabinowitz for pointing out my previous error.