Develop Reference

What is the best way to implement persistent data model in Django?

What I need is basically a database model with version control. So that every time a record is modified/deleted, the data isn't lost, and the change can be undone.
I've been trying to implement it myself with something like this:
from django.db import models
class AbstractPersistentModel(models.Model):
time_created = models.DateTimeField(auto_now_add=True)
time_changed = models.DateTimeField(null=True, default=None)
time_deleted = models.DateTimeField(null=True, default=None)
class Meta:
abstract = True
Then every model would inherit from AbstractPersistentModel.
Problem is, if I override save() and delete() to make sure they don't actually touch the existing data, I'll still be left with the original object, and not the new version.
After trying to come up with a clean, safe and easy-to-use solution for some hours, I gave up.
Is there some way to implement this functionality that isn't overwhelming?
It seems common enough problem that I thought it would be built into Django itself, or at least there'd be a well documented package for this, but I couldn't find any.

When I hear version control for models and Django, I immediately think of django-reversion.
Then, if you want to access the versions of an instance, and not the actual instance, simply use the Version model.
from reversion.models import Version
versions = Version.objects.get_for_object(instance)

I feel you can work around your issue not by modifying your models but by modifying the logic that access them.
So, you could have two models for your same object: one that can be your staging area, in which you store values as the ones you mention, such as time_created, time_modified, and modifying_user, or others. From there, in the code for your views you go through that table and select the records you want/need according to your design and store in your definitive table.

Django many-to-many generic relationship

I think I need to create a 'many-to-many generic relationship'.
I have two types of Participants:
class MemberParticipant(AbstractParticipant):
class Meta:
app_label = 'participants'
class FriendParticipant(AbstractParticipant):
"""
Abstract participant common information shared for all rewards.
"""
pass
These Participants can have 1 or more rewards of 2 different kinds (rewards model is from another app):
class SingleVoucherReward(AbstractReward):
"""
Single-use coupons are coupon codes that can only be used once
"""
pass
class MultiVoucherReward(AbstractReward):
"""
A multi-use coupon code is a coupon code that can be used unlimited times.
"""
So now I need to link these all up. This is how I was thinking of creating the relationship (see below) would this work, any issues you see?
Proposed linking model below:
class ParticipantReward(models.Model):
participant_content_type = models.ForeignKey(ContentType, editable=False,
related_name='%(app_label)s_%(class)s_as_participant',
)
participant_object_id = models.PositiveIntegerField()
participant = generic.GenericForeignKey('participant_content_type', 'participant_object_id')
reward_content_type = models.ForeignKey(ContentType, editable=False,
related_name='%(app_label)s_%(class)s_as_reward',
)
reward_object_id = models.PositiveIntegerField()
reward = generic.GenericForeignKey('reward_content_type', 'reward_object_id')
Note: I'm using Django 1.6

Your approach is exactly the right way to do it given your existing tables. While there's nothing official (this discussion, involving a core developer in 2007, appears not to have gone anywhere), I did find this blog post which takes the same approach (and offers it in a third-party library), and there's also a popular answer here which is similar, except only one side of the relationship is generic.
I'd say the reason this functionality has never made it into django's trunk is that while it's a rare requirement, it's fairly easy to implement using the existing tools. Also, the chance of wanting a custom "through" table is probably quite high so most end-user implementations are going to involve a bit of custom code anyway.
The only other potentially simpler approach would be to have base Participant and Reward models, with the ManyToMany relationship between those, and then use multi-table inheritance to extend these models as Member/Friend etc.
Ultimately, you'll just need to weigh up the complexity of a generic relation versus that of having your object's data spread across two models.

Late reply, but I found this conversation when looking for a way to implement generic m2m relations and felt my 2 cents would be helpful for future googlers.
As Greg says, the approach you chose is a good way to do it.
However, I would not qualify generic many to many as 'easy to implement using existing tools' when you want to use features such as reverse relations or prefetching.
The 3rd party app django-genericm2m is nice but has several shortcomings in my opinion (the fact that the 'through' objects are all in the same database table by default and that you don't have 'add' / 'remove' methods - and therefore bulk add/remove).
With that in view, because I needed something to implement generic many-to-many relations 'the django way' and also because I wanted to learn a little bit about django internals, I recently released django-gm2m. It has a very similar API to django's built-in GenericForeignKey and ManyToManyField (with prefetching, through models ...) and adds deletion behavior customisation. The only thing it lacks for the moment is a suitable django admin interface.

Two-way querying in Django without circular reference

Let's say you have two models: Post and Category. Each Post has a category_id.
Getting a post's category is straightforward: post.category. What if you want to get all the posts for a certain category? I suppose you would do
def posts(self):
return Post.filter(category__pk=self.id)
But what if the Post model and Category model are in separate files? Because Post and Category now require each other, you would end up with a circular reference.
Maybe you say the solution is to put Post and Category into the same file. But what if your app has 50 different models, many of them quite large, all in separate files? Should you combine Post and Category into one file and leave all the others separate? Should you combine all 50 models into one gigantic file?
I'm hoping to find one of two things:
An answer to this problem that doesn't involve combining files
A good, logical reason for grouping models into the same file with one another. All my models are related to some extent, so where do you draw the line as far as grouping goes? If you draw the line with foreign keys, all my models would end up in the same file.

For part 1. of your question:
Django automatically sets up the reverse relationship for you. on your Category you have access to a post_set attribute, which is itself a Manager, so you can do:
def posts(self):
return self.post_set.all()
Check out the docs for more on this.
For part 2. I have a less complete answer... If you are experiencing a need to break out models into so many files you probably have a more fundamental problem. I'd say you should think about breaking down that huge app into some smaller ones.
I think the best advice I've had in terms of reducing the size of my apps is to "do one thing". If your app can't be described in a relatively short phrase, it's worth considering how to break it out into a number of smaller apps - each of which will "do one thing".
That advice is, of course, rather loaded. It takes quite a bit of planning to figure out how to break down some complex tasks, and sometimes hindsight is the only way to see where things got out of hand. And refactoring something of the size of which you speak can be quite daunting. (If you can tell I'm speaking from experience!). My only advice in this case is to take it a step at a time. Try to break down what look like big, application-spanning issues into small, manageable chunks and do them when you can.
To address your immediate need, I'd suggest grouping your models into files topically or categorically. It might lead you in the direction of refactoring this one (what sounds like a mammoth) app into a number of smaller apps. I think that's the direction you should really be heading with this.

You can refer to models by their name (i.e., a string) rather than the actual object. Are they in separate applications? In which case you can refer to them using dot notation as described in the documentation on ForeignKey:
#in mysite/categories/models.py
class Category(models.Model):
...
#in mysite/posts/models.py
class Post(models.Model):
category = models.ForeignKey('categories.Category')
You can also import the model within a function, rather than at the top of the file. I often do that to avoid circular references:
def posts(self):
from posts.models import Post
return Post.filter(category__pk=self.id)

Reverse ForeignKey lookup

I'm new to Django and am still trying to break old PHP habits. Below are two models. To make things confusing they live in separate files, in different apps...
#article.models
from someapp.author.models import Author
class Article(model.Model):
...
author = models.ForeignKey(Author)
# author.models
class Author(model.Model):
...
From this schema I want to be able to get all the articles by an author. Something like:
author = Author(pk=1)
articles = author.articles
My first reaction was to write a method that did a simple look up in the article model based on the authors ID. What happened here was a never ending inclusion loop because of the separate files. Article needed Author imported to use for the ForeignKey and Author needed article included to use for the model look up. This felt hacky and wrong. I would much rather do it the right way... So, what is the Django way?

I think this is what you're asking for...
class Article(model.Model):
...
author = models.ForeignKey(Author, related_name='articles')
On a side note, by default without changing anything you've got, I think this would work for you...
article.author_set
But to maintain the article.authors syntax you mention above, you can specify that yourself with related_name.

models.py getting huge, what is the best way to break it up?

Directions from my supervisor:
"I want to avoid putting any logic in the models.py. From here on out, let's use that as only classes for accessing the database, and keep all logic in external classes that use the models classes, or wrap them."
I feel like this is the wrong way to go. I feel that keeping logic out of the models just to keep the file small is a bad idea. If the logic is best in the model, that's where it really should go regardless of file size.
So is there a simple way to just use includes? In PHP-speak, I'd like to propose to the supervisor that we just have models.py include() the model classes from other places. Conceptually, this would allow the models to have all the logic we want, yet keep file size down via increasing the number of files (which leads to less revision control problems like conflicts, etc.).
So, is there a simple way to remove model classes from the models.py file, but still have the models work with all of the Django tools? Or, is there a completely different yet elegant solution to the general problem of a "large" models.py file? Any input would be appreciated.

It's natural for model classes to contain methods to operate on the model. If I have a Book model, with a method book.get_noun_count(), that's where it belongs--I don't want to have to write "get_noun_count(book)", unless the method actually intrinsically belongs with some other package. (It might--for example, if I have a package for accessing Amazon's API with "get_amazon_product_id(book)".)
I cringed when Django's documentation suggested putting models in a single file, and I took a few minutes from the very beginning to figure out how to split it into a proper subpackage.
site/models/__init__.py
site/models/book.py
__init__.py looks like:
from .book import Book
so I can still write "from site.models import Book".
The following is only required for versions prior to Django 1.7, see
https://code.djangoproject.com/ticket/3591
The only trick is that you need to explicitly set each model's application, due to a bug in Django: it assumes that the application name is the third-to-last entry in the model path. "site.models.Book" results in "site", which is correct; "site.models.book.Book" makes it think the application name is "models". This is a pretty nasty hack on Django's part; it should probably search the list of installed applications for a prefix match.
class Book(models.Model):
class Meta: app_label = "site"
You could probably use a base class or metaclass to generalize this, but I haven't bothered with that yet.

Django is designed to let you build many small applications instead of one big application.
Inside every large application are many small applications struggling to be free.
If your models.py feels big, you're doing too much. Stop. Relax. Decompose.
Find smaller, potentially reusable small application components, or pieces. You don't have to actually reuse them. Just think about them as potentially reusable.
Consider your upgrade paths and decompose applications that you might want to replace some day. You don't have to actually replace them, but you can consider them as a stand-alone "module" of programming that might get replaced with something cooler in the future.
We have about a dozen applications, each model.py is no more than about 400 lines of code. They're all pretty focused on less than about half-dozen discrete class definitions. (These aren't hard limits, they're observations about our code.)
We decompose early and often.

I can't quite get which of many possible problems you might have. Here are some possibilities with answers:
multiple models in the same file
Put them into separate files. If there are dependencies, use import to pull in the
additional models.
extraneous logic / utility functions in models.py
Put the extra logic into separate files.
static methods for selecting some model instances from database
Create a new Manager in a separate file.
methods obviously related to the model
save, __unicode__ and get_absolute_url are examples.