What I need is basically a database model with version control. So that every time a record is modified/deleted, the data isn't lost, and the change can be undone.
I've been trying to implement it myself with something like this:
from django.db import models
class AbstractPersistentModel(models.Model):
time_created = models.DateTimeField(auto_now_add=True)
time_changed = models.DateTimeField(null=True, default=None)
time_deleted = models.DateTimeField(null=True, default=None)
class Meta:
abstract = True
Then every model would inherit from AbstractPersistentModel.
Problem is, if I override save() and delete() to make sure they don't actually touch the existing data, I'll still be left with the original object, and not the new version.
After trying to come up with a clean, safe and easy-to-use solution for some hours, I gave up.
Is there some way to implement this functionality that isn't overwhelming?
It seems common enough problem that I thought it would be built into Django itself, or at least there'd be a well documented package for this, but I couldn't find any.
When I hear version control for models and Django, I immediately think of django-reversion.
Then, if you want to access the versions of an instance, and not the actual instance, simply use the Version model.
from reversion.models import Version
versions = Version.objects.get_for_object(instance)
I feel you can work around your issue not by modifying your models but by modifying the logic that access them.
So, you could have two models for your same object: one that can be your staging area, in which you store values as the ones you mention, such as time_created, time_modified, and modifying_user, or others. From there, in the code for your views you go through that table and select the records you want/need according to your design and store in your definitive table.
Related
Context
I'm working on refactoring a Django 2.X app, particularly the core model, CoreModel. There's a single database (Postgres) containing all related tables.
Instances of CoreModel will no longer live in Postgres after this refactor, they will live somewhere else but outside the scope of the Django project, let's say some AWS No-SQL database service.
There also several satellites models SateliteModel to CoreModel which will continue to live on Postgres, but CoreModelis currently modelled as a foreign key field.
class CordeModel(models.Model):
pass
class SatelliteModel(models.Model):
core = models.ForeignKey(CoreModel)
def some_instance_method(self):
return self.core.calculate_stuff() # <- override self.core!
Problem
The code is filled with mentions to the CoreModel relation, and I haven't been able to successfully solved this issue.
My first naive approach was to implement a #property getter method, that way I had enough flexibility to do something like:
#property
def core(self):
try:
# ORM
return self.core
except CoreNotFound:
# External datastore
return aws_client.fetch_core()
With this snippet I have a circular dependency on the core name, so the idea is out.
I could rename the foreign key: but I would much rather not touch the database schema. After all I'm already refactoring the central part of the app, and that's an very error-prone process. I'd do this if there's no other choice.
I could rename the #property field, to something like current_core: This way I avoid the infinite recursion part, but this in turn would imply a very big task of searching the whole code base for mentions of the relation, and this being the central model, it would take a lot of time.
After some hours of research I'm beginning to doubt if the concept of overriding a getter for a foreign key field is possible, as I need it. Maybe this is isn't exactly what I'm looking for, it's a very unusual use case, but the requirement is also very unusual.
Any insights you can give are greatly appreciated.
UPDATE
I've forgotten to add the most crucial piece of information.
Most CoreModel will be removed for Postgres (the historic ones), but there's a tiny part of CoreModels that will remain and will be moved after a while. In essence, only the "active" CoreModels will stay in Postgres, but all will eventually be moved out, while new CoreModel will be created.
So that rules out the possibility of change the ForeignKey field for an integer.
You could retain but rename the foreign key and then add a property with the old name
class SatelliteModel(models.Model):
old_core = models.ForeignKey(CoreModel, null=True, blank=True, on_delete=models.SET_NULL)
#property
def core(self):
try:
return self.old_core
except CoreModel.DoesNotExist:
return aws_client.fetch_core()
This would change the column name in your schema, although you could override the column name to prevent this
old_core = models.ForeignKey(CoreModel, db_column='core_id', null=True, blank=True, on_delete=models.SET_NULL)
It may be possible to create a subclass of ForeignKey that would perform as you wished, if this answer is not sufficient I can share some thoughts
Suppose, I want to build a simple TODO-app. I want to make it possible to create todo-items, and also it should be possible to rearrange items manually.
I made following model:
class Item(models.Model):
title = models.CharField(max_length=500)
Now, I need to add a special field, let's call it order, to keep custom ordering. It should be unique, and it should be greater for any new record, so I tried to make an AutoField.
class Item(models.Model):
title = models.CharField(max_length=500)
order = models.AutoField(primary_key=False)
But it turned out that Django doesn't support several auto fields for a single model.
I guess, it should be possible to write custom raw SQL code and use Postgres sequences directly, but it would look really ugly, and I don't want to write DB-specific code for such simple functionality.
So, here is a question: what is a common way to implement items ordering in Django? It sounds like a very common requirement, and, I think, it should be a simple way to do this.
As it turns out there is no straightforward way to implement this in Django. There are packages which help you, like this one
But I would recommend just look at their model implementation and fit your needs. models.py
You could use Item.objects.count() to automatically increment your field. Plug it in the save() method of your model so that your field is calculated each time you create an instance.
I'm building my first Django app to manage multiple SaaS products.
This entails storing custom attributes for each Version of each Product.
For example, a new version of a Product is released that includes new configuration options that the earlier versions of the Product do not support.
I need to be able to keep track of those new values for each instance of the new Version.
I'm thinking I want the Admins to be able to add "custom fields" at the Product level by Version.
Looking for suggestions as to the best approach.
Thanks.
The common way of tracking model versions is to use django-reversion.
It sounds like each instance needs its own custom attributes. That means that changing the Models relating to Product and Version need not occur. This is good, because models can only change with the code (unless you get into dynamically generating Models which is usually not a good idea).
So, you need to be able to model attributes for each Product instance, regardless of Version. This should be a simple data modelling exercise, not necessarily related to Django.
A Product has a set of fields
A Product has a Version
A Product has a set of Attributes
This is quite easily modelled, depending on how you want to manage attributes.
class Version(models.Model):
version = models.CharField(max_length=10)
class ProductAttributes(models.Model):
name = models.CharField(max_length=64)
description = models.CharField(max_length=255)
# other fields as necessary
class Product(models.Model):
name = models.CharField(max_length=64)
version = models.ForeignKey(Version)
attributes = models.ManyToManyField(ProductAttributes, related_name='products')
That should be your modelling sorted in a very basic way. Now, let's create some instances.
v1 = Version(version='1.0.0')
v1.save()
hosted = ProductAttributes(name='Hosted', description='We host the apps!')
hosted.save()
shiny = ProductAttributes(name='Shiny', description='I like shiny')
shiny.save()
p = Product(name='Web Based Email', version=v1)
p.save()
p.attributes.add(hosted)
p.attributes.add(shiny)
p.attributes.all()
# shows shiny and hosted!
You can tweak the ModelAdmin for Product such that you can add ProductAttributes inline when adding or editing a Product. You can also have a separate ModelAdmin for ProductAttributes so you can create a list of known Attributes that can be applied to products at a later date.
There are two basic approaches for this.
Use a document based db (ie, "NoSQL") like Couch or Mongo. These have flexible schemas, so allow for multiple variations on a product.
Use the Entity Attribute Value (wikipedia) schema pattern. django-eav is an app that provides this.
Decide to go with sub-classes with each Product as each has a limited set of specific attributes that won't change much or at all over time. Thanks for all the great feedback. Learned a lot :-)
Half way through my current project, after suffering the pain of spending uncountable minutes on debugging, I have decided to adopt TDD. To start, I am planning to write a set of unit tests for each existing models. But for models that only have attributes defined (ie. no additional methods/properties) I am not sure what I need to test nor how.
class Product(models.Model):
name = models.CharField(max_length=50)
description = models.TextField(default='', blank=True)
retails = models.ManyToManyField(Retail, verbose_name='Retail stores that carry the product')
manufacturer = models.ForeignKey(Manufacturer, related_name='products')
date_created = models.DateTimeField(auto_now_add=True)
date_modified = models.DateTimeField(auto_now=True)
Using Product as an example, what are the things about it that unit tests should cover? And how should ForeignKey and ManyToManyField be covered?
This was an article I found helpful: A Guide to Testing in Django (archived link). Here is a good summary of what to test:
Another common setback for developers/designers new to testing is the
question of 'what should (or shouldn't) I test?' While there are no
hard & fast rules here that neatly apply everywhere, there are some
general guidelines I can offer on making the decision:
If the code in question is a built-in Python function/library, don't test it. Examples like the datetime library.
If the code in question is built into Django, don't test it. Examples like the fields on a Model or testing how the built-in
template.Node renders included tags.
If your model has custom methods, you should test that, usually with unit tests.
Same goes for custom views, forms, template tags, context processors, middleware, management commands, etc. If you implemented
the business logic, you should test your aspects of the code.
So, for your example, there wouldn't really be anything to test until you write some custom functions.
In my opinion, testing ForeignKey and ManyToManyField links would fall under the second category (code built into Django), so I wouldn't test these, as you are really testing whether or not Django is functioning properly. If you have a method which creates an instance of your product, including foreign relationships and M2Ms, you could verify the data has been created, that would be testing your custom method, not Django functionality.
Using the TDD paradigm, the tests are built to verify business logic, and design requirements.
My CS350 class TDD stipulated that it's best practice to test all accessors and mutators. So for a model, you would first write tests that call each assessor function and make sure that it returns the proper value.
For each function which changes a data field in the model, you would not only test the result of that data field in particular, but you would also test all of the other fields in the model instance to make sure that none of them were modified erroneously.
To restat:, if a model has fields a, b, and c, you would create an instance using your constructor, then asset that all three are set properly. Say there's another function, set_a(). You would assert that not only the value of 'a' has changed, but that the values of b and c remain unchanged.
Is it possible to use Django's user authentication features with more than one profile?
Currently I have a settings.py file that has this in it:
AUTH_PROFILE_MODULE = 'auth.UserProfileA'
and a models.py file that has this in it:
from django.db import models
from django.contrib.auth.models import User
class UserProfileA(models.Model):
company = models.CharField(max_length=30)
user = models.ForeignKey(User, unique=True)
that way, if a user logs in, I can easily get the profile because the User has a get_profile() method. However, I would like to add UserProfileB. From looking around a bit, it seems that the starting point is to create a superclass to use as the AUTH_PROFILE_MODULE and have both UserProfileA and UserProfileB inherit from that superclass. The problem is, I don't think the get_profile() method returns the correct profile. It would return an instance of the superclass. I come from a java background (polymorphism) so I'm not sure exactly what I should be doing.
Thanks!
Edit:
Well I found a way to do it via something called an "inheritance hack" that I found at this site http://djangosnippets.org/snippets/1031/
It works really well, however, coming from a java background where this stuff happens automatically, I'm a little unsettled by the fact that someone had to code this up and call it a "hack" to do it in python. Is there a reason why python doesn't enable this?
So the issue you're going to have is that whatever you want for your profile, you need to persist it in a database of some sort. Basically all of the back-ends for django are relational, and thus every field in a persisted object is present in every row of the table. there are a few ways for getting what you want.
Django provides some support for inheritance. You can use the techniques listed and get reasonable results in a polymorphic way.
The most direct approach is to use multiple table inheritance. Roughly:
class UserProfile(models.Model):
# set settings.AUTH_PROFILE_MODULE to this class!
pass
class UserProfileA(UserProfile):
pass
class UserProfileB(UserProfile):
pass
To use it:
try:
profile = user.get_profile().userprofilea
# user profile is UserProfileA
except UserProfileA.DoesNotExist:
# user profile wasn't UserProfileB
pass
try:
profile = user.get_profile().userprofileb
# user profile is UserProfileB
except UserProfileB.DoesNotExist:
# user profile wasn't either a or b...
Edit: Re, your comment.
The relational model implies a number of things that seem to disagree with object oriented philosophy. For a relation to be useful, it requires that every element in the relation to have the same dimensions, so that relational queries are valid for the whole relation. Since this is known a-priori, before encountering an instance of a class stored in the relation, then the row cannot be a subclass. django's orm overcomes this impedance mismatch by storing the subclass information in a different relation (one specific to the subclass), There are other solutions, but they all obey this basic nature of the relational model.
If it helps you come to terms with this, I'd suggest looking at how persistence on a RDBMs works for applications in the absence of an ORM. In particular, relational databases are more about collections and summaries of many rows, rather than applying behaviors to data once fetched from the database.
The specific example of using the profile feature of django.contrib.auth is a rather uninteresting one, especially if the only way that model is ever used is to fetch the profile data associated with a particular django.contrib.auth.models.User instance. If there are no other queries, you don't need a django.models.Model subclass at all. You can pickle a regular python class and store it in a blob field of an otherwise featureless model.
On the other hand, if you want to do more interesting things with profiles, like search for users that live in a particular city, then it will be important for all profiles to have an index for their city property. That's got nothing to do with OOP, and everything to do with relational.
The idios app by the Pinax team aimed at solving the multiple-profile problem. You can tweak the model to make the inheritance of the base profile class either abstract or non-abstract.
https://github.com/eldarion/idios.
Here is the answer to my question of how to get multiple profiles to work:
from django.contrib.contenttypes.models import ContentType
class Contact(models.Model):
content_type = models.ForeignKey(ContentType,editable=False,null=True)
def save(self):
if(not self.content_type):
self.content_type = ContentType.objects.get_for_model(self.__class__)
self.save_base()
def as_leaf_class(self):
content_type = self.content_type
model = content_type.model_class()
if(model == Contact):
return self
return model.objects.get(id=self.id)
I don't really understand why it works or why the developers of django/python made inheritance work this way
If you have app-specific options for each user, I would rather recommend to put them into a separate model.
A simplified example:
class UserSettings(models.Model):
user = models.ForeignKey(User, primary_key = True)
# Settings go here
defaultLocale = models.CharField(max_length = 80, default = "en_US")
...
This would be used like so:
def getUserSettings(request):
try:
return UserSettings.objects.get(pk = request.user)
except UserSettings.DoesNotExist:
# Use defaults instead, that's why you should define reasonable defaults
# in the UserSettings model
return UserSettings()