Develop Reference

Possible source of cascade deletion on Django 1.9 project?

I'm working on a Django 1.9 project with a huge source and lots of is inherited from previous developers.
When I delete a User object from the shell there are many other related objects that get deleted as well. I get a print output with a Tuple containing:
(# of deleted objects, {u'CLASS_NAME': #number of deleted objects...})
I've searched all the code throughly and there are no Signals attached to User pre_ post_ or on_ delete, neither I can find where this print output is constructed... so, I have no idea where this is coming from.
Any ideas for other possible ways this can be happening?

A ForeignKey [Django-doc] has an on_delete=… parameter. Prior to django-2.0, you did not have to state this explicitly, in that case it used uses CASCADE, as is specified in the documentation:
on_delete will become a required argument in Django 2.0. In older versions it defaults to CASCADE.
This thus means that if you did not specify an on_delete=… parameter, or you specified this as CASCADE, then if the target record is removed, that model record is removed as well.
You can specify another handler for this. In fact you can even write your own handler, but unless you have to do something very sophisticated, the builtin handlers will likely be sufficient, you can choose between:
CASCADE
PROTECT
SET_NULL
SET_DEFAULT
SET(…)
DO_NOTHING
There are basically four categories: a cascade (CASCADE), prevent the removal (PROTECT), set it to another value (SET_NULL, SET_DEFAULT, SET(…), and DO_NOTHING.
DO_NOTHING is often not a good idea, since most databases enforce referential integrity and will thus refuse to update/delete, since then the foreign key column no longer points to a valid record. By SET_NULL, the field needs to be NULLable (so null=True), you can also set t to a different view.
You should thus pick the strategy that you think is the best, and then the ForeignKey looks like:
class MyModel(models.Model):
my_field = models.ForeignKey(OtherModel, null=True, on_delete=models.SET_NULL)

is it possible to override a foreign key relation with a custom method/property

Context
I'm working on refactoring a Django 2.X app, particularly the core model, CoreModel. There's a single database (Postgres) containing all related tables.
Instances of CoreModel will no longer live in Postgres after this refactor, they will live somewhere else but outside the scope of the Django project, let's say some AWS No-SQL database service.
There also several satellites models SateliteModel to CoreModel which will continue to live on Postgres, but CoreModelis currently modelled as a foreign key field.
class CordeModel(models.Model):
pass
class SatelliteModel(models.Model):
core = models.ForeignKey(CoreModel)
def some_instance_method(self):
return self.core.calculate_stuff() # <- override self.core!
Problem
The code is filled with mentions to the CoreModel relation, and I haven't been able to successfully solved this issue.
My first naive approach was to implement a #property getter method, that way I had enough flexibility to do something like:
#property
def core(self):
try:
# ORM
return self.core
except CoreNotFound:
# External datastore
return aws_client.fetch_core()
With this snippet I have a circular dependency on the core name, so the idea is out.
I could rename the foreign key: but I would much rather not touch the database schema. After all I'm already refactoring the central part of the app, and that's an very error-prone process. I'd do this if there's no other choice.
I could rename the #property field, to something like current_core: This way I avoid the infinite recursion part, but this in turn would imply a very big task of searching the whole code base for mentions of the relation, and this being the central model, it would take a lot of time.
After some hours of research I'm beginning to doubt if the concept of overriding a getter for a foreign key field is possible, as I need it. Maybe this is isn't exactly what I'm looking for, it's a very unusual use case, but the requirement is also very unusual.
Any insights you can give are greatly appreciated.
UPDATE
I've forgotten to add the most crucial piece of information.
Most CoreModel will be removed for Postgres (the historic ones), but there's a tiny part of CoreModels that will remain and will be moved after a while. In essence, only the "active" CoreModels will stay in Postgres, but all will eventually be moved out, while new CoreModel will be created.
So that rules out the possibility of change the ForeignKey field for an integer.

You could retain but rename the foreign key and then add a property with the old name
class SatelliteModel(models.Model):
old_core = models.ForeignKey(CoreModel, null=True, blank=True, on_delete=models.SET_NULL)
#property
def core(self):
try:
return self.old_core
except CoreModel.DoesNotExist:
return aws_client.fetch_core()
This would change the column name in your schema, although you could override the column name to prevent this
old_core = models.ForeignKey(CoreModel, db_column='core_id', null=True, blank=True, on_delete=models.SET_NULL)
It may be possible to create a subclass of ForeignKey that would perform as you wished, if this answer is not sufficient I can share some thoughts

What does adding on_delete to models.py do, and what should I put in it? [duplicate]

I'm quite familiar with Django, but I recently noticed there exists an on_delete=models.CASCADE option with the models. I have searched for the documentation for the same, but I couldn't find anything more than:
Changed in Django 1.9:
on_delete can now be used as the second positional argument (previously it was typically only passed as a keyword argument). It will be a required argument in Django 2.0.
An example case of usage is:
from django.db import models
class Car(models.Model):
manufacturer = models.ForeignKey(
'Manufacturer',
on_delete=models.CASCADE,
)
# ...
class Manufacturer(models.Model):
# ...
pass
What does on_delete do? (I guess the actions to be done if the model is deleted.)
What does models.CASCADE do? (any hints in documentation)
What other options are available (if my guess is correct)?
Where does the documentation for this reside?

This is the behaviour to adopt when the referenced object is deleted. It is not specific to Django; this is an SQL standard. Although Django has its own implementation on top of SQL. (1)
There are seven possible actions to take when such event occurs:
CASCADE: When the referenced object is deleted, also delete the objects that have references to it (when you remove a blog post for instance, you might want to delete comments as well). SQL equivalent: CASCADE.
PROTECT: Forbid the deletion of the referenced object. To delete it you will have to delete all objects that reference it manually. SQL equivalent: RESTRICT.
RESTRICT: (introduced in Django 3.1) Similar behavior as PROTECT that matches SQL's RESTRICT more accurately. (See django documentation example)
SET_NULL: Set the reference to NULL (requires the field to be nullable). For instance, when you delete a User, you might want to keep the comments he posted on blog posts, but say it was posted by an anonymous (or deleted) user. SQL equivalent: SET NULL.
SET_DEFAULT: Set the default value. SQL equivalent: SET DEFAULT.
SET(...): Set a given value. This one is not part of the SQL standard and is entirely handled by Django.
DO_NOTHING: Probably a very bad idea since this would create integrity issues in your database (referencing an object that actually doesn't exist). SQL equivalent: NO ACTION. (2)
Source: Django documentation
See also the documentation of PostgreSQL for instance.
In most cases, CASCADE is the expected behaviour, but for every ForeignKey, you should always ask yourself what is the expected behaviour in this situation. PROTECT and SET_NULL are often useful. Setting CASCADE where it should not, can potentially delete all of your database in cascade, by simply deleting a single user.
Additional note to clarify cascade direction
It's funny to notice that the direction of the CASCADE action is not clear to many people. Actually, it's funny to notice that only the CASCADE action is not clear. I understand the cascade behavior might be confusing, however you must think that it is the same direction as any other action. Thus, if you feel that CASCADE direction is not clear to you, it actually means that on_delete behavior is not clear to you.
In your database, a foreign key is basically represented by an integer field which value is the primary key of the foreign object. Let's say you have an entry comment_A, which has a foreign key to an entry article_B. If you delete the entry comment_A, everything is fine. article_B used to live without comment_A and don't bother if it's deleted. However, if you delete article_B, then comment_A panics! It never lived without article_B and needs it, it's part of its attributes (article=article_B, but what is article_B???). This is where on_delete steps in, to determine how to resolve this integrity error, either by saying:
"No! Please! Don't! I can't live without you!" (which is said PROTECT or RESTRICT in Django/SQL)
"All right, if I'm not yours, then I'm nobody's" (which is said SET_NULL)
"Good bye world, I can't live without article_B" and commit suicide (this is the CASCADE behavior).
"It's OK, I've got spare lover, I'll reference article_C from now" (SET_DEFAULT, or even SET(...)).
"I can't face reality, I'll keep calling your name even if that's the only thing left to me!" (DO_NOTHING)
I hope it makes cascade direction clearer. :)
Footnotes
(1) Django has its own implementation on top of SQL. And, as mentioned by #JoeMjr2 in the comments below, Django will not create the SQL constraints. If you want the constraints to be ensured by your database (for instance, if your database is used by another application, or if you hang in the database console from time to time), you might want to set the related constraints manually yourself. There is an open ticket to add support for database-level on delete constraints in Django.
(2) Actually, there is one case where DO_NOTHING can be useful: If you want to skip Django's implementation and implement the constraint yourself at the database-level.

The on_delete method is used to tell Django what to do with model instances that depend on the model instance you delete. (e.g. a ForeignKey relationship). The on_delete=models.CASCADE tells Django to cascade the deleting effect i.e. continue deleting the dependent models as well.
Here's a more concrete example. Assume you have an Author model that is a ForeignKey in a Book model. Now, if you delete an instance of the Author model, Django would not know what to do with instances of the Book model that depend on that instance of Author model. The on_delete method tells Django what to do in that case. Setting on_delete=models.CASCADE will instruct Django to cascade the deleting effect i.e. delete all the Book model instances that depend on the Author model instance you deleted.
Note: on_delete will become a required argument in Django 2.0. In older versions it defaults to CASCADE.
Here's the entire official documentation.

FYI, the on_delete parameter in models is backwards from what it sounds like. You put on_delete on a foreign key (FK) on a model to tell Django what to do if the FK entry that you are pointing to on your record is deleted. The options our shop have used the most are PROTECT, CASCADE, and SET_NULL. Here are the basic rules I have figured out:
Use PROTECT when your FK is pointing to a look-up table that really shouldn't be changing and that certainly should not cause your table to change. If anyone tries to delete an entry on that look-up table, PROTECT prevents them from deleting it if it is tied to any records. It also prevents Django from deleting your record just because it deleted an entry on a look-up table. This last part is critical. If someone were to delete the gender "Female" from my Gender table, I CERTAINLY would NOT want that to instantly delete any and all people I had in my Person table who had that gender.
Use CASCADE when your FK is pointing to a "parent" record. So, if a Person can have many PersonEthnicity entries (he/she can be American Indian, Black, and White), and that Person is deleted, I really would want any "child" PersonEthnicity entries to be deleted. They are irrelevant without the Person.
Use SET_NULL when you do want people to be allowed to delete an entry on a look-up table, but you still want to preserve your record. For example, if a Person can have a HighSchool, but it doesn't really matter to me if that high-school goes away on my look-up table, I would say on_delete=SET_NULL. This would leave my Person record out there; it just would just set the high-school FK on my Person to null. Obviously, you will have to allow null=True on that FK.
Here is an example of a model that does all three things:
class PurchPurchaseAccount(models.Model):
id = models.AutoField(primary_key=True)
purchase = models.ForeignKey(PurchPurchase, null=True, db_column='purchase', blank=True, on_delete=models.CASCADE) # If "parent" rec gone, delete "child" rec!!!
paid_from_acct = models.ForeignKey(PurchPaidFromAcct, null=True, db_column='paid_from_acct', blank=True, on_delete=models.PROTECT) # Disallow lookup deletion & do not delete this rec.
_updated = models.DateTimeField()
_updatedby = models.ForeignKey(Person, null=True, db_column='_updatedby', blank=True, related_name='acctupdated_by', on_delete=models.SET_NULL) # Person records shouldn't be deleted, but if they are, preserve this PurchPurchaseAccount entry, and just set this person to null.
def __unicode__(self):
return str(self.paid_from_acct.display)
class Meta:
db_table = u'purch_purchase_account'
As a last tidbit, did you know that if you don't specify on_delete (or didn't), the default behavior is CASCADE? This means that if someone deleted a gender entry on your Gender table, any Person records with that gender were also deleted!
I would say, "If in doubt, set on_delete=models.PROTECT." Then go test your application. You will quickly figure out which FKs should be labeled the other values without endangering any of your data.
Also, it is worth noting that on_delete=CASCADE is actually not added to any of your migrations, if that is the behavior you are selecting. I guess this is because it is the default, so putting on_delete=CASCADE is the same thing as putting nothing.

As mentioned earlier, CASCADE will delete the record that has a foreign key and references another object that was deleted. So for example if you have a real estate website and have a Property that references a City
class City(models.Model):
# define model fields for a city
class Property(models.Model):
city = models.ForeignKey(City, on_delete = models.CASCADE)
# define model fields for a property
and now when the City is deleted from the database, all associated Properties (eg. real estate located in that city) will also be deleted from the database
Now I also want to mention the merit of other options, such as SET_NULL or SET_DEFAULT or even DO_NOTHING. Basically, from the administration perspective, you want to "delete" those records. But you don't really want them to disappear. For many reasons. Someone might have deleted it accidentally, or for auditing and monitoring. And plain reporting. So it can be a way to "disconnect" the property from a City. Again, it will depend on how your application is written.
For example, some applications have a field "deleted" which is 0 or 1. And all their searches and list views etc, anything that can appear in reports or anywhere the user can access it from the front end, exclude anything that is deleted == 1. However, if you create a custom report or a custom query to pull down a list of records that were deleted and even more so to see when it was last modified (another field) and by whom (i.e. who deleted it and when)..that is very advantageous from the executive standpoint.
And don't forget that you can revert accidental deletions as simple as deleted = 0 for those records.
My point is, if there is a functionality, there is always a reason behind it. Not always a good reason. But a reason. And often a good one too.

Using CASCADE means actually telling Django to delete the referenced record.
In the poll app example below: When a 'Question' gets deleted it will also delete the Choices this Question has.
e.g Question: How did you hear about us?
(Choices: 1. Friends 2. TV Ad 3. Search Engine 4. Email Promotion)
When you delete this question, it will also delete all these four choices from the table.
Note that which direction it flows.
You don't have to put on_delete=models.CASCADE in Question Model put it in the Choice.
from django.db import models
class Question(models.Model):
question_text = models.CharField(max_length=200)
pub_date = models.dateTimeField('date_published')
class Choice(models.Model):
question = models.ForeignKey(Question, on_delete=models.CASCADE)
choice_text = models.CharField(max_legth=200)
votes = models.IntegerField(default=0)

simply put, on_delete is an instruction to specify what modifications will be made to the object in case the foreign object is deleted:
CASCADE: will remove the child object when the foreign object is deleted
SET_NULL: will set the child object foreign key to null
SET_DEFAULT: will set the child object to the default data given while creating the model
RESTRICT: raises a RestrictedError under certain conditions.
PROTECT: prevents the foreign object from being deleted so long there are child objects inheriting from it
additional links:
https://docs.djangoproject.com/en/4.0/ref/models/fields/#foreignkey

Here is answer for your question that says: why we use on_delete?
When an object referenced by a ForeignKey is deleted, Django by default emulates the behavior of the SQL constraint ON DELETE CASCADE and also deletes the object containing the ForeignKey. This behavior can be overridden by specifying the on_delete argument. For example, if you have a nullable ForeignKey and you want it to be set null when the referenced object is deleted:
user = models.ForeignKey(User, blank=True, null=True, on_delete=models.SET_NULL)
The possible values for on_delete are found in django.db.models:
CASCADE: Cascade deletes; the default.
PROTECT: Prevent deletion of the referenced object by raising ProtectedError, a subclass of django.db.IntegrityError.
SET_NULL: Set the ForeignKey null; this is only possible if null is True.
SET_DEFAULT: Set the ForeignKey to its default value; a default for the ForeignKey must be set.

Let's say you have two models, one named Person and another one named Companies, and that, by definition, one person can create more than one company.
Considering a company can have one and only one person, we want that when a person is deleted that all the companies associated with that person also be deleted.
So, we start by creating a Person model, like this
class Person(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=20)
def __str__(self):
return self.id+self.name
Then, the Companies model can look like this
class Companies(models.Model):
title = models.CharField(max_length=20)
description=models.CharField(max_length=10)
person= models.ForeignKey(Person,related_name='persons',on_delete=models.CASCADE)
Notice the usage of on_delete=models.CASCADE in the model Companies. That is to delete all companies when the person that owns it (instance of class Person) is deleted.

Reorient your mental model of the functionality of "CASCADE" by thinking of adding a FK to an already existing cascade (i.e. a waterfall). The source of this waterfall is a primary key (PK). Deletes flow down.
So if you define a FK's on_delete as "CASCADE," you're adding this FK's record to a cascade of deletes originating from the PK. The FK's record may participate in this cascade or not ("SET_NULL"). In fact, a record with a FK may even prevent the flow of the deletes! Build a dam with "PROTECT."

Deletes all child fields in the database when parent object is deleted then we use on_delete as so:
class user(models.Model):
commodities = models.ForeignKey(commodity, on_delete=models.CASCADE)

CASCADE will also delete the corresponding field connected with it.

More than one profile in Django?

Is it possible to use Django's user authentication features with more than one profile?
Currently I have a settings.py file that has this in it:
AUTH_PROFILE_MODULE = 'auth.UserProfileA'
and a models.py file that has this in it:
from django.db import models
from django.contrib.auth.models import User
class UserProfileA(models.Model):
company = models.CharField(max_length=30)
user = models.ForeignKey(User, unique=True)
that way, if a user logs in, I can easily get the profile because the User has a get_profile() method. However, I would like to add UserProfileB. From looking around a bit, it seems that the starting point is to create a superclass to use as the AUTH_PROFILE_MODULE and have both UserProfileA and UserProfileB inherit from that superclass. The problem is, I don't think the get_profile() method returns the correct profile. It would return an instance of the superclass. I come from a java background (polymorphism) so I'm not sure exactly what I should be doing.
Thanks!
Edit:
Well I found a way to do it via something called an "inheritance hack" that I found at this site http://djangosnippets.org/snippets/1031/
It works really well, however, coming from a java background where this stuff happens automatically, I'm a little unsettled by the fact that someone had to code this up and call it a "hack" to do it in python. Is there a reason why python doesn't enable this?

So the issue you're going to have is that whatever you want for your profile, you need to persist it in a database of some sort. Basically all of the back-ends for django are relational, and thus every field in a persisted object is present in every row of the table. there are a few ways for getting what you want.
Django provides some support for inheritance. You can use the techniques listed and get reasonable results in a polymorphic way.
The most direct approach is to use multiple table inheritance. Roughly:
class UserProfile(models.Model):
# set settings.AUTH_PROFILE_MODULE to this class!
pass
class UserProfileA(UserProfile):
pass
class UserProfileB(UserProfile):
pass
To use it:
try:
profile = user.get_profile().userprofilea
# user profile is UserProfileA
except UserProfileA.DoesNotExist:
# user profile wasn't UserProfileB
pass
try:
profile = user.get_profile().userprofileb
# user profile is UserProfileB
except UserProfileB.DoesNotExist:
# user profile wasn't either a or b...
Edit: Re, your comment.
The relational model implies a number of things that seem to disagree with object oriented philosophy. For a relation to be useful, it requires that every element in the relation to have the same dimensions, so that relational queries are valid for the whole relation. Since this is known a-priori, before encountering an instance of a class stored in the relation, then the row cannot be a subclass. django's orm overcomes this impedance mismatch by storing the subclass information in a different relation (one specific to the subclass), There are other solutions, but they all obey this basic nature of the relational model.
If it helps you come to terms with this, I'd suggest looking at how persistence on a RDBMs works for applications in the absence of an ORM. In particular, relational databases are more about collections and summaries of many rows, rather than applying behaviors to data once fetched from the database.
The specific example of using the profile feature of django.contrib.auth is a rather uninteresting one, especially if the only way that model is ever used is to fetch the profile data associated with a particular django.contrib.auth.models.User instance. If there are no other queries, you don't need a django.models.Model subclass at all. You can pickle a regular python class and store it in a blob field of an otherwise featureless model.
On the other hand, if you want to do more interesting things with profiles, like search for users that live in a particular city, then it will be important for all profiles to have an index for their city property. That's got nothing to do with OOP, and everything to do with relational.

The idios app by the Pinax team aimed at solving the multiple-profile problem. You can tweak the model to make the inheritance of the base profile class either abstract or non-abstract.
https://github.com/eldarion/idios.

Here is the answer to my question of how to get multiple profiles to work:
from django.contrib.contenttypes.models import ContentType
class Contact(models.Model):
content_type = models.ForeignKey(ContentType,editable=False,null=True)
def save(self):
if(not self.content_type):
self.content_type = ContentType.objects.get_for_model(self.__class__)
self.save_base()
def as_leaf_class(self):
content_type = self.content_type
model = content_type.model_class()
if(model == Contact):
return self
return model.objects.get(id=self.id)
I don't really understand why it works or why the developers of django/python made inheritance work this way

If you have app-specific options for each user, I would rather recommend to put them into a separate model.
A simplified example:
class UserSettings(models.Model):
user = models.ForeignKey(User, primary_key = True)
# Settings go here
defaultLocale = models.CharField(max_length = 80, default = "en_US")
...
This would be used like so:
def getUserSettings(request):
try:
return UserSettings.objects.get(pk = request.user)
except UserSettings.DoesNotExist:
# Use defaults instead, that's why you should define reasonable defaults
# in the UserSettings model
return UserSettings()

Django signals vs. overriding save method

I'm having trouble wrapping my head around this. Right now I have some models that looks kind of like this:
def Review(models.Model)
...fields...
overall_score = models.FloatField(blank=True)
def Score(models.Model)
review = models.ForeignKey(Review)
question = models.TextField()
grade = models.IntegerField()
A Review is has several "scores", the overall_score is the average of the scores. When a review or a score is saved, I need to recalculate the overall_score average. Right now I'm using a overridden save method. Would there be any benefits to using Django's signal dispatcher?

Save/delete signals are generally favourable in situations where you need to make changes which aren't completely specific to the model in question, or could be applied to models which have something in common, or could be configured for use across models.
One common task in overridden save methods is automated generation of slugs from some text field in a model. That's an example of something which, if you needed to implement it for a number of models, would benefit from using a pre_save signal, where the signal handler could take the name of the slug field and the name of the field to generate the slug from. Once you have something like that in place, any enhanced functionality you put in place will also apply to all models - e.g. looking up the slug you're about to add for the type of model in question, to ensure uniqueness.
Reusable applications often benefit from the use of signals - if the functionality they provide can be applied to any model, they generally (unless it's unavoidable) won't want users to have to directly modify their models in order to benefit from it.
With django-mptt, for example, I used the pre_save signal to manage a set of fields which describe a tree structure for the model which is about to be created or updated and the pre_delete signal to remove tree structure details for the object being deleted and its entire sub-tree of objects before it and they are deleted. Due to the use of signals, users don't have to add or modify save or delete methods on their models to have this management done for them, they just have to let django-mptt know which models they want it to manage.

You asked:
Would there be any benefits to using Django's signal dispatcher?
I found this in the django docs:
Overridden model methods are not called on bulk operations
Note that the delete() method for an object is not necessarily called
when deleting objects in bulk using a QuerySet or as a result of a
cascading delete. To ensure customized delete logic gets executed, you
can use pre_delete and/or post_delete signals.
Unfortunately, there isn’t a workaround when creating or updating
objects in bulk, since none of save(), pre_save, and post_save are
called.
From: Overriding predefined model methods

Small addition from Django docs about bulk delete (.delete() method on QuerySet objects):
Keep in mind that this will, whenever possible, be executed purely in
SQL, and so the delete() methods of individual object instances will
not necessarily be called during the process. If you’ve provided a
custom delete() method on a model class and want to ensure that it is
called, you will need to “manually” delete instances of that model
(e.g., by iterating over a QuerySet and calling delete() on each
object individually) rather than using the bulk delete() method of a
QuerySet.
https://docs.djangoproject.com/en/1.11/topics/db/queries/#deleting-objects
And bulk update (.update() method on QuerySet objects):
Finally, realize that update() does an update at the SQL level and,
thus, does not call any save() methods on your models, nor does it
emit the pre_save or post_save signals (which are a consequence of
calling Model.save()). If you want to update a bunch of records for a
model that has a custom save() method, loop over them and call save()
https://docs.djangoproject.com/en/2.1/ref/models/querysets/#update

If you'll use signals you'd be able to update Review score each time related score model gets saved. But if don't need such functionality i don't see any reason to put this into signal, that's pretty model-related stuff.

It is a kind sort of denormalisation. Look at this pretty solution. In-place composition field definition.