I've been reading through the Django documentation and looking over some of the other answers on the site for a couple of hours now, yet I still can't get it to sink in. I know this isn't Django specific, but the examples I use will be from a Django project.
My question boils down to when is it appropriate to use each:
Many-to-many relationships
Many-to-one relationships
One-to-one relationships
One-to-one, more or less makes sense to me.
Now for the other two. While I understand the differences between them in isolation, when it comes to using them practically in a project, I get confused. Here is an example:
class User(AbstractUser):
pass
class Listing(models.Model):
title = models.CharField(max_length=64)
description = models.TextField()
class Watchlist(models.Model):
user = models.ForeignKey(User, related_name='watchlist', on_delete=models.CASCADE)
item = models.ManyToManyField(Listing)
class Comment(models.Model):
user = models.ForeignKey(User, related_name='comments', on_delete=models.SET_NULL)
comment = models.TextField()
Would this be the correct use of Many-to-one(ForeignKey) and Many-to-many?
Should Watchlist.item be a ForeignKey? Or is M2M correct?
Wouldn't it simplify to make the 'Watchlist' part of the User class? (give them an empty list to populate with listing ID's)
Why is Watchlist.user not a One-to-one relationship, if each watchlist belongs to a single user, and a user can only have one list?
Apologies for my stupidity, I just can't get this to sink in!
Thank you.
edit: Context, the models are from a 'learning' project I was working on intended to be an auction site, similar to eBay. The watchlist is sort of a 'wish' list... for the user to watch an item, not for site to watch a user!
To explain it simply these django-models or objects represents tables in your database and the fields are like the columns in them. So with a one-to-one relation you can only have one row in one table relating to one row in another table. For example one user in the user table (represented by one row) can only relate to one row in a profile table. But your user can have many comments, so this would be a one-to-many/foreignkey relation (if you set unique=true on a fk, it will in practice function as 1:1). If the users can collaborate on writing comments, or for example as here on stackoverflow where users can edit other users comments, that would be a many-to-many relation.
Database design can be complicated/complex, especially using an ORM without basic knowledge of SQL and how it all works beneath. In general it requires a bit of planning even for a simple application.
I'm quite familiar with Django, but I recently noticed there exists an on_delete=models.CASCADE option with the models. I have searched for the documentation for the same, but I couldn't find anything more than:
Changed in Django 1.9:
on_delete can now be used as the second positional argument (previously it was typically only passed as a keyword argument). It will be a required argument in Django 2.0.
An example case of usage is:
from django.db import models
class Car(models.Model):
manufacturer = models.ForeignKey(
'Manufacturer',
on_delete=models.CASCADE,
)
# ...
class Manufacturer(models.Model):
# ...
pass
What does on_delete do? (I guess the actions to be done if the model is deleted.)
What does models.CASCADE do? (any hints in documentation)
What other options are available (if my guess is correct)?
Where does the documentation for this reside?
This is the behaviour to adopt when the referenced object is deleted. It is not specific to Django; this is an SQL standard. Although Django has its own implementation on top of SQL. (1)
There are seven possible actions to take when such event occurs:
CASCADE: When the referenced object is deleted, also delete the objects that have references to it (when you remove a blog post for instance, you might want to delete comments as well). SQL equivalent: CASCADE.
PROTECT: Forbid the deletion of the referenced object. To delete it you will have to delete all objects that reference it manually. SQL equivalent: RESTRICT.
RESTRICT: (introduced in Django 3.1) Similar behavior as PROTECT that matches SQL's RESTRICT more accurately. (See django documentation example)
SET_NULL: Set the reference to NULL (requires the field to be nullable). For instance, when you delete a User, you might want to keep the comments he posted on blog posts, but say it was posted by an anonymous (or deleted) user. SQL equivalent: SET NULL.
SET_DEFAULT: Set the default value. SQL equivalent: SET DEFAULT.
SET(...): Set a given value. This one is not part of the SQL standard and is entirely handled by Django.
DO_NOTHING: Probably a very bad idea since this would create integrity issues in your database (referencing an object that actually doesn't exist). SQL equivalent: NO ACTION. (2)
Source: Django documentation
See also the documentation of PostgreSQL for instance.
In most cases, CASCADE is the expected behaviour, but for every ForeignKey, you should always ask yourself what is the expected behaviour in this situation. PROTECT and SET_NULL are often useful. Setting CASCADE where it should not, can potentially delete all of your database in cascade, by simply deleting a single user.
Additional note to clarify cascade direction
It's funny to notice that the direction of the CASCADE action is not clear to many people. Actually, it's funny to notice that only the CASCADE action is not clear. I understand the cascade behavior might be confusing, however you must think that it is the same direction as any other action. Thus, if you feel that CASCADE direction is not clear to you, it actually means that on_delete behavior is not clear to you.
In your database, a foreign key is basically represented by an integer field which value is the primary key of the foreign object. Let's say you have an entry comment_A, which has a foreign key to an entry article_B. If you delete the entry comment_A, everything is fine. article_B used to live without comment_A and don't bother if it's deleted. However, if you delete article_B, then comment_A panics! It never lived without article_B and needs it, it's part of its attributes (article=article_B, but what is article_B???). This is where on_delete steps in, to determine how to resolve this integrity error, either by saying:
"No! Please! Don't! I can't live without you!" (which is said PROTECT or RESTRICT in Django/SQL)
"All right, if I'm not yours, then I'm nobody's" (which is said SET_NULL)
"Good bye world, I can't live without article_B" and commit suicide (this is the CASCADE behavior).
"It's OK, I've got spare lover, I'll reference article_C from now" (SET_DEFAULT, or even SET(...)).
"I can't face reality, I'll keep calling your name even if that's the only thing left to me!" (DO_NOTHING)
I hope it makes cascade direction clearer. :)
Footnotes
(1) Django has its own implementation on top of SQL. And, as mentioned by #JoeMjr2 in the comments below, Django will not create the SQL constraints. If you want the constraints to be ensured by your database (for instance, if your database is used by another application, or if you hang in the database console from time to time), you might want to set the related constraints manually yourself. There is an open ticket to add support for database-level on delete constraints in Django.
(2) Actually, there is one case where DO_NOTHING can be useful: If you want to skip Django's implementation and implement the constraint yourself at the database-level.
The on_delete method is used to tell Django what to do with model instances that depend on the model instance you delete. (e.g. a ForeignKey relationship). The on_delete=models.CASCADE tells Django to cascade the deleting effect i.e. continue deleting the dependent models as well.
Here's a more concrete example. Assume you have an Author model that is a ForeignKey in a Book model. Now, if you delete an instance of the Author model, Django would not know what to do with instances of the Book model that depend on that instance of Author model. The on_delete method tells Django what to do in that case. Setting on_delete=models.CASCADE will instruct Django to cascade the deleting effect i.e. delete all the Book model instances that depend on the Author model instance you deleted.
Note: on_delete will become a required argument in Django 2.0. In older versions it defaults to CASCADE.
Here's the entire official documentation.
FYI, the on_delete parameter in models is backwards from what it sounds like. You put on_delete on a foreign key (FK) on a model to tell Django what to do if the FK entry that you are pointing to on your record is deleted. The options our shop have used the most are PROTECT, CASCADE, and SET_NULL. Here are the basic rules I have figured out:
Use PROTECT when your FK is pointing to a look-up table that really shouldn't be changing and that certainly should not cause your table to change. If anyone tries to delete an entry on that look-up table, PROTECT prevents them from deleting it if it is tied to any records. It also prevents Django from deleting your record just because it deleted an entry on a look-up table. This last part is critical. If someone were to delete the gender "Female" from my Gender table, I CERTAINLY would NOT want that to instantly delete any and all people I had in my Person table who had that gender.
Use CASCADE when your FK is pointing to a "parent" record. So, if a Person can have many PersonEthnicity entries (he/she can be American Indian, Black, and White), and that Person is deleted, I really would want any "child" PersonEthnicity entries to be deleted. They are irrelevant without the Person.
Use SET_NULL when you do want people to be allowed to delete an entry on a look-up table, but you still want to preserve your record. For example, if a Person can have a HighSchool, but it doesn't really matter to me if that high-school goes away on my look-up table, I would say on_delete=SET_NULL. This would leave my Person record out there; it just would just set the high-school FK on my Person to null. Obviously, you will have to allow null=True on that FK.
Here is an example of a model that does all three things:
class PurchPurchaseAccount(models.Model):
id = models.AutoField(primary_key=True)
purchase = models.ForeignKey(PurchPurchase, null=True, db_column='purchase', blank=True, on_delete=models.CASCADE) # If "parent" rec gone, delete "child" rec!!!
paid_from_acct = models.ForeignKey(PurchPaidFromAcct, null=True, db_column='paid_from_acct', blank=True, on_delete=models.PROTECT) # Disallow lookup deletion & do not delete this rec.
_updated = models.DateTimeField()
_updatedby = models.ForeignKey(Person, null=True, db_column='_updatedby', blank=True, related_name='acctupdated_by', on_delete=models.SET_NULL) # Person records shouldn't be deleted, but if they are, preserve this PurchPurchaseAccount entry, and just set this person to null.
def __unicode__(self):
return str(self.paid_from_acct.display)
class Meta:
db_table = u'purch_purchase_account'
As a last tidbit, did you know that if you don't specify on_delete (or didn't), the default behavior is CASCADE? This means that if someone deleted a gender entry on your Gender table, any Person records with that gender were also deleted!
I would say, "If in doubt, set on_delete=models.PROTECT." Then go test your application. You will quickly figure out which FKs should be labeled the other values without endangering any of your data.
Also, it is worth noting that on_delete=CASCADE is actually not added to any of your migrations, if that is the behavior you are selecting. I guess this is because it is the default, so putting on_delete=CASCADE is the same thing as putting nothing.
As mentioned earlier, CASCADE will delete the record that has a foreign key and references another object that was deleted. So for example if you have a real estate website and have a Property that references a City
class City(models.Model):
# define model fields for a city
class Property(models.Model):
city = models.ForeignKey(City, on_delete = models.CASCADE)
# define model fields for a property
and now when the City is deleted from the database, all associated Properties (eg. real estate located in that city) will also be deleted from the database
Now I also want to mention the merit of other options, such as SET_NULL or SET_DEFAULT or even DO_NOTHING. Basically, from the administration perspective, you want to "delete" those records. But you don't really want them to disappear. For many reasons. Someone might have deleted it accidentally, or for auditing and monitoring. And plain reporting. So it can be a way to "disconnect" the property from a City. Again, it will depend on how your application is written.
For example, some applications have a field "deleted" which is 0 or 1. And all their searches and list views etc, anything that can appear in reports or anywhere the user can access it from the front end, exclude anything that is deleted == 1. However, if you create a custom report or a custom query to pull down a list of records that were deleted and even more so to see when it was last modified (another field) and by whom (i.e. who deleted it and when)..that is very advantageous from the executive standpoint.
And don't forget that you can revert accidental deletions as simple as deleted = 0 for those records.
My point is, if there is a functionality, there is always a reason behind it. Not always a good reason. But a reason. And often a good one too.
Using CASCADE means actually telling Django to delete the referenced record.
In the poll app example below: When a 'Question' gets deleted it will also delete the Choices this Question has.
e.g Question: How did you hear about us?
(Choices: 1. Friends 2. TV Ad 3. Search Engine 4. Email Promotion)
When you delete this question, it will also delete all these four choices from the table.
Note that which direction it flows.
You don't have to put on_delete=models.CASCADE in Question Model put it in the Choice.
from django.db import models
class Question(models.Model):
question_text = models.CharField(max_length=200)
pub_date = models.dateTimeField('date_published')
class Choice(models.Model):
question = models.ForeignKey(Question, on_delete=models.CASCADE)
choice_text = models.CharField(max_legth=200)
votes = models.IntegerField(default=0)
simply put, on_delete is an instruction to specify what modifications will be made to the object in case the foreign object is deleted:
CASCADE: will remove the child object when the foreign object is deleted
SET_NULL: will set the child object foreign key to null
SET_DEFAULT: will set the child object to the default data given while creating the model
RESTRICT: raises a RestrictedError under certain conditions.
PROTECT: prevents the foreign object from being deleted so long there are child objects inheriting from it
additional links:
https://docs.djangoproject.com/en/4.0/ref/models/fields/#foreignkey
Here is answer for your question that says: why we use on_delete?
When an object referenced by a ForeignKey is deleted, Django by default emulates the behavior of the SQL constraint ON DELETE CASCADE and also deletes the object containing the ForeignKey. This behavior can be overridden by specifying the on_delete argument. For example, if you have a nullable ForeignKey and you want it to be set null when the referenced object is deleted:
user = models.ForeignKey(User, blank=True, null=True, on_delete=models.SET_NULL)
The possible values for on_delete are found in django.db.models:
CASCADE: Cascade deletes; the default.
PROTECT: Prevent deletion of the referenced object by raising ProtectedError, a subclass of django.db.IntegrityError.
SET_NULL: Set the ForeignKey null; this is only possible if null is True.
SET_DEFAULT: Set the ForeignKey to its default value; a default for the ForeignKey must be set.
Let's say you have two models, one named Person and another one named Companies, and that, by definition, one person can create more than one company.
Considering a company can have one and only one person, we want that when a person is deleted that all the companies associated with that person also be deleted.
So, we start by creating a Person model, like this
class Person(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=20)
def __str__(self):
return self.id+self.name
Then, the Companies model can look like this
class Companies(models.Model):
title = models.CharField(max_length=20)
description=models.CharField(max_length=10)
person= models.ForeignKey(Person,related_name='persons',on_delete=models.CASCADE)
Notice the usage of on_delete=models.CASCADE in the model Companies. That is to delete all companies when the person that owns it (instance of class Person) is deleted.
Reorient your mental model of the functionality of "CASCADE" by thinking of adding a FK to an already existing cascade (i.e. a waterfall). The source of this waterfall is a primary key (PK). Deletes flow down.
So if you define a FK's on_delete as "CASCADE," you're adding this FK's record to a cascade of deletes originating from the PK. The FK's record may participate in this cascade or not ("SET_NULL"). In fact, a record with a FK may even prevent the flow of the deletes! Build a dam with "PROTECT."
Deletes all child fields in the database when parent object is deleted then we use on_delete as so:
class user(models.Model):
commodities = models.ForeignKey(commodity, on_delete=models.CASCADE)
CASCADE will also delete the corresponding field connected with it.
I am getting this error for a django model:
ProgrammingError at /admin/notifications/eventtoken/
column notifications_eventtoken.user_id does not exist
LINE 1: ...ications_eventtoken" INNER JOIN "users_user" ON ( "notificat...
^
HINT: Perhaps you meant to reference the column "notifications_eventtoken.used_at".
I added a foreign key for user like this:
class EventToken(models.Model):
token = models.CharField(max_length=255, db_index=True)
user = models.ForeignKey(User, null=False, blank=False)
used_at = models.DateTimeField(null=True, blank=True)
event = models.ForeignKey(Event)
it works when I remove the user field, but breaks otherwise and I can't tell much from the error message.
It works when I remove the user field.
This is because your database is missing a column that you have programmed in your EventToken class which I'm guessing is inside of notifications/models.py. This is why it is called a ProgrammingError.
There are several ways to solve this:
1. Delete your entire database and sync it again! Not recommended for anything running on production, but for test projects really doesn't hurt much.
2. Use a tool like south or the django migrate app (depending on which django version you're using) to automate adding a table to your database using the manage.py script. Edit: this solution is the most viable as it's quick and adheres to DRY principles when working across multiple development environments.
3a. Manually go into your database using a command line tool and add the missing column. If I haven't made it clear... you're saying "I want to save the user_id" foreign key in this table" while your table is saying back "There is nowhere to save this ID." ...)
3b. Use a database GUI (like PGadmin for postgresql) and add the user_id column to your notifications_eventtoken database table.
4. Delete the user field in the EventToken class! Don't do this, but understand it works because it clears the logical discrepancy.
With Django 1.5 and the introduction of custom user models the AUTH_PROFILE_MODULE became deprecated. In my existing Django application I use the User model and I also have a Profile model with a foreign key to the User and store other stuff about the user in the profile. Currently using AUTH_PROFILE_MODULE and this is set to 'app.profile'.
So obviously, my code tends to do lots of user.get_profile() and this now needs to go away.
Now, I could create a new custom user model (by just having my profile model extend User) but then in all other places where I currently have a foreign key to a user will need to be changed also... so this would be a large migration in my live service.
Is there any way - and with no model migration - and only by creating/overriding the get_profile() function with something like my_user.userprofile_set.all()[0]) somewhere?
Anyone out there that has gone down this path and can share ideas or experiences?
If I where to do this service again now - would obviously not go this way but with a semi-large live production system I am open for short-cuts :-)
Using a profile model with a relation to the built-in User is still a totally legitimate construct for storing additional user information (and recommended in many cases). The AUTH_PROFILE_MODULE and get_profile() stuff that is now deprecated just ended up being unnecessary, given that built-in Django 1-to-1 syntax works cleanly and elegantly here.
The transition from the old usage is actually easy if you're already using a OneToOneField to User on your profile model, which is how the profile module was recommended to be set up before get_profile was deprecated.
class UserProfile(models.Model):
user = OneToOneField(User, related_name="profile")
# add profile fields here, e.g.,
nickname = CharField(...)
# usage: no get_profile() needed. Just standard 1-to-1 reverse syntax!
nickname = request.user.profile.nickname
See here if you're not familiar with the syntactic magic for OneToOneField's that makes this possible. It ends up being a simple search and replace of get_profile() for profile or whatever your related_name is (auto related name in the above case would be user_profile). Standard django reverse 1-1 syntax is actually nicer than get_profile()!
Change a ForeignKey to a OneToOneField
However, I realize this doesn't answer your question entirely. You indicate that you used a ForeignKey to User in your profile module rather than a OneToOne, which is fine, but the syntax isn't as simple if you leave it as a ForeignKey, as you note in your follow up comment.
Assuming you were using your ForeignKey in practice as an unique foreign key (essentially a 1-to-1), given that in the DB a OneToOneField is just a ForeignKey field with a unique=True constraint, you should be able to change the ForeignKey field to a OneToOneField in your code without actually having to make a significant database migration or incurring any data loss.
Dealing with South migration
If you're using South for migrations, the code change from the previous section may confuse South into deleting the old field and creating a new one if you do a schemamigration --auto, so you may need to manually edit the migration to do things right. One approach would be to create the schemamigration and then blank out the forwards and backwards methods so it doesn't actually try to do anything, but so it still freezes the model properly as a OneToOneField going forward. Then, if you want to do things perfectly, you should add the unique constraint to the corresponding database foreign key column as well. You can either do this manually with SQL, or via South (by either editing the migration methods manually, or by setting unique=True on the ForeignKey and creating a first South migration before you switch it to a OneToOneField and do a second migration and blank out the forwards/backwards methods).
This one is driving me nuts right now. It was not happening before (even got screenshots I had to do for the user-manual since the customer required it).
I first noticed it on production server and then I checked and also happens in the dev server that comes with Django. The model appears on the main-page of the django admin, I can click it and it will display the list of point of sales. The problem comes whenever I want to edit an existing instance or create a new one.
I just click on the link (or put it on the bar) and it just hangs.
class PointOfSaleAdmin(admin.ModelAdmin):
list_display = ('id','business', 'user', 'zipcode', 'address','date_registered')
list_filter = ('business',)
filter_horizontal = ('services',)
admin.site.register(models.PointOfSale, PointOfSaleAdmin)
That's the registration of the model. All models are registered in the admin application and the user to test this is a super user. The model is:
class PointOfSale(models.Model):
user = models.ForeignKey(User)
zipcode = models.ForeignKey(Zipcode)
business = models.ForeignKey(Business)
services = models.ManyToManyField(Service,
verbose_name='available services')
date_registered = models.DateField(auto_now_add=True)
address = models.CharField(max_length=300)
Plus a few methods that shouldn't really matter much. Plus, last time before this that I tested the admin was right after I created all those methods, so it shouldn't matter on this.
The administrator very very rarely has to access this page. Usually it's just listing the PoS, but it still bothers me. Any idea of why it could be hanging? All other models are working just fine.
This is happening on both Django 1.2.5 and 1.3
EDIT:
I modified the timeout limits. It IS working, but somehow it takes several minutes for it to actually happen. So, there is something in the background that is taking ages. I don't understand how come it happens only for this model and it happens in different environments (and with small datasets)
I almost feel like slapping myself. My fault for not sleeping for so long.
The problem is that the zipcode list is pretty big (dozens of thousands) and the foreign key field is loaded as an html select tag, which means it loads every single entry. It's an issue with how much data there is simply.
Now I wonder how to control the way the foreign key is displayed in the admin. Anyone could help with that?
In your admin.py file, under the appropriate admin class, set
raw_id_fields = ('zipcode',)
This will display the zipcode's PK instead of a dropdown.
Is there a reason that you are setting up zipcode as it's own model instead of using a CharField or an actual zipcode modelfield?
I just wanted to add that another option here is creating a read_only_fields list. In cases where there is a relationship to a model with a large number of choices(in my case a rel table cataloging flags between a large number of users and discussion threads) but you don't need to edit the field. You can add it to the read_only_fields list will just print the value rather than the choices.
class FlaggedCommentsAdmin(ModelAdmin):
list_display = ('user', 'discussion', 'flagged_on')
readonly_fields = ('user', 'discussion')
For people still landing on this page: As Mamsaac points out in his original post, the timeout happens because django tries to load all instances of a ForeignKey into an html-select. Django 2 lets you add an auto-complete field which asynchronously lets you search for the ForeignKey to deal with this. In your admin.py do something like this:
from django.contrib import admin
from .models import Parent, Child
#admin.register(Parent)
class ParentAdmin(admin.ModelAdmin):
# tell admin to autocomplete-select the "Parent"-field 'children'
autocomplete_fields = ['children']
#admin.register(Child)
class ChildAdmin(admin.ModelAdmin):
# when using an autocomplete to find a child, search in the field 'name'
search_fields = ['name']
Have you tried checking the apache logs (if you're using apache obviously) or any other HTTP server related logs? That might give you an idea of where to start.
That's the only model that is affected? You mentioned methods on the model. Try commenting out those methods and trying again (including the __unicode__ method), just to see if they somehow affect it. Reduce everything down to the bare minimum (as much as possible obviously), to try and deduce where the regression started.
Try to monitor server resources when you request this page. Does CPU spike dramatically? What about network I/O? Could be a database issue (somehow?).
Sorry this doesn't really answer your question, but those are the first debugging techniques that I'd attempt trying to diagnose the problem.