A version of this question has been asked here several times but none of the answers provided solve my exact problem.
I'm trying to bulk_create a batch of objects of a model with a ManyToMany field.
In this case, the ManyToMany field refers to the same model, though I'd also be interested in the general case.
Let's say this is my model:
class Person(models.Model):
name = models.CharField(max_length=20, blank=True, null=True)
friends = models.ManyToMany("self", related_name="friends_with", null=True)
After bulk_creating a large number of Person objects, I want to add the information who's friends with whom within this group.
Is there a more efficient way to go about this than looping through each new Person and calling .set(friend_pks) or .add(*friend_pks)?
I.e., an analogue of bulk_update.
I've achieved some speed-up by wrapping the loop into with transaction.atomic() (from this answer) but it's still quite slow.
Okay, my post was premature -- it seems that this answers the question.
The key is to bulk_create the through models. In this example:
friends_relation_1 = Person.friends.through(from_person_id=1, to_person_id=2)
friends_relation_2 = Person.friends.through(from_person_id=2, to_person_id=8)
Person.friends.through.objects.bulk_create([friends_relation_1, friends_relation_2, ...])
I know the foreign key concept,Djongo Array field , however is there an alternative?
The issue with the foreign key concept is that I would need to make multiple hits to the database and the issue with Array field is the lack of information and the errors emerging without a known solution.
What I would like to do basically is in fact add multiple instances of a model say comments to a single field in another model but I would like to embed it rather than creating a new table for comments which I tried using an abstract model however it did not work out.
I would like to know any alternative solution.
You can use foreign keys, and to avoid making separate query for every related record you can extract them all at once using prefetch_related - https://docs.djangoproject.com/en/3.1/ref/models/querysets/#prefetch-related :
Returns a QuerySet that will automatically retrieve, in a single batch, related objects for each of the specified lookups.
Code example:
# models.py
class Post(models.Model):
title = models.CharField(max_length=100)
content = models.TextField()
class Comment(models.Model):
post = models.ForeignKey(Post, models.CASCADE)
text = models.TextField()
# Somewhere else, fetch posts together with related comments:
# It will take 2 requests in total
# - one to fetch posts, another to fetch all related comments
for post in Post.objects.all().prefetch_related('comment_set'):
print([c.text for c in post.comment_set.all()])
I'm developing a Property Management System with Django, right now I'm working on an app named by "Property Check", basically the purpose of it is to provide a form with a list of tasks like "Diswasher: clean & empty?", those tasks need to be checked at a property by a staff member.
The main idea is to allow admin to create Tasks and their Categories on the admin side.
Example: Task - Dishwater: clean & empty belongs to Category - Kitchen.
Each Property Check belongs to a property, it has the list of tasks and those tasks have different status, like "Checked" or "Needs attention".
So far this is what I've created:
models.py
class Task(models.Model):
name = models.CharField(db_column='SafetyTaskName', max_length=100, blank=False, null=False)
category = models.ForeignKey(Categories, db_column='category')
task_check = models.ForeignKey(TaskCheck)
class Categories(models.Model):
name = models.CharField(db_column='Categories', max_length=40, null=False, blank=False)
class TaskCheck(models.Model):
status = models.CharField(db_column='Status', choices=STATUS_CHOICES, default='nd')
image = models.ImageField(upload_to='property_check',null=True)
notes = models.CharField(db_column='Notes', max_length=500, blank=True, null=True) # Field name made lowercase.
class Propertycheck(models.Model):
property = models.ForeignKey(Property, models.DO_NOTHING, db_column='ID_Property') # Field name made lowercase.
task = models.CharField(TaskCheck)
name = models.CharField(db_column='Name', max_length=150)
date = models.DateField(db_column='Date', default=timezone.now) # Field name made lowercase.
next_visit = models.DateField(db_column='Next Visit')
staff = models.ForeignKey(User, db_column='Staff', max_length=25)
notes = models.CharField(db_column='Notes', max_length=500, blank=True, null=True) # Field name made lowercase.
Functional example of what I pretend:
A staff member goes to a property that needs to be checked, he fills
the form that contains all the tasks. In case of needing more tasks,
the admin goes to the admin panel and adds a new one. The same status
applies to every task.
Requirements:
A property has many property checks;
A property check has a list of tasks;
Admin must be capable to add tasks and categories;
Tasks belong to one category;
Property checks are made by a staff member;
The task list is the same to every property;
Every task must have a status (Ex.: completed state);
Problem:
I'm a bit confused about where to use the foreignkeys. I need property check to show the list of tasks, and for each one, their status.
Due to my experience I'm stuck at this right now, so I need some help with this.
Could you please take a look at what've done and let me know a better solution?
* **Update ***
Thanks to Bruno Desthuilliers answer, I could restructure my models by following his advices. I think this solution is closer to what I need, but my question is, are my changes 100% correct according to the requirements on Bruno's answer?
class Task(models.Model):
name = models.CharField(max_length=100)
category = models.ForeignKey(Categories)
property = models.ManyToManyField(Property)
class Categories(models.Model):
name = models.CharField(max_length=40)
class TaskCheck(models.Model):
status = models.CharField(choices=STATUS_CHOICES, default='nd')
image = models.ImageField(upload_to='task_check', null=True)
notes = models.TextField(max_length=500)
task = models.ForeignKey(Task)
property_check = models.ForeignKey(Propertycheck)
class Propertycheck(models.Model):
property = models.ForeignKey(Property, models.DO_NOTHING)
name = models.CharField(max_length=150)
date = models.DateField(default=timezone.now)
next_visit = models.DateField()
staff = models.ForeignKey(User, max_length=25)
notes = models.TextField(max_length=500, default='')
My english ain't the best and I wasn't sure about the best title for my question.
A property has many property checks;
This describes only half of the relationship's cardinality - you also need to specify how many properties a property check can belong to. In this case the answer seems rather obvious (I can't see a case where a same property check would belong to more than one property), BUT unless you have a real and deep working knowledge of the domain, you should STILL ask your customer - sometimes "obvious" things are actually wrong ;-)
But if we consider that "a property has many property checks" AND "a property check belongs to one single property", we have a one to many relationship. At the db schema level, this is materialized by a foreign key on the "one" side in the "many" side, ie PropertyCheck must have a fk on Property.
This is logical when you remember that in the relational model, fields are atomic values (one single value in each field). You couldn't store a list of related PropertyCheck ids in Property, but you can store a single Property id in each PropertyCheck.
This is also logical when you think of the constraints - a Property can actually have "zero to many" related property checks (you can have a property that has never been "checked" so far), but a PropertyCheck MUST have a related property (it wouldn't make sense to have a property check without property, would it ?). If property checks ids where stored as a list in Property, you could still create property checks without property (and you would have consistency issues too if a property check was deleted and the property's list of property checks not updated).
So, to make a long story short: for a one to many relationship, the fk resides on the "many" side and points to the "one" side.
A property check has a list of tasks;
Are you sure this one is right ? It seems to me that you're confusing the user's view of the application with the database schema.
Sure, what the user views when he's on the "property check" page is a list of tasks to perform (and a checkbox etc for each task) - but this doesn't mean the tasks belong to the property check. If that was the case, the admin would have to create a new list of tasks for each property check... As fad as I understand the domain, the point is that there's a list of tasks for each property, and that the system builds a list of (not yet checked) task checks for each property check. Which FWIW is already what you started to design.
So (assuming I got the problem right), your rule is actually "each property has a list of tasks". Now we have the other cardinality to sort out: does a task belong to one single property, or can the same task be shared by many properties ?
We already covered the first case (cf above). In the second case - which is actually more likely since there are certainly quite a few tasks that will be the same for most properties -, you have a many to many relationship. Those are materialized by a relationship table which has a fk on each side of the relationship, with a unicity constraint on the pair of fks (you don't want to have the same task listed twice for a same property). Note that with Django's ORM, you don't need to explicitely declare a model for this (unless of course you need to add some other field to the relationship, but so far I don't see a need for this here) - just declare a many2many field on any side of the relationship (doesn't really matter) and the ORM will create the intermediary table for you.
Then you have a relationship between property check and task check. Here it's a simple one to many relationship - a property check has many task checks, a task check belongs to one single property check. The only constraint here is that those task checks's task must belong to the same property as the property check's property (yes, it's a bit confused when written that way xD). To say it more simply: the task list of a property is used as a blueprint to create the tasks check list for a property check.
IOW you have:
a task belongs to one or many property
a property has many tasks
a property has many property checks
a property check belongs to one single property
a task check references one single task
a task has many task checks
a task check's task must be one of the tasks of the task check's property check's property (duh!)
Admin must be capable to add tasks and categories;
This is a requirement indeed, but it's not related to what interest us here since this is handled at code level (permissions), not at the db schema level.
Tasks belong to one category;
and a category can have many tasks - one to many relationship, cf above.
Property checks are made by a staff member;
and a staff member can do many property checks - one to many relationship, cf above.
The task list is the same to every property;
Ah, this one is interesting. If this is true, it means that you actually don't need any relationship between Task and Property.
But that's still something I'd double-check with the customer - from experience, customers tend to only think of the general case when they explain the domain, then when they start testing the software a whole lot of corner cases appear out of the blue, and you suddenly realize you will have to rewrite half or more of your schema and code. I actually had the case on one of the very first application I was involved in - not as a developper actually, I was just one of the app's users, and the first thing I had to do with the app revealed such shortcomings, leading to a full month of additional development (which the company that employed me had to pay for since they had signed on the - wrong - requirements). Needless to say the persons responsible for this costly mistake were either blamed or, for one, just plain fired.
Every task must have a status (Ex.: completed state);
This one is wrong too. The status belongs to the task check, not to the task.
Ok, so the models you posted are not too far off. As I already mentionned in a comment, you have some one to many relationships wrong (fk on the wrong side of the relationship) but with the explanations above you should be able to sort this out. You may also want to double check some of the rules with the customer and adjust your models accordingly.
A couple other things now:
First, unless you're working with a legacy database (which is obviously not the case here), you'd be better leaving the model fields db_column attribute alone - the ORM will use the model field's name as db column name, and that's most often the best default - at least you don't have to check your models.py file for column names when you want to do raw SQL queries. Note that for foreign keys, the model's attribute will yield the related model instance, but will create a "fieldname_id" column.
Second point: if a textfield or charfield is not required, do NOT use "null=True" - else you'd have two possible values indicating "no data", either SQL "NULL" or an empty string. Better to only have one of them, in this case the empty string, so remove the "null=True" and use "default=''" instead. Also, for free text (the "notes" field for example), you may want to use a textfield instead of a charfield. This avoids placing useless contraints on the maximum length (that you can bet the users WILL ask you to extend), and will also be translated by Django's ModelForms to a proper html "text" widget instead of a (single line) html "input".
Third point: "blank=False" and "null=False" are already the defaults - a field is required unless specified otherwise - so explicitely passing them for required fields only adds "code noise". The most readable code is no code at all ;-)
Hope that clears up things for you, if not feel free to ask for details / explanations in a comment.
I want to create an Instruction that has multiple Steps. A simplified model would look like this:
class Step(models.Model):
description = models.CharField(max_length=255)
class Instruction(models.Model):
steps = models.ForeignKey(Step)
The problem is that I want to create a new Instruction with multiple steps, but when I create one in the admin, I should have a repeatable form field. For each step I can add a field and create a new Step. I do not need to be able to select an already existing step. I'm not sure if there is something OOTB of a package that already does that... Any ideas how to tackle this?
To give an example of what I'm trying to accomplish: the ACF repeater field in WP:
In my case I would only need a description field with the description of the step
You have things a bit backwards. The ForeignKey relationship should be the other way around (since an instruction can have many steps, but each step only has one related instruction...a Many-to-One relationship).
class Step(models.Model):
description = models.CharField(max_length=255)
instruction = models.ForeignKey(Instruction, related_name='steps')
class Instruction(models.Model):
# some fields
Now, in your Admin, you can use inlines to display these fields in a "repeatable" manner, similar to ACF.
I have a fairly simple model in Django, let's say Book:
class Book(models.Model):
name = models.CharField(max_length=255, default="")
description = models.TextField(null=True, blank=True)
creation_date = models.DateTimeField()
Because the structure and data basically comes from an external API, I have contained it in its own app and I rather not want to modify it.
Now I want to add another source of different books to the project and I'm not really sure what the best solution is here.
Let's say the second model is:
class NextBook(models.Model):
title = models.CharField(max_length=255, default="")
long_description = models.TextField(null=True, blank=True)
created = models.DateTimeField()
So the basic fields are there, but have different names. To get the two together, I can probably use another model with a GenericForeignKey:
class BaseBook(models.Model):
book_type = models.ForeignKey(ContentType)
book_id = models.PositiveIntegerField()
book_object = generic.GenericForeignKey('book_type', 'book_id')
But then I cannot query for all the book as e.g. BaseBook.objects.all().order_by('created') wouldn't work. Surely, I could go on and duplicate a datetime field in the base model and then the title and so on, but I think that would be the wrong direction. Also inheritance seems not a choice if I don't want to touch the specific models.
I am looking for some design pattern or something that let's me efficiently 'plug in' more providers and query for all objects, while not making a huge mess of model structure and database queries.
There’s a fundamental contradiction in your question. Do you assume that, for a given book, the created reported by different providers will be the same?
If yes, why can’t you make a created field on your BaseBook and order by that? You can then drop the creation_date from your Book (or just ignore it, if you don’t want to touch it).
If not, how do you want to order books (not provider entries) by created in the first place?
Also, what you’re trying to do sounds like a good case for a free-schema database like MongoDB. There, you can embed provider entries directly in your book documents, then order by something like “created from the first provider entry for the book”. Thus, you maintain self-contained provider documents without denormalization. You still need created to mean the same thing for all providers, though.