I need to dynamically create a series of models and relationships in a Postgres database for the purpose of demoing some features.
The issue is these models can't interfere with or be returned by queries in the normal functioning of the Django app. However they must have the full behavior set of the model and the ability to be returned by queries during the demo.
Using factories (we use Factory Boy primarily) is imperfect because the models can't be queried without being saved to the database at which point they interact with regular operation queries.
A second database is imperfect as well because the demo occurs during normal app operation and thus the "valid" model instances still need to be available.
Flagging the models as is_demo also won't work as there's a half dozen different types of models that need to be instantiated for the demo to work as intended.
I could clear the specific models that are created with a cron job or similar but they'd effect the database between creation and the scheduled job.
Running everything in a staging environment is acceptable, but the ideal answer allows me to run the demos on the production build.
Is there an accepted approach or useful library that I've missed?
A sample model that's involved in the demo:
class Transaction(models.Model):
amount = models.DecimalField(max_digits=8, decimal_places=2, db_index=True,
editable=False)
assigned_user = models.ForeignKey(User, on_delete=models.CASCADE)
date = models.DateField(db_index=True, editable=False)
description = models.CharField(max_length=255, null=True, blank=True)
is_declined = models.BooleanField(default=False)
is_pending = models.BooleanField(db_index=True)
fi_account = models.ForeignKey(
FinancialAccount, on_delete=models.CASCADE)
#property
def organization(self):
org = self.fi_account.organization
return org
We'd need to be able to query this model and use its .organization property successfully. Without having it effect other queries.
You could create a clone of your production environment (database and django app), then update your production codebase to include the demo features/fixes. This way you'd have a copy of the "valid" production models in addition to the new models you're demoing.
It sounds like you should have a "staging" environment that closely matches your production environment for situations just like this.
Edit
One option I just thought of would be to create a Demo model which keeps track of records created as part of a demo. This Demo model would be have a ForeignKey to your User model, so each User can create/destroy their own demo (or your view can create it).
This would let you delete any records created as part of a specified Demo object.
However, you'd have to make sure you add an exclude for all of your current queries/filters of the models you have now.
Honestly, I'm not sure if what you're asking for can be done without exhaustively modifying all of your Model filters to exclude whatever demo records you create.
Related
What I need is basically a database model with version control. So that every time a record is modified/deleted, the data isn't lost, and the change can be undone.
I've been trying to implement it myself with something like this:
from django.db import models
class AbstractPersistentModel(models.Model):
time_created = models.DateTimeField(auto_now_add=True)
time_changed = models.DateTimeField(null=True, default=None)
time_deleted = models.DateTimeField(null=True, default=None)
class Meta:
abstract = True
Then every model would inherit from AbstractPersistentModel.
Problem is, if I override save() and delete() to make sure they don't actually touch the existing data, I'll still be left with the original object, and not the new version.
After trying to come up with a clean, safe and easy-to-use solution for some hours, I gave up.
Is there some way to implement this functionality that isn't overwhelming?
It seems common enough problem that I thought it would be built into Django itself, or at least there'd be a well documented package for this, but I couldn't find any.
When I hear version control for models and Django, I immediately think of django-reversion.
Then, if you want to access the versions of an instance, and not the actual instance, simply use the Version model.
from reversion.models import Version
versions = Version.objects.get_for_object(instance)
I feel you can work around your issue not by modifying your models but by modifying the logic that access them.
So, you could have two models for your same object: one that can be your staging area, in which you store values as the ones you mention, such as time_created, time_modified, and modifying_user, or others. From there, in the code for your views you go through that table and select the records you want/need according to your design and store in your definitive table.
rather than using the models.ManyToMany field in django
i just set up a intermediary field with a bunch of foreign keys.
is there any reason why this wouldn't work. I can't think of any but why not see if any of you have tried the same.
class Authorization(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL,
on_delete=models.CASCADE,
)
permission = models.ForeignKey( 'venueadmin.Permissions', blank=True, null=True)
#venue = models.ForeignKey(venue) <-- commented out cause I haven't made the model its referencing yet.
That is exactly how ManyToMany relationships work, except few things to consider.
First, you might want to check how Django generates ManyToManyField here. It does almost the same you did here.
Now, before starting using separate model think about next things:
database constraint is missing. It means that there is no validation
on what has been put in to the Authorization table as example -
Duplicate rows;
there is no indexation which means that search will become slow once Authorization grows. Interesting enough I did not find it in Django, maybe nobody cares? Two columns table might not be so critical;
there is no reason to keep an authorization record for user with no permissions. What is the reason of setting permissions to NULL? Does it carry any useful information for DB administrator in future?
I am using django to create a video library website.
I have 3 models for the library: Video, VideoTopic, and Course.
I have a TimedeltaField for the duration of each Video. Upon saving each video in admin, I want to get the sum of each TimedeltaField for all Videos within a Course, then save that sum to a TimedeltaField in my Course model.
I'm using django-timedeltafield, which will handle the summarization very well. What I have yet to find a solution for is how to update the Course model after I save each Video in admin. Any advice?
My models are as follows:
class Video(models.Model):
title = models.CharField(max_length=200)
topic = models.ForeignKey('VideoTopic')
duration = timedelta.fields.TimedeltaField(default="3 minutes, 30 seconds", help_text="(x minutes, x seconds)")
class VideoTopic(models.Model):
name = models.CharField(max_length=200, blank=True)
course = models.ForeignKey('course.Course')
class Course(models.Model):
name = models.CharField(max_length=200)
duration = timedelta.fields.TimedeltaField()
I recommend you use a django signal. This will ensure your models are synchronized regardless of where they are updated (in the app, admin, etc.).
Register the post save signal on the Video. In the signal, you will receive the video instance. Simply query for its related records in VideoTopic and Course and do the math there. I recommend doing all this in the transaction if you don't want it to ever fail. If you don't need the information always up-to-date, it's also an opportunity to do some background work to make the save lighter weight.
Why a signal?
Separates concerns
Ensures your logic always runs
Can help decouple things out of the model directly if it has dependencies, which can cut down on some situations of loading order/circular dependencies.
Alternatively, you could simply override the Video model's save method and do the same thing there. In either case, just use the related properties in the ORM from your foreign keys if you don't want to write the queries yourself.
An alternative is to override the admin save method to update the course model. See https://docs.djangoproject.com/en/dev/ref/contrib/admin/#modeladmin-methods.
Half way through my current project, after suffering the pain of spending uncountable minutes on debugging, I have decided to adopt TDD. To start, I am planning to write a set of unit tests for each existing models. But for models that only have attributes defined (ie. no additional methods/properties) I am not sure what I need to test nor how.
class Product(models.Model):
name = models.CharField(max_length=50)
description = models.TextField(default='', blank=True)
retails = models.ManyToManyField(Retail, verbose_name='Retail stores that carry the product')
manufacturer = models.ForeignKey(Manufacturer, related_name='products')
date_created = models.DateTimeField(auto_now_add=True)
date_modified = models.DateTimeField(auto_now=True)
Using Product as an example, what are the things about it that unit tests should cover? And how should ForeignKey and ManyToManyField be covered?
This was an article I found helpful: A Guide to Testing in Django (archived link). Here is a good summary of what to test:
Another common setback for developers/designers new to testing is the
question of 'what should (or shouldn't) I test?' While there are no
hard & fast rules here that neatly apply everywhere, there are some
general guidelines I can offer on making the decision:
If the code in question is a built-in Python function/library, don't test it. Examples like the datetime library.
If the code in question is built into Django, don't test it. Examples like the fields on a Model or testing how the built-in
template.Node renders included tags.
If your model has custom methods, you should test that, usually with unit tests.
Same goes for custom views, forms, template tags, context processors, middleware, management commands, etc. If you implemented
the business logic, you should test your aspects of the code.
So, for your example, there wouldn't really be anything to test until you write some custom functions.
In my opinion, testing ForeignKey and ManyToManyField links would fall under the second category (code built into Django), so I wouldn't test these, as you are really testing whether or not Django is functioning properly. If you have a method which creates an instance of your product, including foreign relationships and M2Ms, you could verify the data has been created, that would be testing your custom method, not Django functionality.
Using the TDD paradigm, the tests are built to verify business logic, and design requirements.
My CS350 class TDD stipulated that it's best practice to test all accessors and mutators. So for a model, you would first write tests that call each assessor function and make sure that it returns the proper value.
For each function which changes a data field in the model, you would not only test the result of that data field in particular, but you would also test all of the other fields in the model instance to make sure that none of them were modified erroneously.
To restat:, if a model has fields a, b, and c, you would create an instance using your constructor, then asset that all three are set properly. Say there's another function, set_a(). You would assert that not only the value of 'a' has changed, but that the values of b and c remain unchanged.
I just got done working through the Django tutorials for the second time, and am understanding things much more clearly now. However, I'm still unclear how apps inside a site interact with one another.
For example, lets say I'm writing a blog application (a rather popular activity, apparently). Blog posts and comments tend to go together, and yet they are distinct enough that they should be built into separate apps, as is the general philosophy of Djano development.
Consider the following example. In reality I would not actually write the comment app myself, as good code for that already exists on the web, but this is for demonstration/practice purposes:
mysite/blog/models.py
from django.db import models
class post(models.Model):
title = models.CharField(max_length=200)
author = models.CharField(max_length=200)
content = models.TextField()
mysite/comments/models.py
from django.db import models
from mysite.blog.models import post
class comment(models.Model):
id = models.AutoField()
post = models.ForeignKey(post)
author = models.CharField(max_length=200)
text = models.TextField()
Is what I wrote above, importing a model from another app and setting it as a foreign key, how Django apps interact? Or is there a different/better method for the apps that comprise a site to interact?
Update
Per the recommendation in one response, I'm reading the documentation for contrib.contenttypes. If I'm reading this correctly, I could rewrite my example comment app like this:
from django.db import models
from django.contrib.contenttypes.models import ContentType
from django.contrib.contentypes import generic
class comment(models.Model):
id = models.AutoField()
author = models.CharField(max_length=200)
text = models.TextField()
content_type = models.ForeignKey(ContentType)
content_object = generic.GenericForeignKey(content_type, id)
Would this be correct?
Take a look at django's built-in contenttypes framework:
django.contrib.contenttypes
It allows you develop your applications as stand-alone units. This is what the django developers used to allow django's built-in comment framework to attach a comment to any model in your project.
For instance, if you have some content object that you want to "attach" to other content objects of different types, like allowing each user to leave a "favorite" star on a blog post, image, or user profile, you can create a Favorite model with a generic relation field like so:
from django.db import models
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
from django.contrib.contenttypes import generic
class Favorite(models.Model):
user = models.ForeignKey(User)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = generic.GenericForeignKey('content_type', 'object_id')
In this way you can add a Favorite star from any user to any model in your project. If you want to add API access via the recipient model class you can either add a reverse generic relation field on the recipient model (although this would be "coupling" the two models, which you said you wanted to avoid), or do the lookup through the Favorite model with the content_type and object_id of the recipient instance, see the official docs for an example.
"Is what I wrote above, importing a model from another app and setting it as a foreign key, how Django apps interact?"
Yep. Works for me.
We have about 10 applications that borrow back and forth among themselves.
This leads to a kind of dependency in our unit test script.
It looks like this.
"ownership". We have a simple data ownership application that defines some core ownership concepts that other applications depend on. There are a few simple tables here.
"thing". [Not the real name]. Our thing application has data elements owned by different user groups. There are actually several complex tables the model for this app. It depends on "ownership".
"tables". [Not the real name]. Some of our users create fairly complex off-line models (probably with spreadsheets) and upload the results of that modeling in "tables". This has a cluster of fairly complex tables. It depends on "ownership".
"result". [Not the real name]. Our results are based on things which have owners. The results are based on things and tables, and are responses to customer requests. This isn't too complex, perhaps only two or three core tables. It depends on "things" and "table". No, it doesn't completely stand-alone. However, it is subject to more change than the other things on which it depends. That's why it's separate.
"processing". We schedule and monitor big batch jobs. This is in this application. It's really generic, and can be used in a variety of ways. It completely stands alone.
"welcome". We have a "welcome" app that presents a bunch of mostly static pages. This doesn't have too many tables. But it's on it's second incarnation because the first was too complex. It completely stands alone.
The only relationship among the dependent apps is some table names. As long as we preserve those tables (and their keys) we can rearrange other apps as we see fit.
There's nothing wrong (imho) with making some app dependent on another. After all, apps are just operations on a set of models. you just have to always be aware of which app depends on which app (I guess you could call that a dependency map).
You can achieve loose coupling with the contenttypes framework. It allows an app to be truely portable/pluggable yet still integrated with other applications.
I wrote a comments app (yea, I re-invented the wheel), that can be integrated into any other application, with a few lines in the template of the page where comments should be posted (using custom tags).
Say you want a model "thread" to be pluggable into any other model. The idea is to create e generic foreign key (see django documentation on that), and write a small function that takes any object and returns a "thread" corresponding to it (or creates one if necessary), and write a custom template tag that uses that functionality, e.g. {% get_thread for arbitrary_object as thread %}. All posts are related to a thread, which is related to the object, which can be of any type.
You can think of the "thread" object as a kind of a proxy, so instead of having a post be related to a certain "article" or a "blog post", it's just related to a thread, which is abstract in a sense, what is is a thread? It's just a collection of posts. The thread then allows itself to be related to any object regardless of its type. (although it does more than that, it could hold extra information such as allowing/disallowing anon. posts, closing/opening comments on the page, etc ..)
EDIT
Here's how you can create a generic foreign key with the content types framework:
from django.contrib.contenttypes import generic
from django.contrib.contenttypes.models import ContentType
class Thread( models.Model ):
object_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
object = generic.GenericForeignKey('object_type', 'object_id')
You can make it more "transparent" by exploiting the implicit "common" interface that django assumes all objects implement ..
#inside the Thread class:
def __unicode__(self):
return unicode(self.object)
def get_absolute_url(self):
return self.object.get_absolute_url()
Your code seems correct. I would keep the post and the comment in a blog app though. I am not saying this is the Django way, but those models are close enough to be in the same app.
How To Divide The Project
I would seperate an app if;
I plan to design it resuable. (and try loose coupling)
(for big projects) It consists of a major section of the project.
On the other hand; having many tiny apps (such as an app with a single model and two views) is hard to read and maintain IMHO.
How Apps Should Interact
This depends on the type of project and the type of the app again. For example if an app is implicitly dependent on another (ie not generic) importing and using references from the other app is acceptable. In this case the second app might be installed alone, but the first one needs the presence of the second.
If you want to make an app highly reusable and generic, such as a commenting app, you might need to integrate some setup mechanism. Maybe some new settings or additional URL configuration, or a special directive/method on your models... django.contrib.admin is a good example for this.
Apps shouldn't interact if it is not necessary though. Designing apps to avoid unnecessary coupling is very useful. It improves your app's flexibility and makes it more maintainable (but possibly with a higher cost in integrating).