I want to make changes in a model instance A, when a second model instance B
is saved,updated or deleted.
All models are in the same Django app.
What would be the optimal way to do it?
Should I use signals?
Override default methods[save, update,delete]?
Something else?
Django documentation warns:
Where possible you should opt for directly calling the handling code, rather than dispatching via a signal.
Can somebody elaborate on that statement?
The performance impact of your signal handlers depends of course on their functionality. But you should be aware of the fact that they are executed synchronously when the signal is fired, so if there's a lot of action going on in the handlers (for example a lot of database calls) the execution will be delayed.
Related
How does one properly structure a larger django website such as to retain testability and maintainability?
In the best django spirit (I hope) we started out by not caring too much about decoupling between different parts of our website. We did separate it into different apps, but those depend rather directly upon each other, through common use of model classes and direct method calls.
This is getting quite entangled. For example, one of our actions/services looks like this:
def do_apply_for_flat(user, flat, bid_amount):
assert can_apply(user, flat)
application = Application.objects.create(
user=user, flat=flat, amount=bid_amount,
status=Application.STATUS_ACTIVE)
events.logger.application_added(application)
mails.send_applicant_application_added(application)
mails.send_lessor_application_received(application)
return application
The function does not only perform the actual business process, no, it also handles event logging and sending mails to the involved users. I don't think there's something inherently wrong with this approach. Yet, it's getting more and more difficult to properly reason about the code and even test the application, as it's getting harder to separate parts intellectually and programmatically.
So, my question is, how do the big boys structure their applications such that:
Different parts of the application can be tested in isolation
Testing stays fast by only enabling parts that you really need for a specific test
Code coupling is reduced
My take on the problem would be to introduce a centralized signal hub (just a bunch of django signals in a single python file) which the single django apps may publish or subscribe to. The above example function would publish an application_added event, which the mails and events apps would listen to. Then, for efficient testing, I would disconnect the parts I don't need. This also increases decoupling considerably, as services don't need to know about sending mails at all.
But, I'm unsure, and thus very interested in what's the accepted practice for these kind of problems.
For testing, you should mock your dependencies. The logging and mailing component, for example, should be mocked during unit testing of the views. I would usually use python-mock, this allows your views to be tested independently of the logging and mailing component, and vice versa. Just assert that your views are calling the right service calls and mock the return value/side effect of the service call.
You should also avoid touching the database when doing tests. Instead try to use as much in memory objects as possible, instead of Application.objects.create(), defer the save() to the caller, so that you can test the services without having to actually have the Application in the database. Alternatively, patch out the save() method, so it won't actually save, but that's much more tedious.
Transfer some parts of your app to different microservices. This will make some parts of your app focused on doing one or two things right (e.g. event logging, emails). Code coupling is also reduced and different parts of the site can be tested in isolation as well.
The microservice architecture style involves developing a single application as a collection of smaller services that communicates usually via an API.
You might need to use a smaller framework like Flask.
Resources:
For more information on microservices click here:
http://martinfowler.com/articles/microservices.html
http://aurelavramescu.blogspot.com/2014/06/user-microservice-python-way.html
First, try to brake down your big task into smaller classes. Connect them with usual method calls or Django signals.
If you feel that the sub-tasks are independent enough, you can implement them as several Django applications in the same project. See the Django tutorial, which describes relation between applications and projects.
I have a class from which all my entity definitions inherit:
class Model(db.Model):
"""Superclass for all others; contains generic properties and methods."""
created = db.DateTimeProperty(auto_now_add=True)
modified = db.DateTimeProperty(auto_now=True)
For various reasons I want to be able to occasionally modify an entity without changing its modified property. I found this example:
Model.__dict__["modified"].__dict__["auto_now"] = False
db.put(my_entity)
Model.__dict__["modified"].__dict__["auto_now"] = True
When I test this locally, it works great. My question is this: could this have wider ramifications for any other code that happens to be saving entities during the small period of time Model is altered? I could see that leading to incredibly confusing bugs. But maybe this little change only affects the current process/thread or whatever?
Any other request coming in to the same instance and being handled whilst the put is in progress will also get auto_now=False, whilst unlikely it is possible
Something else other thing to consider
You don't have try block around this code, if you get a timeout or error during the put() your code will leave the model in the modified state with auto_now=False .
Personally in think its a bad idea and will definatley be a source of errors.
There are a number of ways of achieving this without manipulating models,
consider setting the default behaviour to auto_now=False, and then have two methods you use for updating. The primary method sets the modified time to datetime.now() just before you do the put(), e.g save() and save_without_modified()
A better method would to override put() in your class, then set modified and then call super put() have put() accept a new argument like modified=False so you don't set the modified date before you call super.
Lastly you could use _pre_put hook to run code before the put() call, but you need to annotate the instance in some way so the _pre_put method can determine if modified needs to be set or not.
I think each of these strategies is a lot more safe than hacking the model
Using wxpython in MVC, I looked for a way to let the models tell the controllers about changes. I found (py)pubsub, which implements a global notification mechanism: Messages are sent to one place (the pubsub Publisher), which sends them to all subscribers. Each subscriber checks whether the message is interesting, and does what is needed.
From Smalltalk times, I know a more "local" approach: Each model object keeps a list of interested controllers, and only sends change notifications to these. No global publisher is involved. This can be implemented as part of the Model class, and works in much the same way, except it's local to the model and the controller.
Now is there a reason to use the global approach (which seems much less performant to me, and might be prone to all the issues related to global approaches)? Is there another package implementing a local observer?
Thanks!
I'm not really seeing the subtle difference here. As far as I know, pubsub is the way to go. It's included in wxPython in wx.lin.pubsub or you can download it from http://pubsub.sourceforge.net/. You can put the listeners just in the models and the publisher(s) just in the controller or however you need to. Here are a couple links to get you started:
http://www.blog.pythonlibrary.org/2010/06/27/wxpython-and-pubsub-a-simple-tutorial/
http://wiki.wxpython.org/WxLibPubSub
I've been playing around for a while to do MVC with wxpython and i know what you mean about pubsub being global.
The latest idea i've come up with is each view and model have there own observer.
The observers have weak references to their handlers and it all works in a separate thread so as to not block the GUI. To call back to the GUI thread I'm using wxAnyThread Gui method decorator.
There are 3 types of signal that get sent, for the model you can set which attributes are observed they automatically send out a signal when they are changed. then on both the model and the view you can send a message signal or a keyword signal. Each of the three signal types have to be unique per view or model as they are used to make a tuple that identify them.
model attributes
controller handlers are decorated with
#onAttr('attributeName')
def onModelAttributeName(self, attributeName)
When you bind to a method that handlers attributes it straight away calls the handler with its current value and then continues to observe changes.
Sending messages
Use the method
view/model.notify('Your message'):
The controller callback is decorated with
#onNotify('Your message')
def onYourMessage(self):
Sending keywords
Use the method
view/model.notifyKw(valid=True, value='this)
The controller callback is decorated with
#onNotifyKw('valid', 'value')
def onValidValueKw(self, valid, value)
The GUI is left knowing nothing about the models the only thing you add to the GUI is the view signaler, the controller attaches it self to this, so if you don't add a controller the view will just happily fire off messages to no one.
I've uploaded what i have so far on github
https://github.com/Yoriz/Y_Signal
https://github.com/Yoriz/Y_Mvc
Both have unit test which should give a bit of an example of what it does, but i will create some wxpython examples.
I'm using python version 2.7 and the Ysignals module requires
https://pypi.python.org/pypi/futures/2.1.3 for the threading.
Please take a look ill be interested in what someone else thinks of this way of approaching mvc or to point out something i seriously overlooked.
I have a view that needs to perform updates to the database involving a shared resource that needs locking (the implementation is complex, but nothing more than a shared counter at heart).
To shield myself from race conditions, I'm using code that looks roughly like this:
#transaction.commit_manually
def do_it(request):
affected_models = Something.objects.select_for_update(blah = 1)
for model in affected_models:
model.modify()
model.save()
transaction.commit()
Is this usage of commit_manually, select_for_update() and save() ok? How can I write a test that confirms this? I can't find, say, a signal that Django fires between transactions; and I can't just run it and hope concurrency issues arise and are dealt with.
Why not to use commit_on_success there?
I think the query itself should look like:
Something.objects.select_for_update().filter(...)
I don't think django does anything special on select_for_update what you could assert on. Only assertion comes to my head is assertTrue(queryset.query.select_for_update). It tests nothing and might be usefull only if somebody accidentaly(?) removes the call.
Even if you come up with some unit test for this run condition problem I don't think it would be wise to put that into the project.
Rather focus on testing your code, not the db behaviour.
How can I save a model, such that signals arent sent. (post_save and pre_save)
It's a bit of a hack, but you can do something like this:
use a unique identifier with a filter and then use the update method of the queryset (which does not trigger the signals)
user_id = 142187
User.objects.filter(id=user_id).update(name='tom')
ModelName.objects.bulk_create([your object/objects])
also you can read more here django docs
This ticket has been marked as "wontfix" because:
In short, it sounds like, given the defined purpose of signals, it is
the attached signal handler that needs to become more intelligent
(like in davedash's suggestion), rather than the code that emits the
signal. Disabling signals is just a quick fix that will work when you
know exactly what handlers are attached to a signal, and it hides the
underlying problem by putting the fix in the wrong place.
There is currently a ticket pending a Django design decision for this feature.
Included in the ticket is a diff for a patch with the proposed implementation.
If you have mutual relations on models and their signals
still you can decouple signal's logic to have more signals of same type, and handle your logic in more sophisticated way:
You can check in signals, the state of object:
kwargs['created']
You can check the state of any pasted extra value:
So in one signal, you will read at first:
if `kwargs['instance'].skip_signals`:
return
and in another place, before save() you will just set that skip_signals on the specific object, in specific situation.
(there is no need to define it as model field)
You can also do not emit signals:
by overriding method(s) on models,
or by adding own save_without_signals(),
or just as mentioned already, doing filter(pk=<>).update(...)
As far as I'm aware there still isn't a 'nice' way to do this, but if you're open to exploring hacky solutions, then I'll add one to the mix.
If you look at the django model source code, specifically save_base() here and here, you'll see that the pre_save() and post_save() signals are both wrapped in a conditional:
if not meta.auto_created:
// Emit signal
We can directly manipulate the meta options of a model or instance through the _meta API which means we're able to 'disable' the signals from firing by setting auto_created = True on the instance we want to save.
For example:
#receiver(post_save, sender=(MyModel))
def save_my_model(sender, instance=None, created=False, **kwargs):
if created:
# Modify the instance
instance.task_id = task.task_hash
# HACK: Prevent `post_save` signal from being called during save
instance._meta.auto_created = True
instance.save()
instance._meta.auto_created = False
elif instance.has_changed("schedule"):
# Modify the instance
instance.task_id = 'abc123'
# HACK: Prevent `post_save` signal from being called during save
instance._meta.auto_created = True
instance.save()
instance._meta.auto_created = False
The major caveat here is that this is undocumented behaviour and it could well change it future releases of django.