When using a Django signal like post_save you can prevent it from firing when an object is first created by doing something like:
#receiver(post_save,sender=MyModel)
def my_signal(sender, instance, created,**kwargs):
if not created:
pass # Do nothing, as the item is new.
else:
logger.INFO("The item changed - %s"%(instance) )
However, ManyToMany relations are applied after an item is initially created, so no such argument is passed in, making it difficult to suppress in these cases.
#receiver(m2m_changed,sender=MyModel.somerelation.though)
def my_signal(sender, instance, created,**kwargs):
if __something__: # What goes here?
pass # Do nothing, as the item is new.
else:
logger.INFO("The item changed - %s"%(instance) )
Is there an easy way to suppress an m2m_changed signal when its being done on an object that has just been created?
I think there is no easy way to do that.
As the Django doc says, you can't associate an item with a relation until it's been saved. Example from the doc:
>>> a1 = Article(headline='...')
>>> a1.publications.add(p1)
Traceback (most recent call last):
...
ValueError: 'Article' instance needs to have a primary key value before a many-to-many relationship can be used.
# should save Article first
>>> a1.save()
# the below statement never know it's just following a creation or not
>>> a1.publications.add(p1)
It's logically not possible for a relation record to know whether it is added to "a just created item" or "an item that already exists for some time", without external info.
Some workarounds I came up with:
Solution 1. add a DatetimeField in MyModel to indicate creation time. m2m_changed handler uses the creation time to check when is the item created. It work practically in some cases, but cannot guarantee correctness
Solution 2. add a 'created' attribute in MyModel, either in a post_save handler or in other codes. Example:
#receiver(post_save, sender=Pizza)
def pizza_listener(sender, instance, created, **kwargs):
instance.created = created
#receiver(m2m_changed, sender=Pizza.toppings.through)
def topping_listener(sender, instance, action, **kwargs):
if action != 'post_add':
# as example, only handle post_add action here
return
if getattr(instance, 'created', False):
print 'toppings added to freshly created Pizza'
else:
print 'toppings added to modified Pizza'
instance.created = False
Demo:
p1 = Pizza.objects.create(name='Pizza1')
p1.toppings.add(Topping.objects.create())
>>> toppings added to freshly created Pizza
p1.toppings.add(Topping.objects.create())
>>> toppings added to modified Pizza
p2 = Pizza.objects.create(name='Pizza2')
p2.name = 'Pizza2-1'
p2.save()
p2.toppings.add(Topping.objects.create())
>>> toppings added to modified Pizza
But be careful using this solution. Since 'created' attribute was assigned to Python instance, not saved in DB, things can go wrong as:
p3 = Pizza.objects.create(name='Pizza3')
p3_1 = Pizza.objects.get(name='Pizza3')
p3_1.toppings.add(Topping.objects.create())
>>> toppings added to modified Pizza
p3.toppings.add(Topping.objects.create())
>>> toppings added to freshly created Pizza
That's all about the answer. Then, caught you here! I'm zhang-z from github django-notifications group :)
#ZZY's answer basically helped me realise that this wasn't possible without storing additional fields. Fortunately, I'm using django-model-utils which includes a TimeStampedModel which includes a created field.
Providing a small enough delta, it was relatively easy to check against the created time when catching the signal.
#receiver(m2m_changed,sender=MyModel.somerelation.though)
def my_signal(sender, instance, created,**kwargs):
if action in ['post_add','post_remove','post_clear']:
created = instance.created >= timezone.now() - datetime.timedelta(seconds=5)
if created:
logger.INFO("The item changed - %s"%(instance) )
For an easier and short way of checking in the object is created or not is using the _state.adding attribute:
def m2m_change_method(sender, **kwargs):
instance = kwargs.pop('instance', None)
if instance:
if instance.adding: #created object
pk_set = list(kwargs.pop('pk_set')) #ids of object added to m2m relation
else:
# do something if the instance not newly created or changed
# if you want to check if the m2m objects is new use pk_set query if exists()
m2m_change.connect(m2m_change_method, sender=YourModel.many_to_many_field.through)
Related
Let's imagine a simple Food model with a name and an expiration date, my goal is to auto delete the object after the expiration date is reached.
I want to delete objects from the database (postgresql in my case) just after exp_date is reached, not filter by exp_date__gt=datetime.datetime.now() in my code then cron/celery once a while a script that filter by exp_date__lt=datetime.datetime.now() and then delete
Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField()
*I could do it with a vanilla view when the object is accessed via an endpoint or even with the DRF like so :
class GetFood(APIView):
def check_date(self, food):
"""
checking expiration date
"""
if food.exp_date <= datetime.datetime.now():
food.delete()
return False
def get(self, request, *args, **kwargs):
id = self.kwargs["id"]
if Food.objects.filter(pk=id).exists():
food = Food.objects.get(pk=id)
if self.check_date(food) == False:
return Response({"error": "not found"}, status.HTTP_404_NOT_FOUND)
else:
name = food.name
return Response({"food":name}, status.HTTP_200_OK)
else:
return Response({"error":"not found"},status.HTTP_404_NOT_FOUND)
but it would not delete the object if no one try to access it via an endpoint.
*I could also set cronjob with a script that query the database for every Food object which has an expiration date smaller than today and then delete themor even setup Celery. It would indeed just need to run once a day if I was using DateField but as I am using DateTimeField it would need to run every minute (every second for the need of ny project).
*I've also thought of a fancy workaround with a post_save signal with a while loop like :
#receiver(post_save, sender=Food)
def delete_after_exp_date(sender, instance, created, **kwargs):
if created:
while instance.exp_date > datetime.datetime.now():
pass
else:
instance.delete()
I don't know if it'd work but it seems very inefficient (if someone could please confirm)
Voila, thanks in advance if you know some ways or some tools to achieve what I want to do, thanks for reading !
I would advice not to delete the objects, or at least not effectively. Sceduling tasks is cumbersome. Even if you manage to schedule this, the time when you remove the items will always be slighlty off the time when you scheduled this from happening. It also means you will make an extra query per element, and not remove the items in bulk. Furthermore scheduling is inherently more complicated: it means you need something to persist the schedule. If later the expiration date of some food is changed, it will require extra logic to "cancel" the current schedule and create a new one. It also makes the system less "reliable": besides the webserver, the scheduler daemon has to run. It can happen that for some reason the daemon fails, and then you will no longer retrieve food that is not expired.
Therefore it might be better to combine filtering the records such that you only retrieve food that did not expire, and remove at some regular interval Food that has expired. You can easily filter the objects with:
from django.db.models.functions import Now
Food.objects.filter(exp_date__gt=Now())
to retrieve Food that is not expired. To make it more efficient, you can add a database index on the exp_date field:
Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField(db_index=True)
If you need to filter often, you can even work with a Manager [Django-doc]:
from django.db.models.functions import Now
class FoodManager(models.Manager):
def get_queryset(*args, **kwargs):
return super().get_queryset(*args, **kwargs).filter(
exp_date__gt=Now()
)
class Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField(db_index=True)
objects = FoodManager()
Now if you work with Food.objects you automatically filter out all Food that is expired.
Besides that you can make a script that for example runs daily to remove the Food objects that have expired:
from django.db.models import Now
Food._base_manager.filter(exp_date__lte=Now()).delete()
Update to the accepted answer. You may run into Super(): No Arguments if you define the method outside the class. I found this answer helpful.
As Per PEP 3135, which introduced "new super":
The new syntax:
super()
is equivalent to:
super(__class__, <firstarg>)
where class is the class that the method
was defined in, and is the first
parameter of the method (normally self for
instance methods, and cls for class methods).
While super is not a reserved word, the parser recognizes the use of super in a method definition and only passes in the class cell when this is found. Thus, calling a global alias of super without arguments will not necessarily work.
As such, you will still need to include self:
class FoodManager(models.Manager):
def get_queryset(self, *args, **kwargs):
return super().get_queryset(*args, **kwargs).filter(
exp_date__gt=Now()
)
Just something to keep in mind.
I have a foreign key relationship in my Django (v3) models:
class Example(models.Model):
title = models.CharField(max_length=200) # this is irrelevant for the question here
not_before = models.DateTimeField(auto_now_add=True)
...
class ExampleItem(models.Model):
myParent = models.ForeignKey(Example, on_delete=models.CASCADE)
execution_date = models.DateTimeField(auto_now_add=True)
....
Can I have code running/triggered whenever an ExampleItem is "added to the list of items in an Example instance"? What I would like to do is run some checks and, depending on the concrete Example instance possibly alter the ExampleItem before saving it.
To illustrate
Let's say the Example's class not_before date dictates that the ExampleItem's execution_date must not be before not_before I would like to check if the "to be saved" ExampleItem's execution_date violates this condition. If so, I would want to either change the execution_date to make it "valid" or throw an exception (whichever is easier). The same is true for a duplicate execution_date (i.e. if the respective Example already has an ExampleItem with the same execution_date).
So, in a view, I have code like the following:
def doit(request, example_id):
# get the relevant `Example` object
example = get_object_or_404(Example, pk=example_id)
# create a new `ExampleItem`
itm = ExampleItem()
# set the item's parent
itm.myParent = example # <- this should trigger my validation code!
itm.save() # <- (or this???)
The thing is, this view is not the only way to create new ExampleItems; I also have an API for example that can do the same (let alone that a user could potentially "add ExampleItems manually via REPL). Preferably the validation code must not be duplicated in all the places where new ExampleItems can be created.
I was looking into Signals (Django docu), specifically pre_save and post_save (of ExampleItem) but I think pre_save is too early while post_save is too late... Also m2m_changed looks interesting, but I do not have a many-to-many relationship.
What would be the best/correct way to handle these requirements? They seem to be rather common, I imagine. Do I have to restructure my model?
The obvious solution here is to put this code in the ExampleItem.save() method - just beware that Model.save() is not invoked by some queryset bulk operations.
Using signals handlers on your own app's models is actually an antipattern - the goal of signal is to allow for your app to hook into other app's lifecycle without having to change those other apps code.
Also (unrelated but), you can populate your newly created models instances directly via their initializers ie:
itm = ExampleItem(myParent=example)
itm.save()
and you can even save them directly:
# creates a new instance, populate it AND save it
itm = ExampleItem.objects.create(myParent=example)
This will still invoke your model's save method so it's safe for your use case.
got a little problem here using the Django-framework and its
.delete() method.
Im using the following model
class Message(models.Model):
...
sender = models.ForeignKey(User, related_name="%(class)s_sender")
user = models.ManyToManyField(User, related_name="%(class)s_recip")
trash = models.ManyToManyField(User, related_name="%(class)s_trash", blank=True)
...
So my aim is to check if both M2M fields "user" and "trash" do not contain objects/keys anymore, and if so, the corresponding Message instance should be deleted from the database.
To archieve this I'm using m2m_changed signals as follows
def set_recip_names(sender, instance, action,**kwargs):
print instance
if action == "post_remove":
if instance.user.all().count() == 0 and instance.trash.all().count() == 0:
print instance.id
instance.delete()
def msg_del(sender, **kwargs):
print "Message deleted"
m2m_changed.connect(set_recip_names, sender=Message.user.through)
m2m_changed.connect(set_recip_names, sender=Message.trash.through)
post_delete.connect(msg_del, sender=Message)
The first two print statements are for debug purpous of course, and both print exactly what Im expecting: instance IS the corresponding instance to be deleted, and instance.id confirms this, as well as telling me that Django actually will execute the .delete() method.
The second post_delete signal is for debugging purpous too, and it is called as expected, so Im getting the output "Message deleted".
So the big problem is, that the "to-be-deleted" instance remains in the database. Ive searched the internet for an hour now but I dont see anything is wrong with my code.
Any suggestions are welcomed, or if you know any better way to do what Im trying to do, please let me know.
Solved
Problem was actually pretty obvious. One of my code snippets, where on user interaction elements from my M2M fields are removed looks like
...
try:
message = Message.objects.get(id=msg)
if usr not in message.trash.all():
continue
if usr != message.sender:
message.user.remove(usr) # m2m_changed triggered
message.trash.remove(usr) # m2m_changed triggered
# if instance was removed by now, this .save() will
# insert the python-local object in the db, so it looked
# like the same instance is still there, but I saw that
# the "old" instance suddenly had a new id
message.save()
except Message.DoesNotExist:
continue
...
You can delete the record directly by doing this:
YourModel.objects.get(id=instance.id).delete()
If the record exists, then it will be deleted in the database.
Your instance is saved in the RAM and may not be deleted, you can do that manually by deleting it from the RAM: del instance
I have a function which currently calls Models.object.get(), which returns either 0 or 1 model objects:
if it returns 0, I create a new model instance in the except DoesNotExist clause of the function.
Otherwise, I would like to update the fields in the pre-existing
instance, without creating a new one.
I was originally attempting to
call .update() on the instance which was found, but .update()
seems to be only callable on a QuerySets. How do I get around
changing a dozen fields, without calling .filter() and comparing
the lengths to know if I have to create or update a pre-existing
instance?
With the advent of Django 1.7, there is now a new update_or_create QuerySet method, which should do exactly what you want. Just be careful of potential race conditions if uniqueness is not enforced at the database level.
Example from the documentation:
obj, created = Person.objects.update_or_create(
first_name='John', last_name='Lennon',
defaults={'first_name': 'Bob'},
)
The update_or_create method tries to fetch an object from database
based on the given kwargs. If a match is found, it updates the
fields passed in the defaults dictionary.
Pre-Django 1.7:
Change the model field values as appropriate, then call .save() to persist the changes:
try:
obj = Model.objects.get(field=value)
obj.field = new_value
obj.save()
except Model.DoesNotExist:
obj = Model.objects.create(field=new_value)
# do something else with obj if need be
if you want only to update model if exist (without create it):
Model.objects.filter(id = 223).update(field1 = 2)
mysql query:
UPDATE `model` SET `field1` = 2 WHERE `model`.`id` = 223
As of Django 1.5, there is an update_fields property on model save. eg:
obj.save(update_fields=['field1', 'field2', ...])
https://docs.djangoproject.com/en/dev/ref/models/instances/
I prefer this approach because it doesn't create an atomicity problem if you have multiple web app instances changing different parts of a model instance.
I don't know how good or bad this is, but you can try something like this:
try:
obj = Model.objects.get(id=some_id)
except Model.DoesNotExist:
obj = Model.objects.create()
obj.__dict__.update(your_fields_dict)
obj.save()
Here's a mixin that you can mix into any model class which gives each instance an update method:
class UpdateMixin(object):
def update(self, **kwargs):
if self._state.adding:
raise self.DoesNotExist
for field, value in kwargs.items():
setattr(self, field, value)
self.save(update_fields=kwargs.keys())
The self._state.adding check checks to see if the model is saved to the database, and if not, raises an error.
(Note: This update method is for when you want to update a model and you know the instance is already saved to the database, directly answering the original question. The built-in update_or_create method featured in Platinum Azure's answer already covers the other use-case.)
You would use it like this (after mixing this into your user model):
user = request.user
user.update(favorite_food="ramen")
Besides having a nicer API, another advantage to this approach is that it calls the pre_save and post_save hooks, while still avoiding atomicity issues if another process is updating the same model.
As #Nils mentionned, you can use the update_fields keyword argument of the save() method to manually specify the fields to update.
obj_instance = Model.objects.get(field=value)
obj_instance.field = new_value
obj_instance.field2 = new_value2
obj_instance.save(update_fields=['field', 'field2'])
The update_fields value should be a list of the fields to update as strings.
See https://docs.djangoproject.com/en/2.1/ref/models/instances/#specifying-which-fields-to-save
I am using the following code in such cases:
obj, created = Model.objects.get_or_create(id=some_id)
if not created:
resp= "It was created"
else:
resp= "OK"
obj.save()
update:
1 - individual instance :
get instance and update manually get() retrieve individual object
post = Post.objects.get(id=1)
post.title = "update title"
post.save()
2 - Set of instances :
use update() method that works only with queryset that what would be returned by filter() method
Post.objects.filter(author='ahmed').update(title='updated title for ahmed')
I am in process of migration from db.Model to ndb.Model. The only issue that I have to solve before finish this migration is that there is no Model.is_saved method. I have used db.Model.is_saved in my application to determine if sharded counters must be updated on put/delete, to check for conflicted keys on creating entities etc.
The documentation says that ndb.Model has no equivalent for is_saved method. I can reimplement some use cases with get_or_insert instead of is_saved. But not all of them.
As a dirty hack I can set flag like _in_memory_instance for every instance I have created by calling constructor. But it does not solve my issue. I still have to update this flag at least after every put() call.
The question is: is there better way to determine if model is persistent in the datastore or not without extra datastore hit?
Edit 1: Forgot to mention: all the entities got keys so check for Model._has_complete_key() does not work for me.
Edit 2: After this discussion https://groups.google.com/d/topic/google-appengine/Tm8NDWIvc70/discussion it seems to be the only way to solve my issue is to use _post_get_hook/_post_put_hook. I wondering why such a trivial thing was not included in official API.
Edit 3: I ended up with next base class for all my models. Now I can leave my codebase (almost) untouched:
class BaseModel(ndb.Model):
#classmethod
def _post_get_hook(cls, key, future):
self = future.get_result()
if self:
self._is_saved = bool(key)
def _post_put_hook(self, future):
self._is_saved = future.state == future.FINISHING
def is_saved(self):
if self._has_complete_key():
return getattr(self, "_is_saved", False)
return False
To get the same kind of state in NDB you would need a combination of
post-get-hook and post-put-hook to set a flag. Here's a working
example:
class Employee(ndb.Model):
<properties here>
saved = False # class variable provides default value
#classmethod
def _post_get_hook(cls, key, future):
obj = future.get_result()
if obj is not None:
# test needed because post_get_hook is called even if get() fails!
obj.saved = True
def _post_put_hook(self, future):
self.saved = True
There's no need to check for the status of the future -- when either
hook is called, the future always has a result. This is because the
hook is actually a callback on the future. However there is a need to
check if its result is None!
PS: Inside a transaction, the hooks get called as soon as the put() call returns; success or failure of the transaction doesn't enter affect them. See https://developers.google.com/appengine/docs/python/ndb/contextclass#Context_call_on_commit for a way to run a hook after a successful commit.
Based on #Tim Hoffmans idea you can you a post hook like so:
class Article(ndb.Model):
title = ndb.StringProperty()
is_saved = False
def _post_put_hook(self, f):
if f.state == f.FINISHING:
self.is_saved = True
else:
self.is_saved = False
article = Article()
print article.is_saved ## False
article.put()
print article.is_saved ## True
I can't guarantee that it's persisted in the datastore. Didn't find anything about it on google :)
On a side not, looking to see if a ndb.Model instance has a key won't probably work since a new instance seems to get a Key before it's ever sent to the datastore. You can look at the source code to see what happens when you create an instance of the ndb.Model class.
If you do not mention a key while creating an instance of the Model, you can use the following implementation of is_saved() to know if the object has been written to the datastore or not atleast once. (should be appropriate if you are migrating from google.appengine.ext.db to google.appengine.ext.ndb)
Using the example given by #fredrik,
class Article(ndb.Model):
title = ndb.StringProperty()
def is_saved(self):
if self.key:
return True
return False
P.S. - I do not know if this would work with google.cloud.ndb