I have a GAE model with several methods that update entities with transactions. Something like this:
class MyModel(ndb.Model):
#staticmethod
#ndb.transactional
def update_foo(ekey):
entity = ekey.get()
entity.foo = "x"
entity.put()
#staticmethod
#ndb.transactional
def update_bar(ekey):
entity = ekey.get()
entity.bar = "y"
entity.put()
To clean up my code, I was thinking, I could centralize the code that does updates with transactions. Something like this:
class MyModel(ndb.Model):
#staticmethod
#ndb.transactional
def update_tx(ekey, **kwargs):
entity = ekey.get()
for prop, value in kwargs.iteritems():
setattr(entity, prop, value)
enitty.put()
def update_foo(self):
self.update_tx(self.key, foo="x")
def update_bar(self):
self.update_tx(self.key, bar="y")
Is this a reasonable idea, or are there dangers in this approach that I haven't considered?
It's reasonable. It really depends on your use case.
I actually had something similar in place, but more and more I have to special case my update_tx() for each entity type to the point that only 1 or 2 of my ~10 models still use it, because there is almost always other logic that needs to be executed.
If I update a user's account state from 'deactivated' to 'activated' I need to perform a few queries to see if they completed all the mandatory onboarding steps.
If a user updates their email I have to send a new verification link to that new email before it can change
Certain fields are only editable if the entity is in a certain state
If a user submit's a new start date and end date, I need to save the old start date and end date in new object for historical purposes
etc
That logic could happen outside of this function, but I often want that to happen in the same transaction.
That aside, one thing it doesn't seem to handle is needing to update a batch of entities in a single transaction, so your pseudo code would need to look more like this if that's important to you:
class MyModel(ndb.Model):
def update_properties(self, **kwargs):
for prop, value in kwargs.iteritems():
setattr(self, prop, value)
#staticmethod
#ndb.transactional(xg=True)
def update_txs(keys, updates):
# keys is an array of ndb.Key()s,
# updates is an array of the same size, containing **kwargs for each key
entities = ndb.get_multi(keys)
for index, entity in enumerate(entities):
entity.update_properties(**updates[index])
ndb.put_multi(entities)
#classmethod
def update_tx(cls, ekey, **kwargs):
cls.update_txs([ekey], [kwargs])
def update_foo(self):
self.update_tx(self.key, foo="x")
def update_bar(self):
self.update_tx(self.key, bar="y")
Related
Let's imagine a simple Food model with a name and an expiration date, my goal is to auto delete the object after the expiration date is reached.
I want to delete objects from the database (postgresql in my case) just after exp_date is reached, not filter by exp_date__gt=datetime.datetime.now() in my code then cron/celery once a while a script that filter by exp_date__lt=datetime.datetime.now() and then delete
Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField()
*I could do it with a vanilla view when the object is accessed via an endpoint or even with the DRF like so :
class GetFood(APIView):
def check_date(self, food):
"""
checking expiration date
"""
if food.exp_date <= datetime.datetime.now():
food.delete()
return False
def get(self, request, *args, **kwargs):
id = self.kwargs["id"]
if Food.objects.filter(pk=id).exists():
food = Food.objects.get(pk=id)
if self.check_date(food) == False:
return Response({"error": "not found"}, status.HTTP_404_NOT_FOUND)
else:
name = food.name
return Response({"food":name}, status.HTTP_200_OK)
else:
return Response({"error":"not found"},status.HTTP_404_NOT_FOUND)
but it would not delete the object if no one try to access it via an endpoint.
*I could also set cronjob with a script that query the database for every Food object which has an expiration date smaller than today and then delete themor even setup Celery. It would indeed just need to run once a day if I was using DateField but as I am using DateTimeField it would need to run every minute (every second for the need of ny project).
*I've also thought of a fancy workaround with a post_save signal with a while loop like :
#receiver(post_save, sender=Food)
def delete_after_exp_date(sender, instance, created, **kwargs):
if created:
while instance.exp_date > datetime.datetime.now():
pass
else:
instance.delete()
I don't know if it'd work but it seems very inefficient (if someone could please confirm)
Voila, thanks in advance if you know some ways or some tools to achieve what I want to do, thanks for reading !
I would advice not to delete the objects, or at least not effectively. Sceduling tasks is cumbersome. Even if you manage to schedule this, the time when you remove the items will always be slighlty off the time when you scheduled this from happening. It also means you will make an extra query per element, and not remove the items in bulk. Furthermore scheduling is inherently more complicated: it means you need something to persist the schedule. If later the expiration date of some food is changed, it will require extra logic to "cancel" the current schedule and create a new one. It also makes the system less "reliable": besides the webserver, the scheduler daemon has to run. It can happen that for some reason the daemon fails, and then you will no longer retrieve food that is not expired.
Therefore it might be better to combine filtering the records such that you only retrieve food that did not expire, and remove at some regular interval Food that has expired. You can easily filter the objects with:
from django.db.models.functions import Now
Food.objects.filter(exp_date__gt=Now())
to retrieve Food that is not expired. To make it more efficient, you can add a database index on the exp_date field:
Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField(db_index=True)
If you need to filter often, you can even work with a Manager [Django-doc]:
from django.db.models.functions import Now
class FoodManager(models.Manager):
def get_queryset(*args, **kwargs):
return super().get_queryset(*args, **kwargs).filter(
exp_date__gt=Now()
)
class Food(models.Model):
name = models.CharField(max_length=200)
exp_date = models.DateTimeField(db_index=True)
objects = FoodManager()
Now if you work with Food.objects you automatically filter out all Food that is expired.
Besides that you can make a script that for example runs daily to remove the Food objects that have expired:
from django.db.models import Now
Food._base_manager.filter(exp_date__lte=Now()).delete()
Update to the accepted answer. You may run into Super(): No Arguments if you define the method outside the class. I found this answer helpful.
As Per PEP 3135, which introduced "new super":
The new syntax:
super()
is equivalent to:
super(__class__, <firstarg>)
where class is the class that the method
was defined in, and is the first
parameter of the method (normally self for
instance methods, and cls for class methods).
While super is not a reserved word, the parser recognizes the use of super in a method definition and only passes in the class cell when this is found. Thus, calling a global alias of super without arguments will not necessarily work.
As such, you will still need to include self:
class FoodManager(models.Manager):
def get_queryset(self, *args, **kwargs):
return super().get_queryset(*args, **kwargs).filter(
exp_date__gt=Now()
)
Just something to keep in mind.
Say I have an object, "Order," a field of which, "items," holds a list of order items. The list of items will never be searched or individually selected in the database so I just want to store it in a DB field as a JSON string.
I'm trying to figure out the best way to embed this functionality so it's fairly transparent to anyone using the model. I think saving the model is pretty easy - just override the save method and serialize the "items" list into an internal "_items" field, and then write that to the db. I'm confused about how to deserialize, though. Having looked into possibly some kind of classmethod for creation, or creating a custom manger, or something to do with signals, I've thoroughly confused myself. I'm sure this has been solved a hundred times over and I'm curious what people consider to be best practice.
Example classes:
class OrderItem():
def __init__(self, desc="", qty=0):
self.desc = desc
self.qty = qty
class Order(Model):
user = ForeignKey(User)
_items = TextField()
def save(self, *args, **kwargs):
self._items = jsonpickle.encode(self.items)
super(Order, self).save(*args, **kwargs)
Example usage:
order = Order()
order.items = [OrderItem("widget", 5)]
order.save()
This would create a record in the DB in which
_items = [{"desc":"widget", "qty":5}]
Now I want to be able to later select the object
order = Order.objects.get(id=whatever)
and have order.items be the unpacked array of items, not the stored JSON string.
EDIT:
The solution turned out to be quite simple, and I'm posting here in case it helps any other newbies. Based on Daniel's suggestion, I went with this custom model field:
class JSONField(with_metaclass(SubfieldBase, TextField)):
def db_type(self, connection):
return 'JSONField'
def to_python(self, value):
if isinstance(value, basestring):
return jsonpickle.decode(value)
else:
return value
def get_prep_value(self, value):
return jsonpickle.encode(value)
A much better approach is to subclass TextField and override the relevant methods to do the serialization/deserialization transparently as required. In fact there are a number of implementations of this already: here's one, for example.
I have class Event and class Participant, which has foreign key to Event.
In Event I have:
model_changed_stamp = models.DateTimeField()
Many participant are taking part in one even.
When any of the instances of class Event changes, or the new one is being created I would like that the value in model_changed_stamp will be updated. In fact I have many other classes like Building, which also have foreign key to Event, and I would like to also keep track of changes.
I came up with idea to use instance class method in Event. I tried:
def model_changed(self):
value = getattr(self, 'model_changed_stamp')
value = datetime.now()
setattr(self, 'model_changed_stamp', value)
and then in save() of Participant, or Building I would like to fire self.event.model_changed()
I would like know how to do it RIGHT. Should I use signals?
UPDATE 0:
According to some reading (e.g Two scoops of Django) use of signals is an overkill for this case.
UPDATE 1: Following suggestions of Daniel Roseman in Participant class in save(self) method I try:
def save(self):
if self.id is None:
self.event.model_changed()
In Event I defined model_changed as follows:
def model_changed(self):
self.model_changed_stamp = datetime.now()
self.save()
And it is not working - not updating the date, when it should i.e when the new Participant is created.
UPDATE 2: WORKING!!! ;-)
after adding: self.save() as last line in model_changed method.
Why not just set it directly? Why all this mucking about with getattr and setattr?
def model_changed(self):
self.model_changed_stamp = datetime.datetime.now()
An even better solution is to define the fields with auto_now=True, so they will be automatically updated with the current time whenever you save.
Yeah, signals is good tool for your task.
You can wite:
model_changed_stamp = models.DateTimeField(auto_now=True)
check docs for auto_now feature.
and then:
from django.db.models.signals import post_save
#receiver(post_save)
def post_save_event_handler(sender, instance, **kwargs):
if not sender in [Building, Participant, ...]:
return
instance.event.save() #updates model_changed_stamp
I wanted to write some code like this:
class SomeModel(models.Model):
field = models.ForeignKey(SomeOtherModel)
def __init__(self, *args, **kwargs):
super(SomeModel, self).__init__(*args, **kwargs)
if self.field is None:
self.field = SomeOtherModel()
...
However this raises self.field.rel.to.DoesNotExist. The Django code is very clear on that:
class ReverseSingleRelatedObjectDescriptor(object):
def __get__(self, instance, instance_type=None):
...
if val is None:
# If NULL is an allowed value, return it.
if self.field.null:
return None
raise self.field.rel.to.DoesNotExist
An obvious workaround would be of course to make the field nullable however as far as I understand that would actually have an effect on the database schema, also I like the integrity checks Django offers. Another one would be to catch the exception and handle it appropriately. However this adds a lot of boilerplate code. Especially when there are multiple fields like that (a separate try...except block for each one of them - now that's ugly). What could I do?
I could use initial however this is quite limited when it comes to foreign keys. I do not always know the kind of default that I would like to have at the moment of creation. I will however know it at the initialization phase. Moreover it could then be dependent on the values of the other fields.
Check if it has the attribute set -
if hasattr(self, 'field')
proper way to refer a field in form is like this:
self.fields['myfield']
so, in your case, the null check should go like this
self.fields['myfield'].value is None
on the other note, don't use reserved/near to reserved words like 'field' for naming your fields.
As you may be able to tell from my questions, I'm new to both python and django. I would like to allow dynamic filter specifications of query sets from my templates using **kwargs. I'm thinking like a select box of a bunch of kwargs. For example:
<select id="filter">
<option value="physician__isnull=True">Unassigned patients</option>
</select>
Does django provide an elegant solution to this problem that I haven't come across yet?
I'm trying to solve this in a generic manner since I need to pass this filter to other views. For example, I need to pass a filter to a paginated patient list view, so the pagination knows what items it's working with. Another example is this filter would have to be passed to a patient detail page so you can iterate through the filtered list of patients with prev/next links.
Thanks a bunch, Pete
Update:
What I came up with was building a FilterSpecification class:
class FilterSpec(object):
def __init__(self, name, *args):
super(FilterSpec, self).__init__()
self.name = name
self.filters = []
for filter in args:
self.add(filter)
def pickle(self):
return encrypt(pickle.dumps(self))
def add(self, f):
self.filters.append(f)
def kwargs(self):
kwargs = {}
for f in self.filters:
kwargs = f.kwarg(**kwargs)
return kwargs
def __unicode__(self):
return self.name
class Filter(object):
def __init__(self, key, value):
super(Filter, self).__init__()
self.filter_key = key
self.filter_value = value
def kwarg(self, **kwargs):
if self.filter_key != None:
kwargs[self.filter_key] = self.filter_value
return kwargs
I then can filter any type of model like this:
filterSpec = FilterSpec('Assigned', Filter('service__isnull', False)))
patients = Patient.objects.filter(**filterSpec.kwargs())
I pass these filterSpec objects from the client to server by serializing, compressing, applying some symmetric encryption, and url-safe base-64 encoding. The only downside is that you end up with URLs looking like this:
http://127.0.0.1:8000/hospitalists/assign_test/?filter=eJwBHQHi_iDiTrccFpHA4It7zvtNIW5nUdRAxdiT-cZStYhy0PHezZH2Q7zmJB-NGAdYY4Q60Tr_gT_Jjy_bXfB6iR8inrNOVkXKVvLz3SCVrCktGc4thePSNAKoBtJHkcuoaf9YJA5q9f_1i6uh45-6k7ZyXntRu5CVEsm0n1u5T1vdMwMnaNA8QzYk4ecsxJRSy6SMbUHIGhDiwHHj1UnQaOWtCSJEt2zVxaurMuCRFT2bOKlj5nHfXCBTUCh4u3aqZZjmSd2CGMXZ8Pn3QGBppWhZQZFztP_1qKJaqSVeTNnDWpehbMvqabpivtnFTxwszJQw9BMcCBNTpvJf3jUGarw_dJ89VX12LuxALsketkPbYhXzXNxTK1PiZBYqGfBbioaYkjo%3D
I would love to get some comments on this approach and hear other solutions.
Rather than face the horrible dangers of SQL injection, why not just assign a value to each select option and have your form-handling view run the selected query based on the value.
Passing the parameters for a DB query from page to view is just asking for disaster. Django is built to avoid this sort of thing.
Concerning your update: FilterSpecs are unfortunately one of those (rare) pieces of Django that lack public documentation. As such, there is no guarantee that they will keep working as they do.
Another approach would be to use Alex Gaynor's django-filter which look really well thought out. I'll be using them for my next project.