I have quite heavy calculation and a lot of database access but reusable in SerializerMethodField of ModelSerializer, how should I result that calculated data across the method.
Example Code
class MySerializer(serializers.ModelSerializer):
a = serializers.SerializerMethodField()
b = serializers.SerializerMethodField()
def get_a(self, obj):
result_x = heavy_calculation(self.context['request'].user, obj)
return result_x + 1
def get_b(self, obj):
result_x = heavy_calculation(self.context['request'].user, obj)
return result_x + 2
heavy_calculation() in both method is the same function so result_x would be the same answer for any requests. I would like to make heavy_calculation() call only once per requests to reduce load work (my real work would like to call 10+ times)
Best Regards,
There're few options for your case:
If you could rewrite your serializer class:
class MySerializer(serializers.ModelSerializer):
a = serializers.IntegerField()
def calc_a(self, obj):
return self._cached_result + 1
def to_representation(self, instance):
data = super().to_representation(instance)
self._cached_result = heavy_calculation(self.context['request'].user, instance)
data['a'] = self.calc_a(instance)
...
return data
Using python3 LRU cache (https://docs.python.org/3/library/functools.html#functools.lru_cache)
#lru_cache
def heavy_calculation(user, obj):
...
There're django-memoize libary that cache your function result (you'll need to setup cache for your Django site)
#memoize(timeout=60)
def heavy_calculation(user, obj):
...
There are several ways to achieve this.
1. You can override init and populate data with object_id and result in a dictionary.
2. You can write list serializer and override data property and populate on your own. Guide of writing list serializer here
Related
I have a serializer for a model with an image field, for which I have saved multiple different sized thumbnail images.
I access them by returning their URL using the SerializerMethodField:
class GalleryImageSerializer(serializers.ModelSerializer):
image_sm = serializers.SerializerMethodField()
image_md = serializers.SerializerMethodField()
image_lg = serializers.SerializerMethodField()
image_compressed = serializers.SerializerMethodField()
def get_image_sm(self, obj):
return default_storage.url(f'{splitext(obj.image.name)[0]}/sm.jpg')
def get_image_md(self, obj):
return default_storage.url(f'{splitext(obj.image.name)[0]}/md.jpg')
def get_image_lg(self, obj):
return default_storage.url(f'{splitext(obj.image.name)[0]}/lg.jpg')
def get_image_compressed(self, obj):
return default_storage.url(f'{splitext(obj.image.name)[0]}/compressed.jpg')
This code works, but it kind of violates the "don't repeat yourself" guideline.
As you can see, these are all duplicate SerializerMethodFields, with the only difference being the filename, eg 'lg.jpg', 'md.jpg', etc.
I'd much prefer to have only one function that I call with an argument for the filename, as an example(pseudocode):
class GalleryImageSerializer(serializers.ModelSerializer):
image_sm = serializers.SerializerMethodField(filename='sm.jpg')
image_md = serializers.SerializerMethodField(filename='md.jpg')
image_lg = serializers.SerializerMethodField(filename='lg.jpg')
image_compressed = serializers.SerializerMethodField(filename='compressed.jpg')
def get_image(self, obj, filename=''):
return default_storage.url(f'{splitext(obj.image.name)[0]}/{filename}')
Currently I am unable to find any way to achieve this. Reading the source code of SerializerMethodField, it doesn't seem to support it.
Is there any way to avoid creating duplicate functions for fields with arbitrary differences?
You can add these fields in the to_representation method.
def to_representation(self, instance):
ret = super().to_representation(instance)
# add img urls to ret dict
for name in ['sm', 'md', 'lg', 'compressed']:
ret['image_' + name] = default_storage.url(f'{splitext(instance.image.name)[0]}/{name}.jpg')
return ret
check the docs for more details:
https://www.django-rest-framework.org/api-guide/serializers/#to_representationself-instance
I've got a many-to-many model like Request ← RequestItem → Item, and I want the response from the Request API endpoint to include a list of Item IDs. I've got a working serializer method like this:
def to_representation(self, instance: Request) -> typing.Dict[str, Any]:
representation: Dict = super().to_representation(instance)
representation["items"] = [
item_id for item_id
in instance.requestitems_set.values_list("item_id", flat=True)
]
return representation
As you can see, this is horrible. What would be an idiomatic way of getting the exact same output?
It can be added quite straightforward like any other field in the serializer
class RequestSerializer(serializers.ModelSerializer):
....
items = serializers.SlugRelatedField(
source='requestitems_set',
slug_field='item_id',
read_only=True,
many=True,
)
And then of course add it to the list of fields
There are few related fields implemented by DRF. But API is open for you to implement your own one. I think this is more readable and clean solution.
class ItemIdRelatedField(serializers.RelatedField):
def to_internal_value(self, data):
pass
# implement if you need it.
def to_representation(self, value):
return value.item_id
And use it as field in serializer like this.
items = ItemIdRelatedField(many=True, source='requestitems_set', queryset=RequestItem.objects.all())
from given info, you could reduce the use of for loop as
def to_representation(self, instance: Request) -> typing.Dict[str, Any]:
representation: Dict = super().to_representation(instance)
representation["items"] = list(instance.requestitems_set.values_list("item_id", flat=True))
return representation
When working with Django model forms, I often do something like this:
def my_view(request):
new_entry = MyModel(name='a')
form = MyModelForm(instance=new_entry)
...
I want to do something similar with a modelformset. Something like this would be ideal:
def my_view(request):
MyFormSet = modelformset_factory(MyModel, form=MyModelForm)
new_entries = [MyModel(name='a'), MyModel(name='b')]
formset = MyFormSet(instances=new_entries) # off course this does not work
...
Since the items are not stored in the database yet, I can't set the instances using a queryset. If I want to use initial I have to declare the fields in the form, which is not ideal and seems a bit hacky.
Any suggestions how I can set the instances of each modelform in a modelformset?
Ok, I think I've found a solution.
class FormSetWithInstances(BaseFormSet):
def __init__(self, *args, **kwargs):
self.instances = kwargs.pop('instances')
super(FormSetWithInstances, self).__init__(*args, **kwargs)
def get_form_kwargs(self, index):
form_kwargs = super(FormSetWithInstances, self).get_form_kwargs(index)
if index < len(self.instances):
form_kwargs['instance'] = self.instances[index]
return form_kwargs
Be careful when using this modelformsets or inlinemodelformsets, as the queryset will override the instance you set.
An alternative approach:
class FormSetWithInstances(BaseFormSet):
def get_form_kwargs(self, index):
kwargs = super(FormSetWithInstances, self).get_form_kwargs(index)
instances = kwargs.pop('instances')
try:
kwargs.update({'instance': instances[index]})
except IndexError:
pass
return kwargs
Then, when creating instances of FormSetWithInstances, you pass in a list of instances as a form kwarg:
form_set = FormSetWithInstances(form_kwargs={'instances': [...]})
I personally prefer this method because it takes advantage of existing class infrastructure instead of defining custom class members in an overridden __init__(). Also, it's in the docs.
I'm not aware of an easy way to pass a list of instances as you are trying to do. Here's a couple of options that might work depending on your use case.
You can provide initial data for the model formset. This should be a list of dictionaries, not model instances:
initial = [{'name': 'a'}, {'name': 'b'}]
formset = MyFormSet(
queryset=MyModel.objects.none(),
initial=initial,
)
Note that I have set the queryset to an empty queryset. If you didn't do this, then the formset would display existing instances, and the initial data would be used for new instances.
If you have initial values for fields that you do not wish to include in the form, then you could be able to set those values when you [save the formset].
instances = formset.save(commit=False)
names = ['a', 'b']
for instance, name in zip(instances, names):
instance.name = name
instance.save()
So I have a serializer that looks like this
class BuildingsSerializer(serializers.ModelSerializer):
masterlisting_set = serializers.PrimaryKeyRelatedField(many=True,
queryset=Masterlistings.objects.all())
and it works great
serializer = BuildingsSerializer(Buildings.objects.get(pk=1))
serializer.data
produces
OrderedDict([
("masterlistings_set", [
"0a06e3d7-87b7-4526-a877-c10f54fa5bc9",
"343643ac-681f-4597-b8f5-ff7e5be65eef",
"449a3ad2-c76c-4cb8-bb86-1be72fafcf64",
])
])
but if I change the queryset in the serializer to
class BuildingsSerializer(serializers.ModelSerializer):
masterlistings_set = serializers.PrimaryKeyRelatedField(many=True, queryset=[])
I still get the same exact result back.
OrderedDict([
("masterlistings_set", [
"0a06e3d7-87b7-4526-a877-c10f54fa5bc9",
"343643ac-681f-4597-b8f5-ff7e5be65eef",
"449a3ad2-c76c-4cb8-bb86-1be72fafcf64",
])
])
Is this supposed to be happening? Am I using querysets incorrectly?
I used [] as an easy example to show that no matter what I put in nothing changes.
Please any insight would be invaluable
It should be noted that masterlistings has a primary key relationship that points to buildings. So a masterlisting belong to a building.
As pointed out by #zymud, queryset argument in PrimaryKeyRelatedField is used for validating field input for creating new entries.
Another solution for filtering out masterlistings_set is to use serializers.SerializerMethodField() as follows:
class BuildingsSerializer(serializers.ModelSerializer):
masterlisting_set = serializers.SerializerMethodField()
def get_masterlisting_set(self, obj):
return MasterListing.objects.filter(building=obj).values_list('pk',flat=True)
queryset in related field limits only acceptable values. So with queryset=[] you will not be able to add new values to masterlisting_set or create new Buildings.
UPDATE. How to use queryset for filtering
This is a little bi tricky - you need to rewrite ManyRelatedField and many_init method in your RelatedField.
# re-define ManyRelatedField `to_representation` method to filter values
# based on queryset
class FilteredManyRelatedField(serializers.ManyRelatedField):
def to_representation(self, iterable):
iterable = self.child_relation.queryset.filter(
pk__in=[value.pk for value in iterable])
return super(FilteredManyRelatedField, self).to_representation(iterable)
# use overridden FilteredManyRelatedField in `many_init`
class FilteredPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
#classmethod
def many_init(cls, *args, **kwargs):
kwargs['child_relation'] = cls(queryset=kwargs.pop('queryset'))
return FilteredManyRelatedField(*args, **kwargs)
I have a problem with serialization of Django inherited models. For example
class Animal(models.Model):
color = models.CharField(max_length=50)
class Dog(Animal):
name = models.CharField(max_length=50)
...
# now I want to serialize Dog model with Animal inherited fields obviously included
print serializers.serialize('xml', Dog.objects.all())
and only Dog model has been serialized.
I can do smth like
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
But it looks ugly and because my models are very big so I have to use SAX parser and with such output it's difficult to parse.
Any idea how to serialize django models with parent class?
**EDIT: ** It use to work ok before this patch has been applied. And the explanation why the patch exist "Model saving was too aggressive about creating new parent class instances during deserialization. Raw save on a model now skips saving of the parent class. " I think there should be an option to be able to serialize "local fields only" by default and second option - "all" - to serialize all inherited fields.
You found your answer in the documentation of the patch.
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
However, if you change Animal to be an abstract base class it will work:
class Animal(models.Model):
color = models.CharField(max_length=50)
class Meta:
abstract = True
class Dog(Animal):
name = models.CharField(max_length=50)
This works as of Django 1.0. See http://docs.djangoproject.com/en/dev/topics/db/models/.
You'll need a custom serializer to support inherited fields, as Django's serializer will only serialize local fields.
I ended up writing my own when dealing with this issue, feel free to copy it: https://github.com/zmathew/django-backbone/blob/master/backbone/serializers.py
In order to use it on its own, you need to do:
serializer = AllFieldsSerializer()
serializer.serialize(queryset, fields=fields)
print serializer.getvalue()
I had the same problem, and i wrote a 'small' queryset serializer which navigates up the inheritance tree and returns all the fields serialized.
It's far from perfect... but works for me :)
a = QuerySetSerializer(MyModel, myqueryset)
a.serialize()
And the snippet:
from __future__ import unicode_literals
import json
import inspect
from django.core import serializers
from django.db.models.base import Model as DjangoBaseModel
class QuerySetSerializer(object):
def __init__(self, model, initial_queryset):
"""
#param model: The model of your queryset
#param initial_queryset: The queryset to serialize
"""
self.model = model
self.initial_queryset = initial_queryset
self.inheritance_tree = self._discover_inheritance_tree()
def serialize(self):
list_of_querysets = self._join_inheritance_tree_objects()
merged_querysets = self._zip_queryset_list(list_of_querysets)
result = []
for related_objects in merged_querysets:
result.append(self._serialize_related_objects(related_objects))
return json.dumps(result)
def _serialize_related_objects(self, related_objects):
"""
In this method, we serialize each instance using the django's serializer function as shown in :
See https://docs.djangoproject.com/en/1.10/topics/serialization/#inherited-models
However, it returns a list with mixed objects... Here we join those related objects into one single dict
"""
serialized_objects = []
for related_object in related_objects:
serialized_object = self._serialize_object(related_object)
fields = serialized_object['fields']
fields['pk'] = serialized_object['pk']
serialized_objects.append(fields)
merged_related_objects = {k: v for d in serialized_objects for k, v in d.items()}
return merged_related_objects
def _serialize_object(self, obj):
data = serializers.serialize('json', [obj, ])
struct = json.loads(data)
return struct[0]
def _discover_inheritance_tree(self):
# We need to find the inheritance tree which excludes abstract classes,
# so we can then join them when serializing the instance
return [x for x in inspect.getmro(self.model) if x is not object and x is not DjangoBaseModel and not x._meta.abstract]
def _join_inheritance_tree_objects(self):
"""
Here we join the required querysets from the non abstract inherited models, which we need so we are able to
serialize them.
Lets say that MyUser inherits from Customer and customer inherits from django's User model
This will return [list(MyUser.objects.filter(...), list(Customer.objects.filter(...), list(User.objects.filter(...)
"""
initial_ids = self._get_initial_ids()
inheritance__querysets = [list(x.objects.filter(id__in=initial_ids).order_by("id")) for x in self.inheritance_tree]
return inheritance__querysets
def _zip_queryset_list(self, list_of_querysets):
"""
At this stage, we have something like:
(
[MyUser1, MyUser2, MyUser3],
[Customer1, Customer2, Customer3],
[User1, User2, User3]
)
And to make it easier to work with, we 'zip' the list of lists so it looks like:
(
[MyUser1, Customer1, User1],
[MyUser2, Customer2, User2],
[MyUser3, Customer3, User3],
)
"""
return zip(*list_of_querysets)
def _get_initial_ids(self):
"""
Returns a list of ids of the initial queryset
"""
return self.initial_queryset.order_by("id").values_list("id", flat=True)
You can define a custom Serializer:
class DogSerializer(serializers.ModelSerializer):
class Meta:
model = Dog
fields = ('color','name')
Use it like:
serializer = DogSerializer(Dog.objects.all(), many=True)
print serializer.data enter code here
Did you look at select_related() ?
as in
serializers.serialize('xml', Dog.objects.select_related().all())