Develop Reference

Converting fields inside a Serializer in django

I have a project where one model, called Request, has two fields (source, dest) that contain two ids which are not known to the user. However, each one is connected to another model User, who let's say that they have one field, username, which is known to the user.
Now, I want to make a serializer that can take usernames, and convert them into ids. (The opposite was simple to achieve, I just modified the to_representation method.) The problem is that when I send {'source': 'john', 'dest': 'jim'} the serializer does not take these data as valid. This was expected behavior, as it expected ids and got strings (usernames). However, even when I overridden the validate_source, validate_dest and validate methods to actually check that the usernames exist (instead of the ids), I am still getting errors that the serializer expected id but got string.
Are the validate, validate_<field> methods the wrong ones to override in this case?
Should I just convert the usernames into ids inside my view?
is it pythonic and good practice, django-wise, to receive some fields from the user and change them inside the serializer (as I change username into id)?
Current Serializer:
class RequestSerializer(serializers.ModelSerializer):
class Meta:
model = Request
fields = '__all__'
def validate_source(self, value):
username = value.get('username')
if username is None:
raise serializers.ValidationError('`user` field is required ')
return value
def validate_dest(self, value):
username = value.get('username')
if username is None:
raise serializers.ValidationError('`user` field is required ')
return value
def validate(self, attrs):
self.validate_source(attrs['source'])
self.validate_dest(attrs['dest'])
return attrs
def to_representation(self, instance):
# do things
pass
Please notice that this is not the whole functionality of my serializer. To convert from an id to a username I have to check the data of another Model, So I cannot use a SlugRelatedField.
Also, username is not the only item returned by the serializer. It also returns a 'class' field, depending on which group the the user has joined. The user may join more than one group, and each user-group combination has its own id. In the same way, when deserializing the data, I will need to read (1) the username, and then (2) the group, and find the correct id.
Thank you.

You probably can work with a SlugRelatedField [drf-doc]:
from rest_framework import serializers
class MyModelSerializer(serializers.ModelSerializer):
source = serializers.SlugRelatedField(
queryset=User.objects.all(),
slug_field='username',
)
dest = serializers.SlugRelatedField(
queryset=User.objects.all(),
slug_field='username',
)
class Meta:
model = MyModel
fields = ('source', 'dest')
This will return the username of the source and dest field of the model object, and in the opposite direction will fetch the User with the corresponding username.

How to auto populate a read-only serializer field in django rest framework?

I have a question regarding django rest framework.
Most of the time, I have a serializer which has some read-only fields. For example, consider this simple model below:
class PersonalMessage(models.Model):
sender = models.ForeignKey(User, related_name="sent_messages", ...)
recipient = models.ForeignKey(User, related_name="recieved_messages", ...)
text = models.CharField(...)
def __str__(self) -> str:
return f"{self.text} (sender={self.sender})"
In this model, the value of sender and recipient should be automatically provided by the application itself and the user shouldn't be able to edit those fields. Alright, now take a look at this serializer:
class PersonalMessageSerializer(serializers.ModelSerializer):
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')
It perfectly prevents users from setting an arbitrary value on the sender and recipient fields. But the problem is, when these fields are marked as read-only in the serializer, the serializer will completely ignore all the values that are passed into the constructor for these fields. So when I try to create a model, no values would be set for these fields:
PersonalMessageSerializer(data={**request.data, 'sender': ..., 'recipient': ...) # Won't work
What's the best way to prevent users from setting an arbitrary value and at the same time auto-populate those restricted fields in django rest framework?

Depending on how you get those two objects, you can use the serializer's save method to pass them, and they will automatically be applied to the object you are saving:
sender = User.objects.first()
recipient = User.objects.last()
serializer = PersonalMessageSerializer(data=request.data)
message = serializer.save(sender=sender, recipient=recipient)
The kwargs should match the field names in your model for this to work. For reference, have a look here

You able to override the serializer context like this;
PersonalMessageSerializer(data={**request.data, context={'sender': sender, 'recipent': recipent})
and catch the context inside serializer.
class PersonalMessageSerializer(serializers.ModelSerializer):
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')
def validate(self, attrs):
attrs = super().validate(attrs)
attrs['sender'] = self.context['sender']
attrs['recipent'] = self.context['recipent']
return attrs
now serializer.validated_data it must returns sender and recipent.

From the question it is not possible to understand what field(s) of the relationship with sender and recipient you want to interact with, but a general answer can be found in the Serializer relations section of Django REST documentation.
Long story short, if you want to interact with one field only, you can use SlugRelatedField, which lets you interact with the target of the relationship using only one of its fields.
If it just the id, you can use PrimaryKeyRelatedField.
If you want to interact with more than one field, the way to go is Nested Relationships. Here you can specify a custom serializer for the target relationship, but you will have to override the create() method in your PersonalMessageSerializer to create the object from your relationship, as nested serializers are read-only by default.

So this is how you can make set a default on create but read only after in DRF. Although in this solution it wont actually be readonly, it's writable, but you now have explicit control on what the logged in user can write, which is the ultimate goal
Given the model
class PersonalMessage(models.Model):
sender = models.ForeignKey(User,...)
recipient = models.ForeignKey(User,..)
text = models.CharField(...)
You would first create your own custom default (I will show an example for only one field)
# Note DRF already has a CurrentUserDefault you can also use
class CurrentSenderDefault:
requires_context = True
def __call__(self, serializer_field):
return serializer_field.context['request'].user
def __repr__(self):
return '%s()' % self.__class__.__name__
Next you make your own field, that knows whats up with the filter.
This queryset prevents people from setting a value they are not allowed to. which is exactly what you want
class SenderField(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
user = self.context['request'].user
if user:
queryset = User.objects.filter(id=user.id)
else:
queryset = User.objects.none()
return queryset
Finally on the serialiser you go
class PersonalMessageSerializer(serializers.ModelSerializer):
sender = SenderField(default=CurrentSenderDefault())
recipient = ...
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')

Why is max_length being ignored in BinaryField?

I am attempting to create an invite system to register users, which stores a md5 hashsum based on the object's ID. Here's how my model is defined:
class Invite(TimestampedModel):
user = models.ForeignKey(User, default=None, null=True, blank=True, on_delete=models.CASCADE)
_code = models.BinaryField(max_length=16, unique=True)
#classmethod
def create(cls):
invite = cls.objects.create()
hash_obj = hashlib.md5()
hash_obj.update(str(invite.id).encode('utf-8'))
invite._code = hash_obj.digest()
invite.save()
return invite
Thus when I call Invite.create(), I see a row in the database that has a _code that is 16 characters long (e.g. ��#�"��Zo�n�%��V).
The problem arises, however, when I adjust max_length to 2. I expected Django to either throw an error or truncate the value to two bytes, but it did neither; I am still seeing 16 bytes in new rows. Yes, I ran makemigrations and migrate.
What is going on here? This is a bit concerning that I can't limit the length at all. I assume it's defaulting to 50 but I have no idea why. Any thoughts are appreciated.
BinaryField description here.

If you look at the source code of the BinaryField class, you will see that having max_length parameter will only add a validator to a field:
class BinaryField(Field):
description = _("Raw binary data")
empty_values = [None, b'']
def __init__(self, *args, **kwargs):
kwargs.setdefault('editable', False)
super().__init__(*args, **kwargs)
if self.max_length is not None:
self.validators.append(validators.MaxLengthValidator(self.max_length))
Now, you go to the docs and search for how validators are run:
Note that validators will not be run automatically when you save a model, but if you are using a ModelForm, it will run your validators on any fields that are included in your form.
Now you see, that if you're not using ModelForm you'll have to run validations by yourself:
There are three steps involved in validating a model:
Validate the model fields - Model.clean_fields()
Validate the model as a whole - Model.clean()
Validate the field uniqueness - Model.validate_unique()
All three steps are performed when you call a model’s full_clean() method.
When you use a ModelForm, the call to is_valid() will perform these validation steps for all the fields that are included on the form. See the ModelForm documentation for more information.
You should only need to call a model’s full_clean() method if you plan to handle validation errors yourself, or if you have excluded fields from the ModelForm that require validation.
So, as i said, it has nothing to do with migrations, you just have to call full_clean method (or just clean_fields one) or use ModelForm and call it's is_valid() method.
refs:
BinaryField implementation
How validators are run
Validating models

Where should i do the django validations for objects and fields?

I'm creating a django application which uses both the Django Rest Framework and the plain django-views as entrypoint for users.
I want to do validation both independant fields of my models, and on objects on a whole. For example:
Field: is the entered licence-plate a correct one based on a regex function. No relation to other fields.
Object: Is the entered zipcode valid for the given country. Relates to zipcode and country in the model.
For the DRF-API i use ModelSerializers which automatically call all the validators i have placed in my Model, for example:
class MyModel(models.Model):
licence_plate = CharField(max_length=20, validators=[LicencePlateValidator])
Since the validator is given in the model, the API POSTS (because i use a ModelSerializer), as well as the objects created in the django admin backend are validated.
But when i want to introduce object level validation i need to do that in the serializer's validate()-method, which means objects are only validated in the API.
I'll have to override the model's save method too, to validate the objects created in the Django admin page.
Question: This seems a bit messy to me, is there a single point where i can put the object-level validators so that they are run at the API and in the admin-page, like i did with the field-level validation (I only have to put them in my model-declaration and everything is handled)

For model-level validation, there is the Model.clean method.
It is called if you are using ModelForm (which is used by default in admin), so this solves django views and admin parts.
On the other hand, DRF does not call models' clean automatically, so you will have to do it yourself in Serializer.validate (as the doc suggests). You can do it via a serializer mixin:
class ValidateModelMixin(object)
def validate(self, attrs):
attrs = super().validate(attrs)
obj = self.Meta.model(**attrs)
obj.clean()
return attrs
class SomeModelSerializer(ValidateModelMixin, serializers.ModelSerializer):
#...
class Meta:
model = SomeModel
or write a validator:
class DelegateToModelValidator(object):
def set_context(self, serializer):
self.model = serializer.Meta.model
def __call__(self, attrs):
obj = self.model(**attrs)
obj.clean()
class SomeModelSerializer(serializers.ModelSerializer):
#...
class Meta:
model = SomeModel
validators = (
DelegateToModelValidator(),
)
Caveats:
an extra instantiation of your models just to call clean
you will still have to add the mixin/validator to your serializers

You can create a separate function validate_zipcode_with_country(zipcode, country) which will take 2 arguments zipcode and country.
Then, we will call this method in the serializer's validate() and in our model's clean().
from django.core.exceptions import ValidationError
def validate_zipcode_with_country(zipcode, country):
# check zipcode is valid for the given country
if not valid_zipcode:
raise ValidationError("Zipcode is not valid for this country.")
Then in your serializers.py, you need to call this function in your validate() function.
class MySerializer(serializers.ModelSerializer):
def validate(self, attrs):
zipcode = attrs.get('zipcode')
country = attrs.get('country')
validate_zipcode_with_country(zipcode, country) # call the function
...
Similarly, you need to override the model's clean() and call this function.
class MyModel(models.Model):
def clean(self):
validate_zipcode_with_country(self.zipcode, self.country) # call this function
...

Django REST Framework - Serializing optional fields

I have an object that has optional fields. I have defined my serializer this way:
class ProductSerializer(serializers.Serializer):
code = serializers.Field(source="Code")
classification = serializers.CharField(source="Classification", required=False)
I thought required=False would do the job of bypassing the field if it doesn't exist. However, it is mentioned in the documentation that this affects deserialization rather than serialization.
I'm getting the following error:
'Product' object has no attribute 'Classification'
Which is happening when I try to access .data of the serialized instance. (Doesn't this mean it's deserialization that's raising this?)
This happens for instances that do not have Classification. If I omit Classification from the serializer class it works just fine.
How do I correctly do this? Serialize an object with optional fields, that is.

Django REST Framework 3.0+
Dynamic fields now supported, see http://www.django-rest-framework.org/api-guide/serializers/#dynamically-modifying-fields -- this approach defines all of the fields in the serializer, and then allows you to selectively remove the ones you don't want.
Or you could also do something like this for a Model Serializer, where you mess around with Meta.fields in the serializer init:
class ProductSerializer(serializers.ModelSerializer):
class Meta:
model = Product
fields = ('code',)
def __init__(self, *args, **kwargs):
if SHOW_CLASSIFICATION: # add logic here for optional viewing
self.Meta.fields = list(self.Meta.fields)
self.Meta.fields.append('classification')
super(ProductSerializer, self).__init__(*args, **kwargs)
You'd have to ask Tom though if this is the "correct way" since it may not fit in with the long term plan.
Django REST Framework < 3.0
Try something like this:
class ProductSerializer(serializers.Serializer):
...
classification = serializers.SerializerMethodField('get_classification')
def get_classification(self, obj):
return getattr(obj, 'classification', None)
Multiple Serializers
Another approach would be to create multiple serializers with different sets of fields. One serializer inherits from another and adds additional fields. Then you can choose the appropriate serializer in the view with the get_serializer_class method. Here's an actual example of how I use this approach to call different serializers to present different user data if the user object is the same as the request user.
def get_serializer_class(self):
""" An authenticated user looking at their own user object gets more data """
if self.get_object() == self.request.user:
return SelfUserSerializer
return UserSerializer
Removing fields from representation
Another approach that I've used in security contexts is to remove fields in the to_representation method. Define a method like
def remove_fields_from_representation(self, representation, remove_fields):
""" Removes fields from representation of instance. Call from
.to_representation() to apply field-level security.
* remove_fields: a list of fields to remove
"""
for remove_field in remove_fields:
try:
representation.pop(remove_field)
except KeyError:
# Ignore missing key -- a child serializer could inherit a "to_representation" method
# from its parent serializer that applies security to a field not present on
# the child serializer.
pass
and then in your serializer, call that method like
def to_representation(self, instance):
""" Apply field level security by removing fields for unauthorized users"""
representation = super(ProductSerializer, self).to_representation(instance)
if not permission_granted: # REPLACE WITH PERMISSION LOGIC
remove_fields = ('classification', )
self.remove_fields_from_representation(representation, remove_fields)
return representation
This approach is straightforward and flexible, but it comes at the cost of serializing fields that are sometimes not displayed. But that's probably okay.

The method describe below did the work for me.
Pretty simple,easy and worked for me.
DRF version used = djangorestframework (3.1.0)
class test(serializers.Serializer):
id= serializers.IntegerField()
name=serializers.CharField(required=False,default='some_default_value')

The serializers are deliberately designed to use a fixed set of fields so you wouldn't easily be able to optionally drop out one of the keys.
You could use a SerializerMethodField to either return the field value or None if the field doesn't exist, or you could not use serializers at all and simply write a view that returns the response directly.
Update for REST framework 3.0 serializer.fields can be modified on an instantiated serializer. When dynamic serializer classes are required I'd probably suggest altering the fields in a custom Serializer.__init__() method.

The serializers Charfield method has a property allow_blank
By default it is set to False.
Setting it to True will allow you to mark the field as optional during "serialization".
This is the code that you should write
classification = serializers.CharField(source="Classification", allow_blank=True)
Note: required property is used for deserialization.

DynamicSerializer for DRF 3, which allows dynamicly specifying which fields will be used in serializer, which will be excluded, and optionally which will become required!
Create Mixin
class DynamicSerializerMixin:
"""
A Serializer that takes an additional `fields` argument that
controls which fields should be used.
"""
def __init__(self, *args, **kwargs):
# Don't pass the 'fields' arg up to the superclass
fields = kwargs.pop("fields", None)
excluded_fields = kwargs.pop("excluded_fields", None)
required_fields = kwargs.pop("required_fields", None)
# Instantiate the superclass normally
super().__init__(*args, **kwargs)
if fields is not None:
# Drop any fields that are not specified in the `fields` argument.
allowed = set(fields)
existing = set(self.fields)
for field_name in existing - allowed:
self.fields.pop(field_name)
if isinstance(fields, dict):
for field, config in fields.items():
set_attrs(self.fields[field], config)
if excluded_fields is not None:
# Drop any fields that are not specified in the `fields` argument.
for field_name in excluded_fields:
self.fields.pop(field_name)
if required_fields is not None:
for field_name in required_fields:
self.fields[field_name].required = True
Initialize/adjust your serializer by adding DynamicSerializerMixin to inheritence
class UserProfileSerializer(DynamicSerializerMixin, serializers.ModelSerializer):
class Meta:
model = User
fields = (
"id",
'first_name', 'last_name'
"email",
"is_staff",
)
Use it :)
class RoleInvitationSerializer(serializers.ModelSerializer):
invited_by = UserProfileSerializer(fields=['id', 'first_name', 'last_name'])
or in action apis
#action(detail=True, serializer_class=YourSerialzierClass)
def teams_roles(self, request, pk=None):
user = self.get_object()
queryset = user.roles.all()
serializer = self.get_serializer(queryset, many=True, excluded_fields=['user'])
return Response(data=serializer.data)

For this purpose the serializers have the partial argument. If when the serializer is initialized you can pass partial=True. If you are using generics or mixins you can overrider the get_serializer function as follows:
def get_serializer(self, *args, **kwargs):
kwargs['partial'] = True
return super(YOUR_CLASS, self).get_serializer(*args, **kwargs)
And that will do the trick.
Note: This allows all fields to be optional and not only a specific one. If you want only specifics, you can override the method (i.e. update) and add validations of existence for various fields.

What has worked well for me is to set the serializer like so:
classification = serializers.CharField(max_length=20, allow_blank=True, default=None)

From the "it's a terrible hack relying on specific implementation details of both DRF and Django, but it works (at least for now)" files, here's the approach I used to include some additional debugging data in the response from a "create" method implementation on a serializer:
def create(self, validated_data)
# Actual model instance creation happens here...
self.fields["debug_info"] = serializers.DictField(read_only=True)
my_model.debug_info = extra_data
return my_model
This is a temporary approach that lets me use the browsable API to display some of the raw response data received from a particular remote service during the creation process. In the future, I'm inclined to keep this capability, but hide it behind a "report debugging info" flag in the creation request rather than returning the lower level info by default.