Develop Reference

DRF - Raise Exception if any defined field is None

I need to serialize model to JSON. Then send this JSON to one API.
But this API requires some fields to be not None.
I have a list of these fields. In this case, let's say it's just ['telephone'] but it can be much more.
For example:
class UserSerializer(serializers.ModelSerializer):
telephone = serializers.CharField(source='userprofile.telephone')
class Meta:
model = User
fields = ['first_name','last_name','telephone']
Serialization:
>>> UserSerializer(user).data
>>> {'first_name':'Michael','last_name':'Jackson','telephone':None}
Since API requires some fields like telephone, I want UserSerializer to raise ValidationError when the required field is None.
So in this case I couldn't serialize user because telephone is None.
I tried many things including adding required=True to the telephone but nothing works.
Is there a way to validate serialized data? Note that I'm not talking about deserialization.

Why validation not working?
The validation process undergoes only while Deserialization process (input is a dict like object) and you are trying a Serialization process. In the case of Serialization, DRF assumes the given object is a valid one and hence it doesn't require a validation.
Source DRF-serializers
How can we make this happen?
Method-1
Make your user object to a user_data (dict object) and pass it to the serializer and run the validation.
user = User.objects.get(id=1)
dict_user_data = {"first_name": user.first_name, "last_name": user.last_name, "telephone": user.userprofile.telephone}
user_serializer = UserSerializer(data=dict_user_data)
user_serializer.is_valid(True)
user_serializer.data
Method-2
Override the to_representation() method
class UserSerializer(serializers.ModelSerializer):
telephone = serializers.CharField(source='userprofile.telephone')
class Meta:
model = User
fields = ['first_name', 'last_name', 'telephone']
def to_representation(self, instance):
data = super().to_representation(instance)
for field, value in data.items():
if value is None:
raise SomeExceptionHere({field: "can't be None"})
return data

You don't need to. DRF serializers can do that right out the box. If a field is setted to be null=False or required=True, just do this.
data = UserSerializer(data=user_data)
data.is_valid(raise_exception=True
And that is it. A 400 error will be raised.
If you want, you can tweak the error message:
data = UserSerializer(data=user_data)
try:
data.is_valid(raise_exception=True)
except ValidationError as e:
return HttpResponse(status_code=400, content=e)

What you want is not to validate data for de-serialization, but to validate for serialization. Expressions like required=True are all used to validate data for de-serialization. Serialization is handled in to_representation method of Serializer class (which is the base for ModelSerializer)
What you can do is, override to_representation, and have the method raise an exception if a required field value is None.
You may need to further subclass default DRF classes like Field (for a serializer field) and use your custom classes to be able to provide your functionality in a systematic manner. You do not want to rely on required=True for this, because it is used for another purpose already (for de-serialization validation).
What I suggest is, subclass Field class, add a property like "required_for_read", and define your serializer field with this property, using your custom field class. Then, in your overridden to_representation method, look for this property, and raise an exception if the field has this property as True but its value is None

Django strange SlugField validation, error not raised before clean(), uncleaned data returned

django 2.0
I have a django model, with different slug fields:
from django.core.validators import validate_slug
class MyModel(models.Model):
# with slug field
slug_field = models.SlugField(max_length=200)
# or charfield with slug validator (should be exactly the same)
char_field = models.CharField(max_length=200, validators=[validate_slug])
The first problem i have, in my form i have a clean method, to validate the values of multiple fields, not individually. This method should theoretically be called after the clean_fields method, but it is called even if clean_fields raise an error.
My forms.py:
class MyForm(forms.ModelForm):
class Meta:
model = MyModel
fields = '__all__'
def clean(self):
cleaned_data = super().clean()
print(cleaned_data.get('slug_field')) # > None
print(cleaned_data.get('char_field')) # > ééé; uncleaned data
print(self.errors) # only from slug_field
return cleaned_data
With SlugField, slug_field is not set in cleaned_data, when it's invalid, and after the error is raised and returned to the user by the form. (I don't see why clean() is even reached, because clean_fields() have raised the error before)
The problem is that with the CharField with any custom validator (validate_slug or a self made one), the uncleaned value is returned in cleaned_data. However, the validation error is still raised, but after.
This is quite dangerous for me, because i used to trust cleaned_data, to modify data not saved inside the model.

The clean() method is called after the field's validator. If the alias is invalid, then it won't be in cleaned_data. Your clean method should handle this case, for example:
def clean():
cleaned_data = super().clean()
print(self.errors) # You should see the alias error here
if 'alias' in cleaned_data:
print(cleaned_data['alias'])
# do any code that relies on cleaned_data['alias'] here
return cleaned_data
See the docs on cleaning fields that depend on each other for more info.

How to access cleaned data in a Django form's clean() method?

I'd like to implement an input field in a Django form, phone_type, which is only required when another field, phone_number, is filled in. I'm reading the example at https://www.fusionbox.com/blog/detail/creating-conditionally-required-fields-in-django-forms/577/ on how to do this:
def clean(self):
shipping = self.cleaned_data.get('shipping')
if shipping:
msg = forms.ValidationError("This field is required.")
self.add_error('shipping_destination', msg)
else:
# Keep the database consistent. The user may have
# submitted a shipping_destination even if shipping
# was not selected
self.cleaned_data['shipping_destination'] = ''
return self.cleaned_data
where the models are defined as
from django.db import models
class ShippingInfo(models.Model):
SHIPPING_DESTINATION_CHOICES = (
('residential', "Residential"),
('commercial', "Commercial"),
)
shipping = models.BooleanField()
shipping_destination = models.CharField(
max_length=15,
choices=SHIPPING_DESTINATION_CHOICES,
blank=True
)
When comparing this code to the documentation at https://docs.djangoproject.com/en/2.0/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other, however, I notice that there is no call to super().clean(). Instead of accessing self.cleaned_data directly, should I do
cleaned_data = super().clean()
shipping = cleaned_data.get('shipping')
in the first lines of the custom clean() method?
(I would also be interested in ways to make the field conditionally visible without requiring additional jQuery/JavaScript code, e.g. using Django Crispy Forms and/or the HiddenInput widget).

forminstance.is_valid or forminstance.full_clean will call your form's clean method implicitly by which time forminstance.cleaned_data will have the dict populated with the data in the right type according to the form fields. The call to super in the example you posted is in case you have inheritance in your form class hierarchy.
For clarification. It won't hurt if you have super but it won't change anything if you're not inheriting from a form class that doesn't have any fields defined.

Where should i do the django validations for objects and fields?

I'm creating a django application which uses both the Django Rest Framework and the plain django-views as entrypoint for users.
I want to do validation both independant fields of my models, and on objects on a whole. For example:
Field: is the entered licence-plate a correct one based on a regex function. No relation to other fields.
Object: Is the entered zipcode valid for the given country. Relates to zipcode and country in the model.
For the DRF-API i use ModelSerializers which automatically call all the validators i have placed in my Model, for example:
class MyModel(models.Model):
licence_plate = CharField(max_length=20, validators=[LicencePlateValidator])
Since the validator is given in the model, the API POSTS (because i use a ModelSerializer), as well as the objects created in the django admin backend are validated.
But when i want to introduce object level validation i need to do that in the serializer's validate()-method, which means objects are only validated in the API.
I'll have to override the model's save method too, to validate the objects created in the Django admin page.
Question: This seems a bit messy to me, is there a single point where i can put the object-level validators so that they are run at the API and in the admin-page, like i did with the field-level validation (I only have to put them in my model-declaration and everything is handled)

For model-level validation, there is the Model.clean method.
It is called if you are using ModelForm (which is used by default in admin), so this solves django views and admin parts.
On the other hand, DRF does not call models' clean automatically, so you will have to do it yourself in Serializer.validate (as the doc suggests). You can do it via a serializer mixin:
class ValidateModelMixin(object)
def validate(self, attrs):
attrs = super().validate(attrs)
obj = self.Meta.model(**attrs)
obj.clean()
return attrs
class SomeModelSerializer(ValidateModelMixin, serializers.ModelSerializer):
#...
class Meta:
model = SomeModel
or write a validator:
class DelegateToModelValidator(object):
def set_context(self, serializer):
self.model = serializer.Meta.model
def __call__(self, attrs):
obj = self.model(**attrs)
obj.clean()
class SomeModelSerializer(serializers.ModelSerializer):
#...
class Meta:
model = SomeModel
validators = (
DelegateToModelValidator(),
)
Caveats:
an extra instantiation of your models just to call clean
you will still have to add the mixin/validator to your serializers

You can create a separate function validate_zipcode_with_country(zipcode, country) which will take 2 arguments zipcode and country.
Then, we will call this method in the serializer's validate() and in our model's clean().
from django.core.exceptions import ValidationError
def validate_zipcode_with_country(zipcode, country):
# check zipcode is valid for the given country
if not valid_zipcode:
raise ValidationError("Zipcode is not valid for this country.")
Then in your serializers.py, you need to call this function in your validate() function.
class MySerializer(serializers.ModelSerializer):
def validate(self, attrs):
zipcode = attrs.get('zipcode')
country = attrs.get('country')
validate_zipcode_with_country(zipcode, country) # call the function
...
Similarly, you need to override the model's clean() and call this function.
class MyModel(models.Model):
def clean(self):
validate_zipcode_with_country(self.zipcode, self.country) # call this function
...

Django REST Framework - Serializing optional fields

I have an object that has optional fields. I have defined my serializer this way:
class ProductSerializer(serializers.Serializer):
code = serializers.Field(source="Code")
classification = serializers.CharField(source="Classification", required=False)
I thought required=False would do the job of bypassing the field if it doesn't exist. However, it is mentioned in the documentation that this affects deserialization rather than serialization.
I'm getting the following error:
'Product' object has no attribute 'Classification'
Which is happening when I try to access .data of the serialized instance. (Doesn't this mean it's deserialization that's raising this?)
This happens for instances that do not have Classification. If I omit Classification from the serializer class it works just fine.
How do I correctly do this? Serialize an object with optional fields, that is.

Django REST Framework 3.0+
Dynamic fields now supported, see http://www.django-rest-framework.org/api-guide/serializers/#dynamically-modifying-fields -- this approach defines all of the fields in the serializer, and then allows you to selectively remove the ones you don't want.
Or you could also do something like this for a Model Serializer, where you mess around with Meta.fields in the serializer init:
class ProductSerializer(serializers.ModelSerializer):
class Meta:
model = Product
fields = ('code',)
def __init__(self, *args, **kwargs):
if SHOW_CLASSIFICATION: # add logic here for optional viewing
self.Meta.fields = list(self.Meta.fields)
self.Meta.fields.append('classification')
super(ProductSerializer, self).__init__(*args, **kwargs)
You'd have to ask Tom though if this is the "correct way" since it may not fit in with the long term plan.
Django REST Framework < 3.0
Try something like this:
class ProductSerializer(serializers.Serializer):
...
classification = serializers.SerializerMethodField('get_classification')
def get_classification(self, obj):
return getattr(obj, 'classification', None)
Multiple Serializers
Another approach would be to create multiple serializers with different sets of fields. One serializer inherits from another and adds additional fields. Then you can choose the appropriate serializer in the view with the get_serializer_class method. Here's an actual example of how I use this approach to call different serializers to present different user data if the user object is the same as the request user.
def get_serializer_class(self):
""" An authenticated user looking at their own user object gets more data """
if self.get_object() == self.request.user:
return SelfUserSerializer
return UserSerializer
Removing fields from representation
Another approach that I've used in security contexts is to remove fields in the to_representation method. Define a method like
def remove_fields_from_representation(self, representation, remove_fields):
""" Removes fields from representation of instance. Call from
.to_representation() to apply field-level security.
* remove_fields: a list of fields to remove
"""
for remove_field in remove_fields:
try:
representation.pop(remove_field)
except KeyError:
# Ignore missing key -- a child serializer could inherit a "to_representation" method
# from its parent serializer that applies security to a field not present on
# the child serializer.
pass
and then in your serializer, call that method like
def to_representation(self, instance):
""" Apply field level security by removing fields for unauthorized users"""
representation = super(ProductSerializer, self).to_representation(instance)
if not permission_granted: # REPLACE WITH PERMISSION LOGIC
remove_fields = ('classification', )
self.remove_fields_from_representation(representation, remove_fields)
return representation
This approach is straightforward and flexible, but it comes at the cost of serializing fields that are sometimes not displayed. But that's probably okay.

The method describe below did the work for me.
Pretty simple,easy and worked for me.
DRF version used = djangorestframework (3.1.0)
class test(serializers.Serializer):
id= serializers.IntegerField()
name=serializers.CharField(required=False,default='some_default_value')

The serializers are deliberately designed to use a fixed set of fields so you wouldn't easily be able to optionally drop out one of the keys.
You could use a SerializerMethodField to either return the field value or None if the field doesn't exist, or you could not use serializers at all and simply write a view that returns the response directly.
Update for REST framework 3.0 serializer.fields can be modified on an instantiated serializer. When dynamic serializer classes are required I'd probably suggest altering the fields in a custom Serializer.__init__() method.

The serializers Charfield method has a property allow_blank
By default it is set to False.
Setting it to True will allow you to mark the field as optional during "serialization".
This is the code that you should write
classification = serializers.CharField(source="Classification", allow_blank=True)
Note: required property is used for deserialization.

DynamicSerializer for DRF 3, which allows dynamicly specifying which fields will be used in serializer, which will be excluded, and optionally which will become required!
Create Mixin
class DynamicSerializerMixin:
"""
A Serializer that takes an additional `fields` argument that
controls which fields should be used.
"""
def __init__(self, *args, **kwargs):
# Don't pass the 'fields' arg up to the superclass
fields = kwargs.pop("fields", None)
excluded_fields = kwargs.pop("excluded_fields", None)
required_fields = kwargs.pop("required_fields", None)
# Instantiate the superclass normally
super().__init__(*args, **kwargs)
if fields is not None:
# Drop any fields that are not specified in the `fields` argument.
allowed = set(fields)
existing = set(self.fields)
for field_name in existing - allowed:
self.fields.pop(field_name)
if isinstance(fields, dict):
for field, config in fields.items():
set_attrs(self.fields[field], config)
if excluded_fields is not None:
# Drop any fields that are not specified in the `fields` argument.
for field_name in excluded_fields:
self.fields.pop(field_name)
if required_fields is not None:
for field_name in required_fields:
self.fields[field_name].required = True
Initialize/adjust your serializer by adding DynamicSerializerMixin to inheritence
class UserProfileSerializer(DynamicSerializerMixin, serializers.ModelSerializer):
class Meta:
model = User
fields = (
"id",
'first_name', 'last_name'
"email",
"is_staff",
)
Use it :)
class RoleInvitationSerializer(serializers.ModelSerializer):
invited_by = UserProfileSerializer(fields=['id', 'first_name', 'last_name'])
or in action apis
#action(detail=True, serializer_class=YourSerialzierClass)
def teams_roles(self, request, pk=None):
user = self.get_object()
queryset = user.roles.all()
serializer = self.get_serializer(queryset, many=True, excluded_fields=['user'])
return Response(data=serializer.data)

For this purpose the serializers have the partial argument. If when the serializer is initialized you can pass partial=True. If you are using generics or mixins you can overrider the get_serializer function as follows:
def get_serializer(self, *args, **kwargs):
kwargs['partial'] = True
return super(YOUR_CLASS, self).get_serializer(*args, **kwargs)
And that will do the trick.
Note: This allows all fields to be optional and not only a specific one. If you want only specifics, you can override the method (i.e. update) and add validations of existence for various fields.

What has worked well for me is to set the serializer like so:
classification = serializers.CharField(max_length=20, allow_blank=True, default=None)

From the "it's a terrible hack relying on specific implementation details of both DRF and Django, but it works (at least for now)" files, here's the approach I used to include some additional debugging data in the response from a "create" method implementation on a serializer:
def create(self, validated_data)
# Actual model instance creation happens here...
self.fields["debug_info"] = serializers.DictField(read_only=True)
my_model.debug_info = extra_data
return my_model
This is a temporary approach that lets me use the browsable API to display some of the raw response data received from a particular remote service during the creation process. In the future, I'm inclined to keep this capability, but hide it behind a "report debugging info" flag in the creation request rather than returning the lower level info by default.