Develop Reference

Django strange SlugField validation, error not raised before clean(), uncleaned data returned

django 2.0
I have a django model, with different slug fields:
from django.core.validators import validate_slug
class MyModel(models.Model):
# with slug field
slug_field = models.SlugField(max_length=200)
# or charfield with slug validator (should be exactly the same)
char_field = models.CharField(max_length=200, validators=[validate_slug])
The first problem i have, in my form i have a clean method, to validate the values of multiple fields, not individually. This method should theoretically be called after the clean_fields method, but it is called even if clean_fields raise an error.
My forms.py:
class MyForm(forms.ModelForm):
class Meta:
model = MyModel
fields = '__all__'
def clean(self):
cleaned_data = super().clean()
print(cleaned_data.get('slug_field')) # > None
print(cleaned_data.get('char_field')) # > ééé; uncleaned data
print(self.errors) # only from slug_field
return cleaned_data
With SlugField, slug_field is not set in cleaned_data, when it's invalid, and after the error is raised and returned to the user by the form. (I don't see why clean() is even reached, because clean_fields() have raised the error before)
The problem is that with the CharField with any custom validator (validate_slug or a self made one), the uncleaned value is returned in cleaned_data. However, the validation error is still raised, but after.
This is quite dangerous for me, because i used to trust cleaned_data, to modify data not saved inside the model.

The clean() method is called after the field's validator. If the alias is invalid, then it won't be in cleaned_data. Your clean method should handle this case, for example:
def clean():
cleaned_data = super().clean()
print(self.errors) # You should see the alias error here
if 'alias' in cleaned_data:
print(cleaned_data['alias'])
# do any code that relies on cleaned_data['alias'] here
return cleaned_data
See the docs on cleaning fields that depend on each other for more info.

Django REST Framework serializer field required=false

from the documentation:
read_only
Set this to True to ensure that the field is used when serializing a representation, but is not used when updating an instance during deserialization.
Defaults to False
required
Normally an error will be raised if a field is not supplied during deserialization. Set to false if this field is not required to be present during deserialization.
Defaults to True.
So I have a model which has a field that's not nullable but I want it to be populated in the pre_save method, so I have set the field to required=False in serializer, but doesn't seem to work. I am still getting error when saving the record.
class FavoriteListSerializer(serializers.ModelSerializer):
owner = serializers.IntegerField(required=False)
class Meta:
model = models.FavoriteList
Update:
I have added serializer_class = serializers.FavoriteListSerializer to the ViewSet, now instead of getting This field is required, which I think got past the validation but then I am getting This field cannot be null. I have checked the pre_save method is not being executed, any ideas?

Yeah, I ran into this issue at some point as well. You need to also update the validation exclusions.
class FavoriteListSerializer(serializers.ModelSerializer):
owner = serializers.IntegerField(required=False)
class Meta:
model = models.FavoriteList
def get_validation_exclusions(self):
exclusions = super(FavoriteListSerializer, self).get_validation_exclusions()
return exclusions + ['owner']

Late Entry to this thread. This issue was fixed in django-rest-framework 2.3.13. Here is the link of the PR.
You use it like this in your case:
class Meta:
model = models.FavoriteList
optional_fields = ['owner', ]

In case somebody lands here with a similar issue, pay attention to the following attributes along with required:
allow_blank:
If set to True then the empty string should be considered a valid value.
allow_null:
Normally an error will be raised if None is passed to a serializer field.
required:
Normally an error will be raised if a field is not supplied during deserialization.
I was straggling to figure out why I was getting a validation error with required=False where I had missed the allow_null attribute.

In 2020, for DRF 3.12.x, the approach that I prefer the approach that relies on
Serializer's extra_kwargs.
So assuming your
class FavoriteListSerializer(serializers.ModelSerializer):
owner = serializers.IntegerField(required=False)
class Meta:
model = models.FavoriteList
fields = ["owner"] # and whatever other fields you want to expose
extra_kwargs = {"owner": {"required": False, "allow_null": True}}

If you have unique_together constraint on one of the fields you are trying to set required=False you need to set validators=[] in serializers Meta like
class FavoriteListSerializer(serializers.ModelSerializer):
owner = serializers.IntegerField(required=False)
class Meta:
model = models.FavoriteList
validators = []
Here is the original answer

You can also do this:
class ASerializer(serializers.HyperlinkedModelSerializer):
owner = serializers.HiddenField(default=serializers.CurrentUserDefault())
...
As referred here: https://www.django-rest-framework.org/api-guide/validators/#advanced-field-defaults
There you can also find the case when you also wanna let the view show owner

I would set model field to allow null value (and possible also default to None)
class FavoriteList(models.Model):
owner = models.PositiveIntegerField(null=True, default=None)
Then it's possible to just leave owner field to Meta section. These fields, without any extra settings, will automatically get all attributes from model field and be non-required.
class FavoriteListSerializer(serializers.ModelSerializer):
class Meta:
model = models.FavoriteList
fields = ('owner',)

Django REST Framework - Serializing optional fields

I have an object that has optional fields. I have defined my serializer this way:
class ProductSerializer(serializers.Serializer):
code = serializers.Field(source="Code")
classification = serializers.CharField(source="Classification", required=False)
I thought required=False would do the job of bypassing the field if it doesn't exist. However, it is mentioned in the documentation that this affects deserialization rather than serialization.
I'm getting the following error:
'Product' object has no attribute 'Classification'
Which is happening when I try to access .data of the serialized instance. (Doesn't this mean it's deserialization that's raising this?)
This happens for instances that do not have Classification. If I omit Classification from the serializer class it works just fine.
How do I correctly do this? Serialize an object with optional fields, that is.

Django REST Framework 3.0+
Dynamic fields now supported, see http://www.django-rest-framework.org/api-guide/serializers/#dynamically-modifying-fields -- this approach defines all of the fields in the serializer, and then allows you to selectively remove the ones you don't want.
Or you could also do something like this for a Model Serializer, where you mess around with Meta.fields in the serializer init:
class ProductSerializer(serializers.ModelSerializer):
class Meta:
model = Product
fields = ('code',)
def __init__(self, *args, **kwargs):
if SHOW_CLASSIFICATION: # add logic here for optional viewing
self.Meta.fields = list(self.Meta.fields)
self.Meta.fields.append('classification')
super(ProductSerializer, self).__init__(*args, **kwargs)
You'd have to ask Tom though if this is the "correct way" since it may not fit in with the long term plan.
Django REST Framework < 3.0
Try something like this:
class ProductSerializer(serializers.Serializer):
...
classification = serializers.SerializerMethodField('get_classification')
def get_classification(self, obj):
return getattr(obj, 'classification', None)
Multiple Serializers
Another approach would be to create multiple serializers with different sets of fields. One serializer inherits from another and adds additional fields. Then you can choose the appropriate serializer in the view with the get_serializer_class method. Here's an actual example of how I use this approach to call different serializers to present different user data if the user object is the same as the request user.
def get_serializer_class(self):
""" An authenticated user looking at their own user object gets more data """
if self.get_object() == self.request.user:
return SelfUserSerializer
return UserSerializer
Removing fields from representation
Another approach that I've used in security contexts is to remove fields in the to_representation method. Define a method like
def remove_fields_from_representation(self, representation, remove_fields):
""" Removes fields from representation of instance. Call from
.to_representation() to apply field-level security.
* remove_fields: a list of fields to remove
"""
for remove_field in remove_fields:
try:
representation.pop(remove_field)
except KeyError:
# Ignore missing key -- a child serializer could inherit a "to_representation" method
# from its parent serializer that applies security to a field not present on
# the child serializer.
pass
and then in your serializer, call that method like
def to_representation(self, instance):
""" Apply field level security by removing fields for unauthorized users"""
representation = super(ProductSerializer, self).to_representation(instance)
if not permission_granted: # REPLACE WITH PERMISSION LOGIC
remove_fields = ('classification', )
self.remove_fields_from_representation(representation, remove_fields)
return representation
This approach is straightforward and flexible, but it comes at the cost of serializing fields that are sometimes not displayed. But that's probably okay.

The method describe below did the work for me.
Pretty simple,easy and worked for me.
DRF version used = djangorestframework (3.1.0)
class test(serializers.Serializer):
id= serializers.IntegerField()
name=serializers.CharField(required=False,default='some_default_value')

The serializers are deliberately designed to use a fixed set of fields so you wouldn't easily be able to optionally drop out one of the keys.
You could use a SerializerMethodField to either return the field value or None if the field doesn't exist, or you could not use serializers at all and simply write a view that returns the response directly.
Update for REST framework 3.0 serializer.fields can be modified on an instantiated serializer. When dynamic serializer classes are required I'd probably suggest altering the fields in a custom Serializer.__init__() method.

The serializers Charfield method has a property allow_blank
By default it is set to False.
Setting it to True will allow you to mark the field as optional during "serialization".
This is the code that you should write
classification = serializers.CharField(source="Classification", allow_blank=True)
Note: required property is used for deserialization.

DynamicSerializer for DRF 3, which allows dynamicly specifying which fields will be used in serializer, which will be excluded, and optionally which will become required!
Create Mixin
class DynamicSerializerMixin:
"""
A Serializer that takes an additional `fields` argument that
controls which fields should be used.
"""
def __init__(self, *args, **kwargs):
# Don't pass the 'fields' arg up to the superclass
fields = kwargs.pop("fields", None)
excluded_fields = kwargs.pop("excluded_fields", None)
required_fields = kwargs.pop("required_fields", None)
# Instantiate the superclass normally
super().__init__(*args, **kwargs)
if fields is not None:
# Drop any fields that are not specified in the `fields` argument.
allowed = set(fields)
existing = set(self.fields)
for field_name in existing - allowed:
self.fields.pop(field_name)
if isinstance(fields, dict):
for field, config in fields.items():
set_attrs(self.fields[field], config)
if excluded_fields is not None:
# Drop any fields that are not specified in the `fields` argument.
for field_name in excluded_fields:
self.fields.pop(field_name)
if required_fields is not None:
for field_name in required_fields:
self.fields[field_name].required = True
Initialize/adjust your serializer by adding DynamicSerializerMixin to inheritence
class UserProfileSerializer(DynamicSerializerMixin, serializers.ModelSerializer):
class Meta:
model = User
fields = (
"id",
'first_name', 'last_name'
"email",
"is_staff",
)
Use it :)
class RoleInvitationSerializer(serializers.ModelSerializer):
invited_by = UserProfileSerializer(fields=['id', 'first_name', 'last_name'])
or in action apis
#action(detail=True, serializer_class=YourSerialzierClass)
def teams_roles(self, request, pk=None):
user = self.get_object()
queryset = user.roles.all()
serializer = self.get_serializer(queryset, many=True, excluded_fields=['user'])
return Response(data=serializer.data)

For this purpose the serializers have the partial argument. If when the serializer is initialized you can pass partial=True. If you are using generics or mixins you can overrider the get_serializer function as follows:
def get_serializer(self, *args, **kwargs):
kwargs['partial'] = True
return super(YOUR_CLASS, self).get_serializer(*args, **kwargs)
And that will do the trick.
Note: This allows all fields to be optional and not only a specific one. If you want only specifics, you can override the method (i.e. update) and add validations of existence for various fields.

What has worked well for me is to set the serializer like so:
classification = serializers.CharField(max_length=20, allow_blank=True, default=None)

From the "it's a terrible hack relying on specific implementation details of both DRF and Django, but it works (at least for now)" files, here's the approach I used to include some additional debugging data in the response from a "create" method implementation on a serializer:
def create(self, validated_data)
# Actual model instance creation happens here...
self.fields["debug_info"] = serializers.DictField(read_only=True)
my_model.debug_info = extra_data
return my_model
This is a temporary approach that lets me use the browsable API to display some of the raw response data received from a particular remote service during the creation process. In the future, I'm inclined to keep this capability, but hide it behind a "report debugging info" flag in the creation request rather than returning the lower level info by default.

Django: 'unique_together' and 'blank=True'

I have a Django model which looks like this:
class MyModel(models.Model):
parent = models.ForeignKey(ParentModel)
name = models.CharField(blank=True, max_length=200)
... other fields ...
class Meta:
unique_together = ("name", "parent")
This works as expected; If there is the same name more than once in the same parent then I get an error: "MyModel with this Name and Parent already exists."
However, I also get an error when I save more than one MyModel with the same parent but with the name field blank, but this should be allowed. So basically I don't want to get the above error when the name field is blank. Is that possible somehow?

Firstly, blank (empty string) IS NOT same as null ('' != None).
Secondly, Django CharField when used through forms will be storing empty string when you leave field empty.
So if your field was something else than CharField you should just add null=True to it. But in this case you need to do more than that. You need to create subclass of forms.CharField and override it's clean method to return None on empty string, something like this:
class NullCharField(forms.CharField):
def clean(self, value):
value = super(NullCharField, self).clean(value)
if value in forms.fields.EMPTY_VALUES:
return None
return value
and then use it in form for your ModelForm:
class MyModelForm(forms.ModelForm):
name = NullCharField(required=False, ...)
this way if you leave it blank it will store null in database instead of empty string ('')

Using unique_together, you're telling Django that you don't want any two MyModel instances with the same parent and name attributes -- which applies even when name is an empty string.
This is enforced at the database level using the unique attribute on the appropriate database columns. So to make any exceptions to this behavior, you'll have to avoid using unique_together in your model.
Instead, you can get what you want by overriding the save method on the model and enforcing the unique restraint there. When you try to save an instance of your model, your code can check to see if there are any existing instances that have the same parent and name combination, and refuse to save the instance if there are. But you can also allow the instance to be saved if the name is an empty string. A basic version of this might look like this:
class MyModel(models.Model):
...
def save(self, *args, **kwargs):
if self.name != '':
conflicting_instance = MyModel.objects.filter(parent=self.parent, \
name=self.name)
if self.id:
# This instance has already been saved. So we need to filter out
# this instance from our results.
conflicting_instance = conflicting_instance.exclude(pk=self.id)
if conflicting_instance.exists():
raise Exception('MyModel with this name and parent already exists.')
super(MyModel, self).save(*args, **kwargs)
Hope that helps.

This solution is very similar to the one given by #bigmattyh, however, i found the below page which describes where the validation should be done:
http://docs.djangoproject.com/en/1.3/ref/models/instances/#validating-objects
The solution i ended up using is the following:
from django import forms
class MyModel(models.Model):
...
def clean(self):
if self.name != '':
instance_exists = MyModel.objects.filter(parent=self.parent,
name=self.name).exists()
if instance_exists:
raise forms.ValidationError('MyModel with this name and parent already exists.')
Notice that a ValidationError is raised instead of a generic exception. This solution has the benefit that when validating a ModelForm, using .is_valid(), the models .clean() method above is automatically called, and will save the ValidationError string in .errors, so that it can be displayed in the html template.
Let me know if you do not agree with this solution.

You can use constraints to set up a partial index like so:
class MyModel(models.Model):
parent = models.ForeignKey(ParentModel)
name = models.CharField(blank=True, max_length=200)
... other fields ...
class Meta:
constraints = [
models.UniqueConstraint(
fields=['name', 'parent'],
condition=~Q(name='')
name='unique_name_for_parent'
)
]
This allow constraints like UniqueTogether to only apply to certain rows (based on conditions you can define using Q).
Incidentally, this happens to be the Django recommended path forward as well: https://docs.djangoproject.com/en/3.2/ref/models/options/#unique-together
Some more documentation: https://docs.djangoproject.com/en/3.2/ref/models/constraints/#django.db.models.UniqueConstraint

bigmattyh gives a good explanation as to what is happening. I'll just add a possible save method.
def save(self, *args, **kwargs):
if self.parent != None and MyModels.objects.filter(parent=self.parent, name=self.name).exists():
raise Exception('MyModel with this name and parent exists.')
super(MyModel, self).save(*args, **kwargs)
I think I chose to do something similar by overriding my model's clean method and it looked something like this:
from django.core.exceptions import ValidationError
def clean(self):
if self.parent != None and MyModels.objects.filter(parent=self.parent, name=self.name).exists():
raise ValidationError('MyModel with this name and parent exists.')

How to show hidden autofield in django formset

A Django autofield when displayed using a formset is hidden by default. What would be the best way to show it?
At the moment, the model is declared as,
class MyModel:
locid = models.AutoField(primary_key=True)
...
When this is rendered using Django formsets,
class MyModelForm(ModelForm):
class Meta:
model = MyModel
fields = ('locid', 'name')
it shows up on the page as,
<input id="id_form-0-locid" type="hidden" value="707" name="form-0-locid"/>
Thanks.
Edit
I create the formset like this -
LocFormSet = modelformset_factory(MyModel)
pformset = LocFormSet(request.POST, request.FILES, queryset=MyModel.objects.order_by('name'))
Second Edit
Looks like I'm not using the custom form class I defined there, so the question needs slight modification..
How would I create a formset from a custom form (which will show a hidden field), as well as use a custom queryset?
At the moment, I can either inherit from a BaseModelFormSet class and use a custom query set, or I can use the ModelForm class to add a custom field to a form. Is there a way to do both with a formset?
Third Edit
I'm now using,
class MyModelForm(ModelForm):
def __init__(self, *args, **kwargs):
super(MyModelForm, self).__init__(*args, **kwargs)
locid = forms.IntegerField(min_value = 1, required=True)
self.fields['locid'].widget.attrs["type"] = 'visible'
self.queryset = MyModel.objects.order_by('name')
class Meta:
model = MyModel
fields = ('locid', 'name')
LocFormSet = modelformset_factory(MyModel, form = MyModelForm)
pformset = LocFormSet()
But this still doesn't
Show locid
Use the custom query that was specified.

Try changing the default field type:
from django import forms
class MyModelForm(ModelForm):
locid = forms.IntegerField(min_value=1, required=True)
class Meta:
model = MyModel
fields = ('locid', 'name')
EDIT: Tested and works...

As you say, you are not using the custom form you have defined. This is because you aren't passing it in anywhere, so Django can't know about it.
The solution is simple - just pass the custom form class into modelformset_factory:
LocFormSet = modelformset_factory(MyModel, form=MyModelForm)
Edit in response to update 3:
Firstly, you have the redefinition for locid in the wrong place - it needs to be at the class level, not inside the __init__.
Secondly, putting the queryset inside the form won't do anything at all - forms don't know about querysets. You should go back to what you were doing before, passing it in as a parameter when you instantiate the formset. (Alternatively, you could define a custom formset, but that seems like overkill.)
class MyModelForm(ModelForm):
locid = forms.IntegerField(min_value=1, required=True)
def __init__(self, *args, **kwargs):
super(MyModelForm, self).__init__(*args, **kwargs)
self.fields['locid'].widget.attrs["type"] = 'visible'
class Meta:
model = MyModel
fields = ('locid', 'name')
LocFormSet = modelformset_factory(MyModel, form = MyModelForm)
pformset = LocFormSet(request.POST, request.FILES,
queryset=MyModel.objects.order_by('name')))

Okay, none of the approaches above worked for me. I solved this issue from the template side, finally.
There is a ticket filed (http://code.djangoproject.com/ticket/10427), which adds a "value" option to a template variable for a form. For instance, it allows,
{{form.locid.value}}
to be shown. This is available as a patch, which can be installed in the SVN version of django using "patch -p0 file.patch"
Remember, the {{form.locid.value}} variable will be used in conjunction with the invisible form - otherwise, the submit and save operations for the formset will crash.
This is Not the same as {{form.locid.data}} - as is explained in the ticket referred to above.

The reason that the autofield is hidden, is that both BaseModelFormSet and BaseInlineFormSet override that field in add_field. The way to fix it is to create your own formset and override add_field without calling super. Also you don't have to explicitly define the primary key.
you have to pass the formset to modelformset_factory:
LocFormSet = modelformset_factory(MyModel,
formset=VisiblePrimaryKeyFormSet)
This is in the formset class:
from django.forms.models import BaseInlineFormSet, BaseModelFormSet, IntegerField
from django.forms.formsets import BaseFormSet
class VisiblePrimaryKeyFormset(BaseModelFormSet):
def add_fields(self, form, index):
self._pk_field = pk = self.model._meta.pk
if form.is_bound:
pk_value = form.instance.pk
else:
try:
pk_value = self.get_queryset()[index].pk
except IndexError:
pk_value = None
form.fields[self._pk_field.name] = IntegerField( initial=pk_value,
required=True) #or any other field you would like to display the pk in
BaseFormSet.add_fields(self, form, index) # call baseformset which does not modify your primary key field