Develop Reference

DRF serializer parsing comma-delimited string into a list field

Is there a way of modifying how DRF serializers parse incoming request payload?
I'm trying to let clients send a comma-delimited list as query parameter but receive it inside the serializer as a list instead but DRF keeps complaining. Right now I'm manually intercepting the request in the view and doing the parsing that field manually before passing it to the serializer which doesn't seem elegant to me.
What I'm doing right now
class ExampleSerializer(...):
list_field = serialzers.ListField(child=serializers.Integerfield(...))
# more fields
def view(request):
payload = request.GET
payload["list_field"] = str(payload.get("list_field", "")).split(",")
serializer = ExampleSerializer(data=payload)
What I'd prefer (using same serializer as above)
def view(request):
serializer = ExampleSerializer(data=request.GET)

The ListField will work with json, or with multi-value query strings or form bodies (as below). It does not parse comma separated strings.
This will work:
GET /path/?list_field=1&list_field=2&list_field=3
What you need is a custom field which implements your parsing logic: accept a string and split it using a separator (,, or :, etc), and then validate it using the child field.
There is no builtin field which works this way, but there is a great example GIST here which you can copy or reference when writing your own field. I have included some snippets from the gist, but as its not mine I don't feel comfortable copying the whole thing.
# https://gist.github.com/imomaliev/77fdfd0ab5f1b324f4e496768534737e
class CharacterSeparatedField(serializers.ListField):
def __init__(self, *args, **kwargs):
self.separator = kwargs.pop("separator", ",")
super().__init__(*args, **kwargs)
def to_internal_value(self, data):
data = data.split(self.separator)
return super().to_internal_value(data)
# continues ...
class TestCharacterSeparatedManyField:
def test_field_from_native_should_return_list_for_given_str(self):
field = CharacterSeparatedField(child=serializers.CharField())
assert field.to_internal_value("a,b,c") == ["a", "b", "c"]
You can also write a custom validate_{fieldname} function to modify the value. This at least keeps it in the serializer. A proper Field is better if possible, though, but this is a common pattern for one-off validation/transformations like this.
class ExampleSerializer(Serializer):
list_field = CharField()
def validate_list_field(self, value):
arr = value.split(",")
arr = [int(x) for x in arr if x.isdigit()]
if len(arr) == 0:
raise ValidationError("Supply at least 1 value.")
return arr

How can I reject a queryset object while serializing the data using a Django Rest ModelSerializer?

I have a queryset that I'm trying to serialize with my CasePartySerializer which inherits the Django REST Framework ModelSerializer class. It has a nested serializer called CaseRuleDocketSerializer.
Here is how I instantiate it.
def get_queryset(self):
self.queryset = CaseParty.objects.filter(user=self.request.user)
def get(self):
serial = CasePartySerializer(
list(
self.queryset[
self.offset:self.offset + self.limit
]
),
many=True,
context={
'tracked': self.tracked.value,
'request': self.request
}
)
Simple Enough, but what if I want to conditionally pass over and reject objects so they are not included in the finalized serial.data.
I'm sure there is probably some exception I can raise that would pass over this database object, but I'm unsure what that would be. I looked through the documentation without any luck; it's something that surprised me considering the quality of the REST Framework documentation. I'm probably missing something simple.
Here is my CasePartySerializer so you can see my conditionals. In this example you can see they are based on results from the nested serializer CaseRuleDocketSerializer output which is not available from the get_queryset method. If one of the ruledocket items is 'red' it does not need to be included in the serializer result. I know I can filter in the get_queryset method also, but it just seems it would be easier to do in the serializer itself.
class CaseRuleDocketSerializer(serializers.ModelSerializer):
table_row_color = serializers.SerializerMethodField()
class Meta:
model = CaseRuleDocket
fields = [
'unique_id',
'created_date',
'updated_date',
'date_time',
'entry',
'party'
]
def get_is_user_only_created(self, obj):
if obj.is_user_created and not obj.is_court_created:
return 'green'
elif obj.is_court_created and not obj.is_user_created:
return 'blue'
else:
return 'red'
class CasePartySerializer(serializers.ModelSerializer):
docket_of_party = CaseRuleDocketSerializer(many=True, read_only=True)
table_row_color = serializers.SerializerMethodField()
class Meta:
model = CaseParty
fields = [
'is_tracked',
'created_date',
'updated_date',
'serve_status',
'serve_status_date',
'defendant',
'plaintiff',
# CONTINUE WITH THE REST OF THE FIELDS....
]
def get_table_row_color(self, obj):
errors = [
x.table_row_color for x in self.obj.docket_party
]
if 'blue' in errors:
return 'blue'
elif 'green' in errors:
return 'green'
else:
# HERE IS WHERE I WANT TO EXCEPT AND REMOVE
# THIS CaseParty OBJECT INSTANCE
I have been using stackoverflow for many years and I have always found the answers I have needed when looking. For some reason I could not find a way to frame the question correctly. Also, I am familiar with the documentation and have read it thoroughly. If it is not suggested to filter inside of the queryset due to efficiency, readability, or some other missight on my part, please include the reason in your answer.
Let me know if you need clarification.
Thanks!

Based on your example, where you are filtering all CaseParty instances that have no related CaseTrack models, your best bet would be to update the view's get_queryset method.
Basically, rather than using queryset = CaseParty.objects.all(), you could write your own get_queryset(self) method, which will filter out the unwanted models.
To give a more concrete example:
class MyViewSet(ModelViewSet):
def get_queryset(self):
# We fetch all party instances
party_instances = CaseParty.objects.all()
# We fetch all case_tracks linked to them
case_tracks = CaseTrack.objects.get(
user=self.request.user,
case__in=party_instances
)
# We extract the party ids from case tracks
party_ids_from_casetracks = {item.case.id for item in case_tracks}
# We only keep those parties in the queryset
party_instances_with_tracks = [item for item in party_instances if item.id in party_ids_from_casetracks ]
return party_instances_with_tracks
You'll find more example in the official documentation
If filterting at the viewset level is problematic because different endpoints/views must have different filters, then simply filter then from within the view action.
As someone else mentionned in the comments, the serializer is not here to filter the data:
The view filters the data (queryset, get_queryset, filter_queryset, detail view, etc)
The serializer receives the data, validates it, create/update/destroy instances, return representation and internal values

get_FIELD_serializer in Django Rest Framework

I'm in a situation where I want to change the serializer field depending on a condition. Where the condition comes doesn't matter but I want to be able to switch between serializer fields as the following example:
class EntrySerializer(serializers.ModelSerializer):
# author defined from ModelSerializer
def get_author_serializer(self):
request = self.context.get('request')
GET = getattr(request, 'GET', {})
if request and GET and GET.get('include_author')=='true':
author_serializer = UserSerializer()
else:
author_serializer = serializers.PrimaryKeyRelatedField(
read_only=True, default=serializers.CurrentUserDefault()
)
return author_serialize
Of course this doesn't work because get_FIELD_serializer doesn't exist, but I'm looking for the simplest solution that can do this.
I've tried writing author as a property in a naive attempt but it didn't work.
I am aware that I can write multiple EntrySerializers and use get_serializer_class but that is just too much boilerplate code for such a small customization.

If you just want to change a serializers' field based on a condition, you could do something like this.
class MySerializer(serializers.ModelSerializer):
author = serializers.SerializerMethodField()
def get_author(self, instance):
# your condition here
if your_condition:
return instance.author
return 'hello'
Check the docs for SerializerMethodField
https://www.django-rest-framework.org/api-guide/fields/#serializermethodfield

django serializers to json - custom json output format

I am quite new to django and recently I have a requirement of a JSON output, for which I use the following django code:
data = serializers.serialize("json", Mymodel.objects.all())
It works great, except that I get a output of:
[{"pk": 8970859016715811, "model": "myapp.mymodel", "fields": {"reviews": "3.5", "title": .....}}]
However, I would like the output to be simply either:
[{"reviews": "3.5", "title": .....}]
or,
[{"id": "8970859016715811", "reviews": "3.5", "title": .....}]
I was wondering if someone could point me to the right direction as to how to achieve this.

You can add 'fields' parameter to the serialize-function, like this:
data = serializers.serialize('xml', SomeModel.objects.all(), fields=('name','size'))
See: https://docs.djangoproject.com/en/dev/topics/serialization/
EDIT 1:
You can customize the serializer to get only the fields you specify.
From Override Django Object Serializer to get rid of specified model:
from django.core.serializers.python import Serializer
class MySerialiser(Serializer):
def end_object( self, obj ):
self._current['id'] = obj._get_pk_val()
self.objects.append( self._current )
# views.py
serializer = MySerialiser()
data = serializer.serialize(some_qs)

You'll need to write a custom Json serializer. Something like this should do the trick:
class FlatJsonSerializer(Serializer):
def get_dump_object(self, obj):
data = self._current
if not self.selected_fields or 'id' in self.selected_fields:
data['id'] = obj.id
return data
def end_object(self, obj):
if not self.first:
self.stream.write(', ')
json.dump(self.get_dump_object(obj), self.stream,
cls=DjangoJSONEncoder)
self._current = None
def start_serialization(self):
self.stream.write("[")
def end_serialization(self):
self.stream.write("]")
def getvalue(self):
return super(Serializer, self).getvalue()
The you can use it like this:
s = FlatJsonSerializer()
s.serialize(MyModel.objects.all())
Or you could register the serializer with django.core.serializers.register_serializer and then use the familiar serializers.serialize shortcut.
Take a look at the django implementation as a reference if you need further customization: https://github.com/django/django/blob/master/django/core/serializers/json.py#L21-62

I just came across this as I was having the same problem. I also solved this with a custom serializer, tried the "EDIT 1" method but it didn't work too well as it stripped away all the goodies that the django JSON encoder already did (decimal, date serialization), which you can rewrite it yourself but why bother. I think a much less intrusive way is to inherit the JSON serializer directly like this.
from django.core.serializers.json import Serializer
from django.utils.encoding import smart_text
class MyModelSerializer(Serializer):
def get_dump_object(self, obj):
self._current['id'] = smart_text(obj._get_pk_val(), strings_only=True)
return self._current
Sso the main culprit that writes the fields and model thing is at the parent level python serializer and this way, you also automatically get the fields filtering that's already built into django's JSON serializer. Call it like this
serializer = MyModelSerializer()
data = serializer.serialize(<queryset>, <optional>fields=('field1', 'field2'))

import json
_all_data = Reporter.objects. all()
json_data = json.dumps([{'name': reporter.full_name} for reporter in _all_data])
return HttpResponse(json_data, content_type='application/json')
Here Reporter is your Model

Case insensitive unique model fields in Django?

I have basically a username is unique (case insensitive), but the case matters when displaying as provided by the user.
I have the following requirements:
field is CharField compatible
field is unique, but case insensitive
field needs to be searchable ignoring case (avoid using iexact, easily forgotten)
field is stored with case intact
preferably enforced on database level
preferably avoid storing an extra field
Is this possible in Django?
The only solution I came up with is "somehow" override the Model manager, use an extra field, or always use 'iexact' in searches.
I'm on Django 1.3 and PostgreSQL 8.4.2.

As of Django 1.11, you can use CITextField, a Postgres-specific Field for case-insensitive text backed by the citext type.
from django.db import models
from django.contrib.postgres.fields import CITextField
class Something(models.Model):
foo = CITextField()
Django also provides CIEmailField and CICharField, which are case-insensitive versions of EmailField and CharField.

Store the original mixed-case string in a plain text column. Use the data type text or varchar without length modifier rather than varchar(n). They are essentially the same, but with varchar(n) you have to set an arbitrary length limit, that can be a pain if you want to change later. Read more about that in the manual or in this related answer by Peter Eisentraut #serverfault.SE.
Create a functional unique index on lower(string). That's the major point here:
CREATE UNIQUE INDEX my_idx ON mytbl(lower(name));
If you try to INSERT a mixed case name that's already there in lower case you get a unique key violation error.
For fast equality searches use a query like this:
SELECT * FROM mytbl WHERE lower(name) = 'foo' --'foo' is lower case, of course.
Use the same expression you have in the index (so the query planner recognizes the compatibility) and this will be very fast.
As an aside: you may want to upgrade to a more recent version of PostgreSQL. There have been lots of important fixes since 8.4.2. More on the official Postgres versioning site.

With overriding the model manager, you have two options. First is to just create a new lookup method:
class MyModelManager(models.Manager):
def get_by_username(self, username):
return self.get(username__iexact=username)
class MyModel(models.Model):
...
objects = MyModelManager()
Then, you use get_by_username('blah') instead of get(username='blah'), and you don't have to worry about forgetting iexact. Of course that then requires that you remember to use get_by_username.
The second option is much hackier and convoluted. I'm hesitant to even suggest it, but for completeness sake, I will: override filter and get such that if you forget iexact when querying by username, it will add it for you.
class MyModelManager(models.Manager):
def filter(self, **kwargs):
if 'username' in kwargs:
kwargs['username__iexact'] = kwargs['username']
del kwargs['username']
return super(MyModelManager, self).filter(**kwargs)
def get(self, **kwargs):
if 'username' in kwargs:
kwargs['username__iexact'] = kwargs['username']
del kwargs['username']
return super(MyModelManager, self).get(**kwargs)
class MyModel(models.Model):
...
objects = MyModelManager()

As of December 2021, with the help of Django 4.0 UniqueConstraint expressions you can add a Meta class to your model like this:
class Meta:
constraints = [
models.UniqueConstraint(
Lower('<field name>'),
name='<constraint name>'
),
]
I'm by no mean a Django professional developer and I don't know technical considerations like performance issues about this solution. Hope others comment on that.

Since a username is always lowercase, it's recommended to use a custom lowercase model field in Django. For the ease of access and code-tidiness, create a new file fields.py in your app folder.
from django.db import models
from django.utils.six import with_metaclass
# Custom lowecase CharField
class LowerCharField(with_metaclass(models.SubfieldBase, models.CharField)):
def __init__(self, *args, **kwargs):
self.is_lowercase = kwargs.pop('lowercase', False)
super(LowerCharField, self).__init__(*args, **kwargs)
def get_prep_value(self, value):
value = super(LowerCharField, self).get_prep_value(value)
if self.is_lowercase:
return value.lower()
return value
Usage in models.py
from django.db import models
from your_app_name.fields import LowerCharField
class TheUser(models.Model):
username = LowerCharField(max_length=128, lowercase=True, null=False, unique=True)
End Note : You can use this method to store lowercase values in the database, and not worry about __iexact.

You can use citext postgres type instead and not bother anymore with any sort of iexact. Just make a note in model that underlying field is case insensitive.
Much easier solution.

You can use lookup='iexact' in UniqueValidator on serializer, like this:
Unique model field in Django and case sensitivity (postgres)

I liked Chris Pratt's Answer but it didn't worked for me, because the models.Manager-class doesn't have the get(...) or filter(...) Methods.
I had to take an extra step via a custom QuerySet:
from django.contrib.auth.base_user import BaseUserManager
from django.db.models import QuerySet
class CustomUserManager(BaseUserManager):
# Use the custom QuerySet where get and filter will change 'email'
def get_queryset(self):
return UserQuerySet(self.model, using=self._db)
def create_user(self, email, password, **extra_fields):
...
def create_superuser(self, email, password, **extra_fields):
...
class UserQuerySet(QuerySet):
def filter(self, *args, **kwargs):
if 'email' in kwargs:
# Probably also have to replace...
# email_contains -> email_icontains,
# email_exact -> email_iexact,
# etc.
kwargs['email__iexact'] = kwargs['email']
del kwargs['email']
return super().filter(*args, **kwargs)
def get(self, *args, **kwargs):
if 'email' in kwargs:
kwargs['email__iexact'] = kwargs['email']
del kwargs['email']
return super().get(*args, **kwargs)
This worked for me in a very simple case but is working pretty good so far.

You can also override get_prep_value() and reuse it through inheritance.
class LowerCaseField:
def get_prep_value(self, value):
value = super().get_prep_value(value)
if value:
value = value.strip().lower()
return value
class LowerSlugField(LowerCaseField, models.SlugField):
pass
class LowerEmailField(LowerCaseField, models.EmailField):
pass
class MyModel(models.Model):
email = LowerEmailField(max_length=255, unique=True)
This way, if you ever want to reuse this field in another model, you can use the same consistent strategy.
From Django Docs:
get_prep_value(value)
value is the current value of the model’s
attribute, and the method should return data in a format that has been
prepared for use as a parameter in a query.
See Converting Python objects to query values for usage.