Is there a way of modifying how DRF serializers parse incoming request payload?
I'm trying to let clients send a comma-delimited list as query parameter but receive it inside the serializer as a list instead but DRF keeps complaining. Right now I'm manually intercepting the request in the view and doing the parsing that field manually before passing it to the serializer which doesn't seem elegant to me.
What I'm doing right now
class ExampleSerializer(...):
list_field = serialzers.ListField(child=serializers.Integerfield(...))
# more fields
def view(request):
payload = request.GET
payload["list_field"] = str(payload.get("list_field", "")).split(",")
serializer = ExampleSerializer(data=payload)
What I'd prefer (using same serializer as above)
def view(request):
serializer = ExampleSerializer(data=request.GET)
The ListField will work with json, or with multi-value query strings or form bodies (as below). It does not parse comma separated strings.
This will work:
GET /path/?list_field=1&list_field=2&list_field=3
What you need is a custom field which implements your parsing logic: accept a string and split it using a separator (,, or :, etc), and then validate it using the child field.
There is no builtin field which works this way, but there is a great example GIST here which you can copy or reference when writing your own field. I have included some snippets from the gist, but as its not mine I don't feel comfortable copying the whole thing.
# https://gist.github.com/imomaliev/77fdfd0ab5f1b324f4e496768534737e
class CharacterSeparatedField(serializers.ListField):
def __init__(self, *args, **kwargs):
self.separator = kwargs.pop("separator", ",")
super().__init__(*args, **kwargs)
def to_internal_value(self, data):
data = data.split(self.separator)
return super().to_internal_value(data)
# continues ...
class TestCharacterSeparatedManyField:
def test_field_from_native_should_return_list_for_given_str(self):
field = CharacterSeparatedField(child=serializers.CharField())
assert field.to_internal_value("a,b,c") == ["a", "b", "c"]
You can also write a custom validate_{fieldname} function to modify the value. This at least keeps it in the serializer. A proper Field is better if possible, though, but this is a common pattern for one-off validation/transformations like this.
class ExampleSerializer(Serializer):
list_field = CharField()
def validate_list_field(self, value):
arr = value.split(",")
arr = [int(x) for x in arr if x.isdigit()]
if len(arr) == 0:
raise ValidationError("Supply at least 1 value.")
return arr
Related
First of all, I'm very new to Django world, there could be a similar question, however i did not find a satisfactory answer.
Here is my scenario, i have few external REST endpoints, which I will hit from my Django app and get say 100-key JSON response. Now, when I'm writing my API in Django app, this response i'll have to trim and send it to outer world. Say for example,
My API is,
GET /api/profiles/1472
which will give user profile with id 1472. Now, this API will inturn call some other REST endpoint and fetch actual profile's data. So, in a way I'm writing a proxy endpoint. This proxy endpoint is supposed to trim out some fields and give it back to caller.
I've not written model classes for this.
What are best ways to achieve this in Django?
Edit 1:
Sample view will be like this,
class GetCompetitorProductsView(APIView):
"""
Get Competitor products view
"""
def post(self, request, format=None):
# I'll be having a list of fields to be trimmed from response.
# It will be separate for every API.
data = request.data
error_checks = system_errors.check_for_competitor_products_input_error(data)
if not error_checks:
response = call_to_rest(data)
return Response(response)
else :
return Response(error_checks, status = status.HTTP_412_PRECONDITION_FAILED)
And one more thing, same behavior is applied to all other APIs. So, I need more generic solution which can be easily applied to other APIs.
Basically this is how to filter in python
allowed_fields = ("first_name", "last_name", "email")
user_info = call_rest_endpoint(id=1472)
result = {key:value for key,value in user_info.items() if key in allowed_fields}
First line define what fields u want to return.
Second line call the endpoint and get the data from theird party API.
Third line consist of 3 statements.
user_info.items() convert dictionary into array key/values paris.
Build dictionary from these tuples
but only if the key was found in allowed_fields tuple
You can create function or mixin that you will put in parents of your view and then use it method for trimming. Here is example
class TrimDataMixin(object):
ALLOWED_FIELDS = None
def trim_data(self, data):
allowed_fields = self.ALLOWED_FIELDS or []
return {k: v for k, v in data.items() if k in allowed_fields}
class GetCompetitorProductsView(TrimDataMixin, APIView):
"""
Get Competitor products view
"""
ALLOWED_FIELDS = ['first_name', 'last_name']
def post(self, request, format=None):
# I'll be having a list of fields to be trimmed from response.
# It will be separate for every API.
data = request.data
error_checks = system_errors.check_for_competitor_products_input_error(data)
if not error_checks:
response = call_to_rest(data)
# trim data
response = self.trim_data(response)
return Response(response)
else:
return Response(error_checks, status = status.HTTP_412_PRECONDITION_FAILED)
I am facing a very weird issue today.
Here is my serializer class.
class Connectivity(serializers.Serializer):
device_type = serializers.CharField(max_length=100,required=True)
device_name = serializers.CharField(max_length=100,required=True)
class Connections(serializers.Serializer):
device_name = serializers.CharField(max_length=100,required=True)
connectivity = Connectivity(required = True, many = True)
class Topologyserializer(serializers.Serializer):
name = serializers.CharField(max_length=100,required=True, \
validators=[UniqueValidator(queryset=Topology.objects.all())])
json = Connections(required=True,many=True)
def create(self, validated_data):
return validated_data
I am calling Topologyserializer from a Django view and I am passing a json like:
{
"name":"tokpwol",
"json": [
]
}
As per my experience with DRF since I have mentioned required = True in json field it should not accept the above json.
But I am able to create record.
Can anyone suggest me why it is not validating the json field and how it accepting empty list as json field?
I am using django rest framework 3.0.3
DRF does not clearly state what required stands for for lists.
In its code, it appears that validation passes as long as a value is supplied, even if that value is an empty list.
If you want to ensure the list is not empty, you'll need to validate its content manually. You would do that by adding the following method on your TopologySerializer:
def validate_json(self, value):
if not value:
raise serializers.ValidationError("Connections list is empty")
return value
I cannot test it right now, but it should work.
I tried to describe it best I could in the title, but basically, I want to write an API using the Django REST Framework, but instead of using the Django db and pre defining models, I want my API to take an HTTP call from the user, use that to call another libraries functions, take the objects the 3rd party lib returns, build models based on what it gets back, serialize to JSON and give it back to the caller in JSON.
right now I'm using an extremeley simple class adn function to test this concept. it's got an object definition and a function that reads from a text file and converts it into an object list:
class myObj:
id = None
port = None
cust = None
product = None
def __init__(self, textLine):
props = [x.strip() for x in textLine.split(',')]
self.id = props[0]
self.port = props[1]
self.cust = props[2]
self.product = props[3]
def getObjList():
lines = [line.strip() for line in open("objFile.txt")]
objList = [myObj(x) for x in lines]
return objList
I want my Django REST project to call that getObjList function when I try to access the associated URL in a browser (or call via curl or somethig), build a model based on the object it gets back, create a list of that model, serialize it and give it back to me so I can view it in the browsable web interface. Is this possible or am I being an idiot?
Thanks, I've been a C# developer for a bit now but now working in Python and with this HTTP stuff is a bit overwhelming.
If anyone cares I figured it out, I had to skip models entirely and just build the serializer directly based on the object I got back, and I had to revert to using more basic django views.
Here is the view:
#api_view(['GET'])
def ObjView(request):
if request.method == 'GET':
objList = myObj.getObjList()
dynamic_serializer = SerializerFactory.first_level(objList)
return Response(dynamic_serializer.data)
the getObjList function is the one posted in my question, but this should work with any function and any object that gets returned, here is what goes on in the serializer factory:
from rest_framework import serializers
def first_level(cur_obj):
isList = False
ser_val = cur_obj
if type(cur_obj) in {list, tuple}:
isList = True
ser_val = cur_obj[0]
dynamic_serializer = create_serializer(ser_val)
return dynamic_serializer(cur_obj, many=isList)
def create_serializer(cur_obj):
if type(cur_obj) in {list, tuple}:
if hasattr(cur_obj[0], "__dict__"):
cur_ser = create_serializer(cur_obj[0])
return cur_ser(many=True)
else:
return serializers.ListField(child=create_serializer(cur_obj[0]))
elif type(cur_obj) == dict:
if hasattr(cur_obj.values()[0], "__dict__"):
child_ser = create_serializer(cur_obj.values()[0])
return serializers.DictField(child=child_ser())
else:
return serializers.DictField(child=create_serializer(cur_obj.values()[0]))
elif hasattr(cur_obj, "__dict__"):
attrs = {}
for key, val in cur_obj.__dict__.items():
if "__" not in key:
cur_field = create_serializer(val)
if hasattr(val, "__dict__"):
attrs.update({key: cur_field()})
else:
attrs.update({key: cur_field})
return type(cur_obj.__name__ + "Serializer", (serializers.Serializer,), attrs)
else:
return serializers.CharField(required=False, allow_blank=True, max_length=200)
as you can see i had to break it into 2 pieces (I'm sure there's a way to make it one but it wasn't worth it to me to spend time on it) and it involved a substantial amount of recursion and just generally playing with the different field types, this should be good for serializing any combination of objects, lists, dictionaries and simple data types. I've tested it on a pretty wide array of objects and it seems pretty solid.
I have custom ZIPCodeField, but it require country to be given to work properly,
it's easy for validation where i can just save it in _country attribute of the form in form's init(with fallback to country field in the same form), it looks like:
class ZIPCodeField(wtforms.TextField):
def pre_validate(self, form):
if not self.data:
return
country = country = getattr(form, '_country', form.data.get('country'))
country = country.upper()
if not validate_zip_code(self.data, country):
raise ValueError(self.gettext(u'Invalid ZIP code.'))
but there is a problem with for process_formdata method (where I want to pass the received data trough simple filter to format the ZIP code correctly), we have no form instance, so seems like there are 2 solutions:
Saving country on field level, like:
class ZIPCodeField(wtforms.TextField):
def process_formdata(self, valuelist):
if valuelist:
self.data = format_zip_code(valuelist[0], self._country)
else:
self.data = ''
class TestForm(wtforms.Form):
zip_code = ZIPCodeField()
form = TestForm(MultiDict([('zip_code', '123455')]))
form.zip_code._country = u'US'
Or, overide process method and pass my extra value to it's data argument, like:
class ZIPCodeField(wtforms.TextField):
def process(self, formdata, data):
# we picking country value from data here
pass
form = TestForm(MultiDict([('zip_code', '123455')]),
zip_code=zip_code={'country': u'US'})
Which of these is proper solution? or there is better solution?
All of them will work but in general I would advise you too keep data access logic out of your forms. This makes for a bad separation of concerns.
You forms have a very specific job and that is to validate the form submission. In your case you want to augment the data submitted in form then validate it. This doesn't require any overriding of wtforms process methods. I would do the country retrieval either in a middleware or in the view, append it to the form's submission data, then validate the form as normal. If the zipcode didn't return a valid country or wasn't found I would abort(400) right there.
I am quite new to django and recently I have a requirement of a JSON output, for which I use the following django code:
data = serializers.serialize("json", Mymodel.objects.all())
It works great, except that I get a output of:
[{"pk": 8970859016715811, "model": "myapp.mymodel", "fields": {"reviews": "3.5", "title": .....}}]
However, I would like the output to be simply either:
[{"reviews": "3.5", "title": .....}]
or,
[{"id": "8970859016715811", "reviews": "3.5", "title": .....}]
I was wondering if someone could point me to the right direction as to how to achieve this.
You can add 'fields' parameter to the serialize-function, like this:
data = serializers.serialize('xml', SomeModel.objects.all(), fields=('name','size'))
See: https://docs.djangoproject.com/en/dev/topics/serialization/
EDIT 1:
You can customize the serializer to get only the fields you specify.
From Override Django Object Serializer to get rid of specified model:
from django.core.serializers.python import Serializer
class MySerialiser(Serializer):
def end_object( self, obj ):
self._current['id'] = obj._get_pk_val()
self.objects.append( self._current )
# views.py
serializer = MySerialiser()
data = serializer.serialize(some_qs)
You'll need to write a custom Json serializer. Something like this should do the trick:
class FlatJsonSerializer(Serializer):
def get_dump_object(self, obj):
data = self._current
if not self.selected_fields or 'id' in self.selected_fields:
data['id'] = obj.id
return data
def end_object(self, obj):
if not self.first:
self.stream.write(', ')
json.dump(self.get_dump_object(obj), self.stream,
cls=DjangoJSONEncoder)
self._current = None
def start_serialization(self):
self.stream.write("[")
def end_serialization(self):
self.stream.write("]")
def getvalue(self):
return super(Serializer, self).getvalue()
The you can use it like this:
s = FlatJsonSerializer()
s.serialize(MyModel.objects.all())
Or you could register the serializer with django.core.serializers.register_serializer and then use the familiar serializers.serialize shortcut.
Take a look at the django implementation as a reference if you need further customization: https://github.com/django/django/blob/master/django/core/serializers/json.py#L21-62
I just came across this as I was having the same problem. I also solved this with a custom serializer, tried the "EDIT 1" method but it didn't work too well as it stripped away all the goodies that the django JSON encoder already did (decimal, date serialization), which you can rewrite it yourself but why bother. I think a much less intrusive way is to inherit the JSON serializer directly like this.
from django.core.serializers.json import Serializer
from django.utils.encoding import smart_text
class MyModelSerializer(Serializer):
def get_dump_object(self, obj):
self._current['id'] = smart_text(obj._get_pk_val(), strings_only=True)
return self._current
Sso the main culprit that writes the fields and model thing is at the parent level python serializer and this way, you also automatically get the fields filtering that's already built into django's JSON serializer. Call it like this
serializer = MyModelSerializer()
data = serializer.serialize(<queryset>, <optional>fields=('field1', 'field2'))
import json
_all_data = Reporter.objects. all()
json_data = json.dumps([{'name': reporter.full_name} for reporter in _all_data])
return HttpResponse(json_data, content_type='application/json')
Here Reporter is your Model