Develop Reference

Best architecture for dynamically validating and saving field

I am looking for the good architecture for my problem. I am using django rest framework for building an API. I receive a list of dict which contains an id and a list of values. The list of values need to be validated according to the id.
Example of my code:
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField()
def validate(self, validated_data):
attribute = validated_data["attribute"]
values = validated_data["values"]
# This function returns the corresponding field according to attribute
values_child_field = get_values_field(attribute)
self.fields["values"].child = values_child_fields
new_values = self.fields["values"].run_child_validation(values)
set_value(validated_data, "values", new_values)
return validated_data
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I want to parse json like this:
{
"categorty_id": 42, # Category pk of the baseobject. which defines some constraints about attributes available
"attributes": [
{"id": 124, "values": ["value"]},
{"id": 321, "values": [42]},
{
"id": 18,
"values": [
{
"location": {"type": "Point", "geometry": {...}},
"address": "an address",
}
],
},
]
}
Currently, this code does not work. DRF seems to try to revalidate all values entries for each iteration with each child field. I do not understand why... I guess I could make it work without using this fields["values"] for making the validation and just retrieve the field and use it directly, but i need this field for making the save later.
Do you think my architecture is ok? What is the good way for parsing this type of data with DRF?
EDIT:
Structure of models are complex but a version simplified following:
class Attribute(models.Model):
class DataType(models.TextChoices):
TEXT = "TEXT", _("datatype_text")
INTEGER = "INTEGER", _("datatype_integer")
DATETIME = "DATETIME", _("datatype_datetime")
BOOL = "BOOL", _("datatype_bool")
# Some examples, but there are about 30 items with
# type very complicated like RecurrenceRule (RFC2445)
# or GeoJSON type
label = models.CharField()
category = models.ForeignKey(Category)
attribute_type = models.CharField(choices=DataType.choices)
class AttributeValue(models.Model):
attribute = models.ForeignKey(Attribute)
# a model which represents an object with list of attributes
baseobject = models.ForeignKey(BaseObject)
value = models.TextField()
AttributeValue is like a through table for manytomany relation between BaseObject model and Attribute model.
My JSON represents the list of attribute/values attached to a baseobject.
In fact I don't understand why DRf doesn't allow delegating registration in the child serializers of the parent serializer. This would allow much greater flexibility in code architecture and separation of responsibilities.
EDIT 2 :
My urls.py
router = routers.DefaultRouter()
router.register("baseobjects", BaseObjectViewSet, basename="baseobjects")
I am using the default router and url for DRF viewset.
The view looks like:
class BaseObjectViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
authentication_classes = [TokenAuthentication]
def create(self, request, *args, **kwargs):
serializer = BaseObjectApiInputSerializer(
data=request.data
)
if not serializer.is_valid():
return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
baseobject: BaseObject = serializer.save()
return Response(
{"results": [{"id": baseobject.pk}]}, status=HTTP_200_OK
)

I think you should use ListField with JSONField as child argument for values field.
validators = {
TinyurlShortener.DataType.TEXT: serializers.CharField(),
TinyurlShortener.DataType.INTEGER: serializers.IntegerField(),
TinyurlShortener.DataType.DATETIME: serializers.DateTimeField(),
TinyurlShortener.DataType.BOOL: serializers.BooleanField(),
}
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField(
child=serializers.JSONField()
)
def validate(self, attrs):
attribute = attrs.get('id')
field = validators[attribute.attribute_type]
for v in attrs['values']:
field.run_validation(json.loads(v.replace("'", '"')))
return super().validate(attrs)
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)

DRF queryset to return specific field

I'm creating a django rest framework application with this structure (assuming imports are correct, so I omit them from the code below.
models.py:
class Door(models.Model):
type = models.CharField(max_length=40)
color = models.CharField(max_length=40)
serializers.py:
class DoorSerializer(serializers.ModelSerializer):
class Meta:
model = Door
fields = ['type', 'color']
views.py:
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
queryset = Door.objects.all()
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
So far this behaves as intended, when I make an api call to localhost/Doors it lists all the doors. And when I make an api call to localhost/Doors/?type=big it lists all the doors that have the value "big" in their "type" field.
The addition I would like to make is another parameter check which would return a list of all the unique door types that exist in the database. This can be achieved in the manage.py shell by using: Door.objects.all().values('type').distinct()
My attempt was the following modifications to views.py:
...
parameter = self.request.query.params.get('type', '')
unique = self.request.query.params.get('unique', '')
if parameter:
...
elif unique:
return Door.objects.all().values('type').distinct()
...
My assumption was that this would return the same as Door.objects.all().values('type').distinct() when I make a call to localhost/Doors/?unique=whatever
However I am getting the error: "Got KeyError when attempting to get a value for field color on serializer DoorSerializer.\nThe serializer field might be named incorrectly and not match any attribute or key on the dict instance.\nOriginal exception text was: 'color'."
I assume this means that the serializer expects an object or a list of objects that contains all the fields of the corresponding model.
Is there some way I could circumvent this by fixing the view or should I create a different serializer? In either case, since I've gotten pretty confused with DRF / django differences and it is possible I won't be able to follow abstract instructions, could you provide a code solution that addresses the issue? Also, in the very likely case that my assumption is completely off, could you also explain what is causing the problem? Thank you for your time!
Edit for clarifying the desired result:
Assuming my database has 4 doors which are:
{
"id": 1,
"type": "big",
"color": "blue"
},
{
"id": 2,
"type": "big",
"color": "yellow"
},
{
"id": 3,
"type": "small",
"color": "green"
},
{
"id": 4,
"type": "big",
"color": "red"
},
I would like to make a get request to some url, for instance localhost/Doors/?unique=Yes and have the api return to me the list {"big", "small}

WRITING YOUR OWN VIEW: Short view that returns the list of type. You need to set up a new path here. I'd personally go for this option as the response you expect is way different to what the rest of your view does.
from rest_framework.decorators import api_view
from rest_framework.response import Response
#api_view()
def Unique_door_types(request):
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
WITHOUT AN ADDITIONAL VIEW:
No need for additional view or serializer. Override the list method. Note that this is closer to a trick than to a good way of programming.
from rest_framework.response import Response
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
def list(self, request):
unique = self.request.query_params.get('unique', '')
if unique:
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
return super().list()

My suggestion would be to create a separate route like /doors/types/. You do this by adding a method to your DoorViewSet class with a #action decorator. See https://www.django-rest-framework.org/api-guide/viewsets/#marking-extra-actions-for-routing for more details about how to do this.

Do not create sub-objects in django

I have a model representing a Status - and have a foreign key to Status from an Object model. I want to be able to create new objects, but do not want to allow the possibility of creating any more Status entries (there are a set of 5 pre-defined ones that are migrated into the database). I believe I have figured out how to structure serializers in such a way as to only re-use existing Status entries, but I'm not sure if it is the best way to go about doing something like this...
Some Simplified Code:
class StatusSerializer(serializers.ModelSerializer):
class Meta:
model = Status
fields = ('name',)
def to_representation(self, obj):
return obj.name
def to_internal_value(self, data):
return {
'name': data
}
class ObjectSerializer(serializers.ModelSerializer):
status = StatusSerializer(read_only=True)
class Meta:
model = Object
fields = ('obj_name', 'status',)
def create(self, validated_data):
# We do not want to create new statuses - only use existing ones
status = Status.objects.get(name=self.initial_data['status'])
return Object.objects.create(status=status, **validated_data)
def update(self, instance, validated_data):
instance.obj_name = validated_data.get('obj_name', instance.obj_name)
# We do not want to create new statuses - only use existing ones
instance.status = Status.objects.get(name=self.initial_data['status']) if 'status' in self.initial_data else instance.status
return instance
As seen above, I also flatten out the Status object when it is displayed - e.g. I turn the following
{
'obj_name': 'ObjectName',
'status': {
'name': 'StatusName'
}
}
Into this
{
'obj_name': 'ObjectName',
'status': 'StatusName'
}
This seems to work fine -- however I am not sure how to handle cases where a user of the API gives me an invalid Status name. Right now, the api would bubble up a Status.DoesNotExist exception from one of the Status.objects.get(...) requests - should I just catch that and re-raise as something the serializer/view would expect?
Thanks!
Edit: Realized my question wasnt exactly clear...
Is the above a good way to prohibit creation of Status objects - and enforce that any Object created will use one of those statuses?
What is the best way to handle the case where a user attempts to create an Object with an invalid Status name?

You can use validate method in the ObjectSerializer.
class ObjectSerializer(serializers.ModelSerializer):
status = StatusSerializer(read_only=True)
class Meta:
model = Object
fields = ('obj_name', 'status',)
def validate(self, attrs):
validated_data = super().validate(attrs)
status = self.initial_data.get('status')
# Here assuming that None is not the valid value for status
if status is not None:
status_obj = Status.objects.filter(name=status)
if not status_obj:
raise serializer.ValidationError('Invalid Status')
status_obj = status_obj[0]
validated_data['status'] = status_obj
return validated_data
# Nothing special to be done in create/update since we are sending data
# in validated_data which will directly sent to the instance.
# def create(self, validated_data):
# We do not want to create new statuses - only use existing ones
# status = Status.objects.get(name=self.initial_data['status'])
#return Object.objects.create(status=status, **validated_data)
#def update(self, instance, validated_data):
# instance.obj_name = validated_data.get('obj_name', instance.obj_name)
# We do not want to create new statuses - only use existing ones
# instance.status = Status.objects.get(name=self.initial_data['status']) if 'status' in self.initial_data else instance.status
# return instance
And if you can use different keys for write and read status, then you can modify ObjectSerializer like this:
class ObjectSerializer(serializer.ModelSerializer):
status = serializer.SlugRelatedField(slug_field='name', queryset=Status.objects.all(), write_only=True)
status_data = StatusSerializer(read_only=True, source='status')
class Meta:
model = Object
fields = ('obj_name', 'status', 'status_data')
In this case, if you pass {'obj_name': 'ObjectName', 'status': 'StatusName'} data to serializer, the serializer will first check for the StatusName value in the name field of the provided queryset (In this case we are using all) and if not valid raises ValidationError. If valid, then saves the status in the field of the instance.

Django & Mongoengine get data from embedded document?

I am having trouble retrieving data from the embedded document in mongoengine.
models.py
from mongoengine import Document, EmbeddedDocument, fields
class ProductFields(EmbeddedDocument):
key_name = fields.StringField(required=True)
value = fields.DynamicField(required=True)
class Product(Document):
name = fields.StringField(required=True)
description = fields.StringField(required=True, null=True)
fields = fields.ListField(fields.EmbeddedDocumentField(ProductFields))
views.py
class ProductListView(APIView):
def get(self, request):
# list_products = Product.objects.all()
result=[]
productfields = ProductFields
for product in Product.objects:
data={
"name":product.name,
"description":product.description,
# "key":product.fields.key_name,
# "value":ProductFields.value,
}
print (data)
# print(productfields.key_name)
result.append(data)
return Response({"products":result,"message":"list of products.","requestStatus":1},status=status.HTTP_200_OK)
Output:
{
"description": "test description",
"name": "product1"
"fields":[
{ "key_name" : value},
{ "key_name" : value},
]
}
How do I get the above-desired output? Print function doesn't work because mongoengine returns object and not the value.

Here I see you're using APIView from Django Rest FrameWork. Have a look at django-rest-framework-mongoengine. If you're already familiar with DRF, you can use this extension to create your API endpoints with MongoDB easily.
You must have found some workaround by now even though you can mark this answer as correct so that if anyone else runs into the same problem in future, they can get the solution.

Django Rest Framework receive primary key value in POST and return model object as nested serializer

I'm not completely sure that the title of my question is as specific as I wanted it to be, but this is the case:
I have a HyperlinkedModelSerializer that looks like this:
class ParentArrivalSerializer(serializers.HyperlinkedModelSerializer):
carpool = SchoolBuildingCarpoolSerializer()
class Meta:
model = ParentArrival
As you can see the carpool is defined as a nested serializer object and what I want is to be able to make a POST request to create a ParentArrival in this way (data as application/json):
{
...
"carpool": "http://localhost:8000/api/school-building-carpools/10/"
...
}
And receive the data in this way:
{
"carpool": {
"url": "http://localhost:8000/api/school-building-carpools/10/"
"name": "Name of the carpool",
...
}
}
Basically, I'm looking for a way to deal with nested serializers without having to send data as an object (but id or url in this case) in POST request, but receiving the object as nested in the serialized response.

I have been happy with my previous solution, but decided to look again and I think I have another solution that does exactly what you want.
Basically, you need to create your own custom field, and just overwrite the to_representation method:
class CarpoolField(serializers.PrimaryKeyRelatedField):
def to_representation(self, value):
pk = super(CarpoolField, self).to_representation(value)
try:
item = ParentArrival.objects.get(pk=pk)
serializer = CarpoolSerializer(item)
return serializer.data
except ParentArrival.DoesNotExist:
return None
def get_choices(self, cutoff=None):
queryset = self.get_queryset()
if queryset is None:
return {}
return OrderedDict([(item.id, str(item)) for item in queryset])
class ParentArrivalSerializer(serializers.HyperlinkedModelSerializer):
carpool = CarpoolField(queryset=Carpool.objects.all())
class Meta:
model = ParentArrival
This will allow you to post with
{
"carpool": 10
}
and get:
{
"carpool": {
"url": "http://localhost:8000/api/school-building-carpools/10/"
"name": "Name of the carpool",
...
}
}

It's simple.
As you know, Django appends "_id" to the field name in the ModelClass, and you can achieve it in the SerializerClass, and the original filed can also be achieved. All you have to do is like this
class ParentArrivalSerializer(serializers.HyperlinkedModelSerializer):
# ...
carpool_id = serializers.IntegerField(write_only=True)
carpool = SchoolBuildingCarpoolSerializer(read_only=True)
# ...
class Meta:
fields = ('carpool_id', 'carpool', ...)
And use carpool_id in POST request.

How about overriding the to_representation method?
class YourSerializer(serializers.ModelSerializer):
class Meta:
model = ModelClass
fields = ["id", "foreignkey"]
def to_representation(self, instance):
data = super(YourSerializer, self).to_representation(instance)
data['foreignkey'] = YourNestedSerializer(instance.foreignkey).data
return data

One way to do it is to keep 'carpool' as the default you get from DRF, and then add a read-only field for the nested object.
Something like this (I don't have time to test the code, so consider this pseudo-code. If you cannot get it to work, let me know, and will spend more time):
class ParentArrivalSerializer(serializers.HyperlinkedModelSerializer):
carpool_info = serializers.SerializerMethodField(read_only=True)
class Meta:
model = ParentArrival
fields = ('id', 'carpool', 'carpool_info',)
def get_carpool_info(self, obj):
carpool = obj.carpool
serializer = SchoolBuildingCarpoolSerializer(carpool)
return serializer.data
If your only nested object is carpool, I would also suggest switching to the regular ModelSerializer so carpool only shows the ID (10) and the nested object then can show the URL.
class ParentArrivalSerializer(serializers.ModelSerializer):
....
and then if it all works, you will be able to do a post with
{
"carpool": 10
}
and your get:
{
"carpool": 10
"carpool_info": {
"url": "http://localhost:8000/api/school-building-carpools/10/"
"name": "Name of the carpool",
...
}
}
I have never found another solution, so this is the trick I have used several times.