Develop Reference

Best architecture for dynamically validating and saving field

I am looking for the good architecture for my problem. I am using django rest framework for building an API. I receive a list of dict which contains an id and a list of values. The list of values need to be validated according to the id.
Example of my code:
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField()
def validate(self, validated_data):
attribute = validated_data["attribute"]
values = validated_data["values"]
# This function returns the corresponding field according to attribute
values_child_field = get_values_field(attribute)
self.fields["values"].child = values_child_fields
new_values = self.fields["values"].run_child_validation(values)
set_value(validated_data, "values", new_values)
return validated_data
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I want to parse json like this:
{
"categorty_id": 42, # Category pk of the baseobject. which defines some constraints about attributes available
"attributes": [
{"id": 124, "values": ["value"]},
{"id": 321, "values": [42]},
{
"id": 18,
"values": [
{
"location": {"type": "Point", "geometry": {...}},
"address": "an address",
}
],
},
]
}
Currently, this code does not work. DRF seems to try to revalidate all values entries for each iteration with each child field. I do not understand why... I guess I could make it work without using this fields["values"] for making the validation and just retrieve the field and use it directly, but i need this field for making the save later.
Do you think my architecture is ok? What is the good way for parsing this type of data with DRF?
EDIT:
Structure of models are complex but a version simplified following:
class Attribute(models.Model):
class DataType(models.TextChoices):
TEXT = "TEXT", _("datatype_text")
INTEGER = "INTEGER", _("datatype_integer")
DATETIME = "DATETIME", _("datatype_datetime")
BOOL = "BOOL", _("datatype_bool")
# Some examples, but there are about 30 items with
# type very complicated like RecurrenceRule (RFC2445)
# or GeoJSON type
label = models.CharField()
category = models.ForeignKey(Category)
attribute_type = models.CharField(choices=DataType.choices)
class AttributeValue(models.Model):
attribute = models.ForeignKey(Attribute)
# a model which represents an object with list of attributes
baseobject = models.ForeignKey(BaseObject)
value = models.TextField()
AttributeValue is like a through table for manytomany relation between BaseObject model and Attribute model.
My JSON represents the list of attribute/values attached to a baseobject.
In fact I don't understand why DRf doesn't allow delegating registration in the child serializers of the parent serializer. This would allow much greater flexibility in code architecture and separation of responsibilities.
EDIT 2 :
My urls.py
router = routers.DefaultRouter()
router.register("baseobjects", BaseObjectViewSet, basename="baseobjects")
I am using the default router and url for DRF viewset.
The view looks like:
class BaseObjectViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
authentication_classes = [TokenAuthentication]
def create(self, request, *args, **kwargs):
serializer = BaseObjectApiInputSerializer(
data=request.data
)
if not serializer.is_valid():
return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
baseobject: BaseObject = serializer.save()
return Response(
{"results": [{"id": baseobject.pk}]}, status=HTTP_200_OK
)

I think you should use ListField with JSONField as child argument for values field.
validators = {
TinyurlShortener.DataType.TEXT: serializers.CharField(),
TinyurlShortener.DataType.INTEGER: serializers.IntegerField(),
TinyurlShortener.DataType.DATETIME: serializers.DateTimeField(),
TinyurlShortener.DataType.BOOL: serializers.BooleanField(),
}
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField(
child=serializers.JSONField()
)
def validate(self, attrs):
attribute = attrs.get('id')
field = validators[attribute.attribute_type]
for v in attrs['values']:
field.run_validation(json.loads(v.replace("'", '"')))
return super().validate(attrs)
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)

Filter Embedded Document List in GraphQL

I'm building a GraphQL application in Python/Graphene using a MongoDB backend (through MongoEngine). Everything has been working well, but noticed that there's not a lot documentation for handling nested lists of embedded documents. I thought one power of GraphQL was the ability to project only the properties you want, but it doesn't appear to be the case fully.
Looking at this collection as an example:
[
{
"name": "John Doe",
"age": 37,
"preferences": [
{
"key": "colour",
"value": "Green"
},
{
"key": "smell",
"value": "onions cooking in butter"
},
...
]
},
...
]
If I want to find a particular object through GraphQL, I would look up through a query like
{
person(name: "John Doe"){edges{node{
name age preferences{edges{node{
key value
}}}
}}}
}
But this could bring back hundreds of nested documents. What I would like to do instead is to identify the requested nested documents as part of the projection request.
{
person(name: "John Doe"){edges{node{
name age preferences(key: "colour"){edges{node{
key value
}}}
}}}
}
My understanding reading the GraphQL spec is these sub-queries are not possible, but wanted to confirm with experts first. And if it is possible, how would I implement it to support these types of requests?
Update Maybe a schema example will provide some more insightful responses.
class PreferenceModel(mongoengine.EmbeddedDocument):
key = mongoengine.fields.StringField()
value = mongoengine.fields.StringField()
class Preference(graphene_mongo.MongoengineObjectType):
class Meta:
interfaces = (graphene.relay.Node, )
model = PreferenceModel
class PersonModel(mongoengine.Document):
meta = {'collection': 'persons'}
name = mongoengine.fields.StringField()
age = mongoengine.fields.IntField()
preferences = mongoengine.fields.EmbeddedDocumentListField(PreferenceModel)
class Person(graphene_mongo.MongoengineObjectType):
class Meta:
interfaces = (graphene.relay.Node, )
model = PersonModel
class Query(graphene.ObjectType):
person = graphene_mongo.MongoengineConnectionField(Person)
schema = graphene.Schema(query=Query, types=[Person])
app = starlette.graphql.GraphQLApp(schema=schema)
Using this above structure, what changes would be necessary to allow for queries/filters on nested objects?

I had a similiar issue but working graphene-django. I solved it using custom resolvers on the DjangoObjectType, like this:
import graphene
from graphene_django import DjangoObjectType
from .models import Question, Choice, SubChoice
class SubChoiceType(DjangoObjectType):
class Meta:
model = SubChoice
fields = "__all__"
class ChoiceType(DjangoObjectType):
sub_choices = graphene.List(SubChoiceType, search_sub_choices=graphene.String())
class Meta:
model = Choice
fields = ("id", "choice_text", "question")
def resolve_sub_choices(self, info, search_sub_choices=None):
if search_sub_choices:
return self.subchoice_set.filter(sub_choice_text__icontains=search_sub_choices)
return self.subchoice_set.all()
class QuestionType(DjangoObjectType):
choices = graphene.List(ChoiceType, search_choices=graphene.String())
class Meta:
model = Question
fields = ("id", "question_text")
def resolve_choices(self, info, search_choices=None):
if search_choices:
return self.choice_set.filter(choice_text__icontains=search_choices)
return self.choice_set.all()
class Query(graphene.ObjectType):
all_questions = graphene.List(QuestionType, search_text=graphene.String())
all_choices = graphene.List(ChoiceType, search_text=graphene.String())
all_sub_choices = graphene.List(SubChoiceType)
def resolve_all_questions(self, info, search_text=None):
qs = Question.objects.all()
if search_text:
qs = qs.filter(question_text__icontains=search_text)
return qs
def resolve_all_choices(self, info, search_text=None):
qs = Choice.objects.all()
if search_text:
qs = qs.filter(choice_text__icontains=search_text)
return qs
def resolve_all_sub_choices(self, info):
qs = SubChoice.objects.all()
return qs
schema = graphene.Schema(query=Query)
you can find the example here: https://github.com/allangz/graphene_subfilters/blob/main/mock_site/polls/schema.py
It may work for you

DRF: Right way to use ListCreateAPIView with nested serializer

I make a page that lists all the existing vendors and modules that apply to each vendor. Here I need to change the status of modules (active or unactive), and if the module does not exist, but need to make it active then create it. It looks roughly like this.
Vendor1 module1/false module2/true module3/true .....
Vendor2 module1/false module2/true module3/true .....
.....
.....
models.py
class RfiParticipation(models.Model):
vendor = models.ForeignKey('Vendors', models.DO_NOTHING, related_name='to_vendor')
m = models.ForeignKey('Modules', models.DO_NOTHING, related_name='to_modules')
active = models.BooleanField(default=False)
user_id = models.IntegerField()
rfi = models.ForeignKey('Rfis', models.DO_NOTHING, related_name='to_rfi', blank=True, null=True)
timestamp = models.DateTimeField(auto_now=True)
To display it, I use ListCreateAPIView() class and nested serializer
serializer.py
class VendorModulesListManagementSerializer(serializers.ModelSerializer):
to_vendor = RfiParticipationSerializer(many=True)
class Meta:
model = Vendors
fields = ('vendorid', 'vendor_name', 'to_vendor',)
read_only_fields = ('vendorid', 'vendor_name', )
def create(self, validated_data):
validated_data = validated_data.pop('to_vendor')
for validated_data in validated_data:
module, created = RfiParticipation.objects.update_or_create(
rfi=validated_data.get('rfi', None),
vendor=validated_data.get('vendor', None),
m=validated_data.get('m', None),
defaults={'active': validated_data.get('active', False)})
return module
class RfiParticipationSerializer(serializers.ModelSerializer):
class Meta:
model = RfiParticipation
fields = ('pk', 'active', 'm', 'rfi', 'vendor', 'timestamp')
read_only_fields = ('timestamp', )
views.py
class AssociateModulesWithVendorView(generics.ListCreateAPIView):
"""
RFI: List of vendors with participated modules and modules status
"""
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
I have a question about using the create serializer method when sending a POST request.
Now the input format looks like this
{
"to_vendor": [
{
"active": false,
"m": 1,
"rfi": "20R1",
"vendor": 15
}]
}
I.e. the dictionary key for the current code implementation is the list of one dictionary. If I remove " [] " from dict value I got
{
"to_vendor": {
"non_field_errors": [
"Expected a list of items but got type \"dict\"."
]
}
}
And this is the reason why I need to add a for loop in the create method to iterate through the list with just one element. I already have any doubts that I'm doing the right thing. Maybe I chose the wrong implementation way?
But now question is why do I get a mistake?
AttributeError: Got AttributeError when attempting to get a value for field `to_vendor` on serializer `VendorModulesListManagementSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `RfiParticipation` instance.
Original exception text was: 'RfiParticipation' object has no attribute 'to_vendor'.
I would be very grateful for your help and advice!
upd
Get request format data:
[
{
"vendorid": 15,
"vendor_name": "Forest Gamp",
"to_vendor": [
{
"pk": 35,
"active": true,
"m": "Sourcing",
"rfi": "1",
"vendor": 15,
"timestamp": "2020-03-29T08:15:41.638427"
},
{
"pk": 39,
"active": false,
"m": "CLM",
"rfi": "20R1",
"vendor": 15,
"timestamp": "2020-03-29T09:09:03.431111"
}
]
},
{
"vendorid": 16,
"vendor_name": "Test21fd2",
"to_vendor": [
{
"pk": 41,
"active": false,
"m": "SA",
"rfi": "20R1",
"vendor": 16,
"timestamp": "2020-03-30T11:05:16.106412"
},
{
"pk": 40,
"active": false,
"m": "CLM",
"rfi": "20R1",
"vendor": 16,
"timestamp": "2020-03-30T10:40:52.799763"
}
]
}
]

It tries to access to_vendor on your model RfiParticipation and it complains that this property does not exist. related_name refers to the back relation on your Vendors model. So that you can do something like Vendors.to_vendor.all() and fetch all the RfiParticipation instances.
This happens when it tries to validate your input data, so even before it gets to the create function of your serializer.
If you are trying to create new RfiParticipation, why would you use a view that defines queryset = Vendors.objects.all()?
Instead define a view that takes care of creating RfiParticipation, since it seems that you already have a reference to Vendors. If I understand correctly, what you are trying to do is basically batch create RfiParticipation, so make a view that points to these.

I have gone through your implementation, and I suppose you might have used wrong generics class inheritance in views.py.
You should try to replace
class AssociateModulesWithVendorView(generics.ListCreateAPIView):
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
With
class AssociateModulesWithVendorView(generics.CreateAPIView, generics.ListAPIView):
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
You can check below links for reference:
CreateAPIView : https://www.django-rest-framework.org/api-guide/generic-views/#createmodelmixin
ListCreateAPIView : https://www.django-rest-framework.org/api-guide/generic-views/#listcreateapiview

Retrieve the object ID in GraphQL

I'd like to know whether it is possible to get the "original id" of an object as the result of the query. Whenever I make a request to the server, it returns the node "global identifier", something like U29saWNpdGFjYW9UeXBlOjEzNTkxOA== .
The query is similar to this one:
{
allPatients(active: true) {
edges {
cursor
node {
id
state
name
}
}
}
and the return is:
{
"data": {
"edges": [
{
"cursor": "YXJyYXljb25uZWN0aW9uOjA=",
"node": {
"id": "U29saWNpdGFjYW9UeXBlOjEzNTkxOA==",
"state": "ARI",
"name": "Brad"
}
}
]
}
}
How can I get the "original" id of the object at the database level (e.g. '112') instead of that node unique identifier?
ps.: I am using graphene-python and Relay on the server side.

Overriding default to_global_id method in Node object worked out for me:
class CustomNode(graphene.Node):
class Meta:
name = 'Node'
#staticmethod
def to_global_id(type, id):
return id
class ExampleType(DjangoObjectType):
class Meta:
model = Example
interfaces = (CustomNode,)

First option, remove relay.Node as interface of your objectNode declaration.
Second option, use custom resolve_id fonction to return id original value.
Example
class objectNode(djangoObjectType):
.... Meta ....
id = graphene.Int(source="id")
def resolve_id("commons args ...."):
return self.id
Hope it helps

To expand on the top answer and for those using SQLAlchemy Object Types, this worked for me:
class CustomNode(graphene.Node):
class Meta:
name = 'myNode'
#staticmethod
def to_global_id(type, id):
return id
class ExampleType(SQLAlchemyObjectType):
class Meta:
model = Example
interfaces = (CustomNode, )
If you have other ObjectTypes using relay.Node as the interface, you will need to use a unique name under your CustomNode. Otherwise you will get and assertion error.

With this you can retrive the real id in database:
def get_real_id(node_id: str):
_, product_id_real = relay.Node.from_global_id(global_id=node_id)
return product_id_real

How can one customize Django Rest Framework serializers output?

I have a Django model that is like this:
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
And two serializers, defined as:
class WindowsFlatMacAddressSerializer(serializers.Serializer):
address = serializers.Field()
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = WindowsFlatMacAddressSerializer(many=True)
clientId = serializers.Field()
When accessing the serializer over a view, I get the following output:
[
{
"id": 1,
"macAddresses": [
{
"address": "aa:aa:aa:aa:aa:aa"
},
{
"address": "bb:bb:bb:bb:bb:bb"
}
],
"clientId": null
}
]
Almost good, except that I'd prefer to have:
[
{
"id": 1,
"macAddresses": [
"aa:aa:aa:aa:aa:aa",
"bb:bb:bb:bb:bb:bb"
],
"clientId": null
}
]
How can I achieve that ?

Create a custom serializer field and implement to_native so that it returns the list you want.
If you use the source="*" technique then something like this might work:
class CustomField(Field):
def to_native(self, obj):
return obj.macAddresses.all()
I hope that helps.
Update for djangorestframework>=3.9.1
According to documentation, now you need override either one or both of the to_representation() and to_internal_value() methods. Example
class CustomField(Field):
def to_representation(self, value)
return {'id': value.id, 'name': value.name}

Carlton's answer will work do the job just fine. There's also a couple of other approaches you could take.
You can also use SlugRelatedField, which represents the relationship, using a given field on the target.
So for example...
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.SlugRelatedField(slug_field='address', many=True, read_only=True)
clientId = serializers.Field()
Alternatively, if the __str__ of the WindowsMacAddress simply displays the address, then you could simply use RelatedField, which is a basic read-only field that will give you a simple string representation of the relationship target.
# models.py
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
def __str__(self):
return self.address
# serializers.py
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.RelatedField(many=True)
clientId = serializers.Field()
Take a look through the documentation on serializer fields to get a better idea of the various ways you can represent relationships in your API.