Develop Reference

Best architecture for dynamically validating and saving field

I am looking for the good architecture for my problem. I am using django rest framework for building an API. I receive a list of dict which contains an id and a list of values. The list of values need to be validated according to the id.
Example of my code:
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField()
def validate(self, validated_data):
attribute = validated_data["attribute"]
values = validated_data["values"]
# This function returns the corresponding field according to attribute
values_child_field = get_values_field(attribute)
self.fields["values"].child = values_child_fields
new_values = self.fields["values"].run_child_validation(values)
set_value(validated_data, "values", new_values)
return validated_data
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I want to parse json like this:
{
"categorty_id": 42, # Category pk of the baseobject. which defines some constraints about attributes available
"attributes": [
{"id": 124, "values": ["value"]},
{"id": 321, "values": [42]},
{
"id": 18,
"values": [
{
"location": {"type": "Point", "geometry": {...}},
"address": "an address",
}
],
},
]
}
Currently, this code does not work. DRF seems to try to revalidate all values entries for each iteration with each child field. I do not understand why... I guess I could make it work without using this fields["values"] for making the validation and just retrieve the field and use it directly, but i need this field for making the save later.
Do you think my architecture is ok? What is the good way for parsing this type of data with DRF?
EDIT:
Structure of models are complex but a version simplified following:
class Attribute(models.Model):
class DataType(models.TextChoices):
TEXT = "TEXT", _("datatype_text")
INTEGER = "INTEGER", _("datatype_integer")
DATETIME = "DATETIME", _("datatype_datetime")
BOOL = "BOOL", _("datatype_bool")
# Some examples, but there are about 30 items with
# type very complicated like RecurrenceRule (RFC2445)
# or GeoJSON type
label = models.CharField()
category = models.ForeignKey(Category)
attribute_type = models.CharField(choices=DataType.choices)
class AttributeValue(models.Model):
attribute = models.ForeignKey(Attribute)
# a model which represents an object with list of attributes
baseobject = models.ForeignKey(BaseObject)
value = models.TextField()
AttributeValue is like a through table for manytomany relation between BaseObject model and Attribute model.
My JSON represents the list of attribute/values attached to a baseobject.
In fact I don't understand why DRf doesn't allow delegating registration in the child serializers of the parent serializer. This would allow much greater flexibility in code architecture and separation of responsibilities.
EDIT 2 :
My urls.py
router = routers.DefaultRouter()
router.register("baseobjects", BaseObjectViewSet, basename="baseobjects")
I am using the default router and url for DRF viewset.
The view looks like:
class BaseObjectViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
authentication_classes = [TokenAuthentication]
def create(self, request, *args, **kwargs):
serializer = BaseObjectApiInputSerializer(
data=request.data
)
if not serializer.is_valid():
return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
baseobject: BaseObject = serializer.save()
return Response(
{"results": [{"id": baseobject.pk}]}, status=HTTP_200_OK
)

I think you should use ListField with JSONField as child argument for values field.
validators = {
TinyurlShortener.DataType.TEXT: serializers.CharField(),
TinyurlShortener.DataType.INTEGER: serializers.IntegerField(),
TinyurlShortener.DataType.DATETIME: serializers.DateTimeField(),
TinyurlShortener.DataType.BOOL: serializers.BooleanField(),
}
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField(
child=serializers.JSONField()
)
def validate(self, attrs):
attribute = attrs.get('id')
field = validators[attribute.attribute_type]
for v in attrs['values']:
field.run_validation(json.loads(v.replace("'", '"')))
return super().validate(attrs)
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)

DRF: Right way to use ListCreateAPIView with nested serializer

I make a page that lists all the existing vendors and modules that apply to each vendor. Here I need to change the status of modules (active or unactive), and if the module does not exist, but need to make it active then create it. It looks roughly like this.
Vendor1 module1/false module2/true module3/true .....
Vendor2 module1/false module2/true module3/true .....
.....
.....
models.py
class RfiParticipation(models.Model):
vendor = models.ForeignKey('Vendors', models.DO_NOTHING, related_name='to_vendor')
m = models.ForeignKey('Modules', models.DO_NOTHING, related_name='to_modules')
active = models.BooleanField(default=False)
user_id = models.IntegerField()
rfi = models.ForeignKey('Rfis', models.DO_NOTHING, related_name='to_rfi', blank=True, null=True)
timestamp = models.DateTimeField(auto_now=True)
To display it, I use ListCreateAPIView() class and nested serializer
serializer.py
class VendorModulesListManagementSerializer(serializers.ModelSerializer):
to_vendor = RfiParticipationSerializer(many=True)
class Meta:
model = Vendors
fields = ('vendorid', 'vendor_name', 'to_vendor',)
read_only_fields = ('vendorid', 'vendor_name', )
def create(self, validated_data):
validated_data = validated_data.pop('to_vendor')
for validated_data in validated_data:
module, created = RfiParticipation.objects.update_or_create(
rfi=validated_data.get('rfi', None),
vendor=validated_data.get('vendor', None),
m=validated_data.get('m', None),
defaults={'active': validated_data.get('active', False)})
return module
class RfiParticipationSerializer(serializers.ModelSerializer):
class Meta:
model = RfiParticipation
fields = ('pk', 'active', 'm', 'rfi', 'vendor', 'timestamp')
read_only_fields = ('timestamp', )
views.py
class AssociateModulesWithVendorView(generics.ListCreateAPIView):
"""
RFI: List of vendors with participated modules and modules status
"""
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
I have a question about using the create serializer method when sending a POST request.
Now the input format looks like this
{
"to_vendor": [
{
"active": false,
"m": 1,
"rfi": "20R1",
"vendor": 15
}]
}
I.e. the dictionary key for the current code implementation is the list of one dictionary. If I remove " [] " from dict value I got
{
"to_vendor": {
"non_field_errors": [
"Expected a list of items but got type \"dict\"."
]
}
}
And this is the reason why I need to add a for loop in the create method to iterate through the list with just one element. I already have any doubts that I'm doing the right thing. Maybe I chose the wrong implementation way?
But now question is why do I get a mistake?
AttributeError: Got AttributeError when attempting to get a value for field `to_vendor` on serializer `VendorModulesListManagementSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `RfiParticipation` instance.
Original exception text was: 'RfiParticipation' object has no attribute 'to_vendor'.
I would be very grateful for your help and advice!
upd
Get request format data:
[
{
"vendorid": 15,
"vendor_name": "Forest Gamp",
"to_vendor": [
{
"pk": 35,
"active": true,
"m": "Sourcing",
"rfi": "1",
"vendor": 15,
"timestamp": "2020-03-29T08:15:41.638427"
},
{
"pk": 39,
"active": false,
"m": "CLM",
"rfi": "20R1",
"vendor": 15,
"timestamp": "2020-03-29T09:09:03.431111"
}
]
},
{
"vendorid": 16,
"vendor_name": "Test21fd2",
"to_vendor": [
{
"pk": 41,
"active": false,
"m": "SA",
"rfi": "20R1",
"vendor": 16,
"timestamp": "2020-03-30T11:05:16.106412"
},
{
"pk": 40,
"active": false,
"m": "CLM",
"rfi": "20R1",
"vendor": 16,
"timestamp": "2020-03-30T10:40:52.799763"
}
]
}
]

It tries to access to_vendor on your model RfiParticipation and it complains that this property does not exist. related_name refers to the back relation on your Vendors model. So that you can do something like Vendors.to_vendor.all() and fetch all the RfiParticipation instances.
This happens when it tries to validate your input data, so even before it gets to the create function of your serializer.
If you are trying to create new RfiParticipation, why would you use a view that defines queryset = Vendors.objects.all()?
Instead define a view that takes care of creating RfiParticipation, since it seems that you already have a reference to Vendors. If I understand correctly, what you are trying to do is basically batch create RfiParticipation, so make a view that points to these.

I have gone through your implementation, and I suppose you might have used wrong generics class inheritance in views.py.
You should try to replace
class AssociateModulesWithVendorView(generics.ListCreateAPIView):
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
With
class AssociateModulesWithVendorView(generics.CreateAPIView, generics.ListAPIView):
permission_classes = [permissions.AllowAny, ]
serializer_class = VendorModulesListManagementSerializer
queryset = Vendors.objects.all()
You can check below links for reference:
CreateAPIView : https://www.django-rest-framework.org/api-guide/generic-views/#createmodelmixin
ListCreateAPIView : https://www.django-rest-framework.org/api-guide/generic-views/#listcreateapiview

Django filter query from many to many fields

models.py
class Keyword(models.Model):
name=models.CharField(max_length=500,unique=True)
image = models.ImageField(upload_to='keywords/', blank=True, null=True)
mood=models.ManyToManyField(Mood,blank=True)
def __str__(self):
return str(self.name)
class ItemVariation(models.Model):
restaurant=models.ForeignKey(Restaurant,on_delete=models.CASCADE)
items_id=models.ForeignKey(Item,on_delete=models.CASCADE)
price=models.IntegerField(blank=True,null=True,default=0)
item_code=models.CharField(max_length=500)
keywords= models.ManyToManyField(Keyword)
image=models.ImageField(upload_to='dishes/', blank=True, null=True)
def __str__(self):
return str(self.id)
views.py
class FoodfeedList(APIView):
def get(self,request,format=None):
keyword = request.GET['keywords'].split(",")
mood=request.GET['mood']
location=request.GET['location'].split(",")
price=request.GET['price']
user_id=request.user.id
items=Item.objects.all()
item_variation=ItemVariation.objects.all()
search_restaurant=SearchRestaurant()
search_result=search_restaurant.searchRes(location[0],location[1],keyword[0],user_id)
all_results=[]
json={}
for i in search_result:
json={}
json['res_id'] = i['restaurant']['id']
# keywords_list = ItemVariation.objects.filter(keywords=keyword[0])
items=ItemVariation.objects.filter(restaurant = request.user.id)
itemserializer=ItemVariationSerializer(items,many =True)
json['itemserializer']=itemserializer.data
all_results.append(json)
# items_serilizer=ItemsSerializer(items,many=True)
return Response(all_results)
Output
"res_id": 2,
"itemserializer": [
{
"id": 1,
"price": 0,
"item_code": "2134ffsd",
"image": null,
"restaurant": 1,
"items_id": 1,
"keywords": [1,2,3]
},
I need fiter query for items where keywords are name from keyword model with query paraters 'Burger' as keyword
my url is like
http://127.0.0.1:8000/api/foodfeeds/?keywords=BURGER,teste&mood=happy&location=2323,7767.323&price=2
if keywords match in ItemVariation model in many to many fields so it should return itemvariation

do the filtering like this:
items=ItemVariation.objects.filter(keywords__name__in=keywords)
or for a single keyword do this:
items=ItemVariation.objects.filter(keywords__name=keywords[0])

Not 100% sure if i understood your question correctly. Here i used a for loop on the list with they keywords, adding each keyword-Entity to the list - selected by name.
class FoodfeedList(APIView):
def get(self,request,format=None):
keywords = request.GET['keywords'].split(",")
mood=request.GET['mood']
location=request.GET['location'].split(",")
price=request.GET['price']
user_id=request.user.id
keyword_names = []
for keyword in keywords:
keyword_names += Item.objects.get(name=keyword)
item_variation=ItemVariation.objects.all()
search_restaurant=SearchRestaurant()
search_result=search_restaurant.searchRes(location[0],location[1],keyword[0],user_id)
all_results=[]
json={}
for i in search_result:
json={}
json['res_id'] = i['restaurant']['id']
json['keywords'] = keyword_names
items=ItemVariation.objects.filter(restaurant = request.user.id)
itemserializer=ItemVariationSerializer(items,many =True)
json['itemserializer']=itemserializer.data
all_results.append(json)
# items_serilizer=ItemsSerializer(items,many=True)
return Response(all_results)

Wagtail - Possible to rename an item in a streamfield in a migration?

I've changed the name of an item in a StreamField from
tiles = StreamField(
[
('items', StructBlock([
('icon', ImageChooserBlock()),
('text', CharBlock()),
])
),
], blank=True)
To
tiles = StreamField(
[
('info', StructBlock([
('icon', ImageChooserBlock()),
('text', CharBlock()),
])
),
], blank=True)
Any idea how I can create a migration to rename the field ?

Streamfield data is stored as JSON, as a list of items with 'type' and 'value' properties. To apply the field name change to all of your pages, you should be able to create an empty migration (or add it to an existing one), then add the following function or something similar to your migration file, then run it:
// other imports
import json
def convert_streamfield_name(apps, schema_editor):
db_alias = schema_editor.connection.alias
MyPageModel = apps.get_model('myapp', 'MyPageModel')
pages = MyPageModel.objects.using(db_alias).all()
for page in pages:
revised_stream_data = []
stream_data = page.tiles.stream_data
for data in stream_data:
if data.get('type') == 'items':
value = data.get('value')
revised_stream_data.append({
'type': 'info'
'value': value
})
else:
revised_stream_data.append(data)
raw_json = json.dumps(revised_stream_data)
page.tiles = raw_json
page.save()
class Migration(migrations.Migration):
dependencies = [...]
operations = [
migrations.RunPython(convert_streamfield_name),
...
]

How can I add a document on a StreamField in Wagtail CMS?

I've a question about Wagtail CMS.
Recently I'm trying to import programmatically some documents in a StreamField of an instance of a Wagtail Page model. I've done some researches but without results.
Currently I'm using:
Wagtail 1.13
Django 1.11.6
Python 2.7
Here the model of the page in which I need to import the documents as attachments (see the homonym field):
class EventPage(TranslatablePage, Page):
# Database fields
uuid = models.UUIDField(verbose_name='UUID', default=uuid.uuid4)
start_date = models.DateField(verbose_name='Start date')
end_date = models.DateField(verbose_name='End date')
location = models.CharField(verbose_name='Place', max_length=255, null=True, blank=True)
body = RichTextField(verbose_name='Body')
attachments = StreamField(blocks.StreamBlock([
('document', DocumentChooserBlock(label='Document', icon='doc-full-inverse')),
]), verbose_name='Attachments', null=True, blank=True)
subscribe = models.BooleanField(verbose_name='Subscribe option', default=False)
# Editor panels configuration
content_panels = [
FieldPanel('title', classname='title'),
MultiFieldPanel([
FieldRowPanel([
FieldPanel('start_date'),
FieldPanel('end_date'),
]),
], heading='Period'),
FieldPanel('location'),
FieldPanel('body'),
StreamFieldPanel('attachments'),
]
promote_panels = Page.promote_panels + [
MultiFieldPanel([
FieldPanel('subscribe'),
], heading='Custom Settings'),
]
settings_panels = TranslatablePage.settings_panels + [
MultiFieldPanel([
FieldPanel('uuid'),
], heading='Meta')
]
parent_page_types = ["home.FolderPage"]
subpage_types = []
On shell, I tried to apply the solution explained on this page but without success.
event = EventPage.objects.get(pk=20)
doc = Document.objects.get(pk=3)
event.attachments = [
('document',
[
StreamValue.StreamChild(
id = None,
block = DocumentChooserBlock(),
value = doc
)
]
)
]
Python give me this error: AttributeError: 'list' object has no attribute 'pk'.

event.attachments = [('document', doc)] should work, I believe. (On the other question you link to, StreamChild was necessary because AccordionRepeaterBlock was a StreamBlock nested in a StreamBlock; that's not the case for your definition.)
To add a document to the existing StreamField content, build a new list and assign that to event.attachments:
new_attachments = [(block.block_type, block.value) for block in blocks]
new_attachments.append(('document', doc))
event.attachments = new_attachments
(Currently you can't append directly to a StreamField value, but this may well be supported in a future Wagtail release...)