Develop Reference

Best architecture for dynamically validating and saving field

I am looking for the good architecture for my problem. I am using django rest framework for building an API. I receive a list of dict which contains an id and a list of values. The list of values need to be validated according to the id.
Example of my code:
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField()
def validate(self, validated_data):
attribute = validated_data["attribute"]
values = validated_data["values"]
# This function returns the corresponding field according to attribute
values_child_field = get_values_field(attribute)
self.fields["values"].child = values_child_fields
new_values = self.fields["values"].run_child_validation(values)
set_value(validated_data, "values", new_values)
return validated_data
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I want to parse json like this:
{
"categorty_id": 42, # Category pk of the baseobject. which defines some constraints about attributes available
"attributes": [
{"id": 124, "values": ["value"]},
{"id": 321, "values": [42]},
{
"id": 18,
"values": [
{
"location": {"type": "Point", "geometry": {...}},
"address": "an address",
}
],
},
]
}
Currently, this code does not work. DRF seems to try to revalidate all values entries for each iteration with each child field. I do not understand why... I guess I could make it work without using this fields["values"] for making the validation and just retrieve the field and use it directly, but i need this field for making the save later.
Do you think my architecture is ok? What is the good way for parsing this type of data with DRF?
EDIT:
Structure of models are complex but a version simplified following:
class Attribute(models.Model):
class DataType(models.TextChoices):
TEXT = "TEXT", _("datatype_text")
INTEGER = "INTEGER", _("datatype_integer")
DATETIME = "DATETIME", _("datatype_datetime")
BOOL = "BOOL", _("datatype_bool")
# Some examples, but there are about 30 items with
# type very complicated like RecurrenceRule (RFC2445)
# or GeoJSON type
label = models.CharField()
category = models.ForeignKey(Category)
attribute_type = models.CharField(choices=DataType.choices)
class AttributeValue(models.Model):
attribute = models.ForeignKey(Attribute)
# a model which represents an object with list of attributes
baseobject = models.ForeignKey(BaseObject)
value = models.TextField()
AttributeValue is like a through table for manytomany relation between BaseObject model and Attribute model.
My JSON represents the list of attribute/values attached to a baseobject.
In fact I don't understand why DRf doesn't allow delegating registration in the child serializers of the parent serializer. This would allow much greater flexibility in code architecture and separation of responsibilities.
EDIT 2 :
My urls.py
router = routers.DefaultRouter()
router.register("baseobjects", BaseObjectViewSet, basename="baseobjects")
I am using the default router and url for DRF viewset.
The view looks like:
class BaseObjectViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
authentication_classes = [TokenAuthentication]
def create(self, request, *args, **kwargs):
serializer = BaseObjectApiInputSerializer(
data=request.data
)
if not serializer.is_valid():
return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
baseobject: BaseObject = serializer.save()
return Response(
{"results": [{"id": baseobject.pk}]}, status=HTTP_200_OK
)

I think you should use ListField with JSONField as child argument for values field.
validators = {
TinyurlShortener.DataType.TEXT: serializers.CharField(),
TinyurlShortener.DataType.INTEGER: serializers.IntegerField(),
TinyurlShortener.DataType.DATETIME: serializers.DateTimeField(),
TinyurlShortener.DataType.BOOL: serializers.BooleanField(),
}
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField(
child=serializers.JSONField()
)
def validate(self, attrs):
attribute = attrs.get('id')
field = validators[attribute.attribute_type]
for v in attrs['values']:
field.run_validation(json.loads(v.replace("'", '"')))
return super().validate(attrs)
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)

Defining an API for complex View generator function (with many configurables)

I'm writing a view generator for my Django project. I have a large number of models from a legacy application (~150 models), that all need the same basic CRUD operations (providing Admin access isn't enough apparently).
So I'm writing a generator that returns 5 Views for each model, and of course each view can potentially take a large number of options, and I'm trying to define sane API/default parameter format for my generator.
My current generator:
def generate_views(model_class, **kwargs):
"""
For a given model, returns a dict of generic class-based views
"""
###
# Forms
# Optionally generate form classes if not already provided
###
# Append these fields with either "create_" or "update_" to have them only
# apply to that specific type of form
form_override_args = ['fields', 'exclude', 'form_method', 'form_class',
'form_layout', 'widgets', 'media_css', 'media_js']
if 'form_class' not in kwargs and 'create_form_class' not in kwargs:
create_form_kwargs = kwargs.copy()
for arg in form_override_args:
if f'create_{arg}' in kwargs:
create_form_kwargs[arg] = kwargs[f'create_{arg}']
kwargs['create_form_class'] = forms.FormFactory(model_class, **create_form_kwargs).form()
if 'form_class' not in kwargs and 'update_form_class' not in kwargs:
update_form_kwargs = kwargs.copy()
for arg in form_override_args:
if f'update_{arg}' in kwargs:
update_form_kwargs[arg] = kwargs[f'update_{arg}']
kwargs['update_form_class'] = forms.FormFactory(model_class, **update_form_kwargs).form()
if 'form_class' not in kwargs:
kwargs['form_class'] = forms.FormFactory(model_class, **kwargs).form()
###
# Tables
# Optionally generate table classes if not already provided
###
# Append these fields with "table_" to have them only
# apply to the table view
table_override_args = ['fields', 'exclude']
if 'table_class' not in kwargs:
update_table_kwargs = kwargs.copy()
for arg in table_override_args:
if f'table_{arg}' in kwargs:
update_table_kwargs[arg] = kwargs[f'table_{arg}']
kwargs['table_class'] = tables.TableFactory(model_class, **update_table_kwargs).table()
###
# Views
# Generate 5 generic views based on the provided model
###
view_factory = views.ViewFactory(model_class, **kwargs)
return {
'list_view': view_factory.list_view(),
'detail_view': view_factory.detail_view(),
'create_view': view_factory.create_view(),
'update_view': view_factory.update_view(),
'delete_view': view_factory.delete_view()
}
I'm currently relying on kwargs, and I wanted to define what a fully filled-out kwargs dict should look like. Something like
{
'forms': {
'all': {
},
'create': {
},
'update': {
}
},
'tables': {
'all': {
},
'list': {
}
},
'views': {
'all': {
},
'list': {
},
'detail': {
},
'create': {
},
'update': {
},
'delete': {
}
}
}
And it's just seeming a bit overworked. I'm mostly looking for recommendations on a potentially better design (because I'm going cross eyed from just working on it).

It seems that you are fighting the way how Django structures discrete functionalities/configurations in class-based views.
Django’s generic class-based views are built out of mixins providing discrete functionality.
So, my suggestion is: using mixins to incoporate the table and form classes into your views for the CRUD operation. In the generator, all configurable parameters should be passed only to the views.
Backgrounds knowledge
Let's look at how django.views.generic.edit.CreateView is designed. It inherits methods and attributes from:
SingleObjectTemplateResponseMixin,
BaseCreateView and
ModelFormMixin.
It can be bound to a model simply with a few lines of codes:
from myapp.models import Author
class AuthorCreateView(CreateView):
model = Author
fields = ['FirstName','FamilyName','BirthDay']
def form_valid(self, form):
# Saves the form instance, sets the current object for the view, and redirects to get_success_url().
Here the model attribute is shared by all the mixins to do their jobs, while fields and form_valid are specific to ModelFormMixin.
Although all configurable parameters/methods are put together under the View class, each mixin just picks up those it needs.
Redesign the API
Keeping this in mind, let's begin to simplify your view generator/factory. For this example, let's say you have the following base classes that include common (default) settings:
from django.views.generic.edit import CreateView, DeleteView, UpdateView
from django.views.generic import ListView, DetailView
from django_tables2 as SingleTableMixin
class TableListView(SingleTableMixin, ListView):
table_pagination = { 'per_page': 10 }
# add common configurable parameters here
class MyOwnCreateView(CreateView):
success_url = "/yeah"
# Introduce a configurable method `form_valid_hook`
def form_valid(self, form):
if hasattr(self,'form_valid_hook'):
self.form_valid_hook(form)
return super().form_valid(form)
Below is the simplified generator function for all 5 views.
BaseViews= {'create': MyOwnCreateView,
'delete': DeleteView,
'update': UpdateView,
'list' : TableListView,
'detail': DetailView }
def generate_views(model_class, **kwargs):
"""
Generate views for `model_class`
Keyword parameters:
{action}=dict(...)
{action}_mixins=tuple(...)
where `action` can be 'list', 'detail', 'create', 'update', 'delete'.
"""
NewViews = {}
for action, baseView in BaseViews.items():
viewName = model_class.__name__ + baseView.__name__
viewAttributes = kwargs.get(action,{})
viewBaseCls = (baseView,) + kwargs.get(f"{action}_mixins",tuple())
v = type(viewName, viewBaseCls, viewAttributes) # create a subclass of baseView
v.model = model_class # bind the view to the model
NewViews[f'{action}_view'] = v
return NewViews
You see, the generator function is simplified to only 10 lines of code.
Moreover, the API will become much cleaner:
def validate_author(self, form):
send_email(form)
AuthorViews = generate_views(Author,
create=dict(
success_url='/thanks/',
form_valid_hook=validate_author),
... )
How to use mixins in this API
In the above example, I use a hook/callback function form_valid_hook to inject an email-sending procedure before the form data are saved. This is ugly because the configurables for the email will be in the module scope. It's better to refactor it into a mixin class.
from django.core.mail import send_mail
class FormEmailMixin:
from_email = 'info#example.com'
subject_template = 'We hear you'
message_template = 'Hi {username}, ...'
def form_valid(self, form):
user_info = dict( username = self.request.user.username
to_email = ... )
send_mail(subject_template.format(**user_info),
message_template.format(**user_info)
self.from_email , [user_info['to_email'],] )
return super().form_valid(form)
Then you can use this mixin class in the API call.
AuthorViews = generate_views( Author,
create={ 'message_template': 'Dear Author {username}, ...' },
create_mixins = (FormEmailMixin,) )

Populating WTForm FormField in a Flask view when object doesn't match form field names

I have a form setup like so:
class AddressForm(FlaskForm):
line1 = StringField()
city = StringField()
postcode = StringField()
class PlaceForm(FlaskForm):
name = StringField()
address = FormField(AddressForm)
And then I have a Flask view something like this:
#bp.route("/places/<ident>", methods=['GET', 'POST'])
def edit_place(ident):
place = api.get_place(ident)
form = PlaceForm(obj=place)
if form.validate_on_submit():
# do stuff with the form data
return render_template('place/edit.html', form=form)
The api.get_place(ident) returns data that doesn't match the shape of the field names in my Form classes, so my forms are always empty when rendered in the browser. For example, the response from the API might look like this:
{
"place": {
"place_name": "Foobar",
"address": {
"address1": "500 5th St",
"locality": "San Francisco",
"post_code": "90210"
}
}
}
How do I customize the code populates the PlaceForm with data when passing in obj?

I can't quite tell if this is actually the pattern you want. Since edit_place has both GET and POST methods do you really want to populate the form in both cases from the api.get_place() function, possibly overwriting the data from the request?
Anyway you might try something like the following:
class PlaceForm(FlaskForm):
name = StringField()
address = FormField(AddressForm)
def __init__(self, *kwargs):
super().__init__()
self.address, self.name = # some code to populate with data from kwargs

django create custom form

We could directly create a simple form by inheriting forms.Form. However, I want forms to be dynamically created based on the key-value specified by the admin. Consider, user provides a map as :
[
{
"key": "name",
"value": "CharField"
},
{
"key": "url",
"value": "URLField"
}
]
Based on the map given above how can I create a form which is equivalent to the class based form creation below:
from django import forms
class CommentForm(forms.Form):
name = forms.CharField(label='name')
url = forms.URLField(label='url')
My current approach might be to iterate across each key-value pair and make a HTML form manually. Is it possible to create dynamic forms in django without having to write a class definition?

Yes, you can do this in the init function of the form.
Pass in the mapping array as an extra keyword argument when creating the form, then pop it from the kwargs before calling the init method from super.
You can then use this to dynamically add fields to the form.
class CommentForm(forms.Form):
def __init__(self , *args, **kwargs):
field_mapping_array = kwargs.pop('field_mapping_array')
super(CommentFrom, self).__init__(*args, **kwargs)
for field_type in field_mapping_array:
field_name = field_type['key'] # make sure keys are unique
if field_type[value] == "CharField":
self.fields[field_name] = form.CharField(...extra args here)
elif field_type['value'] == 'UrlField' :
self.fields[field_name] = form.UrlField(...extra args here)
elif .... map other field types here....
When creating the form pass the field_mapping_array into the form class.
field_mapping = [
{
"key": "name",
"value": "CharField"
},
{
"key": "url",
"value": "URLField"
}
]
comment_form = CommentForm(field_mapping_array=field_mapping)

How to create a filtered dropdown using Django forms?

I've got this model :
class QuestionInstance(models.Model):
questionsSet = models.ForeignKey(QuestionsSet)
question = models.ForeignKey(Question)
parent = models.ForeignKey('self',null=True,blank=True)
optional = models.BooleanField(default=False)
I'd like to create a dropdown, which user could choose one QuestionInstance.
It has to be filtered with questionsSet.
I've tested using a modelform like this, but it's not working :
(based on this How do I filter values in a Django form using ModelForm?)
class FormQuestionSelect(ModelForm):
instanceList = forms.ChoiceField(choices=[(questionInstance.id, questionInstance.question) for questionInstance in QuestionInstance.objects.all()])
class Meta:
model = QuestionInstance
fields = ('instanceList', )
widgets = {
'instanceList': Select(attrs={'class': 'select'}),
}
def __init__(self, questionsSet=None, **kwargs):
super(FormQuestionSelect, self).__init__(**kwargs)
if questionsSet:
#Tested many code here to filter, None of them worked :(
#Is that possible to create instanceList there ?
I'm not sure using a modelform is a good idea for this kind of purpose.
A modelform is great when create or update a model instance.
When using specific forms, like in this case, I'm using a custom form in template :
View
questionInstanceList = QuestionInstance.objects.filter(questionsSet=questionsSet)
Template
<select name="questionInstanceSelect">
{% for instance in questionInstanceList %}
<option value="{{ instance.id }}">{{ instance.question.text }}</option>
{% endfor %}
</select>
and process them this way :
instanceList = request.POST.get('questionInstanceSelect')
I'm quite sure there's a proper way.

You can change queryset of ModelChoiceField after form instantiation either in form __init__ or in view. but this wouldn't solve issue on client side. When someone change QuestionSet Question selectbox will remain the same
To update queryset just update form field's one
form.fields['parent'].queryset = (QuestionInstance.objects
.filter(questionsSet=questionsSet))
or if You change form __init__
self.fields['parent'].queryset = (QuestionInstance.objects
.filter(questionsSet=questionsSet))
But One should remember that if questionsSet is changed on client side parent list will remain the same.
Would You consider to add client side code updating parent's choice list
Let me explain a bit.
You have model
class QuestionInstance(models.Model):
questionsSet = models.ForeignKey(QuestionsSet)
question = models.ForeignKey(Question)
parent = models.ForeignKey('self',null=True,blank=True)
optional = models.BooleanField(default=False)
Here parent field link to self(the same model).
Let us use `Model form for this model
class FormQuestionSelect(ModelForm):
class Meta:
model = QuestionInstance
ModelForm will create fields for each model field with the same name
then after Its creation we update ModelChoiceField (created for ForeignKey) queryset

If you want your field to be dynamic, you need to use jQuery and ajax for this functionality. I have given code for use in django admin. You can tweak it a bit, if you want to use it in custom pages. But the concept remains same for both.
question_set_change.js
(function($){
$(function(){
$(document).ready(function() {
$('#id_questionsSet').bind('change', question_set_change);
$('#id_question > option').show();
if ($('#id_questionsSet').val() != '') {
var question_set_id = $('#id_questionsSet').val();
$.ajax({
"type" : "GET",
"url" : "/product_change/?question_set_id="+question_set_id,
"dataType" : "json",
"cache" : false,
"success" : function(json) {
$('#id_question >option').remove();
for(var j = 0; j < json.length; j++){
$('#id_question').append($('<option></option>').val(json[j][0]).html(json[j][1]));
}
}
});
}
});
});
})(django.jQuery);
// based on the questionsSet, questions will be loaded
var $ = django.jQuery.noConflict();
function question_set_change()
{
var question_set_id = $('#id_questionsSet').val();
$.ajax({
"type" : "GET",
"url" : "/product_change/?question_set_id="+question_set_id,
"dataType" : "json",
"cache" : false,
"success" : function(json) {
$('#id_question > option').remove();
for(var j = 0; j < json.length; j++){
$('#id_question').append($('<option></option>').val(json[j][0]).html(json[j][1]));
}
}
})(jQuery);
}
Include the following in views.py:
import simplejson
from django.shortcuts import HttpResponse
from app.models import Question
def question_choices(request):
question_list = []
question_set_id = request.GET.get('question_set_id')
questions = Question.objects.filter(question_set = question_set_id)
[question_list.append((each_question.pk,each_question.name)) for each_question in questions]
json = simplejson.dumps(question_list)
return HttpResponse(json, mimetype='application/javascript')
In urls.py:
from app.views import question_choices
urlpatterns = patterns(
(r'^question_set_change/', question_choices),
)
In admin.py where you want to load question based on question_set:
class Media:
js = ['/path/to/question_set_change.js',]