I've got an issue with Django where I am attempting to insert a new row into a MySQL database without providing NULLs for unspecified data.
For example, consider the following:
class MyModel(models.Model):
class Meta:
db_table = 'model_table'
managed = False
a = models.AutoField(primary_key=True)
b = models.CharField(max_length=255)
c = models.CharField(max_length=255, blank=True)
When providing the following data to the Serializer for a save():
data = list()
data['b'] = 'Some Text'
serializer = serializers.MyModelSerializer(data=data)
serializer.save()
The resulting SQL generated has NULL for every unspecified field, in this case 'c'. The table in the database does not allow NULL but has default values in place to handle unspecified data.
Is it possible to override this Serializer functionality to allow fields to be entirely omitted when attempting to insert, or will I have to create a new model with those fields omitted and have a 'creation' model for this specific case that does not have any awareness of these fields?
Note: the Serializer only has the field 'b' in its fields variable, so it has no immediate awareness of the 'c' column.
Edit: to try and clarify a little, this is the SQL that is generated, which due to nulls not being allowed in the column, is failing. 'c' does have default values however.
INSERT INTO `model_table` (`b`, `c`) VALUES ('Some Text', NULL)
It's fine for the AutoField of 'a' the PK, this is omitted by default, but 'c' is replaced with a NULL due to no value being provided in the data provided to the serializer, instead of being omitted form the SQL entirely. As a workaround I've duplicated the model and removed the fields I don't need from it just to get it working, and the SQL generated is valid due to the model itself having no awareness of the columns, but this does not seem like the best way to do this. The following is what I want this model and serializer combination to output
INSERT INTO `model_table` (`b`) VALUES ('Some Text')
so that 'c' is provided the default column value provided by MySQL itself. The model and serializer attmepting to insert NULL when I have provided not provided any data to the serializer is the problem.
you can override serializer create method like following
class UserSerializer(serializers.ModelSerializer):
profile = ProfileSerializer()
class Meta:
model = User
fields = ('username', 'email', 'profile')
def create(self, validated_data):
profile_data = validated_data.pop('profile')
user = User.objects.create(**validated_data)
Profile.objects.create(user=user, **profile_data)
return user
Related
I have a project where one model, called Request, has two fields (source, dest) that contain two ids which are not known to the user. However, each one is connected to another model User, who let's say that they have one field, username, which is known to the user.
Now, I want to make a serializer that can take usernames, and convert them into ids. (The opposite was simple to achieve, I just modified the to_representation method.) The problem is that when I send {'source': 'john', 'dest': 'jim'} the serializer does not take these data as valid. This was expected behavior, as it expected ids and got strings (usernames). However, even when I overridden the validate_source, validate_dest and validate methods to actually check that the usernames exist (instead of the ids), I am still getting errors that the serializer expected id but got string.
Are the validate, validate_<field> methods the wrong ones to override in this case?
Should I just convert the usernames into ids inside my view?
is it pythonic and good practice, django-wise, to receive some fields from the user and change them inside the serializer (as I change username into id)?
Current Serializer:
class RequestSerializer(serializers.ModelSerializer):
class Meta:
model = Request
fields = '__all__'
def validate_source(self, value):
username = value.get('username')
if username is None:
raise serializers.ValidationError('`user` field is required ')
return value
def validate_dest(self, value):
username = value.get('username')
if username is None:
raise serializers.ValidationError('`user` field is required ')
return value
def validate(self, attrs):
self.validate_source(attrs['source'])
self.validate_dest(attrs['dest'])
return attrs
def to_representation(self, instance):
# do things
pass
Please notice that this is not the whole functionality of my serializer. To convert from an id to a username I have to check the data of another Model, So I cannot use a SlugRelatedField.
Also, username is not the only item returned by the serializer. It also returns a 'class' field, depending on which group the the user has joined. The user may join more than one group, and each user-group combination has its own id. In the same way, when deserializing the data, I will need to read (1) the username, and then (2) the group, and find the correct id.
Thank you.
You probably can work with a SlugRelatedField [drf-doc]:
from rest_framework import serializers
class MyModelSerializer(serializers.ModelSerializer):
source = serializers.SlugRelatedField(
queryset=User.objects.all(),
slug_field='username',
)
dest = serializers.SlugRelatedField(
queryset=User.objects.all(),
slug_field='username',
)
class Meta:
model = MyModel
fields = ('source', 'dest')
This will return the username of the source and dest field of the model object, and in the opposite direction will fetch the User with the corresponding username.
I have a list dictionaries which I've parsed to JSON with json.dumps(). I would now like to POST this data to my database using Django REST framework.
# Example Data to POST
[
{
"key_1":"data_1",
"key_2":"data_2",
},
{
"key_1":"data_1",
"key_2":"data_2",
},
{
"key_1":"data_3",
"key_2":"data_4",
}
]
If we imagine that all entries are unique (which isn't the case with the above example dataset), we can successfully batch POST this data with:
# models.py
class data(models.Model):
key_1 = models.CharField(max_length=64, blank=True, null=True)
key_2 = models.CharField(max_length=64, blank=True, null=True)
class Meta:
unique_together = (( "key_1", "key_2"))
# serializers.py
class dataSerializer(serializers.ModelSerializer):
class Meta:
model = data
fields = '__all__'
# views.py
class dataViewSet(viewsets.ModelViewSet):
queryset=data.objects.all()
serializer_class=dataSerializer
filter_backends=[DjangoFilterBackend]
filterset_fields=['key_1', 'key_2']
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data, many=isinstance(request.data,list))
serializer.is_valid(raise_exception=True)
self.perform_create(serializer)
headers = self.get_success_headers(serializer.data)
return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)
# Initiating the POST request
api_url="localhost:8000/app/api/"
requests.post(
f"{api_url}data/",
data=my_json_serialised_data,
headers=headers
)
However, this will fail if some records already exist in the database ("fields must be unique together").
As per the example data, entries in the list will occasionally already be present in the database and I would therefore like to avoid POST'ing duplicates (based on the combination of fields in the model; I have specified unique_together to be explicit about which fields).
What's the best DRF way of checking whether a record exists and--if it does--to skip it, when you're dealing with a bulk POST? Should I use viewsets.ModelViewSet and override the create() method within that class? Or is there a better approach? I note that there are several ways to create records in DRF, so I'm seeking clarity on the best approach when we're dealing with a batch of data like in the above example.
To solve this, you can use a custom list serializer to be able to get the validated data as a list, which will then support creating your data objects in bulk using bulk_create.
In addition, you can ignore the constraint errors (like unique constraints in your case) with bulk_create's ignore_conflicts:
On databases that support it (all but Oracle), setting the ignore_conflicts parameter to True tells the database to ignore failure to insert any rows that fail constraints such as duplicate unique values. Enabling this parameter disables setting the primary key on each model instance (if the database normally supports it).
So all in all you can do something like:
class dataListSerializer(serializers.ListSerializer)
def create(self, validated_data):
return data.objects.bulk_create([
data(**validated) for validated in validated_data
], ignore_conflicts=True)
class dataSerializer(serializers.ModelSerializer):
class Meta:
model = data
fields = '__all__'
list_serializer_class = dataListSerializer
When many=True is flagged in the serializer, dataListSerializer's create will be used instead of dataSerializer which will support the abovementioned create in bulk.
I have a question regarding django rest framework.
Most of the time, I have a serializer which has some read-only fields. For example, consider this simple model below:
class PersonalMessage(models.Model):
sender = models.ForeignKey(User, related_name="sent_messages", ...)
recipient = models.ForeignKey(User, related_name="recieved_messages", ...)
text = models.CharField(...)
def __str__(self) -> str:
return f"{self.text} (sender={self.sender})"
In this model, the value of sender and recipient should be automatically provided by the application itself and the user shouldn't be able to edit those fields. Alright, now take a look at this serializer:
class PersonalMessageSerializer(serializers.ModelSerializer):
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')
It perfectly prevents users from setting an arbitrary value on the sender and recipient fields. But the problem is, when these fields are marked as read-only in the serializer, the serializer will completely ignore all the values that are passed into the constructor for these fields. So when I try to create a model, no values would be set for these fields:
PersonalMessageSerializer(data={**request.data, 'sender': ..., 'recipient': ...) # Won't work
What's the best way to prevent users from setting an arbitrary value and at the same time auto-populate those restricted fields in django rest framework?
Depending on how you get those two objects, you can use the serializer's save method to pass them, and they will automatically be applied to the object you are saving:
sender = User.objects.first()
recipient = User.objects.last()
serializer = PersonalMessageSerializer(data=request.data)
message = serializer.save(sender=sender, recipient=recipient)
The kwargs should match the field names in your model for this to work. For reference, have a look here
You able to override the serializer context like this;
PersonalMessageSerializer(data={**request.data, context={'sender': sender, 'recipent': recipent})
and catch the context inside serializer.
class PersonalMessageSerializer(serializers.ModelSerializer):
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')
def validate(self, attrs):
attrs = super().validate(attrs)
attrs['sender'] = self.context['sender']
attrs['recipent'] = self.context['recipent']
return attrs
now serializer.validated_data it must returns sender and recipent.
From the question it is not possible to understand what field(s) of the relationship with sender and recipient you want to interact with, but a general answer can be found in the Serializer relations section of Django REST documentation.
Long story short, if you want to interact with one field only, you can use SlugRelatedField, which lets you interact with the target of the relationship using only one of its fields.
If it just the id, you can use PrimaryKeyRelatedField.
If you want to interact with more than one field, the way to go is Nested Relationships. Here you can specify a custom serializer for the target relationship, but you will have to override the create() method in your PersonalMessageSerializer to create the object from your relationship, as nested serializers are read-only by default.
So this is how you can make set a default on create but read only after in DRF. Although in this solution it wont actually be readonly, it's writable, but you now have explicit control on what the logged in user can write, which is the ultimate goal
Given the model
class PersonalMessage(models.Model):
sender = models.ForeignKey(User,...)
recipient = models.ForeignKey(User,..)
text = models.CharField(...)
You would first create your own custom default (I will show an example for only one field)
# Note DRF already has a CurrentUserDefault you can also use
class CurrentSenderDefault:
requires_context = True
def __call__(self, serializer_field):
return serializer_field.context['request'].user
def __repr__(self):
return '%s()' % self.__class__.__name__
Next you make your own field, that knows whats up with the filter.
This queryset prevents people from setting a value they are not allowed to. which is exactly what you want
class SenderField(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
user = self.context['request'].user
if user:
queryset = User.objects.filter(id=user.id)
else:
queryset = User.objects.none()
return queryset
Finally on the serialiser you go
class PersonalMessageSerializer(serializers.ModelSerializer):
sender = SenderField(default=CurrentSenderDefault())
recipient = ...
class Meta:
model = PersonalMessage
fields = '__all__'
read_only_fields = ('sender', 'recipient')
I am totally new to Django/DRF, and trying to work with Create/Read/Update of a Model.
Here are the dummy model/serializers:
AddressModel and UserModel:
class AddressModel(models.Model):
name = models.CharField(max_length=255)
street = models.CharField(max_length=255)
class UserModel(models.Model):
email = models.CharField(max_length=255)
address = models.ForeignKey(Address, on_delete=models.PROTECT)
And I have BaseSerializer and WriteSerializer:
class UserBaseSerializer(serializers.ModelSerializer):
email = serializers.CharField(required=True)
address = AddressSerializer()
class Meta:
model = User
fields = ['email', 'address']
class UserWriteSerializer(UserBaseSerializer):
class Meta(UserBaseSerializer.Meta):
read_only_fields = ["created_by"]
def create(self, validated_data):
return super().create(validated_data)
Now the problem is, reading data through BaseSerializer, is working fine, I am able to display User and Address on UI correctly. But having issues with Create/Update.
For creating new user from UI, I have a select dropdown for Address, which has some constant values, these constant values are on UI side, it's not getting fetched from Backend, but backend will have related row in database.
And the issue is, I am not sending primary key of the address, I am sending name field of the address in the post call, so how can I still handle name field on Create serializer to store correct address, and return it in success?
validated_data in create method is not having Address instance. It's omitting that, may be due to non-primary field value.
Solution 1#:
One solution can be to fetch address from backend to display in select dropdown, so that I can send address id in api call.
Solution 2#:
Another solution could be to use serializers.PrimaryKeyRelatedField and convert data using to_internal_values and to_representation_value:
class UserWriteSerializer(UserBaseSerializer):
address = serializers.PrimaryKeyRelatedField(
queryset=Address.objects.all(), required=True
)
def to_internal_value(self, data):
address = Address.objects.filter(name=data.get("name")).first()
data["address"] = address.id
return super().to_internal_value(data)
def to_representation(self, instance):
data = super().to_representation(instance)
data["address"] = AddressSerializer(instance.address).data
return data
Above solution works, but mypy throws error that I am changing data type of address in WriteSerializer, while BaseSerializer has different type.
Is there another way to handle this non-primary field value in WriteSerializer or somewhere else?
This scenario is supported when using the SlugRelatedField. Just specify your name field as the "slug" and ensure it is unique.
class UserWriteSerializer(serializers.ModelSerializer):
address = serializers.SlugRelatedField(
slug_field="name",
queryset=Address.objects.all()
)
class Meta:
model = User
fields = ["id", "address", "email", "..."]
It is a good idea to split serializers, as you have done, for create/update vs. list. Be careful with inheriting, though, since any fields you add in the list view will automatically be editable in the create. Its safer to just make them entirely separate.
In one of my django models, I have a field like this:
modified = models.DateTimeField(auto_now=True)
I thought, when creating a ModelForm for this model, I could just skip this field, and Django would populate it automatically.
My ModelForm:
class FooForm(forms.ModelForm):
class Meta:
model = Foo
fields = ['text', 'name', 'description'] # notice - modified field not included -
# should not be shown to the user
But, even though it didn't show up in the form, when submitting, when creating a new object, I got an exception:
IntegrityError at /url/ - null value in column "modified" violates not-null constraint
How can I make this work?
The auto_now field is not populated automatically in such a case.
You need to use auto_add_now, so the field definition looks like this:
modified = models.DateTimeField(auto_now_add=True)
Then, django will add the date automatically if the field is not shown.
You just need to provide a default value to the field.
This can be done as follows:
from datetime import datetime
modified = models.DateTimeField(default=datetime.now)