Develop Reference

Why am I getting 422 status code during the post endpoint testing, unit testing?

During the running test case of my application, I keep on seeing test failed even when the data is well formatted. The following are my code snippets:
I was able to create a new user through its application interface, but trying its behaviour, it was giving me an unexpected error status code, 422. I don't really know what was going wrong with the snippets. Here I have included all the following code for better look into the stated issue.
For the endpoint /users
#app.route('/users/, methods=['POST'])
def create_user():
try:
body = request.get_json((
new_user = body get('user_name', None)
if user_name is None:
about(405)
new_entry = User(new_user)
new_entry.insert()
return jsonify({
'success': True
)}
except:
abort(422)
Here is my model_class:
class User(db.Model):
__tablename__='users'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
score = Column(Integer, nullable=False)
def __init__(self, name, score=0):
self.name = name
self.score = score
def insert(self):
db.session.add(self)
db.session.commit()
def format(self):
return {
'id': self.id,
'name': self.name,
'score': self.score
And here is my test_case_file:
class TriviaTestCase(unit test.TestCase):
def setUp(self):
self.app = create_app()
self.client = self.app.test_client
self.database_path = "PostgreSQL://postgres:postgresspass#localhos:5432/user_test_db"
setup_db(self.app, self database_path)
self.new_user = {
"username":"P.Son",
"score": 0
}
def test_create_user(self):
res = self.client().post("/users", json=self.new_user)
data = json.loads(res.data)
self.assertEqual(res.status_code, 200)
self.assertEqual(data['success'], True)
if __name__=="__main__":
unittest.main()
Output of my test:
============================
FAIL: test_create_user (__main__.TriviaTestCase)
----------------------------
Traceback (most recent call last):
File "C:\...\test_flaskr.py", line 201, in test_create_user
self.assertEqual(res.status_code, 200)
AssertionError: 422 != 200
Note Other endpoints pass the test but the above endpoint has been failing test. I don't know what was wrong with the snippets I have written.

First of all, you should always be logging such errors when they happen in your endpoints.
Second, you are returning the 422 error yourself in case of "any" exception and there is some in your code. One of which is the fact that you are just passing user_name to the constructor of the User class but there must be a score variable present too.
Another problem is that you should pass key-value arguments to the User class constructor, not a dictionary.
is should be something like this:
body = request.get_json()
user_name = body.get('user_name', '')
score = body.get('score', '')
if not all([user_name, score]):
# return some customized error here
new_entry = User(user_name=user_name, score=score)
Also, you should consider some form of input validation rather than just checking the data yourself. Something like Marshmellow would be fine.

How is throttling disabled for testing in Django Rest Framework?

Upon implementing a throttle for a REST API, I'm encountering an issue when running my tests all at once.
Upon isolating the subject TestCase and running the test runner, the TestCase passes its assertions. However when all the tests are ran I get the following error: AssertionError: 429 != 400. Which that type of error of course is due to the requests exceeding a rate limit.
How can I disable throttling for the tests so the assertion error is not raised. I decorated the TestCase with #override_settings but that doesn't have any effect.
from copy import deepcopy
from django.conf import settings
from django.test import TestCase, override_settings
from django.contrib.auth.models import User
from rest_framework.test import APITestCase, APIClient
from django.urls import reverse
from ..models import QuestionVote, Question
from users.models import UserAccount
from tags.models import Tag
from .model_test_data import mock_questions_submitted
REST_FRAMEWORK = deepcopy(settings.REST_FRAMEWORK)
del REST_FRAMEWORK['DEFAULT_THROTTLE_RATES']
#override_settings(REST_FRAMEWORK=REST_FRAMEWORK)
class TestUserVoteOnOwnQuestion(APITestCase):
'''Verify that a User cannot vote on their own Question'''
#classmethod
def setUpTestData(cls):
cls.user1 = User.objects.create_user("Me", password="topsecretcode")
cls.user1_account = UserAccount.objects.create(user=cls.user1)
cls.tag = Tag.objects.create(name="Tag")
cls.q = mock_questions_submitted[2]
cls.q.update({'user_account': cls.user1_account})
cls.question = Question(**cls.q)
cls.question.save()
cls.question.tags.add(cls.tag)
def test_vote_on_own_posted_question(self):
self.client.login(username="Me", password="topsecretcode")
response = self.client.put(
reverse("questions_api:vote", kwargs={'id': 1}),
data={"vote": "upvote"}
)
self.assertEqual(response.status_code, 400)
self.assertEquals(
response.data['vote'],
"Cannot vote on your own question"
)
REST_FRAMEWORK = {
'TEST_REQUEST_DEFAULT_FORMAT': 'json',
'DEFAULT_THROTTLE_RATES': {
'voting': '5/minute'
}
}
class UserQuestionVoteView(APIView):
renderer_classes = [JSONRenderer, ]
parser_classes = [JSONParser, ]
permission_classes = [IsAuthenticated, ]
authentication_classes = [SessionAuthentication, ]
throttle_classes = [ScopedRateThrottle, ]
throttle_scope = "voting"
def put(self, request, id):
# import pdb; pdb.set_trace()
account = UserAccount.objects.get(user=request.user)
question = Question.objects.get(id=id)
if account == question.user_account:
return Response(data={
'vote': "Cannot vote on your own question"
}, status=400)
try:
stored_vote = QuestionVote.objects.get(
account=account, question=question
)
serializer = QuestionVoteSerializer(stored_vote, request.data)
except QuestionVote.DoesNotExist:
serializer = QuestionVoteSerializer(data=request.data)
finally:
if serializer.is_valid(raise_exception=True):
question_vote = serializer.save(
account=account,
question=question
)
vote = serializer.validated_data['vote']
if vote == "downvote":
question.vote_tally = F('vote_tally') - 1
else:
question.vote_tally = F('vote_tally') + 1
question.save()
question.refresh_from_db()
return Response(data={
'id': question.id,
'tally': question.vote_tally
})
return Response(serializer.errors)

One way to do this is by setting your config files up to support testing versions:
# config.py
REST_FRAMEWORK = {
'TEST_REQUEST_DEFAULT_FORMAT': 'json',
'DEFAULT_THROTTLE_RATES': {
'voting': '5/minute'
}
}
TESTING = len(sys.argv) > 1 and sys.argv[1] == 'test'
if TESTING:
del REST_FRAMEWORK['DEFAULT_THROTTLE_RATES']
The pro of this approach is you're not hacking away at your application in tests and hiding modifications to the config file - all of your testing based changes are in the same config file as their true values.
The con of this approach is unless your developers know this setting and values are there, they may be scratching their heads as to why the throttling doesn't work in tests, but does at runtime.

My solution was to apply some monkey patching.
I have a throttles.py file where I have custom throttles, such as
class UserBurstRateThrottle(UserRateThrottle):
rate = '120/minute'
What I've done is create a stub allow_request function to always return true, so something like
def apply_monkey_patching_for_test():
def _allow_request(self, request, view):
return True
UserBurstRateThrottle.allow_request = _allow_request
Then, in the test_whatever.py file, I add the following at the top.
from my_proj import throttles
throttles.apply_monkey_patching_for_test()

Another easy way is to disable the cache that's responsible for storing the clients' meta data (https://www.django-rest-framework.org/api-guide/throttling/#setting-up-the-cache). So you need this in your test settings:
CACHES = {
'<throttling-cache-name|default>': {
'BACKEND': 'django.core.cache.backends.dummy.DummyCache', # to prevent API throttling
}
}

Queryset return response customize

I'm pretty new to Django restframework, what i'm trying now is to return object with foreignkey.
class User(models.Model):
name = models.CharField(max_length=255,blank=True)
date_created = models.DateTimeField(auto_now_add=True)
date_modiefied = models.DateTimeField(auto_now=True)
area = models.CharField(max_length=255,blank=True)
uuid = models.CharField(max_length=255)
home = models.CharField(max_length=255,blank=True)
work = models.CharField(max_length=255,blank=True)
mobileNo = models.CharField(max_length=255,blank=True)
email = models.CharField(max_length=255,blank=True)
appVersionCode = models.CharField(max_length=255,blank=True)
photoUrl = models.CharField(max_length=255,blank=True)
serverTime = models.CharField(max_length=255,blank=True)
fcmTokenId = models.CharField(max_length=255,blank=True)
def __str__(self):
return self.name
class LocationData(models.Model):
user = models.ForeignKey(
User, related_name='user', on_delete=models.DO_NOTHING)
source_id = models.CharField(max_length=255)
latitude = models.CharField(max_length=255)
longitude = models.CharField(max_length=255)
speed = models.CharField(max_length=255)
kms = models.CharField(max_length=255)
date_created = models.DateTimeField(auto_now=True)
date_modiefied = models.DateTimeField(auto
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = '__all__'
class LocationDataSerializer(serializers.ModelSerializer):
class Meta:
model = LocationData
fields = '__all__'
depth = 1
I'm using def get_queryset(self):
class SyncIndexLastDataViewSet(viewsets.ModelViewSet):
serializer_class = LocationDataSerializer
def get_queryset(self):
userid = self.request.query_params.get('user_id', None)
userExist = User.objects.filter(id=userid)
if userExist.exists():
# call the original 'list' to get the original response
queryset = LocationData.objects.values('source_id').filter(user__id=userid).order_by('-source_id')[:1]
lastSourceId = queryset[0]['source_id']
response = {"collection": {"data": lastSourceId,"statusCode": status.HTTP_200_OK,"version":"1.0"}}
json = JSONRenderer().render(response)
# customize the response data
if response is not None:
return json
else:
# return response with this custom representation
response = {"collection": {"data": "","statusCode":status.HTTP_404_NOT_FOUND,"version":"1.0","error":"Not found"}}
return response
Right now the result is inside the response is below and immediately it throws this error
But i want that queryset to return as below one, Hence i can read those key-pair values in android
{ "collection": {
"data": {
"id": 31,
"source_id": "55",
"latitude": "24654",
"longitude": "454654",
"date_created": "2019-02-08T17:10:09.318644Z",
"date_modiefied": "2019-02-08T17:10:09.318714Z",
"area": "54546",
"user": {
"id": 1,
"name": "Dormy",
"date_created": "1992-01-18T03:29:53.388000Z",
"date_modiefied": "2018-02-19T05:17:00.164000Z",
"serverTime": "",
"fcmTokenId": ""
}
},
"statusCode": 200,
"version": "1.0"
}
Now the error throws
AttributeError: Got AttributeError when attempting to get a value for field source_id on serializer LocationDataSerializer.
The serializer field might be named incorrectly and not match any attribute or key on the int instance.
Original exception text was: 'int' object has no attribute 'source_id'.
Thanks!

The answer to this depends on what type of view you are using but the bottom line is you don't do this in get_queryset you do this in the method for the type of reguest.
For example if you are using a RetrieveAPIView you should override the retrieve method from the RetrieveModelMixin like so:
class MyAPIView(RetrieveAPIView):
queryset = MyModel.objects.all()
serializer_class = MySerializer
def retrieve(self, request, *args, **kwargs):
instance = self.get_object()
serializer = self.get_serializer(instance)
data = {
"collection": {
"data": serializer.data
},
"statusCode": 200,
"version": "1.0"
}
return Response(data)
If you are using something else like a ListAPIView then you want to see what is used by that in the relevant method and override that to wrap your data.
The main thing to realise here is that it has nothing to do with getting the queryset - which is just about getting data from the database. This is about transforming the data into the correct format when sending back a response. As a result the work should be done at the point the response is made.

There are couple of solution possible for this problem. NDevox already mention how we can overwrite our retrive function and get our expected response. But If we want this will be done with every response for every api end-point and if we go this way we need to overwrite every function then its quite burden and its DRY we should avoid this as possible. One of the possible way to introduce a middleware or overwrite Response so that we can get our generic response for every-api end-point without explicitly overwrite every functionality.
Possible Solution One
As we are using DRF here we can add our own return responses with various media types, say for application/json.
First We need to add in our settings.py
REST_FRAMEWORK = {
...
'DEFAULT_RENDERER_CLASSES': (
'app_name.renderers.ApiRenderer', # our own render middleware
),
...
}
And in our custom render middleware
from rest_framework.renderers import BaseRenderer
from rest_framework.utils import json
class ApiRenderer(BaseRenderer):
def render(self, data, accepted_media_type=None, renderer_context=None):
our_response_dict = {
'version': '1.0'
'data': {},
'message': '',
}
if data.get('data'):
our_response_dict['data'] = data.get('data')
if data.get('status'):
our_response_dict['statusCode'] = data.get('status')
if data.get('message'):
our_response_dict['message'] = data.get('message')
data = our_response_dict
return json.dumps(data)
Reference Link
Possible Solution Two
If we are using ModelViewset then there is another way we can achievement that. Say Our Views.py are like following
class A(serializer.ModelSerializer):
........
class B(serializer.ModelSerializer):
........
class C(serializer.ModelSerializer):
........
Our goal is to overwrite ModelViewset's to_representation function and return our custom result. This will like as following
from collections import OrderedDict
class OurParentViewset(serializer.ModelSerializer):
......
def to_representation(self, instance):
data = super(serializers.ModelSerializer, self).to_representation(instance)
result = OrderedDict()
result['data'] = data
result['version'] = '1.0'
result['statusCode'] = '2xx' # i am not fully sure how to customize this
return result
class A(OurParentViewset):
........
class B(OurParentViewset):
........
class C(OurParentViewset):
........

Implementing a custom renderer here seems to be a ways to go.
You can have requests from your android client include in the Accept header a way to identify the client to the renderer. 1 e.g.
Accept: application/json; android=true
Then compose a renderer using the JSONRenderer class to provide the format for your Android client.
# ./formatters/android_format.py
from rest_framework.renderers import JSONRenderer, BaseRenderer
from django.http.multipartparser import parse_header
class AndroidV1FormatRenderer(BaseRenderer):
media_type = 'application/json'
format = 'json'
json_renderer = JSONRenderer()
def android(self, accepted_media_type):
base_media_type, params = parse_header(accepted_media_type.encode('ascii'))
return 'android' in params
def render(self, data, accepted_media_type=None, renderer_context=None):
response = renderer_context['response']
android = self.android(accepted_media_type)
if android:
data = {
"collection": {"data": data},
"statusCode": response.status_code,
"version": "1.0"
}
return json_renderer.render(
wrapped_data, accepted_media_type, renderer_context)
This can then be used where you require response formatted that way using renderer_classes attribute of your APIView. 2

Since get_queryset won't allow you to customize the response data. I decide to take the query value that's important to me.
http://localhost/api/users/?user_id=1 --> changed into ...api/users/1
def retrieve(self, request, *args, **kwargs):
""" userid = self.request.query_params.get('user_id', None) """
userid = kwargs.get('pk')
userExist = User.objects.filter(id=userid)
if userExist.exists():
# call the original 'list' to get the original response
queryset = LocationData.objects.values('source_id').filter(user__id=userid).order_by('-source_id')[:1]
lastSourceId = queryset[0]['source_id']
response = {"collection": {"data": lastSourceId,"statusCode": status.HTTP_200_OK,"version":"1.0"}}
# customize the response data
if response is not None:
return Response(response)
else:
# return response with this custom representation
response = {"collection": {"data": "","statusCode":status.HTTP_404_NOT_FOUND,"version":"1.0","error":"Not found"}}
return response

Avoiding repeated names in JSON with MongoEngine - Python

I'm looking for a way/function/method to make it impossible to save two identical names on my JSON, for example, I got this JSON with repeated names:
[
[
{
"id": "59a5c80dc75969297837c51e",
"name": "uza",
"password": "3648726"
},
{}
],
[
{
"id": "59a5c811c75969297837c51f",
"name": "kuza",
"password": "3648726"
},
{}
],
[
{
"id": "59a5c83ec75969297837c520",
"name": "kuza",
"password": "3648726"
},
{}
]
]
My code that creates an user is this one:
#api.route('/', methods=['POST'])
def create():
# Grabs the data from the requisition
user_json = request.get_json(silent=True)
if not user_json:
return "FAIL"
# creates an entity JSON
user, errors = schema.load(user_json)
if bool(errors):
return jsonify(errors)
user.save()
return "SUCCESS"
Again, I'm using mongoengine, anybody knows how to do it?
Edited to add my model.py
rom mongoengine import Document
from mongoengine import StringField, ReferenceField
import marshmallow_mongoengine as ma
from marshmallow import Schema, fields
from .service import ImageService
class User(Document):
name = StringField(unique=True)
password = StringField(unique=True)
class Face(Document):
user = ReferenceField(User)
image = StringField()
embedding = StringField()
def get_embedding(self):
return ImageService().from_base64_flat(self.embedding.encode())
def get_image(self):
return ImageService().from_base64(self.image.encode())
class UserSchema(ma.ModelSchema):
class Meta:
model = User
class FaceSchema(ma.ModelSchema):
class Meta:
model = Face
image = ma.fields.Method(deserialize="img_to_base64", serialize="img_to_base64")
embedding = ma.fields.Method(deserialize="to_base64", serialize="to_base64")
def img_to_base64(self, data):
return ImageService().to_base64(data)
def to_base64(self, data):
return ImageService().np_to_base64(data)
In the class User i changed it from required to unique, now i can't add one with the same name but in exchange it returns an INTERNAL SERVER ERROR on Insomnia, and as you can see i put another unique on the password for test and it didn't worked it saves even if there is another user with the same password.

Try this
class User(Document):
name = StringField(unique=True)
password = StringField(unique=True)
meta = {
'indexes': [{'fields': ['name'], 'unique': True}]
}

A Test to Support Both JSON and File Multipart Uploads in DRF

I would like to write a test for my DRF app that posts both json and a file using multipart.
This is what I have tried so far but collection_items (in the create method) is blank. Do I need to modify my view to make this work correctly, or am I doing something incorrectly within my test case below?
My Test:
image = Image.new('RGB', (100, 100))
tmp_file = tempfile.NamedTemporaryFile(suffix='.jpg')
image.save(tmp_file)
files = {"collection_items": [{"image": tmp_file}]}
payload = json.dumps({
"title": "Test Collection",
})
self.api_factory.credentials(Authorization='Bearer ' + self.token)
response = self.api_factory.post(url, data=payload, files=files, format='multipart')
This is the model:
class Collection(models.Model):
title = models.CharField(max_length=60)
collection_items = models.ManyToManyField('collection.Item')
class Item(models.Model):
image = models.ImageField(upload_to="/",null=True, blank=True)
Serializers:
class ItemCollectionDetailSerializer(serializers.ModelSerializer):
class Meta:
model = Item
fields = ('id', 'image')
read_only_fields = ('image',)
class CollectionListSerializer(serializers.ModelSerializer):
url = serializers.HyperlinkedIdentityField(view_name='col_detail')
collection_items = ItemCollectionDetailSerializer(many=True, required=True)
class Meta:
model = Collection
fields = ('url', 'id', 'collection_items')
def create(self, validated_data):
item_data = validated_data.pop('collection_items')
print(item_data) # <----- **EMPTY HERE???**
etc ....edited for brevity
So print(item_data) is empty [], why? How to I resolve this?
This is my entire view: below, do I need to do something here?
class CollectionListView(generics.ListCreateAPIView):
queryset = Collection.objects.all()
serializer_class = CollectionListSerializer
I am using Django Rest Framework 3.x, Django 1.8.x and Python 3.4.x.
Update
I have tried below but still no joy! collection_items is empty in my create. This either has to do with the fact it's a nested object or something has to happen in my view.
stream = BytesIO()
image = Image.new('RGB', (100, 100))
image.save(stream, format='jpeg')
uploaded_file = SimpleUploadedFile("temp.jpeg", stream.getvalue())
payload = {
"title": "Test Collection",
"collection_items": [{"image": uploaded_file}],
}
self.api_factory.credentials(Authorization='Bearer ' + self.test_access.token)
response = self.api_factory.post(url, data=payload, format='multipart')
Update 2
If I change my payload to use json.dumps it seems to now see the file but of course this cannot work!
payload = json.dumps({
"title": "Test Collection",
"collection_items": [{"image": uploaded_file}],
})
Error
<SimpleUploadedFile: temp.jpeg (text/plain)> is not JSON serializable
PS
I know the file is being uploaded because if I do the following in my serializer...
print(self.context.get("request").data['collection_items'])
I get
{'image': <SimpleUploadedFile: temp.jpeg (text/plain)>}

Using the multipart parser you can just pass the file handler in the post arguments (see this). In your code you are submitting a json-encoded part as the data payload and the file part in a files argument, and I don't think it can work that way.
Try this code:
from PIL import Image
from io import BytesIO
from django.core.files.uploadedfile import SimpleUploadedFile
stream = BytesIO()
image = Image.new('RGB', (100, 100))
image.save(stream, format='jpeg')
uploaded_file = SimpleUploadedFile("file.jpg", stream.getvalue(), content_type="image/jpg")
payload = {
"title": "Test collection",
"collection_items": [{"image": uf}],
}
self.api_factory.credentials(Authorization='Bearer ' + self.token)
self.api_factory.post(url, data=payload, format='multipart')
...
I'm not entirely sure the nested serialization works, but at least the file upload should work.