Develop Reference

Cannot send JSONs from an external app but my Django test works

Tl;dr: Im trying to send a JSON over http with python, is my first script alright?
Im trying to build a django app where I would send information to add to the database by POSTing json strings from another python application.
To do that, I run this code to send the json (trimmed to the important part)
out_data = {
'session_id': session_id,
'count': count,
'gas_reading': gas_received,
'latitude': latitude,
'longitude': longitude,
'date_time': date_time.strftime("%Y-%m-%d %X")
}
out_json = json.dumps(out_data)
url = 'http://localhost:8000/sessions/post/'
response = requests.post(url, headers={'Content-type':'application/json'}, json=out_json)
print("Posted to ",url," and got ",response)
And this is the definition of the view that catches it.
def post(request):
if request.method=='POST':
received_data=json.loads(request.body)
session_id = int(received_data['session_id'])
if Session.objects.filter(session_id=session_id):
session = Session.objects.get(session_id=session_id)
else:
session = Session(session_id=session_id)
session.save()
session.measurement_set.create(
gas_reading=int(received_data['gas_reading']),
count=int(received_data['count']),
latitude=float(received_data['latitude']),
longitude=float(received_data['longitude']),
date_time = parse_datetime(received_data['date_time'])
)
return HttpResponseRedirect(reverse('index'))
elif request.method=='GET':
return HttpResponse("This is only for posting data")
I tried the View by using this test, which works:
class PostViewTests(TestCase):
def test_post_into_database(self):
data = {
'session_id': 69,
'count':100,
'gas_reading': 420,
'latitude': 4.13,
'longitude': 80.08,
'date_time': '2020-07-13 20:30:00'
}
headers = {'content-type':'application/json'}
self.client.post(reverse('readings:post'), content_type='application/json',data=data)
session=Session.objects.get(session_id=69)
measurement=session.measurement_set.last()
local_date_time = timezone.localtime(measurement.date_time)
self.assertEqual(session.session_id, 69)
self.assertEqual(measurement.count, 100)
self.assertEqual(measurement.gas_reading,420)
self.assertEqual(measurement.latitude,4.13)
self.assertEqual(measurement.longitude,80.08)
self.assertEqual(local_date_time.day,13)
self.assertEqual(local_date_time.month,7)
self.assertEqual(local_date_time.year,2020)
self.assertEqual(local_date_time.hour,20)
self.assertEqual(local_date_time.minute,30)
self.assertEqual(local_date_time.second,00)
I get a TypeError: string indices must be integers and fooling around with the debugger, I see that, indeed, I get a string with the json data instead of a dictionary object. Am I missing something in sending the json?

What is wrong: You're converting your out_data to JSON and still using json argument to pass out_json.
What you should be doing is directly using json argument with dictionary or using data argument with converted json.
response = requests.post(
url, headers={"Content-type": "application/json"}, json=out_data
)
OR
response = requests.post(
url, headers={"Content-type": "application/json"}, data=out_json
)

Why serializing a QuerySet I iobtain a string?

I'm using a js function to obtain some data from my django models. Concretely, I want to obtain the last value from my sensors.
I'm doing the following,
from django.core import serializers
def getData(request):
ctx = {}
if request.method == 'POST':
select = int(request.POST['Select'])
last_val = DevData.objects.order_by('dev_id','-data_timestamp').distinct('dev_id')
data = serializers.serialize('json', last_val)
print(data)
print('****************')
print(data[0]) # I just obtain a "[" then is a string not a list
ctx = {'Select':data}
return JsonResponse(ctx)
My question is, why the output is a string? How can I convert it to a Json object and then pass it to my js function?
Thank you very much!!

You obtain a string, because JSON is a text format. You can for example use json.loads to convert it back to a list of dictionaries:
from json import loads as jsonloads
from django.core import serializers
def getData(request):
ctx = {}
if request.method == 'POST':
select = int(request.POST['Select'])
last_val = DevData.objects.order_by('dev_id','-data_timestamp').distinct('dev_id')
data = jsonloads(serializers.serialize('json', last_val))
ctx = {'Select':data}
return JsonResponse(ctx)
The JSON serialization in Django is just a special JsonEncoder named DjangoJSONEncoder [GitHub], that has some special cases for a datetime object, etc.

Rendering a "<class 'django.template.response.TemplateResponse'>" Object in Django

I have a requirement to incorporate my existing working openstack horizon with our SSO using python-saml.
Hence i referred to demo docs which is written here:
https://github.com/onelogin/python-saml/blob/master/demo-django/demo/views.py#L113
So here as per the guide, I need to render the page as mentioned.
return render(request, 'auth/login.html', {'errors': errors, 'not_auth_warn': not_auth_warn, 'success_slo': success_slo, 'paint_logout': paint_logout, 'SSO': True})
When I am trying the same I am not getting the expected result on page, to be exact, I am not getting the exact desired page with full details. here is the more detail on the same.
Expected page(This is screenshot of page working fine, before i am enabling saml):
Actual page now(Username and password text box not visible, Because login.html file is not enough to render it, because it need form to fully display, using django_auth_views response object "res" is created for the same):
Code:
def login(request, template_name=None, extra_context=None, **kwargs):
"""Logs a user in using the :class:`~openstack_auth.forms.Login` form."""
if not request.is_ajax():.
if (request.user.is_authenticated() and
auth.REDIRECT_FIELD_NAME not in request.GET and
auth.REDIRECT_FIELD_NAME not in request.POST):
return shortcuts.redirect(settings.LOGIN_REDIRECT_URL)
# Get our initial region for the form.
initial = {}
current_region = request.session.get('region_endpoint', None)
requested_region = request.GET.get('region', None)
regions = dict(getattr(settings, "AVAILABLE_REGIONS", []))
if requested_region in regions and requested_region != current_region:
initial.update({'region': requested_region})
if request.method == "POST":
if django.VERSION >= (1, 6):
form = functional.curry(forms.Login)
else:
form = functional.curry(forms.Login, request)
else:
form = functional.curry(forms.Login, initial=initial)
if extra_context is None:
extra_context = {'redirect_field_name': auth.REDIRECT_FIELD_NAME}
if not template_name:
if request.is_ajax():
template_name = 'auth/_login.html'
extra_context['hide'] = True
else:
template_name = 'auth/login.html'
res = django_auth_views.login(request,
template_name=template_name,
authentication_form=form,
extra_context=extra_context,
**kwargs)
# Save the region in the cookie, this is used as the default
# selected region next time the Login form loads.
if request.method == "POST":
utils.set_response_cookie(res, 'login_region',
request.POST.get('region', ''))
req = prepare_django_request(request)
auth = init_saml_auth(req)
errors = []
not_auth_warn = False
success_slo = False
attributes = False
paint_logout = False
if 'sso' in req['get_data']:
return HttpResponseRedirect(auth.login())
elif 'slo' in req['get_data']:
name_id = None
session_index = None
if 'samlNameId' in request.session:
name_id = request.session['samlNameId']
if 'samlSessionIndex' in request.session:
session_index = request.session['samlSessionIndex']
slo_built_url = auth.logout(name_id=name_id, session_index=session_index)
request.session['LogoutRequestID'] = auth.get_last_request_id()
print ('set logout id'+auth.get_last_request_id())
return HttpResponseRedirect(slo_built_url)
if 'samlUserdata' in request.session:
paint_logout = True
if len(request.session['samlUserdata']) > 0:
attributes = request.session['samlUserdata'].items()
return render(request, 'auth/login.html', {'errors': errors, 'not_auth_warn': not_auth_warn, 'success_slo': success_slo, 'paint_logout': paint_logout, 'SSO': True})
Would like to know what may be the problem here, What can be the fix to overcome this issue.
I have given try as below, tried returning the TemplateResponse object which was created in above mentioned code correct working code was just returning 'res' object from code.
return res
hence I tried to return the object instead of html file. ie: 'res' instead of 'auth/login.html'.
return render(request, res, {'errors': errors, 'not_auth_warn': not_auth_warn, 'success_slo': success_slo, 'paint_logout': paint_logout, 'SSO': True})
It returns error as follow:
Getting an error as follows:
ContentNotRenderedError at /auth/login/
The response content must be rendered before it can be accessed.
During analysis, I can see that template object(res), Which is of type: class 'django.template.response.TemplateResponse'
Someone please drop thoughts to figure out how we can resolve.

You cannot access body after reading from request's data stream after starting py.test

Good afternoon.
I'm testing api based on django-rest-framework using pytest. I have the following method that creates a new object (method taken from here):
class JSONResponse(HttpResponse):
"""
An HttpResponse that renders its content into JSON.
"""
def __init__(self, data, **kwargs):
content = JSONRenderer().render(data)
kwargs['content_type'] = 'application/json'
super(JSONResponse, self).__init__(content, **kwargs)
#csrf_exempt
#api_view(('POST',))
#permission_classes((IsAuthenticated, ))
def create_transaction(request):
"""
The method takes the data in JSON-format.
If the data is correct Transaction object will created, otherwise it returns an error also in JSON-format.
"""
stream = StringIO('[' + request.raw_post_data + ']')
data = JSONParser().parse(stream)
serializer = NewTransactionSerializer(data=data, many=True)
if serializer.is_valid():
serializer.save()
return JSONResponse(serializer.data, status=201)
else:
return JSONResponse(serializer.errors, status=400)
I wrote to it next test:
#pytest.mark.django_db
def test_create_method(client):
correct_data = '''{ "var1": "111",
"var2": "222",
"var3": 2 }'''
client.login(username='test2#github.com', password='test')
data = json.loads(correct_data)
response = client.post('/rest2/create_transaction/', data, format='json')
content = json.loads(response.content)
assert content[0]['var1'] == '111'
assert content[0]['var2'] == '222'
assert content[0]['var3'] == 2
assert response['Content-Type'] == 'application/json'
assert response.status_code == 201
When starting pytest displays the following: Exception: You cannot access body after reading from request's data stream. Its broke when i post data to url.
When I run the same code in the shell, the code runs without problems. I am new to testing, may miss something, help please.

If you're using django-rest-framework, then you can just use request.data instead of trying to parse json from the request yourself
http://www.django-rest-framework.org/api-guide/requests/
stream = StringIO('[' + request.raw_post_data + ']')
data = JSONParser().parse(stream)
this can be replaced with
data = request.data

Filter only if the value is defined in Django

I have the following view:
def process(request):
if request.method == 'POST':
data = request.POST
results = Specs.objects.filter(screenGroup = data['screen_user'], storage = data['storage_user'], mSystem = data['system_user'] )
context = {'results' : results}
return render(request, 'process.html', context)
When the user inputs the three values it filters correctly, but when it just inputs one or two (or nothing), then it filters passing the value None. Is there any way to ignore the filter if it's not set?
Thanks!
EDIT:
The following code is working, but it's obviously a very unefficient way:
def process(request):
if request.method == 'POST':
data = request.POST
if(data['screen_user'] != None):
results = Specs.objects.filter(screenGroup = data['screen_user'])
elif (data['storage_user'] != None):
results = Specs.objects.filter(storage = data['storage_user'])
else:
results = Specs.objects.all()
#plus all the other options...
context = {'results' : results}
return render(request, 'process.html', context)

You can build the filter beforehand:
def process(request):
if request.method == 'POST':
data = request.POST
spec_filter = {}
for attribute in ['screenGroup', 'storage', 'mSystem']:
if attribute in data and data[attribute]:
spec_filter[attribute] = data[attribute]
results = Specs.objects.filter(**spec_filter)
context = {'results' : results}
return render(request, 'process.html', context)
NB: To use this verbatim you would have to change the names of the variables being passed in the request.POST to match those in the Specs model. I did this just to illustrate, but you can easily use the same principle with your variable names. In that case you'll have to be a bit more verbose.

It's called validating your form.. There are two ways of doing this:
create a django form and use myform.is_valid(). You can read about it in the docs
validate it yourself with a few 'if' statements (either on server side or with javascript before sending the ajax call)