Develop Reference

How to correctly decode text content POST with Flask

What is the correct way to handle character encoding when processing a POST request containing "text/plain" data in Flask?
Test application:
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route("/put-text", method=['POST'])
def hello_world():
print("data", request.data)
return {}
Example request
import requests
url = 'http://127.0.0.1:5000/put-text'
data = 'The quick brown fox jumped over the lazy dog."
headers = {
"Content-Type:":"text/plain",
}
res = requests.post(url, data=data, headers=headers)
With this, the request.data is bytes, and I could decode it relatively easily, but how do I make this code robust to support unicode or different text encodings?

According to the Flask docs, request.get_data(as_text=True) probably does what you want:
#app.route("/put-text", method=['POST'])
def hello_world():
text = request.get_data(as_text=True)
# do something with the `text`...
return {}
Presumably, Flask will decode the text with the correct encoding, as specified by the client in the respective header. Provided the client specifies the correct encoding, of course...

Sending a POST request to RESTFUL API(Flask), but receiving a TypeError

So I am trying to send certain values to my Flask API from my streamlit application, but I am not sure as to why I am getting a type error(Using this as reference: Sending a POST request to my RESTful API(Python-Flask), but receiving a GET request).
Type error: The view function for 'get_data' did not return a valid response. The function either returned None or ended without a return statement
app.py
import requests
import streamlit as st
...
api_url = requests.get("http://127.0.0.1:5000/") # Flask url
create_row_data = {'name': name, 'type': get_type(name), 'token_value': token_value, 'external': external, 'start': start, 'end': end, 'step': step}
print(create_row_data)
r = requests.post(url=api_url, json = create_row_data)
The url in the main.py has https
main.py
#app.route('/', methods=['GET', 'POST'])
def get_data():
if request.method =='POST':
p_name = request.json['name']
p_type = request.json['type']
...
p_end = request.json['end']
p_step = request.json['step']
create_row_data = {'p_name': str(p_name), 'p_type': str(p_type), ... , 'p_end': str(p_end), 'p_step': str(p_step)}
print(create_row_data)
response = requests.post(url, data=json.dumps(create_row_data), headers= {'Content-type': 'application/json'}
return response.content
What I want to do is send the values to my Flask API which would then use those values, return a dataframe, and send the dataframe to the streamlit application.
Any help is greatly appreciated.

Your route does not return anything for a GET request (return response.content is handled within the POST).
e.g. if you add return 'ok', 200 below your POST handler, it will handle the GET request

How can i read the data of headers of a post request sent by python requests?

Looking for the help, I am making a post request to an django application with AUTH-TOKEN in the header and trying to access it on view but not able to.
python app
import requests
import json
URL = "http://127.0.0.1:8000/test/api/"
request_data ={
'order':{
'id':'order.id',
'payment_collection_status': 'transaction_status',
'payment_collection_message': 'transaction_message'
}
}
request_headers = {'AUTH-TOKEN':'webhook_auth_token'}
json_data = json.dumps(request_data)
response = requests.post(url = URL,headers=request_headers, data = json_data)
print(response)
django application view
from django.shortcuts import render,HttpResponse
from django.views.decorators.csrf import csrf_exempt
#csrf_exempt
def request_handle(request):
if (request.method=='POST'):
print(request.body)
print(request.META["AUTH-TOKEN"])
return HttpResponse('<h1>hello</h1>')
but not able read the header data it is throwing an error: KeyError: 'AUTH-TOKEN'

Try to use
request.META.get('HTTP_AUTH_TOKEN')
request.headers.get('AUTH-TOKEN') if you're on Django>=2.2
More details on Documentation

AWS Chalice, can't get image from POST request

I'm trying to invoke my sagemaker model using aws chalice, a lambda function, and an API Gateaway.
I'm attempting to send the image over POST request but I'm having problem receiving it on the lambda function.
My code looks like:
from chalice import Chalice
from chalice import BadRequestError
import base64
import os
import boto3
import ast
import json
app = Chalice(app_name='foo')
app.debug = True
#app.route('/', methods=['POST'], content_types=['application/json'])
def index():
body = ''
try:
body = app.current_request.json_body # <- I suspect this is the problem
return {'response': body}
except Exception as e:
return {'error': str(e)}
It's just returning
<Response [200]> {'error': 'BadRequestError: Error Parsing JSON'}
As I mentioned before, my end goal is to receive my image and make a sagemaker request with it. But I just can't seem to read the image.
My python test client looks like this:
import base64, requests, json
def test():
url = 'api_url_from_chalice'
body = ''
with open('b1.jpg', 'rb') as image:
f = image.read()
body = base64.b64encode(f)
payload = {'data': body}
headers = {'Content-Type': 'application/json'}
r = requests.post(url, data=payload, headers=headers)
print(r)
r = r.json()
# r = r['response']
print(r)
test()
Please help me, I spent way to much time trying to figure this out

That error message is because you're not sending a JSON body over to your Chalice app. One way to check this is by using the .raw_body property to confirm:
#app.route('/', methods=['POST'], content_types=['application/json'])
def index():
body = ''
try:
#body = app.current_request.json_body # <- I suspect this is the problem
return {'response': app.current_request.raw_body.decode()}
except Exception as e:
return {'error': str(e)}
You'll see that the body is form-encoded and not JSON.
$ python client.py
<Response [200]>
{'response': 'data=c2FkZmFzZGZhc2RmYQo%3D'}
To fix this, you can use the json parameter in the requests.post() call:
r = requests.post(url, json=payload, headers=headers)
We can then confirm we're getting a JSON body in your chalice app:
$ python client.py
<Response [200]>
{'response': '{"data": "c2FkZmFzZGZhc2RmYQo="}'}

So I was able to figure it out with the help of an aws engineer (i got lucky I suppose). I'm including the complete lambda function. Nothing changed on the client.
from chalice import Chalice
from chalice import BadRequestError
import base64
import os
import boto3
import ast
import json
import sys
from chalice import Chalice
if sys.version_info[0] == 3:
# Python 3 imports.
from urllib.parse import urlparse, parse_qs
else:
# Python 2 imports.
from urlparse import urlparse, parse_qs
app = Chalice(app_name='app_name')
app.debug = True
#app.route('/', methods=['POST'])
def index():
parsed = parse_qs(app.current_request.raw_body.decode())
body = parsed['data'][0]
print(type(body))
try:
body = base64.b64decode(body)
body = bytearray(body)
except e:
return {'error': str(e)}
endpoint = "object-detection-endpoint_name"
runtime = boto3.Session().client(service_name='sagemaker-runtime', region_name='us-east-2')
response = runtime.invoke_endpoint(EndpointName=endpoint, ContentType='image/jpeg', Body=body)
print(response)
results = response['Body'].read().decode("utf-8")
results = results['predictions']
results = json.loads(results)
results = results['predictions']
return {'result': results}

Force Content-Type or expose request.data in Flask for known content-type

I am recreating a service in Python/Flask and am running into an issue with the way the existing clients authenticate. I have to match the existing clients scheme for compatibility reasons.
The existing clients take the username, password and base64 encode it. This is not HTTP Basic Authentication, despite sounding similar. Below is some sample code that would create this login request.
credentials = {
'username': 'test#example.com',
'password': 'password'
}
data = b64encode(urlencode(credentials))
request = urllib2.Request(loginURL)
request.add_data(data)
# request.add_header('Content-Type', 'application/gooblygop')
# 'application/x-www-form-urlencoded' seems to be a default Content-Type
login = urllib2.urlopen(request)
On the server side, I take the POST data and base64 decode it to get the username and password information again.
flask server:
#app.route('/login', methods=['POST'])
def login():
error = None
if request.method == 'POST':
# post data: cGFzc3dvcmQ9ZGVmYXVsdCZlbWFpbD10ZXN0JTQwZXhhbXBsZS5jb20=
data = b64decode(request.data)
# decoded data: password=default&email=test%40example.com
return('ok')
The problem is the Content Type. If I specify an unknown Content-Type in the client (application/gooblygop), Flask exposes the POST data to request.data and I can decode the base64 string. If I leave the Content-Type as default (application/x-www-form-urlencoded), the raw data is not exposed to request.data and I don't know how to retrieve the base64 encoded string and make use of it.
The existing client software all pretty much defaults to x-www-form-urlencoded, but I can't rely on that always being the case.
Essentially, I need a reliable, server-side method for accessing that encoded string no matter what Content-Type the client program states.
Other notes: I am very new to Python, coming from a PHP background. So I am very open to suggestions. Also, this project is primarily for personal use.

You want to look at the request.form object when dealing with urlencoded posts with normal mimetypes. In this case you have an unusual form, but here is a way to do it:
# mkreq.py
from urllib import urlencode
import urllib2
from base64 import b64encode
credentials = {
'username': 'test#example.com',
'password': 'password'
}
data = b64encode(urlencode(credentials))
request = urllib2.Request("http://localhost:5000/login")
request.add_data(data)
request.add_header('Content-Type', 'application/gooblygop')
# 'application/x-www-form-urlencoded' seems to be a default Content-Type
login1 = urllib2.urlopen(request).read()
print(login1)
request2 = urllib2.Request("http://localhost:5000/login")
request2.add_data(data)
login2 = urllib2.urlopen(request2).read()
print(login2)
You probably want to modify the login bit to check the mimetype, here is a version with minimal changes to your current setup:
#app.route('/login', methods=['POST'])
def login():
error = None
if request.method == 'POST':
# post data: cGFzc3dvcmQ9ZGVmYXVsdCZlbWFpbD10ZXN0JTQwZXhhbXBsZS5jb20=
data = b64decode(request.data)
# decoded data: password=default&email=test%40example.com
if not data:
data = b64decode(request.form.keys()[0])
special_mimetype = request.mimetype
return(special_mimetype + '\n' + data)
This is the output of the first code sample, with two requests:
bvm$ python mkreq.py
application/gooblygop
username=test%40example.com&password=password
application/x-www-form-urlencoded
username=test%40example.com&password=password

Have you thought about using json to pass your data in the POST? Flask has built in support for passing json data. In addition, if you set the Content-Type in the headers to application/json then flask will automatically dejson the POST data for you and put it in request.json
Here is the requesting application
import urllib2
import json
if __name__ == "__main__":
headers = {'Content-Type':'application/json'}
post_data = {"user":"test_user"}
print "Posting request"
req = urllib2.Request("http://localhost:5000/login", json.dumps(post_data), headers)
resp = urllib2.urlopen(req)
print "Response was %s" % resp.read()
This is the Flask view
from flask import request
#app.route('/login', methods=['POST'])
def login():
user = request.json['user']
return user
I suggest you test using curl as well if you are using the linux terminal. Here is an example
curl -X POST -H "Content-Type:application/json" -s -d '{"user":"This is the username"}' 'localhost:5000/login'
This is the username