What is the correct way to handle character encoding when processing a POST request containing "text/plain" data in Flask?
Test application:
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route("/put-text", method=['POST'])
def hello_world():
print("data", request.data)
return {}
Example request
import requests
url = 'http://127.0.0.1:5000/put-text'
data = 'The quick brown fox jumped over the lazy dog."
headers = {
"Content-Type:":"text/plain",
}
res = requests.post(url, data=data, headers=headers)
With this, the request.data is bytes, and I could decode it relatively easily, but how do I make this code robust to support unicode or different text encodings?
According to the Flask docs, request.get_data(as_text=True) probably does what you want:
#app.route("/put-text", method=['POST'])
def hello_world():
text = request.get_data(as_text=True)
# do something with the `text`...
return {}
Presumably, Flask will decode the text with the correct encoding, as specified by the client in the respective header. Provided the client specifies the correct encoding, of course...
Related
From Python I make request to Rails app, and get this response:
{u'answer': u'ok'}
and in Rails side code like that:
#res = {'answer'=> 'ok'}
render json: #res
May this problem on python side? Here code of python:
import requests
import json
URL1 = 'http://'
PARAMS = {'login': 'asd'}
r = requests.get(url = URL1, params = PARAMS)
data = r.json()
print(data)
How I can get clear json
You can try as data = r.as_json
You are specifically parsing the JSON contained in the response body into Python objects using r.json(). Use r.text to see the textual representation of the response body.
In a Rails controller you can do:
render :json => data.to_json and return
In the CLI you'd do:
puts data.to_json
Make sure to include json or a compatible gem in your CLI code.
the code does not work with multiple keywords at the same time
Ex. keyword1 keyword2 keyword3 referral to bad URL formatting:
/get_all_related/keyword1%20keyword2%20keyword3
instead of this
/get_all_related/keyword1+keyword2+keyword3
How to get URL formatting as desired?
here the code:
from flask import Flask, render_template, jsonify
from urllib.parse import quote_plus
import json
import sys
import urllib.request
app = Flask(__name__)
url = 'http://suggestqueries.google.com/complete/search?client=firefox&q={}'
#app.route('/')
def index():
return render_template('index.html')
def get_related(entity):
search_term = quote_plus('{} vs '.format(entity))
request = urllib.request.Request(url.format(search_term))
result = urllib.request.urlopen(request)
suggestions = json.loads(result.read())
return [x.replace(suggestions[0],'') for x in suggestions[1] if 'vs' not in x.replace(suggestions[0],'')]
Thank's for help
if search_term has any spaces in it, they will be converted to %20 when calling a function to encode for URI (url.format()). You will have to replace them with + before encoding into a URL safe format:
request = urllib.request.Request(url.format(search_term.replace(' ','+')))
This will replace all spaces with +, then encode the entire string as URI safe.
Im trying to create a script, either by Python or PHP, that can receive JSON from Dialogflow webhooks.
The data value will be from resolvedQuery.
I want to parse the data then send to the below URL:
https://joinjoaomgcd.appspot.com/_ah/api/messaging/v1/sendPush?text=***JSON GOES HERE ***&deviceId=18c972b753ad&apikey=5b48aed7
The data from resolvedQuery needs to be sent to the about URL where *JSON GOES HERE * is.
Here is the Python code I have been trying:
from flask import Flask
from flask import request
from flask import make_response
import json
import logging as l
import requests
app = Flask(__name__)
#app.route('/')
def hello():
"""Return a friendly HTTP greeting."""
l.info("reached the hello() module...")
return 'Hello Beautiful World!\n'
#app.route('/apiai', methods=['POST'])
def apiai_response():
requests_session = requests.session()
requests_session.headers.update({'Content-Type': 'application/json'})
requests_session.headers.update({'charset':'utf-8'})
post_data = '[{ "resolvedQuery": "" }]'
requests_response = requests_session.post(url="https://joinjoaomgcd.appspot.com/_ah/api/messaging/v1/sendPush?text=***JSON GOES HERE ***&deviceId=18c972b753ad&apikey=5b48aed7", data=post_data)
print requests_response.content
#app.errorhandler(404)
def page_not_found(e):
"""Return a custom 404 error."""
return 'Sorry, nothing at this URL.', 404
I would try something like this:
import json
import requests
json_string = json.dumps({"hello": "world"})
these_params = {"text": json_string,
"deviceId": "18c972b753ad&apikey=5b48aed7"}
this_url = "https://joinjoaomgcd.appspot.com/_ah/api/messaging/v1/sendPush"
r.get(url=this_url, params=these_params)
notice it is a get request and not a post request
It is a matter of opinion however using url encoded params on a GET request differ in how stuff gets read. As far as the data field is concerned on a post request, the handling is a bit different.
On a get request it will look something like part of the url as :
text=%7B%22hello%22%3A%22world%22%7D
Where as a post request the server will try to read in a application/json encoded body which expects the "data" to be valid json text.
also note:
How to urlencode a querystring in Python?
I have two servers where one is trying to get a file from the other. I am using Flask get requests to send simple data back and forth (strings, lists, JSON objects, etc.).
I also know how to send just a file, but I need to send an error code with my data.
I'm using something along the following lines:
Server 1:
req = requests.post('https://www.otherserver.com/_download_file', data = {'filename':filename})
Server 2:
#app.route('/_download_file', methods = ['POST'])
def download_file():
filename = requests.form.get('filename')
file_data = codecs.open(filename, 'rb').read()
return file_data
Server 1:
with codecs.open('new_file.xyz', 'w') as f:
f.write(req.content)
...all of which works fine. However, I want to send an error code variable along with file_data so that Server 1 knows the status (and not the HTTP status, but an internal status code).
Any help is appreciated.
One solution that comes to my mind is to use a custom HTTP header.
Here is an example server and client implementation.
Of course, you are free to change the name and the value of the custom header as you need.
server
from flask import Flask, send_from_directory
app = Flask(__name__)
#app.route('/', methods=['POST'])
def index():
response = send_from_directory(directory='your-directory', filename='your-file-name')
response.headers['my-custom-header'] = 'my-custom-status-0'
return response
if __name__ == '__main__':
app.run(debug=True)
client
import requests
r = requests.post(url)
status = r.headers['my-custom-header']
# do what you want with status
UPDATE
Here is another version of the server based on your implementation
import codecs
from flask import Flask, request, make_response
app = Flask(__name__)
#app.route('/', methods=['POST'])
def index():
filename = request.form.get('filename')
file_data = codecs.open(filename, 'rb').read()
response = make_response()
response.headers['my-custom-header'] = 'my-custom-status-0'
response.data = file_data
return response
if __name__ == '__main__':
app.run(debug=True)
I am recreating a service in Python/Flask and am running into an issue with the way the existing clients authenticate. I have to match the existing clients scheme for compatibility reasons.
The existing clients take the username, password and base64 encode it. This is not HTTP Basic Authentication, despite sounding similar. Below is some sample code that would create this login request.
credentials = {
'username': 'test#example.com',
'password': 'password'
}
data = b64encode(urlencode(credentials))
request = urllib2.Request(loginURL)
request.add_data(data)
# request.add_header('Content-Type', 'application/gooblygop')
# 'application/x-www-form-urlencoded' seems to be a default Content-Type
login = urllib2.urlopen(request)
On the server side, I take the POST data and base64 decode it to get the username and password information again.
flask server:
#app.route('/login', methods=['POST'])
def login():
error = None
if request.method == 'POST':
# post data: cGFzc3dvcmQ9ZGVmYXVsdCZlbWFpbD10ZXN0JTQwZXhhbXBsZS5jb20=
data = b64decode(request.data)
# decoded data: password=default&email=test%40example.com
return('ok')
The problem is the Content Type. If I specify an unknown Content-Type in the client (application/gooblygop), Flask exposes the POST data to request.data and I can decode the base64 string. If I leave the Content-Type as default (application/x-www-form-urlencoded), the raw data is not exposed to request.data and I don't know how to retrieve the base64 encoded string and make use of it.
The existing client software all pretty much defaults to x-www-form-urlencoded, but I can't rely on that always being the case.
Essentially, I need a reliable, server-side method for accessing that encoded string no matter what Content-Type the client program states.
Other notes: I am very new to Python, coming from a PHP background. So I am very open to suggestions. Also, this project is primarily for personal use.
You want to look at the request.form object when dealing with urlencoded posts with normal mimetypes. In this case you have an unusual form, but here is a way to do it:
# mkreq.py
from urllib import urlencode
import urllib2
from base64 import b64encode
credentials = {
'username': 'test#example.com',
'password': 'password'
}
data = b64encode(urlencode(credentials))
request = urllib2.Request("http://localhost:5000/login")
request.add_data(data)
request.add_header('Content-Type', 'application/gooblygop')
# 'application/x-www-form-urlencoded' seems to be a default Content-Type
login1 = urllib2.urlopen(request).read()
print(login1)
request2 = urllib2.Request("http://localhost:5000/login")
request2.add_data(data)
login2 = urllib2.urlopen(request2).read()
print(login2)
You probably want to modify the login bit to check the mimetype, here is a version with minimal changes to your current setup:
#app.route('/login', methods=['POST'])
def login():
error = None
if request.method == 'POST':
# post data: cGFzc3dvcmQ9ZGVmYXVsdCZlbWFpbD10ZXN0JTQwZXhhbXBsZS5jb20=
data = b64decode(request.data)
# decoded data: password=default&email=test%40example.com
if not data:
data = b64decode(request.form.keys()[0])
special_mimetype = request.mimetype
return(special_mimetype + '\n' + data)
This is the output of the first code sample, with two requests:
bvm$ python mkreq.py
application/gooblygop
username=test%40example.com&password=password
application/x-www-form-urlencoded
username=test%40example.com&password=password
Have you thought about using json to pass your data in the POST? Flask has built in support for passing json data. In addition, if you set the Content-Type in the headers to application/json then flask will automatically dejson the POST data for you and put it in request.json
Here is the requesting application
import urllib2
import json
if __name__ == "__main__":
headers = {'Content-Type':'application/json'}
post_data = {"user":"test_user"}
print "Posting request"
req = urllib2.Request("http://localhost:5000/login", json.dumps(post_data), headers)
resp = urllib2.urlopen(req)
print "Response was %s" % resp.read()
This is the Flask view
from flask import request
#app.route('/login', methods=['POST'])
def login():
user = request.json['user']
return user
I suggest you test using curl as well if you are using the linux terminal. Here is an example
curl -X POST -H "Content-Type:application/json" -s -d '{"user":"This is the username"}' 'localhost:5000/login'
This is the username