Attempting to send an image to the client browser - the following code works perfectly - please assume that I am receiving the correct base64.b64encoded byte stream:
def get_user_image(usr_rec):
with open("config.json",) as config:
data = json.load(config)
avatar = data["PATHS"]["AVATAR"]
file_ext = os.path.splitext(usrdata["avatar"])[1] if usrdata["avatar"] else ".png"
file_fmt = ".jpeg" if file_ext.upper()==".JPG" else file_ext
path = "{}/{}{}".format(avatar,usr_rec["recordid"],file_ext)
img = Image.open(path,mode="r")
barr = io.BytesIO()
img.save(barr,format="{}".format(file_fmt[1:].upper()))
return b64encode(barr.getvalue())
def format_record (usr_rec):
avatar = get_user_image(usr_rec["recordid"])
return jsonify({"avatar": str(avatar)})
On my development box. Move it to a flask production running under gunicorn and I get JSON serialization errors: TypeError: Object of type bytes is not JSON serializable I am also getting this error for Decimal types as well.
How can I get around this issue?
This is because your get_user_image function returns a stream of bytes and not a string, so you have to cast the bytes read into a string: get_user_image(usr_rec["recordid"]).decode("utf-8"). The same happens for the object of type Decimal.
The jsonify function only serializes objects of type string, as you can also see here and here
Related
I am attempting to send an image to a Flask API. So far, I am using base64 to encode the image to be sent as a string. On the server side, I am receiving that string and decoding it, and attempting to write over a file on the server side. The code runs, but the resulting JPG is not viewable, and it shows "It looks like we don't support this format." I have also attempted saving the file as other photo file formats. It is a jpg that I convert to string so that is why I am saving as jpg.
Here is my client side code:
with open(filename, "rb") as img:
string = base64.b64encode(img.read())
print(type(string))
print(string)
name = 'John Doe'
EmpID = 1
company = 1
def test():
api_url = "http://192.168.8.13:5000/register-new?emp_name=%s&company_id=%s&emp_id=%s&user_photo=%s" % (name, company, EmpID, string)
response = requests.post(url= api_url)
assert response.status_code == 200
And here is the server side code for receiving the photo.
photo = request.args.get('user_photo')
photo1 = photo.replace(" ", "")
f = (base64.b64decode(photo1))
a = io.BytesIO()
with open("compare.jpg", "wb") as file:
file.write(f)
If you really want to upload this as base64 data, I suggest putting this as JSON in the post body, rather than as a GET parameter.
On the client side, open the file like this:
with open(filename, "rb") as img:
string = base64.b64encode(img.read()).decode('utf-8')
Then in your test function, take that image_data string out of the URL, and use the request.post argument json to pass this across with the correct Content Type. You could consider sending the other variables by adding them to the dictionary passed with this arg:
def test():
api_url = "http://192.168.8.13:5000/register-new?emp_name=%s&company_id=%s&emp_id=%s" % (name, company, EmpID)
response = requests.post(url= api_url, json={'user_photo':string})
Then on the backend, grab this with Flask's request.get_json function and initialise the BytesIO object, before writing the data to that, and finally writing to a file:
#app.route('/register-new', methods=['POST'])
def register_new():
photo = request.get_json()['user_photo']
photo_data = base64.b64decode(photo)
with open("compare.jpg", "wb") as file:
file.write(photo_data)
With a test image this works correctly, as confirmed by the Linux file command:
$ file compare.jpg
compare.jpg: JPEG image data, baseline, precision 8, 500x750, components 3
Is there a way to download a file through FastAPI? The files we want are located in an Azure Datalake and retrieving them from the lake is not an issue, the problem occurs when we try to get the bytes we get from the datalake down to a local machine.
We have tried using different modules in FastAPI such as starlette.responses.FileResponse and fastapi.Response with no luck.
In Flask this is not an issue and can be done in the following manner:
from io import BytesIO
from flask import Flask
from werkzeug import FileWrapper
flask_app = Flask(__name__)
#flask_app.route('/downloadfile/<file_name>', methods=['GET'])
def get_the_file(file_name: str):
the_file = FileWrapper(BytesIO(download_file_from_directory(file_name)))
if the_file:
return Response(the_file, mimetype=file_name, direct_passthrough=True)
When running this with a valid file name the file automatically downloads. Is there equivalent way to this in FastAPI?
Solved
After some more troubleshooting I found a way to do this.
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
if the_file:
return Response(the_file)
So after a lot of troubleshooting and hours of looking through documentation, this was all it took, simply returning the bytes as Response(the_file).
After some more troubleshooting I found a way to do this.
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
if the_file:
return Response(the_file)
So after a lot of troubleshooting and hours of looking through documentation, this was all it took, simply returning the bytes as Response(the_file) with no extra parameters and no extra formatting for the raw bytes object.
As far as I know, you need to set media_type to the adequate type. I did that with some code a year ago and it worked fine.
#app.get("/img/{name}")
def read(name: str, access_token_cookie: str=Cookie(None)):
r = internal.get_data(name)
if r is None:
return RedirectResponse(url="/static/default.png")
else:
return Response(content=r["data"], media_type=r["mime"])
r is a dictionary with the data as raw bytes and mime the type of the data as given by PythonMagick.
To add a custom filename to #Markus's answer, in case your api's path doesn't end with a neat filename or you want to determine a custom filename from server side and give to the user:
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
filename1 = make_filename(file_name) # a custom filename
headers1 = {'Content-Disposition': f'attachment; filename="{filename1}"'}
if the_file:
return Response(the_file, headers=headers1)
I am working with webhooks from Bold Commerce, which are validated using a hash of the timestamp and the body of the webhook, with a secret key as the signing key. The headers from the webhook looks like this :
X-Bold-Signature: 06cc9aab9fd856bdc326f21d54a23e62441adb5966182e784db47ab4f2568231
timestamp: 1556410547
Content-Type: application/json
charset: utf-8
According to their documentation, the hash is built like so (in PHP):
$now = time(); // current unix timestamp
$json = json_encode($payload, JSON_FORCE_OBJECT);
$signature = hash_hmac('sha256', $now.'.'.$json, $signingKey);
I am trying to recreate the same hash using python, and I am always getting the wrong value for the hash. I've tried several combinations, with and without base64 encoding. In python3, a bytes object is expected for the hmac, so I need to encode everything before I can compare it. At this point my code looks like so :
json_loaded = json.loads(request.body)
json_dumped = json.dumps(json_loaded)
# if I dont load and then dump the json, the message has escaped \n characters in it
message = timestamp + '.' + json_dumped
# => 1556410547.{"event_type" : "order.created", "date": "2020-06-08".....}
hash = hmac.new(secret.encode(), message.encode(), hashlib.sha256)
hex_digest = hash.hexdigest()
# => 3e4520c869552a282ed29b6523eecbd591fc19d1a3f9e4933e03ae8ce3f77bd4
# hmac_to_verify = 06cc9aab9fd856bdc326f21d54a23e62441adb5966182e784db47ab4f2568231
return hmac.compare_digest(hex_digest, hmac_to_verify)
Im not sure what I am doing wrong. For the other webhooks I am validating, I used base64 encoding, but it seems like here, that hasnt been used on the PHP side. I am not so familiar with PHP so maybe there is something I've missed in how they built the orginal hash. There could be complications coming from the fact that I have to convert back and forth between byte arrays and strings, maybe I am using the wrong encoding for that ? Please someone help, I feel like I've tried every combination and am at a loss.
EDIT : Tried this solution by leaving the body without encoding it in json and it still fails :
print(type(timestamp)
# => <class 'str'>
print(type(body))
# => <class 'bytes'>
# cant concatenate bytes to string...
message = timestamp.encode('utf-8') + b'.' + body
# => b'1556410547.{\n "event_type": "order.created",\n "event_time": "2020-06-08 11:16:04",\n ...
hash = hmac.new(secret.encode(), message, hashlib.sha256)
hex_digest = hash.hexdigest()
# ...etc
EDIT EDIT :
Actually it is working in production ! Thanks to the solution described above (concatenating everything as bytes). My Postman request with the faked webhook was still failing, but that's probably because of how I copied and pasted the webhook data from my heroku logs :) .
Hi I am new to writing web APIs in python. And my understanding of REST is limited
I have a simple Flask API that takes in a python dict {'pdf':pdf_as_bytes, 'filename':string}
The below is my server script:
#app.route("/predict", methods=["POST"])
def predict():
data = {"success": False}
if flask.request.method == "POST":
pdf = flask.request.files["pdf"].read()
filename = flask.request.files["filename"].read().decode("utf-8")
assert isinstance(filename, str), "Got {}".format(type(filename))
assert isinstance(pdf, bytes), "Got {}".format(type(pdf))
# further processing happens and returns a json
This works as intended when I write a python client as follows:
import requests
import os
ip = "localhost"
port = 8605
url = "http://{}:{}/predict".format(ip,port)
path_to_pdf = "./617339931.pdf"
with open(path_to_pdf, "rb") as f:
pdf = f.read() # pdf is a bytes
# the payload must have the following fields: "pdf": bytes, "filename": string object
payload = {"pdf":pdf,"filename":os.path.basename(path_to_pdf).split(".")[0]}
# post request
result = requests.post(url=url, files=payload).json()
# the resulting json always has a field called as success, which returns True or False accordingly
if result["success"] == True:
print(result["data"].keys())
But, When I send a request using Postman I get a 400 Error! Below is the screen shot of the error
I don't understand. How can I change my server code so that it works with Postman and also Python client programs
I just did the same thing, and I think it's because of the double quotes you are putting in key and value, try to take them out.
I'm getting a POST request inside a Flask app from Slack. The request is sent when a user presses on an interactive message button. According to Slack docs I must extract the body of the request to verify the signature.
My computed signature doesn't match the one sent by Slack, though.
In fact, the body of the request comes as some encoded string. The string is actually an encoded dictionary instead of a query str parameters, as expected.
Here's the beginning of my view:
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8')
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated, digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...
I've tried to format the received data to make it look like:
token=fdjkgjl&user_id=1234... but I am not aware of all of the necessary parameters that have to be present in the data.
Any ideas are highly appreciated.
The body of the message is following - after being URL decoded (note I've modified possibly sensitive data):
b'payload={"type":"interactive_message","actions":
[{"name":"yes_button","type":"button","value":"236"}],"callback_id":"visit_button","team":{"id":"fffff","domain":"ffff"},"channel":{"id":"ffff","name":"directmessage"},"user":{"id":"ffffff","name":"fffft"},"action_ts":"1540403943.419120","message_ts":"1541403949.000100","attachment_id":"1","token":"8LpjBuv13J7xAjhl2lEajoBU","is_app_unfurl":false,"original_message":{"text":"Test","bot_id":"DDDDDDDDD","attachments":[{"callback_id":"visit_button","text":"Register","id":1,"color":"3AA3E3","actions":[{"id":"1","name":"yes_button","text":"Yes","type":"button","value":"236","style":""}],"fallback":"Register"}],"type":"message","subtype":"bot_message","ts":"1540413949.000100"},"response_url":"https://hooks.slack.com/actions/ffffff/ffffff/tXJjx1XInaUhrikj6oEzK08e","trigger_id":"464662548327.425084163429.dda35a299eedb940ab98dbb9386b56f0"}'
The reason you are getting the "garbled" data is that you are using request.get_data(). That method will return the raw data of a request, but not do any decoding for you.
Much more convenient is to use request.form.get('payload'), which will directly give you the JSON string of the request object. You can then convert that into a dict object with json.loads() to process it further in your app.
Note that the format you received is the correct format for interactive messages. You will not get a query string (e.g. "token=abc;user_id?def...") as you suggested (like for slash command requests). Interactive message request will always contain the request as JSON string in a payload form property. See here for reference.
Here is a simple working example, which will reply a greeting to the user that pressed the button. It will work directly with Slack, but I recommend using Postman to test it.
#app.py
from flask import Flask, request #import main Flask class and request object
import json
app = Flask(__name__) #create the Flask app
#app.route('/register', methods=['POST'])
def register_visit():
slack_req = json.loads(request.form.get('payload'))
response = '{"text": "Hi, <#' + slack_req["user"]["id"] + '>"}'
return response, 200, {'content-type': 'application/json'}
if __name__ == '__main__':
app.run(debug=True, port=5000) #run app in debug mode on port 5000
OK, the issue wasn't related to how Slack sends me the message. It was about misunderstanding which data comes as bytes and which data is unicode. The culprit was string formatting in my case - the line concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8') should have been concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data)). Data is already bytes, timestamp meanwhile is unicode.
Cannot believe I've banged my head on this for hours -_-
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data))
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated,
digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...
This worked for me
from urllib import parse
parsed_text = parse.unquote('your bytes text here')