Develop Reference

Python POST Request x-www-form-urlencoded received error "Object of type Response is not JSON serializable"

i'm having an issue while write codes for consume another API using Python + Flask + Requests.
their API using PHP with x-www-form-urlencoded instead of RAW JSON, here is my code :
app = Flask(__name__)
#app.route('/api/test/getinfo', methods=['POST'])
def get_house_info():
if request.method == 'POST':
try:
payloads = {'no_house':'001234123', 'cd_agent' : '01', 'nm_agent' : 'ABC'}
response_data = requests.post(url=url, headers=headers, data=payloads)
return response_data
except Exception as e:
return(str(e))
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
after that i run this flask and tried to call this endpoint using postman
but i received error Object of type Response is not JSON serializable is there something wrong in my codes ?

Try json.dumps() for payloads
python dictionary and json is not the same, refer to its usage here:
Note: Keys in key/value pairs of JSON are always of the type str. When a dictionary is converted into JSON, all the keys of the dictionary are coerced to strings. As a result of this, if a dictionary is converted into JSON and then back into a dictionary, the dictionary may not equal the original one. That is, loads(dumps(x)) != x if x has non-string keys. source
From python to json : json.dumps(x)
From json to python : json.loads(x)

How to post a kafka schema using python

I am trying to post a kafka schema using python.
From the CLI I would use a syntax like:
curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema": "{\"type\":\"record\",\"name\":\"VisualDetections\",\"namespace\":\"com.namespace.something\",\"fields\":[{\"name\":\"vehicle_id\",\"type\":\"int\"},{\"name\":\"source\",\"type\":\"string\"},{\"name\":\"width\",\"type\":\"int\"},{\"name\":\"height\",\"type\":\"int\"},{\"name\":\"annotated_frame\",\"type\":[\"string\",\"null\"]},{\"name\":\"version\",\"type\":\"string\"},{\"name\":\"fps\",\"type\":\"int\"},{\"name\":\"mission_id\",\"type\":\"int\"},{\"name\":\"sequence\",\"type\":{\"type\":\"array\",\"items\":{\"type\":\"record\",\"name\":\"sequence_record\",\"fields\":[{\"name\":\"frame_id\",\"type\":\"int\"},{\"name\":\"timestamp\",\"type\":\"long\"},{\"name\":\"localization\",\"type\":{\"type\":\"array\",\"items\":{\"type\":\"record\",\"name\":\"localization_record\",\"fields\":[{\"name\":\"latitude\",\"type\":\"double\"},{\"name\":\"longitude\",\"type\":\"double\"},{\"name\":\"class\",\"type\":\"string\"},{\"name\":\"object_id\",\"type\":\"int\"},{\"name\":\"confidence\",\"type\":\"double\"},{\"name\":\"bbox\",\"type\":{\"type\":\"record\",\"name\":\"bbox\",\"fields\":[{\"name\":\"x_min\",\"type\":\"int\"},{\"name\":\"y_min\",\"type\":\"int\"},{\"name\":\"x_max\",\"type\":\"int\"},{\"name\":\"y_max\",\"type\":\"int\"}]}}]}}}]}}}]}"}' http://server_ip:8081/subjects/VisualDetections-value/versions/
When I tried to tranfer this function to python I tried something like:
import requests
import json
topic = 'VisualDetections'
headers = {'Content-Type': 'application/vnd.schemaregistry.v1+json'}
with open(avro_path) as fp:
data = {'schema': json.load(fp)}
data_json = json.dumps(data)
cmd = 'http://server_ip:8081/subjects/{}-value/versions/'.format(topic)
response = requests.post(cmd, headers=headers, data=data_json)
The above returns a code {"error_code":500,"message":"Internal Server Error"}. I have tried other options like:
with open(avro_path) as fp:
data = json.load(fp)
with error code:
"error_code":422,"message":"Unrecognized field: name"
In the above the avro_path just contains the avro schema in a json file (can be uploaded if useful also).
I am not sure how I could post this data exactly. Also, I did not take into consideration the -H argument of post in CLI since I couldn't find a equivalent python argument (not sure it plays any role though). Can anyone provide a solution to this issue.

For the second error, the payload needs to be {'schema': "schema string"}
For the first, I think its a matter of the encoding; json.load will read the file to a dict rather than just a string.
Notice
>>> import json
>>> schema = {"type":"record"} # example when using json.load() ... other data excluded
>>> json.dumps({'schema': schema})
'{"schema": {"type": "record"}}' # the schema value is not a string
>>> json.dumps({'schema': json.dumps(schema)})
'{"schema": "{\\"type\\": \\"record\\"}"}' # here it is
Try just reading the file
url = 'http://server_ip:8081/subjects/{}-value/versions/'.format(topic)
with open(avro_path) as fp:
data = {'schema': fp.read().strip()}
response = requests.post(cmd, headers=headers, data=json.dumps(data))
Otherwise, you would json.load then use json.dumps twice as shown above
You may also try json=data rather than data=json.dumps(data)

POST/PUT input to Rest API

I am launching REST API build in python.
I am inputting a list as input and getting the required data.
example:
your.api.com/birth?name=James&date=2015-02-01&name=Robert&date=2020-01-01
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
As the number of inputs I have is huge, the end point URL is becoming huge. Is there any way to do the same using PUT or POST where I dump a doc in some format (say json) as my input?

Sure, put the request fields in a JSON and go through the dictionary
#app.route('/birth', methods=['POST'])
def birth():
data = request.get_json()
names = data["names"]
# etc.

When the input grows it is suggested to use a POST call rather than GET call.
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
Convert /birth to the following
from flask import request
#app.route('/birth', methods=['POST'])
def birth():
input = request.get_json()
# <Do the processing>
And in the POST call from client-side use a JSON like the following
{
name: [<array of values>],
date: [<array of values>]
}

Slack Interactive Messages: POST request payload has an unexpected format

I'm getting a POST request inside a Flask app from Slack. The request is sent when a user presses on an interactive message button. According to Slack docs I must extract the body of the request to verify the signature.
My computed signature doesn't match the one sent by Slack, though.
In fact, the body of the request comes as some encoded string. The string is actually an encoded dictionary instead of a query str parameters, as expected.
Here's the beginning of my view:
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8')
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated, digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...
I've tried to format the received data to make it look like:
token=fdjkgjl&user_id=1234... but I am not aware of all of the necessary parameters that have to be present in the data.
Any ideas are highly appreciated.
The body of the message is following - after being URL decoded (note I've modified possibly sensitive data):
b'payload={"type":"interactive_message","actions":
[{"name":"yes_button","type":"button","value":"236"}],"callback_id":"visit_button","team":{"id":"fffff","domain":"ffff"},"channel":{"id":"ffff","name":"directmessage"},"user":{"id":"ffffff","name":"fffft"},"action_ts":"1540403943.419120","message_ts":"1541403949.000100","attachment_id":"1","token":"8LpjBuv13J7xAjhl2lEajoBU","is_app_unfurl":false,"original_message":{"text":"Test","bot_id":"DDDDDDDDD","attachments":[{"callback_id":"visit_button","text":"Register","id":1,"color":"3AA3E3","actions":[{"id":"1","name":"yes_button","text":"Yes","type":"button","value":"236","style":""}],"fallback":"Register"}],"type":"message","subtype":"bot_message","ts":"1540413949.000100"},"response_url":"https://hooks.slack.com/actions/ffffff/ffffff/tXJjx1XInaUhrikj6oEzK08e","trigger_id":"464662548327.425084163429.dda35a299eedb940ab98dbb9386b56f0"}'

The reason you are getting the "garbled" data is that you are using request.get_data(). That method will return the raw data of a request, but not do any decoding for you.
Much more convenient is to use request.form.get('payload'), which will directly give you the JSON string of the request object. You can then convert that into a dict object with json.loads() to process it further in your app.
Note that the format you received is the correct format for interactive messages. You will not get a query string (e.g. "token=abc;user_id?def...") as you suggested (like for slash command requests). Interactive message request will always contain the request as JSON string in a payload form property. See here for reference.
Here is a simple working example, which will reply a greeting to the user that pressed the button. It will work directly with Slack, but I recommend using Postman to test it.
#app.py
from flask import Flask, request #import main Flask class and request object
import json
app = Flask(__name__) #create the Flask app
#app.route('/register', methods=['POST'])
def register_visit():
slack_req = json.loads(request.form.get('payload'))
response = '{"text": "Hi, <#' + slack_req["user"]["id"] + '>"}'
return response, 200, {'content-type': 'application/json'}
if __name__ == '__main__':
app.run(debug=True, port=5000) #run app in debug mode on port 5000

OK, the issue wasn't related to how Slack sends me the message. It was about misunderstanding which data comes as bytes and which data is unicode. The culprit was string formatting in my case - the line concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8') should have been concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data)). Data is already bytes, timestamp meanwhile is unicode.
Cannot believe I've banged my head on this for hours -_-
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data))
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated,
digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...

This worked for me
from urllib import parse
parsed_text = parse.unquote('your bytes text here')

Kivy UrlRequest

My API works fine and I see a 200 status when I test it using Postman. However I'm trying to access it using a Kivy application but I'm seeing a 400 response from the server after some waiting or quitting the app. By the way when testing with Postman I specify header as Content-Type: application/json and in body I see my parameters
{
"search_text": "Hello",
"num_results": 1
}
being sent as raw data.
My code
def search(self, search_text):
header = {'Content-Type':'application/json'}
req = UrlRequest('http://127.0.0.1:5000/search',req_body={"search_text": search_text,"num_results": 1},on_success=Test.got_json,req_headers=header)
print("Search method called")
#staticmethod
def got_json(req,result):
print(result)
Kivy docs say that you don't have to specify a method as this would send a POST request so I've not specified that here
Edit: The code for the server is kind of irrelevant for my issue here so I've removed it

UrlRequest should be passed a str object as request body. You can serialize the request dictionary as a string object by dumping it. Pass this dumped dictionary as the request body to UrlRequest.
import json
req_body=json.dumps({'search_text': search_text, 'num_results': 1})
req = UrlRequest(
'http://127.0.0.1:5000/search',
req_body=req_body,
on_success=Test.got_json,
req_headers=header)

req_body is a string parameter, might be a bit confusing as req_headers is a dict. You can use:
req_body=json.dumps({"search_text": search_text,"num_results": 1})