Develop Reference

How to take in JSON as input to python tornado "post" method

I am trying to use tornado to do a simple get and post method. Quite new to tornado framework. For the post I would like to take in a json as input, use that input to feed into another function that I have to execute another part of code. However I can't get tornado post method to work even with a simple self.write().
For my get method I am reading from an SQL database to get the status of a sensor and write that in a json format. The get method works perfectly! When I go to localhost:port# it reads out my get json file. For my post method I would like to take in a simple json of just one key:value which is a float number. I want to take that float number that the user specified in the json and use it in my flowMKS.set() function that will change the setpoint parameter of the sensor. I am not sure how to input a json into the post method and read it into a variable. I have some #commented code below that I tried and didn't work. However I went back to the basics and just did a self.write("Hello World") to see if the post was working. I can't get self.write to work either. Keep getting a 500 error message when i go to localhost:port#/flow_post. The variable flow_status was used in my get method.
The intended result would be to take in a json {"setpoint":45.5} into the post method. Use the number and insert into my flowMKS method to change a parameter on the sensor.
How would you take in a json to a post method and take the number from the json input and store in a variable?
class Current(tornado.web.RequestHandler):
def get(self):
global flow_status
time = flow_status[0]
ip = flow_status[1]
rate = flow_status[2]
setp = flow_status[3]
tempc = flow_status[4]
status = {"flow_controller":{
"time":time,
"ip":ip,
"rate_sccm":rate,
"setpoint":setp,
"temperature_c":tempc,
}
}
self.write(status)
class Update(tornado.web.RequestHandler):
# def prepare(self):
# if self.request.haders["Content-Type"].startswith("application/json"):
# self.json_args = json.loads(self.request.body)
# else:
# self.json_args = None
def post(self):
# #expecting body data to contain JSON so we use json.loads to decrypt the JSON into a dict
# data = json.loads(self.request.body)
#
# #Getting what the setpoint should be
# setpoint = self.json_args["setpoint"]
#
# #making the input a float
# setpoint = float(setpoint)
#
# #setting up connection with sensor
# flowMKS = FlowController(flow_status[1])
#
# #sending setpoint to sensor
# flowMKS.set(setpoint)
self.write("Hello World")
if __name__ == '__main__':
# global flow_status
#Below is creating the Tornado based API for get and post methods
tornado.options.parse_command_line()
app = tornado.web.Application(
handlers=[(r'/',Current), (r'/flow_post', Update)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
#using PeriodicCallback to get info from the SQL database every 500 ms
PeriodicCallback(get_sql_status,500).start()
#starting the entire Tornado IOLoop
tornado.ioloop.IOLoop.current().start()

For uploading a file using Tornado you can use this function tornado.httputil.parse_body_arguments which will split the uploaded file content in a dictionary file_dict and other arguments in the FormData in the args_dict.
Sample code:
import tornado.httputil
import tornado.web
import tornado.escape
import json
import os
import sys
import traceback
class FileHandler(tornado.web.RequestHandler):
def _return_response(self, request, message_to_be_returned: dict, status_code):
"""
Returns formatted response back to client
"""
try:
request.set_header("Content-Type", "application/json; charset=UTF-8")
request.set_status(status_code)
#If dictionary is not empty then write the dictionary directly into
if(bool(message_to_be_returned)):
request.write(message_to_be_returned)
request.finish()
except Exception:
raise
def set_default_headers(self, *args, **kwargs):
self.set_header('Content-Type','text/csv')
self.set_header("Access-Control-Allow-Origin", "*")
self.set_header("Access-Control-Allow-Headers", "x-requested-with")
self.set_header("Access-Control-Allow-Methods", "*")
def post(self):
"""
This function reads an uploaded file
"""
try:
arg_dict = {}
file_dict = {}
tornado.httputil.parse_body_arguments(self.request.headers["Content-Type"], self.request.body, arg_dict, file_dict)
uploaded_file = file_dict['TestFile'][0]
if not uploaded_file:
return self._return_response(self, { 'message': 'No test file uploaded, please upload a test file' }, 400)
# File contents here
file_contents = str(uploaded_file['body'], "utf-8")
self.set_status(200)
self.finish()
except Exception as ex:
return self._return_response(self, { "message": 'Could not complete the request because of some error at the server!', "cause": ex.args[0], "stack_trace": traceback.format_exc(sys.exc_info()) }, 500)
You can alternatively use tornado.escape.json_decode to deserialize the request body into a dictionary and do something with it.
Sample code:
import tornado.gen
import tornado.web
import tornado.escape
import json
import os
import sys
import traceback
class JSONHandler(tornado.web.RequestHandler):
def _return_response(self, request, message_to_be_returned: dict, status_code):
"""
Returns formatted response back to client
"""
try:
request.set_header("Content-Type", "application/json; charset=UTF-8")
request.set_status(status_code)
#If dictionary is not empty then write the dictionary directly into
if(bool(message_to_be_returned)):
request.write(message_to_be_returned)
request.finish()
except Exception:
raise
def set_default_headers(self, *args, **kwargs):
self.set_header("Content-Type", "application/json")
self.set_header("Access-Control-Allow-Origin", "*")
self.set_header("Access-Control-Allow-Headers", "x-requested-with")
self.set_header("Access-Control-Allow-Methods", "*")
def post(self):
"""
This function parses the request body and does something
"""
try:
# Do something with request body
request_payload = tornado.escape.json_decode(self.request.body)
return self._return_response(self, request_payload, 200)
except json.decoder.JSONDecodeError:
return self._return_response(self, { "message": 'Cannot decode request body!' }, 400)
except Exception as ex:
return self._return_response(self, { "message": 'Could not complete the request because of some error at the server!', "cause": ex.args[0], "stack_trace": traceback.format_exc(sys.exc_info()) }, 500)

Django http request to api error

As this is the first time I'm trying this out, I do not know what is wrong with the problem. So it would be great if someone can help me solve this problem
The code I'm using is at the bottom page of this website: https://www.twilio.com/blog/2014/11/build-your-own-pokedex-with-django-mms-and-pokeapi.html
Where it give example on how you can make HTTP request function and retrieve database on your query.
The code on the website is this.
query.py
import requests
import json
BASE_URL = 'http://pokeapi.co'
def query_pokeapi(resource_url):
url = '{0}{1}'.format(BASE_URL, resource_url)
response = requests.get(url)
if response.status_code == 200:
return json.loads(response.text)
return None
charizard = query_pokeapi('/api/v1/pokemon/charizard/')
sprite_uri = charizard['sprites'][0]['resource_uri']
description_uri = charizard['descriptions'][0]['resource_uri']
sprite = query_pokeapi(sprite_uri)
description = query_pokeapi(description_uri)
print
charizard['name']
print
description['description']
print
BASE_URL + sprite['image']
In my edit, I only change these print line at the bottom of this
query.py
print(charizard['name'])
print(description['description'])
print(BASE_URL + sprite['image'])
But i got this error instead
Traceback (most recent call last): File "query2.py", line 46, in
sprite_uri = charizard['sprites'][0]['resource_uri'] TypeError: 'NoneType' object is not subscriptable

query_pokeapi must be returning None, which would mean that your API call is not receiving a 200 HTTP response. I'd check your URL, to make sure it's properly formed. Test it in your web browser.
best practice would be to try-except your API call with an error message letting you know that your API call failed and otherwise routing the thread.
Update: reread and the sub scripting issue could be in any layer of your nested object.
Evaluate charizard['sprites'][0]['resource_uri']
step by step in your debugger.

When you call api requests.get(url) then its response is
More than one resource is found at this URI
you are using charizard['sprites'][0]['resource_uri'] on result and it's raising exception.
When I tried to get response then status code is 300 so
def query_pokeapi(resource_url) returning None value.
'{0}{1}'.format(BASE_URL, resource_url)
Update
it means at {0} BASE_URL will be places and at {1} resource_url will be places.
Complete url will be
url = '{0}{1}'.format(BASE_URL, resource_url)
url = 'http://pokeapi.co/api/v1/pokemon/charizard/'.
update
you can try
import json
charizard = query_pokeapi('/api/v1/pokemon/')
data = json.loads(charizard.content)
print data['objects'][0]['descriptions'][0]
result will be
{u'name': u'ekans_gen_1', u'resource_uri': u'/api/v1/description/353/'}
Update with complete code
import requests
import json
BASE_URL = 'http://pokeapi.co'
def query_pokeapi(resource_url):
url = '{0}{1}'.format(BASE_URL, resource_url)
response = requests.get(url)
if response.status_code == 200:
return json.loads(response.text)
return None
charizard = query_pokeapi('/api/v1/pokemon/')
print charizard['objects'][0]['descriptions'][0]
result will be:
{u'name': u'ekans_gen_1', u'resource_uri': u'/api/v1/description/353/'}

OpenShift REST API for scaling, invalid character 's' looking for beginning of value

I'm trying to scale out my deployments using openshift rest api, but I'm encountering the error "invalid character 's' looking for beginning of value".
I can successfully get the deployment config details but it's the patch request which is troubling me.
From the documents I have tried Content-Type as below 3 but nothing works:
application/json-patch+json
application/merge-patch+json
application/strategic-merge-patch+json
Here's my code:
data = {'spec':{'replicas':2}}
headers = {"Authorization": token, "Content-Type": "application/json-patch+json"}
def updateReplicas():
url = root + "namespaces" + namespace + "deploymentconfigs" + dc + "scale"
resp = requests.patch(url, headers=headers, data=data, verify=False)
print(resp.content)
Thank you.

Ok, I found out the issue. Silly thing first, data should be inside single quotes data = '{'spec':{'replicas':2}}'.
Then, we need few more info in our data, which finally looks like :
data = '{"kind":"Scale","apiVersion":"extensions/v1beta1","metadata":{"name":"deployment_name","namespace":"namespace_name"},"spec":{"replicas":1}}'
Thank you for your time.

I had the same use case and the hint of #GrahamDumpleton to run oc with --loglevel 9 was very helpful.
This is what oc scale does:
It makes a get request to the resource, receiving some JSON object
Then it makes a put request to the resource, with a modified JSON
object (the number of replicas changed) as payload
If you're doing this you don't have to worry about setting the apiVersion, you just reuse what you get in the first place.
Here is a small python script, that follows this approach:
"""
Login into your project first `oc login` and `oc project <your-project>` before running this script.
Usage:
pip install requests
python scale_pods.py --deployment-name <your-deployment> --nof-replicas <number>
"""
import argparse
import requests
from subprocess import check_output
import warnings
warnings.filterwarnings("ignore") # ignore insecure request warnings
def byte_to_str(bs):
return bs.decode("utf-8").strip()
def get_endpoint():
byte_str = check_output("echo $(oc config current-context | cut -d/ -f2 | tr - .)", shell=True)
return byte_to_str(byte_str)
def get_namespace():
byte_str = check_output("echo $(oc config current-context | cut -d/ -f1)", shell=True)
return byte_to_str(byte_str)
def get_token():
byte_str = check_output("echo $(oc whoami -t)", shell=True)
return byte_to_str(byte_str)
def scale_pods(deployment_name, nof_replicas):
url = "https://{endpoint}/apis/apps.openshift.io/v1/namespaces/{namespace}/deploymentconfigs/{deplyoment_name}/scale".format(
endpoint=get_endpoint(),
namespace=get_namespace(),
deplyoment_name=deployment_name
)
headers = {
"Authorization": "Bearer %s" % get_token()
}
get_response = requests.get(url, headers=headers, verify=False)
data = get_response.json()
data["spec"]["replicas"] = nof_replicas
print(data)
response_put = requests.put(url, headers=headers, json=data, verify=False)
print(response_put.status_code)
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--deployment-name", type=str, required=True, help="deployment name")
parser.add_argument("--nof-replicas", type=int, required=True, help="nof replicas")
args = parser.parse_args()
scale_pods(args.deployment_name, args.nof_replicas)
if __name__ == "__main__":
main()

Catch http-status code in Flask

I lately started using Flask in one of my projects to provide data via a simple route. So far I return a json file containing the data and some other information. When running my Flask app I see the status code of this request in terminal. I would like to return the status code as a part of my final json file. Is it possible to catch the same code I see in terminal?
Some simple might look like this
from flask import Flask
from flask import jsonify
app = Flask(__name__)
#app.route('/test/<int1>/<int2>/')
def test(int1,int2):
int_sum = int1 + int2
return jsonify({"result":int_sum})
if __name__ == '__main__':
app.run(port=8082)
And in terminal I get:

You are who set the response code (by default 200 on success response), you can't catch this value before the response is emited. But if you know the result of your operation you can put it on the final json.
#app.route('/test/<int1>/<int2>/')
def test(int1, int2):
int_sum = int1 + int2
response_data = {
"result": int_sum,
"sucess": True,
"status_code": 200
}
# make sure the status_code on your json and on the return match.
return jsonify(response_data), 200 # <- the status_code displayed code on console
By the way if you access this endpoint from a request library, on the response object you can find the status_code and all the http refered data plus the json you need.
Python requests library example
import requests
req = requests.get('your.domain/test/3/3')
print req.url # your.domain/test/3/3
print req.status_code # 200
print req.json() # {u'result': 6, u'status_code: 200, u'success': True}

You can send HTTP status code as follow:
#app.route('/test')
def test():
status_code = 200
return jsonify({'name': 'Nabin Khadka'}, status_code) # Notice second element of the return tuple(return)
This way you can control what status code to return to the client (typically to web browser.)

requests - Gateway Timeout

this is a test script to request data from Rovi API, provided by the API itself.
test.py
import requests
import time
import hashlib
import urllib
class AllMusicGuide(object):
api_url = 'http://api.rovicorp.com/data/v1.1/descriptor/musicmoods'
key = 'my key'
secret = 'secret'
def _sig(self):
timestamp = int(time.time())
m = hashlib.md5()
m.update(self.key)
m.update(self.secret)
m.update(str(timestamp))
return m.hexdigest()
def get(self, resource, params=None):
"""Take a dict of params, and return what we get from the api"""
if not params:
params = {}
params = urllib.urlencode(params)
sig = self._sig()
url = "%s/%s?apikey=%s&sig=%s&%s" % (self.api_url, resource, self.key, sig, params)
resp = requests.get(url)
if resp.status_code != 200:
# THROW APPROPRIATE ERROR
print ('unknown err')
return resp.content
from another script I import the module:
from roviclient.test import AllMusicGuide
and create an instance of the class inside a mood function:
def mood():
test = AllMusicGuide()
print (test.get('[moodids=moodids]'))
according to documentation, the following is the syntax for requests:
descriptor/musicmoods?apikey=apikey&sig=sig [&moodids=moodids] [&format=format] [&country=country] [&language=language]
but running the script I get the following error:
unknown err
<h1>Gateway Timeout</h1>:
what is wrong?

"504, try once more. 502, it went through."
Your code is fine, this is a network issue. "Gateway Timeout" is a 504. The intermediate host handling your request was unable to complete it. It made its own request to another server on your behalf in order to handle yours, but this request took too long and timed out. Usually this is because of network congestion in the backend; if you try a few more times, does it sometimes work?
In any case, I would talk to your network administrator. There could be any number of reasons for this and they should be able to help fix it for you.