I am misunderstanding something very basic probably. I am new to tornado and web servers in general. I used some tutorials and a lot of googling to get started but I still find myself stuck at the basics.
The Situation
I am using python 3.6.9 on an Ubuntu 18.04 server with tornado 6.0.4.
I have a tornado server that accepts GET requests via a tornado.web.RequestHandler class get () function and does some computation on it. This all works properly.
I need the tornado server to return the results (a numpy array) to the client that sent the request.
To my knowledge everything I am doing is synchronous as I did not add any async code myself.
My code in a nutshell:
class MainHandler(tornado.web.RequestHandler):
def get(self):
base_data = self.get_argument("base_data")
compute_data(base_data)
#Here I want to return the data back to the client
application = tornado.web.Application(handlers=[ (r"/calculator", MainHandler)])
if __name__ == "__main__":
http_server=tornado.httpserver.HTTPServer(application)
http_server.listen(__PORT__)
tornado.ioloop.IOLoop.instance().start()
The problem
I do not have info about the client.
I do not have any idea and cannot find any tutorial explaining how to respond back to a client from a GET request.
What I tried
I tried simply returning the np.array at the end of my get() function but I got:
TypeError: object numpy.ndarray can't be used in 'await' expression
I thought what I need to do is make a POST request back to the client, but I do not (that I know of) have the IP and port of the client.
I also found randomly maybe I should use tornado.ioloop.IOLoop.current().spawn_callback(data) but that wasn't right I guess because it asked me for a callable function.
What I want to happen
I want to send back the computed data to the client that requested it.
Thanks in advance for any help available. I know I am probably misunderstanding the very basics of what tornado is meant to do or how it works, but I can't find any place addressing this question specifically.
See official documentation:
Many methods in RequestHandler are designed to be overridden in
subclasses and be used throughout the application. It is common to
define a BaseHandler class that overrides methods such as write_error
and get_current_user and then subclass your own BaseHandler instead of
RequestHandler for all your specific handlers.
So in your example it is also possible to write a write_response method that could make it easier to write responses in MainHandler as well as in other handlers.
See a simple example:
from tornado.web import RequestHandler
from http import HTTPStatus
import json
class BaseHandler(RequestHandler):
def write_response(self, status_code, result=None, message=None):
self.set_status(status_code)
if result:
self.finish(json.dumps(result))
elif message:
self.finish(json.dumps({
"message": message
}))
elif status_code:
self.set_status(status_code)
self.finish()
class MainHandler(BaseHandler):
def get(self):
self.write_response(status_code=HTTPStatus.OK, message='Hello calculator!')
If the data you return to the client is in the form below, then use write_response with the result argument
data = ['foo', {'bar': ('baz', None, 1.0, 2)}]
self.write_response(status_code=HTTPStatus.OK, result=data)
# and so you will send to the client:
["foo", {"bar": ["baz", null, 1.0, 2]}]
# or
your_numpy_list = your_numpy_object.tolist()
self.write_response(status_code=HTTPStatus.OK, result=your_numpy_list)
So I was missing the most basic thing.
Apparently in Tornado self.write({"data_name":data}) in the get() function will return the data.
Now I am still running into an issue of not being able to return byte data (my circumstances have changed and now I need to turn the numpy array into a wav file and send the wav file over) and I am getting a different error that Object of type 'bytes' is not JSON serializable but if I wont be able to figure it out I will open a new question for it.
Related
I tried to establish a simple websocket connection through this code:
import websocket as ws
tiingo_key = #my respective API key
conn = ws.create_connection('wss://api.tiingo.com/crypto')
subscribe = {'eventName':'subscribe','authorization':tiingo_key,
'eventData': {'thresholdLevel': 5}}
conn.send(subscribe)
conn.recv()
But I get the following error from line 7:
TypeError: unsupported operand type(s) for +=: 'dict' and 'bytes'
Any thoughts?
So you are trying to send a Python Dictionary object through an API. You need to look at the documentation for tiingo and determine what sort of payload that API requires. It doesn't look like you can even call just the straight /crypto endpoint, but rather you need a further extension beyond that (something like /crypto/top. So figure out the endpoint you are trying to call, then base the subscribe payload around that. It shouldn't be a Python dictonary, but rather just a string, probably formatted in JSON. You could also use libraries like json to help.
I'd recommend reading up on how to call APIs so you can understand what the actual call to the API should be, then you just make Python code to create the needed payload.
I'm at the moment in the middle of writing my Bachelor thesis and for it creating a database system with Postgres and Flask.
To ensure the safety of my data, I was working on a file to prevent SQL injections, since a user should be able to submit a string via Http request. Since most of my functions which I use to analyze the Http request use Kwargs and a dict based on JSON in the request I was wondering if it is possible to inject python code into those kwargs.
And If so If there are ways to prevent that.
To make it easier to understand what I mean, here are some example requests and code:
def calc_sum(a, b):
c = a + b
return c
#app.route(/<target:string>/<value:string>)
def handle_request(target,value):
if target == 'calc_sum':
cmd = json.loads(value)
calc_sum(**cmd)
example Request:
Normal : localhost:5000/calc_sum/{"a":1, "b":2}
Injected : localhost:5000/calc_sum/{"a":1, "b:2 ): print("ham") def new_sum(a=1, b=2):return a+b":2 }
Since I'm not near my work, where all my code is I'm unable to test it out. And to be honest that my code example would work. But I hope this can convey what I meant.
I hope you can help me, or at least nudge me in the right direction. I've searched for it, but all I can find are tutorials on "who to use kwargs".
Best regards.
Yes you, but not in URL, try to use arguments like these localhost:5000/calc_sum?func=a+b&a=1&b=2
and to get these arguments you need to do this in flask
#app.route(/<target:string>)
def handle_request(target):
if target == 'calc_sum':
func= request.args.get('func')
a = request.args.get('a')
b = request.args.get('b')
result = exec(func)
exec is used to execute python code in strings
I am trying to implement an mitmproxy addon script, in order to tamper with a particular https packet data - which is by the way decrypted on the fly through mitmproxy's certificate injection.
I am following this Stack Overflow answer to a rather similar question, as well as this tutorial from the mitmproxy docs, but without any success so far.
The packet I'm targeting comes from https://api.example.com/api/v1/user/info.
Now here is the whole python script I wrote so as to tamper with this packet data, based upon the aforementioned sources :
from mitmproxy import ctx
class RespModif:
def _init_(self):
self.num = 0
def response(self, flow):
ctx.log.info("RespModif triggered")
if flow.request.url == "https://api.example.com/api/v1/user/info":
ctx.log.info("RespModif triggered -- In the if statement")
self.num = self.num + 1
ctx.log.info("RespModif -- Proceeded to %d response modifications "
"of the targetted URLs" % self.num)
addons = [
RespModif()
]
Checking out the events log, I'm able to see that the first log information ("RespModif triggered") is being reported onto the log, but the two other log infos (done from inside the if statement) are never reported, which means I think that the if statement does never succeed.
Is there something wrong with my code ?
How can I get the if statement to succeed ?
PS: The target URL is definitely correct, plus I'm using it with a registered account from the client application that is being sniffed with mitmproxy.
Have you tried to use pretty_url attribute ?
Something like :
if flow.request.pretty_url == "https://api.example.com/api/v1/user/info":
....
pretty_url attribute handles full domain name whereas url only deals with corresponding ip address.
Also logging the content of pretty_url should allow to see what exact URL is going through and give more visibility as to what the code is actually doing.
So I have a simple application using Pyro4 (Python Remote Objects). There is an exposed class API, and I have a file that calls the function api.parse(input,userid), which returns some JSON dependent on input. However, instead of simply returning the result as a string (which it did previously), it returns {'data': 'eyJlcnJvciI6ICJDb21tYW5kIG5vdCByZWNvZ25pc2VkIn0=', 'encoding': 'base64'}
, where the base64 is the JSON string that parse should returned.
I'm very confused as to why this is not working - I have tested it previously and there was no issue, the string was just returned with no weird base64 encoding. Only thing I can think of is I have changed networks (School connection to home connection) but I don't think this should be a problem? I have prepared an MVE of some code that indicates the problem.
testserver.py
import Pyro4;
import json;
#Pyro4.expose
class API:
def parse(self,buff,userid):
return prep({"error":"Command not recognised"});
def prep(obj):
return json.dumps(obj).encode("utf-8");
# Pyro stuff
daemon = Pyro4.Daemon() # make a Pyro daemon
ns = Pyro4.locateNS() # find the name server
uri = daemon.register(API) # register the greeting maker as a Pyro object
ns.register("testAPI", uri) # register the object with a name in the name server
daemon.requestLoop()
testclient.py
import Pyro4;
import json;
api = Pyro4.Proxy("PYRONAME:testAPI");
resp = api.parse("whatever","something");
print(resp); # Produces {'data': 'eyJlcnJvciI6ICJDb21tYW5kIG5vdCByZWNvZ25pc2VkIn0=', 'encoding': 'base64'}
# I just want b'{"error":"Command not recognised"}'
Note - Printing at the stage where prep is applied in parse() gives the expected result b'{"error":"Command not recognised"}'. I'm using the command python3 -m Pyro4.naming to start the nameserver if that matters as well. I'm thinking there's probably some global setting/constant I haven't set correctly or something - All responses welcome, thankyou!
The default serialization protocol that Pyro uses is serpent, and that is a text-based protocol. This means it cannot transmit binary data (bytes) unless it encodes them to a text format, which it does in the way you discovered.
There's a little helper function (serpent.tobytes) available that you could use in your client code that automatically detects and converts the response if needed, see the info box in this paragraph in the manual: https://pythonhosted.org/Pyro4/tipstricks.html?highlight=base64#binary-data-transfer-file-transfer
Ofcourse, if you make the data sent by your server strings in the first place, there's no need for this. (This can't be done if the data really is binary though, such as an image or sound clip or whatever)
In your case with the json text it gets transformed into bytes if you encode it. As you discovered there is no need for this at all, just leave it as a string and you won't run into issues. (which leaves me with the question why you're still doing the encode to utf-8 bytes in your client?)
Turns out Pyro doesn't like trying to send raw bytes - It'd rather convert the bytes to base64 then send it as JSON. So to fix, I changed:
def prep(obj):
return json.dumps(obj).encode("utf-8");
to
def prep(obj):
return json.dumps(obj);
And put the encode bit in the client.
As you will be able to see from my previous questions, I have been working on a project, and really want to know how I can get this last part finished.
Quick summary of project: I have a Raspberry Pi that is running a Web Server (Lighttpd) and Flask. It has an RF USB Transmitter connected, which controls the power of a plug via a Python script. (Power.pyon GitHub). This works.
I now need to create an Endpoint in Flask so that Salesforce can send it some JSON, and it will understand it.
I want to keep this as simple as I can, so I understand what it's actually doing. In my last question, someone did provide me with something, but I thought it'd be better have a specific question relating to it, rather than trying to cover too much in one.
All I need to be able to send is 'power=on/off', 'device=0,1,2', 'time=(secondsasinteger)' and 'pass=thepassword' I can send this as URL variables, or a POST to my existing power.py linked above, and it does it.
I would like a simple, clear way of sending this from Salesforce in JSON, to Flask and make it understand the request.
Literally all I need to do now is go to: ip/cgi-bin/power.py?device=0&power=on&time=10&pass=password
That would load a Python script, and turn device 0 on for 10 seconds. (0 is unlimited).
How can I convert that to JSON? What code do I need to put into Flask for it to be able to comprehend that? Can I forward the variables onto the power.py so the Flask file only has to find the variables and values?
I have downloaded Postman in Chrome, and this allows me to send POST's to the Pi to test things.
Where can I find out more info about this, as a beginner?
Can I send something like this?
'requestNumber = JSONRequest.post(
"ip/api.py",
{
deviceid: 0,
pass: "password",
time: 60,
power: "on"
},'
I don't know how you can get saleforce to send a POST request with an associated JSON, but capturing it with Flask is fairly easy. Consider the following example:
from flask import request
from yourmodule import whatever_function_you_want_to_launch
from your_app import app
#app.route('/power/', methods=['POST'])
def power():
if request.headers['Content-Type'] == 'application/json':
return whatever_function_you_want_to_launch(request.json)
else:
return response("json record not found in request", 415)
when saleforce visits the url http://example.com/power/ your applications executes the power() function passing it, as a parameter, a dictionary containing the JSON contents. The whatever_function_you_want_to_launch function can use the dictionary to trigger whatever action you want to take, and return a response back to the power() function. The power() function would return this respose back to salesforce.
For example:
def whatever_function_you_want_to_launch(data):
device = data['deviceid']
power = data['power']
message = ""
if power == "on":
turn_power_on(device)
message = "power turned on for device " + device
else:
turn_power_off(device)
message = "power turned off for device " + device
return make_response(message, 200)
this is just a short example, of course. You'd need to add some additional stuff (e.g. handle the case that the JSON is malformed, or does not contain one of the requested keys).
in order to test the whole stuff you can also use curl command (available on Linux, don't know on other OSs) with this type of syntax:
curl -H "Content-type: application/json" -X POST http://localhost:5000/power/ -d '{"deviceid": "0", "pass": "password", "time": "60", "power": "on"}'