I'm trying to create a Python script which running Tornado Async http client with fetch and trying to get the response and print the response.body to the screen.
my class code is:
class MyAsyncHTTPClient(httpclient.AsyncHTTPClient):
#gen.coroutine
def _fetch(self, url):
print('send Asynchronous request ...['+url+"] ")
http_response = yield gen.Task(self.fetch, url)
print(http_response.body)
print('got Asynchronous response !!'+str(http_response))
raise gen.Return(http_response.body)
and I'm calling it this way:
async_http_client = MyAsyncHTTPClient()
res_body = async_http_client._fetch(url)
The issue is that I'm not sure how to deal with this code to get the returned value once it's ready.
can you please help?
Thanks!
Editing
I have also tried implementing this function like:
class MyAsyncHTTPClient(httpclient.AsyncHTTPClient):
#gen.coroutine
def _fetch(self, url):
print('send Asynchronous request ...['+url+"] "+str(self))
http_response = yield self.fetch(url)
print('got Asynchronous response !!')
return http_response.body
But I'm having the same results :(
Editing again
I have succeeded running the async class...but without the inherited object self.
I did it like that:
#gen.coroutine
def _fetch_async(self, url):
print('send Asynchronous request ...[' + url + "] ")
http_response = yield httpclient.AsyncHTTPClient().fetch(url)
#http_response = yield self.fetch(url)
print('got Asynchronous response !!')
return http_response.body
and it worked fine.
The issue is that I need to use the inherited object self, and I'm not sure what am I missing here when defining the class.
When debugging I can see that self is very "thin" with its content..
Please let me know what am I doing wrong here.
Thanks!
Asynchronous functions can only be called from other asynchronous functions. You must call _fetch like this:
#gen.coroutine
def f():
async_http_client = MyAsyncHTTPClient()
res_body = yield async_http_client._fetch(url)
If you're not doing this in the context of a tornado server, the best way to call a coroutine from your __main__ block is like this:
tornado.ioloop.IOLoop.current().run_sync(f)
Related
I want to attach a middleware to specific handler and if client is not authorized then want to return an error response. However with the following code :
async def middleware_factory(app, handler):
async def auth_handler(request):
if request.headers.get('Authorization') == 'Basic test1234':
return await handler(request)
return web.Response(text='Forbidden', status='403')
return auth_handler
I am getting an exception that :
AssertionError: Handler <function AbstractRoute.__init__.
<locals>.handler_wrapper at 0x10da56bf8> should return response
instance, got <class 'NoneType'> [middlewares [<function
middleware_factory at 0x1104cb268>]]
Documentation states that I should return a response object which I am doing. Still this error. Where am I going wrong?
You can look to an example from official documentation.
But the main concern that if you want to have Middleware Factory - needs to be a function not a coroutine. Also, recommend to use #web.middleware decorator for that.
from aiohttp import web
def middleware_factory(text):
#web.middleware
async def sample_middleware(request, handler):
resp = await handler(request)
resp.text = resp.text + text
return resp
return sample_middleware
I'm trying to write a post request for a Python Tornado server that sleeps for a second before sending a response back to a client. The server must handle many of these post requests per minute. The following code doesn't work because of BadYieldError: yielded unknown object <generator object get at 0x10d0b8870>
#asynchronous
def post(self):
response = yield IOLoop.instance().add_timeout(time.time() + 1, self._process)
self.write(response)
self.finish()
#gen.coroutine
def _process(self, callback=None):
callback("{}")
The server is to receive a post request, wait a second, and then return the result without blocking other requests. This is Python 2.7. How to resolve this? Thanks!
Either use callbacks or "yield", not both. So you could do:
#asynchronous
def post(self):
IOLoop.instance().add_timeout(time.time() + 1, self._process)
def _process(self):
self.write("{}")
self.finish()
Or, better:
#gen.coroutine
def post(self):
yield gen.sleep(1)
self.write("{}")
# Tornado calls self.finish when coroutine exits.
So I have repeating code that I do for many GETs -- checking whether the response was cached previously and returning that if it is available.
The code I'd like to get working looks like this:
class Handler(web.RequestHandler):
#gen.coroutine
def get_cache(self):
try:
response = yield gen.Task(get_redis)
except:
logging.log()
if response:
self.finish(response)
raise gen.Return()
#gen.coroutine
#asynchronous
def get(self):
self.get_cache()
response = do_sql_get()
self.set_cache(key, response)
self.finish(response)
What's happening now is that it gets the cache if there but continues running the rest of the code in self.get. That it does this makes sense to me, but I'm not sure how to refactor it properly with it stopping as soon as self.finish is called in the self.get_cache method.
get_cache should return a value that indicates whether it finished the request or not (or it should return the cached data and leave it to the caller to finish the request). I would do one of the following:
#gen.coroutine
def serve_from_cache(self):
response = yield gen.Task(get_redis)
if response:
self.finish(response)
raise gen.Return(True)
else:
raise gen.Return(False)
#gen.coroutine
def get(self):
if (yield self.serve_from_cache()):
return
# do work
yield self.set_cache(...)
or
#gen.coroutine
def get_cache(self):
return yield gen.Task(get_redis)
#gen.coroutine
def get(self):
resp = yield self.get_cache()
if resp:
self.finish(resp)
return
# do work...
I'm trying to use AsyncHTTPClient in Tornado to do multiple callouts to a "device" available over http:
def ext_call(self, params):
device = AsyncHTTPClient()
request = HTTPRequest(...)
return partial(device.fetch, request)
#coroutine
def _do_call(self, someid):
acall = self.ext_call(params)
waitkey = str(someid)
acall(callback = (yield Callback(waitkey)))
response = yield Wait(waitkey)
raise Return(response)
def get_device_data(self, lst):
for someid in lst:
r = self._do_call(someid)
print 'response', r
But instead of HTTP responses as AsyncHTTPClient should return after .fetch, I'm getting this:
response <tornado.concurrent.TracebackFuture object at 0x951840c>
Why this is not working like examples in http://www.tornadoweb.org/en/stable/gen.html ?
Got this one solved. It appears that #coroutine has to be applied all the way down from the get/post method of your class inheriting from RequestHandler, otherwise #coroutine/yield magic does not work.
Apparently this is a case of Tornado newbiness combined with bad design on my part: according to a colleague one should not do "callback spaghetti" of nested #coroutine and yield()s, but rather move all the synchronous code out of request handler and call before or after async code it and have #coroutine call hierarchy flat rather than deep.
I am implementing a SOAP web service using tornado (and the third party tornadows module). One of the operations in my service needs to call another so I have the chain:
External request in (via SOAPUI) to operation A
Internal request (via requests module) in to operation B
Internal response from operation B
External response from operation A
Because it is all running in one service it is being blocked somewhere though. I'm not familiar with tornado's async functionality.
There is only one request handling method (post) because everything comes in on the single url and then the specific operation (method doing processing) is called based on the SOAPAction request header value. I have decorated my post method with #tornado.web.asynchronous and called self.finish() at the end but no dice.
Can tornado handle this scenario and if so how can I implement it?
EDIT (added code):
class SoapHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def post(self):
""" Method post() to process of requests and responses SOAP messages """
try:
self._request = self._parseSoap(self.request.body)
soapaction = self.request.headers['SOAPAction'].replace('"','')
self.set_header('Content-Type','text/xml')
for operations in dir(self):
operation = getattr(self,operations)
method = ''
if callable(operation) and hasattr(operation,'_is_operation'):
num_methods = self._countOperations()
if hasattr(operation,'_operation') and soapaction.endswith(getattr(operation,'_operation')) and num_methods > 1:
method = getattr(operation,'_operation')
self._response = self._executeOperation(operation,method=method)
break
elif num_methods == 1:
self._response = self._executeOperation(operation,method='')
break
soapmsg = self._response.getSoap().toprettyxml()
self.write(soapmsg)
self.finish()
except Exception as detail:
#traceback.print_exc(file=sys.stdout)
wsdl_nameservice = self.request.uri.replace('/','').replace('?wsdl','').replace('?WSDL','')
fault = soapfault('Error in web service : {fault}'.format(fault=detail), wsdl_nameservice)
self.write(fault.getSoap().toxml())
self.finish()
This is the post method from the request handler. It's from the web services module I'm using (so not my code) but I added the async decorator and self.finish(). All it basically does is call the correct operation (as dictated in the SOAPAction of the request).
class CountryService(soaphandler.SoapHandler):
#webservice(_params=GetCurrencyRequest, _returns=GetCurrencyResponse)
def get_currency(self, input):
result = db_query(input.country, 'currency')
get_currency_response = GetCurrencyResponse()
get_currency_response.currency = result
headers = None
return headers, get_currency_response
#webservice(_params=GetTempRequest, _returns=GetTempResponse)
def get_temp(self, input):
get_temp_response = GetTempResponse()
curr = self.make_curr_request(input.country)
get_temp_response.temp = curr
headers = None
return headers, get_temp_response
def make_curr_request(self, country):
soap_request = """<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:coun='CountryService'>
<soapenv:Header/>
<soapenv:Body>
<coun:GetCurrencyRequestget_currency>
<country>{0}</country>
</coun:GetCurrencyRequestget_currency>
</soapenv:Body>
</soapenv:Envelope>""".format(country)
headers = {'Content-Type': 'text/xml;charset=UTF-8', 'SOAPAction': '"http://localhost:8080/CountryService/get_currency"'}
r = requests.post('http://localhost:8080/CountryService', data=soap_request, headers=headers)
try:
tree = etree.fromstring(r.content)
currency = tree.xpath('//currency')
message = currency[0].text
except:
message = "Failure"
return message
These are two of the operations of the web service (get_currency & get_temp). So SOAPUI hits get_temp, which makes a SOAP request to get_currency (via make_curr_request and the requests module). Then the results should just chain back and be sent back to SOAPUI.
The actual operation of the service makes no sense (returning the currency when asked for the temperature) but i'm just trying to get the functionality working and these are the operations I have.
I don't think that your soap module, or requests is asyncronous.
I believe adding the #asyncronous decorator is only half the battle. Right now you aren't making any async requests inside of your function (every request is blocking, which ties up the server until your method finishes)
You can switch this up by using tornados AsynHttpClient. This can be used pretty much as an exact replacement for requests. From the docoumentation example:
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
http = tornado.httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed-api.com/v2/feed/bret",
callback=self.on_response)
def on_response(self, response):
if response.error: raise tornado.web.HTTPError(500)
json = tornado.escape.json_decode(response.body)
self.write("Fetched " + str(len(json["entries"])) + " entries "
"from the FriendFeed API")
self.finish()
Their method is decorated with async AND they are making asyn http requests. This is where the flow gets a little strange. When you use the AsyncHttpClient it doesn't lock up the event loop (PLease I just started using tornado this week, take it easy if all of my terminology isn't correct yet). This allows the server to freely processs incoming requests. When your asynchttp request is finished the callback method will be executed, in this case on_response.
Here you can replace requests with the tornado asynchttp client realtively easily. For your soap service, though, things might be more complicated. You could make a local webserivce around your soap client and make async requests to it using the tornado asyn http client???
This will create some complex callback logic which can be fixed using the gen decorator
This issue was fixed since yesterday.
Pull request:
https://github.com/rancavil/tornado-webservices/pull/23
Example: here a simple webservice that doesn't take arguments and returns the version.
Notice you should:
Method declaration: decorate the method with #gen.coroutine
Returning results: use raise gen.Return(data)
Code:
from tornado import gen
from tornadows.soaphandler import SoapHandler
...
class Example(SoapHandler):
#gen.coroutine
#webservice(_params=None, _returns=Version)
def Version(self):
_version = Version()
# async stuff here, let's suppose you ask other rest service or resource for the version details.
# ...
# returns the result.
raise gen.Return(_version)
Cheers!