While doing an async mongodb query like the one in the below class how is this call really non-blocking if I still have access to an argument like self.get_argument("ip_address") inside the callback function? Or should I not access to the argument like this to keep the call async?
class MainHandler(tornado.web.RequestHandler):
def get(self):
app_key = self.get_argument("app_key")
#async call to mongodb. call _valid_app afterwards
db.apps.find_one({'app_key': app_key}, callback=self._valid_app);
def _valid_app(self, response, error):
if error:
raise tornado.web.HTTPError(500)
if response:
ip_address = self.get_argument("ip_address")
#rest of the code
else:
print("invalid app_key")
The instance self referenced in the callback function will be hanging around until the end of the callback function, therefore self.arguments will always be available inside _valid_app.
Maybe you can be confused by what would happen if another request to the same handler were made during the async call to Mongo. This would not be a problem because, for any new request a new instance of MainHandler is created, not interfering with the previous one.
Related
I'm building an async library with aiohttp. The library has a single client that on instantiation creates a ClientSession and uses it to make requests to an API (it's an REST API wrapper)
The problem i'm facing is how to cleanly close the client session on exit?
If the session is not explicitly closed a whole lot of errors come out but i can't simply use context managers to close the session since i don't know when the program will end.
A tipical use would be this:
from mylibrary import Client
client = Client()
async main():
await client.get_foo(...)
await client.patch_bar(...)
asyncio.run(main())
I could add await client.close_session() on main but I want to remove this responsability from the enduser so ideally the client would automatically close the ClientSession when the program ends.
How can I do this?
I have tried using __del__ on the client to get the loop and close the session without success as well as using the atexit library, but it seems that by the time these run the asyncio loop has already been destroyed and I still get the warnings.
The specific error is:
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000013ACFD54AF0>
transport: <_ProactorSocketTransport fd=1052 read=<_OverlappedFuture cancelled>>
I did some research on this error and google seems to think it's because I need to implement flow control, I have however and this error only occurs if I don't explicitly close the session.
Unfortunately, it seems like the only clean pattern that can apply there is to make your client itself an (async) context manager, and require that your users use it in a with block.
The __del__ method could work in some cases - but it would require that code from your users would not "leak" the Client instance itself.
so, the code is trivial - the burden on your users is not zero:
class Client:
...
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc_value, tb):
await self.close_session()
Creating a pseudo-hook on loop.stop:
Another way, though not "clean" and not guaranteed to work, could be to decorate the running loop stop function to add a call to close_session.
If the user code just "halts" and does not tear down the loop properly, this can't help anyway - but I guess it might be an option for "well behaved" users.
The big problem here is this is not documented - but taking a pick on asyncio internals, it looks it always will go through self.stop().
import asyncio
class ShutDownCb:
def __init__(self, cb):
self.cb = cb
self.stopping = False
loop = self.loop = asyncio.get_running_loop()
self.original_stop = loop.stop
loop.stop = self.new_stop
async def _stop(self):
self.task.result()
return self.original_stop()
def new_stop(self):
if not self.stopping:
self.stopping = True
self.task = asyncio.create_task(self.cb())
asyncio.create_task(self._stop())
return
return self.original_stop()
class Client:
def __init__(self, ...):
...
ShutDownCb(self.close_session)
I'm relatively new to Twisted and crossbar.io and I'm currently working on some database abstractions using sqlalchemy and alchimia (a layer to use sqlalchemy with twisted). The db abstractions I build so far are working as expected but I get problems when making asynchronous db calls inside my crossbar procedures. I guess that is because i have nested asynchronous calls with the outer call being some crossbar remote procedure and the inner being the database access.
What i want to do is the following:
#inlineCallbacks
def onJoin(self, details):
...
def login(user, password): # <-- outer call
...
db_result = yield db.some_query(user, password) # <-- inner call
for row in db_result: # <-- need to wait for db_result
return 'authentication_ticket'
return 'error'
To be able to authenticate the user i have to wait for the data from the db and then either issue a valid ticket or return an error. Currently I get an error that i cannot iterate over an deferred, because i don't wait for the db query to be finished.
Now, how do i wait inside my login RPC for the inner db call, then authenticate the user and then return. I know it is possible to chain asynchronous calls in twisted, but i don't know how to do this in this special case with crossbar and when my outer function uses the #inlineCallbacks shortcut.
Update:
Tinkering around I now can add a callback to the db query. Unfortunately now I don't know how to pass a return value from the inner function to the outer function (the outer function needs to return the ticket the user gets authenticated with):
#inlineCallbacks
def onJoin(self, details):
...
def login(user, password): # <-- outer call
db_result = db.some_query(user, password) # <-- inner call
def my_callback(query_result):
if not query_result.is_empty():
return 'user_ticket' # <-- needs to be returned by outer method
db_result.addCallback(my_callback)
return my_callback_result # <-- need something like that
I tried defer.returnValue('user_ticket') inside my callback function but this gives me an error.
Your problem appears to be that while login is expecting to yield Deferreds, it isn't decorated with #inlineCallbacks.
I am using Tornado Webserver and want to internally call a WebSocketHandler from a RequestHandler.
It is not possible to use the redirect /redirectHandler functionality, because the WebSocketHandler class to call ("IndexHandlerDynamic1" in the example below) will be created with a classFactory.
Using the definition of Requesthandler (here) my example looks like:
class IndexHandlerDynamic1(tornado.web.WebSocketHandler):
def initialize(self):
print "Forwarded to Websocket"
def open(self):
print "WebSocket opened"
class IndexHandlerDistributor(tornado.web.RequestHandler):
def get(self, channelId):
IndexHandlerDynamic1(self.application, self.request)
If I request the related url he jumps into IndexHandlerDistributor and IndexHandlerDynamic1.initialize() is called.
But on Clientside the Browser console outputs the following error:
Error during WebSocket handshake: Unexpected response code: 200
Obviously the socket connection is not opened correctly, what's my mistake ?
EDIT:
Thanks to Ben for his help!
Sadly I still have trouble to route the user to a dynamically created class named like a url parameter. I hope you can understand my problem by having a look on my example:
app = tornado.web.Application(
[(r"/", IndexHandler)] +
[(r"/channel/(?P<channelId>[^\/]+)?", ClassFactory(channelId))]
)
How to use channelId as a parameter for my call of ClassFactory as Requesthandler?
Or is there maybe another way to dynamically change the routing of my application while the application is running? If so, i could use this way to solve my initial task.
The problem is that you're attaching two RequestHandlers to the same request. I'm not sure that dynamically creating handler classes is a great idea, but if you want to do it just pass your factory function (which is not itself a RequestHandler) to the url routing table. The routing table doesn't necessarily need a RequestHandler subclass, it just needs an object which can be called with (app, request) and return a RequestHandler instance.
I am making a web application using Python + Tornado. Wrote a simple handler for example:
class AdminHandler(BaseHandler):
#tornado.web.authenticated
#tornado.web.asynchronous
#gen.engine
def get(self):
response = yield gen.Task(self.acync_func_test, 'my')
print response
self.render('admin/index.html')
def acync_func_test(self, argument, callback):
for i in xrange(1,59999000):
i**2+2-12
callback(argument)
But function is not performed asynchronously. Other clients are waiting until the query is executed for the first. How to make non-blocking execution?
Update:
Added decorator '#gen.engine' to your async_func_test() function, but still blocked((
Add decorator '#gen.engine' to your async_func_test() function
I'm new to Twisted and I'm having trouble with some necessary subclassing for the static.File in twisted. i'm trying to set request headers within the subclass.
class ResponseFile(static.File):
def render_GET(self, request):
request.setHeader('Content-Disposition', ['attachment ; filename="tick_db_export.csv"'])
static.File.render_GET(self, request)
if __name__ == "__main__":
from twisted.internet import reactor
root = ResponseFile('WebFolder')
testHandler = TestHandler()
root.putChild('main', testHandler)
reactor.listenTCP(3650, server.Site(root))
reactor.run()
The first bit of code is the subclass definition itself (pretty straightforward), while the second bit is the initialization portion from my code (this isn't all of my code). I've have also subclassed a resource.Resource object called TestHandler. WebFolder is another folder containing many static files.
However, I am getting most of these types of exception when making calls to the server.
Unhandled Error
Traceback (most recent call last):
Failure: exceptions.RuntimeError: Producer was not unregistered for /
With many different paths other than root.
The problem in your code is in render_GET method. It returns nothing. Basically it must return string for synchronous response and NOT_DONE_YET value for asynchronous response. In your case render_GET returns None (and your connection get closed immediately).
So you have to make a smaller change in your render_GET (add proper return):
def render_GET(self, request):
request.setHeader('Content-Disposition', ['attachment ; filename="tick_db_export.csv"'])
return static.File.render_GET(self, request)
If you inspect twisted.web.static.py module you'll find that File.render_GET makes producer and returns NOT_DONE_YET which makes connection to hold on until it is not explicitly closed (in our case, after file is downloaded).