I am running a simple Flask app with Tornado, but the view only handles one request at a time. How can I make it handle multiple concurrent requests?
The fix I'm using is to fork and use the multiple processes to handle requests, but I don't like that solution.
from flask import Flask
app = Flask(__name__)
#app.route('/flask')
def hello_world():
return 'This comes from Flask ^_^'
from tornado.wsgi import WSGIContainer
from tornado.ioloop import IOLoop
from tornado.web import FallbackHandler, RequestHandler, Application
from flasky import app
class MainHandler(RequestHandler):
def get(self):
self.write("This message comes from Tornado ^_^")
tr = WSGIContainer(app)
application = Application([
(r"/tornado", MainHandler),
(r".*", FallbackHandler, dict(fallback=tr)),
])
if __name__ == "__main__":
application.listen(8000)
IOLoop.instance().start()
The immediate answer is that you should use a dedicated WSGI server, such as uWSGI or Gunicorn, and configure it to use multiple workers. Do not Tornado as a WSGI server.
Your fix of spawning processes is correct in as much as using WSGI with Tornado is "correct". WSGI is a synchronous protocol: one worker handles one request at a time. Flask doesn't know about Tornado, so it can't play nice with it by using coroutines: handling the request happens synchronously.
Tornado has a big warning in their docs about this exact thing.
WSGI is a synchronous interface, while Tornado’s concurrency model is based on single-threaded asynchronous execution. This means that running a WSGI app with Tornado’s WSGIContainer is less scalable than running the same app in a multi-threaded WSGI server like gunicorn or uwsgi. Use WSGIContainer only when there are benefits to combining Tornado and WSGI in the same process that outweigh the reduced scalability.
In other words: to handle more concurrent requests with a WSGI application, spawn more workers. The type of worker also matters: threads vs. processes vs. eventlets all have tradeoffs. You're spawning workers by creating processes yourself, but it's more common to use a WSGI server such as uWSGI or Gunicorn.
Related
I have this tornado application wrapped with django function as WSGI app (using in windows)
from django.core.wsgi import get_wsgi_application
from django.conf import settings
from waitress import serve
settings.configure()
wsgi_app = tornado.wsgi.WSGIContainer(django.core.wsgi.WSGIHandler())
def tornado_app():
url = [(r"/models//predict", PHandler),
(r"/models//explain", EHandler),
('.*', tornado.web.FallbackHandler, dict(fallback=wsgi_app))]
return Application(url, db=ELASTIC_URL, debug=options.debug, autoreload=options.debug)
if __name__ == "__main__":
application = tornado_app()
http_server = HTTPServer(application)
http_server.listen(LISTEN_PORT)
IOLoop.current().start()
Not sure how to use waitress,
To serve using waitress I tried http_server = serve(application), server is starting, now sure is it correct, getting error when hitting the endpoint
waitress is a WSGI server; Tornado is not based on or compatible with WSGI. You cannot use waitress to serve the Tornado portions of your application.
To serve both Tornado and WSGI applications in one thread, you need to use Tornado's HTTPServer as you've done in the original example. For better scalability, I'd recommend splitting the Tornado and Django portions of your application into separate processes and putting a proxy like nginx or haproxy in front of them.
I'm setting up a Flask server, and want to use a Twisted ReverseProxyResource to proxy over another local server. In Flask, I have a current_user.is_authenticated boolean which I use to protect pages. How can I lock the ReverseProxyResource using this variable, so that it cannot be accessed when a user is logged out? The twisted.web.guard module requires me to set up another HTTP authentication system entirely, and the library doesn't seem to have any other built-in solution.
I've set up some demo code, (based off a previous question) attempting to place the Flask server inside of the Twisted reactor. I'm using the ReverseProxyResource to allow for two-way communication with a Shellinabox server on port 4200.
from flask import Flask
from twisted.internet import reactor
from twisted.web.proxy import ReverseProxyResource
from twisted.web.resource import Resource
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
app = Flask(__name__)
#app.route('/example')
def index():
return 'Flask within Twisted?'
flask_site = WSGIResource(reactor, reactor.getThreadPool(), app)
root = Resource()
site_example = ReverseProxyResource('localhost', 4200, b'')
root.putChild(b'ssh', site_example)
reactor.listenTCP(8081, Site(root))
reactor.run()
I'd be okay switching to another reverse proxy, but only Twisted has worked so far. I'm also not wanting to switch to the Klein framework due to what I already have established in Flask.
I wrote a flask app as kind of a proxy, to analyse the data passing through it and provide a web page where I get the result. All seems to went well when using the default development server that comes with flask, say using:
app.run()
But when I tried to deploy the app to a server, for example tornado or wsgiref.simple_server in Python standard library using:
from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from app import app
http_server = HTTPServer(WSGIContainer(app))
http_server.listen(5000)
IOLoop.instance().start()
or
from wsgiref.simple_server import make_server
from app import app
httpd = make_server('', 5000, app)
httpd.serve_forever()
This two ways result in getting 404 error on requests got 200 OK previously.
The requests my app gets, since it serves as a proxy, are with absolute urls in the request lines like POST http://example.com/test HTTP/1.1. When I'm using the development server, this request is handled by the function registered under /test normally, something like:
#app.route('/test', methods=['GET', 'POST'])
def handle_test():
...
and the request.url, as I checked, is http://example.com/test.
When using the other two ways, the request is handled by the error handler with code 404, and the handle_test() function is never invoked. The request.url, which seems to cause the problem, is http://example.com/http://example.com/test, definitely not what I want.
So I want to know:
What changed the url to the wrong one, and when did this happen.
Why the app behaves differently on default development server and servers like tornado.
Of course, how to get rid of this problem.
Thanks!
Tornado does not currently support proxy-style requests; see https://github.com/tornadoweb/tornado/issues/1036
Tornado's WSGIContainer is also a poor choice for proxies because of its single-threaded concurrency model (see http://www.tornadoweb.org/en/stable/wsgi.html#tornado.wsgi.WSGIContainer). Even if the aforementioned bug were fixed, your proxy would perform poorly. I recommend either using a multithreaded WSGI server (like gunicorn or uwsgi, although I don't know whether they support proxy-style requests) or rewriting the proxy as a native Tornado app (without flask) to take advantage of asynchronous features.
How do I server my Django application using gevent-socketio's SocketIOServer though Apache via uWSGI?
I have the following uWSGI .ini file:
[uwsgi]
socket = 127.0.0.1:3031
master = true
processes = 2
env = DJANGO_SETTINGS_MODULE=demo.settings
module = app:serve
then I have the following app.py:
from gevent import monkey
from socketio.server import SocketIOServer
import django.core.handlers.wsgi
import os
import sys
monkey.patch_all()
PORT = 3031
os.environ['DJANGO_SETTINGS_MODULE'] = 'demo.settings'
def serve():
application = django.core.handlers.wsgi.WSGIHandler()
SocketIOServer(('', PORT), application, namespace="socket.io").serve_forever()
But it just keeps on loading, basically my problem is how do I tell uWSGI to use SocketIOServer when serving?
It is not clear if you want uWSGI to serve both or you want an additional process with the socketio server.
generally you cannot mix blocking apps (like django) with non-blocking (like gevent-based) in the same process and even if you are using monkey patching your database adapter will not be monkeypatched (unless you are using a native python-adapter, and this is uncommon in django).
So i suppose you want to spawn the SocketIOServer as a different process. Just move the last 2 lines out of serve() so the uwsgi importer will parse/run both
I want a simple python web server for the following use case:
I want to write a simple server that will accept HTTP requests from my application running on Google App Engine.
The server will accept HTTP requests, and then send iphone notifications. (Basically, I need this extra server to account for the lack of socket support in google app engine).
I guess I need the server to be able to maintain this persistent connection with Apple's Push Notification Service. So I'll need to have some sort of thread always open for this. So I need some sort of web server that can accept the request pass it off to the other thread with the persistent connection to APNS.
Maybe multiple processes and one of pythons queuing tools to communicate between them? Accept the HTTP request, then enqueue a message to the other process?
I was wondering what someone with a bit of experience would suggest. I'm starting to think that maybe even writing my own simple server is a good option (http://fragments.turtlemeat.com/pythonwebserver.php).
One option would be the (appropriately named) SimpleHTTPServer, which is part of the Python standard library. Another, more flexible but more complicated option would be to write your server in Twisted.
I've been writing simple http servers using gevent and bottle -- an example:
#!/usr/bin/env python
import gevent.monkey
gevent.monkey.patch_all()
import bottle
bottle.debug(True)
import gevent.wsgi
from bottle import route, run, request, response, static_file, abort
#route('/echo')
def echo():
s = request.GET.get('s', 'o hai')
return '<html><head><title>echo server</title></head><body>%s</body></html>\r\n' % (s)
#route('/static/:filename')
def send_static(filename):
root = os.getcwd() + '/static'
return static_file(filename, root=root)
if __name__ == '__main__':
app = bottle.app()
wsgi_server = gevent.wsgi.WSGIServer(('0.0.0.0', 8000), app)
print 'Starting wsgi search on port 8000'
wsgi_server.serve_forever()
So you could write a simple server that sticks a job into a Queue (see gevent.queue) and have another worker greenlet that handles reading requests from the queue and processing them...