Develop Reference

Non-blocking tornado instance on Openshift?

I am trying to create tornado webserver and quick start made me standard project of tornado, but according to documentation this configuration is blocking.
I am new to non-blocking python.
I have this wsgi file that lies in root folder in my PAAS server
#!/usr/bin/env python
import os
import imp
import sys
#
# Below for testing only
#
if __name__ == '__main__':
ip = 'localhost'
port = 8051
zapp = imp.load_source('application', 'wsgi/application')
from wsgiref.simple_server import make_server
httpd = make_server(ip, port, zapp.application)
httpd.serve_forever()
This is main handler file
#!/usr/bin/env python
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
And Application folder contains this
# Put here yours handlers.
import tornado.wsgi
from . import handlers
handlers = [(r'/',MainHandler),]
application = tornado.wsgi.WSGIApplication(handlers, **settings)
In WSGI mode asynchronous methods are not supported
uses WSGI to deploy the python applications
Is it possible to configure python application on openshift to be fully non-blocking
Though i have seen project that seemed to work

If you are talking about OpenShift V2 (not V3 which uses Kubernetes/Docker), then you need to use the app.py file as described in:
http://blog.dscpl.com.au/2015/08/running-async-web-applications-under.html

Reverse Proxy in Python using WSGI

app, built with Flask.
I want to have the following routing :
"/" -> my Flask app
"/foo/" -> Reverse proxy toward http://bar/
So far, I don't have any reverse proxy, so my application looks like :
import app
[...]
if __name__ == '__main__':
app.app.secret_key = 'XXX'
app.app.run(debug=True, use_reloader=False)
I would like to have the whole project only in python. I don't want any Apache or nginx stack (the project is not meant to be on public network). I saw that I could use a Python WSGI server, such as "wsgiserver" in cherrypy, so my app would be:
from cherrypy import wsgiserver
import app
d = wsgiserver.WSGIPathInfoDispatcher({
'/': app.app.wsgi_app
})
server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 8080), d)
if __name__ == '__main__':
server.start()
And if I want to add a reverse proxy in "/foo", I guess I'll just need to :
from cherrypy import wsgiserver
import app
d = wsgiserver.WSGIPathInfoDispatcher({
'/': app.app.wsgi_app,
'/foo/': SOME_WSGI_REVERSE_PROXY
})
server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 8080), d)
if __name__ == '__main__':
server.start()
So my questions are :
Is there any reverse proxy, written in python, that are WSGI compliant ? (I don't know if any SOME_WSGI_REVERSE_PROXY exists)
Would it work with this kind of implementation ?
Anwser:
As mentioned by accepted anwser, here is the final code:
from cherrypy import wsgiserver
import wsgiproxy.app
import app
app = app.app.wsgi_app
proxy = wsgiproxy.app.WSGIProxyApp("http://bar/")
d = wsgiserver.WSGIPathInfoDispatcher({
'/': app,
'/foo/':proxy
})
server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 8080), d)
if __name__ == '__main__':
try:
server.start()
except KeyboardInterrupt:
server.stop()

See the Paste proxy middleware.
http://pythonpaste.org/wsgiproxy/

This seems to be an actively maintained alternative to the original wsgiproxy now:
https://pypi.org/project/WSGIProxy2/

mitmproxy: mitmproxy.org
It's a python proxy with reverse proxy capability.

using twisted adbapi in ZSI soap

I'm new to python and currently researching its viability to be used as a soap server. I currently have a very rough application that uses the mysql blocking api, but would like to try twisted adbapi. I've successfully used twisted adbapi on regular twisted code using reactors, but can't seem to make it work with code below using ZSI framework. It's not returning anything from mysql. Anyone ever used twisted adbapi with ZSI?
import os
import sys
from dpac_server import *
from ZSI.twisted.wsgi import (SOAPApplication,
soapmethod,
SOAPHandlerChainFactory)
from twisted.enterprise import adbapi
import MySQLdb
def _soapmethod(op):
op_request = GED("http://www.example.org/dpac/", op).pyclass
op_response = GED("http://www.example.org/dpac/", op + "Response").pyclass
return soapmethod(op_request.typecode, op_response.typecode,operation=op, soapaction=op)
class DPACServer(SOAPApplication):
factory = SOAPHandlerChainFactory
#_soapmethod('GetIPOperation')
def soap_GetIPOperation(self, request, response, **kw):
dbpool = adbapi.ConnectionPool("MySQLdb", '127.0.0.1','def_user', 'def_pwd', 'def_db', cp_reconnect=True)
def _dbSPGeneric(txn, cmts):
txn.execute("call def_db.getip(%s)", (cmts, ))
return txn.fetchall()
def dbSPGeneric(cmts):
return dbpool.runInteraction(_dbSPGeneric, cmts)
def returnResults(results):
response.Result = results
def showError(msg):
response.Error = msg
response.Result = ""
response.Error = ""
d = dbSPGeneric(request.Cmts)
d.addCallbacks(returnResults, showError)
return request, response
def main():
from wsgiref.simple_server import make_server
from ZSI.twisted.wsgi import WSGIApplication
application = WSGIApplication()
httpd = make_server('127.0.0.1', 8080, application)
application['dpac'] = DPACServer()
print "listening..."
httpd.serve_forever()
if __name__ == '__main__':
main()

The code you posted creates a new ConnectionPool per (some kind of) request and it never stops the pool. This means you'll eventually run out of resources and you won't be able to service any more requests. "Eventually" is probably after one or two or three requests.
If you never get any responses perhaps this isn't the problem you've encountered. It will be a problem at some point though.
On closer inspection, I wonder if this code even runs the Twisted reactor at all. On first read, I thought you were using some ZSI Twisted integration to run your server. Now I see that you're using wsgiref.simple_server. I am moderately confident that this won't work.
You're already using Twisted, use Twisted's WSGI server instead.
Beyond that, verify that ZSI executes your callbacks in the correct thread. The default for WSGI applications is to run in a non-reactor thread. Twisted APIs are not thread-safe, so if ZSI doesn't do something to correct for this, you'll have bugs introduced by using un-thread-safe APIs in threads.

Integrating fanstatic with tornado web?

I'm attempting to integrate fanstatic.org with tornadoweb and was curious if anyone has any prior knowledge or any code that may reflect how this is done? I've been reading the documentation and i believe it can be done since tornado does provide a wsgi interface.
thanks

import tornado.wsgi
from fanstatic import Fanstatic
from your_tornado_app import MainHandler # tornado.web.RequestHandler
app = tornado.wsgi.WSGIApplication([
(r"/", MainHandler),
])
app = Fantastic(app)
if __name__ == '__main__':
from wsgiref.simple_server import make_server
server = make_server('127.0.0.1', 8080, app)
server.serve_forever()

How do you run the Tornado web server locally?

Is it possible to run Tornado such that it listens to a local port (e.g. localhost:8000). I can't seem to find any documentation explaining how to do this.

Add an address argument to Application.listen() or HTTPServer.listen().
It's documented here (Application.listen) and here (TCPServer.listen).
For example:
application = tornado.web.Application([
(r'/blah', BlahHandler),
], **settings)
# Create an HTTP server listening on localhost, port 8080.
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8080, address='127.0.0.1')

In the documetaion they mention to run on the specific port like
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8000)
tornado.ioloop.IOLoop.instance().start()
You will get more help from http://www.tornadoweb.org/documentation/overview.html and http://www.tornadoweb.org/documentation/index.html

Once you've defined an application (like in the other answers) in a file (for example server.py), you simply save and run that file.
python server.py

If you want to daemonize tornado - use supervisord. If you want to access tornado on address like http://mylocal.dev/ - you should look at nginx and use it like reverse proxy. And on specific port it can be binded like in Lafada's answer.