I'm wondering if it is possible in the Tornado framework to register multiple Application on the same IOLoop ?
Something like
application1 = web.Application([
(r"/", MainPageHandler),
])
http_server = httpserver.HTTPServer(application1)
http_server.listen(8080)
application2 = web.Application([
(r"/appli2", MainPageHandler2),
])
http_server2 = httpserver.HTTPServer(application2)
http_server2.listen(8080)
ioloop.IOLoop.instance().start()
Basically I'm trying to structure my webapp so that:
functional applications are separated
multiple handlers with the same purpose (e.g. admin/monitoring/etc) are possible on each webapp
The simple thing is if you were to bind your applications to different ports:
...
http_server = httpserver.HTTPServer(application1)
http_server.listen(8080) # NOTE - port 8080
...
http_server2 = httpserver.HTTPServer(application2)
http_server2.listen(8081) # NOTE - port 8081
ioloop.IOLoop.instance().start()
This is the base case that Tornado makes easy. The challenge is that by routing to applications at the URI level you're crossing a design boundary which is that each application is responsible for all of the URIs that that are requested by it.
If they all really need to be serviced at the URI level not port, it would probably be best to host different applications on different ports and have Nginx/Apache do the URI routing - anything that involves messing with the Application/Request handling is going to be a world of hurt.
Related
Here is my code.
application1 = web.Application([
(r"/", MainPageHandler),
])
http_server = httpserver.HTTPServer(application1)
http_server.listen(8080)
application2 = web.Application([
(r"/appli2", MainPageHandler2),
])
http_server2 = httpserver.HTTPServer(application2)
http_server2.listen(8081)
ioloop.IOLoop.instance().start()
I want to use multi-process like https://www.tornadoweb.org/en/stable/httpserver.html.
What should I modify my codes?
To support multiple Applications with multi-process mode, you must use the "advanced" mode with the add_sockets method:
sockets1 = tornado.netutil.bind_sockets(8080)
sockets2 = tornado.netutil.bind_sockets(8081)
tornado.process.fork_processes(0)
server1 = HTTPServer(app1)
server2 = HTTPServer(app2)
server1.add_sockets(sockets1)
server2.add_sockets(sockets2)
IOLoop.current().start()
Bind all the sockets (and do nothing else) before the call to fork_processes, then create the servers and add the sockets to them.
How can I manage my rabbit-mq connection in Pyramid app?
I would like to re-use a connection to the queue throughout the web application's lifetime. Currently I am opening/closing connection to the queue for every publish call.
But I can't find any "global" services definition in Pyramid. Any help appreciated.
Pyramid does not need a "global services definition" because you can trivially do that in plain Python:
db.py:
connection = None
def connect(url):
global connection
connection = FooBarBaz(url)
your startup file (__init__.py)
from db import connect
if __name__ == '__main__':
connect(DB_CONNSTRING)
elsewhere:
from db import connection
...
connection.do_stuff(foo, bar, baz)
Having a global (any global) is going to cause problems if you ever run your app in a multi-threaded environment, but is perfectly fine if you run multiple processes, so it's not a huge restriction. If you need to work with threads the recipe can be extended to use thread-local variables. Here's another example which also connects lazily, when the connection is needed the first time.
db.py:
import threading
connections = threading.local()
def get_connection():
if not hasattr(connections, 'this_thread_connection'):
connections.this_thread_connection = FooBarBaz(DB_STRING)
return connections.this_thread_connection
elsewhere:
from db import get_connection
get_connection().do_stuff(foo, bar, baz)
Another common problem with long-living connections is that the application won't auto-recover if, say, you restart RabbitMQ while your application is running. You'll need to somehow detect dead connections and reconnect.
It looks like you can attach objects to the request with add_request_method.
Here's a little example app using that method to make one and only one connection to a socket on startup, then make the connection available to each request:
from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response
def index(request):
return Response('I have a persistent connection: {} with id {}'.format(
repr(request.conn).replace("<", "<"),
id(request.conn),
))
def add_connection():
import socket
s = socket.socket()
s.connect(("google.com", 80))
print("I should run only once")
def inner(request):
return s
return inner
if __name__ == '__main__':
config = Configurator()
config.add_route('index', '/')
config.add_view(index, route_name='index')
config.add_request_method(add_connection(), 'conn', reify=True)
app = config.make_wsgi_app()
server = make_server('0.0.0.0', 8080, app)
server.serve_forever()
You'll need to be careful about threading / forking in this case though (each thread / process will need its own connection). Also, note that I am not very familiar with pyramid, there may be a better way to do this.
I have a simple Cherrypy web application, including two classes. The init code looks like this:
c = MyClass()
c.updates = AnotherClass()
app = cherrypy.tree.mount(c, '/', 'myapp.config')
c.setConfig(app.config)
c.updates.setConfig(app.config)
cherrypy.engine.start()
cherrypy.engine.block()
The setConfig method for both classes is just a line of code to store some database configuration:
def setConfig(self, conf):
self.config = conf['Database']
The configuration file myapp.config looks like this:
[global]
server.socket_host = "0.0.0.0"
server.socket_port = 80
[/]
tools.staticdir.root = com.stuff.myapp.rootDir + '/html'
[Database]
dbtable: "mydbtable"
username: "user"
password: "pass"
When I start the lot, the application gets the database config data, and correctly serves static files from the /html directory, but it only listens on localhost on 8080. I get this on the console:
[11/Apr/2013:10:03:58] ENGINE Bus STARTING
[11/Apr/2013:10:03:58] ENGINE Started monitor thread 'Autoreloader'.
[11/Apr/2013:10:03:58] ENGINE Started monitor thread '_TimeoutMonitor'.
[11/Apr/2013:10:03:58] ENGINE Serving on 127.0.0.1:8080
[11/Apr/2013:10:03:58] ENGINE Bus STARTED
I definitely must have done something wrong. It's as if the global part of the configuration doesn't get applied. How can I fix it?
I think I figured out how to solve it. I added this line:
cherrypy.config.update('myapp.config')
after the line that says
app = cherrypy.tree.mount(c, '/', 'myapp.config')
I think the reason why my classes were getting the Database configuration is that I pass it manually with the setConfig() calls. This passes the application configuration only, not the global configuration. The mount() call apparently doesn't propagate the configuration data to the objects it mounts, as I thought it would do.
Furthermore, the update() call must be after the mount() call, or an exception is raised.
I'm not sure whether this is the best way to organize this code. This works for now, but better ideas are always welcome.
Is it possible to run Tornado such that it listens to a local port (e.g. localhost:8000). I can't seem to find any documentation explaining how to do this.
Add an address argument to Application.listen() or HTTPServer.listen().
It's documented here (Application.listen) and here (TCPServer.listen).
For example:
application = tornado.web.Application([
(r'/blah', BlahHandler),
], **settings)
# Create an HTTP server listening on localhost, port 8080.
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8080, address='127.0.0.1')
In the documetaion they mention to run on the specific port like
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8000)
tornado.ioloop.IOLoop.instance().start()
You will get more help from http://www.tornadoweb.org/documentation/overview.html and http://www.tornadoweb.org/documentation/index.html
Once you've defined an application (like in the other answers) in a file (for example server.py), you simply save and run that file.
python server.py
If you want to daemonize tornado - use supervisord. If you want to access tornado on address like http://mylocal.dev/ - you should look at nginx and use it like reverse proxy. And on specific port it can be binded like in Lafada's answer.
I am new to Python and Tornado WebServer.
I am trying to figure out the number of request and number of requests/second in my server side code. I am using Tornadio2 to implement websockets.
Kindly take a look at the following code and let me know, what modification can be done to it.
I am using the RequestHandler.prepare() to bottleneck all the requests and using a list as it is immutable to store the count.
Consider all modules are included
count=[0]
class IndexHandler(tornado.web.RequestHandler):
"""Regular HTTP handler to serve the chatroom page"""
def prepare(self):
count[0]=count[0]+1
def get(self):
self.render('index1.html')
class SocketIOHandler(tornado.web.RequestHandler):
def get(self):
self.render('../socket.io.js')
partQue=Queue.Queue()
class ChatConnection(tornadio2.conn.SocketConnection):
participants = set()
def on_open(self, info):
self.send("Welcome from the server.")
self.participants.add(self)
def on_message(self, message):
partQue.put(message)
time.sleep(10)
self.qmes=partQue.get()
for p in self.participants:
p.send(self.qmes+" "+str(count[0]))
partQue.task_done()
def on_close(self):
self.participants.remove(self)
partQue.join()
# Create tornadio server
ChatRouter = tornadio2.router.TornadioRouter(ChatConnection)
# Create socket application
sock_app = tornado.web.Application(
ChatRouter.urls,
flash_policy_port = 843,
flash_policy_file = op.join(ROOT, 'flashpolicy.xml'),
socket_io_port = 8002)
# Create HTTP application
http_app = tornado.web.Application(
[(r"/", IndexHandler), (r"/socket.io.js", SocketIOHandler)])
if __name__ == "__main__":
import logging
logging.getLogger().setLevel(logging.DEBUG)
# Create http server on port 8001
http_server = tornado.httpserver.HTTPServer(http_app)
http_server.listen(8001)
# Create tornadio server on port 8002, but don't start it yet
tornadio2.server.SocketServer(sock_app, auto_start=False)
# Start both servers
tornado.ioloop.IOLoop.instance().start()
Also, I am confused about every Websocket messages. Does each Websocket event got to server in the form of an HTTP request? or a Socket.IO request?
I use Siege - excellent tool for testing requests if your running on linux. Example
siege http://localhost:8000/?q=yourquery -c10 -t10s
-c10 = 10 concurrent users
-t10s = 10 seconds
Tornadio2 has built-in statistics module, which includes incoming connections/s and other counters.
Check following example: https://github.com/MrJoes/tornadio2/tree/master/examples/stats
When testing applications, always approach performance testing with a healthy appreciation for the uncertainty principle..
If you want to test a server, hook up two PCs to a HUB where you can monitor traffic from one going to the other. Then bang the hell out of the server. There are a variety of tools for doing this, just look for web load testing tools.
Normal HTTP requests in Tornado create a new RequestHandler instance, which persists until the connection is terminated.
WebSockets use persistent connections. One WebSocketHandler instance is created, and each message sent by the browser to the server calls the on_message method.
From what I understand, Socket.IO/Tornad.IO will use WebSockets if supported by the browser, falling back to long polling.