I have a multiproccessing tornado web server and I want to create another process that will do some things in the background.
I have a server with to following code
start_background_process
app = Application([<someurls>])
server = HTTPServer(app)
server.bind(8888)
server.start(4) # Forks multiple sub-processes
IOLoop.current().start()
def start_background_process():
process = multiprocessing.Process(target=somefunc)
process.start()
and everything is working great.
However when I try to close the server (by crtl c or send signal)
I get AssertionError: can only join a child process
I understood the cause of this problem:
when I create a process with multiprocess a call for the process join method
is registered in "atexit" and because tornado does a simple fork all its childs also call the join method of the process I created and the can't since the process is their brother and not their son?
So how can I open a process normally in tornado?
"HTTPTserver start" uses os.fork to fork the 4 sub-processes as it can be seen in its source code.
If you want your method to be executed by all the 4 sub-processes, you have to call it after the processes have been forked.
Having that in mind your code can be changed to look as below:
import multiprocessing
import tornado.web
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
# A simple external handler as an example for completion
from handlers.index import IndexHandler
def method_on_sub_process():
print("Executing in sub-process")
def start_background_process():
process = multiprocessing.Process(target=method_on_sub_process)
process.start()
def main():
app = tornado.web.Application([(r"/", IndexHandler)])
server = HTTPServer(app)
server.bind(8888)
server.start(4)
start_background_process()
IOLoop.current().start()
if __name__ == "__main__":
main()
Furthermore to keep the behavior of your program clean during any keyboard interruption , surround the instantiation of the sever by a try...except clause as below:
def main():
try:
app = tornado.web.Application([(r"/", IndexHandler)])
server = HTTPServer(app)
server.bind(8888)
server.start(4)
start_background_process()
IOLoop.current().start()
except KeyboardInterrupt:
IOLoop.instance().stop()
Related
I'm looking for a possibility to use uvicorn.run() with a FastAPI app but without uvicorn.run() is blocking the thread. I already tried to use processes, subprocessesand threads but nothing worked.
My problem is that I want to start the Server from another process that should go on with other tasks after starting the server. Additinally I have problems closing the server like this from another process.
Has anyone an idea how to use uvicorn.run() non blocking and how to stop it from another process?
Approach given by #HadiAlqattan will not work because uvicorn.run expects to be run in the main thread. Errors such as signal only works in main thread will be raised.
Correct approach is:
import contextlib
import time
import threading
import uvicorn
class Server(uvicorn.Server):
def install_signal_handlers(self):
pass
#contextlib.contextmanager
def run_in_thread(self):
thread = threading.Thread(target=self.run)
thread.start()
try:
while not self.started:
time.sleep(1e-3)
yield
finally:
self.should_exit = True
thread.join()
config = uvicorn.Config("example:app", host="127.0.0.1", port=5000, log_level="info")
server = Server(config=config)
with server.run_in_thread():
# Server is started.
...
# Server will be stopped once code put here is completed
...
# Server stopped.
Very handy to run a live test server locally using a pytest fixture:
# conftest.py
import pytest
#pytest.fixture(scope="session")
def server():
server = ...
with server.run_in_thread():
yield
Credits: uvicorn#742 by florimondmanca
This is an alternate version which works and was inspired by Aponace uvicorn#1103. The uvicorn maintainers want more community engagement with this issue, so if you are experiencing it, please join the conversation.
Example conftest.py file.
import pytest
from fastapi.testclient import TestClient
from app.main import app
import multiprocessing
from uvicorn import Config, Server
class UvicornServer(multiprocessing.Process):
def __init__(self, config: Config):
super().__init__()
self.server = Server(config=config)
self.config = config
def stop(self):
self.terminate()
def run(self, *args, **kwargs):
self.server.run()
#pytest.fixture(scope="session")
def server():
config = Config("app.main:app", host="127.0.0.1", port=5000, log_level="debug")
instance = UvicornServer(config=config)
instance.start()
yield instance
instance.stop()
#pytest.fixture(scope="module")
def mock_app(server):
client = TestClient(app)
yield client
Example test_app.py file.
def test_root(mock_app):
response = mock_app.get("")
assert response.status_code == 200
When I set reload to False, fastapi will start a multi-process web service. If it is true, there will only be one process for the web service
import uvicorn
from fastapi import FastAPI, APIRouter
from multiprocessing import cpu_count
import os
router = APIRouter()
app = FastAPI()
#router.post("/test")
async def detect_img():
print("pid:{}".format(os.getpid()))
return os.getpid
if __name__ == '__main__':
app.include_router(router)
print("cpu个数：{}".format(cpu_count()))
workers = 2*cpu_count() + 1
print("workers:{}".format(workers))
reload = False
#reload = True
uvicorn.run("__main__:app", host="0.0.0.0", port=8082, reload=reload, workers=workers, timeout_keep_alive=5,
limit_concurrency=100)
According to Uvicorn documentation there is no programmatically way to stop the server.
instead, you can stop the server only by pressing ctrl + c (officially).
But I have a trick to solve this problem programmatically using multiprocessing standard lib with these three simple functions :
A run function to run the server.
A start function to start a new process (start the server).
A stop function to join the process (stop the server).
from multiprocessing import Process
import uvicorn
# global process variable
proc = None
def run():
"""
This function to run configured uvicorn server.
"""
uvicorn.run(app=app, host=host, port=port)
def start():
"""
This function to start a new process (start the server).
"""
global proc
# create process instance and set the target to run function.
# use daemon mode to stop the process whenever the program stopped.
proc = Process(target=run, args=(), daemon=True)
proc.start()
def stop():
"""
This function to join (stop) the process (stop the server).
"""
global proc
# check if the process is not None
if proc:
# join (stop) the process with a timeout setten to 0.25 seconds.
# using timeout (the optional arg) is too important in order to
# enforce the server to stop.
proc.join(0.25)
With the same idea you can :
use threading standard lib instead of using multiprocessing standard lib.
refactor these functions into a class.
Example of usage :
from time import sleep
if __name__ == "__main__":
# to start the server call start function.
start()
# run some codes ....
# to stop the server call stop function.
stop()
You can read more about :
Uvicorn server.
multiprocessing standard lib.
threading standard lib.
Concurrency to know more about multi processing and threading in python.
Using pywin32 and gevent, I'm creating a Windows service that serves two functions:
It runs a web server for a simple web application (using bottle's gevent server adapter, which runs WSGIServer's serve_forever()).
It listens for incoming SIP calls (using a gevent-based SIP client) and runs some simple code to respond to calls.
I'd like the service to just keep the web server and SIP client running forever, but stop both immediately and gracefully if I try to stop the Windows service. This seems like it should be pretty simple.
What I'm currently doing to run the app is basically to run the web server and SIP client each in a greenlet, and run kill on both greenlets when I want to stop the app (simplified mockup):
from bottle import run
from mysipclient import SipClient
import gevent
def sip_listen():
client = SipClient()
try:
client.wait() # This method blocks on a gevent queue.get call
finally:
client.close() # This does some cleanup, like deregistering from the SIP server, that I really want to run when the service stops!
class App(object):
def start(self):
self.stop_event = gevent.event.Event()
self.server_greenlet = gevent.spawn(run, server='gevent', host='0.0.0.0', port=8080)
self.sip_greenlet = gevent.spawn(sip_listen)
gevent.joinall([self.server_greenlet, self.sip_greenlet])
def stop(self):
self.server_greenlet.kill()
self.sip_greenlet.kill()
if __name__ == "__main__":
app = App()
gevent.spawn_later(10, app.stop)
app.start()
If I run this from the command line, it works great: it starts the app with both greenlets working, then ten seconds later it shuts itself down, running the cleanup code for the SIP client and all.
Now, though, I try to make this into a Windows service, using pywin32's win32serviceutil:
import win32serviceutil
import win32service
import win32event
import servicemanager
from app import App
class TestService(win32serviceutil.ServiceFramework):
_svc_name_ = 'TestService'
_svc_display_name_ = 'TestService'
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.service_obj = App()
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
self.service_obj.stop()
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE, servicemanager.PYS_SERVICE_STARTED, (self._svc_name_, ''))
self.service_obj.start()
if __name__ == '__main__':
if len(sys.argv) == 1:
servicemanager.Initialize()
servicemanager.PrepareToHostSingle(TestService)
servicemanager.StartServiceCtrlDispatcher()
else:
win32serviceutil.HandleCommandLine(TestService)
When I install this as a service and run it, I get an exception when trying to kill the greenlets: LoopExit: This operation would block forever. Then the service fails to stop, and I have to kill it manually. (I can avoid this by catching the exception, using an event instead of joining the greenlets and setting the event on stop - but this means the cleanup doesn't run at all.)
I'm pretty new to both gevent and working with Windows services, and Google hasn't been terribly helpful. I thought maybe the difference was about how with the command line version I was running stop in another greenlet, so I tried replacing self.service_obj.stop() in SvcStop with gevent.spawn(self.service_obj.stop).join(), but that way it doesn't even throw the exception and just hangs completely until I kill the process.
What's going on here? Am I doing something fundamentally wrong? How do I stop the greenlets gracefully on SvcStop?
You need to trap the signal event
from gevent import signal
def shutdown():
print('Shutting down ...')
server.stop(timeout=60)
exit(signal.SIGTERM)
gevent.signal(signal.SIGTERM, shutdown)
gevent.signal(signal.SIGQUIT, shutdown)
gevent.signal(signal.SIGINT, shutdown) #CTRL C
... start server code
I am running a Python 3.7 Flask application which uses flask_socketio to setup a socketio server for browser clients, another python process to connect to a separate remote socketio server & exchange messages, and another python process to read input from a PIR sensor.
Both python processes communicate over multiprocessing.Queue - but, the socketio process always gets either [WinError6] - Invalid Handle or [WinError5] - Permission Denied. I have absolutely no idea what I'm doing wrong.
Here's the top-level (server) code; it does not appear to have issues:
from shotsocket import init as shotsocket_init
from shotsocket import util as matchmaking_util
import multiprocessing, os, config, uuid
match_queue = multiprocessing.Queue()
shot_queue = multiprocessing.Queue()
app = Flask(__name__, static_url_path='', static_folder='templates')
socketio = SocketIO(app)
_rooms = [] # I don't plan to keep this in memory, just doing it for debug / dev
...
The above works fine and dandy. The 2nd to last line in the following block is the issue.
# THIS IS THE FUNC WHERE WE ARE TRYING TO USE
# THE BROKEN QUEUE
#socketio.on('connect')
def listen():
room_key = str(uuid.uuid4())
join_room(room_key)
_rooms.append((room_key, request.sid))
possible_match = matchmaking_util.match_pending_clients(_rooms)
if possible_match:
shot_queue.put_nowait(possible_match)
print('put it in there')
Here's how I start these processes:
if __name__ == '__main__':
debug = os.environ.get('MOONSHOT_DEBUG', False)
try:
proc = multiprocessing.Process(target=start, args=(debug,match_queue))
proc.start()
shot_proc = multiprocessing.Process(target=shotsocket_init, args=(shot_queue,))
shot_proc.start()
socketio.run(app, host='0.0.0.0')
except KeyboardInterrupt:
socketio.stop()
proc.join()
shot_proc.join()
And here's the entirety of shotsocket (the code that cannot read the queue)
import socketio, multiprocessing # mp for the type
sio = socketio.Client(engineio_logger=True)
sio.connect('redacted woot', transports=['websocket'])
#sio.on('connect')
def connect():
print("connected to shot server")
def init(queue: multiprocessing.Queue):
while True:
try:
# WE NEVER GET PAST THIS LINE
print(queue.get())
except Exception as e:
continue
if not queue.empty():
print('queue empty')
shot = queue.get()
print(shot)
match_id, opponents = shot
sio.emit('start', {'id': match_id, 'opponents': [opponents[0], opponents[1]]})
I'm pulling my hair out. What the heck am I doing wrong?
Solution
I have no idea why this fixes the problem, but switching from multiprocessing.Queue to queue.Queue and multiprocessing.Process to threading.Thread did it.
How to listen for events that happen on stdin in Tornado loop?
In particular, in a tornado-system, I want to read from stdin, react on it, and terminate if stdin closes. At the same time, the Tornado web service is running on the same process.
While looking for this, the most similar I could find was handling streams of an externally spawned process. However, this is not what I want: I want to handle i/o stream of the current process, i.e. the one that has the web server.
Structurally, my server is pretty much hello-world tornado, so we can base the example off that. I just need to add an stdin handler.
You can use the add_handler method on the IOLoop instance to watch for events on stdin.
Here's a minimal working example:
from tornado.ioloop import IOLoop
from tornado.web import Application, RequestHandler
import sys
class MainHandler(RequestHandler):
def get(self):
self.finish("foo")
application = Application([
(r"/", MainHandler),
])
def on_stdin(fd, events):
content = fd.readline()
print "received: %s" % content
if __name__ == "__main__":
application.listen(8888)
IOLoop.instance().add_handler(sys.stdin, on_stdin, IOLoop.READ)
IOLoop.instance().start()
Let's take the hello world application in the Tornado home page:
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
Is there a way, after the IOloop has been started and without stopping it, to essentially stop the application and start another one (on the same port or on another)?
I saw that I can add new application (listening on different ports) at runtime, but I do not know how I could stop existing ones.
Application.listen() method actually creates a HTTPServer and calls its listen() medthod. HTTPServer objects has stop() method which is probably what you need. But in order to do it you have to explicitly create HTTPServer object in your script.
server = HTTPServer(application)
server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
#somewhere in your code
server.stop()
Here is a gist about how to gracefully and safely shutdown the tornado ioloop.
https://gist.github.com/nicky-zs/6304878
However, you can refer to this implementation to achieve your goal.
To add to #Alex Shkop's answer a few years later, as of Tornado 4.3 .listen() returns a reference to its HTTPServer!
https://www.tornadoweb.org/en/stable/web.html#tornado.web.Application.listen
server = app.listen()
... # later
server.stop()
Further, if you're working in a Jupyter notebook and for some reason need a Tornado server, you can try to close the HTTPServer before you recreate it to avoid OSError: [Errno 98] Address already in use on re-running the cell
# some Jupyter cell
#
import tornado.web
try:
server.stop() # NameError on first cell run
except Exception as ex:
print(f"server not started to stop: {repr(ex)}")
else: # did not raise NameError: server was running
print(f"successfully stopped server: {server}")
app = tornado.web.Application(...)
server = app.listen(9006) # arbitrary listening port