I have the following code:
import SimpleHTTPServer
import SocketServer
def http_server():
PORT = 80
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
httpd.serve_forever()
The problem with this is that, because of httpd.serve_forever(), it hangs the rest of the program. I'm assuming I could use threading to run this on its own thread, so the rest of the program can execute independently of the server, but I'm not sure how to implement this.
Simplest way, straight from the docs:
from threading import Thread
t = Thread(target=http_server)
t.start()
Note that this thread will be difficult to kill as-is, KeyboardInterrupts do not propagate to random threads that you've start()ed. You may want to set daemon=True or have some more sophisticated method to shut it down.
Related
Hypothesis:
thread....start() blocks until start completes.
Question:
Is hypothesis True or False?
Start http web server then open browser has the following code.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
This works fine. However, the following also works.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
Hypothesis:
thread....start() actually blocks until start completes. So,webbrowser.open_new(url) does not execute until start completes. Thus making the following unnecessary.
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
I have not been able to prove or disprove the Hypothesis after extensive searching.
There is no blocking when calling Thread.start() in the way you suggest.
The call is blocking in the sense that a call is placed that initalizes the new-thread internal state, and a system call is made to start the actual OS Thread - but that should take less than 1ms. The function that is the target of the thread is only called on the new thread, and the main thread will continue to run, regardless of what takes place inside that function.
If you want your program not to end, there is no need to resort to a complicated pausing loop like the one you setup - just place a call to threading.join() instead. This will block until all other threads end running, and only them the threading calling join() will proceed.
Used this code to run a python server:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
How to make it stop?
Your question is ambiguous - if your running the server via shell i.e. python myscript.py, simply press crtl + C.
If you want to close it elegantly using code, you must decide on some condition, or point, or exception to call it shutdown. You can add a block and call httpd.shutdown() - as HttpServer itself is a SocketServer.TCPSServer subclass:
The first class, HTTPServer, is a SocketServer.TCPServer subclass, and
therefore implements the SocketServer.BaseServer interface. It creates
and listens at the HTTP socket, dispatching the requests to a handler.
So the BaseServer has a method shutdown(), hence being a subclass HttpServer has it too.
for example:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
try:
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
except Exception:
httpd.shutdown()
Helpful relevant question -
How do I shutdown an HTTPServer from inside a request handler in Python?
How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?
You can send a SIGTERM signal from the handler thread if you are ok with killing the whole process:
os.kill(os.getpid(), signal.SIGTERM)
If you need the Python HTTP server in a unit test then it is advisable to run it in a separate thread and stop it from another one, like this:
import unittest
from threading import Thread
from http.server import HTTPServer
class TestWithHTTP(unittest.TestCase):
"""
My unit test that needs a HTTP server
NOTE: skeleton code
"""
def setUp(self):
# you need to provide the host, port and request handler class
self.myserver = HTTPServer((host, port), HandlerClass)
# start HTTP server in another thread
httpthread = Thread(target=self.myserver.serve_forever)
httpthread.start()
# ... any other setup operations ...
def test_something(self):
# ... your unit testing code ...
pass
def tearDown(self):
# shut down the server from yet another thread
killerthread = Thread(target = self.myserver.shutdown)
killerthread.start()
Just use ^C (control+c) to shut down python server.
I have this methods:
class dJobs():
def server(self):
address = ('127.0.0.1', dConfig.cgiport)
handler = CGIHTTPServer.CGIHTTPRequestHandler
handler.cgi_directories = ['/cgi-bin']
self.logger.info("starting http server on port %s" %str(port))
httpd = BaseHTTPServer.HTTPServer(address, handler)
httpd.serve_forever()
def job(self):
self.runNumber = 0
while True:
self.logger.info("Counting: %s" %str(self.runNumber))
self.runNumber+=1
time.sleep(1)
I want run job while waiting for http and cgi requests, handle requests and then continue job method.
Is it possibile to do this using gevent (and how), or i need to use threading ?
i.e. I want to run both method concurrently without creating threads.
This solution seems to work for me:
import monkey:
from gevent import monkey
monkey.patch_all(thread=False)
add and run this method:
def run(self):
jobs = [gevent.spawn(self.server),gevent.spawn(self.job)]
gevent.joinall(jobs)
Please try it out in your program.
If your job is CPU-bound, you must not use Python threads because of GIL. Alternative is multiprocessing module.
Also you could use uWSGI--it can do CGI and run jobs. Look at the WSGI which is the main feature of uWSGI, you may want to use that instead of CGI.
This is what I have:
http.py:
class HTTPServer():
def __init__(self, port):
self.port = port
self.thread = None
self.run = True
def serve(self):
self.thread = threading.Thread(target=self._serve)
self.thread.start()
def _serve(self):
serverAddress = ("", self.port)
self.server = MyBaseHTTPServer(serverAddress,MyRequestHandler)
logging.log(logging.INFO, "HTTP server started on port %s"%self.port)
while self.run:
self.server.handle_request()
def stop(self):
self.run = False
self.server.server_close()
Then in another file, to restart it:
def restartHTTP(self):
try:
self.httpserver.stop()
reload(http)
self.httpserver = http.HTTPServer(80)
self.httpserver.serve()
except:
traceback.print_exc()
This gives me an address already in use error, so it seems the HTTP server isn't stopping properly. What else do I need to do to stop it?
EDIT:
Where I call restartHTTP:
def commandHTTPReload(self, parts, byuser, overriderank):
self.client.factory.restartHTTP()
self.client.sendServerMessage("HTTP server reloaded.")
I do know the command is executing because I get the message it's supposed to send.
You just need to let the OS know that you really do want to reuse the port immediately after closing it. Normally it's held in a closed state for a while, in case any extra packets show up. You do this with SO_REUSEADDR:
mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
..after opening mysocket. A good place to do this with HTTPServer could be in an overridden server_bind method:
def server_bind(self):
HTTPServer.server_bind(self)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Edit: Having looked more closely at your code, I see that your threading model is also likely causing problems here. You're closing the socket in the main(?) thread while the other thread is waiting on a connection on that same socket (in accept()). This arrangement does not have well-defined semantics, and I believe it does different things on different OSes. In any case, it is something you ought to avoid in order to minimize confusion (already lots of that to go around in a multithreaded program). Your old thread will not actually go away until after it gets a connection and handles its request (because it won't re-check self.run until then), and so the port may not be re-bindable until after that.
There isn't really a simple solution to this. You could add a communication pipe between the threads, and then use select()/poll() in the server thread to wait for activity on either of them, or you could timeout the accept() calls after a short amount of time so that self.run gets checked more frequently. Or you could have the main thread connect to the listening socket itself. But whatever you do, you're probably approaching the level of complexity where you ought to look at using a "real" httpd or network framework instead of rolling your own: apache, lighttpd, Tornado, Twisted, etc.
For gracefully stop HTTPServer and close socket one should use:
# Start server
httpd = HTTPServer(...)
httpd.serve_forever()
# Stop server
httpd.shutdown()
httpd.server_close()
I've got an application that runs Twisted by starting the reactor with reactor.run() in my main thread after starting some other threads, including the CherryPy web server. Here's a program that shuts down cleanly when Ctrl+C is pressed on Linux but not on Windows:
from threading import Thread
from signal import signal, SIGINT
import cherrypy
from twisted.internet import reactor
from twisted.web.client import getPage
def stop(signum, frame):
cherrypy.engine.exit()
reactor.callFromThread(reactor.stop)
signal(SIGINT, stop)
class Root:
#cherrypy.expose
def index(self):
reactor.callFromThread(kickoff)
return "Hello World!"
cherrypy.server.socket_host = "0.0.0.0"
Thread(target=cherrypy.quickstart, args=[Root()]).start()
def print_page(html):
print(html)
def kickoff():
getPage("http://acpstats/account/login").addCallback(print_page)
reactor.run()
I believe that CherryPy is the culprit here, because here's a different program that I wrote without CherryPy that does shutdown cleanly on both Linux and Windows when Ctrl+C is pressed:
from time import sleep
from threading import Thread
from signal import signal, SIGINT
from twisted.internet import reactor
from twisted.web.client import getPage
keep_going = True
def stop(signum, frame):
global keep_going
keep_going = False
reactor.callFromThread(reactor.stop)
signal(SIGINT, stop)
def print_page(html):
print(html)
def kickoff():
getPage("http://acpstats/account/login").addCallback(print_page)
def periodic_downloader():
while keep_going:
reactor.callFromThread(kickoff)
sleep(5)
Thread(target=periodic_downloader).start()
reactor.run()
Does anyone have any idea what the problem is? Here's my conundrum:
On Linux everything works
On Windows, I can call functions from signal handlers using reactor.callFromThread when CherryPy is not running
When CherryPy is running, no function that I call using reactor.callFromThread from a signal handler will ever execute (I've verified that the signal handler itself does get called)
What can I do about this? How can I shut down Twisted on Windows from a signal handler while running CherryPy? Is this a bug, or have I simply missed some important part of the documentation for either of these two projects?
CherryPy handles signals by default when you call quickstart. In your case, you should probably just unroll quickstart, which is only a few lines, and pick and choose. Here's basically what quickstart does in trunk:
if config:
cherrypy.config.update(config)
tree.mount(root, script_name, config)
if hasattr(engine, "signal_handler"):
engine.signal_handler.subscribe()
if hasattr(engine, "console_control_handler"):
engine.console_control_handler.subscribe()
engine.start()
engine.block()
In your case, you don't need the signal handlers, so you can omit those. You also don't need to call engine.block if you're not starting CherryPy from the main thread. Engine.block() is just a way to make the main thread not terminate immediately, but instead wait around for process termination (this is so autoreload works reliably; some platforms have issues calling execv from any thread but the main thread).
If you remove the block() call, you don't even need the Thread() around quickstart. So, replace your line:
Thread(target=cherrypy.quickstart, args=[Root()]).start()
with:
cherrypy.tree.mount(Root())
cherrypy.engine.start()