Develop Reference

Serve single HTTP request without blocking

I am writing a script which involves opening an HTTP server and serving a single file. However, the request for this file is also instigated from further down the script. Currently, I am doing it like this:
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", 8000), Handler)
Thread(target=httpd.handle_request).start()
This works to handle a single request, but also creates some issues with keyboard input. What is the most efficient, non-blocking way to serve a single HTTP request? Ideally the server would close and release the port upon the completion of the request.

You can try many workarounds but flask is the way to go. It is not the simplest or fastest solution but it is the most relieble one.
Example for serving a single file with flask:
from flask import Flask, send_file
app = Flask(__name__)
#app.route('/file-downloads/')
def file_downloads():
try:
return render_template('downloads.html')
except Exception as e:
return str(e)
app.run()
for a non blocking solution you can do this instead of app.run():
Thread(target=app.run).start()
But I don't recommend running the flask app in a thread because of the GIL

You can use the handle_request method to handle a single request, and if you use the server inside a with statement then Python will close the server and release the port when the statement exits. (Alternatively, you can use the server_close method to close the server and release the port if you want, but the with statement provides better error handling.) If you do all of that in a separate thread, you should get the behaviour you are looking for.
Using Python 3:
from threading import Thread
from http.server import HTTPServer, SimpleHTTPRequestHandler
def serve_one_request():
with HTTPServer(("0.0.0.0", 8000), SimpleHTTPRequestHandler) as server:
server.handle_request()
thread = Thread(target=serve_one_request)
thread.start()
# Do other work
thread.join()
I'm not sure if this will fix the issues with keyboard input you mentioned. If you elaborate on that some more I will take a look.

How to shut down python server

Used this code to run a python server:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
How to make it stop?

Your question is ambiguous - if your running the server via shell i.e. python myscript.py, simply press crtl + C.
If you want to close it elegantly using code, you must decide on some condition, or point, or exception to call it shutdown. You can add a block and call httpd.shutdown() - as HttpServer itself is a SocketServer.TCPSServer subclass:
The first class, HTTPServer, is a SocketServer.TCPServer subclass, and
therefore implements the SocketServer.BaseServer interface. It creates
and listens at the HTTP socket, dispatching the requests to a handler.
So the BaseServer has a method shutdown(), hence being a subclass HttpServer has it too.
for example:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
try:
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
except Exception:
httpd.shutdown()
Helpful relevant question -
How do I shutdown an HTTPServer from inside a request handler in Python?
How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?

You can send a SIGTERM signal from the handler thread if you are ok with killing the whole process:
os.kill(os.getpid(), signal.SIGTERM)

If you need the Python HTTP server in a unit test then it is advisable to run it in a separate thread and stop it from another one, like this:
import unittest
from threading import Thread
from http.server import HTTPServer
class TestWithHTTP(unittest.TestCase):
"""
My unit test that needs a HTTP server
NOTE: skeleton code
"""
def setUp(self):
# you need to provide the host, port and request handler class
self.myserver = HTTPServer((host, port), HandlerClass)
# start HTTP server in another thread
httpthread = Thread(target=self.myserver.serve_forever)
httpthread.start()
# ... any other setup operations ...
def test_something(self):
# ... your unit testing code ...
pass
def tearDown(self):
# shut down the server from yet another thread
killerthread = Thread(target = self.myserver.shutdown)
killerthread.start()

Just use ^C (control+c) to shut down python server.

Python CGI simultaneous AJAX requests

So here's the deal : I'm writing a simple lightweight IRC app, hosted locally, that basically does the same job as Xchat and works in your browser, just as Sabnzbd. I display search results in the browser as an html table, and using an AJAX GET request with an on_click event, the download is launched. I use another AJAX GET request in a 1 second loop to request the download information (status, progress, speed, ETA, etc.). I hit a bump with the simultaneous AJAX requests, since my CGI handler seems to only be able to handle one thread at a time : indeed, the main thread processes the download, while requests for download status are sent too.
Since I had a Django app somewhere, I tried implementing this IRC app and everything works fine. Simultaneous requests are handled properly.
So is there something I have to know with the HTTP handler ? Is it not possible with the basic CGI handle to deal with simultaneous requests ?
I use the following for my CGI IRC app :
from http.server import BaseHTTPRequestHandler, HTTPServer, CGIHTTPRequestHandler
If it's not about theory but about my code, I can gladly post various python scripts if it helps.

A little bit deeper into the documentation:
These four classes process requests synchronously; each request must be completed before the next request can be started.
TL;DR: Use a real web server.

So, after further research, here's my code, whick works :
from http.server import BaseHTTPRequestHandler, HTTPServer, CGIHTTPRequestHandler
from socketserver import ThreadingMixIn
import threading
import cgitb; cgitb.enable() ## This line enables CGI error reporting
import webbrowser
class HTTPRequestHandler(CGIHTTPRequestHandler):
"""Handle requests in a separate thread."""
def do_GET(self):
if "shutdown" in self.path:
self.send_head()
print ("shutdown")
server.stop()
else:
self.send_head()
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
allow_reuse_address = True
daemon_threads = True
def shutdown(self):
self.socket.close()
HTTPServer.shutdown(self)
class SimpleHttpServer():
def __init__(self, ip, port):
self.server = ThreadedHTTPServer((ip,port), HTTPRequestHandler)
self.status = 1
def start(self):
self.server_thread = threading.Thread(target=self.server.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def waitForThread(self):
self.server_thread.join()
def stop(self):
self.server.shutdown()
self.waitForThread()
if __name__=='__main__':
HTTPRequestHandler.cgi_directories = ["/", "/ircapp"]
server = SimpleHttpServer('localhost', 8020)
print ('HTTP Server Running...........')
webbrowser.open_new_tab('http://localhost:8020/ircapp/search.py')
server.start()
server.waitForThread()

Threading/Multiprocessing in Python

I have the following code:
import SimpleHTTPServer
import SocketServer
def http_server():
PORT = 80
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
httpd.serve_forever()
The problem with this is that, because of httpd.serve_forever(), it hangs the rest of the program. I'm assuming I could use threading to run this on its own thread, so the rest of the program can execute independently of the server, but I'm not sure how to implement this.

Simplest way, straight from the docs:
from threading import Thread
t = Thread(target=http_server)
t.start()
Note that this thread will be difficult to kill as-is, KeyboardInterrupts do not propagate to random threads that you've start()ed. You may want to set daemon=True or have some more sophisticated method to shut it down.

How do I write a python HTTP server to listen on multiple ports?

I'm writing a small web server in Python, using BaseHTTPServer and a custom subclass of BaseHTTPServer.BaseHTTPRequestHandler. Is it possible to make this listen on more than one port?
What I'm doing now:
class MyRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def doGET
[...]
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
pass
server = ThreadingHTTPServer(('localhost', 80), MyRequestHandler)
server.serve_forever()

Sure; just start two different servers on two different ports in two different threads that each use the same handler. Here's a complete, working example that I just wrote and tested. If you run this code then you'll be able to get a Hello World webpage at both http://localhost:1111/ and http://localhost:2222/
from threading import Thread
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/plain")
self.end_headers()
self.wfile.write("Hello World!")
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
daemon_threads = True
def serve_on_port(port):
server = ThreadingHTTPServer(("localhost",port), Handler)
server.serve_forever()
Thread(target=serve_on_port, args=[1111]).start()
serve_on_port(2222)
update:
This also works with Python 3 but three lines need to be slightly changed:
from socketserver import ThreadingMixIn
from http.server import HTTPServer, BaseHTTPRequestHandler
and
self.wfile.write(bytes("Hello World!", "utf-8"))

Not easily. You could have two ThreadingHTTPServer instances, write your own serve_forever() function (don't worry it's not a complicated function).
The existing function:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()
So our replacement would be something like:
def serve_forever(server1,server2):
while True:
r,w,e = select.select([server1,server2],[],[],0)
if server1 in r:
server1.handle_request()
if server2 in r:
server2.handle_request()

I would say that threading for something this simple is overkill. You're better off using some form of asynchronous programming.
Here is an example using Twisted:
from twisted.internet import reactor
from twisted.web import resource, server
class MyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
return 'gotten'
site = server.Site(MyResource())
reactor.listenTCP(8000, site)
reactor.listenTCP(8001, site)
reactor.run()
I also thinks it looks a lot cleaner to have each port be handled in the same way, instead of having the main thread handle one port and an additional thread handle the other. Arguably that can be fixed in the thread example, but then you're using three threads.