Reference: http://docs.python.org/2/library/basehttpserver.html
I have the following code snippet which uses Python BaseHTTPServer to run a basic HTTP server.
from BaseHTTPServer import HTTPServer
from BaseHTTPServer import BaseHTTPRequestHandler
# http request handler
class HttpHandler(BaseHTTPRequestHandler):
def do_POST(self):
print "I have just received a HTTP request through POST"
try:
server = HTTPServer((<ip>, <port>), HttpHandler)
# wait forever for incoming http requests!
server.serve_forever()
except KeyboardInterrupt:
server.socket.close()
What I am looking for is a way to get a callback whenever the http server is started/stopped using server.serve_forever()/server.socket.close() methods.
Say we have the following functions:
def http_server_start_callback():
print "http server has just been started"
def http_server_stop_callback():
print "http server has just been stopped"
I want http_server_start_callback function to be called right after (whenever) I start the server i.e. server.serve_forever() and I want http_server_stop_callback function to be called right after (whenever) I stop the server i.e. server.socket.close()
It would be excellent to configure the http server with the following callbacks:
before starting the server
after starting the server
before stopping the server
after stopping the server
Is there a way to setup these callbacks in Python BaseHTTPServer.HTTPServer?!
It would be excellent to configure the http server with the following
callbacks:
before starting the server
after starting the server
before stopping the server
after stopping the server
Bear in mind that the OS will start accepting and queuing TCP connections the moment the socket starts listening, which is done in the constructor of BaseHTTPServer, so if you want to perform lengthy tasks before starting the server, it's probably better to do them before the OS starts accepting connections.
There's a server_activate() method which makes the call to socket.listen(), so it's probably best to override that.
Similarly, the OS will continue to accept connections until the call to socket.close(), so if you want to be able to define a 'pre-stop' handler which has the capacity to prevent itself from being shutdown, it's probably better to use the server_close() method, rather than calling socket.close() directly.
I've put together a simple example, using class methods on the request handler to handle the four new events, although you can move them somewhere else...
from BaseHTTPServer import HTTPServer
from BaseHTTPServer import BaseHTTPRequestHandler
# Subclass HTTPServer with some additional callbacks
class CallbackHTTPServer(HTTPServer):
def server_activate(self):
self.RequestHandlerClass.pre_start()
HTTPServer.server_activate(self)
self.RequestHandlerClass.post_start()
def server_close(self):
self.RequestHandlerClass.pre_stop()
HTTPServer.server_close(self)
self.RequestHandlerClass.post_stop()
# HTTP request handler
class HttpHandler(BaseHTTPRequestHandler):
#classmethod
def pre_start(cls):
print 'Before calling socket.listen()'
#classmethod
def post_start(cls):
print 'After calling socket.listen()'
#classmethod
def pre_stop(cls):
print 'Before calling socket.close()'
#classmethod
def post_stop(cls):
print 'After calling socket.close()'
def do_POST(self):
print "I have just received an HTTP POST request"
def main():
# Create server
try:
print "Creating server"
server = CallbackHTTPServer(('', 8000), HttpHandler)
except KeyboardInterrupt:
print "Server creation aborted"
return
# Start serving
try:
print "Calling serve_forever()"
server.serve_forever()
except KeyboardInterrupt:
print "Calling server.server_close()"
server.server_close()
if __name__ == '__main__':
main()
Note that I've also moved the call to the constructor into its own try...except block, since the server variable won't exist if you hit CTRL-C during its construction.
You have to subclass HTTPServer and use your class instead HTTPServer
from __future__ import print_function
from BaseHTTPServer import HTTPServer
from BaseHTTPServer import BaseHTTPRequestHandler
class MyHTTPServer(HTTPServer):
def __init__(self, *args, **kwargs):
self.on_before_serve = kwargs.pop('on_before_serve', None)
HTTPServer.__init__(self, *args, **kwargs)
def serve_forever(self, poll_interval=0.5):
if self.on_before_serve:
self.on_before_serve(self)
HTTPServer.serve_forever(self, poll_interval)
# http request handler
class HttpHandler(BaseHTTPRequestHandler):
def do_POST(self):
print("I have just received a HTTP request through POST")
try:
server = MyHTTPServer(('0.0.0.0', 8080), HttpHandler,
on_before_serve = lambda server: print('Server will start to serve in the moment...'))
# wait forever for incoming http requests!
server.serve_forever()
except KeyboardInterrupt:
server.socket.close()
Related
I created an http server. My task is to make sure that each new post request to the server is processed in a separate thread.I tried using socketserver.ThreadingTCPServer(),but this way I can't control the threads. I need to start a thread only after the previous one is completed or after 5 minutes have passed.How can I implement the server differently, but only so that requests are processed in a separate request and each new thread is processed after the previous one has completed?
from http.server import HTTPServer, BaseHTTPRequestHandler, ThreadingHTTPServer
import socketserver
import threading
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
self.send_response(200)
self.end_headers()
query_args = parse_qs(urlparse(self.path).query)
print(query_args)
if __name__ == "__main__":
try:
with socketserver.ThreadingTCPServer(("", 8080), Handler) as httpd:
while True:
httpd.serve_forever()
except Exception as e:
httpd.server_close()
Used this code to run a python server:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
How to make it stop?
Your question is ambiguous - if your running the server via shell i.e. python myscript.py, simply press crtl + C.
If you want to close it elegantly using code, you must decide on some condition, or point, or exception to call it shutdown. You can add a block and call httpd.shutdown() - as HttpServer itself is a SocketServer.TCPSServer subclass:
The first class, HTTPServer, is a SocketServer.TCPServer subclass, and
therefore implements the SocketServer.BaseServer interface. It creates
and listens at the HTTP socket, dispatching the requests to a handler.
So the BaseServer has a method shutdown(), hence being a subclass HttpServer has it too.
for example:
import os
from http.server import SimpleHTTPRequestHandler, HTTPServer
os.chdir('c:/users/owner/desktop/tom/tomsEnyo2.5-May27')
server_address = ('', 8000)
try:
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
except Exception:
httpd.shutdown()
Helpful relevant question -
How do I shutdown an HTTPServer from inside a request handler in Python?
How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?
You can send a SIGTERM signal from the handler thread if you are ok with killing the whole process:
os.kill(os.getpid(), signal.SIGTERM)
If you need the Python HTTP server in a unit test then it is advisable to run it in a separate thread and stop it from another one, like this:
import unittest
from threading import Thread
from http.server import HTTPServer
class TestWithHTTP(unittest.TestCase):
"""
My unit test that needs a HTTP server
NOTE: skeleton code
"""
def setUp(self):
# you need to provide the host, port and request handler class
self.myserver = HTTPServer((host, port), HandlerClass)
# start HTTP server in another thread
httpthread = Thread(target=self.myserver.serve_forever)
httpthread.start()
# ... any other setup operations ...
def test_something(self):
# ... your unit testing code ...
pass
def tearDown(self):
# shut down the server from yet another thread
killerthread = Thread(target = self.myserver.shutdown)
killerthread.start()
Just use ^C (control+c) to shut down python server.
Hello! I have this code:
from twisted.web import proxy, http
from twisted.internet import reactor
class akaProxy(proxy.Proxy):
"""
Local proxy = bridge between browser and web application
"""
def dataReceived(self, data):
print "Received data..."
headers = data.split("\n")
request = headers[0].split(" ")
method = request[0].lower()
action = request[1]
print action
print "ended content manipulation"
return proxy.Proxy.dataReceived(self, data)
class ProxyFactory(http.HTTPFactory):
protocol = akaProxy
def intercept(port):
print "Intercept"
try:
factory = ProxyFactory()
reactor.listenTCP(port, factory)
reactor.run()
except Exception as excp:
print str(excp)
intercept(1337)
I use above code to intercept everything between browser and web site. When using above, I configure my browser settings: to IP: 127.0.0.1 and Port: 1337. I put this script in remote server to act my remote server as proxy server. But when I change browser proxy IP settings to my server's it does not work. What I do wrong? What else I need to configure?
Presumably your dataReceived is raising an exception during its attempts to parse the data passed to it. Try enabling logging so you can see more of what's going on:
from twisted.python.log import startLogging
from sys import stdout
startLogging(stdout)
The reason your parser is likely to raise exceptions is that dataReceived is not called only with a complete request. It is called with whatever bytes are read from the TCP connection. This may be a complete request, a partial request, or even two requests (if pipelining is in use).
dataReceived in the Proxy context is handling "translation of rawData into lines", so it may be too early for trying your manipulation code. You can try overriding allContentReceived instead and you will have access to the complete headers and content. Here is an example that I believe does what you are after:
#!/usr/bin/env python
from twisted.web import proxy, http
class SnifferProxy(proxy.Proxy):
"""
Local proxy = bridge between browser and web application
"""
def allContentReceived(self):
print "Received data..."
print "method = %s" % self._command
print "action = %s" % self._path
print "ended content manipulation\n\n"
return proxy.Proxy.allContentReceived(self)
class ProxyFactory(http.HTTPFactory):
protocol = SnifferProxy
if __name__ == "__main__":
from twisted.internet import reactor
reactor.listenTCP(8080, ProxyFactory())
reactor.run()
I've created a class for a server with the declaration:
class myServer(socketserver.BaseRequestHandler):
def handle(self):
pass
And started it with:
someServer = socketserver.UDPServer((HOST, PORT), myServer)
someServer.serve_forever()
My question is: how can I get the server to shutdown itself? I've seen it has a base class (of a base class) called BaseServer with a shutdown method. It can be called on someServer with someServer.shutdown() but this is from the outside of the server itself.
By using threads. Serving by one thread and going via another after your timeout.
Consider this working example. Modify it for your UDPServer
import threading
import time
import SimpleHTTPServer
import SocketServer
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
def worker():
# minimal web server. serves files relative to the
print "serving at port", PORT
httpd.serve_forever()
def my_service():
time.sleep(3)
print "I am going down"
httpd.shutdown()
h = threading.Thread(name='httpd', target=worker)
t = threading.Thread(name='timer', target=my_service)
h.start()
t.start()
You could use twisted. Its about the best networking lib for python, here is an example of a UDP server here is (taken from the twisted documentation) the simplest UDP server ever written;
#!/usr/bin/env python
# Copyright (c) 2001-2009 Twisted Matrix Laboratories.
# See LICENSE for details.
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
# Here's a UDP version of the simplest possible protocol
class EchoUDP(DatagramProtocol):
def datagramReceived(self, datagram, address):
self.transport.write(datagram, address)
def main():
reactor.listenUDP(8000, EchoUDP())
reactor.run()
if __name__ == '__main__':
main()
You can then close this down by calling self.transport.loseConnection() When you are ready or a specific event happens.
The server instance is available as self.server in the handler class. So you can call self.server.shutdown() in the handle method.
I'm writing a small web server in Python, using BaseHTTPServer and a custom subclass of BaseHTTPServer.BaseHTTPRequestHandler. Is it possible to make this listen on more than one port?
What I'm doing now:
class MyRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def doGET
[...]
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
pass
server = ThreadingHTTPServer(('localhost', 80), MyRequestHandler)
server.serve_forever()
Sure; just start two different servers on two different ports in two different threads that each use the same handler. Here's a complete, working example that I just wrote and tested. If you run this code then you'll be able to get a Hello World webpage at both http://localhost:1111/ and http://localhost:2222/
from threading import Thread
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/plain")
self.end_headers()
self.wfile.write("Hello World!")
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
daemon_threads = True
def serve_on_port(port):
server = ThreadingHTTPServer(("localhost",port), Handler)
server.serve_forever()
Thread(target=serve_on_port, args=[1111]).start()
serve_on_port(2222)
update:
This also works with Python 3 but three lines need to be slightly changed:
from socketserver import ThreadingMixIn
from http.server import HTTPServer, BaseHTTPRequestHandler
and
self.wfile.write(bytes("Hello World!", "utf-8"))
Not easily. You could have two ThreadingHTTPServer instances, write your own serve_forever() function (don't worry it's not a complicated function).
The existing function:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()
So our replacement would be something like:
def serve_forever(server1,server2):
while True:
r,w,e = select.select([server1,server2],[],[],0)
if server1 in r:
server1.handle_request()
if server2 in r:
server2.handle_request()
I would say that threading for something this simple is overkill. You're better off using some form of asynchronous programming.
Here is an example using Twisted:
from twisted.internet import reactor
from twisted.web import resource, server
class MyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
return 'gotten'
site = server.Site(MyResource())
reactor.listenTCP(8000, site)
reactor.listenTCP(8001, site)
reactor.run()
I also thinks it looks a lot cleaner to have each port be handled in the same way, instead of having the main thread handle one port and an additional thread handle the other. Arguably that can be fixed in the thread example, but then you're using three threads.