I am new to python and wrote a simple httpserver in python. I am trying to shut down the server from the request to the server. How can I achieve this functionality of calling a function of the server from the handler?
class MyHandler(SimpleHTTPRequestHandler):
def do_GET(self):
if self.path == '/shutdown':
pass # I want to call MainServer.shutdown from here
class MainServer()
def __init__(self, port = 8123):
self._server = HTTPServer(('0.0.0.0', port), MyHandler)
self._thread = threading.Thread(target = self._server.serve_forever)
self._thread.deamon = True
def start(self):
self._thread.start()
def shut_down(self):
self._thread.close()
In short, do not use server.serve_forver(..). The request handler has a self.server attribute that you can use to communicate with the main server instance to set some sort of flag that tell the server when to stop.
import threading
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/shutdown':
self.server.running = False
class MainServer:
def __init__(self, port = 8123):
self._server = HTTPServer(('0.0.0.0', port), MyHandler)
self._thread = threading.Thread(target=self.run)
self._thread.deamon = True
def run(self):
self._server.running = True
while self._server.running:
self._server.handle_request()
def start(self):
self._thread.start()
def shut_down(self):
self._thread.close()
m = MainServer()
m.start()
The server is normally accessible from the handler through the server attribute. A HTTPServer that was started with server_forerver can be shut down with its... shutdown() method. Unfortunately, even if it is not documented, you cannot call shutdown from the thread that runs the server loop because it causes a deadlock. So you could write this in your do_GET handler method:
def do_GET(self):
# send something to requester...
if self.path == '/shutdown':
t = threading.Thread(target = self.server.shutdown)
t.daemon = True
t.start()
This will cleanly let the thread to terminate, and you should also use it as you server shut_down method, because Python threads cannot be closed abruptly:
def shut_down(self):
self._server.shutdown()
Related
I am implementing a Server class in CPython 3.7 on Windows 10 with a Server.serve method that starts the serving forever and a Server.shutdown method that stops the serving. I need to run multiple server instances in subprocesses.
Running a server instance in a subthread stops the instance as expected:
import threading
import time
class Server:
def __init__(self):
self.shutdown_request = False
def serve(self):
print("serving")
while not self.shutdown_request:
print("hello")
time.sleep(1)
print("done")
def shutdown(self):
print("stopping")
self.shutdown_request = True
if __name__ == "__main__":
server = Server()
threading.Thread(target=server.serve).start()
time.sleep(5)
server.shutdown()
However running a server instance in a subprocess does not stop the instance, unexpectedly:
import multiprocessing
import time
class Server:
def __init__(self):
self.shutdown_request = False
def serve(self):
print("serving")
while not self.shutdown_request:
print("hello")
time.sleep(1)
print("done")
def shutdown(self):
print("stopping")
self.shutdown_request = True
if __name__ == "__main__":
server = Server()
multiprocessing.Process(target=server.serve).start()
time.sleep(5)
server.shutdown()
I suspect that in the multiprocessing case, the self.shutdown_request attribute is not shared between the parent process and the subprocess, and therefore the server.shutdown() call does not affect the running server instance in the subprocess.
I know I could solve this with multiprocessing.Event:
import multiprocessing
import time
class Server:
def __init__(self, shutdown_event):
self.shutdown_event = shutdown_event
def serve(self):
print("serving")
while not self.shutdown_event.is_set():
print("hello")
time.sleep(1)
print("done")
if __name__ == "__main__":
shutdown_event = multiprocessing.Event()
server = Server(shutdown_event)
multiprocessing.Process(target=server.serve).start()
time.sleep(5)
shutdown_event.set()
But I want to keep the Server.shutdown method instead of changing the Server interface according to its usage (single processing v. multiprocessing) and I don't want clients to deal with multiprocessing.Event.
I have finally figured out a solution by myself:
import multiprocessing
import time
class Server:
def __init__(self):
self.shutdown_event = multiprocessing.Event()
def serve(self):
print("serving")
while not self.shutdown_event.is_set():
print("hello")
time.sleep(1)
print("done")
def shutdown(self):
print("stopping")
self.shutdown_event.set()
if __name__ == "__main__":
server = Server()
multiprocessing.Process(target=server.serve).start()
time.sleep(5)
server.shutdown()
It works in either case: single processing (multithreading) and multiprocessing.
Remark. — With a multiprocessing.Event() in the __init__ method, Server instances are no longer pickable. That might be a problem if one wants to call a Server instance in a process pool (either with multiprocessing.pool.Pool or concurrent.futures.ProcessPoolExecutor). In this case, one should replace multiprocessing.Event() with multiprocessing.Manager().Event() in the __init__ method.
I have a script which sets up a BasicHTTPServer in a thread so that the main script can automatically open a web browser pointing to the server url to download a file. After the file is downloaded, I want to shut down that server but I have no clue how to go about it. This is an example of what I've done currently:
def server():
HandlerClass = SimpleHTTPRequestHandler
ServerClass = BaseHTTPServer.HTTPServer
Protocol = 'HTTP/1.0'
server_address = ('127.0.0.1', 8000)
HandlerClass.protocol_version = Protocol
httpd = ServerClass(server_address, HandlerClass)
httpd.serve_forever()
def download():
t = threading.Thread(name='server', target=server)
t.start()
webbrowser.open('safari-http://127.0.0.1:8000/')
I want to shut down the server after webbrowser.open().
Thank you
I tried the example given here. Can you check if it worked for you.
runFlag = True
def server(server_class=BaseHTTPServer.HTTPServer,
handler_class=BaseHTTPServer.BaseHTTPRequestHandler):
global runFlag
server_address = ('127.0.0.1', 8000)
HandlerClass.protocol_version = Protocol
httpd = ServerClass(server_address, HandlerClass)
while runFlag:
httpd.handle_request()
httpd.shutdown()
def download():
t = threading.Thread(name='server', target=server)
t.start()
webbrowser.open('https:\\www.google.com')
global runFlag
runFlag = False
So after digging deep through many articles I managed to find a kind of messy solution that works for me which closes the server completely. To do it I incorporated code from the following sources:
The Green Place
Corey Goldberg
UI is a module exclusive to the iOS Python IDE Pythonista which basically just creates buttons "start", "stop" and "visit" which bind to their respective _t functions. The ui.in_background decorator just lets the ui remain responsive while things happen in the background.
self.httpd.socket.close() is what really closes the server but it's messy and prints an ugly error to stdout/err so I had no choice but to suppress it by redirecting stdout/err to a dead class so the error is dropped. Standard stdout/err behaviour is restored immediately after. Thank you all for the time and effort you took to help me, I appreciate it greatly.
import console
import BaseHTTPServer
import SocketServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
import sys
import threading
import webbrowser
from time import sleep
import ui
original_stdout = sys.stdout
original_stderr = sys.stderr
class BasicServer(SocketServer.TCPServer):
allow_reuse_address = True
class NullifyOutput():
def write(self, s):
pass
class ServerThread(threading.Thread):
def __init__(self, ip, port):
super(ServerThread, self).__init__()
self.ip = ip
self.port = port
self.HandlerClass = SimpleHTTPRequestHandler
self.Protocol = 'HTTP/1.0'
self.server_address = (self.ip, self.port)
self.HandlerClass.protocol_version = self.Protocol
try:
self.httpd = BasicServer(self.server_address, self.HandlerClass)
except:
self.port += 1
self.server_address = (self.ip, self.port)
self.httpd = BasicServer(self.server_address, self.HandlerClass)
self.stoptheserver = threading.Event()
def run(self):
while not self.stoptheserver.isSet():
self.httpd.handle_request()
def join(self, timeout=None):
self.stoptheserver.set()
self.httpd.socket.close()
super(ServerThread, self).join(timeout)
server = ServerThread('127.0.0.1', 8000)
def start_t(sender):
print server.isAlive()
if not server.isAlive():
server.start()
def visit_t(sender):
webbrowser.open('http://127.0.0.1:' + str(server.port))
#webbrowser.open('safari-http://127.0.0.1' + str(server.port))
# Use the safari- prefix to open in safari. You may need to switch to
# pythonista then back to safari to get the page to load.
#ui.in_background
def stop_t(sender):
sys.stdout, sys.stderr = NullifyOutput(), NullifyOutput()
server.join(3)
sys.stdout, sys.stderr = original_stdout, original_stderr
ui.load_view('SimpleServer').present('sheet')
Here is an example from cryptoassets.core project, status server:
class StatusHTTPServer(threading.Thread):
def __init__(self, ip, port):
threading.Thread.__init__(self)
self.running = False
self.ready = False
self.ip = ip
self.port = port
def run(self):
self.running = True
self.ready = True
self.httpd.serve_forever()
self.running = False
def start(self):
server_address = (self.ip, self.port)
try:
self.httpd = HTTPServer(server_address, StatusGetHandler)
except OSError as e:
raise RuntimeError("Could not start cryptoassets helper service status server at {}:{}".format(self.ip, self.port)) from e
threading.Thread.start(self)
def stop(self):
if self.httpd and self.running:
self.httpd.shutdown()
self.httpd = None
import SocketServer
import sys
from Queue import *
import threading
class CustomTCPServer(SocketServer.TCPServer):
def __init__(self, server_address, RequestHandlerClass, commandQueue=Queue):
self.queue = commandQueue
SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass, bind_and_activate=False)
SocketServer.TCPServer.allow_reuse_address = True
self.server_bind()
self.server_activate()
class SingleTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
commandQueue = self.server.queue
self.data = self.request.recv(1024).strip()
try:
commandQueue.put(self.data)
except Queue.Empty:
print 'Sorry.. Cannot accept any more.. Queue is full..'
def main():
HOST = ''
PORT = 50099
commandQueue = Queue()
server = CustomTCPServer((HOST, PORT), SingleTCPHandler, commandQueue)
threadObject = threading.Thread(target=server.serve_forever)
threadObject.setDaemon(True)
threadObject.start()
threadObject.join()
if __name__ == '__main__':
main()
But whenever I run the code for second time I get the message that
socket.error: [Errno 98] Address already in use
I am confused..
Any suggestion would be appreciated..
Instead of:
SocketServer.TCPServer.allow_reuse_address = True
Which sets the value on the base class that you've already subclassed yourself from, (and created an instance of),
Use:
self.allow_reuse_address = True
It doesn't look like you're ever closing the socket at the end. Try adding a server.shutdown() (as well as changing the allow_reuse_address) to shut it down at the end after the thread join.
I'm trying to send a message (a string) to a server (say server A) as a response to a particular string message received from the server. I've used a client connection to send messages to the server. I've created my own server (say server B) using socket module to receive the messages from the the above mentioned server. If a certain message is received, to client should send a message to that server A. I used threads so that both my client and server can work concurrently.
Here's my client script (socket1.py).
import socket
import time
s = socket.socket()
s2 = socket.socket()
class MyClient():
def __init__(self):
self.s=socket.socket()
def start(self):
self.s.connect(('localhost',6000))
self.s.settimeout(1000)
self.s.send('JOIN#')
while True:
if self.s.gettimeout():
break
def move(self):
self.s.send('LEFT#')
def close(self):
self.s.close()
def socStart():
global s,s2
s.connect(('localhost',6000))
#s2.connect(('localhost',6000))
s.send('JOIN#')
#time.sleep(10)
#s.send('DOWN#')
#s.close()
def move():
global s1
s1.send('DOWN#')
def close():
global s,s2
s.close()
#s2.close()
Here's my server script (server1.py)
import socket
class MyServer():
def __init__(self):
self.s= socket.socket()
def serStart(self):
self.s.bind(('localhost',7000))
#self.s.settimeout(30)
self.s.listen(5)
self.started = False
while True:
self.c, self.addr = self.s.accept()
self.msg = self.c.recv(1024)
if self.msg[0] == 'S':
self.started = True
print self.addr, '>>', self.msg
self.c.close()
if self.s.gettimeout():
break
self.s.close()
def getStarted(self):
return self.started
Here's the script in which I used threads with both socket1 & server1 module.
import threading
import datetime
import socket1
import server1
import time
class ThreadClass(threading.Thread):
def initialize(self,client,server):
self.client = client
self.server = server
def run(self):
self.client.start()
if self.server.getStarted():
self.client.move()
self.client.close()
class ThreadServer(threading.Thread):
def initialize(self, client, server):
self.client = client
self.server = server
def run(self):
self.server.serStart()
client = socket1.MyClient()
server = server1.MyServer()
t = ThreadClass()
tS=ThreadServer()
t.initialize(client, server)
tS.initialize(client, server)
t.start()
tS.start()
Though server class changes it's started varialble to True the move method doesn't work in ThreadClass. How do I perform such event triggering and responding accordingly with a outside server in python?
I'm trying to figure out why I can't kill my multi threaded SocketServer via a CRTL-C.
Basically I have that :
import SocketServer,threading
class TEST(SocketServer.BaseRequestHandler):
def server_bind(self):
self.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR,SO_REUSEPORT, 1)
self.socket.bind(self.server_address)
self.socket.setblocking(0)
def handle(self):
request, socket = self.request
data = request
if data[0] == "\x01":
buff = "blablabla"
socket.sendto(str(buff), self.client_address)
class TEST1(SocketServer.BaseRequestHandler):
def server_bind(self):
self.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR,SO_REUSEPORT, 1)
self.socket.bind(self.server_address)
self.socket.setblocking(0)
def handle(self):
request, socket = self.request
data = request
if data[0] == "\x01":
buff = "blablabla"
socket.sendto(str(buff), self.client_address)
class TEST2(SocketServer.BaseRequestHandler):
def server_bind(self):
self.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR,SO_REUSEPORT, 1)
self.socket.bind(self.server_address)
self.socket.setblocking(0)
def handle(self):
request, socket = self.request
data = request
if data[0] == "\x01":
buff = "blablabla"
socket.sendto(str(buff), self.client_address)
class TEST3(SocketServer.BaseRequestHandler):
def server_bind(self):
self.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR,SO_REUSEPORT, 1)
self.socket.bind(self.server_address)
self.socket.setblocking(0)
def handle(self):
request, socket = self.request
data = request
if data[0] == "\x01":
buff = "blablabla"
socket.sendto(str(buff), self.client_address)
def serve_thread_udp(host, port, handler):
server = SocketServer.UDPServer((host, port), handler)
server.serve_forever()
def serve_thread_tcp(host, port, handler):
server = SocketServer.TCPServer((host, port), handler)
server.serve_forever()
def main():
try:
threading.Thread(target=serve_thread_tcp,args=('', 4045,TEST)).start()
threading.Thread(target=serve_thread_tcp,args=('', 239,TEST1)).start()
threading.Thread(target=serve_thread_udp,args=('', 1246,TEST2)).start()
threading.Thread(target=serve_thread_tcp,args=('', 12342,TEST3)).start()
except KeyboardInterrupt:
os._exit()
if __name__ == '__main__':
try:
main()
except:
raise
I'm trying to understand what i've done wrong and what would be the best way to be able to kill the whole script via a crtl-c.
Any help would be greatly appreciated !
Thanks
Here is a solution:
def main():
import thread
try:
thread.start_new(serve_thread_tcp, ('', 4045,TEST))
thread.start_new(serve_thread_tcp,('', 239,TEST1))
thread.start_new(serve_thread_udp,('', 1246,TEST2))
thread.start_new(serve_thread_tcp,('', 12342,TEST3))
except KeyboardInterrupt:
os._exit()
if __name__ == '__main__':
try:
main()
except:
raise
raw_input()
To close the server you can type return or close the stdin.
The Problem is with the Thread class that will not allow closing the application before all Threads are closed.
serve_forever() will not end until you close the belonging to server(an other solution) on KeyboardInterrupt.
When creating threads, set them as daemon :
Thread.__init__(self)
self.setDaemon(True)
In this way all the thread will terminate when you have killed the main thread.
Based on python documentation in here :
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.