I'm trying to run a python http server in the background using threading. I came across several references that do the following:
import threading
import http.server
import socket
from http.server import HTTPServer, SimpleHTTPRequestHandler
debug = True
server = http.server.ThreadingHTTPServer((socket.gethostname(), 6666), SimpleHTTPRequestHandler)
if debug:
print("Starting Server in background")
thread = threading.Thread(target = server.serve_forever)
thread.daemon = True
thread.start()
else:
print("Starting Server")
print('Starting server at http://{}:{}'.format(socket.gethostname(), 6666))
server.serve_forever()
When thread.daemon is set to True, the program will finish without starting the server (nothing running on port 6666).
And when I set thread.daemon to False, it starts the server in foreground and blocks the terminal until I kill it manually.
Any idea on how to make this work?
In both cases the server is launched in the background, in the separate thread. This means that thread.start() launches the server and python continues executing the rest of the code in the main thread.
However, there seems to be nothing else to execute in your program. Python reaches the end of the file and the main thread is done.
The OS requires all non-daemon threads to be done before the process could be finished. When thread.daemon is set to False the OS waits until the server thread exits (which will never happen, as the name serve_forever implies). When it is True the process is closed immediately after the main thread is done.
Put whatever code you want to be executed asynchronously after the thread.start() and you're done!
Related
I want to kill a Thread that runs a server via pymodbus. When I use the .join() method, the server will continue running if a client is connected.
address = ("ip.of.my.pi", 5020)
server = Thread(target=StartTcpServer, args=(context, identity, address))
server.start()
def threaded(context, server):
if master.poll:
thread = Thread(target=server_loop, args=(context,))
thread.start()
time.sleep(5)
master.after(100, lambda: threaded(context, server))
else:
server.join()
threaded(context, server)
The function server_loop runs a measurement, which will stop in this constellation. I cannot use a normal loop, because, as you see at master.after, I'm using Tkinter as a GUI.
To stop a pymodbus server inside a thread, you have to start the server.join
From another thread.
I'm writing a python script that will start a local fileserver, and while that server is alive it will be writing to a file every 30 seconds. I would like to have the server and writer function running synchronously so I made the writer function a daemon thread... My main question is, since this daemon thread will quit once the server is stopped, if the daemon is in the middle of writing to a file will it complete that operation before exiting? It would be really bad to be left with 1/2 a file. Here's the code, but the actual file it will be writing is about 3k lines of JSON, hence the concern.
import http.server
import socketserver
from time import sleep
from threading import Thread
class Server:
def __init__(self):
self.PORT = 8000
self.Handler = http.server.SimpleHTTPRequestHandler
self.httpd = socketserver.TCPServer(("", self.PORT), self.Handler)
print("Serving at port", self.PORT)
def run(self):
try:
self.httpd.serve_forever()
except KeyboardInterrupt:
print("Server stopped")
def test():
while True:
with open('test', mode='w') as file:
file.write('testing...')
print('file updated')
sleep(5)
if __name__ == "__main__":
t = Thread(target=test, daemon=True)
t.start()
server = Server()
server.run()
It looks like you may have made an incorrect decision making the writer thread daemonic.
Making a daemonic thread does not mean it will run synchronously. It will still be affected by the GIL.
If you want synchronous execution, you'll have to use multiprocessing
From the Python docs:
Daemon threads are abruptly stopped at shutdown. Their resources (such
as open files, database transactions, etc.) may not be released
properly. If you want your threads to stop gracefully, make them
non-daemonic and use a suitable signalling mechanism such as an Event.
So that means that daemon threads are only suitable for the tasks that only make sense in context of the main thread and don't matter when the main thread has stopped working. File I/O, particularly data saving, is not suitable for a daemon thread.
So it looks like the most obvious and logical solution would be to make the writer thread non-daemonic.
Then, even if the main thread exits, the Python process won't be ended until all non-daemonic threads have finished. This allows for file I/O to complete and exit safely.
Explanation of daemonic threads in Python can be found here
I want to start a local server and then open a link with a browser from the same python program.
This is what I tried (a very naive and foolish attempt):
from subprocess import call
import webbrowser
call(["python", "-m", "SimpleHTTPServer"]) #This creates a server at port:8000
webbrowser.open_new_tab("some/url")
However, the program doesn't go to the second statement because the server is still running in the background. To open the browser, I need to exit the server which defeats the purpose of running the server.
Can anyone help me by suggesting a working solution?
You could start your web server in a daemon thread (a Python program exits if only daemon threads are left) and then make your requests from the main thread.
The only problem then is to synchronize your main thread to the server thread, since the HTTP-server will need some time to start up and won't handle any requests until this point. I am not aware of an easy and clean solution to do that, but you could (somewhat hackish) just pause your main thread for a number seconds (possibly shorter) and start making requests only after this. Another option would be to just send requests to the webserver from the very beginning and expect them to fail for some amount of time.
Here is a small sample script with a simple HTTP webserver that serves content from the local file system over TCP on localhost:8080 and a sample request, requesting a file foo.txt from the directory the webserver (and in this case also the script) was started in.
import sys
import requests
import threading
import time
from BaseHTTPServer import HTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
# set up the HTTP server and start it in a separate daemon thread
httpd = HTTPServer(('localhost', 8080), SimpleHTTPRequestHandler)
thread = threading.Thread(target=httpd.serve_forever)
thread.daemon = True
# if startup time is too long we might want to be able to quit the program
try:
thread.start()
except KeyboardInterrupt:
httpd.shutdown()
sys.exit(0)
# wait until the webserver finished starting up (maybe wait longer or shorter...)
time.sleep(5)
# start sending requests
r = requests.get('http://localhost:8080/foo.txt')
print r.status_code
# => 200 (hopefully...)
I have a flask controller which starts a tcpserver on demand. I am posting a simplified version of the code.
#app.route("/")
def hello():
scenario_server = MyTCPServer(('localhost', 9000), MyTCPHandler)
server_thread = threading.Thread(target=scenario_server.serve_forever)
server_thread.daemon = True
server_thread.start()
return "TCP server started"
The MyTCPServer is a normal SocketServer.TCPServer. This is an example of a simple MyTCPHandler.
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
while True:
data = self.request.recv(1024).strip().decode('ascii')
if data == "HELLO":
self.request.sendall('COUCOU'.encode('ascii'))
else:
break
And I start my app application in the __main__ the standard way (using the built in server app.run(threaded=True))
After I launch the tcpserver, I connect to it using telnet and send HELLO
With this code and the previous scenario, if I press CTRL+C, the main thread will caught it but it won't kill the program. It seems that the thread running the TCPServer is not killed even if the thread is run with daemon = True
However If I modify my handler to change this :
data = self.request.recv(1024).strip().decode('ascii')
if data == "":
break
On CTRL+C, it properly exits the program.
Anybody could explain why ?
I start multiple servers using the following:
from threading import Thread
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/plain")
self.end_headers()
self.wfile.write("Hello World!")
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
pass
def serve_on_port(port):
server = ThreadingHTTPServer(("localhost",port), Handler)
server.serve_forever()
Thread(target=serve_on_port, args=[1111]).start()
Thread(target=serve_on_port, args=[2222]).start()
I want to stop these threads on KeyboardInterrupt.
How can I do that?
You can kill lots of threads at the end of your program by defining them as daemon threads. To do this, set their daemon property to true. According to the documentation,
This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.
The entire Python program exits when no alive non-daemon threads are left.
So, something like this should work:
for port in [1111, 2222]:
t = Thread(target=serve_on_port, args=[port])
t.daemon = True
t.start()
try:
while True:
time.sleep(1000000)
except KeyboardInterrupt:
pass
Note that any threads that are non-daemon and still running will keep your program from exiting. If you have other threads that you also want to be killed on exit, set their daemon properties to True before starting them, too.
To stop one of these servers, you can use its shutdown() method. This means you will need a reference to the server from the code that catches the KeyboardInterrupt. For example:
servers = []
for port in [11111, 22222]:
servers.append(ThreadingHTTPServer(("localhost",port), Handler))
for server in servers:
Thread(target=server.serve_forever).start()
try:
while True:
time.sleep(1000000)
except KeyboardInterrupt:
for server in servers:
server.shutdown()