Develop Reference

In python is the .start() command blocking for the time it takes the thread start to complete?

Hypothesis:
thread....start() blocks until start completes.
Question:
Is hypothesis True or False?
Start http web server then open browser has the following code.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
This works fine. However, the following also works.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
Hypothesis:
thread....start() actually blocks until start completes. So,webbrowser.open_new(url) does not execute until start completes. Thus making the following unnecessary.
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
I have not been able to prove or disprove the Hypothesis after extensive searching.

There is no blocking when calling Thread.start() in the way you suggest.
The call is blocking in the sense that a call is placed that initalizes the new-thread internal state, and a system call is made to start the actual OS Thread - but that should take less than 1ms. The function that is the target of the thread is only called on the new thread, and the main thread will continue to run, regardless of what takes place inside that function.
If you want your program not to end, there is no need to resort to a complicated pausing loop like the one you setup - just place a call to threading.join() instead. This will block until all other threads end running, and only them the threading calling join() will proceed.

How can I set a timeout in Python HTTPServer

I have a Python (2.7.13) HTTP Server running in Debian, I want to stop any GET request that takes longer than 10 seconds, but can't find a solution anywhere.
I already tried all the snippets posted in the following question: How to implement Timeout in BaseHTTPServer.BaseHTTPRequestHandler Python
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
class handlr(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text-html')
self.end_headers()
self.wfile.write(os.popen('sleep 20 & echo "this took 20 seconds"').read())
def run():
server_address = ('127.0.0.1', 8080)
httpd = HTTPServer(server_address, handlr)
httpd.serve_forever()
if __name__ == '__main__':
run()
As a test, I'm running a shell command that takes 20 seconds to execute, so I need the server stop before that.

Put your operation on a background thread, and then wait for your background thread to finish. There isn't a general-purpose safe way to abort threads, so this implementation unfortunately leaves the function running in the background even though it had already given up.
If you can, you might consider putting a proxy server (like say nginx) in front of your server and let it handle timeouts for you, or perhaps use a more robust HTTP server implementation that allows this as a configuration option. But the answer below should basically cover it.
#!/usr/bin/env python
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import os
import threading
class handlr(BaseHTTPRequestHandler):
def do_GET(self):
result, succeeded = run_with_timeout(lambda: os.popen('sleep 20 & echo "this took 20 seconds"').read(), timeout=3)
if succeeded:
self.send_response(200)
self.send_header('Content-type','text-html')
self.end_headers()
self.wfile.write(os.popen('sleep 20 & echo "this took 20 seconds"').read())
else:
self.send_response(500)
self.send_header('Content-type','text-html')
self.end_headers()
self.wfile.write('<html><head></head><body>Sad panda</body></html>')
self.wfile.close()
def run():
server_address = ('127.0.0.1', 8080)
httpd = HTTPServer(server_address, handlr)
httpd.serve_forever()
def run_with_timeout(fn, timeout):
lock = threading.Lock()
result = [None, False]
def run_callback():
r = fn()
with lock:
result[0] = r
result[1] = True
t = threading.Thread(target=run_callback)
t.daemon = True
t.start()
t.join(timeout)
with lock:
return tuple(result)
if __name__ == '__main__':
run()

Run WebSocket server in Thread

this is probably more a question about threading than about my websocket.
I'm using "SimpleWebSocket" from github ( https://github.com/dpallot/simple-websocket-server )
The example works fine:
from SimpleWebSocketServer import SimpleWebSocketServer, WebSocket
class SimpleEcho(WebSocket):
def handleMessage(self):
# echo message back to client
self.sendMessage(self.data)
def handleConnected(self):
print self.address, 'connected'
def handleClose(self):
print self.address, 'closed'
server = SimpleWebSocketServer('', 8000, SimpleEcho)
server.serveforever()
The Server is running, I can connect and send Messages.
Now i try to run it as a Thread with those classes:
This one is supposed to create many threads including the WebSocketServer
from websockethread import WebSocketThread
class startManyThreads:
def __init__(self):
self.thread1 = WebSocketThread()
self.thread1.start()
if __name__ == "__main__":
startManyThreads = startManyThreads()
This class should run as my thread:
import threading
from SimpleWebSocketServer import SimpleWebSocketServer
from webSocketServer import WebSocketServer
class WebSocketThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
server = SimpleWebSocketServer('', 8000, WebSocketServer)
server.serveforever()
And this is the "customized" echo example:
from SimpleWebSocketServer import SimpleWebSocketServer, WebSocket
class SimpleEcho(WebSocket):
def handleMessage(self):
# echo message back to client
self.sendMessage(self.data)
def handleConnected(self):
print self.address, 'connected'
def handleClose(self):
print self.address, 'closed'
I have also tried to derive this: class SimpleEcho(WebSocket, threading.Thread):
Any Ideas what i'm doing wrong?
&
Thanks alot in advance!
Edit:
The result when i run "simpleEcho" is that i get a prompt can connect via the websocket.html (provided on github), send and receive Messages
The result when i put it in a thread (anyone of the 3 ways i tried) is the same behaviour except when i try to "connect" from the websocket.html i get a "error: undefined". With nmap i checked and the Server seems so be running & listening to port 8000
Edit 2: Derived new Class from SimpleWebSocketServer
import threading
from SimpleWebSocketServer import SimpleWebSocketServer
class ThreadSimpleWebSocketThread(threading.Thread, SimpleWebSocketServer):
def __init__(self, serversocket):
threading.Thread.__init__(self)
self.serversocket = serversocket
def serveforever(self):
SimpleWebSocketServer.serversocket = self.serversocket
SimpleWebSocketServer.selectInterval = 0.1
SimpleWebSocketServer.listeners = [self.serversocket]
super(ThreadSimpleWebSocketThread, self).serveforever()
def run(self):
self.serveforever()

The main problem seems to be where you're starting the server. The Thread.__init__() method runs inside the main thread (of the caller), not the actual WebSocketThread(). This needs to be done in the Thread.run() method:
class WebSocketThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
server = SimpleWebSocketServer('', 8000, WebSocketServer)
server.serveforever()
The code inside run() actually runs inside the thread.
Note that because of the Global Interpreter Lock, threads won't improve performance much, and you'll probably need multiprocessing. However, if you just want to offload the I/O waiting, this should work fine.
Edit: From looking at this GitHub project, and rethinking what you're trying to do, this isn't trivial. You'll have to override WebSocket.serveforever() in your SimpleEcho() class and change it to accept the socket and pass the accepted socket to a Thread (see here).

How do I kill SimpleHTTPServer from within a Python script?

I am trying to use http.server to test all the links in a Python project. I can get my script to work if I start the server before running my script, and then the server stops when I close the terminal window. But I'd really like the script itself to start and stop the server.
I made a test script to simply start the server, get a page and prove the server is running, and then stop the server. I can't seem to get the pid of the server. When I try to kill the pid that this script reports after the script runs, I get a message that there is no such process; but the server is still running.
How do I get the correct pid for the server, or more generally how do I stop the server from the script?
import os
import requests
from time import sleep
# Start server, in background.
print("Starting server...")
os.system('python -m http.server &')
# Make sure server has a chance to start before making request.
sleep(1)
print "Server pid: "
os.system('echo $$')
url = 'http://localhost:8000/index.html'
print("Testing request: ", url)
r = requests.get(url)
print("Status code: ", r.status_code)

Here is what I am doing:
import threading
try:
from http.server import HTTPServer, SimpleHTTPRequestHandler # Python 3
except ImportError:
from SimpleHTTPServer import BaseHTTPServer
HTTPServer = BaseHTTPServer.HTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler # Python 2
server = HTTPServer(('localhost', 0), SimpleHTTPRequestHandler)
thread = threading.Thread(target = server.serve_forever)
thread.daemon = True
thread.start()
def fin():
server.shutdown()
print('server running on port {}'.format(server.server_port))
# here is your program
If you call fin in your program, then the server shuts down.

A slight modification to User's code above:
import threading
try:
from http.server import HTTPServer, BaseHTTPRequestHandler # Python 3
except ImportError:
import SimpleHTTPServer
from BaseHTTPServer import HTTPServer # Python 2
from SimpleHTTPServer import SimpleHTTPRequestHandler as BaseHTTPRequestHandler
server = HTTPServer(('localhost', 0), BaseHTTPRequestHandler)
thread = threading.Thread(target = server.serve_forever)
thread.deamon = True
def up():
thread.start()
print('starting server on port {}'.format(server.server_port))
def down():
server.shutdown()
print('stopping server on port {}'.format(server.server_port))

This is a closure solution to the problem. Works on python 3.
import os
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
def simple_http_server(host='localhost', port=4001, path='.'):
server = HTTPServer((host, port), SimpleHTTPRequestHandler)
thread = threading.Thread(target=server.serve_forever)
thread.deamon = True
cwd = os.getcwd()
def start():
os.chdir(path)
thread.start()
webbrowser.open_new_tab('http://{}:{}'.format(host, port))
print('starting server on port {}'.format(server.server_port))
def stop():
os.chdir(cwd)
server.shutdown()
server.socket.close()
print('stopping server on port {}'.format(server.server_port))
return start, stop
simple_http_server which will return start and stop functions
>>> start, stop = simple_http_server(port=4005, path='/path/to/folder')
which you can use as
>>> start()
starting server on port 4005
127.0.0.1 - - [14/Aug/2016 17:49:31] "GET / HTTP/1.1" 200 -
>>> stop()
stopping server on port 4005

Here's a context-flavored version which I prefer because it cleans up automatically and you can specify the directory to serve:
from contextlib import contextmanager
from functools import partial
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
#contextmanager
def http_server(host: str, port: int, directory: str):
server = ThreadingHTTPServer(
(host, port), partial(SimpleHTTPRequestHandler, directory=directory)
)
server_thread = Thread(target=server.serve_forever, name="http_server")
server_thread.start()
try:
yield
finally:
server.shutdown()
server_thread.join()
def usage_example():
import time
with http_server("127.0.0.1", 8087, "."):
# now you can use the web server
time.sleep(100)

I got this to run, but I'm curious to hear how this compares to User's answer above. I came up with this after looking at the accepted answer here.
import subprocess
import requests
import os
import signal
from time import sleep
print "Starting server..."
cmd = 'python -m SimpleHTTPServer'
pro = subprocess.Popen(cmd, shell=True, preexec_fn=os.setsid)
# Make sure server has a chance to start before making request.
sleep(1)
url = 'http://localhost:8000/index.html'
print "Testing request: ", url
r = requests.get(url)
print "Status code: ", r.status_code
os.killpg(pro.pid, signal.SIGTERM)

My solution with browser opening:
File: http.py
import SimpleHTTPServer
import SocketServer
import threading
import webbrowser
import platform
from socket import SOL_SOCKET,SO_REUSEADDR
class HTTPServer():
def __init__(self,port=8000,url='http://localhost'):
self.port = port
self.thread = None
self.httpd = None
self.run = False
self.url = url
os = platform.system()
if os=='Linux':
self.browser_path = "/usr/bin/google-chrome %s"
elif os == 'Windows':
self.browser_path = "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe %s"
else:
print("Chrome not found!")
def start(self):
self.run = True
self.httpd = SocketServer.TCPServer(("", self.port), SimpleHTTPServer.SimpleHTTPRequestHandler)
self.httpd.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
self.thread = threading.Thread(target = self._serve)
self.thread.start()
webbrowser.get(str(self.browser_path)).open(self.url+":"+str(self.port)+"/")
def _serve(self):
while self.run:
self.httpd.handle_request()
def stop(self):
self.run = False
self.httpd.server_close()
After, just run:
from http import HTTPServer
server = HTTPServer()
server.start()
raw_input("Enter to close")
server.stop()

In my opinion, what I did is:
go to activity monitor
then go to network
search for word 'python'
(only if you ran the cmd/terminal command in python, because I think the way to stop it in cmd/terminal is ctrl+c or just quit the cmd/terminal)
note: the pid is also there
quit the process 'python' (if it doesn't, use force quit process)
if this works in cmd/terminal, then you are done! enjoii~
Hope this helps :D!!

Using Python 3.8
import http.server
import socketserver
import threading
PORT = 8000
Handler = http.server.SimpleHTTPRequestHandler
server = socketserver.TCPServer(("", PORT), Handler)
thread = threading.Thread(target = server.serve_forever)
thread.daemon = True
thread.start()

Python thread blocking further execution

I have been trying to write a python script that initiates a thread to listen on a socket and send HTTP data to another application to be launched by the same program. There is a requirement for the socket server to be running prior to executing the application. However, the thread running the socket server blocks further execution of the program and it freezes where it is listening. Putting some dummy code.
In module 1:
def runServer(Port, Host, q):
HTTPServerObj = HTTPServer((Host, Port), RequestHandler)
HTTPServerObj.handle_request()
HTTPServerObj.server_close()
q.put((True, {'messageDoNotDuplicate': 'Data sent successfully by the server'}))
class SpoofHTTPServer(object):
def runServerThread(self):
q = Queue.Queue()
serverThread=Thread(target=runServer, args=(self.Port, self.Host, q))
serverThread.daemon=True
serverThread.start()
result = q.get()
print result
return result
In module 2:
from module1 import SpoofHTTPServer
spoofHTTPServer = SpoofHTTPServer()
result = spoofHTTPServer.runServerThread()
rc = myApp.start()
The myApp.start() never gets executed as the thread is blocking it.

It looks to me like the method that blocks execution is not the thread but q.get(). It will listen to the Queue until an item is available, but since it's executed before running the client application nothing ever gets posted into the queue. Maybe you should return q instead and listen to the queue in module 2 after calling myApp.start()?

This may work for you from Python 3. Make a connection to ('localhost', 8080) to see it work.
import queue as Queue
from threading import Thread
from http.server import HTTPServer
from socketserver import BaseRequestHandler as RequestHandler
def runServer(Port, Host, q):
HTTPServerObj = HTTPServer((Host, Port), RequestHandler)
HTTPServerObj.handle_request()
HTTPServerObj.server_close()
q.put((True, {'messageDoNotDuplicate':
'Data sent successfully by the server'}))
class SpoofHTTPServer(object):
Port = 8080
Host = ''
def runServerThread(self):
q = Queue.Queue()
serverThread=Thread(target=runServer, args=(self.Port, self.Host, q))
serverThread.daemon=True
serverThread.start()
result = q.get()
print(result)
return result
spoofHTTPServer = SpoofHTTPServer()
result = spoofHTTPServer.runServerThread()
##rc = myApp.start()