I created an http server. My task is to make sure that each new post request to the server is processed in a separate thread.I tried using socketserver.ThreadingTCPServer(),but this way I can't control the threads. I need to start a thread only after the previous one is completed or after 5 minutes have passed.How can I implement the server differently, but only so that requests are processed in a separate request and each new thread is processed after the previous one has completed?
from http.server import HTTPServer, BaseHTTPRequestHandler, ThreadingHTTPServer
import socketserver
import threading
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
self.send_response(200)
self.end_headers()
query_args = parse_qs(urlparse(self.path).query)
print(query_args)
if __name__ == "__main__":
try:
with socketserver.ThreadingTCPServer(("", 8080), Handler) as httpd:
while True:
httpd.serve_forever()
except Exception as e:
httpd.server_close()
I´m using HTTPServer to listen for incoming POST requests and serving them. All is working fine with that.
I need to add some periodic tasks in the script (every X seconds: do something). As the HTTP server takes full command after
def run(server_class=HTTPServer, handler_class=S, port=9999):
server_address = (ethernetIP, port)
httpd = server_class(server_address, handler_class)
httpd.serve_forever()
I guess if there´s any way to include a check for time.time() as part of:
class S(BaseHTTPRequestHandler):
def _set_response(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
def do_GET(self):
self._set_response()
self.wfile.write("GET request for {}".format(self.path).encode('utf-8'))
def do_POST(self):
# my stuff here
Any ideas are welcome. Thanks!
Thanks to #rdas for pointing me to the separate thread solution. I tried schedule but it didn´t work with the HTTP server, because I can´t tell the script to run the pending jobs.
I tried with threading, running my periodic task as deamon.. and it worked! Here´s the code structure:
import time
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
polTime = 60 # how often we touch the file
polFile = "myfile.abc"
# this is the deamon thread
def polUpdate():
while True:
thisSecond = int(time.time())
if thisSecond % polTime == 0: # every X seconds
f = open(polFile,"w")
f.close() # touch and close
time.sleep(1) # avoid loopbacks
return "should never come this way"
# here´s the http server starter
def run(server_class=HTTPServer, handler_class=S, port=9999):
server_address = (ethernetIP, port)
httpd = server_class(server_address, handler_class)
try:
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()
sys.exit(1)
# init the thread as deamon
d = threading.Thread(target=polUpdate, name='Daemon')
d.setDaemon(True)
d.start()
# runs the HTTP server
run(port=conf_port)
The HTTP server doesn´t block the thread, so it works great.
By the way, I´m using the file 'touching' as proof of life for the process.
I am trying to create multi threaded web server in python, but the requests are handled one by one. After searching few hours, I found this link but the approved answer seems to be incorrect as the request over there is also handled one by one.
Here is the code:
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import threading
from time import sleep
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
sleep(5)
message = threading.currentThread().getName()
self.wfile.write(message)
self.wfile.write('\n')
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
if __name__ == '__main__':
server = ThreadedHTTPServer(('localhost', 8080), Handler)
print 'Starting server, use <Ctrl-C> to stop'
server.serve_forever()
I added "sleep(5)" for 5 second delay to handle the request. After that I send multiple requests but all the requests are handled one by one and each request took 5 seconds. I am unable to find the reason. Help me.
The key requirement here is to be able to have a 5-second delay between the send_response and the data returned. That means you need streaming; you can't use ThreadingMixIn, gunicorn, or any other such hack.
You need something like this:
import time, socket, threading
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 8000
sock.bind((host, port))
sock.listen(1)
HTTP = "HTTP/1.1 200 OK\nContent-Type: text/html; charset=UTF-8\n\n"
class Listener(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True # stop Python from biting ctrl-C
self.start()
def run(self):
conn, addr = sock.accept()
conn.send(HTTP)
# serve up an infinite stream
i = 0
while True:
conn.send("%i " % i)
time.sleep(0.1)
i += 1
[Listener() for i in range(100)]
time.sleep(9e9)
I have a project that I'm working on where I hope to be able to:
start a wsgiserver on its own thread
do stuff (some of which involves interacting with the wsgiserver
close the thread
end the program
I can do the first two steps, but I'm having trouble with the last two. I've provided a simpler version of my project that exhibits the issue I have where I can do the first two steps from above, just not the last two.
A couple questions:
How do I get the thread to stop the wsgi server?
Do I just need to pull out the wsgiserver code and start it on its own process?
Some details of my project that may head off some questions:
My project currently spins up other processes that are intended to talk to my wsgi server. I can spin everything up and get my processes to talk to my server, but I'm not able to get a graceful shutdown. This code sample is intended to provide a 'relatively simple' sample that can be more easily reviewed.
there are remnants of failed attempts at solving this in the code, hopefully, they aren't too distracting.
#Simple echo program
#listens on port 3000 and returns anything posted by http to that port
#installing required libraries
#download/install Microsoft Visual C++ 9.0 for Python
#https://www.microsoft.com/en-us/download/details.aspx?id=44266
#pip install greenlet
#pip install gevent
import sys
import threading
import urllib
import urllib2
import time
import traceback
from gevent.pywsgi import WSGIServer, WSGIHandler
from gevent import socket
server = ""
def request_error(start_response):
global server
# Send error to atm - must provide start_response
start_response('500', [])
#server.stop()
return ['']
def handle_transaction(env, start_response):
global server
try:
result = env['wsgi.input'].read()
print("Received: " + result)
sys.stdout.flush()
start_response('200 OK', [])
if (result.lower()=="exit"):
#server.stop()
return result
else:
return result
except:
return request_error(start_response)
class ErrorCapturingWSGIHandler(WSGIHandler):
def read_requestline(self):
result = None
try:
result = WSGIHandler.read_requestline(self)
except:
protocol_error()
raise # re-raise error, to not change WSGIHandler functionality
return result
class ErrorCapturingWSGIServer(WSGIServer):
handler_class = ErrorCapturingWSGIHandler
def start_server():
global server
server = ErrorCapturingWSGIServer(
('', 3000), handle_transaction, log=None)
server.serve_forever()
def main():
global server
#start server on it's own thread
print("Echoing...")
commandServerThread = threading.Thread(target=start_server)
commandServerThread.start()
#now that the server is started, send data
req = urllib2.Request("http://127.0.0.1:3000", data='ping')
response = urllib2.urlopen(req)
reply = response.read()
print(reply)
#take a look at the threading info
print(threading.active_count())
#try to exit
req = urllib2.Request("http://127.0.0.1:3000", data='exit')
response = urllib2.urlopen(req)
reply = response.read()
print(reply)
#Now that I'm done, exit
#sys.exit(0)
return
if __name__ == '__main__':
main()
I am trying to use http.server to test all the links in a Python project. I can get my script to work if I start the server before running my script, and then the server stops when I close the terminal window. But I'd really like the script itself to start and stop the server.
I made a test script to simply start the server, get a page and prove the server is running, and then stop the server. I can't seem to get the pid of the server. When I try to kill the pid that this script reports after the script runs, I get a message that there is no such process; but the server is still running.
How do I get the correct pid for the server, or more generally how do I stop the server from the script?
import os
import requests
from time import sleep
# Start server, in background.
print("Starting server...")
os.system('python -m http.server &')
# Make sure server has a chance to start before making request.
sleep(1)
print "Server pid: "
os.system('echo $$')
url = 'http://localhost:8000/index.html'
print("Testing request: ", url)
r = requests.get(url)
print("Status code: ", r.status_code)
Here is what I am doing:
import threading
try:
from http.server import HTTPServer, SimpleHTTPRequestHandler # Python 3
except ImportError:
from SimpleHTTPServer import BaseHTTPServer
HTTPServer = BaseHTTPServer.HTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler # Python 2
server = HTTPServer(('localhost', 0), SimpleHTTPRequestHandler)
thread = threading.Thread(target = server.serve_forever)
thread.daemon = True
thread.start()
def fin():
server.shutdown()
print('server running on port {}'.format(server.server_port))
# here is your program
If you call fin in your program, then the server shuts down.
A slight modification to User's code above:
import threading
try:
from http.server import HTTPServer, BaseHTTPRequestHandler # Python 3
except ImportError:
import SimpleHTTPServer
from BaseHTTPServer import HTTPServer # Python 2
from SimpleHTTPServer import SimpleHTTPRequestHandler as BaseHTTPRequestHandler
server = HTTPServer(('localhost', 0), BaseHTTPRequestHandler)
thread = threading.Thread(target = server.serve_forever)
thread.deamon = True
def up():
thread.start()
print('starting server on port {}'.format(server.server_port))
def down():
server.shutdown()
print('stopping server on port {}'.format(server.server_port))
This is a closure solution to the problem. Works on python 3.
import os
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
def simple_http_server(host='localhost', port=4001, path='.'):
server = HTTPServer((host, port), SimpleHTTPRequestHandler)
thread = threading.Thread(target=server.serve_forever)
thread.deamon = True
cwd = os.getcwd()
def start():
os.chdir(path)
thread.start()
webbrowser.open_new_tab('http://{}:{}'.format(host, port))
print('starting server on port {}'.format(server.server_port))
def stop():
os.chdir(cwd)
server.shutdown()
server.socket.close()
print('stopping server on port {}'.format(server.server_port))
return start, stop
simple_http_server which will return start and stop functions
>>> start, stop = simple_http_server(port=4005, path='/path/to/folder')
which you can use as
>>> start()
starting server on port 4005
127.0.0.1 - - [14/Aug/2016 17:49:31] "GET / HTTP/1.1" 200 -
>>> stop()
stopping server on port 4005
Here's a context-flavored version which I prefer because it cleans up automatically and you can specify the directory to serve:
from contextlib import contextmanager
from functools import partial
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
#contextmanager
def http_server(host: str, port: int, directory: str):
server = ThreadingHTTPServer(
(host, port), partial(SimpleHTTPRequestHandler, directory=directory)
)
server_thread = Thread(target=server.serve_forever, name="http_server")
server_thread.start()
try:
yield
finally:
server.shutdown()
server_thread.join()
def usage_example():
import time
with http_server("127.0.0.1", 8087, "."):
# now you can use the web server
time.sleep(100)
I got this to run, but I'm curious to hear how this compares to User's answer above. I came up with this after looking at the accepted answer here.
import subprocess
import requests
import os
import signal
from time import sleep
print "Starting server..."
cmd = 'python -m SimpleHTTPServer'
pro = subprocess.Popen(cmd, shell=True, preexec_fn=os.setsid)
# Make sure server has a chance to start before making request.
sleep(1)
url = 'http://localhost:8000/index.html'
print "Testing request: ", url
r = requests.get(url)
print "Status code: ", r.status_code
os.killpg(pro.pid, signal.SIGTERM)
My solution with browser opening:
File: http.py
import SimpleHTTPServer
import SocketServer
import threading
import webbrowser
import platform
from socket import SOL_SOCKET,SO_REUSEADDR
class HTTPServer():
def __init__(self,port=8000,url='http://localhost'):
self.port = port
self.thread = None
self.httpd = None
self.run = False
self.url = url
os = platform.system()
if os=='Linux':
self.browser_path = "/usr/bin/google-chrome %s"
elif os == 'Windows':
self.browser_path = "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe %s"
else:
print("Chrome not found!")
def start(self):
self.run = True
self.httpd = SocketServer.TCPServer(("", self.port), SimpleHTTPServer.SimpleHTTPRequestHandler)
self.httpd.socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
self.thread = threading.Thread(target = self._serve)
self.thread.start()
webbrowser.get(str(self.browser_path)).open(self.url+":"+str(self.port)+"/")
def _serve(self):
while self.run:
self.httpd.handle_request()
def stop(self):
self.run = False
self.httpd.server_close()
After, just run:
from http import HTTPServer
server = HTTPServer()
server.start()
raw_input("Enter to close")
server.stop()
In my opinion, what I did is:
go to activity monitor
then go to network
search for word 'python'
(only if you ran the cmd/terminal command in python, because I think the way to stop it in cmd/terminal is ctrl+c or just quit the cmd/terminal)
note: the pid is also there
quit the process 'python' (if it doesn't, use force quit process)
if this works in cmd/terminal, then you are done! enjoii~
Hope this helps :D!!
Using Python 3.8
import http.server
import socketserver
import threading
PORT = 8000
Handler = http.server.SimpleHTTPRequestHandler
server = socketserver.TCPServer(("", PORT), Handler)
thread = threading.Thread(target = server.serve_forever)
thread.daemon = True
thread.start()