Develop Reference

Python HTTPServer and periodic tasks

I´m using HTTPServer to listen for incoming POST requests and serving them. All is working fine with that.
I need to add some periodic tasks in the script (every X seconds: do something). As the HTTP server takes full command after
def run(server_class=HTTPServer, handler_class=S, port=9999):
server_address = (ethernetIP, port)
httpd = server_class(server_address, handler_class)
httpd.serve_forever()
I guess if there´s any way to include a check for time.time() as part of:
class S(BaseHTTPRequestHandler):
def _set_response(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
def do_GET(self):
self._set_response()
self.wfile.write("GET request for {}".format(self.path).encode('utf-8'))
def do_POST(self):
# my stuff here
Any ideas are welcome. Thanks!

Thanks to #rdas for pointing me to the separate thread solution. I tried schedule but it didn´t work with the HTTP server, because I can´t tell the script to run the pending jobs.
I tried with threading, running my periodic task as deamon.. and it worked! Here´s the code structure:
import time
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
polTime = 60 # how often we touch the file
polFile = "myfile.abc"
# this is the deamon thread
def polUpdate():
while True:
thisSecond = int(time.time())
if thisSecond % polTime == 0: # every X seconds
f = open(polFile,"w")
f.close() # touch and close
time.sleep(1) # avoid loopbacks
return "should never come this way"
# here´s the http server starter
def run(server_class=HTTPServer, handler_class=S, port=9999):
server_address = (ethernetIP, port)
httpd = server_class(server_address, handler_class)
try:
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()
sys.exit(1)
# init the thread as deamon
d = threading.Thread(target=polUpdate, name='Daemon')
d.setDaemon(True)
d.start()
# runs the HTTP server
run(port=conf_port)
The HTTP server doesn´t block the thread, so it works great.
By the way, I´m using the file 'touching' as proof of life for the process.

Force reload on SimpleHTTP Server in Python

I have a very simple HTTPServer implemented in Python. The code is the following:
import SimpleHTTPServer
import SocketServer as socketserver
import os
import threading
class MyHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
path_to_image = 'RGBWebcam1.png'
img = open(path_to_image, 'rb')
statinfo = os.stat(path_to_image)
img_size = statinfo.st_size
print(img_size)
def do_HEAD(self):
self.send_response(200)
self.send_header("Content-type", "image/png")
self.send_header("Content-length", img_size)
self.end_headers()
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "image/png")
self.send_header("Content-length", img_size)
self.end_headers()
f = open(path_to_image, 'rb')
self.wfile.write(f.read())
f.close()
class MyServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_adress, RequestHandlerClass):
self.allow_reuse_address = True
socketserver.TCPServer.__init__(self, server_adress, RequestHandlerClass, False)
if __name__ == "__main__":
HOST, PORT = "192.168.2.10", 9999
server = MyServer((HOST, PORT), MyHandler)
server.server_bind()
server.server_activate()
server_thread = threading.Thread(target=server.serve_forever)
server_thread.start()
while(1):
print "test"
If I connect to the given IP-Adress the page loads and everything is fine. Now it would be nice if the page would automatically refresh every n seconds.
I am very new to python and especially new to webcoding. I have found LiveReload however I cannot get my head around how I merge these two libraries together.
Thank you for your help

You'll require a connection to the client if you want the server to tell it to refresh. A HTTP server means you've sent information (HTML) and the client will process it. There is no communication beyond that. That would require AJAX or Websockets to be implemented - both protocols that allow frequent communication.
Since you can't communicate, you should automate the refresh in the content you initially send. In our example we'll say we want the page to refresh every 30 seconds. This is possible to do in either HTML or Javascript:
<meta http-equiv="refresh" content="30" />
or
setTimeout(function(){
window.location.reload(1);
}, 30000);

Issues in creating web app with SSE + long polling

i am new in web development. i am creating a web app for my home automation project, in which i need bi-directional communication. any theft-security alert from home will be send to client from server or if client want to control the main gate, he'll sent a POST request to server. I am still confused what thing to use, SSE or Web sockets. my question is, is it possible to develop an app that uses both, SSE as well as handles traditional (long-polling) HTTP requests from client (GET/POST) ? i have tested each of them individually and they work fine but i am unable to make them work together. i am using python BaseHTTPServer. Or at last, do i have to move to WebSocket? Any suggestion will be highly appreciated. my code here is;
import time
import BaseHTTPServer
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import os
import requests
import threading
from threading import Thread
from chk_connection import is_connected
import socket
HOST_NAME = socket.gethostbyname(socket.gethostname())
PORT_NUMBER = 8040 # Maybe set this to 9000.
ajax_count=0
ajax_count_str=""
switch=0
IP_Update_time=2
keep_alive=0
connected=False
###############################
##############
my_dir = os.getcwd()
class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_HEAD(s):
s.send_response(200)
s.send_header("Content-type", "text/html")
s.end_headers()
def do_POST(s):
global keep_alive
"""Respond to a POST request."""
s.send_response(200)
s.send_header('Access-Control-Allow-Origin', '*')
s.send_header("Content-type", "text/html")
s.end_headers()
HTTP_request=s.requestline
if HTTP_request.find('keep_alive')>-1:
keep_alive += 1
keep_alive_str = str(keep_alive)
s.wfile.write(keep_alive_str) #sending ajax calls for keep alive
def do_GET(s):
global ajax_count
global my_dir
global switch
global ajax_count_str
global keep_alive
#print 'got Get',
"""Respond to a GET request."""
s.send_response(200)
#s.send_header('Access-Control-Allow-Origin', '*')
s.send_header('content-type', 'text/html')
s.end_headers()
print s.headers
HTTP_request=s.requestline
index_1=HTTP_request.index("GET /")
index_2=HTTP_request.index(" HTTP/1.1")
file_name=HTTP_request[index_1+5:index_2]
#print 'file_name:',file_name
#print 'HTTP_request:',HTTP_request
#if len(file_name)>0:
#if HTTP_request.find('L2ON')>-1:
# print 'sending SSE'
# s.wfile.write('event::'.format(time.time()))
elif HTTP_request.find('GET / HTTP/1.1')>-1:
print 'send main'
file1=open('Index.html','r')
file_read=file1.read()
s.wfile.write(file_read)
elif file_name.find("/")==-1:
for root, dirs, files in os.walk(my_dir):
#print 'in for 1'
for file in files:
#print 'in for'
if HTTP_request.find(file)>-1:
file_path=os.path.join(root,file)
file1=open(file_path,'r')
file_read=file1.read()
s.wfile.write(file_read)
print 'send',file
elif file_name.find("/")>-1:
#print 'get /...'
slash_indexes=[n for n in xrange(len(file_name)) if file_name.find('/', n)==n]
length=len(slash_indexes)
slash=slash_indexes[length-1]
file_path=file_name[0:slash]
root_dir=(my_dir + '/' + file_path + '/')
for root, dirs, files in os.walk(root_dir):
for file in files:
if HTTP_request.find(file)>-1:
image_path=os.path.join(root,file)
image=open(image_path,'r')
image_read=image.read()
s.wfile.write(image_read)
print 'send',file
#else:
#print 'file not found'
class MyHandler_SSE(BaseHTTPRequestHandler):
print 'SSE events class'
def do_GET(self):
print 'this is SSE'
self.send_response(200)
self.send_header('content-type', 'text/event-stream')
self.end_headers()
while True:
print 'SSE sent'
self.wfile.write('event: message\nid: 1\ndata: {0}\ndata:\n\n'.format(time.time()))
time.sleep(2)
class chk_connection(threading.Thread):
"""
# this thread checks weather there is internet connection available ?
"""
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global connected
while 1:
########################################################## INSIDE THE chk_connection import is_connected
#import socket
#REMOTE_SERVER = "www.google.com"
#def is_connected():
#try:
# # see if we can resolve the host name -- tells us if there is
# # a DNS listening
# host = socket.gethostbyname(REMOTE_SERVER)
# connect to the host -- tells us if the host is actually
# reachable
# s = socket.create_connection((host, 80), 2)
# return True
#except:
# pass
#return False
##########################################################
connected=is_connected()
#print 'internet:', connected
time.sleep(1)
class server_main(threading.Thread):
"""
"""
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global connected
#print 'shutdown started'
server_class = BaseHTTPServer.HTTPServer
HOST_NAME = socket.gethostbyname(socket.gethostname())
last_HOST_NAME = HOST_NAME
httpd = server_class((HOST_NAME, PORT_NUMBER), MyHandler)
#http_SSE = server_class((HOST_NAME, PORT_NUMBER), MyHandler_SSE)
print time.asctime(), "Server Starts - %s:%s" % (HOST_NAME, PORT_NUMBER)
while(1):
while is_connected():
httpd._handle_request_noblock()
#print 'no block'
#http_SSE._handle_request_noblock()
time.sleep(1)
HOST_NAME = socket.gethostbyname(socket.gethostname())
if HOST_NAME != last_HOST_NAME:
print 'Serving at new host:', HOST_NAME
httpd = server_class((HOST_NAME, PORT_NUMBER), MyHandler)
def start():
tx_socket_thread3 = chk_connection() # this thread checks weather there is internet connection available ?
tx_socket_thread3.start()
tx_socket_thread5 = server_main()
tx_socket_thread5.start()
print 's1:',tx_socket_thread1.is_alive()
if __name__ == '__main__':
start()
i might need to modify the code in a new manner, but don't know how. What i want is, if any interrupt happens at server side, it pulls data to client, and mean while it also responds to the GET and POST requests from client. Help Plx...

It is definitely possible to develop a web application which uses a mixture of normal HTTP traffic, server-side events and WebSockets. However, the web server classes in Python standard library are not designed for this purpose, though one can probably make them to work with enough hammering. You should to install a proper web server and use it facilities.
Examples include
uWSGI and server-side events with WSGI applications
Tornado and WebSockets
Furthermore
Installing Python packages

How to run Python CGI script

I have never setup a server (let alone a python server) before and i am a bit lost. How do i utilize the following code? I have tried to put in in the cgi bin directory but that didnt work. It returned an internal server error. have a look at this here
#!/usr/bin/env python
#
# Funf: Open Sensing Framework
# Copyright (C) 2010-2011 Nadav Aharony, Wei Pan, Alex Pentland.
# Acknowledgments: Alan Gardner
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn
import sys
import cgi
import urlparse
import os.path
import shutil
import time
server_dir = os.path.dirname(__file__)
config_path = '/config'
config_file_path = os.path.join(server_dir, 'config.json')
upload_path = '/data'
upload_dir = os.path.join(server_dir, 'uploads')
def read_config():
config = None
try:
with open(config_file_path) as config_file:
config = config_file.read()
except IOError:
pass
return config
def backup_file(filepath):
shutil.move(filepath, filepath + '.' + str(int(time.time()*1000)) + '.bak')
def write_file(filename, file):
if not os.path.exists(upload_dir):
os.mkdir(upload_dir)
filepath = os.path.join(upload_dir, filename)
if os.path.exists(filepath):
backup_file(filepath)
with open(filepath, 'wb') as output_file:
while True:
chunk = file.read(1024)
if not chunk:
break
output_file.write(chunk)
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
parsed_url = urlparse.urlparse(self.path)
if parsed_url.path == config_path:
config = read_config()
if config:
self.send_response(200)
self.end_headers()
self.wfile.write(config)
else:
self.send_error(500)
elif parsed_url.path == upload_path:
self.send_error(405)
else:
self.send_error(404)
def do_POST(self):
parsed_url = urlparse.urlparse(self.path)
path = parsed_url.path
ctype, pdict = cgi.parse_header(self.headers['Content-Type'])
if path == upload_path:
if ctype=='multipart/form-data':
form = cgi.FieldStorage(self.rfile, self.headers, environ={'REQUEST_METHOD':'POST'})
try:
fileitem = form["uploadedfile"]
if fileitem.file:
try:
write_file(fileitem.filename, fileitem.file)
except Exception as e:
print e
self.send_error(500)
else:
self.send_response(200)
self.end_headers()
self.wfile.write("OK")
return
except KeyError:
pass
# Bad request
self.send_error(400)
elif parsed_url.path == config_path:
self.send_error(405)
else:
self.send_error(404)
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
if __name__ == '__main__':
if sys.argv[1:]:
port = int(sys.argv[1])
else:
port = 8000
server_address = ('', port)
httpd = ThreadedHTTPServer(server_address, RequestHandler)
sa = httpd.socket.getsockname()
print "Serving HTTP on", sa[0], "port", sa[1], "..."
print 'use <Ctrl-C> to stop'
httpd.serve_forever()

If you want to run a CGI on something like Apache (as opposed via custom server code like you pasted above), you can create a .py file like this in a (.py) CGI-enabled directory.
#!/usr/bin/env python
print "Content-Type: text/html"
print
print 'Hello World'
If you're using Apache, here's some info on how to set up CGI executables.
edit: (As Adrien P. says, the Python script should be made executable.)

You do not have to place it into a cgi-bin directory.
If you are running windows, you can launch Idle from your start menu under the python entry. Paste the code in, and hit F5 to run the code.
If you are running *nix, look to Adrien's answer for the commands and copy what is output when you run ./your_script.py.
Are you attempting to program a website in Python? This is code to create a web server, not site, so navigating to the program in a web browser will yield no results.

$ chmod +x your_script.py
$ ./your_script.py
A quick look a your code: it launch a simple http server who listen on port 8000

Heroku is a good place to host and python scripts.
Pre-req
pythonscripts.py
procfile
requirements.txt
and After add, commit and push the scripts to heroku app. Just run the following command on terminal to run the scripts.
heroku run python your_scripts.py
More if you want to run this scripts on a schedule timing. then heroku provides lots of adds-on. just search it on heroku

How to serve an mp3 file with built-in python http server

I am currently trying to serve MP3 Files using Python. The problem is that I can only play the MP3 once. Afterwards media controls stop responding and I need to reload entirely the page to be able to listen again to the MP3. (tested in Chrome)
Problem: running the script below, and entering http://127.0.0.1/test.mp3 on my browser will return an MP3 files which can be replayed only if I refresh the page
Notes:
Saving the page as HTML and loading it directly with Chrome (without Python server) would make the problem disappear.
Serving the file with Apache would solve the problem, but this is overkilled: I want to make the script very easy to use and not require installing Apache.
Here is the code I use:
import string
import os
import urllib
import socket
# Setup web server import string,cgi,time
import string,cgi,time
from os import curdir, sep
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import hashlib
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
try:
# serve mp3 files
if self.path.endswith(".mp3"):
print curdir + sep + self.path
f = open(curdir + sep + self.path, 'rb')
st = os.fstat( f.fileno() )
length = st.st_size
data = f.read()
md5 = hashlib.md5()
md5.update(data)
md5_key = self.headers.getheader('If-None-Match')
if md5_key:
if md5_key[1:-1] == md5.hexdigest():
self.send_response(304)
self.send_header('ETag', '"{0}"'.format(md5.hexdigest()))
self.send_header('Keep-Alive', 'timeout=5, max=100')
self.end_headers()
return
self.send_response(200)
self.send_header('Content-type', 'audio/mpeg')
self.send_header('Content-Length', length )
self.send_header('ETag', '"{0}"'.format(md5.hexdigest()))
self.send_header('Accept-Ranges', 'bytes')
self.send_header('Last-Modified', time.strftime("%a %d %b %Y %H:%M:%S GMT",time.localtime(os.path.getmtime('test.mp3'))))
self.end_headers()
self.wfile.write(data)
f.close()
return
except IOError:
self.send_error(404,'File Not Found: %s' % self.path)
from SocketServer import ThreadingMixIn
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
pass
if __name__ == "__main__":
try:
server = ThreadedHTTPServer(('', 80), MyHandler)
print 'started httpserver...'
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down server'
server.socket.close()

BaseServer is single-threaded, you should use either ForkingMixIn or ThreadingMixIn to support multiple connections.
For example replace line:
server = HTTPServer(('', 80), MyHandler)
with
from SocketServer import ThreadingMixIn
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
pass
server = ThreadedHTTPServer(('', 80), MyHandler)

EDIT: I wrote much of this before I realized Mapadd only planned to use this in a lab. WSGI probably is not required for his use case.
If you are willing to run this as a wsgi app (which I would recommend over vanilla CGI for any real scalability), you can use the script I have included below.
I took the liberty of modifying your source... this works with the assumptions above.. btw, you should spend some time checking that your html is reasonably compliant... this will help ensure that you get better cross-browser compatibility... the original didn't have <head> or <body> tags... mine (below) is strictly prototype html, and could be improved.
To run this, you just run the python executable in your shell and surf to the ipaddress of the machine on 8080. If you were doing this for a production website, we should be using lighttpd or apache for serving files, but since this is simply for lab use, the embedded wsgi reference server should be fine. Substitute the WSGIServer line at the bottom of the file if you want to run in apache or lighttpd.
Save as mp3.py
from webob import Request
import re
import os
import sys
####
#### Run with:
#### twistd -n web --port 8080 --wsgi mp3.mp3_app
_MP3DIV = """<div id="musicHere"></div>"""
_MP3EMBED = """<embed src="mp3/" loop="true" autoplay="false" width="145" height="60"></embed>"""
_HTML = '''<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head></head><body> Hello %s %s</body></html> ''' % (_MP3DIV, _MP3EMBED)
def mp3_html(environ, start_response):
"""This function will be mounted on "/" and refer the browser to the mp3 serving URL."""
start_response('200 OK', [('Content-Type', 'text/html')])
return [_HTML]
def mp3_serve(environ, start_response):
"""Serve the MP3, one chunk at a time with a generator"""
file_path = "/file/path/to/test.mp3"
mimetype = "application/x-mplayer2"
size = os.path.getsize(file_path)
headers = [
("Content-type", mimetype),
("Content-length", str(size)),
]
start_response("200 OK", headers)
return send_file(file_path, size)
def send_file(file_path, size):
BLOCK_SIZE = 4096
fh = open(file_path, 'r')
while True:
block = fh.read(BLOCK_SIZE)
if not block:
fh.close()
break
yield block
def _not_found(environ,start_response):
"""Called if no URL matches."""
start_response('404 NOT FOUND', [('Content-Type', 'text/plain')])
return ['Not Found']
def mp3_app(environ,start_response):
"""
The main WSGI application. Dispatch the current request to
the functions andd store the regular expression
captures in the WSGI environment as `mp3app.url_args` so that
the functions from above can access the url placeholders.
If nothing matches call the `not_found` function.
"""
# map urls to functions
urls = [
(r'^$', mp3_html),
(r'mp3/?$', mp3_serve),
]
path = environ.get('PATH_INFO', '').lstrip('/')
for regex, callback in urls:
match = re.search(regex, path)
if match is not None:
# assign http environment variables...
environ['mp3app.url_args'] = match.groups()
return callback(environ, start_response)
return _not_found(environ, start_response)
Run from the bash shell with: twistd -n web --port 8080 --wsgi mp3.mp3_app from the directory where you saved mp3.py (or just put mp3.py somewhere in $PYTHONPATH).
Now surf to the external ip (i.e. http://some.ip.local:8080/) and it will serve the mp3 directly.
I tried running your original app as it was posted, and could not get it to source the mp3, it barked at me with an error in linux...