How can I keep my Python HTTP server connected(streaming) to my browser in real time?
(Update image to infinity) Like raspberry pi's motion eye
class MyHttpRequestHandler(http.server.SimpleHTTPRequestHandler):
def _set_response(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.send_header("Connection", "keep-alive")
self.send_header("keep-alive", "timeout=999999, max=99999")
self.end_headers()
def do_GET(self):
#self.send_response(204)
#self.end_headers()
if self.path == '/':
self.path = 'abc.jpg'
return http.server.SimpleHTTPRequestHandler.do_GET(self)
# Create an object of the above class
handler_object = MyHttpRequestHandler
PORT = 8000
my_server = socketserver.TCPServer(("", PORT), handler_object)
# Star the server
my_server.serve_forever()
Just keep writing, as in:
while True:
self.wfile.write(b"data")
This however won't get you into eventstream / server sent events territory, without using helper external libraries, as far as I'm aware.
I came across the same issue, I then found by chance (after much debugging) that you need to send linebreaks (\r\n or \n\n) to have the packets sent:
import http.server
import time
class MyHttpRequestHandler(http.server.BaseHTTPRequestHandler):
value = 0
# One can also set protocol_version = 'HTTP/1.1' here
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.send_header("Connection", "keep-alive")
self.end_headers()
while True:
self.wfile.write(str(self.value).encode())
self.wfile.write(b'\r\n') # Or \n\n, necessary to flush
self.value += 1
time.sleep(1)
PORT = 8000
my_server = http.server.HTTPServer(("", PORT), MyHttpRequestHandler)
# Start the server
my_server.serve_forever()
This enables you to send Server-sent Events (SSE) or HTTP long poll, or even json/raw http streams with the http.server library.
As the comment in the code says, you can also set the protocol version to HTTP/1.1 to enable keepalive by default. If you do so, you will have to specify Content-Length for every sent packet, otherwise the connection will never be terminated.
It is probably best to combine this with a threaded server to allow concurrent connections, as well as maybe setting a keepalive on the socket itself.
Related
I've been working on an application which contains a small, simple http server to handle post requests on occasion. The server and all functionality around it works fine, but each time the server runs, log output would tell me that my code is being run multiple times, plus one time for each request the http server handles.
class HttpApp:
def __init__(self, host="localhost", port=25565):
logging = Util.configure_logging(__name__)
server_address = (host, port)
httpd = HTTPServer(server_address, ServerObject)
logging.info('Starting httpd...\n')
try:
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()
logging.info('Stopping httpd...\n')
class ServerObject(BaseHTTPRequestHandler):
def _set_response(self):
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
def do_GET(self):
print("GET request,\nPath: %s\nHeaders:\n%s\n", str(self.path), str(self.headers))
self._set_response()
self.wfile.write("GET request for {}".format(self.path).encode('utf-8'))
def do_POST(self):
content_length = int(self.headers['Content-Length'])
content_type = str(self.headers['Content-Type'])
# print(content_length)
post_data = self.rfile.read(content_length)
if content_type == "application/json":
parsed_data = json.loads(post_data.decode('utf-8'))
else:
print("Bad request!")
self._set_response()
self.wfile.write(json.dumps({"Response": "Bad Request"}).encode('utf-8'))
print("POST request,\nPath: %s\nHeaders:\n%s\n\nBody:\n%s\n" %
(str(self.path), str(self.headers), parsed_data))
print("Parsed Params: %s" % parsed_data)
def free_port():
free_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
free_socket.bind(('0.0.0.0', 0))
free_socket.listen(5)
port = free_socket.getsockname()[1]
free_socket.close()
return port
rand_port = free_port()
SpawnSlave(category=parsed_data["category"], tag=parsed_data["tag"], filename=parsed_data["filename"], port=rand_port)
self._set_response()
self.wfile.write(json.dumps({"port": rand_port}).encode('utf-8'))
A cli application passes information to HttpApp, which then starts based on that information. Once it receives a connection the first time, it goes through its steps normally and only prints output once. The second time, output is printed twice, and so on. Only post requests are handled by this server. I have gone over my code a few times to make sure I'm not calling it more than once, but I seem to be stumped. For more context, more of this code is available on github, but this is the only relevant piece.
It turns out that this wasn't an issue with my code, but rather an issue with the logger I was using which was adding multiple console handlers for the same logger, causing output to be repeated. I fixed this in my cli library.
I am putting together an http-post server client example in order to send and request data from a client to a server that handle multiple connections. I am using the HTTPServer module from the standard library. The code seems to work fine, but the communication slows down randomly. I have checked the communication traffic using Wireshark and I can see some strange messages going on.
I have checked different solutions on internet, but I have not found anything unusual in my code.
The code for the client it is just a simple http post request
Server code:
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
body = self.rfile.read(content_length)
data = {
'ids': [5, 6]
}
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
self.wfile.write(json.dumps(data).encode())
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
test = HTTPServer((SV_HOST, SV_PORT), Handler)
test.timeout = 5
print('Starting server, use <Ctrl-C> to stop')
test.serve_forever()
Here are the Wireshark messages that I see:
I would appreciate if someone can clarify what I am doing wrong, if there is something wrong. Is "TCP segment of a reassembled PDU" normal?
I have a very simple HTTPServer implemented in Python. The code is the following:
import SimpleHTTPServer
import SocketServer as socketserver
import os
import threading
class MyHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
path_to_image = 'RGBWebcam1.png'
img = open(path_to_image, 'rb')
statinfo = os.stat(path_to_image)
img_size = statinfo.st_size
print(img_size)
def do_HEAD(self):
self.send_response(200)
self.send_header("Content-type", "image/png")
self.send_header("Content-length", img_size)
self.end_headers()
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "image/png")
self.send_header("Content-length", img_size)
self.end_headers()
f = open(path_to_image, 'rb')
self.wfile.write(f.read())
f.close()
class MyServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_adress, RequestHandlerClass):
self.allow_reuse_address = True
socketserver.TCPServer.__init__(self, server_adress, RequestHandlerClass, False)
if __name__ == "__main__":
HOST, PORT = "192.168.2.10", 9999
server = MyServer((HOST, PORT), MyHandler)
server.server_bind()
server.server_activate()
server_thread = threading.Thread(target=server.serve_forever)
server_thread.start()
while(1):
print "test"
If I connect to the given IP-Adress the page loads and everything is fine. Now it would be nice if the page would automatically refresh every n seconds.
I am very new to python and especially new to webcoding. I have found LiveReload however I cannot get my head around how I merge these two libraries together.
Thank you for your help
You'll require a connection to the client if you want the server to tell it to refresh. A HTTP server means you've sent information (HTML) and the client will process it. There is no communication beyond that. That would require AJAX or Websockets to be implemented - both protocols that allow frequent communication.
Since you can't communicate, you should automate the refresh in the content you initially send. In our example we'll say we want the page to refresh every 30 seconds. This is possible to do in either HTML or Javascript:
<meta http-equiv="refresh" content="30" />
or
setTimeout(function(){
window.location.reload(1);
}, 30000);
I've created a RESTful server and I can successfully make requests and get responses when I use my local IP address. I would like it to be exposed externally to the internet. I set up a port forwarding rule but I cannot seem to get things working. From what I'm reading the "Connection Refused" with (7) means something is blocking it whether it's a firewall or ISP issue. Any ideas on what to do?
Here's my curl command that works with my local IP:
Pako-2:Pokebot pako$ curl -X GET http://192.168.1.8:30000/api/v1/getrecord/test
{"data": [{"id": 1, "title": "learn python"}, {"id": 2, "title": "get paid"}]}
This is what I see when I try using my external IP address given to me by What's my ip
Pako-2:Pokebot pako$ curl -X GET http://MyIpHere:30000/api/v1/getrecord/test
curl: (7) Failed to connect to myIPAddress port 30000: Connection refused
Here's the port forwarding rule I made in my router/modem
Here are my router/modem advanced settings options. I tried tweaking my firewall settings, but no luck. I tried with NAT only and also with Low Security Level with all ports checked.
-----------------Edit-----------------
Here is the port forwarding screen, should I just set 30000 as the begin and end?
Here is some python code for my server:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn
import threading
import argparse
import re
import cgi
import json
TODOS = [
{'id': 1, 'title': 'learn python'},
{'id': 2, 'title': 'get paid'},
]
class LocalData(object):
records = {}
class HTTPRequestHandler(BaseHTTPRequestHandler):
print "HTTPRequestHandler BaseHTTPRequestHandler = ", BaseHTTPRequestHandler
def do_POST(self):
print "do_POST"
if None != re.search('/api/v1/addrecord/*', self.path):
ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
if ctype == 'application/json':
length = int(self.headers.getheader('content-length'))
data = cgi.parse_qs(self.rfile.read(length), keep_blank_values=1)
recordID = self.path.split('/')[-1]
LocalData.records[recordID] = data
print "record %s is added successfully" % recordID
else:
data = {}
self.send_response(200)
self.end_headers()
else:
self.send_response(403)
self.send_header('Content-Type', 'application/json')
self.end_headers()
return
def do_GET(self):
print "do_GET"
self.send_response(200)
self.end_headers()
self.wfile.write(json.dumps({'data': TODOS}))
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
allow_reuse_address = True
def shutdown(self):
self.socket.close()
HTTPServer.shutdown(self)
class SimpleHttpServer():
def __init__(self, ip, port):
self.server = ThreadedHTTPServer((ip, port), HTTPRequestHandler)
def start(self):
self.server_thread = threading.Thread(target=self.server.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def waitForThread(self):
self.server_thread.join()
def addRecord(self, recordID, jsonEncodedRecord):
LocalData.records[recordID] = jsonEncodedRecord
def stop(self):
self.server.shutdown()
self.waitForThread()
if __name__ == '__main__':
server = SimpleHttpServer("0.0.0.0", 30000)
print 'HTTP Server Running...........'
server.start()
server.waitForThread()
------------------- Edit 2--------------------
Tried only having 1 port, not a begin/end range and it's expecting a range...
I find it strange that you'd be getting connection refused from within the local network because the router should be able to detect that the external IP is a reference to itself.
That ability may depend on the router, but that's been my experience with networking.
That being said, if you'd like to test the external firewall after playing with the port forwarding rules, then you need a device not on the local network.
If another device like a smart phone with a cell plan isn't available to you, then you could try to use CanYouSeeMe to test if the port is being opened.
You may also want to set DHCP reservation for your server. In other words, set a static IP address.
I have written this HTTP web server in python which simply sends reply "Website Coming Soon!" to the browser/client, but I want that this web server should sends back the URL given by the client, like if I write
http://localhost:13555/ChessBoard_x16_y16.bmp
then server should reply back the same url instead of "Website Coming Soon!" message.
please tell how can I do this?
Server Code:
import sys
import http.server
from http.server import HTTPServer
from http.server import SimpleHTTPRequestHandler
#import usb.core
class MyHandler(SimpleHTTPRequestHandler): #handles client requests (by me)
#def init(self,req,client_addr,server):
# SimpleHTTPRequestHandler.__init__(self,req,client_addr,server)
def do_GET(self):
response="Website Coming Soon!"
self.send_response(200)
self.send_header("Content-type", "application/json;charset=utf-8")
self.send_header("Content-length", len(response))
self.end_headers()
self.wfile.write(response.encode("utf-8"))
self.wfile.flush()
print(response)
HandlerClass = MyHandler
Protocol = "HTTP/1.1"
port = 13555
server_address = ('localhost', port)
HandlerClass.protocol_version = Protocol
try:
httpd = HTTPServer(server_address, MyHandler)
print ("Server Started")
httpd.serve_forever()
except:
print('Shutting down server due to some problems!')
httpd.socket.close()
You can do what you're asking, sort of, but it's a little complicated.
When a client (e.g., a web browser) connects to your web server, it sends a request that look like this:
GET /ChessBoard_x16_y16.bmp HTTP/1.1
Host: localhost:13555
This assumes your client is using HTTP/1.1, which is likely true of anything you'll find these days. If you expect HTTP/1.0 or earlier clients, life is much more difficult because there is no Host: header.
Using the value of the Host header and the path passed as an argument to the GET request, you can construct a URL that in many cases will match the URL the client was using.
But it won't necessarily match in all cases:
There may be a proxy in between the client and your server, in which case both the path and hostname/port seen by your code may be different from that used by the client.
There may be packet manipulation rules in place that modify the destination ip address and/or port, so that the connection seen by your code does not match the parameters used by the client.
In your do_GET method, you can access request headers via the
self.headers attribute and the request path via self.path. For example:
def do_GET(self):
response='http://%s/%s' % (self.headers['host'],
self.path)