Develop Reference

Communicating with a server on a device using that server as a wifi hotspot

I have some python server code running on a laptop, and am trying to send GET/POST to the server's URL. This works fine using a python client or browser on the laptop, as well as on other devices connected to my home network.
However, I'd like to now use that laptop as a wifi hotspot, and connect to the server using devices that are connecting to that hotspot; for example, connecting to the hotspot on a phone, opening a browser, and entering the IP address and port into the address bar (eg http://192.181.1.XXX:80XX/example).
The URL fails to resolve only on devices using the server as a hotspot; switching back to the normal wifi network without changing the URL then successfully resolves, and a browser on the server itself can access it fine.
Do I need to change the URL in some manner when using the target server as a wifi hotspot, or is there something else I'm missing? Or is this actually impossible due to some networking reasons?
In case it's relevant, here is the server code:
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
import cgi
import time
hostName = "192.168.1.XXX"
serverPort = 80XX
class MyServer(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
data = json.dumps({'wifi period': '60', 'poll period': '1', '#':'1'})
self.wfile.write(bytes(data, "utf-8"))
def do_POST(self):
if hasattr(self.headers, 'getheader'):
ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
print(ctype)
if ctype == 'application/x-www-form-urlencoded':
#todo: save file
return
if ctype != 'application/json':
print("invalid ctype.")
print(ctype)
self.send_response(400)
self.end_headers()
return
length = int(self.headers.getheader('content-length'))
else:
header = self.headers.get('content-type')
if header == 'application/x-www-form-urlencoded':
length = int(self.headers.get('content-length'))
content = self.rfile.read(length)
#TODO: save file
self.send_response(200)
self.end_headers()
return
elif header != 'application/json':
print(f"Header: {header}")
self.send_response(400)
self.end_headers()
return
length = int(self.headers.get('content-length'))
# read the message and convert it into a python dictionary
message = json.loads(self.rfile.read(length))
print(message)
self.send_response(200)
self.end_headers()
return
if __name__ == "__main__":
webServer = HTTPServer((hostName, serverPort), MyServer)
print("Server started http://%s:%s" % (hostName, serverPort))
try:
webServer.serve_forever()
except KeyboardInterrupt:
pass
webServer.server_close()
print("Server stopped.")

Web server in python in plainText

I am looking for a way to expose a text file with Python web server.
I get some python code to run a web server :
import http.server
import socketserver
port = 9500
address = ("", port)
handler = http.server.SimpleHTTPRequestHandler
httpd = socketserver.TCPServer(address, handler)
print(f"Serveur démarré sur le PORT {port}")
httpd.serve_forever()
It's working fine. but i would :
Run a web sever exposing textplain content (and not Html content).
Set manually the workpath and name of index file (default: index.html)
keep Python server Code simple and light
I found some help on the web :
handler.extensions_map['Content-type'] = 'text/plain'
or
handler.send_header('Content-Type','text/plain')
But none os this proposition work.
Could you help me to build a simple python code to do this ?
Thanks a lot,

Script for Python 2 with using only built-in modules, just place the absolute path of the file which you want to be served <INSERT_FILE>:
#!/usr/bin/python
from SimpleHTTPServer import SimpleHTTPRequestHandler
import BaseHTTPServer
from io import StringIO
import sys
import os
class MyHandler(SimpleHTTPRequestHandler):
def send_head(self):
# Place here the absolute path of the file
with open("<INSERT_FILE>", "r") as f:
body = unicode("".join( f.readlines()))
self.send_response(200)
self.send_header("Content-type", "text/html; charset=UTF-8")
self.send_header("Content-Length", str(len(body)))
#self.send_header("Server", "SimpleHTTP/1.1 Python/2.7.5")
self.end_headers()
# text I/O binary, and raw I/O binary
# initial value must be unicode or None
return StringIO(body)
if __name__ == "__main__":
HandlerClass = MyHandler
ServerClass = BaseHTTPServer.HTTPServer
Protocol = "HTTP/1.1"
server_address = ('', 5555)
HandlerClass.protocol_version = Protocol
httpd = ServerClass (server_address, HandlerClass)
print("serving on port 5555")
httpd.serve_forever()
For python3 (SimpleHTTPServer module has been merged into http.server), place absolute path <INSERT_FILE>:
from http.server import HTTPServer, BaseHTTPRequestHandler
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
# place absolute path here
f_served = open('<INSERT_FILE>','rb')
f_content = f_served.read()
f_served.close()
self.wfile.write(f_content)
if __name__ == "__main__":
httpd = HTTPServer(('localhost', 5555), SimpleHTTPRequestHandler)
httpd.serve_forever()

I recommend using aiohttp with its lowlevel server, which is described here:
You can either return plain text, or you change the content type of your web.Response to text/html to send data that will be interpreted as html.
You can just replace the "OK" in the text="OK" with whatever plain text you wish. Or you replace it with the content of your *.html and change the content_type.
import asyncio
from aiohttp import web
async def handler(request):
return web.Response(text="OK")
async def main():
server = web.Server(handler)
runner = web.ServerRunner(server)
await runner.setup()
site = web.TCPSite(runner, 'localhost', 8080)
await site.start()
print("======= Serving on http://127.0.0.1:8080/ ======")
# pause here for very long time by serving HTTP requests and
# waiting for keyboard interruption
await asyncio.sleep(100*3600)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main())
except KeyboardInterrupt:
pass
loop.close()

AJAX HTTP Request and Python web server HTTP response return

I'm just new in integrating ajax http request and return a python webserver http response. I really no have an idea how to use it.
For example my web page is on another IP . like 192.168.1.1 and i will get a data or a response from 192.168.1.2
then on my view:
function test(){
$.ajax({
url : "http://192.168.1.2:8012/",
type : "GET",
success : function(data) {
alert(data);
}
});
}
now on my python web server
import string,cgi,time
from os import curdir, sep
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import MySQLdb
from lxml import etree
from lxml.builder import E as buildE
import urllib
global db, cnn
db = MySQLdb.connect("localhost","root","password","schema" )
cnn = db.cursor()
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
global cnn, sql
self.wfile.write("Cannot GET "+self.path)
print "test"
self.send_response(200, "testing")
self.send_header('Content-type', 'xml')
self.end_headers()
self.wfile.write("testing")
def do_POST(self):
global rootnode
try:
ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
if ctype == 'multipart/form-data':
query=cgi.parse_multipart(self.rfile, pdict)
self.send_response(301)
self.end_headers()
upfilecontent = query.get('upfile')
print "filecontent", upfilecontent[0]
self.wfile.write("<HTML>POST OK.<BR><BR>");
self.wfile.write(upfilecontent[0]);
except :
pass
def main():
try:
server = HTTPServer(('', 8012), MyHandler)
print 'started httpserver...'
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down server'
server.socket.close()
if __name__ == '__main__':
main()
I just want a returned data from the webserver. But i think im doing it wrong.

I think you shouldn't send data to self.wfile before sending ALL headers. And maybe you should send a 'Content-length' header to let your page know when it must stop waiting for data. Something like this:
data="Cannot GET "+self.path
self.send_response(200)
self.send_header('Content-type','text/xml')
self.send_header('Content-length',str(len(data))
self.end_headers()
self.wfile.write(data)

Multithreaded web server in python

I'm trying to create multithreaded web server in python, but it only responds to one request at a time and I can't figure out why. Can you help me, please?
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
from time import sleep
class ThreadingServer(ThreadingMixIn, HTTPServer):
pass
class RequestHandler(SimpleHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/plain')
sleep(5)
response = 'Slept for 5 seconds..'
self.send_header('Content-length', len(response))
self.end_headers()
self.wfile.write(response)
ThreadingServer(('', 8000), RequestHandler).serve_forever()

Check this post from Doug Hellmann's blog.
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import threading
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
message = threading.currentThread().getName()
self.wfile.write(message)
self.wfile.write('\n')
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
if __name__ == '__main__':
server = ThreadedHTTPServer(('localhost', 8080), Handler)
print 'Starting server, use <Ctrl-C> to stop'
server.serve_forever()

I have developed a PIP Utility called ComplexHTTPServer that is a multi-threaded version of SimpleHTTPServer.
To install it, all you need to do is:
pip install ComplexHTTPServer
Using it is as simple as:
python -m ComplexHTTPServer [PORT]
(By default, the port is 8000.)

In python3, you can use the code below (https or http):
from http.server import HTTPServer, BaseHTTPRequestHandler
from socketserver import ThreadingMixIn
import threading
USE_HTTPS = True
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
self.wfile.write(b'Hello world\t' + threading.currentThread().getName().encode() + b'\t' + str(threading.active_count()).encode() + b'\n')
class ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
pass
def run():
server = ThreadingSimpleServer(('0.0.0.0', 4444), Handler)
if USE_HTTPS:
import ssl
server.socket = ssl.wrap_socket(server.socket, keyfile='./key.pem', certfile='./cert.pem', server_side=True)
server.serve_forever()
if __name__ == '__main__':
run()
You will figure out this code will create a new thread to deal with every request.
Command below to generate self-sign certificate:
openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
If you are using Flask, this blog is great.

It's amazing how many votes these solutions that break streaming are getting. If streaming might be needed down the road, then ThreadingMixIn and gunicorn are no good because they just collect up the response and write it as a unit at the end (which actually does nothing if your stream is infinite).
Your basic approach of combining BaseHTTPServer with threads is fine. But the default BaseHTTPServer settings re-bind a new socket on every listener, which won't work in Linux if all the listeners are on the same port. Change those settings before the serve_forever() call. (Just like you have to set self.daemon = True on a thread to stop ctrl-C from being disabled.)
The following example launches 100 handler threads on the same port, with each handler started through BaseHTTPServer.
import time, threading, socket, SocketServer, BaseHTTPServer
class Handler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
if self.path != '/':
self.send_error(404, "Object not found")
return
self.send_response(200)
self.send_header('Content-type', 'text/html; charset=utf-8')
self.end_headers()
# serve up an infinite stream
i = 0
while True:
self.wfile.write("%i " % i)
time.sleep(0.1)
i += 1
# Create ONE socket.
addr = ('', 8000)
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(addr)
sock.listen(5)
# Launch 100 listener threads.
class Thread(threading.Thread):
def __init__(self, i):
threading.Thread.__init__(self)
self.i = i
self.daemon = True
self.start()
def run(self):
httpd = BaseHTTPServer.HTTPServer(addr, Handler, False)
# Prevent the HTTP server from re-binding every handler.
# https://stackoverflow.com/questions/46210672/
httpd.socket = sock
httpd.server_bind = self.server_close = lambda self: None
httpd.serve_forever()
[Thread(i) for i in range(100)]
time.sleep(9e9)

A multithreaded https server in python3.7
from http.server import BaseHTTPRequestHandler, HTTPServer
from socketserver import ThreadingMixIn
import threading
import ssl
hostName = "localhost"
serverPort = 8080
class MyServer(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(bytes("<html><head><title>https://pythonbasics.org</title></head>", "utf-8"))
self.wfile.write(bytes("<p>Request: %s</p>" % self.path, "utf-8"))
self.wfile.write(bytes("<p>Thread: %s</p>" % threading.currentThread().getName(), "utf-8"))
self.wfile.write(bytes("<p>Thread Count: %s</p>" % threading.active_count(), "utf-8"))
self.wfile.write(bytes("<body>", "utf-8"))
self.wfile.write(bytes("<p>This is an example web server.</p>", "utf-8"))
self.wfile.write(bytes("</body></html>", "utf-8"))
class ThreadingSimpleServer(ThreadingMixIn,HTTPServer):
pass
if __name__ == "__main__":
webServer = ThreadingSimpleServer((hostName, serverPort), MyServer)
webServer.socket = ssl.wrap_socket(webServer.socket, keyfile='./privkey.pem',certfile='./certificate.pem', server_side=True)
print("Server started http://%s:%s" % (hostName, serverPort))
try:
webServer.serve_forever()
except KeyboardInterrupt:
pass
webServer.server_close()
print("Server stopped.")
you can test it in a browser: https://localhost:8080
the running result is:
enter image description here
enter image description here
remind that you can generate your own keyfile and certificate use
$openssl req -newkey rsa:2048 -keyout privkey.pem -x509 -days 36500 -out certificate.pem
To learn details about creating self-signed certificate with openssl:https://www.devdungeon.com/content/creating-self-signed-ssl-certificates-openssl

Python basehttpserver not serving requests properly

I'm trying to write down a simple local proxy for javascript: since I need to load some stuff from javascript within a web page, I wrote this simple daemon in python:
import string,cgi,time
from os import curdir, sep
import urllib
import urllib2
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
class MyHandler(BaseHTTPRequestHandler):
def fetchurl(self, url, post, useragent, cookies):
headers={"User-Agent":useragent, "Cookie":cookies}
url=urllib.quote_plus(url, ":/?.&-=")
if post:
req = urllib2.Request(url,post,headers)
else:
req=urllib2.Request(url, None, headers)
try:
response=urllib2.urlopen(req)
except urllib2.URLError, e:
print "URLERROR: "+str(e)
return False
except urllib2.HTTPError, e:
print "HTTPERROR: "+str(e)
return False
else:
return response.read()
def do_GET(self):
if self.path != "/":
[callback, url, post, useragent, cookies]=self.path[1:].split("%7C")
print "callback = "+callback
print "url = "+url
print "post = "+post
print "useragent = "+useragent
print "cookies = "+cookies
if useragent=="":
useragent="pyjproxy v. 1.0"
load=self.fetchurl(url, post, useragent, cookies)
pack=load.replace("\\", "\\\\").replace("\"", "\\\"").replace("\n", "\\n").replace("\r", "\\r").replace("\t", "\\t").replace(" </script>", "</scr\"+\"ipt>")
response=callback+"(\""+pack+"\");"
if load:
self.send_response(200)
self.send_header('Content-type', 'text/javascript')
self.end_headers()
self.wfile.write(response)
self.wfile.close()
return
else:
self.send_error(404,'File Not Found: %s' % self.path)
return
else:
embedscript="function pyjload(datadict){ if(!datadict[\"url\"] || !datadict[\"callback\"]){return false;} if(!datadict[\"post\"]) datadict[\"post\"]=\"\"; if(!datadict[\"useragent\"]) datadict[\"useragent\"]=\"\"; if(!datadict[\"cookies\"]) datadict[\"cookies\"]=\"\"; var oHead = document.getElementsByTagName('head').item(0); var oScript= document.createElement(\"script\"); oScript.type = \"text/javascript\"; oScript.src=\"http://localhost:1180/\"+datadict[\"callback\"]+\"%7C\"+datadict[\"url\"]+\"%7C\"+datadict[\"post\"]+\"%7C\"+datadict[\"useragent\"]+\"%7C\"+datadict[\"cookies\"]; oHead.appendChild( oScript);}"
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(embedscript)
self.wfile.close()
return
def main():
try:
server = HTTPServer(('127.0.0.1', 1180), MyHandler)
print 'started httpserver...'
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down server'
server.socket.close()
if __name__ == '__main__':
main()
And I use within a web page like this one:
<!DOCTYPE HTML>
<html><head>
<script>
function miocallback(htmlsource)
{
alert(htmlsource);
}
</script>
<script type="text/javascript" src="http://localhost:1180"></script>
</head><body>
<a onclick="pyjload({'url':'http://www.google.it','callback':'miocallback'});"> Take the Red Pill</a>
</body></html>
Now, on Firefox and Chrome looks like it works always. On Opera and Internet Explorer, however, I noticed that sometimes it doesn't work, or it hangs for a lot of time... what's up, I wonder? Did I misdo something?
Thank for any help!
Matteo

You have to understand that (modern) browsers try to optimize their browsing speed using different techniques, which is why you get different results on different browsers.
In your case, the technique that caused you trouble is concurrent HTTP/1.1 session setup: in order to utilize your bandwidth better, your browser is able to start several HTTP/1.1 sessions at the same time. This allows to retrieve multiple resources (e.g. images) simultaneously.
However, BaseHTTPServer is not threaded: as soon as your browser tries to open another connection, it will fail to do so because BaseHTTPServer is already blocked by the first session that's still open. The request will never reach the server and run into a timeout. This also means that only one user can access your service at a given time. Inconvenient? Aye, but help is here:
Threads! .. and python makes this one rather easy:
Derive a new class from HTTPServer using a MixIn from socketserver.
.
Example:
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import threading
class Handler(BaseHTTPRequestHandler):
def do_HEAD(self):
pass
def do_GET(self):
pass
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
""" This class allows to handle requests in separated threads.
No further content needed, don't touch this. """
if __name__ == '__main__':
server = ThreadedHTTPServer(('localhost', 80), Handler)
print 'Starting server on port 80...'
server.serve_forever()
From now on, BaseHTTPServer is threaded and ready to serve multiple connections ( and therefore requests ) at the same time which will solve your problem.
Instead of the ThreadingMixIn, you can also use the ForkingMixIn in order to spawn another process instead of another thread.
all the best,
creo

Note that Python basehttpserver is a very basic HTTP server far to be perfect, but that's not your first issue.
What is happening if you put the two scripts at the end of the document just before the </body> tag? Does it help?