I'm trying to figure out how to set up a basic web server in Python, but I'm having a lot of difficulty.
My main issue is that I am only able to get my server to serve the webpage once. The html displays a message in the browser and the Javascript displays another message in the console.
When I start the server and go to http://127.0.0.1:8080, both of my messages display and everything is fine. However, I run into problems when I open up a second browser tab and go there again. I get the GET HTTP request in the terminal, but not the GET Javascript request. And nothing displays in either the browser window or the console.
What am I doing wrong? Any suggestions would be appreciated.
Here is my Python code:
import BaseHTTPServer
from os import curdir, sep
htmlfile="htmltest.html"
htmlpage =open(curdir+sep+htmlfile, 'rb')
jsfile="jstest.js"
jspage=open(curdir+sep+jsfile, 'rb')
notfound = "File not found"
class WelcomeHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_OPTIONS(self):
self.send_response(200)
self.send_header('Access-Control-Allow-Origin', '*')
self.send_header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS')
self.send_header("Access-Control-Allow-Headers", "X-Requested-With")
def do_GET(self):
if self.path == "/":
print "get html"
self.send_response(200)
self.send_header("Content-type","text/html")
self.end_headers()
self.wfile.write(htmlpage.read())
elif self.path=="/jstest.js":
print "get js"
self.send_response(200)
self.send_header("Content-type","text/js")
self.end_headers()
self.wfile.write(jspage.read())
else:
self.send_error(404, notfound)
httpserver = BaseHTTPServer.HTTPServer(("127.0.0.1",8080), WelcomeHandler)
#httpserver.serve_forever()
while True:
httpserver.handle_request()
When you open a file in Python and read its contents, the "file pointer" (i.e. where the next read will start from) is then at the end of the file. You'll either have to re-open it or rewind to the beginning of the file in order to read it again.
Unless you expect your files to change frequently, you might want to just read them at the start and store the contents in a variable, then serve that. Alternatively, you could move your opening into your do_GET method so it opens it fresh for each request.
Related
I am using Python HttpServer in the server side. One GET request will take more time to respond and I want to update the user the current status of it, such as 'Fetching module X. Please wait', 'Fetching module Y. Please wait'.
But, it is not getting updated in the client side even though I sending it in between the modules. I have tried flushing the stream, but no luck.
self.wfile.write('Fetching module X. Please wait')
self.wfile.flush()
How can I force the HttpServer to send the information instantly, instead of waiting for the completion of full response ?
You can use python threading
from threading import Thread
t = threading.Thread(target=function to be call, args=[request])
t.setDaemon(False)
t.start()
This code will force to return response instantly and run your function in background.
Suggest you put the user indication to header not body. Then you can use stream to reach your requirements.
NOTE: Next code is base on python2, you can change http server related to python3 related api if you like.
server.py:
import BaseHTTPServer
import time
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
Page = "Main content here."
def do_GET(self):
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.send_header("Content-Length", str(len(self.Page)))
self.send_header("User-Information", "Fetching module X. Please wait")
self.end_headers()
time.sleep(10)
self.wfile.write(self.Page)
if __name__ == '__main__':
serverAddress = ('', 8080)
server = BaseHTTPServer.HTTPServer(serverAddress, RequestHandler)
server.serve_forever()
client.py:
import requests
r = requests.get('http://127.0.0.1:8080', stream=True)
print(r.headers['User-Information'])
print(r.content)
Explain:
If use stream, the header information will still be fetched by client at once, so you can print it to user at once with print(r.headers['User-Information'])
But with stream, the body information will not transmit, it's be delayed until client use r.content to require it(Response.iter_lines() or Response.iter_content() also ok), so when you do print(r.content), it will need 10 seconds to see the main content as it cost 10s in server code.
Output: (The first line will be shown to user at once, but the second line will be shown 10 seconds later)
Fetching module X. Please wait
Main content here.
Attach the guide for your reference, hope it helpful.
I want to write some data, e.g. "hello" on a file that is located on a remote server, not a local server. This is the code that I used to read from server:
import urllib2
var = "hello"
url = "http://url:port/log/log.txt"
response = urllib2.urlopen(url)
txt = response.read();
print txt
As an output I was able to get the data from the log file.
Now I want to write some data, e.g. "hello" onto the same file. How to achieve that?
What your code is doing is actually not "reading a file" but sending an HTTP get request to a given url and print the HTTP response's body. What you get as content for this url is up to the HTTP server serving this url, and the fact it actually comes from a file stored on this server is totally irrelevant - it might as well come from a database, from another web server, or be typed in real time by a monkey FWIW.
If you want to use the HTTP protocol to modify the content of a given file on this server, the server must provide this service (as a given url where you're supposed to send a POST or PUT http request with the new content).
In my python application I have to read many web pages to collect data. To decrease the http calls I would like to fetch only changed pages. My problem is that my code always tells me that the pages have been changed (code 200) but in reality it is not.
This is my code:
from models import mytab
import re
import urllib2
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
def url_change():
urls = mytab.objects.all()
# this is some urls:
# http://www.venere.com/it/pensioni/venezia/pensione-palazzo-guardi/#reviews
# http://www.zoover.it/italia/sardegna/cala-gonone/san-francisco/hotel
# http://www.orbitz.com/hotel/Italy/Venice/Palazzo_Guardi.h161844/#reviews
# http://it.hotels.com/ho292636/casa-del-miele-susegana-italia/
# http://www.expedia.it/Venezia-Hotel-Palazzo-Guardi.h1040663.Hotel-Information#reviews
# ...
for url in urls:
request = urllib2.Request(url.url)
if url.last_date == None:
now = datetime.now()
stamp = mktime(now.timetuple())
url.last_date = format_date_time(stamp)
url.save()
request.add_header("If-Modified-Since", url.last_date)
try:
response = urllib2.urlopen(request) # Make the request
# some actions
now = datetime.now()
stamp = mktime(now.timetuple())
url.last_date = format_date_time(stamp)
url.save()
except urllib2.HTTPError, err:
if err.code == 304:
print "nothing...."
else:
print "Error code:", err.code
pass
I do not understand what has gone wrong. Can anyone help me?
Web servers aren't required to send a 304 header as the response when you send an 'If-Modified-Since' header. They're free to send a HTTP 200 and send the entire page again.
Sending a 'If-Modified-Since' or 'If-None-Since' alerts the server that you'd like a cached response if available. It's like sending an 'Accept-Encoding: gzip, deflate' header -- you're just telling the server you'll accept something, not requiring it.
A good way to check if a site returns 304 is to use google chromes dev tools. E.g. below is an annotated example of using chrome on the bls website. Keep refreshing and you will see that the server keeps returning 304. If you force refresh with Ctrl+F5 (windows), you will see that instead it returns status code 200.
You can use this technique on your example to find out if the server does not return 304, or if you have incorrectly formatted your request headers somehow. Sometimes a webpage has a resource imported on to it which does not respect the If- headers and so it returns 200 whatever you do (If any resource on the page does not return 304, the whole page will return 200), but sometimes you are only looking at a specific part of a website and you can cheat by loading the resource directly and bypassing the whole document.
I'm using SimpleHTTPServer to test some webpages I'm working on. It works great, however I need to do some cross-domain requests. That requires setting a Access-Control-Allow-Origin header with the domains the page is allowed to access.
Is there an easy way to set a header with SimpleHTTPServer and serve the original content? The header would be the same on each request.
This is a bit of a hack because it changes end_headers() behavior, but I think it's slightly better than copying and pasting the entire SimpleHTTPServer.py file.
My approach overrides end_headers() in a subclass and in it calls send_my_headers() followed by calling the superclass's end_headers().
It's not 1 - 2 lines either, less than 20 though; mostly boilerplate.
#!/usr/bin/env python
try:
from http import server # Python 3
except ImportError:
import SimpleHTTPServer as server # Python 2
class MyHTTPRequestHandler(server.SimpleHTTPRequestHandler):
def end_headers(self):
self.send_my_headers()
server.SimpleHTTPRequestHandler.end_headers(self)
def send_my_headers(self):
self.send_header("Access-Control-Allow-Origin", "*")
if __name__ == '__main__':
server.test(HandlerClass=MyHTTPRequestHandler)
I'd say there's no simple way of doing it, where simple means "just add 1-2 lines that will write the additional header and keep the existing functionality". So, the best solution would be to subclass the SimpleHTTPRequestHandler class and re-implement the functionality, with the addition of the new header.
The problem behind why there is no simple way of doing this can be observed by looking at the implementation of the SimpleHTTPRequestHandler class in the Python library: http://hg.python.org/cpython/file/19c74cadea95/Lib/http/server.py#l654
Notice the send_head() method, particularly the lines at the end of the method which send the response headers. Notice the invocation of the end_headers() method. This method writes the headers to the output, together with a blank line which signals the end of all headers and the start of the response body: http://docs.python.org/py3k/library/http.server.html#http.server.BaseHTTPRequestHandler.end_headers
Therefore, it would not be possible to subclass the SimpleHTTPRequestHandler handler, invoke the super-class do_GET() method, and then just add another header -- because the sending of the headers has already finished when the call to the super-class do_GET() method returns. And it has to work like this because the do_GET() method has to send the body (the file that is requested), and to send the body - it has to finalize sending the headers.
So, again, I think you're stuck with sub-classing the SimpleHTTPRequestHandler class, implementing it exactly as the code in the library (just copy-paste it?), and add another header before the call to the end_headers() method in send_head():
...
self.send_header("Last-Modified", self.date_time_string(fs.st_mtime))
# this below is the new header
self.send_header('Access-Control-Allow-Origin', '*')
self.end_headers()
return f
...
# coding: utf-8
import SimpleHTTPServer
import SocketServer
PORT = 9999
def do_GET(self):
self.send_response(200)
self.send_header('Access-Control-Allow-Origin', 'http://example.com')
self.end_headers()
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
Handler.do_GET = do_GET
httpd = SocketServer.TCPServer(("", PORT), Handler)
httpd.serve_forever()
While this is an older answer, its the first result in google...
Basically what #iMon0 suggested..Seems correct?..Example of doPOST
def do_POST(self):
self.send_response()
self.send_header('Content-type','application/json')
self.send_header('Access-Control-Allow-Origin','*')
self.end_headers()
sTest = {}
sTest['dummyitem'] = "Just an example of JSON"
self.wfile.write(json.dumps(sTest))
By doing this, the flow feels correct..
1: You get a request
2: You apply the headers and response type you want
3: You post back the data you want, be this what or how ever you want.,
The above example is working fine for me and can be extended further, its just a bare bone JSON post server. So i'll leave this here on SOF incase someone needs it or i myself come back in a few months for it.
This does produce a valid JSON file with only the sTest object, Same as a PHP generated page/file.
I wanted to design a simple site where one person can upload a file, and pass off the random webaddress to someone, who can then download it.
At this point, I have a webpage where someone can successfully upload a file which gets stored under /files/ on my webserver.
The python script also generates a unique, random 5 letter code that gets stored in a database identifying the file
I have another page called retrieve, where a person should go, put in the 5 letter code, and it should pop up a filebox asking where to save the file.
My Problem is that: 1) How do I retrieve the file for download? At this point my retrieve script, takes the code, gets the location of the file on my server, but how do I get the brower to start downloading?
2)How do I stop people from directly going to the file? Should I change permissions on the file?
How do you serve the file-upload page, and how do you let your users upload files?
If you are using Python's built-in HTTP server modules you shouldn't have any problems.
Anyway, here's how the file serving part is done using Python's built-in modules (just the basic idea).
Regarding your second question, if you were using these modules in the first place you probably wouldn't have asked it because you'd have to explicitly serve specific files.
import SocketServer
import BaseHTTPServer
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
# The URL the client requested
print self.path
# analyze self.path, map the local file location...
# open the file, load the data
with open('test.py') as f: data = f.read()
# send the headers
self.send_response(200)
self.send_header('Content-type', 'application/octet-stream') # you may change the content type
self.end_headers()
# If the file is not found, send error code 404 instead of 200 and display a message accordingly, as you wish.
# wfile is a file-like object. writing data to it will send it to the client
self.wfile.write(data)
# XXX: Obviously, you might want to send the file in segments instead of loading it as a whole
if __name__ == '__main__':
PORT = 8080 # XXX
try:
server = SocketServer.ThreadingTCPServer(('', 8080), RequestHandler)
server.serve_forever()
except KeyboardInterrupt:
server.socket.close()
You should send the right HTTP Response, containing the binary data and making the browser react on it.
Try this (I haven't) if you're using Django:
response = HttpResponse()
response['X-Sendfile'] = os.path.join(settings.MEDIA_ROOT, file.file.path)
content_type, encoding = mimetypes.guess_type(file.file.read())
if not content_type:
content_type = 'application/octet-stream'
response['Content-Type'] = content_type
response['Content-Length'] = file.file.size
response['Content-Disposition'] = 'attachment; filename="%s"' % file.file.name
return response
Source: http://www.chicagodjango.com/blog/permission-based-file-serving/