I would like to make a HTTP server using python, to host my website. I've implemented do_GET() method and it works fine, but when I try the POST method, server is not responding until I reload the page. Then it wakes up, tring to complete the POST, but realizing that the connection has ended. After that it serves the GET method (because I reloaded the page) and continue running normally.
I'm using jQuery on client side, but i also tried html form. It's no change even if i open the page in another browser, so it must be an exception on server side.
Here is the client javascript:
function Send()
{
jQuery.post("address", "Blah!", function (data) {
//get response
alert(data);
});
}
And here is the server code (I'm using python 3.3.4):
class MyHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
def do_POST(self):
print("Data: " + str(self.rfile.read(), "utf-8")) #show request
response = bytes("This is the response.", "utf-8") #create response
self.send_response(200) #create header
self.send_header("Content-Length", str(len(response)))
self.end_headers()
self.wfile.write(response) #send response
This is the log from server:
10.175.72.200 - - [01/Apr/2014 19:11:26] "GET / HTTP/1.1" 200 -
Data: Blah!
10.175.72.200 - - [01/Apr/2014 19:11:47] "POST /address HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('10.175.72.200', 2237)
Traceback (most recent call last):
File "C:\Python33\lib\socketserver.py", line 306, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Python33\lib\socketserver.py", line 332, in process_request
self.finish_request(request, client_address)
File "C:\Python33\lib\socketserver.py", line 345, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python33\lib\socketserver.py", line 666, in __init__
self.handle()
File "C:\Python33\lib\http\server.py", line 400, in handle
self.handle_one_request()
File "C:\Python33\lib\http\server.py", line 388, in handle_one_request
method()
File "C:\Users\Jirek\Desktop\Server\server.py", line 45, in do_POST
self.wfile.write(response) #send response
File "C:\Python33\lib\socket.py", line 317, in write
return self._sock.send(b)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
----------------------------------------
10.175.72.200 - - [01/Apr/2014 19:11:47] "GET / HTTP/1.1" 200 -
You can see, on the first line is classical GET request. Then I call the POST from browser and nothing happens until the reload. At that point the server writes the rest of the log.
Problem solved! It's in the line where the code loads received data (in do_POST()). I just had to specify the amount of data to read.
So, just don't use rfile.read(), but rfile.read(numberOfBytes). How big is the POST message is saved in headers: numberOfBytes = int(self.headers["Content-Length"])
The repaired method:
def do_POST(self):
length = int(self.headers["Content-Length"])
print("Data: " + str(self.rfile.read(length), "utf-8"))
response = bytes("This is the response.", "utf-8") #create response
self.send_response(200) #create header
self.send_header("Content-Length", str(len(response)))
self.end_headers()
self.wfile.write(response) #send response
Related
I create a simple python http server and I want it to display only files from a directory I want to, thus changing always returning "Hello world!" and also how could I handle the broken pipe errors? I tried to do a try catch there but I'm not sure if it's working:
#!/usr/bin/python
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
PORT_NUMBER = 8089
#This class will handles any incoming request from
#the browser
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write("Hello World !")
return
try:
#Create a web server and define the handler to manage the
#incoming request
server = HTTPServer(('', PORT_NUMBER), myHandler)
print 'Started http server on port ' , PORT_NUMBER
#Wait forever for incoming http requests
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down the web server'
server.socket.close()
except socket.error:
pass
This is my Error:
someIP - - [25/Dec/2019 09:17:11] "GET someFILE HTTP/1.0" 200 -
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 293, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 657, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 716, in finish
self.wfile.close()
File "/usr/lib/python2.7/socket.py", line 283, in close
self.flush()
File "/usr/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
Example which follows is the basic example of creating basic web server for serving the static files. Additional comments you can find inside the code, with one additional note: 403 Forbidden implementation can be replaced with file indexing page, for what you need to do the additional generation of it (according to your question, this is out of scope for now.)
from http.server import HTTPServer, BaseHTTPRequestHandler
from os import curdir, sep, path
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200) #Request received, sending OK status
self.end_headers()
try:
if(path.isdir(self.path)): #Checking if user requested access to dirrectory (not to a particular file)
self.send_response(403)
self.wfile.write(str.encode("Listing of directories not permited on this server")) #Preventing directory listing, in case you dont want to allow file indexing.
else: #If user is requesting access to a file, file content is read and displayed.
f = open(curdir + sep + self.path, 'rb')
self.wfile.write(f.read())
f.close()
except IOError: #Covering the 404 error, in case user requested non-existing file
print("File "+self.path+" not found")
self.send_response(404)
httpd = HTTPServer(('localhost', 8000), SimpleHTTPRequestHandler)
httpd.serve_forever()
I used to get the same error earlier
My Code :
from signal import signal, SIGPIPE, SIG_DFL
signal(SIGPIPE,SIG_DFL)
Paste it in the end and it should work!
I'm trying to make a POST request script in Python which will send data to database automatically. Since my web application (made in Django) is still in progress (on localhost) I tried to send this POST request from another computer on same network via IPv4:port.
While doing this I get HTTP Error 500: Internal server error with traceback to the line "response = urllib2.urlopen(req)"
My script looks like this:
import urllib, urllib2, cookielib
import re
cookie_jar = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie_jar))
urllib2.install_opener(opener)
url_1 = 'http://192.168.1.104:8000/legacy'
req = urllib2.Request(url_1)
rsp = urllib2.urlopen(req)
url_2 = 'http://192.168.1.104:8000/legacy'
params = {
'temperature_max' : '186',
'temperature_min' : '88',
'submit': 'Submit',
}
data = urllib.urlencode(params)
req = urllib2.Request(url_2, data)
response = urllib2.urlopen(req)
the_page = response.read()
pat = re.compile('Title:.*')
print pat.search(the_page).group()
On the other computer that is hosting the server I get the following error:
Exception happened during processing of request from ('192.168.1.65', 56996)
Traceback (most recent call last):
File "c:\python27\Lib\SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "c:\python27\Lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\Alex\Envs\rango\lib\site-packages\django\core\servers\basehttp.
py", line 129, in __init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "c:\python27\Lib\SocketServer.py", line 657, in __init__
self.finish()
File "c:\python27\Lib\SocketServer.py", line 716, in finish
self.wfile.close()
File "c:\python27\Lib\socket.py", line 283, in close
self.flush()
File "c:\python27\Lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10054] An existing connection was forcibly closed by the remote host
EDIT: I'd like to let you know that I can post data from the other computer to my database if I connect to my app with my browser.
You will need to get a session cookie and then read the generated cookie for that session.
The best way to go about that is to fetch the page with the form first, saving all cookies. You can use cookielib for that as demonstrated here. If you want to make life easier for yourself, use requests as well instead of urllib. The code then becomes a lot simpler.
To get the CSRF token, you can scrape the form page with BeautifulSoup. There are lots of tutorials for this which you can easily find on Google.
Try:
import urllib.request
from urllib.error import HTTPError
url = "<enter_URL_here>"
try:
res = urllib.request.urlopen(url)
except HTTPError as e:
content = e.read()
print("HTTP error encountered while reading URL")
I have a daemon which uploads json to http://localhost:9000, I can read json easily using hapi with this code:
'use strict';
const Hapi = require('hapi');
const server = new Hapi.Server();
server.connection({
host: 'localhost',
port: 9000
});
server.route({
method: 'POST',
path:'/push/',
handler: function (request, reply) {
console.log('Server running at:', request.payload);
return reply('hello world');
}
});
server.start((err) => {
if (err) {
throw err;
}
console.log('Server running at:', server.info.uri);
});
but I'm trying to handle this in python, I found a answer here, so here is my code:
import urllib, json
url = 'http://localhost:9000/push/'
response = urllib.urlopen(url)
data = json.load(response.read())
print data
but when I run it I get Exception:
Traceback (most recent call last):
File "pythonServer.py", line 3, in <module>
response = urllib.urlopen(url)
File "/usr/lib/python2.7/urllib.py", line 87, in urlopen
return opener.open(url)
File "/usr/lib/python2.7/urllib.py", line 213, in open
return getattr(self, name)(url)
File "/usr/lib/python2.7/urllib.py", line 350, in open_http
h.endheaders(data)
File "/usr/lib/python2.7/httplib.py", line 997, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 812, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 793, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
IOError: [Errno socket error] [Errno 111] Connection refused
How can I solve this?
Your server route is expecting a POST request.
Your Python code is sending a GET request.
Notice that your JavaScript code is using the following:
method: 'POST'
Your Python code by contrast will be default to a GET method, since you didn't explicitly specify it. You can look here for how to make a POST request using urllib, but I would recommend you look into using the requests library instead.
It's a simple as
import requests
r = requests.post("http://localhost:9000/push/", data = {"key":"value"})
Alternatively, just change your server (JavaScript/Node.js code) to use method: 'GET'.
Your server is sending down POST request and you are fetching a GET call.
response = urllib.urlopen(url)
urllib.urlopen opens the network URL for reading.
Instead of sending a GET request you can send POST request to server.
Here is the complete code:
import urllib, json, urllib2
url = 'http://localhost:9000/push/'
# This packages the request (it doesn't make it)
request = urllib2.Request(url)
# Sends the request and catches the response
response = urllib2.urlopen(request)
data = json.load(response.read())
print data
I have a django listener that sends an email. It works perfectly under normal circumstances, but when it is triggered by AJAX, I see this error on the console:
[Errno 32] Broken pipe
I am using python manage.py runserver to test it, hence the error on the console.
My suspicion is that because it only happens when the AJAX POST occurs, it must be something to do with the AJAX socket closing before the email has had time to be completely sent, and when it finally is eventually sent, there is nowhere for the response to go. Does this sound feasible? The error is reported by /usr/lib/python2.6/socket.py in the class _fileobject : flush() method.
Here's a snippet of the listener setup:
signals.satchmo_order_status_changed.connect(capture_new_order_listener)
def capture_new_order_listener(sender, oldstatus="", newstatus="", order=None, **kwargs):
email_notify(order, template = 'shop/email/foo.xt', template_html = 'shop/email/foo.html')
Then I have this:
def email_notify(order, template='', template_html=''):
...
send_store_mail(subject, c, template, [order.contact.email],
template_html=template_html, format_subject=True,
sender=order_confirmation_sender)
The AJAX python handler looks like so:
def ajax_update(request, **kwargs):
order.add_status('New') # Triggers sending an email
The AJAX jquery looks like this:
<script src="/static/js/jquery-1.4.2.js" type="text/javascript"></script>
<script>
$(document).ready(function(){
$("#submit").mousedown(function(){
$.post("{% url ajax_update %}",
{
some_var:$("#some_var").val(),
},
function(data,status){
alert("Data: " + data + "\nStatus: " + status);
});
});
});
</script>
How do I get rid of this error? Or should I just ignore it? I'm not sure how it will affect the Apache/WSGI server. It seems I can't catch the exception at send_store_mail()
Edit 1:
Here's the full text of the exception:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 36564)
Traceback (most recent call last):
File "/usr/lib/python2.6/SocketServer.py", line 283, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.6/SocketServer.py", line 309, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/pymodules/python2.6/django/core/servers/basehttp.py", line 562, in __init__
BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
File "/usr/lib/python2.6/SocketServer.py", line 618, in __init__
self.finish()
File "/usr/lib/python2.6/SocketServer.py", line 661, in finish
self.wfile.flush()
File "/usr/lib/python2.6/socket.py", line 304, in flush
self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 32] Broken pipe
----------------------------------------
Many thanks!
I'm having some difficulty attempting to create an SSL socket in Python to use a proxy that requires authentication. I am very sorry for the length, but I felt it was best to include as much detail as possible.
First, the server code looks like this:
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
def __init__(self, server_address, RequestHandlerClass, client_manager, recv_queue):
SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass, bind_and_activate=True)
<snipped out extra code>
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def setup(self):
while True:
try:
print 'trying to wrap in ssl'
self.request = ssl.wrap_socket(self.request,
certfile=(os.getcwd() + '/ssl_certs/newcert.pem'),
keyfile=(os.getcwd() + '/ssl_certs/webserver.nopass.key'),
server_side=True,
cert_reqs=ssl.CERT_NONE,
ssl_version=ssl.PROTOCOL_TLSv1,
do_handshake_on_connect=False,
suppress_ragged_eofs=True)
break
except Exception, ex:
print 'error trying to wrap in ssl %s' % ex
def handle(self):
# Display message that client has connected
print '\r[*] Received connection from %s:%s\r' % (self.client_address[0], self.client_address[1])
while self.stopped() == False:
recv_msg = self.request.read(1024)
if recv_msg == '':
self.stop.set()
server.recv_queue.put(recv_msg)
break
else:
server.recv_queue.put(recv_msg)
if self.stopped():
print '[!] Received STOP signal from %s:%s; Exiting!' % (self.client_address[0], self.client_address[1])
Second, this is the client code where I set up the information needed to connect via the proxy that requires authentication:
class proxyCommsHandler():
def __init__(self, user, password, remote_host, remote_port, list_of_proxies):
# information needed to connect
self.user = 'username'
self.passwd = 'password'
self.remote_host = 'remote_host_ip'
self.remote_port = 8008
self.list_of_proxies = [['proxyserver.hostname.com', 8080]]
# setup basic authentication to send to the proxy when we try to connect
self.user_pass = base64.encodestring(self.user + ':' + self.passwd)
self.proxy_authorization = 'Proxy-authorization: Basic ' + self.user_pass + '\r\n'
self.proxy_connect = 'CONNECT %s:%s HTTP/1.1\r\n' % (self.remote_host, self.remote_port)
self.user_agent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1\r\n"
self.proxy_pieces = self.proxy_connect + self.proxy_authorization + self.user_agent + '\r\n'
Now, here's where I initially connect to the proxy, where I get no errors (I get a '200' status code):
self.proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.proxy.connect( (proxy_host, proxy_port) )
self.proxy.sendall(self.proxy_pieces)
self.response = proxy.recv(1024)
Here's where the client fails (I think). I try to take self.proxy and wrap it in SSL, like this:
sslsock = ssl.wrap_socket(self.proxy, server_side=False, do_handshake_on_connect=True,
ssl_version=ssl.PROTOCOL_TLSv1)
This is the error that I see on the client:
Traceback (most recent call last):
File "C:\Python27\pyrevshell.py", line 467, in <module>
proxyCommsHandler(None, None, None, None, list_of_proxies).run()
File "C:\Python27\pyrevshell.py", line 300, in run
ssl_version=ssl.PROTOCOL_TLSv1)
File "C:\Python27\lib\ssl.py", line 372, in wrap_socket
ciphers=ciphers)
File "C:\Python27\lib\ssl.py", line 134, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 296, in do_handshake
self._sslobj.do_handshake()
SSLError: [Errno 8] _ssl.c:503: EOF occurred in violation of protocol
The client does connect, like shown from the output here:
trying to wrap in ssl
[*] Received connection from x.x.x.x:47144
[*] x.x.x.x:47144 added to the client list
But then it's immediately followed by an exception:
----------------------------------------
Exception happened during processing of request from ('x.x.x.x', 47144)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 582, in process_request_thread
self.finish_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 639, in __init__
self.handle()
File "shell_server.py", line 324, in handle
recv_msg = self.request.read(1024)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 138, in read
return self._sslobj.read(len)
SSLError: [Errno 1] _ssl.c:1348: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
----------------------------------------
While I realize that this sounds like an obvious problem based on the Exceptions that were thrown, the interesting parts are that:
I successfully connect initially via the proxy as shown above
I can successfully connect with a web browser behind behind the same proxy, and no exceptions are thrown; I can pass data to the browser
I've tried different SSL Protocol versions on both the server and client side, as shown in the table here in the Python documentation; it errors on the client-side each time
I've used Wireshark on both ends of the connection. While using a normal browser and connecting to the server, I see the entire SSL handshake and negotiation process, and everything runs smoothly.
However, when I use the client shown above, as soon as I connect with it, I see the client send a Client Hello message, but then my server sends a RST packet to kill the connection (I haven't determined if this is before or after the Exception is thrown).
Again, I apologize for the length, but I am in dire need of expert advice.
I've figured out the issue to my problem. I am sending the self.user_agent to the remote host when I connect via the proxy for the first time, which interferes with the SSL Handshake.
To solve this, I put an initial self.request.recv() in the def setup(self) function before I call ssl.wrap_socket on the socket.