I have the following bare-bones server in order to make sure that I'm able to receive a socket connection:
import socket
HOST, PORT = '', 8888
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print(f'Serving HTTP on port {PORT} ...')
while True:
print ('1 - start')
client_connection, client_address = listen_socket.accept()
request_data = client_connection.recv(1024)
print(request_data.decode('utf-8'))
client_connection.sendall(b"""HTTP/1.1 200 OK\n\nHello, World!\n""")
client_connection.close()
print ('2 - end')
And when I view it in Chrome, it loads the page, but doesn't return any response. Here is what the server prints:
1 - start
GET / HTTP/1.1
Host: localhost:8888
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Sec-Fetch-Site: none
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: csrftoken=ekiwd2n8m6BjVe0Uoyif6OZ7pmWmULDs; _xsrf=2|02c4087f|7e8be522d7bb6404ace2d2117e42deed|1571691550; username-localhost-8888="2|1:0|10:1571694083|23:username-localhost-8888|44:ODk2NzY4Yzg2ZjgyNDRlMzg0ZWEwMjU1MGQ0OTU5NzE=|87587a46d228990c91d93d828c44e2a79155d03332047321e1f882dced5c67dd"; _hp2_id.1263915336=%7B%22userId%22%3A%223410356020127146%22%2C%22pageviewId%22%3A%223017774512831795%22%2C%22sessionId%22%3A%228642071851852288%22%2C%22identity%22%3Anull%2C%22trackerVersion%22%3A%224.0%22%
2 - end
1 - start
So it seems to be going through...But the client (Chrome) just receives a blank response:
Why isn't it returning "Hello, World!" ? When I try it with telnet it's the same as well.
Note that this does with with requests -- just not telnet or chrome:
>>> print(requests.get('http://127.0.0.1:8888').text)
Hello, World!
It seems to be related to this line:
request_data = client_connection.recv(1024)
As you said problem can be when browser sends more then 1024 bytes.
This example uses while True loop to read in chunk and check if request_data ends with b'\r\n\r\n' - and it works with GET but problem makes POST which send body directly after b'\r\n\r\n'. To resolve this problem I simply use .recv(1) and check b'\r\n\r\n' after every bytes but you could try to find better method to check b'\r\n\r\n' in request_data
import socket
HOST, PORT = '', 8888
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print(f'Serving HTTP on port {PORT} ...')
while True:
print ('1 - start')
client_connection, client_address = listen_socket.accept()
request_data = b''
while True:
chunk = client_connection.recv(1)
request_data += chunk
#print(chunk)
if request_data.endswith(b'\r\n\r\n'):
break
print(request_data.decode('utf-8'))
client_connection.sendall(b"""HTTP/1.1 200 OK\n\nHello, World!\n""")
client_connection.close()
print ('2 - end')
This code reads only header. To read body in POST you can use function with bigger value in recv() and POST should send header content-length with size of body so it should be easier to read body then header.
Related
Handle http protocol using socketserver
I'm structuring a web server (...), aiming to interpret the python language (just like apache does with php ). I establish the connection (client x server) in the transport layer through the TCP protocol using python's socketserver module. Now I need to capture the request header data and send an http response to the client (browser), how could I do that? Below is the code and result of the processing:
File : httptcpipv4.py
# Module Name : socketserver -> https://docs.python.org/3/library/socketserver.html#module-socketserver
from socketserver import BaseRequestHandler, ThreadingTCPServer
# Transport Layer Protocol : TCP
# This is the superclass of all request handler objects. It defines the interface, given below.
class HTTPTCPIPv4(BaseRequestHandler):
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).decode()
print(self.data)
if __name__ == "__main__":
HOST, PORT = "", 8080
# Create the asynchronous server, binding to localhost on port 8080
with ThreadingTCPServer((HOST, PORT), HTTPTCPIPv4) as server:
print("Server : Action v0.0.1, running address http://127.0.0.1:8080,")
print("cancel program with Ctrl-C")
server.serve_forever()
Output :
GET / HTTP/1.1
Host: 127.0.0.1:8080
Connection: keep-alive
sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"
sec-ch-ua-mobile: ?0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.106 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Purpose: prefetch
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7
I'm a fresh man to computer network, and i'm tring to make a proxy server of my own.
But when I send the request i received from the client to the server, i can't get the response from the server. My code get an exception here :
try:
# connect
serverSock.connect((hostName, 80))
# get the client's request
fp = open("requestCache.txt", "r")
message = fp.read()
fp.close()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)
except:
print('connect failed!')
serverSock.close()
the following is the request received from the client
GET /www.baidu.com HTTP/1.1
Host: localhost:3009
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9
You generally want to avoid enclosing huge chunks of code in try...except blocks unless you understand exactly what will happen when an exception is raised. I usually keep the try...except blocks as minimal as possible and catch as specific errors as possible:
try:
serverSock.connect((hostName, 80))
except OSError as e:
# handle e
You're actually catching and throwing away a very useful error:
TypeError Traceback (most recent call last)
<ipython-input-13-78a255a190f8> in <module>()
10
11 # send to the target server
---> 12 serverSock.send(message)
13 response = serverSock.recv(4096)
14
TypeError: a bytes-like object is required, not 'str'
Your message is a string, but sockets deal with bytes. To fix it, read the file's contents as bytes instead ('rb' mode instead of just 'r'):
# connect
serverSock.connect((hostName, 80))
# get the client's request
with open("requestCache.txt", "rb") as handle:
message = handle.read()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)
Im currently creating a python socket http server, and I'm working on my GET and POST requests. I got my GET implementation working fine, but the body element of the POST requests won't show up.
Code snippet:
self.host = ''
self.port = 8080
self.listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.listener.bind((self.host, self.port))
self.listener.listen(1)
while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
This code yields the http header after processing the post request from the webpage:
POST /test.txt HTTP/1.1
Host: localhost:8080
Content-Type: application/x-www-form-urlencoded
Origin: http://localhost:8080
Content-Length: 21
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17
Referer: http://localhost:8080/
Accept-Language: nb-no
Accept-Encoding: gzip, deflate
But there is no body, so the question is why am i not receiving the http body when i know it is sent?
Thanks!
while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
recv does not read exactly 2048 bytes but it reads up to 2048 bytes. If some data arrive recv will return with the data even if more data might follow. My guess is that in your case the client is first sending the HTTP header and then the body. If NAGLE algorithms is off at the client side (common) it is likely that your first recv will only get the header and that you would need another recv for the body. This would explain what happens in your case: you get the header but not the body since you don't do another recv.
But even that would be a too simple implementation which will go wrong sooner or later. To make it correctly you should implement the HTTP protocol correctly: first read the HTTP header which might need multiple recv if the header is large. Then you should parse the header, figure out the size of the body (Content-length header) and read the remaining bytes.
I have a server in python that listens to GET requests:
host = '127.0.0.1' # listen to localhost
port = 8001
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((host, port))
sock.listen(5) # don't queue up any requests
while True:
csock, caddr = sock.accept()
print "Connection from: " + repr(caddr)
req = csock.recv(1024)
print req
And I get the following request:
Connection from: ('127.0.0.1', 42311)
GET /?categories[]=100&categories[]=200 HTTP/1.1
Host: localhost:8001
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
The requests have the form http://localhost:8000/?categories[]=100&categories[]=200 and I want to get the categories that were passed.
Should I write a regular expression to parse req or I can get 'categories' parameters as attribute parameter of req?
It depends on how you intend to use these requests. If you want to respond with HTML pages (or the such) depending on the categories, you should take a look at frameworks like Flask. If you just want to parse the headers, take a look at this. It's a thread on how to parse HTTP headers.
Making a simple python web server using sockets to start understanding how they work, but I think I'm lost on this on. My python server is supposed to access a basic html file in the same directory and display it, once for every time it's requested. But this code for some reason sends the request 3 to 5 times...
from socket import *
server = socket(AF_INET, SOCK_STREAM)
port = 12030
server.bind((gethostname(), port))
server.listen(1)
while True:
print 'Ready to serve'
conection, addr = server.accept()
try:
print 'Working'
message = conection.recv(1024)
filename = message.split()[1] #cuts off the '/' in the request page
f = open(filename[1:])
print message
outputdata = f.read()
print outputdata
conection.send('HTTP/1.1 200 OK\r\n')
for i in range(0, len(outputdata)):
conection.send(outputdata[i])
conection.close()
except IOError:
print 'IO ERROR'
print message
print outputdata
conection.close()
except KeyboardInterrupt:
server.close()
conection.close()
break;
This is the output from me opening the page in the browser.
-en 14:59:54 # ・ー ・
python project.py
Ready to serve
Working
<html><body><h1>Wurld</body></html>
Ready to serve
Working
IO ERROR
<html><body><h1>Wurld</body></html>
Ready to serve
Working
IO ERROR
<html><body><h1>Wurld</body></html>
Ready to serve
I've tried adding a server.listen(1)
and a conection.send("Content-Type:text/html\r\n") , but neither of these do anything.
I'm not sure what the problem could be other than blocking how many times can be requested per minute?
Updated to print message every time
-en 15:33:26 # ・ー ・
python project.py
Ready to serve
Working
GET /HelloWorld.html HTTP/1.1
Host: seppala:12030
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36
DNT: 1
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
<html><body><h1>Wurld</body></html>
Ready to serve
Working
IO ERROR
GET /favicon.ico HTTP/1.1
Host: seppala:12030
Connection: keep-alive
Accept: */*
DNT: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
<html><body><h1>Wurld</body></html>
Ready to serve
Working
IO ERROR
GET /favicon.ico HTTP/1.1
Host: seppala:12030
Connection: keep-alive
Accept: */*
DNT: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
<html><body><h1>Wurld</body></html>
Ready to serve
It seems your browser is requesting favicon.ico . Try adding a favicon.ico to your document root, or perhaps try a different browser. This problem isn't because of your script.