I have the following code for a simple Server that handles GET requests from the browser over TCP.
while True:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(('127.0.0.1', int(args.port)))
print(f'TCP Server listening on port {args.port}...')
s.listen()
connection, addr = s.accept()
print(f'Connected by {addr}')
while True:
data = connection.recv(args.buf)
if not data:
print('Connection closed')
break
message = data.decode()
response = handle_request(message)
connection.sendall(response)
connection.close()
This works somewhat, but not really. When the browser GETs a PNG or html file, that it has cached (if-modified-since), it works seamlessly. However, when it has to serve a new file, the browser (Safari) stalls until I shut down the server but then mysteriously shows the actual file the browser was trying to GET.
I wonder if this has something to do with my implementation of the connection, but I assume that it more likely has to do with my http headers.
This is what my simple headers and response to the browser look like:
header = f"HTTP/1.1 {status}\r\n" \
f"Host: {header_dict['host']}\r\n" \
f"Date: {datetime.today().strftime(time_format)}\r\n" \
f"Last-Modified: {mod_time}\r\n" \
f"Connection: Keep-Alive\r\n" \
f"\r\n".encode()
with open(filename, "rb") as f:
payload = f.read()
response = header + payload
Is there anything important missing from the header for Safari (or any other browser) to work with it? Thanks!
As KompjoeFriek commented, the browser needs to know how much data it is getting from the server. So the content-length header is necessary when handling a get request to serve some data. Without it, it waits until the connection is closed to show the data it has received. Adding a content-length header solved the problem.
Related
I have a simple Python HTTP server which also connects other HTTP servers to fetch some data. While, connecting to other servers, my server acts as an http client, but the socket created for incoming connection requests still keeps listening from port 8080 (I have a different socket for the client).
The list of other servers that I need to connect and fetch data is stored in a JSON file and I have code like this
with open(myjsonfile, 'r') as json_file:
entries = json.load(json_file)
for entry in entries.keys():
address = entries[entry]['address']
port = int(entries[entry]['port'])
client_port = config.server_port + 50000
host = gethostname()
# request the TXT file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.txt_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = ''
response = sock.recv(2048).decode()
pos = response.find("\r\n\r\n")
txt_data = response[pos+4:]
# processing the received data here
sock.close()
# request the IMG file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.img_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = b''
while True:
recvdata = sock.recv(2048)
if (len(recvdata) < 1):
break
response = response + recvdata
pos = response.find(b"\r\n\r\n")
img_data = response[pos+4:]
# working with the image here
sock.close()
I have to use a set port number for my client because this is how the server identifies me. However, I sometimes get an "Address already in use" error for the second socket.bind() call (the one for the image). Without the bind() calls, my code works fine.
I tried setting socket options (commented out in the code above) and using pycurl with the LOCALPORT property set to client_port value above, but still getting the same error.
What could be the reason behind the error message? I think I open and close the sockets so the operating system should free the port for further use (I think)?
Thanks
PS : This is a small project, not a production system, hence do not bother with "why use port numbers to identify clients"
There is a TIME_WAIT after the session is shutdown to make sure that there are still no live packets in the network.When you re-create the same tuple and one of those packets shows up, it would be treated as a valid packet for your connection this will cause an error state.Usually 2xpacket max age, before the packet is discarded
Before you create a connection with the same tuple, all the packets from the previous session must be dead.
Try using;
...
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.listen([backlog])
sock.bind((host, client_port))
...
socket.listen([backlog])
So I am very new to Python and am now trying to understand how to send a request using TCP in Python. The sample code and document is not very helpful (to me, as I don't understand Java).
The document:
https://www.sharekhan.com/Upload/General/TradeTigerAPIForClient.pdf
I have the following till now
import socket
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server = '192.168.43.211'
port = 800
request = "DataLength = 196|Transcode = 1|LoginId = ***|MemberPassword = sh*|TradingPassword = S77*| IP = 192.618.31.211|Reserved = |"
s.connect((server,port))
s.send(request.encode())
result = s.recv(4096)
If I use this the program shows I am connected and the result is b''
I also tried
request = "|DataLength =108|Transcode = 21|Exchange Code=NC|Reserved=|"
result is b'Hurray you are connected'
How do I use the commands from the document to get data?
According to this, you can send a TCP packet like this:
import socket
server = '192.168.31.211'
port = 80
buffer_size = 4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((server, port))
sock.send(message)
data = sock.recv(buffer_size)
sock.close()
Note I didn't include your message here, nor did I encode your data. However, as was pointed out in the comments, your IP address is incorrectly written and refers to a private IP address, according to the IANA so if you're trying to send out over a public network, this won't work. Additionally, I have noticed a couple of other problems with your code:
Your message looks like you've manually included TCP header information. Be aware that by using the socket library the way you are and the way I have suggested, you are making a TCP request. The header information will therefore be included with your request so you don't need to include it yourself.
What you have here is the client code and you didn't include any server code. Have you written code for your server? If not, you'll need some.
Otherwise, I can't see any problems with your code.
I am sending some data after html content (it has a little delay) in the same response during keep-alive session and want browser to show html before the whole response is downloaded.
For example, I have text 'hello, ' and a function that computes 'world' with delay (let it be 1 sec). So I want browser to show 'hello, ' immediately and 'world' with its delay. Is it possible within one request (so, without ajax)
Here is example python code of what I do (highlighted: https://pastebin.com/muUJyR36):
import socket
from time import sleep
sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)
conn, addr = sock.accept()
def give_me_a_world():
sleep(1)
return b'world'
while True:
data = conn.recv(1024)
response = b'HTTP/1.1 200 OK\r\n'\
b'Content-Length: 12\r\n'\
b'Connection: keep-alive\r\n'\
b'\r\n'\
b'hello, '
conn.send(response) # send first part
conn.send(give_me_a_world()) # make a delay and send other part
conn.close()
First and foremost, read How the web works: HTTP and CGI explained to understand why and where your current code violates HTTP and thus doesn't and shouldn't work.
Now, as per Is Content-Length or Transfer-Encoding is mandatory in a response when it has body , after fixing the violation, you should
omit the Content-Length header and close the socket after sending all the data, OR
calculate the length of the entire data to send beforehand and specify it in the Content-Length header
You could use Transfer-Encoding: chunked and omit Content-Length.
It works fine on text browsers like curl and Links WWW Browser. But, modern graphical browsers don't really start rendering until it reaches some sort of buffer boundaries.
import socket
from time import sleep
sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)
conn, addr = sock.accept()
def give_me_a_world():
sleep(1)
return b'5\r\n'\
b'world\r\n'\
b'0\r\n'\
b'\r\n'
while True:
data = conn.recv(1024)
response = b'HTTP/1.1 200 OK\r\n'\
b'Transfer-Encoding: chunked\r\n'\
b'Connection: keep-alive\r\n'\
b'\r\n'\
b'7\r\n'\
b'hello, \r\n'
conn.send(response) # send first part
conn.send(give_me_a_world()) # make a delay and send other part
conn.close()
So I built a very simple HTTP server in Python. It's purpose is to send a file when it gets a request.
This works in chrome but in Firefox it keeps downloading without making any progress. I also noticed that in Chrome, the name of the downloaded file is download.png where as the actual name of the file is s.png. Could someone tell me what is wrong with this code? Also I tried printing a message and sending html code too in firefox, it just keeps on showing the message "waiting on localhost" and does nothing.
import socket
serversocket = socket.socket()
serversocket.bind(("127.0.0.1", 80))
serversocket.listen(800)
msg = open("s.png", "r").read()
msg = "HTTP/1.0 200 OK\r\nServer: ls\r\nContent-Type: image/png\r\nContent-Disposition: attachement\r\nfilename: s.png\r\n\r\n" + msg + "\r\n\r\n"
while 1:
(clientsocket, address) = serversocket.accept()
clientsocket.send(msg)
Do not insert newline between Content-Disposition and the name of the file.
Using : between filename and the name of the file is also wrong.
I think you shouldn't add useless newlines after the image data.
Using binary mode is good for reading binary files.
You should close the connection after sending the message. Otherwise, the client cannot tell where the end of file is because you didn't send Content-Length header.
It seems good for Firefox to read the request before sending the response.
Try this (tested with Python 3.4.2 and Python 2.7.11):
import socket
serversocket = socket.socket()
serversocket.bind(("127.0.0.1", 80))
serversocket.listen(800)
msg = open("s.png", "rb").read()
msg = "HTTP/1.0 200 OK\r\nServer: ls\r\nContent-Type: image/png\r\nContent-Disposition: attachement; filename=s.png\r\n\r\n".encode('UTF-8') + msg
while True:
(clientsocket, address) = serversocket.accept()
recvdata = ''.encode('UTF-8')
while True:
recvdata += clientsocket.recv(4096)
if "\r\n\r\n".encode('UTF-8') in recvdata:
break
clientsocket.send(msg)
clientsocket.close()
I am trying to implement a server in Python. When the browser connects to localhost with port number 9999, it will open the file index.html with the images.jpg in that page, but the image can not be shown. How can I make the web server handle the image as well?
Here is my code so far:
from socket import *
import os
serversocket = socket(AF_INET, SOCK_STREAM)
port = 5000
host = '127.0.0.1'
size = os.path.getsize("index.html")
myfile = open('index.html', 'rb')
mycontent = "Welcome to Very Simple Web Server"
size = len(mycontent)
header = "HTTP/1.0 200 OK \r\n Content_Length:" + str(size) + "\r\n\r\n"
mycontent = myfile.read()
serversocket.bind((host, port))
serversocket.listen(5)
print('Server is listening on port 9999')
while (1):
conn, addr = serversocket.accept()
print('Connected by', addr)
conn.send(bytes(header))
conn.send(mycontent)
conn.close()
Your code creates an infinite loop that will just only send one file, and never accepts other connections.
In order for the image to show, the browser has to send another request to the URL of the image, and this request is not being serviced by your code.
In order for your server to work, you need to:
Start a loop
Listen for connections
Interpret the headers of the incoming request, and then act appropriately. Lets assume that you only deal with GET requests and not other things like POST, HEAD, PUT, etc.
Look at the requested resource (the URL)
Find the resource on the file system (so now, you have to parse the URL)
Package the resource into a HTTP response (read the file set the appropriate mime type)
Send the response back to the client with the appropriate headers (the server response headers)
Repeat
To display a HTML page with one image, it takes two requests, one for the HTML page, and another for the image. If the HTML code has a link to a CSS file, now you need three requests - one for the HTML page, one for the CSS file and a final one for the image. All these requests need to be completed successfully in order for the browser to render the page.
You never need to do this by hand, use a web development framework which will take care of all this "boring" stuff so you can then deal with solving the actual problem.