I'm a fresh man to computer network, and i'm tring to make a proxy server of my own.
But when I send the request i received from the client to the server, i can't get the response from the server. My code get an exception here :
try:
# connect
serverSock.connect((hostName, 80))
# get the client's request
fp = open("requestCache.txt", "r")
message = fp.read()
fp.close()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)
except:
print('connect failed!')
serverSock.close()
the following is the request received from the client
GET /www.baidu.com HTTP/1.1
Host: localhost:3009
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9
You generally want to avoid enclosing huge chunks of code in try...except blocks unless you understand exactly what will happen when an exception is raised. I usually keep the try...except blocks as minimal as possible and catch as specific errors as possible:
try:
serverSock.connect((hostName, 80))
except OSError as e:
# handle e
You're actually catching and throwing away a very useful error:
TypeError Traceback (most recent call last)
<ipython-input-13-78a255a190f8> in <module>()
10
11 # send to the target server
---> 12 serverSock.send(message)
13 response = serverSock.recv(4096)
14
TypeError: a bytes-like object is required, not 'str'
Your message is a string, but sockets deal with bytes. To fix it, read the file's contents as bytes instead ('rb' mode instead of just 'r'):
# connect
serverSock.connect((hostName, 80))
# get the client's request
with open("requestCache.txt", "rb") as handle:
message = handle.read()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)
Related
Was wondering why I am getting a 408 request timeout when sending an HTTP GET request using sockets. I just copied the GET request that was sent through Chrome and then pasted it into python figuring that I would get a 200 response, but clearly, I am missing something.
def GET():
headers = ("""GET / HTTP/1.1\r
Host: {insert host here}\r
Connection: close\r
Cache-Control: max-age=0\r
DNT: 1\r
Upgrade-Insecure-Requests: 1\r
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\r
Accept-Encoding: gzip, deflate\r
Accept-Language: en-US,en;q=0.9\r
Cookie: accept_cookies=1\r\n""").encode('ascii')
payload = headers
return payload
def activity1():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
user = GET()
sock.sendall(user)
poop = sock.recv(10000)
print(poop)
sock.close()
Assuming the hostname and port are defined correctly is there anything wrong with this request that would cause it to timeout? Thanks.
The initial problem is that the HTTP header is not properly finished, i.e. it is missing the final \r\n (empty line). Once this is done you will likely run into multiple other problems, like:
You are assuming that everything can be read within a single recv, which will only be true for short answers.
You likely assume that the body is a single byte buffer. But it can be transferred in chunks since HTTP/1.1 support this Transfer-Encoding.
You likely assume that the body is in plain. But it can be compressed since you explicitly accept gzip-compressed responses.
HTTP is not the simple protocol as it might look. Please read the actual standard before implementing it, see RFC 7230. Or just use a library which does the hard work for you.
So I created a code which a client uploads a file to the server folder and he has an option to download it back, it works perfectly fine in chrome, I click on the item I want to download and it downloads it
def send_image(request, cs):
request = request.split('=')
try:
name = request[1]
except:
name = request[0]
print('using send_iamge!')
print('Na ' + name)
path = 'C:\\Users\\x\\Desktop\\webroot\\uploads' + '\\file-name=' + name
print(path)
with open(path, 'rb') as re:
print('exist!')
read = re.read()
cs.send(read)
the code above reads the file that you choose and sends the data as bytes to the client back.
In chrome, it downloads the file as I showed you already but in for example internet explorer, it just prints the data to the client and doesn't download it The real question is why doesn't it just prints the data in chrome, why does it download it and doesn't print it as internet explorer does and how can I fix it?(for your info: all the files that I download have the name file-name before them that's why I put it there)
http request:
UPDATE:
POST /upload?file-name=Screenshot_2.png HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive
Content-Length: 3534
Accept: */*
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type: application/octet-stream
Origin: http://127.0.0.1
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Referer: http://127.0.0.1/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,he;q=0.7
It looks like that you don't send a HTTP/1 response but a HTTP/0.9 response (Note that I'm talking about the response send from the server not the request send from the client). A HTTP/1 response consists of a HTTP header and a HTTP body, similar to how a HTTP request is constructed. A HTTP/0.9 response instead only consists of the actual body, i.e. no header and thus no meta information in the header which tell the browser what to do with the body.
HTTP/0.9 is obsolete for 25 years but some browsers still support it. When a browser gets a HTTP/0.9 request it could anything with it since there is no defined meaning from the HTTP header. Browsers might try to interpret is as HTML, as plain text, offer it for download, refuse it in total ... - whatever.
The way to fix the problem is to send an actual HTTP response header before sending the body, i.e. something like this
cs.send("HTTP/1.0 200 ok\r\nContent-type: application/octet-stream\r\n\r\n")
with open(path, 'rb') as re:
...
cs.send(read)
In any case: HTTP is way more complex than you might think. There are established libraries to deal with this complexity. If you insist on not using any library please study the standard in order to avoid such problems.
So I have an issue with a websocket in Python. I'm trying to interface with a website that uses websockets for some of its content communication. Here is the javascript code on their site:
var $j = jQuery.noConflict(); // Use $j to reference JQuery selectors instead of $
function sockify() {
var ws = new WebSocket("ws://website:1234");
ws.onmessage = function (evt) {
console.log(evt.data)
$j('#output').html(evt.data);
}
ws.onopen = function () {
ws.send(JSON.stringify($j('#srctext').val()));
}
ws.onerror = function () {
alert("socket down");
}
}
So the site works fine, and theres no issue with it, however when I try this python code, I get an error stating that the socket has been closed immediately after it opens:
ws = create_connection("ws://website:1234/")
print "Sending 'Hello, World'..."
ws.send("Hello, World")
print "Sent"
print "Receiving..."
result = ws.recv()
print "Received '%s'" % result
ws.close()
This is sample code pulled from the websocket man page on python.org, and it does work if I do not change the host to the website i'm trying to pull from, but rather leave the example's host as it is in the example.
Here is the error I receive:
Traceback (most recent call last):
File "irc.py", line 462, in <module>
tmpmsg = getSocket()
File "irc.py", line 64, in getTrump
result = ws.recv()
File "/Library/Python/2.7/site-packages/websocket/_core.py", line 293, in recv
opcode, data = self.recv_data()
File "/Library/Python/2.7/site-packages/websocket/_core.py", line 310, in recv_data
opcode, frame = self.recv_data_frame(control_frame)
File "/Library/Python/2.7/site-packages/websocket/_core.py", line 323, in recv_data_frame
frame = self.recv_frame()
File "/Library/Python/2.7/site-packages/websocket/_core.py", line 357, in recv_frame
return self.frame_buffer.recv_frame()
File "/Library/Python/2.7/site-packages/websocket/_abnf.py", line 336, in recv_frame
self.recv_header()
File "/Library/Python/2.7/site-packages/websocket/_abnf.py", line 286, in recv_header
header = self.recv_strict(2)
File "/Library/Python/2.7/site-packages/websocket/_abnf.py", line 371, in recv_strict
bytes_ = self.recv(min(16384, shortage))
File "/Library/Python/2.7/site-packages/websocket/_core.py", line 427, in _recv
return recv(self.sock, bufsize)
File "/Library/Python/2.7/site-packages/websocket/_socket.py", line 93, in recv
"Connection is already closed.")
websocket._exceptions.WebSocketConnectionClosedException: Connection is already closed.
Any idea why its closing right away?
Edit:
Ran with enableTrace true.
Here is the error I get:
--- request header ---
GET / HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: website
Origin: website
Sec-WebSocket-Key: 6jsV5DEWXPGTTTLKSEwz6g==
Sec-WebSocket-Version: 13
-----------------------
--- response header ---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: CX4DYsItQC6utXvt8JH641455mM=
-----------------------
send: '\x81\x8b\x98\x8d\x81\xce\xd0\xe8\xed\xa2\xf7\xad\xd6\xa1\xea\xe1\xe5'
Please note that I dont actually have control over the websocket, so any fixes would need to be on my end. I'm on Python 2.7.10
I also noticed that if I intercept the websocket request in Burp while using the website, the websocket initial connection request is different. Here it is as captured from the website:
GET / HTTP/1.1
Host: website
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Sec-WebSocket-Version: 13
Origin: website
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: uyG2WBK51ZtPhy9RXLNTmg==
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
You have for sure an error (most probably your socket is exploding silently..)
set instead in the configuration a callback for the errors in the socket and print the msg you get..
example: (taken from here)
websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://echo.websocket.org/",
on_message = on_message,
on_error = on_error,
on_close = on_close)
and define the method
def on_error(ws, error):
print(error)
For anyone who is having the same issue I fixed it by tweaking the websocket module and setting a custom header in the get request try forcing it to use this header
GET / HTTP/1.1
Host: website
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Sec-WebSocket-Version: 13
Origin: website
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: uyG2WBK51ZtPhy9RXLNTmg==
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
instead of this
GET / HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: website
Origin: website
Sec-WebSocket-Key: 6jsV5DEWXPGTTTLKSEwz6g==
Sec-WebSocket-Version: 13
Im currently creating a python socket http server, and I'm working on my GET and POST requests. I got my GET implementation working fine, but the body element of the POST requests won't show up.
Code snippet:
self.host = ''
self.port = 8080
self.listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.listener.bind((self.host, self.port))
self.listener.listen(1)
while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
This code yields the http header after processing the post request from the webpage:
POST /test.txt HTTP/1.1
Host: localhost:8080
Content-Type: application/x-www-form-urlencoded
Origin: http://localhost:8080
Content-Length: 21
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17
Referer: http://localhost:8080/
Accept-Language: nb-no
Accept-Encoding: gzip, deflate
But there is no body, so the question is why am i not receiving the http body when i know it is sent?
Thanks!
while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
recv does not read exactly 2048 bytes but it reads up to 2048 bytes. If some data arrive recv will return with the data even if more data might follow. My guess is that in your case the client is first sending the HTTP header and then the body. If NAGLE algorithms is off at the client side (common) it is likely that your first recv will only get the header and that you would need another recv for the body. This would explain what happens in your case: you get the header but not the body since you don't do another recv.
But even that would be a too simple implementation which will go wrong sooner or later. To make it correctly you should implement the HTTP protocol correctly: first read the HTTP header which might need multiple recv if the header is large. Then you should parse the header, figure out the size of the body (Content-length header) and read the remaining bytes.
I have made a simple Python web server.
The server runs fine, connects to browser, and sends data.
When I have files included to to the HTML doc I'm sending (like <img src="image_path"/>)
It doesn't want to send the data, nor other ones included, like style sheets. I am using Chrome Web browser, and getting thoose headers:
GET /img.jpg HTTP/1.1
Host: localhost
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.17 (KHTML, like Gecko)
Chrome/24.0.1312.57 Safari/537.17
Accept: */*
Referer: http://localhost/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: pl,en-US;q=0.8,en;q=0.6,nb;q=0.4
Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.3
And I get this error message:
Unhandled exception in thread started by <function handler at 0x02C28150>
Traceback (most recent call last):
File "C:\Users\CamIce\Desktop\PyServer v2\server.py", line 16, in handler
data = clientsock.recv(BUFSIZ)
OSError: [WinError 10038] Próbowano przeprowadzić próbę na obiekcie, który nie jest
gniazdem
The code is too long for this message, so I had to post it on pastebin:
http://pastebin.com/n19J0y6A
Have you tried binding your incoming socket to a different, non-privileged port instead of port 80 (say, something above 1024)? This seems to me like the Windows firewall is interfering somehow, or possibly some other permission issue. It just seems odd to me that you didn't get an error before the recv() call, but I don't know WIndows too well.
I don't see anything immediately wrong with your code.