I'm trying to make an http proxy in python. So far I've got everything except https working, hence the next step is to implement the CONNECT method.
I'm slightly confused with the chain of events that need to occur when doing https tunnelling.
From my understanding I should have this when connecting to google:
Broswer -> Proxy
CONNECT www.google.co.uk:443 HTTP/1.1\r\n\r\n
Then the proxy should establish a secure connection to google.co.uk, and confirm it by sending:
Proxy -> Browser
HTTP/1.1 200 Connection established\r\n\r\n
At this point I'd expect the browser to now go ahead with whatever it was going to do in the first place, however, I either get nothing, or get a string of bytes that I can't decode(). I've been reading anything and everything to do with ssl tunnelling, and I think I'm supposed to be forwarding any and all bytes from browser to server, as well as the other way around. However, when doing this, I get a:
HTTP/1.0 400 Bad Request\r\n...\r\n
Once I've sent the 200 code, what should I be doing next?
My code snippet for the connect method:
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
if headers["Method"] == "CONNECT":
client = ssl.wrap_socket(client)
try:
client.connect(( headers["Host"], headers["Port"] ))
reply = "HTTP/1.0 200 Connection established\r\n"
reply += "Proxy-agent: Pyx\r\n"
reply += "\r\n"
browser.sendall( reply.encode() )
except socket.error as err:
print(err)
break
while True:
now not sure
Help is much appreciated!
After finding this answer to a related question: HTTPS Proxy Implementation (SSLStream)
I realised that the initial connection on port 443 of the target server (in this case google.co.uk) should NOT be encrypted. I therefore removed the
client = ssl.wrap_socket(client)
line to continue with a plain text tunnel rather than ssl. Once the
HTTP/1.1 200 Connection established\r\n\r\n
message is sent, the browser and end server will then form their own ssl connection through the proxy, and so the proxy doesn't need to do anything related to the actual https connection.
The modified code (includes byte forwarding):
# If we receive a CONNECT request
if headers["Method"] == "CONNECT":
# Connect to port 443
try:
# If successful, send 200 code response
client.connect(( headers["Host"], headers["Port"] ))
reply = "HTTP/1.0 200 Connection established\r\n"
reply += "Proxy-agent: Pyx\r\n"
reply += "\r\n"
browser.sendall( reply.encode() )
except socket.error as err:
# If the connection could not be established, exit
# Should properly handle the exit with http error code here
print(err)
break
# Indiscriminately forward bytes
browser.setblocking(0)
client.setblocking(0)
while True:
try:
request = browser.recv(1024)
client.sendall( request )
except socket.error as err:
pass
try:
reply = client.recv(1024)
browser.sendall( reply )
except socket.error as err:
pass
References:
HTTPS Proxy Implementation (SSLStream)
https://datatracker.ietf.org/doc/html/draft-luotonen-ssl-tunneling-03
http://www.ietf.org/rfc/rfc2817.txt
Related
I have the a python server coded to work on http://127.0.0.1:9999/. The server prints out the incoming http request. I have also coded on what headers to be sent during the response and also the content. Here is the code:
import socket
from time import sleep
c = None #Client socket1
addr = None #Client address1
server_socket1 = socket.socket() #by default it is SOCK_STREAM (TCP) and has porotocal AF_INET (IPv4)
server_socket1.bind(('127.0.0.1',9999)) #server machine's ip and port on which it will send and recieve connections from
server_socket1.listen(2) #We will only accept two connections as of now , one for each client
print("Server started successfully!!!")
print("Waiting for connections...\n\n")
while (((c is None)and(addr is None))):
if((c is None) and (addr is None)):
c,addr = server_socket1.accept()
print("User connected to client1 socket!!")
c.send(bytes("Connected to the apps server!!!","utf-8"))
print("Client connected ip address "+str(addr))
while True:
msg = c.recv(4096)
if(msg!=None):
#print(msg)
headers, sep, body = msg.partition(b'\r\n\r\n')
headers = headers.decode('utf-8')
print(headers)
html_body = "<html><body><h1>This is a test</h1><p>More content here</p></body></html>"
response_headers = {
'Content-Type': 'text/html; encoding=utf8',
'Content-Length': len(html_body),
'Connection': 'close',
}
response_headers_raw = ''.join('%s: %s\r\n' % (k, v) for k, v in response_headers.items())
response_proto = 'HTTP/1.1'
response_status = '200'
response_status_text = 'OK' # this can be random
# sending all this stuff
r = '%s %s %s\r\n' % (response_proto, response_status, response_status_text)
c.sendall(r.encode())
c.sendall(response_headers_raw.encode())
c.sendall(b'\r\n') # to separate headers from body
c.send(html_body.encode(encoding="utf-8"))
sleep(5)
The code works without compilation errors, starts the server and captures the request i send from the browser intended. But, while sending the response, the socket connection closes with an error as [WinError 10053] An established connection was aborted by the software in your host machine.
Request sent from the browser :
The Output in the terminal :
The error displayed in browser :
What may be causing this error? Previously python prompted me with an error that the object must be a byte type and not type 'str' while sending response_headers_raw variable. Hence I used the encode() function to convert it to a byte type object which has led me to this error.
Any solutions would be greatly appreciated!
~regards
c,addr = server_socket1.accept()
print("User connected to client1 socket!!")
c.send(bytes("Connected to the apps server!!!","utf-8"))
You send "Connected to the apps server!!!" to the client immediately after connect. The client is expecting a HTTP response though. Since it gets your non-HTTP data the client closes the connection. Later c.sendall will write to a socket closed by the peer which results in "An established connection was aborted".
In addition to this ...
msg = c.recv(4096)
if(msg!=None):
#print(msg)
headers, sep, body = msg.partition(b'\r\n\r\n')
Your expectation seems to be that c.recv will return None when the socket is closed. This is not true, it will return '' instead. This means that even after the first error is fixed your code will again run into a similar problem if the peer has closed the connection after successfully reading the request and sending the response.
I have a simple Python HTTP server which also connects other HTTP servers to fetch some data. While, connecting to other servers, my server acts as an http client, but the socket created for incoming connection requests still keeps listening from port 8080 (I have a different socket for the client).
The list of other servers that I need to connect and fetch data is stored in a JSON file and I have code like this
with open(myjsonfile, 'r') as json_file:
entries = json.load(json_file)
for entry in entries.keys():
address = entries[entry]['address']
port = int(entries[entry]['port'])
client_port = config.server_port + 50000
host = gethostname()
# request the TXT file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.txt_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = ''
response = sock.recv(2048).decode()
pos = response.find("\r\n\r\n")
txt_data = response[pos+4:]
# processing the received data here
sock.close()
# request the IMG file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.img_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = b''
while True:
recvdata = sock.recv(2048)
if (len(recvdata) < 1):
break
response = response + recvdata
pos = response.find(b"\r\n\r\n")
img_data = response[pos+4:]
# working with the image here
sock.close()
I have to use a set port number for my client because this is how the server identifies me. However, I sometimes get an "Address already in use" error for the second socket.bind() call (the one for the image). Without the bind() calls, my code works fine.
I tried setting socket options (commented out in the code above) and using pycurl with the LOCALPORT property set to client_port value above, but still getting the same error.
What could be the reason behind the error message? I think I open and close the sockets so the operating system should free the port for further use (I think)?
Thanks
PS : This is a small project, not a production system, hence do not bother with "why use port numbers to identify clients"
There is a TIME_WAIT after the session is shutdown to make sure that there are still no live packets in the network.When you re-create the same tuple and one of those packets shows up, it would be treated as a valid packet for your connection this will cause an error state.Usually 2xpacket max age, before the packet is discarded
Before you create a connection with the same tuple, all the packets from the previous session must be dead.
Try using;
...
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.listen([backlog])
sock.bind((host, client_port))
...
socket.listen([backlog])
import socket
host = 'www.google.com'
port = 80
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try :
client.connect((host, port))
except socket.error:
print ("Err")
package = '00101'
package.encode('utf-8')
client.sendall(package.encode(encoding = 'utf-8'))
response = client.recv(4096)
print (response.decode('UTF-8')
I kept getting b'' as my return, so I'm trying to decode it. The error I receive is unexpected EOF while parsing. Should I not include the decoding() function in my printing? I've tried printing only response, the .decode() function did not decode. What should I try?
You need to send a valid HTTP request. For example:
package = b'''GET /HTTP/1.1
Host: www.google.com
'''
client.sendall(package)
Which correctly returns a redirect on my machine. Note the empty line at the end of package, which ends the request.
When you send b'00101' and start reading, the google server has not yet processed your request and returns nothing. By sending a trailing newline (package = b'00101\n') it will start processing your request, and you will get:
...
<p>Your client has issued a malformed or illegal request. <ins>That’s all we know.</ins>
"Firts of all this stuff is for my university(a challeng) nothing bad is happening in RL"!
Im not quite familier with phyton but, i give it a trie.
The task is to performe a ssl Stripping attack through a phython script.
The output off the script musst be Username and PW (stdout)
My problem is now:
When i download the ssl strip binarie i see a python script which should include the relevant stuff (sslstrip) but, im not soure where i should start here?
I noticed that the two classes ClientsRequest & Stripping Proxy are essential so they are an the bottom.
My problem is how can i i connect the client request class so that it reads the data from the client ?
Im not sure but this could be the config for the port:8080
(the script must listen on 8080)
# Standard socket stuff:
host = '' # do we need socket.gethostname() ?
port = 8080
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((host, port))
sock.listen(1) # don't queue up any requests
# Loop forever, listening for requests:
while True:
csock, caddr = sock.accept()
print "Connection from: " + `caddr`
req = csock.recv(1024) # get the request, 1kB max
I need a little spark in the right direction.
best regards
Rolento
# Loop forever, listening for requests:
while True:
csock, caddr = sock.accept() #< - this is a blocking call
print "Connection from: " + `caddr`
#you never end up getting to this instruction
req = csock.recv(1024) # get the request, 1kB max
you have some problems
sock.accept will block waiting for a connection ... once you get the connection you print their data and loop right back to the waiting for connection ... when in reality you want to do something with that connection
while True:
csock, caddr = sock.accept() #< - this is a blocking call
print "Connection from: " + `caddr`
req = csock.recv(1024) # get the request, 1kB max
will at least recieve a message from the client after accepting the incomming connection
what you probably need to do is hand the connection off to a seperate thread to work with so you can continue to accept new connections , while processing the current connection
I am developing a TCP client on Python, and I have the next problem. I connect with the server, I send it some data, it response me with the data expected but after this the my own application (client) send a [FIN, ACK] (checked with wireshark). Here is my client app:
try:
sock = socket(AF_INET, SOCK_STREAM)
sock.bind((my_ip,my_port))
sock.connect((sendAddress,sendPort))
sock.send(joinRequest)
joinResponse = sock.recv(18)
print joinResponse
except socket.timeout:
sock.close()
This is the default behavior of SocketServer, accept a connection, get the request, and then close the connection.
The simple way will be to use while loop to keep it connected, You can also use sock.settimeout to tune the timeout