I'm currently making a proxy which sits between the browser and the web. Everything works except https. I'm having troubles understanding some passages of it and haven't found many resources on the web. And so I'm stuck.
The code I'm using is:
conn, addr = server.accept()
request = conn.recv(9999) #get a CONNECT request
conn.send(b'HTTP/1.1 200 Connection estabilished\n\n')
enc_req = conn.recv(9999) #this gets an encrypted request
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #plaintext client
client.connect((host, 443)) #connect to chosen host
client.send(enc_req)
resp1 = client.recv(9999) #this gets something unreadable (encrypted?)
#could it be the certificate?
#now what?
Is the resp1 I'm getting the certificate? And what do I need to do after that? (Or, which is the same, what does usually happens next with https?)
P.S. I know the question is somewhat generic, but please don't judge me too harshly. I've tried researching on the web but all I keep finding is the encryption method used for ssl. I really don't know how to proceed.
I haven't tested this code (and it's mainly pseudo code), but this should give you an idea of what you need to do.
conn, addr = server.accept()
request = conn.recv(9999) #get a CONNECT request
# Here, parse the CONNECT string and get the host and port (not sure if you were doing that already.
# Then, try to connect *before* you tell the client the connection was established (in case it fails)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #plaintext client
client.connect((host, 443)) #connect to chosen host
conn.send(b'HTTP/1.1 200 Connection estabilished\n\n')
# Then loop until the connections are closed.
while True:
# Read from the client, send the data to the server.
enc_req = conn.recv(9999) #this gets an encrypted request
client.send(enc_req)
# Read from the server, send the data to the client.
resp1 = client.recv(9999) #this gets something unreadable (encrypted?)
#could it be the certificate?
#now what?
# The first time it's certainly the Client Hello message, not encrypted, but in a binary format indeed.
# Just send everything you've just read to the server.
conn.send(resp1)
This is just a quick overview of the idea of the loop you need to write. In reality, you may be able to process both in parallel. You'd also want to be a bit more careful when closing the connection (allowing it to happen in any order while still relaying the last data sent by either party).
As mentioned in the comments, a proxy handling encrypted end-to-end traffic can only pass it on.
Here is a fully working proxy written using circuits that has been fully tested with passing and proxying SSH traffic so it should work equally as well as a pass-through TCP proxy even if SSL is involved:
#!/usr/bin/env python
from uuid import uuid4 as uuid
from circuits import Component
from circuits.net.events import close, connect, write
from circuits.net.sockets import TCPClient, TCPServer
class Client(Component):
channel = "client"
def init(self, sock, host, port, channel=channel):
self.sock = sock
self.host = host
self.port = port
TCPClient(channel=self.channel).register(self)
def ready(self, *args):
self.fire(connect(self.host, self.port))
def disconnect(self, *args):
self.fire(close(self.sock), self.parent.channel)
def read(self, data):
self.fire(write(self.sock, data), self.parent.channel)
class Proxy(Component):
channel = "server"
def init(self, bind, host, port):
self.bind = bind
self.host = host
self.port = port
self.clients = dict()
TCPServer(self.bind).register(self)
def connect(self, sock, host, port):
channel = uuid()
client = Client(
sock, self.host, self.port, channel=channel
).register(self)
self.clients[sock] = client
def disconnect(self, sock):
client = self.clients.get(sock)
if client is not None:
client.unregister()
del self.clients[sock]
def read(self, sock, data):
client = self.clients[sock]
self.fire(write(data), client.channel)
app = Proxy(("0.0.0.0", 3333), "127.0.0.1", 22)
from circuits import Debugger
Debugger().register(app)
app.run()
Related
So in other words it's just stuck after 1st response from the browser.
I'm new to socket programming, but here is stupid question, do I need
to close client socket after the response from the server is sent? In
handleData after self.client.send(). Or what could be the reason
of this behaviour? Tried to use select here to make it asynchronous
but it didn't help. Same issue. After I get request from the client
with headers, I go to the next iteration of the loop, exactly to the
recv() and it is stuck. The client is not sending info anymore and
doesn't show the info that is sent from the server. In this example it
is "Info/Message was retrieved!".
P.S. DEFAULT_ENCODING = "utf-8"
DEFAULT_CLIENTS_AMOUNT = 5
SERVER_IP = "127.0.0.1"
SERVER_PORT = 8080
class Server:
def __init__(self, socket_family: int = socket.AF_INET, socket_type: int = socket.SOCK_STREAM, ip_address: str = SERVER_IP, port: int = SERVER_PORT):
self.addr: Tuple[str, int] = ip_address, port
self.serv_heart: socket.socket = socket.socket(socket_family, socket_type)
self.client: Optional[socket.socket] = None
def _launchSyncServer(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
print("Starting server...")
self.serv_heart.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
self.serv_heart.bind(self.addr)
self.serv_heart.listen(clients)
print(f"Server started at {str(self.addr)}")
print(f"Visible: http://{self.addr[0]}:{self.addr[1]}")
def _acceptConnections(self):
client_socket, client_addr = self.serv_heart.accept()
print(f"Client connected from {str(client_addr)}")
self.client = client_socket
def _handleData(self):
while True:
rawData: str = self.client.recv(128).decode(DEFAULT_ENCODING)
if not rawData:
self.client.close()
break
self.client.send("Info/Message was retrieved!".encode(DEFAULT_ENCODING))
self.client.close()
def serverLoop(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
self._launchSyncServer(clients=clients)
while True:
self._acceptConnections()
self._handleData()
it's just stuck after 1st response from the browser.
Your server doesn't implement HTTP nor any other browser protocol, so a browser is not an appropriate client for it; better use something like nc localhost 8080.
do I need to close client socket after the response from the server is sent?
That depends on how you want the communication to be conducted. If you want the client to only send one message, you can close the server's connection right after responding. If you want the client to be able to send multiple messages, the server as it is is prepared for that, and the client needs to indicate that it's finished by closing its socket or at least shutting down writing.
Note that in general, you can't be sure that a message is reveived in whole with a single recv call - it might be split and returned piece by piece through several calls of recv.
The task is building two files client.py and server.py. I am able to connect the client to the server. The problem I encounter is when I trying to send a get request like client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n"), I do not how to return the file suc.txt to the client from the server side. The scene is a client request file from a server and what the server returns is the respond header and the requested file.
What I wrote so far :
Client:
import socket
target_host = "127.0.0.1"
target_port = 5050
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host,target_port))
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
response = client.recv(1024)
print(response.decode())
Server:
import socket
import threading
import urllib.request
HEADER = 64
PORT = 5050
HOST = socket.gethostbyname(socket.gethostname())
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST,PORT))
def handleClient(conn, addr):
print (f"[NEW CONNECTION {addr} connected. ")
connected = True
while connected:
conn.send()
def start():
server.listen()
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handleClient, args=(conn,addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.activeCount()} ")
print ("server is starting...")
start()
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
The "b..." should be b"...", i.e. you want to specify a sequence of bytes and not a string with a b as first character.
I do not how to return the file suc.txt to the client from the server side
You basically ask very broadly how to read an HTTP request, extract information from it, create a proper response and send it. All what you code so far does is create a listener socket, so you are far away from your ultimate goal.
There are two major ways to tackle this: the easy one is to use a library like http.server to implement the complexity of HTTP for you. The documentation contains actual examples on how to do this and there are many more examples on the internet for this.
The harder option is to study the actual HTTP standard and implement everything yourself based on this standard. Expecting that somebody explains the complex standard here and describes how to implement it would be a too broad question.
I am trying to set up a local server so that other PCs on the same local network can connect to it. When trying to do so, on the client side, I get the following error:
[Errno 10061] No connection could be made because the target machine actively refused it
I have been searching around for hours and still couldn't resolve this issue. I tried turning off my Firewall too, but nothing.
These are my server and client codes:
Server Code:
import socket
import threading
import SocketServer
import datetime
ver_codes = []
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
print threading.current_thread().isDaemon()
data = self.request.recv(1024)
command = data.split()[0]
if(command=="login"):
if(logged_in(data.split()[1])==False):
self.request.sendall(login(data.split()[1], data.split()[2]))
else:
self.request.sendall("already in")
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print "Received: {}".format(response)
finally:
sock.close()
def logged_in(id_num):
for i in ver_codes:
if(i[0]==id_num):
return True
return False
def login(username, password):
login_file = open("Login.txt", "r")
match = login_file.readline()
while(match!="*"):
if(match.split()[0]==username):
if(match.split()[1]==password):
ver_codes.append([match.split()[0], encryption_code(match.split()[2])])
login_file.close()
return "{} {}".format(match.split()[2], encryption_code(match.split()[2]))
print "And Here"
match = login_file.readline()
return "Denied"
login_file.close()
def encryption_code(to_encrypt):
now = datetime.datetime.now()
return int(str(now.microsecond)) * int(to_encrypt)
if __name__ == "__main__":
HOST, PORT = "localhost", 7274
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
print server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = False
server_thread.start()
print "Server loop running in thread:", server_thread.name
Client Code:
import socket
import sys
HOST, PORT = "localhost", 7274
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.sendall("login mahdiNolan m1373")
received = sock.recv(1024)
finally:
sock.close()
I really appreciate any help you could give me!
Thanks A LOT beforehand!
Your issue is because you're listening on localhost - this will only accept connections from the local machine.
If you want to accept connections from anywhere, instead of "localhost" just pass the empty string "". This is equivalent to specifying INADDR_ANY to the C sockets API - see the ip man page for more information, or this page also looks like it has some useful explanation. In short, this means "accept connections on any local interface".
Instead of the empty string you can instead specify an IP address of a local interface to only accept connections on that interface - it's unlikely you need to do this unless you machine has multiple network cards inside it (e.g. acting as a gateway) and you only want to serve requests on one of the networks.
Also, on the client side you should use the actual address of the machine - replace "localhost" with the IP address or hostname of the server machine. For example, something like "192.168.0.99". If you want to find the IP address of the server under Windows, open a DOS window and run the ipconfig command, look for the line with IPv4 Address (assuming you've got an IPv4 network which is very likely).
The Windows firewall will also block the server from accepting connections as you've already found, but you shouldn't need to disable it - as soon as you run your server you should see a popup window where you can instruct it to accept connections (that was on Windows 7, it might be different on other versions). In any case, turning the software firewall off should allow everything to work, although whether that's a security risk is a matter outside of the scope of this question.
I'm writing an application that will make use of Python's HTTPServer and BaseHTTPRequest. At some point I figured that due to the sensitive nature of the data user might want to send, an implementation of SOCKS would be useful. The problem is - the application is planned to run on a non-standard port and thus it would be useful if it could talk to both plaintext and SSL connections. I've found there a way to make HTTPServer use SSL:
import BaseHTTPServer, SimpleHTTPServer
import ssl
httpd = BaseHTTPServer.HTTPServer(('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='path/to/localhost.pem', server_side=True)
httpd.serve_forever()
Is there a way to create a socket class that would handle both SSL and plaintext connections? A neat way to detect SSL (i.e. some magic bytes)? The alternative would be to allocate two ports, but that's way less cool.
I've investigated the problem a little bit.
It's easy to make a socket behave like two different servers (depending on the type of data received). What's bad here is that python's _ssl library reads directly from socket._socket, which is a native python object and therefore can't be hooked normally.
One way is to write a C module that will hook native python socket.
Another solution is to have 1 frontend and 2 backends (https and http). Frontend listens on 4443 and decides whether it should commutate connection with https backend or http backend. You can add the same handlers to the both servers and they'll behave in the same way. Another problem is that on backend we don't know the ip of the client, but there are workarounds (Like the dict {(Frontend to backend source port number): Client IP} that frontend will be keeping and backends will be looking at).
Comparing with the C solution, the second looks quite dirty, but here it is.
import BaseHTTPServer, SimpleHTTPServer
import ssl
import socket
import select
import threading
FRONTEND_PORT = 4443
BACKEND_PORT_SSL = 44431
BACKEND_PORT_HTTP = 44432
HOST = 'localhost'
httpd_ssl = BaseHTTPServer.HTTPServer((HOST, BACKEND_PORT_SSL), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd_ssl.socket = ssl.wrap_socket (httpd_ssl.socket, certfile='key.pem', server_side=True)
httpd_direct = BaseHTTPServer.HTTPServer((HOST, BACKEND_PORT_HTTP), SimpleHTTPServer.SimpleHTTPRequestHandler)
def serve_forever(http_server):
http_server.serve_forever()
def categorize(sock, addr):
data = sock.recv(1)
if data == '\x16':
port = BACKEND_PORT_SSL
else:
port = BACKEND_PORT_HTTP
other_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
other_sock.connect((HOST, port))
other_sock.send(data)
inp = [sock, other_sock]
select_timeout = 1.0
try:
while 1:
r,w,x = select.select(inp,[],[],select_timeout)
if not r:
continue
for s in r:
o_s = inp[1] if inp[0]==s else inp[0]
buf = s.recv(4096)
if not buf:
raise socket.error
o_s.send(buf)
except socket.error:
pass
finally:
for s in inp:
s.close()
threading.Thread(target=serve_forever, args=(httpd_ssl,)).start()
threading.Thread(target=serve_forever, args=(httpd_direct,)).start()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((HOST, FRONTEND_PORT))
sock.listen(10)
while True:
conn, addr = sock.accept()
threading.Thread(target=categorize, args=(conn, addr)).start()
I solved the problem with a helper class that peeks into the received data upon accept(), and then returns either a wrapped or the naked socket:
class SmartServerSocket:
def __init__( self, sock ):
self.sock = sock
# delegate methods as needed
_delegate_methods = [ "fileno" ]
for method in _delegate_methods:
setattr( self, method, getattr( sock, method ) )
def accept( self ):
(conn, addr) = self.sock.accept()
if conn.recv( 1, socket.MSG_PEEK ) == "\x16":
return (ssl.wrap_socket( conn, certfile='path/to/localhost.pem', server_side=True ), addr)
else:
return (conn, addr)
httpd = BaseHTTPServer.HTTPServer( ('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler )
httpd.socket = SmartServerSocket( httpd.socket )
httpd.serve_forever()
If you like, you can give the server object to the SmartServerSocket constructor and have accept() set a special member variable there to the detected protocol, so you can examine this in the RequestHandler instance.
Yes, there acutally is.
But I don't know of any client or server that implements STARTLS for HTTP. It's commonly used for IMAP and SMTP, but for HTTP unfortunately there don't seem to be any implementations, it's still common practice to serve HTTP on a different port then HTTPS.
For fun, I'm writing a minimal IRC server with asynchat. I'm trying to clear up a few fundamentals (my specific questions follow the code). I've decided not to use anything in Twisted just so I can implement a little more myself. First, the code I have:
import asyncore,asynchat
import socket
class Connection(asynchat.async_chat):
def __init__(self, server, sock, addr):
asynchat.async_chat.__init__(self, sock)
self.set_terminator('\n')
self.data = ""
print "client connecting:",addr
# do some IRC protocol initialization stuff here
def collect_incoming_data(self, data):
self.data = self.data + data
def found_terminator(self):
print self.data
self.data = ''
class Server(asyncore.dispatcher):
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.bind((host, port))
self.listen(5)
def handle_accept(self):
conn, addr = self.accept()
Connection(self, conn, addr)
def handle_close(self):
self.close()
s = Server('127.0.0.1',5006)
asyncore.loop()
So, in my mind, this code structure is similar to a Twisted client factory: the Server class is initialized once and basically instantiates Connection every time a client connects. First question: is the best way to keep track of all connected clients by storing all of the Connections in a list within Server?
Also, I don't understand how I am to know when a specific client closes their connection to my socket? Connection implements asynchat (and by extension asyncore) but adding the handle_close() callback to the Connection class doesn't fire when a client disconnects. It seems to be only for when the bound socket on the server is destroyed. I don't see any methods for this purpose. This socket always stays open, whether or not clients connect, right?
to handle client side closed connections check the handle_error method, does your client issue a clean close connection?
handle_error() :Called when an exception is raised and not otherwise handled. The default version prints a condensed traceback.
hope it helps.