I'm writing an application that will make use of Python's HTTPServer and BaseHTTPRequest. At some point I figured that due to the sensitive nature of the data user might want to send, an implementation of SOCKS would be useful. The problem is - the application is planned to run on a non-standard port and thus it would be useful if it could talk to both plaintext and SSL connections. I've found there a way to make HTTPServer use SSL:
import BaseHTTPServer, SimpleHTTPServer
import ssl
httpd = BaseHTTPServer.HTTPServer(('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='path/to/localhost.pem', server_side=True)
httpd.serve_forever()
Is there a way to create a socket class that would handle both SSL and plaintext connections? A neat way to detect SSL (i.e. some magic bytes)? The alternative would be to allocate two ports, but that's way less cool.
I've investigated the problem a little bit.
It's easy to make a socket behave like two different servers (depending on the type of data received). What's bad here is that python's _ssl library reads directly from socket._socket, which is a native python object and therefore can't be hooked normally.
One way is to write a C module that will hook native python socket.
Another solution is to have 1 frontend and 2 backends (https and http). Frontend listens on 4443 and decides whether it should commutate connection with https backend or http backend. You can add the same handlers to the both servers and they'll behave in the same way. Another problem is that on backend we don't know the ip of the client, but there are workarounds (Like the dict {(Frontend to backend source port number): Client IP} that frontend will be keeping and backends will be looking at).
Comparing with the C solution, the second looks quite dirty, but here it is.
import BaseHTTPServer, SimpleHTTPServer
import ssl
import socket
import select
import threading
FRONTEND_PORT = 4443
BACKEND_PORT_SSL = 44431
BACKEND_PORT_HTTP = 44432
HOST = 'localhost'
httpd_ssl = BaseHTTPServer.HTTPServer((HOST, BACKEND_PORT_SSL), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd_ssl.socket = ssl.wrap_socket (httpd_ssl.socket, certfile='key.pem', server_side=True)
httpd_direct = BaseHTTPServer.HTTPServer((HOST, BACKEND_PORT_HTTP), SimpleHTTPServer.SimpleHTTPRequestHandler)
def serve_forever(http_server):
http_server.serve_forever()
def categorize(sock, addr):
data = sock.recv(1)
if data == '\x16':
port = BACKEND_PORT_SSL
else:
port = BACKEND_PORT_HTTP
other_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
other_sock.connect((HOST, port))
other_sock.send(data)
inp = [sock, other_sock]
select_timeout = 1.0
try:
while 1:
r,w,x = select.select(inp,[],[],select_timeout)
if not r:
continue
for s in r:
o_s = inp[1] if inp[0]==s else inp[0]
buf = s.recv(4096)
if not buf:
raise socket.error
o_s.send(buf)
except socket.error:
pass
finally:
for s in inp:
s.close()
threading.Thread(target=serve_forever, args=(httpd_ssl,)).start()
threading.Thread(target=serve_forever, args=(httpd_direct,)).start()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((HOST, FRONTEND_PORT))
sock.listen(10)
while True:
conn, addr = sock.accept()
threading.Thread(target=categorize, args=(conn, addr)).start()
I solved the problem with a helper class that peeks into the received data upon accept(), and then returns either a wrapped or the naked socket:
class SmartServerSocket:
def __init__( self, sock ):
self.sock = sock
# delegate methods as needed
_delegate_methods = [ "fileno" ]
for method in _delegate_methods:
setattr( self, method, getattr( sock, method ) )
def accept( self ):
(conn, addr) = self.sock.accept()
if conn.recv( 1, socket.MSG_PEEK ) == "\x16":
return (ssl.wrap_socket( conn, certfile='path/to/localhost.pem', server_side=True ), addr)
else:
return (conn, addr)
httpd = BaseHTTPServer.HTTPServer( ('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler )
httpd.socket = SmartServerSocket( httpd.socket )
httpd.serve_forever()
If you like, you can give the server object to the SmartServerSocket constructor and have accept() set a special member variable there to the detected protocol, so you can examine this in the RequestHandler instance.
Yes, there acutally is.
But I don't know of any client or server that implements STARTLS for HTTP. It's commonly used for IMAP and SMTP, but for HTTP unfortunately there don't seem to be any implementations, it's still common practice to serve HTTP on a different port then HTTPS.
Related
The task is building two files client.py and server.py. I am able to connect the client to the server. The problem I encounter is when I trying to send a get request like client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n"), I do not how to return the file suc.txt to the client from the server side. The scene is a client request file from a server and what the server returns is the respond header and the requested file.
What I wrote so far :
Client:
import socket
target_host = "127.0.0.1"
target_port = 5050
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host,target_port))
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
response = client.recv(1024)
print(response.decode())
Server:
import socket
import threading
import urllib.request
HEADER = 64
PORT = 5050
HOST = socket.gethostbyname(socket.gethostname())
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST,PORT))
def handleClient(conn, addr):
print (f"[NEW CONNECTION {addr} connected. ")
connected = True
while connected:
conn.send()
def start():
server.listen()
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handleClient, args=(conn,addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.activeCount()} ")
print ("server is starting...")
start()
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
The "b..." should be b"...", i.e. you want to specify a sequence of bytes and not a string with a b as first character.
I do not how to return the file suc.txt to the client from the server side
You basically ask very broadly how to read an HTTP request, extract information from it, create a proper response and send it. All what you code so far does is create a listener socket, so you are far away from your ultimate goal.
There are two major ways to tackle this: the easy one is to use a library like http.server to implement the complexity of HTTP for you. The documentation contains actual examples on how to do this and there are many more examples on the internet for this.
The harder option is to study the actual HTTP standard and implement everything yourself based on this standard. Expecting that somebody explains the complex standard here and describes how to implement it would be a too broad question.
I have a client class, in which it should connect to the server (in this case, iKettle) and sends and receives data. The IP address should come from another main server (so another class), and this IP address can change.
Below is the client code:
#command codes
_ON = "0x4<LF>"
_OFF = "0x0<LF>"
#base command
_BASE_COMMAND = "set sys output "
_SLEEP_TIME = 0.5
#size of buffer when receiving data
_BUFFER_SIZE = 1024
ip_address = ""
port = 2000
def initialiseSocket(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
return s
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def setOff(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send(self._BASE_COMMAND + self._OFF)
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def kettleResponse(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
data = s.recv(self._BUFFER_SIZE)
print(data)
And this is an example of commands from another class:
kettle.setOn(KETTLEIP)
However, when running this code, it does not print anything.
Can anyone help please?
You're probably connecting and reconnecting too much. Sometimes that simplifies things, but in this case I believe it's making you lose your response.
Try just connecting once, and reusing the socket.
Also, keep in mind that TCP is a byte-oriented protocol, not a message-oriented protocol. IOW, if you send 10k, 10k, 10k, the other side of the TCP socket may receive 5k, 8k, 17k - or even more bizarre possibilities. The total number of bytes will be the same, and the data will arrive in the correct order, but the sizes of the chunks could be totally scrambled. For this reason, most folks use REST with http these days - it's simpler in some ways.
If you're married to TCP, perhaps try my bufsock module (or actually, it's Opensource with a Univ Calif Irvine copyright, but I wrote it while I was working for them, and obtained permission from them to release it). It's at http://stromberg.dnsalias.org/~strombrg/bufsock.html . Its methods behave closer to what most people expect out of TCP.
HTH.
My guess is that you should reuse the socket as #dstromberg indicates:
class Kettle:
# other stuff omitted...
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(s) # pass the socket on
def kettleResponse(self, s):
# don't open a new socket, just reuse s
data = s.recv(self._BUFFER_SIZE)
print(data)
I'm trying to access socket objects from memory address "socket._socketobject object at 0x7f4c39d78b40" and use it for another function at different times. The clients are connected to port 9999 and I want the server to react with each one at a later stage while keeping the connection up.
def sock_con(host,port):
host = host
port = port
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((host, port))
sock.listen(5)
while True:
client, address = sock.accept()
print client
print type(client)
print "Server (%s, %s) connected" % address
mongoconn = connectionx('IP_Clients')
key = {'addresses':'192.168.11.1'}
data = {'client':str(client), 'addresses':address}
mongoconn.update(key, data)
client.settimeout(60)
The next code is at a different module which can be used at anytime:
import os,sys
import socket
currentdir = os.path.dirname(os.path.realpath(__file__))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
from mgodb import connectionx
mongoconn = connectionx('IP_Clients')
x= mongoconn.find_one({'addresses':'192.168.11.1'})
client= eval(x['client'])
def send_stuff(client,addresses,arg1):
while True:
try:
#data = client.recv(size)
print data
client.send(arg1)
return data
except:
#raise error('Client disconnected')
client.close()
return False
send_stuff(client,x['addresses'],'test10')
To use sockets later in the same process, just store them at their arrival and find them later. Something like this:
...
clients = {}
while True:
client, addr = server.accept()
clients[addr[0]] = client
So, if you stop the listening loop, or run it in a thread, or you run something else in a thread (doesn't matter), you can get the opened socket object from dictionary clients by the client's IP address.
client = clients.get("192.168.1.1")
But you should count in the port as well for detection, because there may be two different clients contacting you from same IP address.
If you want to send an opened socket to another process, well, it is doable but not worth the trouble.
You would need to send the socket's filedescriptor ( socket.fileno() ) to another process, and that can be done using Python module sendfds. It can be found on pypi.python.org.
Then, in receiving process, you would have to construct the socket wrapper object around it manually or trick somehow the existing _socket.dll/.so and socket.py modules to do it for you.
A lot of work and success dubious. What you should do instead is to use the dictionary to store sockets and create an interface (over socket, PIPE or whatever IPC) to forward messages to and from needed connected sockets.
Finally, you do not have to worry about this mess at all, because Python has asyncore module.
It already does the socket storing into dictionary and other useful stuff. The thing is, you need to know what you want to achieve to be able to adequately tune the asyncore client handler. Set correct buffer sizes etc. etc. But asyncore is elegant and you can easily mix it with existing GUI event loop. asyncore and asynchat are often used when creating push servers or instant-messaging-like systems.
I'm currently making a proxy which sits between the browser and the web. Everything works except https. I'm having troubles understanding some passages of it and haven't found many resources on the web. And so I'm stuck.
The code I'm using is:
conn, addr = server.accept()
request = conn.recv(9999) #get a CONNECT request
conn.send(b'HTTP/1.1 200 Connection estabilished\n\n')
enc_req = conn.recv(9999) #this gets an encrypted request
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #plaintext client
client.connect((host, 443)) #connect to chosen host
client.send(enc_req)
resp1 = client.recv(9999) #this gets something unreadable (encrypted?)
#could it be the certificate?
#now what?
Is the resp1 I'm getting the certificate? And what do I need to do after that? (Or, which is the same, what does usually happens next with https?)
P.S. I know the question is somewhat generic, but please don't judge me too harshly. I've tried researching on the web but all I keep finding is the encryption method used for ssl. I really don't know how to proceed.
I haven't tested this code (and it's mainly pseudo code), but this should give you an idea of what you need to do.
conn, addr = server.accept()
request = conn.recv(9999) #get a CONNECT request
# Here, parse the CONNECT string and get the host and port (not sure if you were doing that already.
# Then, try to connect *before* you tell the client the connection was established (in case it fails)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #plaintext client
client.connect((host, 443)) #connect to chosen host
conn.send(b'HTTP/1.1 200 Connection estabilished\n\n')
# Then loop until the connections are closed.
while True:
# Read from the client, send the data to the server.
enc_req = conn.recv(9999) #this gets an encrypted request
client.send(enc_req)
# Read from the server, send the data to the client.
resp1 = client.recv(9999) #this gets something unreadable (encrypted?)
#could it be the certificate?
#now what?
# The first time it's certainly the Client Hello message, not encrypted, but in a binary format indeed.
# Just send everything you've just read to the server.
conn.send(resp1)
This is just a quick overview of the idea of the loop you need to write. In reality, you may be able to process both in parallel. You'd also want to be a bit more careful when closing the connection (allowing it to happen in any order while still relaying the last data sent by either party).
As mentioned in the comments, a proxy handling encrypted end-to-end traffic can only pass it on.
Here is a fully working proxy written using circuits that has been fully tested with passing and proxying SSH traffic so it should work equally as well as a pass-through TCP proxy even if SSL is involved:
#!/usr/bin/env python
from uuid import uuid4 as uuid
from circuits import Component
from circuits.net.events import close, connect, write
from circuits.net.sockets import TCPClient, TCPServer
class Client(Component):
channel = "client"
def init(self, sock, host, port, channel=channel):
self.sock = sock
self.host = host
self.port = port
TCPClient(channel=self.channel).register(self)
def ready(self, *args):
self.fire(connect(self.host, self.port))
def disconnect(self, *args):
self.fire(close(self.sock), self.parent.channel)
def read(self, data):
self.fire(write(self.sock, data), self.parent.channel)
class Proxy(Component):
channel = "server"
def init(self, bind, host, port):
self.bind = bind
self.host = host
self.port = port
self.clients = dict()
TCPServer(self.bind).register(self)
def connect(self, sock, host, port):
channel = uuid()
client = Client(
sock, self.host, self.port, channel=channel
).register(self)
self.clients[sock] = client
def disconnect(self, sock):
client = self.clients.get(sock)
if client is not None:
client.unregister()
del self.clients[sock]
def read(self, sock, data):
client = self.clients[sock]
self.fire(write(data), client.channel)
app = Proxy(("0.0.0.0", 3333), "127.0.0.1", 22)
from circuits import Debugger
Debugger().register(app)
app.run()
I am trying to set up a local server so that other PCs on the same local network can connect to it. When trying to do so, on the client side, I get the following error:
[Errno 10061] No connection could be made because the target machine actively refused it
I have been searching around for hours and still couldn't resolve this issue. I tried turning off my Firewall too, but nothing.
These are my server and client codes:
Server Code:
import socket
import threading
import SocketServer
import datetime
ver_codes = []
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
print threading.current_thread().isDaemon()
data = self.request.recv(1024)
command = data.split()[0]
if(command=="login"):
if(logged_in(data.split()[1])==False):
self.request.sendall(login(data.split()[1], data.split()[2]))
else:
self.request.sendall("already in")
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print "Received: {}".format(response)
finally:
sock.close()
def logged_in(id_num):
for i in ver_codes:
if(i[0]==id_num):
return True
return False
def login(username, password):
login_file = open("Login.txt", "r")
match = login_file.readline()
while(match!="*"):
if(match.split()[0]==username):
if(match.split()[1]==password):
ver_codes.append([match.split()[0], encryption_code(match.split()[2])])
login_file.close()
return "{} {}".format(match.split()[2], encryption_code(match.split()[2]))
print "And Here"
match = login_file.readline()
return "Denied"
login_file.close()
def encryption_code(to_encrypt):
now = datetime.datetime.now()
return int(str(now.microsecond)) * int(to_encrypt)
if __name__ == "__main__":
HOST, PORT = "localhost", 7274
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
print server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = False
server_thread.start()
print "Server loop running in thread:", server_thread.name
Client Code:
import socket
import sys
HOST, PORT = "localhost", 7274
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.sendall("login mahdiNolan m1373")
received = sock.recv(1024)
finally:
sock.close()
I really appreciate any help you could give me!
Thanks A LOT beforehand!
Your issue is because you're listening on localhost - this will only accept connections from the local machine.
If you want to accept connections from anywhere, instead of "localhost" just pass the empty string "". This is equivalent to specifying INADDR_ANY to the C sockets API - see the ip man page for more information, or this page also looks like it has some useful explanation. In short, this means "accept connections on any local interface".
Instead of the empty string you can instead specify an IP address of a local interface to only accept connections on that interface - it's unlikely you need to do this unless you machine has multiple network cards inside it (e.g. acting as a gateway) and you only want to serve requests on one of the networks.
Also, on the client side you should use the actual address of the machine - replace "localhost" with the IP address or hostname of the server machine. For example, something like "192.168.0.99". If you want to find the IP address of the server under Windows, open a DOS window and run the ipconfig command, look for the line with IPv4 Address (assuming you've got an IPv4 network which is very likely).
The Windows firewall will also block the server from accepting connections as you've already found, but you shouldn't need to disable it - as soon as you run your server you should see a popup window where you can instruct it to accept connections (that was on Windows 7, it might be different on other versions). In any case, turning the software firewall off should allow everything to work, although whether that's a security risk is a matter outside of the scope of this question.