Develop Reference

Python websocket-client WebSocketApp proxy not connecting?

I'm trying to use proxies with a python WebSocketApp included in the websocket module. However, whenever I use this code,
ws=websocket.WebSocketApp('ws://echo.websocket.org:443')
ws.run_forever(http_proxy_host='my proxy here',http_proxy_port=80, # 80 is the proxy port
on_close=on_close)
I get this in the console, and then the socket closes
Connecting proxy...
--- request header ---
CONNECT echo.websocket.org:443 HTTP/1.0
-----------------------
--- response header ---
## SOCKET CLOSED ##
Help please?

If you want a secure connection use a scheme of wss instead of ws in the URI. The default port is 443, so specifying the port is optional. Try this:
websocket.enableTrace(True)
ws=websocket.WebSocketApp('wss://echo.websocket.org') # N.B. use secure socket
ws.run_forever(http_proxy_host='my proxy here', http_proxy_port=80, on_close=on_close)
The reason that it was failing is that the remote server is closing the connection because you were connecting on a secure port on the server, but not using a secure connection on the client.

Unable to change DNS resolver to localhost

I configured my windows machine's DNS server to 127.0.0.1 and on localhost I created a basic python server:
from socket import *
serverPort = 53
serverSocket = socket(AF_INET, SOCK_DGRAM)
serverSocket.bind(('127.0.0.1', serverPort))
print "The server is ready to receive on port: {}".format(serverPort)
while 1:
try:
message, clientAddress = serverSocket.recvfrom(512)
except:
continue
print clientAddress, message
modifiedMessage = "127.0.0.1"
serverSocket.sendto(modifiedMessage, clientAddress)
PS :I know that DNS is a binary protocol and sending ASCII text won't do any good, but I am not trying to make a resolver, I am trying to see with transperancy that how the former works.
When I srarted the server, I am greated with the following output:
(('127.0.0.1', 53945), '.\x9c\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 53945), '.\x9c\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 53945), '.\x9c\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 61362), '\xefc\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 50065), '\xb5\xfc\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 61362), '\xefc\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 61362), '\xefc\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01')
(('127.0.0.1', 52718), '\xc7\x15\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x05tiles\x08services\x07mozilla\x03com\x00\x00\x01\x00\x01')
But unlike as I enticipated, I am still able to open websites. And Wireshark told me that I am making connection to 8.8.8.8(IDK how?).
I tried flushing the DNS cashe from my machine, nada.
What am I missing?
PPS: If I remove the try/catch clause I get this error(a few seconds after the execution of the program):
error: [Errno 10054] An existing connection was forcibly closed by the remote host

You probably have configured Googles 8.8.8.8 as a fallback DNS server.
And since you are destroying the DNS answers, whoever is receiving these broken answers is falling back to the secondary server. The whole path of DNS queries on a typical UN*X machine is quite complicated and the whole system is usually quite robust.

Sending SSL data over a TCP proxy connection in Python

I am facing the following scenario:
I am forced to use an HTTP proxy to connect to an HTTPS server. For several reasons I need access to the raw data (before encryption) so I am using the socket library instead of one of the HTTP specific libraries.
I thus first connect a TCP socket to the HTTP proxy and issue the connect command.
At this point, the HTTP proxy accepts the connection and seemingly forwards all further data to the target server.
However, if I now try to switch to SSL, I receive
error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
indicating that the socket attempted the handshake with the HTTP proxy and not with the HTTPS target.
Here's the code I have so far:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('proxy',9502))
s.send("""CONNECT en.wikipedia.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1
Proxy-Connection: keep-alive
Host: en.wikipedia.org
""")
print s.recv(1000)
ssl = socket.ssl(s, None, None)
ssl.connect(("en.wikipedia.org",443))
What would be the correct way to open an SSL socket to the target server after connecting to the HTTP proxy?

(Note that in generally, it would be easier to use an existing HTTPS library such as PyCurl, instead of implementing it all by yourself.)
Firstly, don't call your variable ssl. This name is already used by the ssl module, so you don't want to hide it.
Secondly, don't use connect a second time. You're already connected, what you need is to wrap the socket. Since Python doesn't do any certificate verification by default, you'll need to verify the remote certificate and verify the host name too.
Here are the steps involved:
Establish your plain-text connection and use CONNECT like you're doing in the first few lines.
Read the HTTP response you get, and make sure you get a 200 status code. (You'll need to read the header line by line).
Use ssl_s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLS1, ca_certs='/path/to/cabundle.pem') to wrap the socket. Then, verify the host name. It's worth reading this answer: the connect method and what it does after wrapping the socket.
Then, use ssl_s as if it was your normal socket. Don't call connect again.

works with python 3
< proxy > is an ip or domain name
< port > 443 or 80 or whatever your proxy is listening to
< endpoint > your final server you want to connect to via the proxy
< cn > is an optional sni field your final server could be expecting
import socket,ssl
def getcert_sni_proxy(cn,endpoint,PROXY_ADDR=("<proxy>", <port>)):
#prepare the connect phrase
CONNECT = "CONNECT %s:%s HTTP/1.0\r\nConnection: close\r\n\r\n" % (endpoint, 443)
#connect to the actual proxy
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(PROXY_ADDR)
conn.send(str.encode(CONNECT))
conn.recv(4096)
#set the cipher for the ssl layer
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
#connect to the final endpoint via the proxy, sending an optional servername information [cn here]
sock = context.wrap_socket(conn, server_hostname=cn)
#retreive certificate from the server
certificate = ssl.DER_cert_to_PEM_cert(sock.getpeercert(True))
return certificate

A Server Behind A Router [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm trying to set up a server behind a router, and I've been able to reduce it to the following problem:
I use:
Siemens SL2-141 Router.
Windows 7 64-bits with Python 2.7.
I run:
server.py:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((IP, 8080))
sock.listen(1)
sock.accept()
print 'success'
sock.close()
client.py:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((IP, 8080))
sock.close()
When IP = '127.0.0.1' it works.
I set up a static IP:
> Network And Sharing Center > Change Adaper Settings > Local Area Connection
> Properties > Internet Protocol Version 4 (TCP/IPv4) > Properties
> Use The Following IP Address:
IP Address: 10.0.0.200
Subnet Mask: 255.0.0.0
Default Gateway: 10.0.0.138
> Use The Following DNS Server Addresses:
Preferred DNS Server: 10.0.0.138
Alternate DNS Server: - - - -
And when IP = '10.0.0.200' it works.
I set up port forwarding on my router:
> http://10.0.0.138/
> Username: Admin
> Password: Admin
> Advanced > Virtual Server > Port Forwarding > Add:
User Defined: Test
From Internal Host IP Address: ALL
Forward to Internal Host IP Address: 10.0.0.200
Protocol: TCP
External Packet Port Start: 8080
External Packet Port End: 8080
Forward To Internal Host Port Start: 8080
Forward To Internal Host Port End: 8080
> Apply (and reboot router, just in case).
To my understanding, if I leave the server IP '10.0.0.200' and set the client IP to my public IP it should work, but it doesn't work ("no connection could be made because the target machine actively refused it").
I tried doing the same thing with an Apache server and the result was similar: browsing localhost worked, browsing the static IP worked, but browsing the public IP didn't work (port 80 gave me the router setup page, and port 8080 just couldn't connect).
Final notes:
I turned off my firewall.
I checked that the port is open (http://www.canyouseeme.org/) and it is.
I checked that the server is listening (netstat -na | find /i "8080") and it does.
Any ideas what's the problem?

To my understanding, if I leave the server IP '10.0.0.200' and set the client IP to my public IP it should work, but it doesn't work.
That's right, if the client is on an external network. On the local network if you use your public address it may or may not work depending on if your router implements NAT reflection (if not it will drop the packets). You should use the your local (private) IP address on your local network. Many routers allow you to configure DNS records for local resources (that override records from the DNS server, implementing a type of "Split DNS"). That way you can use one DNS name to get the correct address.
If you're problem is with a connection from a client on the external network, it sounds like somehow your NAT router is not port forwarding. I don't have your model of router, but I see this line:
From Internal Host IP Address: ALL
And wonder if you have to allow From External.
Also are you sure this rule is enabled? (I only ask because the last time I had a problem like this I had created the forwarding rule correctly but it wasn't enabled.)
If you're still stuck, try removing the rule and re-test if the port looks open to http://www.canyouseeme.org/. I would also re-test when NOT running your server program (to test if the external port scan is misleading, which can happen).
Finally, when you write:
I turned off my firewall.
Do you mean Windows Firewall? You'll want to double check that too because Windows Firewall can allow local connections while blocking remote connections.

How to ssh over HTTP proxy in Python Paramiko?

I am adapting a Python script to be OS independent and run on Windows. I have changed its ssh system calls to calls to paramiko functions. I am stuck with the issue of http proxy authentication. In Unix (actually Cygwin) environment I would use ~/.ssh/config
Host *
ProxyCommand corkscrew http-proxy.example.com 8080 %h %p
Is there a way to obtain the same using paramiko (or the Python ssh module) either using or not using corkscrew? This post seems to suggest that, but I don't know how.
Note: I am behind a firewall that allows me to use only port 80. I need to control Amazon ec2 instances so I configured the sshd server on those machines to listen to port 80. Everything is working fine in my cygwin+corkscrew prototype, but I would like to have a Python script that works without Cygwin.

You can use any pre-established session to paramiko via the sock parameter in SSHClient.connect(hostname,username,password,...,sock).
Below is a code-snippet that tunnels SSH via HTTP-Proxy-Tunnel (HTTP-CONNECT). At first the connection to the proxy is established and the proxy is instructed to connect to localhost:22. The result is a TCP tunnel over the established session that is usually used to tunnel SSL but can be used for any tcp based protocol.
This scenario works with a default installation of tinyproxy with Allow <yourIP> and ConnectPort 22 being set in /etc/tinyproxy.conf. The proxy and the sshd are running on the same host in my example but all you need is any proxy that allows you to CONNECT to your ssh port. Usually this is restricted to port 443 (hint: if you make your sshd listen on 443 this will work with most of the public proxies even thought I do not recommend to do this for interop and security reasons). If this ultimately allows you to bypass your firewall depends on what kind of firewall is employed. If there's no DPI/SSL-Interception features involved, you should be fine. If there's SSL-Interception involved you could still try to tunnel it via ssl or as part of HTTP payload :)
import paramiko
import socket
import logging
logging.basicConfig(loglevel=logging.DEBUG)
LOG = logging.getLogger("xxx")
def http_proxy_tunnel_connect(proxy, target,timeout=None):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(timeout)
sock.connect(proxy)
LOG.debug("connected")
cmd_connect = "CONNECT %s:%d HTTP/1.1\r\n\r\n"%target
LOG.debug("--> %s"%repr(cmd_connect))
sock.sendall(cmd_connect)
response = []
sock.settimeout(2) # quick hack - replace this with something better performing.
try:
# in worst case this loop will take 2 seconds if not response was received (sock.timeout)
while True:
chunk = sock.recv(1024)
if not chunk: # if something goes wrong
break
response.append(chunk)
if "\r\n\r\n" in chunk: # we do not want to read too far ;)
break
except socket.error, se:
if "timed out" not in se:
response=[se]
response = ''.join(response)
LOG.debug("<-- %s"%repr(response))
if not "200 connection established" in response.lower():
raise Exception("Unable to establish HTTP-Tunnel: %s"%repr(response))
return sock
if __name__=="__main__":
LOG.setLevel(logging.DEBUG)
LOG.debug("--start--")
sock = http_proxy_tunnel_connect(proxy=("192.168.139.128",8888),
target=("192.168.139.128",22),
timeout=50)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname="192.168.139.128",sock=sock, username="xxxx", password="xxxxx")
print "#> whoami \n%s"% ssh.exec_command("whoami")[1].read()
output:
DEBUG:xxx:--start--
DEBUG:xxx:connected
DEBUG:xxx:--> 'CONNECT 192.168.139.128:22 HTTP/1.1\r\n\r\n'
DEBUG:xxx:<-- 'HTTP/1.0 200 Connection established\r\nProxy-agent: tinyproxy/1.8.3\r\n\r\n'
#> whoami
root
here are some other resources on how to tunnel through proxies. Just do whatever is needed to establish your tunnel and pass the socket to SSHClient.connect(...,sock)

There's paraproxy, which implements proxy support for Paramiko.
The post you linked to suggets that Paramiko can operate over an arbitrary socket, but that doesn't appear to be the case. In fact, paraproxy works by completing replacing specific methods inside paramiko, since the existing code simply calls socket.socket() to obtain a socket and does not offer any way of hooking in a proxy.