Develop Reference

Translate Oracle SQL Developer SSH Host w/ Local Port Forward Connection to Python

I'm trying to create a Python connection to a remote server through an SSH Jump Host (one I've successfully created in Oracle SQL Developer) but can't replicate in Python. Can connect to SSH Host successfully but fail to forward the port to the remote server due to timeout or error opening tunnels. Safe to assume my code is incorrect rather than server issues. Also need a solution that doesn't use the "with SSHTunnelForwarder() as server:" approach because I need a continuous session similar to OSD/cx_Oracle session rather than a batch processing function.
Similar examples provided here (and elsewhere) using paramiko, sshtunnel, and cx_Oracle haven't worked for me. Many other examples don't require (or at least clearly specify) separate login credentials for the remote server. I expect the critical unclear piece is which local host + port to use, which my SQL Developer connection doesn't require explicitly (although I've tried using the ports OSD chooses, not at the same time).
Closest match I think was best answer from paramiko-port-forwarding-around-a-nat-router
OSD Inputs
SSH Host
- host = proxy_hostname
- port = proxy_port = 22
- username = proxy_username
- password = proxy_password
Local Port Forward
- host = remote_hostname
- port = remote_port = 1521
- automatically assign local port = True
Connection
- username = remote_username
- password = remote_password
- connection type = SSH
- SID = remote_server_sid
Python Code
i.e., analogous code from paramiko-port-forwarding-around-a-nat-router
import paramiko
from paramiko import SSHClient
# Instantiate a client and connect to the proxy server
proxy_client = SSHClient()
proxy_client.connect(
proxy_hostname,
port=proxy_port,
username=proxy_username,
password=proxy_password)
# Get the client's transport and open a `direct-tcpip` channel passing
# the destination hostname:port and the local hostname:port
transport = proxy_client.get_transport()
dest_addr = (remote_hostname,remote_port)
local_addr = ('localhost',55587)
channel = transport.open_channel("direct-tcpip", dest_addr, local_addr)
# Create a NEW client and pass this channel to it as the `sock` (along
# with whatever credentials you need to auth into your REMOTE box
remote_client = SSHClient()
remote_client.connect(
'localhost',
port=55587,
username=remote_username,
password=remote_password,
sock=channel)
Rather than a connection to the remote server I get
transport.py in start_client()
SSHException: Error reading SSH protocol banner

Solution
Finally figured out a solution! Analogous to OSD's automatic local port assignment and doesn't require SSHTunnelForwarder's with statement. Hope it can help someone else- use the question's OSD input variables with...
from sshtunnel import SSHTunnelForwarder
import cx_Oracle
server=SSHTunnelForwarder(
(proxy_hostname,proxy_port),
ssh_username=proxy_username,
ssh_password=proxy_password,
remote_bind_address=(remote_hostname,remote_port))
server.start()
db=cx_Oracle.connect('%s/%s#%s:%s/%s'%(remote_username,remote_password,'localhost',server.local_bind_port,remote_server_sid))
# do something with db
server.close()

Python piramiko, Connection Timed Out Error While Establishing SSH Connection

I am working on a project in Python that will create an Amazon ec2 instance, and establish a SSH and SFTP connection to transfer files and commands between my machine and ec2 instance.
So I began to code, I coded the function that creates an ec2 instance using boto3 library.
# creating a file named sefa.pem that will store the private key
outfile = open('sefa.pem', 'w')
keypair = ec2.meta.client.create_key_pair(KeyName='sefakeypair') # creates key pair
keyout= str(keypair['KeyMaterial']) # reads the key material
outfile.write(keyout) # writes the key material in sefa.pem
# creates the instance finally
response = ec2.create_instances(ImageId='ami-34913254', MinCount=1, MaxCount=1, InstanceType='t2.micro')
After that, I should establish a SSH Connection between my machine and ec2 instance to send command and I also should transfer and bring back files between my machine and ec2 instance.
After research, I found out that there is a Python library called piramiko for establishing SSH Connection and SFTP Connection between my computer and ec2 instance.
I tried to establish a SSH Connection between my computer and ec2 instance, but I have been  facing with the "[Errrno 110]Connection Timed Out Error" for a day. I have been searching the internet for hours, but I couldn't find anything useful.
Here is the code that emerges "Connection Time Out Error":
con = paramiko.SSHClient() # ssh client using paramiko library
con.set_missing_host_key_policy(paramiko.AutoAddPolicy()) # this is needed because of adding policy automautically
k = paramiko.RSAKey.from_private_key_file("sefa.pem") # k reads sefa.pem and stores private key
time.sleep(30) # added this because ec2 should do 2/2 checks before connecting
print("connecting")
con.connect(hostname=PUB_DNS, username="ubuntu", pkey=k, look_for_keys=True) # HERE IS THE ERROR, I CAN'T CONNECT
print("connected")
stdin, stdout, stderr = con.exec_command('echo "TEST"')
print(stdout.readlines())
con.close()
I can not go any further without establishing a connection between my machine and ec2 instance.
 
Do you have any suggestions to solve this problem?
Is there any alternative library to piramiko?

I managed to solve the problem. The problem is my ec2 instance. These solved the issue:
Make sure that instance's security group has ssh deamon and allows you to connect.
Make sure that you have the keypair that you created while creating the instance.
Make sure that you execute chmod 400 keypair.pem

I was facing the same error and here is how I solved it :-
Install openssh-client on your client VM and openssh-server on your server VM.
Don't execute ssh#ip address as this will log you in to your host vm and then IP on both your client and server will be the same, this is the main reason for the error.
3.Instead use
ssh-keyscan ip_address >> ~/.ssh/known_hosts
So know host key is in known hosts and original IP remains.

SSH server routes tunnel by user

Diagram of what I'm trying to accomplish:
$ sftp joe#gatewayserver.horse
SSHCLIENT_JOE --------> GATEWAY_SERVER (does logic by username to determine
| socket to forward the connection to.)
|
\ 127.0.0.1:1030 CONTAINRR_SSHD_SALLY
---> 127.0.0.1:1031 CONTAINER_SSHD_JOE
127.0.0.1:1032 CONTAINER_SSHD_MRAYMOND
This seems closest to what I'm trying to do:
paramiko server mode port forwarding
http://bitprophet.org/blog/2012/11/05/gateway-solutions/
But instead of the client doing a ProxyCommand or requesting a "direct-tcpip" channel, I want the forwarding to be done by the server, invisibly for the client.
I have been trying to do this with a paramiko server by taking the Transport object of the connecting client and making a direct-tcpip channel on behalf of the client, but I'm running into roadblocks.
I'm Using https://github.com/paramiko/paramiko/blob/master/demos/demo_server.py as a template
# There's a ServerInterface class definition that overrides check_channel_request
# (allowing for direct-tcpip and session), and the other expected overides like
# check_auth_password, etc that I'm leaving out for brevity.
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', 2200))
except Exception as e:
print('*** Bind failed: ' + str(e))
traceback.print_exc()
sys.exit(1)
try:
sock.listen(100)
print('Listening for connection ...')
client, addr = sock.accept()
except Exception as e:
print('*** Listen/accept failed: ' + str(e))
traceback.print_exc()
sys.exit(1)
print('Got a connection!')
try:
t = paramiko.Transport(client)
t.add_server_key(host_key)
server = Server()
try:
t.start_server(server=server)
except paramiko.SSHException:
print('*** SSH negotiation failed.')
sys.exit(1)
# Waiting for authentication.. returns an unwanted channel object since
# it isn't the right "kind" of channel
unwanted_chan = t.accept(20)
dest_addr = ("127.0.0.1", 1030) # target container port
local_addr = ("127.0.0.1", 1234) # Arbitrary port on gateway server
# Trying to put words in the client's mouth here.. fails
# What should I do?
print(" Attempting creation of direct-tcpip channel on client Transport")
tunnel_chan = t.open_channel("direct-tcpip", dest_addr, local_addr)
print("tunnel_chan created.")
tunnel_client = SSHClient()
tunnel_client.load_host_keys(host_key)
print("attempting connection using tunnel_chan")
tunnel_client.connect("127.0.0.1", port=1234, sock=tunnel_channel)
stdin, stdout, stderr = tunnel_client.exec_command('hostname')
print(stdout.readlines())
except Exception as e:
print('*** Caught exception: ' + str(e.__class__) + ': ' + str(e))
traceback.print_exc()
try:
t.close()
except:
pass
sys.exit(1)
current output:
Read key: bc1112352a682284d04f559b5977fb00
Listening for connection ...
Got a connection!
Auth attempt with key: 5605063f1d81253cddadc77b2a7b0273
Attempting creation of direct-tcpip channel on client Transport
*** Caught exception: <class 'paramiko.ssh_exception.ChannelException'>: (1, 'Administratively prohibited')
Traceback (most recent call last):
File "./para_server.py", line 139, in <module>
tunnel_chan = t.open_channel("direct-tcpip", dest_addr, local_addr)
File "/usr/lib/python2.6/site-packages/paramiko/transport.py", line 740, in open_channel
raise e
ChannelException: (1, 'Administratively prohibited')
We currently have a straight-forward sftp server where clients connect and they are chroot-ed to their respective ftp directories.
We are wanting to move the clients into lxc containers but don't want to alter how they connect to sftp.. (Since they are probably using gui ftp clients like filezilla.) I'm also not wanting to make a bridge interface and assign new ips to all the containers. Thus the containers don't have separate ips from the host, they share the same network space.
The client containers' sshds would bind to separate ports on localhost. That way they can have unique ports, and the logic of which port is chosen could conceptually be moved out to... a simple server on the physical host.
This is more of a proof-of-concept, and general curiosity on my part.

As I mentioned in a comment above, I don't know anything about paramiko, but I can comment on this from an OpenSSH perspective, and perhaps you can translate the concepts to what you need.
NAT was mentioned in comments. NAT is something done at a lower level than SSH, not something that would be set up on the basis of an SSH login (SOCK5 notwithstanding). You'd implement it in your firewall, not in your SSH configuration. The way ProxyCommand works is to negotiate the SSH connection, then hand the client to the next hop saying "Here, negotiate with this guy too." It's something implemented right inside the SSH protocol.
You may not be totally out of luck.
A standard ProxyCommand setup might look like this, with the target port specified on the client side:
host joecontainer
User joe
ProxyCommand ssh -x -a -q -Wlocalhost:1031 gatewayserver.horse
An older fashioned version of this might have used Netcat:
host joecontainer
User joe
ProxyCommand ssh -x -a -q gatewayserver.horse nc localhost 1031
The idea here is that nc localhost 1031 is the command which provides SSH access to the "next hop" in the SSH chain. You could run any command here as long as the result of that command is a connection to an SSH daemon.
But you want the port selection to be handled by the GATEWAY rather than by the client. And therein lies a bit of a crunch, because the SSH daemon is only using the target username to select which user account's authorized_keys file to read. It's the keys which are important, not the user. By the time the server gets around to running an script or command associated with a user, the SSH negotiation is complete, and it's too late to forward the connection on to the next hop.
So ... you might consider having everyone connect to a common user, and then have the port selection done on the basis of SSH key. This is the way, for example, gitolite handles users. In your case, Joe and Sally could both connect to common#gatewayserver.horse using their DSA or RSA key.
The fun part is that all your port selection gets handled within the "common" user's .ssh/authorized_keys file. The file would look something like this:
command="/usr/bin/nc localhost 1030",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa ... sally#office
command="/usr/bin/nc localhost 1031",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa ... joe#home
You can read about this under the "AUTHORIZED_KEYS FILE FORMAT" section of the sshd(8) man page.
To use this technique, we still need a client-side ProxyCommand, but because port selection happens server-side based on the client's key, everyone can use exactly the same ProxyCommand:
host mycontainer
ProxyCommand ssh -xaq common#gatewayserver.horse
Sally and Joe will run ssh-keygen to create a key pair if they haven't already. They'll send you the public key which you'll add to ~common/.ssh/authorized_keys using the format above.
When Joe connects using his key, the ssh server only runs the nc command associated with his key. And because of the ProxyCommand, that netcat's output is interpreted as a "next hope" for SSH.
I've tested this with sftp (running on my eventual target, akin to your container) and it appears to work for me.
SSH is magic. :-)

Attempting creation of direct-tcpip channel on client Transport
*** Caught exception: <class '[...]'>: (1, 'Administratively prohibited')
The container ssh server is rejecting your direct-tcpip channel request because it has been configured to refuse these requests. I gather the intent here is to proxy SFTP sessions to the correct container? And I imagine the container SSH server has been configured in the usual fashion to only permit these people to do SFTP? SFTP sessions go through a session channel, not a direct-tcpip channel.
I'm not a python coder and can't give you the specific paramiko code, but your relay agent should open a session channel to the container server and invoke the "sftp" subsystem. And if possible, your relay agent should only do this when the remote client requested an SFTP session, not for other types of channel requests.

Accessing a socket coded using python TCPServer from outside (external machine)

I am making python server using TCPServer. Things that I can do are:
1. use curl from other terminal (curl 10.157.41.14:8444 --data "var1=10&var2=15")
2. use firefox in Xming and type "localhost:8444"
The problem is that when I try to access the server from outside, I can not
I thought this is a problem with the code but I could not find any error with my code.
This is how I configure host and port on my python code:
from SocketServer import TCPServer, StreamRequestHandler
import socket
class MyRequestHandler(StreamRequestHandler):
def handle(self):
print "A client tried to connect";
self.wfile.write("success/n this is a replay from the server");
server = TCPServer((socket.gethostname(), 8444), MyRequestHandler)
host, port = server.socket.getsockname()
address = host + ":" + str(port)
message = "Started string-length server at " + address
print message
server.serve_forever()
I tried changing
server = TCPServer((socket.gethostname(), 8444), MyRequestHandler)
to
server = TCPServer('', 8444), MyRequestHandler);
and to:
server = TCPServer('0.0.0.0', 8444), MyRequestHandler);
None of these works on my case. So what I did next is trying to find if it is a problem in my network configuration or firewall. The problem is that I am not an export on these. Here is what I did:
user#ip-10-157-41-14:/var/www/server$ netstat -tnlpen | grep "8444\|PID"
(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
tcp 0 0 10.157.41.14:8444 0.0.0.0:* LISTEN 1014 106226915 31541/python
Then, I did this to find out more about the firewall:
user#ip-10-157-41-14:/var/www/server$ sudo iptables -L
[sudo] password for user:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
another thing I did was opening two terminals (in the same machine)
on terminal-1 I did "nc -l 5000"
on terminal-2 I did "nc 10.157.41.14 5000"
It seems to work. but I do not have access to another Linux machine to try it from another machine.
The problem is that I do not understand the above 2 commands. I spent hours trying to search stackoverflow and other sites for a solution but I did not find a solution that works for me.
In the past(in the same machine), I was able to write cgi python server where I call it using ajax call but I had to write a .htaccess file in the same directory of the python that I was using. The following is my .htaccess file:
Options +ExecCGI
AddHandler cgi-script .py
But the problem I am having now is completely different problem.

My problem turned out to be an Amazon EC2 specific problem.
I am running my server in an instance in Amazon EC2
Amazon instance is located in a Virtual Private Cloud (VPC) with an IP.
You decide if the instance is exposed to the Internet or to remain private.
So, there is an extra level of protection on top of the the EC2 instance.
from the EC2 Dashboard, you can specify open port numbers and close other ones.
It does not matter what I do in the instance level because the configuration in the dashboard is not allowing ports to be open. So, basically I added open ports to the security group in Amazon EC2 and it worked perfectly.

How to ssh over HTTP proxy in Python Paramiko?

I am adapting a Python script to be OS independent and run on Windows. I have changed its ssh system calls to calls to paramiko functions. I am stuck with the issue of http proxy authentication. In Unix (actually Cygwin) environment I would use ~/.ssh/config
Host *
ProxyCommand corkscrew http-proxy.example.com 8080 %h %p
Is there a way to obtain the same using paramiko (or the Python ssh module) either using or not using corkscrew? This post seems to suggest that, but I don't know how.
Note: I am behind a firewall that allows me to use only port 80. I need to control Amazon ec2 instances so I configured the sshd server on those machines to listen to port 80. Everything is working fine in my cygwin+corkscrew prototype, but I would like to have a Python script that works without Cygwin.

You can use any pre-established session to paramiko via the sock parameter in SSHClient.connect(hostname,username,password,...,sock).
Below is a code-snippet that tunnels SSH via HTTP-Proxy-Tunnel (HTTP-CONNECT). At first the connection to the proxy is established and the proxy is instructed to connect to localhost:22. The result is a TCP tunnel over the established session that is usually used to tunnel SSL but can be used for any tcp based protocol.
This scenario works with a default installation of tinyproxy with Allow <yourIP> and ConnectPort 22 being set in /etc/tinyproxy.conf. The proxy and the sshd are running on the same host in my example but all you need is any proxy that allows you to CONNECT to your ssh port. Usually this is restricted to port 443 (hint: if you make your sshd listen on 443 this will work with most of the public proxies even thought I do not recommend to do this for interop and security reasons). If this ultimately allows you to bypass your firewall depends on what kind of firewall is employed. If there's no DPI/SSL-Interception features involved, you should be fine. If there's SSL-Interception involved you could still try to tunnel it via ssl or as part of HTTP payload :)
import paramiko
import socket
import logging
logging.basicConfig(loglevel=logging.DEBUG)
LOG = logging.getLogger("xxx")
def http_proxy_tunnel_connect(proxy, target,timeout=None):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(timeout)
sock.connect(proxy)
LOG.debug("connected")
cmd_connect = "CONNECT %s:%d HTTP/1.1\r\n\r\n"%target
LOG.debug("--> %s"%repr(cmd_connect))
sock.sendall(cmd_connect)
response = []
sock.settimeout(2) # quick hack - replace this with something better performing.
try:
# in worst case this loop will take 2 seconds if not response was received (sock.timeout)
while True:
chunk = sock.recv(1024)
if not chunk: # if something goes wrong
break
response.append(chunk)
if "\r\n\r\n" in chunk: # we do not want to read too far ;)
break
except socket.error, se:
if "timed out" not in se:
response=[se]
response = ''.join(response)
LOG.debug("<-- %s"%repr(response))
if not "200 connection established" in response.lower():
raise Exception("Unable to establish HTTP-Tunnel: %s"%repr(response))
return sock
if __name__=="__main__":
LOG.setLevel(logging.DEBUG)
LOG.debug("--start--")
sock = http_proxy_tunnel_connect(proxy=("192.168.139.128",8888),
target=("192.168.139.128",22),
timeout=50)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname="192.168.139.128",sock=sock, username="xxxx", password="xxxxx")
print "#> whoami \n%s"% ssh.exec_command("whoami")[1].read()
output:
DEBUG:xxx:--start--
DEBUG:xxx:connected
DEBUG:xxx:--> 'CONNECT 192.168.139.128:22 HTTP/1.1\r\n\r\n'
DEBUG:xxx:<-- 'HTTP/1.0 200 Connection established\r\nProxy-agent: tinyproxy/1.8.3\r\n\r\n'
#> whoami
root
here are some other resources on how to tunnel through proxies. Just do whatever is needed to establish your tunnel and pass the socket to SSHClient.connect(...,sock)

There's paraproxy, which implements proxy support for Paramiko.
The post you linked to suggets that Paramiko can operate over an arbitrary socket, but that doesn't appear to be the case. In fact, paraproxy works by completing replacing specific methods inside paramiko, since the existing code simply calls socket.socket() to obtain a socket and does not offer any way of hooking in a proxy.