Develop Reference

Paramiko with continuous stdout while running remote python script

I'm trying to run a remote Python script using Paramiko and have it forward anything Python prints back to the client in realtime (i.e. continuous stdout). I connect to my server by calling my class with the following:
class SSH:
client = None
def __init__(self, address, username, password):
self.client = client.SSHClient()
self.client.set_missing_host_key_policy(client.AutoAddPolicy())
self.client.connect(address, username=username, password=password, look_for_keys=False)
I then send commands to the server through my send_command function:
def send_command(self, command):
if(self.client):
stdin, stdout, stderr = self.client.exec_command(command)
for i in range(5): # just print 5 bytes
print(stdout.channel.recv(1))
time.sleep(0.1)
else:
print("Connection not opened.")
Normally, this would work with any continuous/looping command that fills the buffer on stdout as it loops. My problem is that for some reason stdout is filled only when the Python script finishes running, and anything that Python would output comes up only AFTER the script finishes. I want it to print while the script is running. This is the test script I'm using:
from time import sleep
print("Test.")
sleep(1)
print("Test again.")
sleep(2)
print("Final test.")
Is there a way around this or am I doing anything wrong? Thanks in advance.

Problem solved. The solution was actually very simple. I had to request a psuedo-terminal from the server when running a Python script (command = 'python3.6 test.py'). This is done in Paramiko by simply setting the get_pty bool flag to True. See below (note the get_pty in exec_command):
class SSH:
client = None
def __init__(self, address, username, password):
self.client = client.SSHClient()
self.client.set_missing_host_key_policy(client.AutoAddPolicy())
self.client.connect(address, username=username, password=password, look_for_keys=False)
def send_command(self, command):
if(self.client):
stdin, stdout, stderr = self.client.exec_command(command, get_pty=True)
while not stdout.channel.exit_status_ready():
OUT = stdout.channel.recv(1024)
print(OUT)
else:
print("Connection not opened.")
I'm now successfully continuously printing the output of the Python script in real-time.

python paramiko ssh client then ssh to another server again [duplicate]

I'm rewriting a Bash script I wrote into Python. The crux of that script was
ssh -t first.com "ssh second.com very_remote_command"
I'm having a problem with the nested authentication with paramiko. I wasn't able to find any examples dealing with my precise situation, but I was able to find examples with sudo on a remote host.
The first method writes to stdin
ssh.connect('127.0.0.1', username='jesse', password='lol')
stdin, stdout, stderr = ssh.exec_command("sudo dmesg")
stdin.write('lol\n')
stdin.flush()
The second creates a channel and uses the socket-like send and recv.
I was able to get stdin.write to work with sudo, but it doesn't work with ssh on the remote host.
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('first.com', username='luser', password='secret')
stdin, stdout, stderr = ssh.exec_command('ssh luser#second.com')
stdin.write('secret')
stdin.flush()
print '---- out ----'
print stdout.readlines()
print '---- error ----'
print stderr.readlines()
ssh.close()
...prints...
---- out ----
[]
---- error ----
['Pseudo-terminal will not be allocated because stdin is not a terminal.\r\n', 'Permission denied, please try again.\r\n', 'Permission denied, please try again.\r\n', 'Permission denied (publickey,password,keyboard-interactive).\r\n']
The pseudo-terminal error reminded me of the -t flag in my original command, so I switched to the second method, using a Channel. Instead of ssh.exec_command and later, I have:
t = ssh.get_transport()
chan = t.open_session()
chan.get_pty()
print '---- send ssh cmd ----'
print chan.send('ssh luser#second.com')
print '---- recv ----'
print chan.recv(9999)
chan = t.open_session()
print '---- send password ----'
print chan.send('secret')
print '---- recv ----'
print chan.recv(9999)
...but it prints '---- send ssh cmd ----' and just hangs until I kill the process.
I'm new to Python and none too knowledgeable about networks. In the first case, why does sending the password work with sudo but not with ssh? Are the prompts different? Is paramiko even the right library for this?

I managed to find a solution, but it requires a little manual work. If anyone have a better solution, please tell me.
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('first.com', username='luser', password='secret')
chan = ssh.invoke_shell()
# Ssh and wait for the password prompt.
chan.send('ssh second.com\n')
buff = ''
while not buff.endswith('\'s password: '):
resp = chan.recv(9999)
buff += resp
# Send the password and wait for a prompt.
chan.send('secret\n')
buff = ''
while not buff.endswith('some-prompt$ '):
resp = chan.recv(9999)
buff += resp
# Execute whatever command and wait for a prompt again.
chan.send('ls\n')
buff = ''
while not buff.endswith('some-prompt$ '):
resp = chan.recv(9999)
buff += resp
# Now buff has the data I need.
print 'buff', buff
ssh.close()
The thing to note is that instead of this
t = ssh.get_transport()
chan = t.open_session()
chan.get_pty()
...you want this
chan = ssh.invoke_shell()
It reminds me of when I tried to write a TradeWars script when I was a kid and gave up coding for ten years. :)

Here is a small example using paramiko only (and port forwarding):
import paramiko as ssh
class SSHTool():
def __init__(self, host, user, auth,
via=None, via_user=None, via_auth=None):
if via:
t0 = ssh.Transport(via)
t0.start_client()
t0.auth_password(via_user, via_auth)
# setup forwarding from 127.0.0.1:<free_random_port> to |host|
channel = t0.open_channel('direct-tcpip', host, ('127.0.0.1', 0))
self.transport = ssh.Transport(channel)
else:
self.transport = ssh.Transport(host)
self.transport.start_client()
self.transport.auth_password(user, auth)
def run(self, cmd):
ch = self.transport.open_session()
ch.set_combine_stderr(True)
ch.exec_command(cmd)
retcode = ch.recv_exit_status()
buf = ''
while ch.recv_ready():
buf += ch.recv(1024)
return (buf, retcode)
# The example below is equivalent to
# $ ssh 10.10.10.10 ssh 192.168.1.1 uname -a
# The code above works as if these 2 commands were executed:
# $ ssh -L <free_random_port>:192.168.1.1:22 10.10.10.10
# $ ssh 127.0.0.1:<free_random_port> uname -a
host = ('192.168.1.1', 22)
via_host = ('10.10.10.10', 22)
ssht = SSHTool(host, 'user1', 'pass1',
via=via_host, via_user='user2', via_auth='pass2')
print ssht.run('uname -a')

You can create ssh connection using channel from another ssh connection. See here for more detail.

For a ready made solution check out pxssh from the pxpect project. Look at the sshls.py and ssh_tunnel.py examples.
http://www.noah.org/wiki/Pexpect

Sinas's answer works well but didn't provide all the output from very long commands for me. However, using chan.makefile() allows me to retrieve all the output.
The below works on a system that requires tty and also prompts for sudo password
ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
ssh.connect("10.10.10.1", 22, "user", "password")
chan=ssh.get_transport().open_session()
chan.get_pty()
f = chan.makefile()
chan.exec_command("sudo dmesg")
chan.send("password\n")
print f.read()
ssh.close()

Paramiko Issues - Channel Closed when Executing a Command

I've been trying to run commands on an Overture Box using paramiko to establish the SSH connection.
My script does connect properly to the box, no issues are seen, the problem comes when the script runs the exec_command(cmd) method:
Traceback (most recent call last):
File "test.py", line 39, in <module>
stdin, stdout, stderr = ssh.exec_command("version")
File "/opt/csw/lib/python2.7/site-packages/paramiko/client.py", line 345, in exec_command
chan.exec_command(command)
File "/opt/csw/lib/python2.7/site-packages/paramiko/channel.py", line 60, in _check
return func(self, *args, **kwds)
File "/opt/csw/lib/python2.7/site-packages/paramiko/channel.py", line 229, in exec_command
self._wait_for_event()
File "/opt/csw/lib/python2.7/site-packages/paramiko/channel.py", line 1086, in _wait_for_event
raise e
paramiko.ssh_exception.SSHException: Channel closed.
Same script using a Linux Box as the target host yields the proper results
The code is borrowed, I just found the snippet Googling:
import sys
import time
import select
import paramiko
host = raw_input("Hostname?: ")
i = 1
password = raw_input("Password?: ")
#
# Try to connect to the host.
# Retry a few times if it fails.
#
while True:
print "Trying to connect to %s (%i/30)" % (host, i)
try:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(host, port=22, username="username", password=password)
print "Connected to %s" % host
break
except paramiko.AuthenticationException:
print "Authentication failed when connecting to %s" % host
sys.exit(1)
except:
print "Could not SSH to %s, waiting for it to start" % host
i += 1
time.sleep(2)
# If we could not connect within time limit
if i == 30:
print "Could not connect to %s. Giving up" % host
sys.exit(1)
# Send the command (non-blocking)
stdin, stdout, stderr = ssh.exec_command("version")
# Wait for the command to terminate
while not stdout.channel.exit_status_ready():
# Only print data if there is data to read in the channel
if stdout.channel.recv_ready():
rl, wl, xl = select.select([stdout.channel], [], [], 0.0)
if len(rl) > 0:
# Print data from stdout
print stdout.channel.recv(1024),
#
# Disconnect from the host
#
print "Command done, closing SSH connection"
ssh.close()
I found other questions where it's said that the issue might be related to the SFTP settings in the host, but I'm able to do:
sftp username#hostname
I've Googled for a while but I can't find anything helpful so far.
Thanks in advance for your help.
EDIT: Almost forgot, I know the issue is exactly at:
stdin, stdout, stderr = ssh.exec_command("version")
I've run the whole script interactively and it breaks when I run that line.

Could be that the Overture does not support the SSH exec_command request.
I have needed to use transport.open_session(), and session.get_pty() and session.invoke_shell() to setup an interactive shell session, then use session.send() and session.recv() to write/read to the shell session for Cisco switches and other network appliances, when exec_command() is not available.

here is a workaround for this,
get a transport for your connection.
open a session.
then exec_command(command) and thenrecv()`, no need for sleep.
client.connect("host", port=22, username="user",password= "pass",look_for_keys=False, allow_agent=False)
transport=client.get_transport()
channel = t.open_session()
channel.exec_command(command)
out = channel.recv(-1)
print(out.decode())

Running Sudo Command with paramiko

I am trying to execute a sudo command on a remote machine using python-paramiko, when I execute the command, I bind it with 3 streams, and I use the input stream to pass the password, but it doesn't work, this is the traceback result:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/paramiko/file.py", line 314, in write
self._write_all(data)
File "/usr/local/lib/python2.7/dist-packages/paramiko/file.py", line 439, in _write_all
count = self._write(data)
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 1263,in _write
self.channel.sendall(data)
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 796, in sendall
raise socket.error('Socket is closed')
error: Socket is closed
and this is my python code:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('192.168.0.104', username='cdc',password='123456')
stdin, stdout, stderr = ssh.exec_command("sudo dmesg")
stdin.write("123456\n")
stdin.flush()
print stdout.readlines()
ssh.close()
Any help? Thanks in advance

First of all, have you tried in console with ssh cdc#192.168.0.104 "sudo -S -p '' dmesg". If it also fails, then you might check the sshd settings and the sudoer settings.
If it works well, please add some echo between lines, so that we can know exactly when the exception was thrown. I highly doubt that you should change sudo dmesg to sudo -S -p '' dmesg.
You might also try my wrapper of paramiko. I can use it smoothly to access any CentOS/SuSE node and perform any commands (w/wo sudo privilege):
#!/usr/bin/python
from StringIO import StringIO
import paramiko
class SshClient:
"A wrapper of paramiko.SSHClient"
TIMEOUT = 4
def __init__(self, host, port, username, password, key=None, passphrase=None):
self.username = username
self.password = password
self.client = paramiko.SSHClient()
self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
if key is not None:
key = paramiko.RSAKey.from_private_key(StringIO(key), password=passphrase)
self.client.connect(host, port, username=username, password=password, pkey=key, timeout=self.TIMEOUT)
def close(self):
if self.client is not None:
self.client.close()
self.client = None
def execute(self, command, sudo=False):
feed_password = False
if sudo and self.username != "root":
command = "sudo -S -p '' %s" % command
feed_password = self.password is not None and len(self.password) > 0
stdin, stdout, stderr = self.client.exec_command(command)
if feed_password:
stdin.write(self.password + "\n")
stdin.flush()
return {'out': stdout.readlines(),
'err': stderr.readlines(),
'retval': stdout.channel.recv_exit_status()}
if __name__ == "__main__":
client = SshClient(host='host', port=22, username='username', password='password')
try:
ret = client.execute('dmesg', sudo=True)
print " ".join(ret["out"]), " E ".join(ret["err"]), ret["retval"]
finally:
client.close()

Im sorry i dont have time for details answer but i was able to implement sudo commands on paramiko using this advise
#!/usr/bin/env python
import paramiko
l_password = "yourpassword"
l_host = "yourhost"
l_user = "yourusername"
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(l_host, username=l_user, password=l_password)
transport = ssh.get_transport()
session = transport.open_session()
session.set_combine_stderr(True)
session.get_pty()
#for testing purposes we want to force sudo to always to ask for password. because of that we use "-k" key
session.exec_command("sudo -k dmesg")
stdin = session.makefile('wb', -1)
stdout = session.makefile('rb', -1)
#you have to check if you really need to send password here
stdin.write(l_password +'\n')
stdin.flush()
for line in stdout.read().splitlines():
print 'host: %s: %s' % (l_host, line)

I know this question is kind of old but I was wanting to use sudo and paramiko together, too. It took me a while to figure out this solution. It may not work for everyone but I figured it was worth adding.
def ssh_handler(hostname, username=USER, password=PASS, command=CMD):
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname,
username=username,
password=password)
# set get_pty to True to get a pseudo terminal
stdin, stdout, stderr = ssh.exec_command(prepare_command(command), get_pty=True)
# write and flush the password to the remote terminal
# note that the password will also be written to the terminal from where the script is executed.
stdin.write(password+'\n')
stdin.flush()
response = stdout.read()
ssh.close()
print(response)
def prepare_command(command):
if (not isinstance(command, basestring)):
command = ' ; '.join(command)
command = command.replace('"','\"')
command = 'sudo -s -- " '+command+' " \n'
return command
# kind of a dumb example but you get the point
mycmd = [];
mycmd.append('cd /dir/this/user/doesnt/have/access/to')
mycmd.append('ls -las')
mycmd.append('cat file_in_dir.txt')
ssh_handler(server, command=mycmd)

I was able to implement this solution which was found in other Stackoverflow threads.
https://www.reddit.com/r/learnpython/comments/60taz7/execute_sudo_su_and_input_password_with_paramiko/df94q7s/
The solution for me was to use invoke_shell() when requiring the use of commands with sudo.

How do you execute multiple commands in a single session in Paramiko? (Python)

def exec_command(self, command, bufsize=-1):
#print "Executing Command: "+command
chan = self._transport.open_session()
chan.exec_command(command)
stdin = chan.makefile('wb', bufsize)
stdout = chan.makefile('rb', bufsize)
stderr = chan.makefile_stderr('rb', bufsize)
return stdin, stdout, stderr
When executing a command in paramiko, it always resets the session when you run exec_command.
I want to able to execute sudo or su and still have those privileges when I run another exec_command.
Another example would be trying to exec_command("cd /") and then run exec_command again and have it be in the root directory. I know you can do something like exec_command("cd /; ls -l"), but I need to do it in separate function calls.

Non-Interactive use cases
This is a non-interactive example... it sends cd tmp, ls and then exit.
import sys
sys.stderr = open('/dev/null') # Silence silly warnings from paramiko
import paramiko as pm
sys.stderr = sys.__stderr__
import os
class AllowAllKeys(pm.MissingHostKeyPolicy):
def missing_host_key(self, client, hostname, key):
return
HOST = '127.0.0.1'
USER = ''
PASSWORD = ''
client = pm.SSHClient()
client.load_system_host_keys()
client.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
client.set_missing_host_key_policy(AllowAllKeys())
client.connect(HOST, username=USER, password=PASSWORD)
channel = client.invoke_shell()
stdin = channel.makefile('wb')
stdout = channel.makefile('rb')
stdin.write('''
cd tmp
ls
exit
''')
print stdout.read()
stdout.close()
stdin.close()
client.close()
Interactive use cases
If you have an interactive ssh use case, paramiko can handle it... I personally would drive interactive ssh sessions with scrapli.
Listing all the ways I can think of to use paramiko interactively:
See nagabhushan's answer which uses paramiko interactively
By default, ansible paramiko usage is configurable
By default, exscript ssh uses paramiko
By default, netmiko ssh uses paramiko
By default, scrapli ssh uses paramiko
I might have missed some libraries that use paramiko, but it should be clear that paramiko is used quite extensively by python libraries that control ssh sessions.

Try creating a command string separated by \n character. It worked for me.
For. e.g. ssh.exec_command("command_1 \n command_2 \n command_3")

Strictly speaking, you can't. According to the ssh spec:
A session is a remote execution of a program. The program may be a
shell, an application, a system command, or some built-in subsystem.
This means that, once the command has executed, the session is finished. You cannot execute multiple commands in one session. What you CAN do, however, is starting a remote shell (== one command), and interact with that shell through stdin etc... (think of executing a python script vs. running the interactive interpreter)

You can do that by invoking shell on the client and sending commands. Please refer here
The page has code for python 3.5. I have modified the code a bit to work for pythin 2.7. Adding code here for reference
import threading, paramiko
strdata=''
fulldata=''
class ssh:
shell = None
client = None
transport = None
def __init__(self, address, username, password):
print("Connecting to server on ip", str(address) + ".")
self.client = paramiko.client.SSHClient()
self.client.set_missing_host_key_policy(paramiko.client.AutoAddPolicy())
self.client.connect(address, username=username, password=password, look_for_keys=False)
self.transport = paramiko.Transport((address, 22))
self.transport.connect(username=username, password=password)
thread = threading.Thread(target=self.process)
thread.daemon = True
thread.start()
def close_connection(self):
if(self.client != None):
self.client.close()
self.transport.close()
def open_shell(self):
self.shell = self.client.invoke_shell()
def send_shell(self, command):
if(self.shell):
self.shell.send(command + "\n")
else:
print("Shell not opened.")
def process(self):
global strdata, fulldata
while True:
# Print data when available
if self.shell is not None and self.shell.recv_ready():
alldata = self.shell.recv(1024)
while self.shell.recv_ready():
alldata += self.shell.recv(1024)
strdata = strdata + str(alldata)
fulldata = fulldata + str(alldata)
strdata = self.print_lines(strdata) # print all received data except last line
def print_lines(self, data):
last_line = data
if '\n' in data:
lines = data.splitlines()
for i in range(0, len(lines)-1):
print(lines[i])
last_line = lines[len(lines) - 1]
if data.endswith('\n'):
print(last_line)
last_line = ''
return last_line
sshUsername = "SSH USERNAME"
sshPassword = "SSH PASSWORD"
sshServer = "SSH SERVER ADDRESS"
connection = ssh(sshServer, sshUsername, sshPassword)
connection.open_shell()
connection.send_shell('cmd1')
connection.send_shell('cmd2')
connection.send_shell('cmd3')
time.sleep(10)
print(strdata) # print the last line of received data
print('==========================')
print(fulldata) # This contains the complete data received.
print('==========================')
connection.close_connection()

cmd = 'ls /home/dir'
self.ssh_stdin, self.ssh_stdout, self.ssh_stderr = self.ssh.exec_command(cmd)
print self.ssh_stdout.read()
cmd2 = 'cat /home/dir/test.log'
self.ssh_stdin2, self.ssh_stdout2, self.ssh_stderr2 = self.ssh.exec_command(cmd2)
print self.ssh_stdout2.read()

You can run multiple command using the below technique. Use semicolon to separate the Linux commands
Eg:
chan.exec_command("date;ls;free -m")

If you wish each command to have an effect on the next command you should use:
stdin, stdout, stderr = client.exec_command("command1;command2;command3")
but in some cases, I found that when ";" doesn't work, using "&&" does work.
stdin, stdout, stderr = client.exec_command("command1 && command2 && command3")

You can execute an entire BASH script file for better use, here is the code for that:
import paramiko
hostname = "192.168.1.101"
username = "test"
password = "abc123"
# initialize the SSH client
client = paramiko.SSHClient()
# add to known hosts
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
client.connect(hostname=hostname, username=username, password=password)
except:
print("[!] Cannot connect to the SSH Server")
exit()
# read the BASH script content from the file
bash_script = open("script.sh").read()
# execute the BASH script
stdin, stdout, stderr = client.exec_command(bash_script)
# read the standard output and print it
print(stdout.read().decode())
# print errors if there are any
err = stderr.read().decode()
if err:
print(err)
# close the connection
client.close()
This will execute the local script.sh file on the remote 192.168.1.101 Linux machine.
script.sh (just an example):
cd Desktop
mkdir test_folder
cd test_folder
echo "$PATH" > path.txt
This tutorial explains this in detail: How to Execute BASH Commands in a Remote Machine in Python.