I've successfully implemented Paramiko using exec_command, however, the command I'm running on the remote machine(s) can sometimes take several minutes to complete.
During this time my Python script has to wait for the remote command to complete and receive stdout.
My goal is to let the remote machine "run in the background", and allow the local Python script to continue once it sends the command via exec_command.
I'm not concerned with stdout at this point, I'm just interested in bypassing waiting for stdout to return so the script can continue on while the command runs on the remote machine.
Any suggestions?
Current script:
def function():
ssh_object = paramiko.SSHClient()
ssh_object.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_object.connect(address, port=22, username='un', password='pw')
command = 'command to run'
try:
stdin, stdout, stderr = ssh_object.exec_command(command)
stdout.readlines()
except:
do something else
Thank you!
Use a separate thread to run the command. Usually threads should be cleaned up with the join command (the exception are daemon threads that you expect to run until your program exits). Exactly how you do that depends on the other stuff your program is running. But an example is:
import threading
def ssh_exec_thread(ssh_object, command):
stdin, stdout, stderr = ssh_object.exec_command(command)
stdout.readlines()
def function():
ssh_object = paramiko.SSHClient()
ssh_object.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_object.connect(address, port=22, username='un', password='pw')
command = 'command to run'
thread = threading.Thread(target=ssh_exec_thread, args=(ssh_object, command)
thread.start()
...do something else...
thread.join()
You can make this fancier by passing a Queue to ssh_exec_command and put the result on the queue for processing by your program later.
Related
I wrote this code in Paramiko:
ssh = SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect(hostname, username=user, password=passwd, timeout=3)
session = ssh.invoke_shell()
session.send("\n")
session.send("echo step 1\n")
time.sleep(1)
session.send("sleep 30\n")
time.sleep(1)
while not session.recv_ready():
time.wait(2)
output = session.recv(65535)
session.send("echo step 2\n")
time.sleep(1)
output += session.recv(65535)
I'm trying execute more commands on my Linux server. The problem is my Python code not wait to finish execute command, for example if I'm try to execute sleep 30, the Python not wait 30 seconds for finish execute commands. How can resolve this problem ? I tried with while recv_ready(), but it still does not wait.
Use exec_command: http://docs.paramiko.org/en/1.16/api/channel.html
stdin, stdout, stderr = ssh.exec_command("my_long_command --arg 1 --arg 2")
The following code works for me:
from paramiko import SSHClient, AutoAddPolicy
import time
ssh = SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect('111.111.111.111', username='myname', key_filename='/path/to/my/id_rsa.pub', port=1123)
sleeptime = 0.001
outdata, errdata = '', ''
ssh_transp = ssh.get_transport()
chan = ssh_transp.open_session()
# chan.settimeout(3 * 60 * 60)
chan.setblocking(0)
chan.exec_command('ls -la')
while True: # monitoring process
# Reading from output streams
while chan.recv_ready():
outdata += chan.recv(1000)
while chan.recv_stderr_ready():
errdata += chan.recv_stderr(1000)
if chan.exit_status_ready(): # If completed
break
time.sleep(sleeptime)
retcode = chan.recv_exit_status()
ssh_transp.close()
print(outdata)
print(errdata)
Please note that command history cannot be executed with ssh as is.
See example here: https://superuser.com/questions/962001/incorrect-output-of-history-command-of-ssh-how-to-read-the-timestamp-info-corre
In case you do not need to read the stdout and stderr separately, you can use way more straightforward code:
stdin, stdout, stderr = ssh_client.exec_command(command)
stdout.channel.set_combine_stderr(True)
output = stdout.readlines()
The readlines reads until the command finishes and returns a complete output.
In case you need the output separately, do not be tempted to remove the set_combine_stderr and call readlines on stdout and stderr separately. That might deadlock. See Paramiko ssh die/hang with big output
For a correct code that reads the outputs separately, see Run multiple commands in different SSH servers in parallel using Python Paramiko.
Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
I wrote this code in Paramiko:
ssh = SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect(hostname, username=user, password=passwd, timeout=3)
session = ssh.invoke_shell()
session.send("\n")
session.send("echo step 1\n")
time.sleep(1)
session.send("sleep 30\n")
time.sleep(1)
while not session.recv_ready():
time.wait(2)
output = session.recv(65535)
session.send("echo step 2\n")
time.sleep(1)
output += session.recv(65535)
I'm trying execute more commands on my Linux server. The problem is my Python code not wait to finish execute command, for example if I'm try to execute sleep 30, the Python not wait 30 seconds for finish execute commands. How can resolve this problem ? I tried with while recv_ready(), but it still does not wait.
Use exec_command: http://docs.paramiko.org/en/1.16/api/channel.html
stdin, stdout, stderr = ssh.exec_command("my_long_command --arg 1 --arg 2")
The following code works for me:
from paramiko import SSHClient, AutoAddPolicy
import time
ssh = SSHClient()
ssh.set_missing_host_key_policy(AutoAddPolicy())
ssh.connect('111.111.111.111', username='myname', key_filename='/path/to/my/id_rsa.pub', port=1123)
sleeptime = 0.001
outdata, errdata = '', ''
ssh_transp = ssh.get_transport()
chan = ssh_transp.open_session()
# chan.settimeout(3 * 60 * 60)
chan.setblocking(0)
chan.exec_command('ls -la')
while True: # monitoring process
# Reading from output streams
while chan.recv_ready():
outdata += chan.recv(1000)
while chan.recv_stderr_ready():
errdata += chan.recv_stderr(1000)
if chan.exit_status_ready(): # If completed
break
time.sleep(sleeptime)
retcode = chan.recv_exit_status()
ssh_transp.close()
print(outdata)
print(errdata)
Please note that command history cannot be executed with ssh as is.
See example here: https://superuser.com/questions/962001/incorrect-output-of-history-command-of-ssh-how-to-read-the-timestamp-info-corre
In case you do not need to read the stdout and stderr separately, you can use way more straightforward code:
stdin, stdout, stderr = ssh_client.exec_command(command)
stdout.channel.set_combine_stderr(True)
output = stdout.readlines()
The readlines reads until the command finishes and returns a complete output.
In case you need the output separately, do not be tempted to remove the set_combine_stderr and call readlines on stdout and stderr separately. That might deadlock. See Paramiko ssh die/hang with big output
For a correct code that reads the outputs separately, see Run multiple commands in different SSH servers in parallel using Python Paramiko.
Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
This is really interesting.
I have following scripts on my linux machine:
sleep.py
import time
from datetime import datetime
print(datetime.now())
time.sleep(20)
print('Over!')
print(datetime.now())
loop.py
import time
for i in range(20):
time.sleep(1)
print(i)
I can terminate them directly by ctrl+c if I login through PuTTY or git-bash.
But when I trying to run the Python scripts on Windows console:
test.py
def ssh_pty_command(cmd, ip, username, passwd=None, key_filename=None):
"""run ssh.exec_command with realtime output and return exit_code."""
try:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
logging.debug('Connecting to remote server {}...'.format(ip))
ssh.connect(ip, 22, username, password=passwd,
key_filename=key_filename, timeout=5)
stdin, stdout, stderr = ssh.exec_command(cmd, get_pty=True)
logging.info('ssh_In: {}'.format(cmd))
# print '$: {}'.format(cmd)
for line in iter(stdout.readline, ""):
logging.info('ssh_Out: {}'.format(
line.rstrip('\n').encode('utf-8')))
for err in iter(stderr.readline, ""):
logging.error('ssh_Error: {}'.format(
err.rstrip().encode('utf-8')))
exit_code = stdout.channel.recv_exit_status()
logging.debug('Task exit with code {}.\n'.format(exit_code))
return exit_code
except Exception as err:
logging.error('*** Caught SSH exception: %s: %s' %
(err.__class__, err))
raise
finally:
ssh.close()
ssh_pty_command('python loop.py',ip,username)
ssh_pty_command('python sleep.py',ip,username)
When I press ctrl+c , the loop.py terminated immediately, but the sleep.py waits until the time.sleep(20) is finished and then terminate the execution.
How can I terminate the sleep.py immediately?
Note I did try to use get_pty=True in my exec_command method in my function, but it didn't help.
I guess it should have something to do with the signal sent by Paramiko, but not sure where to dig in...
Ctrl+C signals an interrupt. Python interpreter checks for the interrupt regularly, and raises KeyboardInterrupt exception, when it detects one.
When Paramiko is waiting for an incoming data on a socket, the checking for interrupts is probably suspended (just guessing). So if a remote command is not producing any output, you cannot break the local script.
Your loop.py produces an output using print(i), so your local Python script can be interrupted at the moment it's processing the output.
In any case, it's not the remote script that cannot be interrupted, it's the local script. So it has probably nothing to do with time.sleep as such.
See also:
Stopping python using ctrl+c
https://docs.python.org/3/library/exceptions.html#KeyboardInterrupt
If you actually do not want to wait for the command to finish, your question is not really about Python (nor Paramiko), but about Linux. See Execute remote commands, completely detaching from the ssh connection.
I was trying to execute a python script in my remote machine
The remote command will take 1 minute to complete [ it is a python script, just print something in a loop]
I am seeing my thread throws the command and come out immediately.
I want my thread to wait till it the remote script completes.
is there a way for this?
class Thread_cls(threading.Thread):
def __init__(self,tid,tname,ip,script):
threading.Thread.__init__(self)
print 'kkk'
self.thread_id=tid
self.thread_name=tname
self.ip=ip
self.script=script
def run(self):
print "ENTERED HERE"
stdin,stdout,stderr=self.connect_to_linux_vm(self.ip).exec_command(self.script)
exit_status = stdout.channel.recv_exit_status()
print "out of ssh command"
#pass
def connect_to_linux_vm(self,ip,uname=None,password=None):
ssh_linux = paramiko.SSHClient()
ssh_linux.set_missing_host_key_policy(
paramiko.AutoAddPolicy())
ssh_linux.connect(ip, username=uname,
password=pwd)
return ssh_linux
obj1=Thread_cls(1,'T1','x.x.x.x',"python TEST.py")
obj1.start()
obj1.join()
Change the line
stdin, stdout, stderr = self.connect_to_linux_vm(self.ip).exec_command(self.script)
to two lines:
ssh = self.connect_to_linux_vm(self.ip)
stdin, stdout, stderr = ssh.exec_command(self.script)
otherwise the SSHClient object returned from connect_to_linux_vm() would be immediately destroyed after it invokes exec_command() since there's no other vars still referenceing to it.
I want to log into a remote computer using the python library paramiko,
then start a daemon process using the python-daemon library which, after
the programm terminates, is still working as some kind of job queue.
This is my code so far:
(in this example the daemon will just open a file and print some random numbers into it)
#client.py
import paramiko
def main():
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('machine1', username='user1')
command = 'python server_daemon.py'
stdin,stdout,stderr = ssh.exec_command(command)
ssh.close()
if __name__=="__main__":
main()
'
#server_daemon.py
import time
import daemon
def main():
with daemon.DaemonContext():
s = [str(x)+"\n" for x in range(1000)]
for i in s:
with open("test.txt", "a") as f:
f.write(i)
time.sleep(0.4)
while True:
pass
if __name__=="__main__":
main()
Unfortunately this doesn't seem to do the thing,
if I remove the daemonizing context from the script it seems to work but I have to wait for the server to finish.
I also tried to redirect the output to /dev/null and this didn't work,
thanks for any suggestions.