I've been using my script for a unix server and it's working perfectly. However when i use the same script( with some minor command changes) to connect to HP Procurve switches , script crashes with error. Part of the script is below:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(address, username=userna, password=passwd)
stdin,stdout,stderr= ssh.exec_command("show ver")
for line in stdout:
print '... ' + line.strip('\n')
ssh.close()
This gives error
Traceback (most recent call last):
File "C:/Users/kucar/Desktop/my_python/switchmodel", line 34, in <module>
stdin,stdout,stderr= ssh.exec_command("show ver")
File "C:\Python27\lib\site-packages\paramiko\client.py", line 379, in exec_command
chan.exec_command(command)
File "C:\Python27\lib\site-packages\paramiko\channel.py", line 218, in exec_command
self._wait_for_event()
File "C:\Python27\lib\site-packages\paramiko\channel.py", line 1122, in _wait_for_event
raise e
SSHException: Channel closed.
I've found similar complaints in the web however seems like solution is not provided at all. Switch is open to ssh and works fine with putty. Appreciate if you give any ideas that could help me. I cannot do "show ver" command manually for 100 switches.
As #dobbo mentioned above you have to do invoke_shell() on the channel so that you can execute multiple commands. Also HP ProCurve has ANSI Escape Codes in the output so you have to strip those out. Finally, HP ProCurve throws up a "Press any key to continue" message which you have to get past at least on some devices.
I have an HP ProCurve handler in this library https://github.com/ktbyers/netmiko
Set device_type to "hp_procurve".
Exscript also has some sort of a ProCurve handler though I haven't dug into it enough to get it to work.
I had the same experience connecting to my Samsung s4 phone with an ssh server.
I had no problem connecting to a SUSE VM or a Rasperry Pi and also tried MobaXterm (putty is SO last week).
I have not found the answer but will share my research.
I had a look at the source and found line 1122 in channel.py (copied below).
With my phone (and possibly your HP switch) I have noticed that there is no login message or MOTD at all and when exiting (with putty/mobaXterm) the session doesn't end properly.
In some other reading, I have found that the parameko is not getting much support from the author any more but others are working to port it to python 3x.
Here is the source code I found.
def _wait_for_send_window(self, size):
"""
(You are already holding the lock.)
Wait for the send window to open up, and allocate up to C{size} bytes
for transmission. If no space opens up before the timeout, a timeout
exception is raised. Returns the number of bytes available to send
(may be less than requested).
"""
# you are already holding the lock
if self.closed or self.eof_sent:
return 0
if self.out_window_size == 0:
# should we block?
if self.timeout == 0.0:
raise socket.timeout()
# loop here in case we get woken up but a different thread has filled the buffer
timeout = self.timeout
while self.out_window_size == 0:
if self.closed or self.eof_sent:
return 0
then = time.time()
self.out_buffer_cv.wait(timeout)
if timeout != None:
timeout -= time.time() - then
if timeout <= 0.0:
raise socket.timeout()
# we have some window to squeeze into
It seems that if you don't clean up the connection buffer Paramiko goes nuts when working with HP Procurves. First off you need to invoke a shell or Paramiko will simply drop the connection after the first command (normal behavior, but confusing).
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(switch_ip, username=switch_user, password=switch_pass,look_for_keys=False)
conn = ssh.invoke_shell()
recieveData() # <-- see below
It's important to actually handle the data, and as I've learned you need to make sure Paramiko has actually received all the data before you ask it to do stuff with it. I do this by using the following function. You can adjust the sleep as needed, in some cases 0.050 will work fine.
def recieveData():
tCheck = 0
while not conn.recv_ready():
time.sleep(1)
tCheck+=1
if tCheck >=10:
print "time out"
cleanThatStuffUp(conn.recv(1024)) # <-- see below
This is an example of the garbage that is returning to your ssh client.
[1;24r[24;1H[24;1H[2K[24;1H[?25h[24;1H[24;1HProCurve Switch 2650# [24;1H[24;23H[24;1H[?5h[24;23H[24;23Hconfigure[24;23H[?25h[24;32H[24;0HE[24;1H[24;32H[24;1H[2K[24;1H[?5h[24;1H[1;24r[24;1H[1;24r[24;1H[24;1H[2K[24;1H[?25h[24;1H[24;1H
There's also exit codes to deal with before each "[". So to deal with that I figured out some regex to clean all of that "stuff" up.
procurve_re1 = re.compile(r'(\[\d+[HKJ])|(\[\?\d+[hl])|(\[\d+)|(\;\d+\w?)')
procurve_re2 = re.compile(r'([E]\b)')
procurve_re3 = re.compile(ur'[\u001B]+') #remove stupid escapes
def cleanThatStuffUp(message):
message = procurve_re1.sub("", message)
message = procurve_re2.sub("", message)
message = procurve_re3.sub("", message)
print message
Now you can go about entering commands, just make sure you clear out the buffer each time using recieveData().
conn.send("\n") # Get past "Press any key"
recieveData()
Related
I'm a software tester, trying to verify that the log on a remote QNX (a BSD variant) machine will contain the correct entries after specific actions are taken. I am able to list the contents of the directory in which the log resides, and use that information in the command to read (really want to use tail -n XX <file>) the file. So far, I always get a "(No such file or directory)" when trying to read the file.
We are using Froglogic Squish for automated testing, because the Windows UI (that interacts with the server piece on QNX) is built using Qt extensions for standard Windows elements. Squish uses Python 2.7, so I am using Python 2.7.
I am using paramiko for the SSH connection to the QNX server. This has worked great for sending commands to the simulator piece that also runs on the QNX server.
So, here's the code. Some descriptive names have been changed to avoid upsetting my employer.
import sys
import time
import select
sys.path.append(r"C:\Python27\Lib\site-packages")
sys.path.append(r"C:\Python27\Lib\site-packages\pip\_vendor")
import paramiko
# Import SSH configuration variables
ssh_host = 'vvv.xxx.yyy.zzz'
thelog_dir = "/logs/the/"
ssh_user = 'un'
ssh_pw = 'pw'
def execute_Command(fullCmd):
outptLines = []
#
# Try to connect to the host.
# Retry a few times if it fails.
#
i = 1
while True:
try:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(ssh_host, 22, ssh_user, ssh_pw)
break
except paramiko.AuthenticationException:
log ("Authentication failed when connecting to %s" % ssh_host)
return 1
except:
log ("Could not SSH to %s, waiting for it to start" % ssh_host)
i += 1
time.sleep(2)
# If we could not connect within time limit
if i == 30:
log ("Could not connect to %s. Giving up" % ssh_host)
return 1
# Send the command (non-blocking?)
stdin, stdout, stderr = ssh.exec_command(fullCmd, get_pty=True)
for line in iter(stdout.readline, ""):
outptLines.append(line)
#
# Disconnect from the host
#
ssh.close()
return outptLines
def get_Latest_Log():
fullCmd = "ls -1 %s | grep the_2" %thelog_dir
files = execute_Command(fullCmd)
theFile = files[-1]
return theFile
def main():
numLines = 20
theLog = get_Latest_Log()
print("\n\nThe latest log is %s\n\n" %theLog)
fullCmd = "cd /logs/the; tail -n 20 /logs/the/%s" %theLog
#fullCmd = "tail -n 20 /logs/the/%s" %theLog
print fullCmd
logLines = execute_Command(fullCmd)
for line in logLines:
print line
if __name__ == "__main__":
# execute only if run as a script
main()
I have tried to read the file using both tail and cat. I have also tried to get and open the file using Paramiko's SFTP client.
In all cases, the response of trying to read the file fails -- despite the fact that listing the contents of the directory works fine. (?!) And BTW, the log file is supposed to be readable by 'world'. Permissions are -rw-rw-r--.
The output I get is:
"C:\Users\xsat086\Documents\paramikoTest>python SSH_THE_MsgChk.py
The latest log is the_20210628_115455_205.log
cd /logs/the; tail -n 20 /logs/the/the_20210628_115455_205.log
(No such file or directory)the/the_20210628_115455_205.log"
The file name is correct. If I copy and paste the tail command into an interactive SSH session with the QNX server, it works fine.
Is it something to do with the 'non-interactive' nature of this method of sending commands? I read that some implementations of SSH are built upon a command that offers a very limited environment. I don't see how that would impact this tail command.
Or am I doing something stupid in this code?
I cannot really explain completely, why you get the results you get.
But in general a corrupted output is a result of enabling and not handling terminal emulation. You enable the terminal emulation using get_pty=True. Remove it. You should not use the terminal emulation, when automating command execution.
Related question:
Is there a simple way to get rid of junk values that come when you SSH using Python's Paramiko library and fetch output from CLI of a remote machine?
I have a follow up question that builds off the question I asked here: Run multiple commands in different SSH servers in parallel using Python Paramiko, which was already answered.
Thanks to the answer on the link above, my python script is as follows:
# SSH.py
import paramiko
import argparse
import os
path = "path"
python_script = "worker.py"
# definitions for ssh connection and cluster
ip_list = ['XXX.XXX.XXX.XXX', 'XXX.XXX.XXX.XXX', 'XXX.XXX.XXX.XXX']
port_list = [':XXXX', ':XXXX', ':XXXX']
user_list = ['user', 'user', 'user']
password_list = ['pass', 'pass', 'pass']
node_list = list(map(lambda x: f'-node{x + 1} ', list(range(len(ip_list)))))
cluster = ' '.join([node + ip + port for node, ip, port in zip(node_list, ip_list, port_list)])
# run script on command line of local machine
os.system(f"cd {path} && python {python_script} {cluster} -type worker -index 0 -batch 64 > {path}/logs/'command output'/{ip_list[0]}.log 2>&1")
# loop for IP and password
stdouts = []
clients = []
for i, (ip, user, password) in enumerate(zip(ip_list[1:], user_list[1:], password_list[1:]), 1):
try:
print("Open session in: " + ip + "...")
client = paramiko.SSHClient()
client.connect(ip, user, password)
except paramiko.SSHException:
print("Connection Failed")
quit()
try:
path = f"C:/Users/{user}/Desktop/temp-ines"
stdin, stdout, stderr = ssh.exec_command(
f"cd {path} && python {python_script} {cluster} -type worker -index {i} -batch 64>"
f"C:/Users/{user}/Desktop/{ip}.log 2>&1 &"
)
clients.append(ssh)
stdouts.append(stdout)
except paramiko.SSHException:
print("Cannot run file. Continue with other IPs in list...")
client.close()
continue
# Wait for commands to complete
for i in range(len(stdouts)):
print("hello")
stdouts[i].read()
print("hello1")
clients[i].close()
print('hello2")
print("\n\n***********************End execution***********************\n\n")
This script, which is run locally, is able to SSH into the servers and run the command (i.e., run a python script called worker.py and log the command output to a log file). I.e., it is able to go through the first for loop with no issues.
My issue is related to the second for loop. Please see the print statements I added in the second for loop to be clear. When I run SSH.py locally, this is what I observe:
As you can see, I ssh into each of the servers and then stay at reading the command output of the first server I ssh over to. The worker.py script can take 30 mins or so to complete and the command output is the same on each server -- so it will take 30 mins to read the command output of the first server, then close the SSH connection of the first server, take a couple seconds to read the command output of the second server (as it is the same as the first one and would already be entirely printed), close its SSH connection, and so on. Please see below some of the command line output, if this helps.
Now, my question is, what if I don't want to wait until the worker.py script finishes, i.e., those entire 30 mins? I cannot/do not know how to raise a KeyboardInterrupt exception. What I have tried is quitting the local SSH.py script. However, as you can see from the print statements, this will not close the SSH connections although the training, and thus the log files, will stop logging info. In addition, after I quit the local SSH.py script, if I try to delete any of the log files, I get an error saying "cannot delete file because it is being used in cmd.exe" -- this only happens sometimes and I believe it is because of not closing the SSH connections?
First run in python console:
It hangs: Local python and log file running and saving but no print statements and no python and log file being run/saved in servers.
I run it again so second process starts:
Now, the first process doesn't hang anymore (python running and log files being saved in server). And can close this second run/process. It is like the second run/process helps with the hang of the first run/process.
If I were to run python SSH.py in the terminal it would just hang.
This was not happening before.
If you know that SSHClient.close cleanly close the connection and abort the remote command, call it on response to KeyboardInterrupt.
For this you cannot use the simple solution with stdout.read, as it blocks and prevents handling of the Ctrl+C on Windows.
Use the waiting code from my answer to Run multiple commands in different SSH servers in parallel using Python Paramiko (the while any(x is not None for x in stdouts): snippet).
And wrap it to try:...except (KeyboardInterrupt):.
try:
while any(x is not None for x in stdouts):
for i in range(len(stdouts)):
stdout = stdouts[i]
if stdout is not None:
channel = stdout.channel
# To prevent losing output at the end, first test for exit,
# then for output
exited = channel.exit_status_ready()
while channel.recv_ready():
s = channel.recv(1024).decode('utf8')
print(f"#{i} stdout: {s}")
while channel.recv_stderr_ready():
s = channel.recv_stderr(1024).decode('utf8')
print(f"#{i} stderr: {s}")
if exited:
print(f"#{i} done")
clients[i].close()
stdouts[i] = None
time.sleep(0.1)
except (KeyboardInterrupt):
print("Aborting")
for i in range(len(clients)):
print(f"#{i} closing")
clients[i].close()
If you do not need to separate the stdout and stderr, you can greatly simplify the code by using Channel.set_combine_stderr. See Paramiko ssh die/hang with big output.
I'm currently working a server-client setup in which I have two separate server scripts. One python script is responsible for running a SSH listener with Paramiko, and that script runs on one machine. I have another server script specifically acting as an SFTP server on another, separate machine, within the same range and subnet as the other one.
My client code is running on a windows 10 system. Both servers are running in unix environments (macOS and Ubuntu 16.04 respectively).
The SFTP server that I am running is aptly titled sftpserver, and is available at https://github.com/rspivak/sftpserver/.
The below code is actually the entirety of my client.py as it stands, minus the import statements.
key = paramiko.RSAKey.from_private_key_file('testkey.key')
transport = paramiko.Transport(('192.168.1.116', 10000))
transport.connect(username='root', password='toor', pkey=key)
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('192.168.1.107', username='root', password='toor')
chan = client.get_transport().open_session()
chan.send("Hey man! I'm connected!")
print(chan.recv(1024))
def sftp(localpath, name):
try:
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(localpath, '/root/uploads/' + name)
sftp.close()
transport.close()
return "<+> Done uploading"
except Exception as e:
return str(e)
while True:
command = chan.recv(1024).decode()
ipdb.set_trace() // <-- debugging purposes only
if 'grab' in command:
_, path, name = command.split(' ')
chan.send(sftp(path, name))
else:
try:
CMD = subprocess.check_output(command, shell=True)
chan.send(CMD)
except Exception as e:
chan.send(str(e))
client.close()
Executing the grab command in my script looks like this:
grab C:\Users\xxx\testing.txt testing.txt
Now, if I write a path exactly like that (with the back slashes), it will append a second back slash after each one. So, the path I supplied now looks like C:\\Users\xxx\\testing.txt, and this is what I imagine is causing me to receive File not found errors. Thanks to pdb I was able to find this issue, but I am unsure how to continue. In all honesty, I am completely unsure if this problem is paramiko related or if it's some weird python behavior that I haven't encountered yet.
Also, sorry for no stack trace. I'll try to obtain one if possible, but I'm a bit pressed for time right this second.
Thinking about making a move from Perl to Python running scripts to automate certain tasks on remote servers and devices. We need to be able to use Expect to check for certain results and get the data back. Taking a look at Paramiko-Expect and I like it, but it's timing out every time.
import paramiko
from paramikoe import SSHClientInteraction
HOSTNAME = "HOST IP"
PASSWORD = "PWORD"
USERNAME = "UNAME"
PROMPT = "(node name)#"
command = "show command"
print PROMPT
file = open("testlog.txt","w")
def main():
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(HOSTNAME, port=22, username=USERNAME, password=PASSWORD)
interact = SSHClientInteraction(client, timeout=10, display=True)
interact.send(command)
interact.expect(PROMPT)
file.write(interact.current_output_clean)
client.close()
return
main()
file.close()
This is the traceback I get:
Traceback (most recent call last):
File "python_test.py", line 40, in <module>
main()
File "python_test.py", line 28, in main
interact.expect(PROMPT)
File "/usr/local/lib/python2.7/site-packages/paramikoe.py", line 122, in expect
buffer = self.channel.recv(self.buffer_size)
File "/usr/local/lib/python2.7/site-packages/paramiko/channel.py", line 598, in recv
raise socket.timeout() socket.timeout
I've tried multiple versions of the PROMPT to expect, from directly putting in the text of the node I'm trying it on to full regex. Nothing works. It always times out when it gets to the client.expect. Paramiko-expect documentation does not help and the only other place I see this question is different enough that it doesn't help.
Any advice is appreciated.
Put prompt to something you expect, as... prompt. Here is paramiko interaction example. Please note lines 21, and 37 -
PROMPT = 'vagrant#paramiko-expect-dev:~\$\s+'
interact.expect(PROMPT)
So, when I've updated part of your code to:
interact = SSHClientInteraction(client, timeout=10, display=True)
interact.expect(PROMPT)
interact.send("ls")
interact.expect(".*Maildir.*")
file.write(interact.current_output_clean)
client.close()
I have testlog.txt filled with listing of the home directory, of remote host.
As a side note - switch to python 3. If you are starting, it is better to use tool that is not well known to be outdated soon. Also you can use ipython, or jupyter - code will be more interactive, faster to test. Maybe netmiko will be interested for you?
I have a class that creates the connection. I can connect and execute 1 command before the channel is closed. On another system i have i can execute multiple commands and the channel does not close. Obviously its a config issue with the systems i am trying to connect to.
class connect:
newconnection = ''
def __init__(self,username,password):
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect('somehost', username=username,password=password,port=2222,timeout=5)
except:
print "Count not connect"
sys.exit()
self.newconnection = ssh
def con(self):
return self.newconnection
Then i use 'ls' command just to print some output
sshconnection = connect('someuser','somepassword').con()
stdin, stdout, stderr = sshconnection.exec_command("ls -lsa")
print stdout.readlines()
print stdout
stdin, stdout, stderr = sshconnection.exec_command("ls -lsa")
print stdout.readlines()
print stdout
sshconnection.close()
sys.exit()
After the first exec_command runs it prints the expected output of the dir list. When i print stdout after the first exec_command it looks like the channel is closed
<paramiko.ChannelFile from <paramiko.Channel 1 (closed) -> <paramiko.Transport at 0x2400f10L (cipher aes128-ctr, 128 bits) (active; 0 open channel(s))>>>
Like i said on another system i am able to keep running commands and the connection doesn't close. Is there a way i can keep this open? or a better way i can see the reason why it closes?
edit: So it looks like you can only run 1 command per SSHClient.exec_command... so i decided to get_transport().open_session() and then run a command. The first one always works. The second one always fails and the scripts just hangs
With just paramiko after the exec_command executes the channel is closed and the ssh returns an auth prompt.
Seems its not possible with just paramiko, try fabric or another tool.
** fabric did not work out too.
Please see the following referece as it provides a way to do this in Paramiko:
How do you execute multiple commands in a single session in Paramiko? (Python)
it's possible with netmiko (tested on windows).
this example is written for connecting to cisco devices but the principle is adaptable for others as well.
import netmiko
from netmiko import ConnectHandler
import json
def connect_enable_silent(ip_address,ios_command):
with open ("credentials.txt") as line:
line_1 = json.load(line)
for k,v in line_1.items():
router=(k,v)
try:
ssh = ConnectHandler(**router[1],device_type="cisco_ios",ip=ip_address)
ssh.enable()
except netmiko.ssh_exception.NetMikoAuthenticationException:
#incorrect credentials
continue
except netmiko.ssh_exception.NetMikoTimeoutException:
#oddly enough if it can log in but not able to authenticate to enable mode the ssh.enable() command does not give an authentication error
#but a time-out error instead
try:
ssh = ConnectHandler(username = router[1]['username'],password = router[1]['password'],device_type="cisco_ios", ip=ip_address)
except netmiko.ssh_exception.NetMikoTimeoutException:
# connection timed out (ssh not enabled on device, try telnet)
continue
except Exception:
continue
else:
output = ssh.send_command(ios_command)
ssh.disconnect()
if "at '^' marker." in output:
#trying to run a command that requires enble mode but not authenticated to enable mode
continue
return output
except Exception:
continue
else:
output = ssh.send_command(ios_command)
ssh.disconnect()
return output
output = connect_enable_silent(ip_address,ios_command)
for line in output.split('\n'):
print(line)
Credentials text is meant to store different credentials in case you are planning to call this function to access multiple devices and not all of them using the same credentials. It is in the format:
{"credentials_1":{"username":"username_1","password":"password_1","secret":"secret_1"},
"credentials_2":{"username":"username_2","password":"password_2","secret":"secret_2"},
"credentials_3": {"username": "username_3", "password": "password_3"}
}
The exceptions can be changed to do different things, in my case i just needed it to not return an error and continue trying the next set, which is why most exceptions are silenced.