I have a Python program (below) and when I run it I get the following error:
% python SSH_Prog.py
About to connect...
stderr: ["bash: -c: line 0: unexpected EOF while looking for matching `''\n", 'bash: -c: line 1: syntax error: unexpected end of file\n']
pwd: []
stderr: ['watch: no process found\n']
pwd: []
^CTraceback (most recent call last):
File "SSH_Prog.py", line 32, in <module>
time.sleep(3)
KeyboardInterrupt
I think it is to do with escape sequence probably, and the "\n" character from stdin, but I lack the experience to deal with it.
Here's the program:
import os
import sys
import time
import paramiko
#from ssh import SSHClient
# Define remote machine
host="<ip>"
user="<usrnm>"
passw="<passw>"
client = paramiko.SSHClient()
#client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# Try SSH connection, catch exception
#if not
print('About to connect...')
client.connect(host, username=user, password=passw)
# ForLoop to iterate through the interactions
for x in range(10):
xx = str(x)
# Commands to execute on local machine
f = os.popen3('tshark -i eth0 -f snmp -F pcapng -w ~/Desktop/traf_logs/n'+(xx))
# commands to execute on remote machine
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
g = os.popen3('snmpget -v 2c -c communitystring <ip> sysContact.0')
time.sleep(3)
stdin, stdout, stderr = client.exec_command('killall watch;')
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
ff = os.popen3('killall tshark')
# terminate connection
client.close()
exit(0)
Do you have any idea to fix it?
Regards.
Your first exec_command looks like this:
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
In other words, the first argument is:
"watch -n 0.1 'ps -p $(pgrep -d"
And your second argument is:
" -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";"
If you fire up bash in a terminal and type that first argument (without the quotes), followed by a newline and a ^D, it'll tell you this:
> -bash: unexpected EOF while looking for matching `''
-bash: syntax error: unexpected end of file
Which is exactly what you're getting back from Paramiko.
And the second error is just killall telling you that there is no process named watch, because your first command never started one.
If you just replace the "," with a space, that'll solve that problem… but without knowing why you thought you wanted a "," there, I'm not sure it'll do what you actually were intending to do.
I'm also not sure what the '\'' is supposed to do. Why do you want to triple-quote the arguments to awk, or why you're doing something so complicated when it ought to be equivalent to just { print $1 }, or why you're explicitly asking ps for multiple columns just to use awk to pick out the first one, or…
Related
I'm trying to setup a system to run some commands on VM's in google cloud, in my case we want to run a tcpdump at a certain time using the 'at' command. Right now I'm just trying to execute any commands succesfully, when I have to pass arguments along with the command and getting confusing behaviour, which appears to be that the command, and the arguments are executed as a single long command instead of seperate arguements.
I first tried in bash, and thinking my issue was one of quoting, I moved to using python to hopefully make things easier to understand, but I appear to be hitting the same issue and figure I must be doing something wrong.
I have the following functions defined in python, and call them
def execute(cmd):
popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
for stdout_line in iter(popen.stdout.readline, ""):
yield stdout_line
popen.stdout.close()
return_code = popen.wait()
if return_code:
raise subprocess.CalledProcessError(return_code, cmd)
def runCapture(project, instance, zone, time, duration):
## Run capture against server
print ("Running capture against Project: " + project + ", Instance: " + instance + ", Zone: " + zone, "at: " + time, "for " + str(duration) + " minutes")
## First connect, schedule capture
## Connect again, schedule upload of capture at capture time + duration time + some overrun.
## gcloud compute ssh --project=${PROJECT} ${INSTANCE} --zone="${ZONE}" --command="...do stuff..." --tunnel-through-iap
## CMD=\${1:-"/usr/sbin/tcpdump -nn -i ens4 -G \$(( ${DURATION}*60 )) -W 1 -w ./\$(uname -n)-%Y-%m-%d_%H.%M.%S.pcap"}
total_time=str(duration*60)
command="/bin/bash -c 'echo \"hello world\"'"
for path in execute(["/usr/bin/gcloud", "compute", "ssh", instance, "--project="+project, "--zone="+zone, "--tunnel-through-iap", "--command=\""+command+"\"", ]):
print(path, end="")
The resulting errors are as follows:
bash: /bin/bash -c 'echo hello: No such file or directory
Traceback (most recent call last):
File "./ingressCapture.py", line 79, in <module>
results = runCapture(project, instance, zone, time, duration)
File "./ingressCapture.py", line 33, in runCapture
for path in execute(["/usr/bin/gcloud", "compute", "ssh", instance, "--project="+project, "--zone="+zone, "--tunnel-through-iap", "--command=\""+command+"\"", ]):
File "./ingressCapture.py", line 17, in execute
raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcloud', 'compute', 'ssh', 'tbtst-test3-app-egress-nztw', '--project=devops-tb-sandbox-250222', '--zone=europe-west1-b', '--tunnel-through-iap', '--command="/bin/bash -c \'echo "hello world"\'"']' returned non-zero exit status 127.
It appears to me, that instead of invoking the bash shell and running the echo command, it is instead invoking a command that includes the bash shell and then all the arguments too. I have a bash shell when I login normally via SSH, and can run the commands manually (and they work). Why are the arguments for the command from --command="....." getting called like this and how do I prevent it?
I'm pretty sure your problem is that you have too many quotes.
When you write --command="bash -c 'echo \"Hello World\"'" on the command line, the shell internally marks all the stuff inside the quotes as being in a quoted state and then removes the quotes. The actual argument that ends up going to the program is --command=bash -c 'echo "Hello World"' as a single string in argv (or your language's equivalent).
Try putting import sys ; print(sys.argv[1]) inside a small python script and calling it with ./test.py --command="bash -c 'echo \"Hello World\"'" to see for yourself.
However, in your arglist to subprocess, you're forming this string: --command="/bin/bash -c 'echo "hello world"'", presumably because you thought you needed to match what you'd normally type on the command line. You can see this in the stacktrace (minus the escaped single quotes, since that's syntax highlighting from python). Since python does not perform quote removal, those quotes are going through to the other side of your ssh connection where the login shell is attempting to reparse it as a shell command. The first "word" on the other end of the connection is /bin/bash -c 'echo hello because of those extra quotes so the shell attempts to find a command with that name on the path, and it clearly doesn't exist.
What you need to put into your arglist for subprocess is simply "--command="+command.
I am trying to run a command line argument through python script. Script triggers the .exe but it throws an error as System.IO.IOException: The handle is invalid..
Following is my code :
import os , sys , os.path
from subprocess import call
import subprocess, shlex
def execute(cmd):
"""
Purpose : To execute a command and return exit status
"""
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(result, error) = process.communicate()
rc = process.wait()
if rc != 0:
print "Error: failed to execute command:",cmd
print error
return result
found_alf = r"C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf"
filter_alvf = r"C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
command = str(r'ALVConsole.exe -e -t -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
print command
os.chdir('C:\Program Files\Anite\LogViewer\ALV2')
print os.getcwd()
print "This process detail: \n", execute(command)
Output is as follows :
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
C:\Program Files\Anite\LogViewer\ALV2
This process detail:
Error: failed to execute command: ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
Unhandled Exception: System.IO.IOException: The handle is invalid.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
at ALV.Console.CommandLineParametersHandler.ConsoleWriteLine(String message, Boolean isError)
at ALV.Console.CommandLineParametersHandler.InvokeActions()
at ALV.Console.Program.Main(String[] args)
When I copy the command line argument from the above output and run manually from cmd it works fine.
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
I am using Windows 7 and Python 2.7.13 for. Please suggest overcoming this issue.
EDIT:
I have also tried to pass command as a list s as per below code but the issue remains the same.
command = str(r'ALVConsole.exe -e --csv -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
s=shlex.split(command)
print s
print "This process detail: \n", execute(s)
Based on your error messages I think that this problem is with ALVConsole.exe, not your Python script.
When you redirect the output, ALVConsole.exe tries to do something to the console (like setting cursor position, or getting the size of the terminal) but fails like this.
Is there a flag to ALVConsole.exe that modifies the output to a machine-readable version? I wasn't able to find the documentation for this program.
I've been working on this for a long time, and any help would be appreciated.
What I am trying to do here is ssh to a testing server, then cd .., and then print a list of the directories in that folder through python. This code is my best attempt:
def subprocess_cmd(command):
process = subprocess.Popen(command,stdout=subprocess.PIPE, shell=True)
print "Test 1"
proc_stdout = process.communicate()[0].strip()
#proc_stdout= process.stdout.readlines() (Gives same outcome as communicate)
#proc_stdout= process.stdout.read() (Gives same outcome as communicate)
print "Test 2"
print proc_stdout
subprocess_cmd('ssh user#server -p 111;cd ..;ls')
For some reason this function always hangs at the "proc_stdout= "step. It never prints "Test 2" or returns a list of files. It works fine if I take out the ssh command though. What I expect to see in the terminal is something like this, but instead the terminal hangs, and I can't interact with it anymore:
dredbounds-computer: python file_name.py
Test 1
Test 2
FileA
FileB
FileC
Update:
I modified the code and and put proc_stdout= process.stderr. communicate().
Here is my updated code:
def subprocess_cmd(command):
process = subprocess.Popen(command,stdout=subprocess.PIPE, shell=True)
print "Test 1"
proc_stderr= process.stderr. communicate()
print "Test 2"
print proc_stderr
print "Test 3"
Running this I am getting the following error in the terminal:
dredbounds-computer: python terminal_test.py
Test 1
Traceback (most recent call last):
File "file_name.py", line 26, in <module>
subprocess_cmd('ssh user#server -p 111;cd ..;ls')
File "terminal_test.py", line 21, in subprocess_cmd
proc_stdout= process.stderr. communicate()
AttributeError: 'NoneType' object has no attribute 'communicate'
Does anyone know how I can fix this code, or another way of doing the same thing. Not sure why this is giving me a none type error. Is there something wrong with how I call my ssh command? I've entered the same commands manually in the terminal and it returns a list of directories, so it should work in theory. Any advice?
If you want just to list directory contents, you can send some command over SSH.
Bash:
ssh 192.168.122.24 ls /tmp
or if you want to use "cd" as in your question:
ssh 192.168.122.24 "cd /tmp; ls"
Python script example:
import subprocess
HOST = 'server'
PORT = '111'
USER = 'user'
CMD = 'cd /tmp; ls'
process = subprocess.Popen(['ssh', '{}#{}'.format(USER, HOST),
'-p', PORT, CMD],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
result = process.stdout.readlines()
if not result:
err = process.stderr.readlines()
print('ERROR: {}'.format(err))
else:
print(result)
sCMD = 'ssh user#host cat some/path | grep "Oct 31\|Oct 30"'
child = pexpect.spawn(sCMD)
try:
child.expect("assword")
except pexpect.EOF:
raise Exception("Cannot connect to host")
child.sendline(ssh_pass)
lData = [s.strip(' \n\r:') for s in child.readlines()]
lData[0] reads "No such file or directory"
if I change the first line to read:
sCMD = 'ssh user#host cat some/path'
Then lData contains all the lines from the file.
But if I execute the exact same command (with the grep) in a terminal it works fine - it returns the text I expect. No complaints.
Any idea why the command performs differently when executed via pexpect?
sCMD = 'ssh user#host \'cat some/path | grep "Oct 31\|Oct 30"\''
for some reason pexpect requires those extra single quotes.
I'm using subprocess.popen with shlex to call a remote bash script using ssh. This command works quite fine on bash itself. But as soon as I try to translate it to python and shlex with subprocess.popen it errs out.
Remote bash script:
#!/bin/bash
tmp="";
while read -r line;
do
tmp="$tmp $line\n";
done;
echo $tmp;
BASH CMD RESULT(Invoking the remote bash script on the command line)
$> ssh x.x.x.x cat < /tmp/bef69a1d-e580-5780-8963-6a9b950e529f.txt " | /path/to/bash/script.sh;"
Bar\n
$>
Python code
import shlex
import subprocess
fn = '/tmp/bef69a1d-e580-5780-8963-6a9b950e529f.txt'
s = """
ssh x.x.x.x cat < {localfile} '| /path/to/bash/script.sh;'
""".format(localfile=fn)
print s
lexer = shlex.shlex(s)
lexer.quotes = "'"
lexer.whitespace_split = True
sbash = list(lexer)
print sbash
# print buildCmd
proc=subprocess.Popen(sbash,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out,err=proc.communicate()
print "Out: " + out
print "Err: " + err
PYTHON SCRIPT RESULT:
$> python rt.py
ssh x.x.x.x cat < /tmp/bef69a1d-e580-5780-8963-6a9b950e529f.txt '| /path/to/bash/script.sh'
['ssh', 'x.x.x.x', 'cat', '<', '/tmp/bef69a1d-e580-5780-8963-6a9b950e529f.txt', "'| /path/to/bash/script.sh'"]
Out:
Err: bash: /tmp/bef69a1d-e580-5780-8963-6a9b950e529f.txt: No such file or directory
$>
What am I missing?
The problem is that you're using shell redirection in the command, but there's no shell spawned when using subprocess.
Consider the following (very simple) program:
import sys
print sys.argv
Now if we run it like you're running ssh (assuming foofile.txt exists), we get:
python argcheck.py ssh cat < foofile.txt " | /path/to/bash/script.sh;"
['argcheck.py', 'ssh', 'cat', ' | /path/to/bash/script.sh;']
Notice that < foofile.txt never make it to python's commandline arguments. That's because the bash parser intercepts the < and the file that comes after it and redirects the contents of that file to your program's stdin. In other words, ssh is reading the file from stdin. You want your file to be passed to stdin of ssh using python as well.
s = """
ssh x.x.x.x cat '| /path/to/bash/script.sh;'
"""
#<snip>
proc=subprocess.Popen(sbash,stdout=subprocess.PIPE,stderr=subprocess.PIPE,
stdin=subprocess.PIPE)
out,err=proc.communicate(open(fn).read())
will work presumably.
The following works for me:
import subprocess
from subprocess import PIPE
with open('foo.h') as f:
p = subprocess.Popen(['ssh','mgilson#XXXXX','cat','| cat'],stdin=f,stdout=PIPE,stderr=PIPE)
out,err = p.communicate()
print out
print '#'*80
print err
And the equivalent command in bash:
ssh mgilson#XXXXX cat < foo.h '| cat'
where foo.h is a file on my local machine.