Develop Reference

Passing variables to a script over ssh using gcloud command -- all variables treated as a single string?

I'm trying to setup a system to run some commands on VM's in google cloud, in my case we want to run a tcpdump at a certain time using the 'at' command. Right now I'm just trying to execute any commands succesfully, when I have to pass arguments along with the command and getting confusing behaviour, which appears to be that the command, and the arguments are executed as a single long command instead of seperate arguements.
I first tried in bash, and thinking my issue was one of quoting, I moved to using python to hopefully make things easier to understand, but I appear to be hitting the same issue and figure I must be doing something wrong.
I have the following functions defined in python, and call them
def execute(cmd):
popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
for stdout_line in iter(popen.stdout.readline, ""):
yield stdout_line
popen.stdout.close()
return_code = popen.wait()
if return_code:
raise subprocess.CalledProcessError(return_code, cmd)
def runCapture(project, instance, zone, time, duration):
## Run capture against server
print ("Running capture against Project: " + project + ", Instance: " + instance + ", Zone: " + zone, "at: " + time, "for " + str(duration) + " minutes")
## First connect, schedule capture
## Connect again, schedule upload of capture at capture time + duration time + some overrun.
## gcloud compute ssh --project=${PROJECT} ${INSTANCE} --zone="${ZONE}" --command="...do stuff..." --tunnel-through-iap
## CMD=\${1:-"/usr/sbin/tcpdump -nn -i ens4 -G \$(( ${DURATION}*60 )) -W 1 -w ./\$(uname -n)-%Y-%m-%d_%H.%M.%S.pcap"}
total_time=str(duration*60)
command="/bin/bash -c 'echo \"hello world\"'"
for path in execute(["/usr/bin/gcloud", "compute", "ssh", instance, "--project="+project, "--zone="+zone, "--tunnel-through-iap", "--command=\""+command+"\"", ]):
print(path, end="")
The resulting errors are as follows:
bash: /bin/bash -c 'echo hello: No such file or directory
Traceback (most recent call last):
File "./ingressCapture.py", line 79, in <module>
results = runCapture(project, instance, zone, time, duration)
File "./ingressCapture.py", line 33, in runCapture
for path in execute(["/usr/bin/gcloud", "compute", "ssh", instance, "--project="+project, "--zone="+zone, "--tunnel-through-iap", "--command=\""+command+"\"", ]):
File "./ingressCapture.py", line 17, in execute
raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcloud', 'compute', 'ssh', 'tbtst-test3-app-egress-nztw', '--project=devops-tb-sandbox-250222', '--zone=europe-west1-b', '--tunnel-through-iap', '--command="/bin/bash -c \'echo "hello world"\'"']' returned non-zero exit status 127.
It appears to me, that instead of invoking the bash shell and running the echo command, it is instead invoking a command that includes the bash shell and then all the arguments too. I have a bash shell when I login normally via SSH, and can run the commands manually (and they work). Why are the arguments for the command from --command="....." getting called like this and how do I prevent it?

I'm pretty sure your problem is that you have too many quotes.
When you write --command="bash -c 'echo \"Hello World\"'" on the command line, the shell internally marks all the stuff inside the quotes as being in a quoted state and then removes the quotes. The actual argument that ends up going to the program is --command=bash -c 'echo "Hello World"' as a single string in argv (or your language's equivalent).
Try putting import sys ; print(sys.argv[1]) inside a small python script and calling it with ./test.py --command="bash -c 'echo \"Hello World\"'" to see for yourself.
However, in your arglist to subprocess, you're forming this string: --command="/bin/bash -c 'echo "hello world"'", presumably because you thought you needed to match what you'd normally type on the command line. You can see this in the stacktrace (minus the escaped single quotes, since that's syntax highlighting from python). Since python does not perform quote removal, those quotes are going through to the other side of your ssh connection where the login shell is attempting to reparse it as a shell command. The first "word" on the other end of the connection is /bin/bash -c 'echo hello because of those extra quotes so the shell attempts to find a command with that name on the path, and it clearly doesn't exist.
What you need to put into your arglist for subprocess is simply "--command="+command.

How to execute a python code from coffee script and get the output

i am developing a coffeescript which needs to execute a python program and get the responses from the python code.
my coffee script is
module.exports = (robot) ->
robot.respond /greetme/i, (msg) ->
sender = msg.message.user.name.toLowerCase()
#exec = require('child_process').exec
command = "python3 ext-scripts/hello.py"
#exec command, (out1) ->
msg.send out1
msg.send "Hello " + sender
msg.finish()
and my python code is hello.py
print("hey indhu")
return "reached the python file"
i need to get the output "reached the python file" to 6th line in coffee script. to send out the message
i am getting error while doing this exec out function.
message: Error: Command failed: python3 ext-scripts/hello.py File "ext-scripts/hello.py", line 28
return "reached the python file"
^ SyntaxError: 'return' outside function
error: Response not OK: no_text
How to make it work. please help me . i am a python developer and new to coffeescript.

The easiest way is with a supprecess that reads the output and filters. The first version is good for Windows machines, the second for Linux systems.
import subprocess
process = subprocess.Popen(['echo', 'More output'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
stdout, stderr
OR
import os
cmd = 'your command here'
terminal_output = os.system(cmd)
print(terminal_output)

How to get return value of a executed command in Python

I am trying to get the output of a shell command I try to execute using python but I get an error.
How can I get the response/return value from executing a bash command
This is what I have done:
import subprocess
import time
# NAMESPACE = input("Namespace: ")
# # Create a namespace
# subprocess.call(["kubectl", "create", "namespace", NAMESPACE])
# build a docker image to deploy the application
DOCKER_OUTPUT = subprocess.call(["docker", "build", "-t", "banuka/node-web-app", "."])
print("Docker output is: " + DOCKER_OUTPUT)
Somehow this gives an error:
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/jananath/Desktop/python-script/Dockerfile: no such file or directory
Traceback (most recent call last):
File "/home/jananath/Desktop/python-script/bitesize-platform-troubleshooter/test/test.py", line 11, in
print("Docker output is: " + DOCKER_OUTPUT)
TypeError: can only concatenate str (not "int") to str
Can someone please help me to print the response without getting this error (from python)?

The result of system commands is usually an integer associated with an exit status. You can do print("Docker output is: " + str(DOCKER_OUTPUT)") To convert the int to a String, or you can use other Python string formatting options (depending on your Python version).
Example: f-string
print(f"Docker output is {DOCKER_OUTPUT}")
Example: .format()
print("Docker output is {}".format(DOCKER_OUTPUT))

If you want a more verbose output (i.e. not just the zero/nonzero exit status) you can do the following:
cmd = 'docker build -t banuka/node-web-app .'
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
output, err = p.communicate()
print('This is the output: {}, and this is the error: {}'.format(output, error))

You shouldn't use subprocess.call it's in the old deprecated API
if you lookup the docs you can find this example.
>>> subprocess.run(["ls", "-l"]) # doesn't capture output
CompletedProcess(args=['ls', '-l'], returncode=0)
>>> subprocess.run("exit 1", shell=True, check=True)
Traceback (most recent call last):
...
subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
>>> subprocess.run(["ls", "-l", "/dev/null"], capture_output=True)
CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n', stderr=b'')
The correct way of capturing the out put would be for example:
out=subprocess.run(["ls", "-l","/foo/bar"], capture_output=True).stdout
Make sure you are running the latest version of python you are able to and the latest set of functions because they tend to be more convenient and easier to use.

Failed to execute command line argument from python script

I am trying to run a command line argument through python script. Script triggers the .exe but it throws an error as System.IO.IOException: The handle is invalid..
Following is my code :
import os , sys , os.path
from subprocess import call
import subprocess, shlex
def execute(cmd):
"""
Purpose : To execute a command and return exit status
"""
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(result, error) = process.communicate()
rc = process.wait()
if rc != 0:
print "Error: failed to execute command:",cmd
print error
return result
found_alf = r"C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf"
filter_alvf = r"C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
command = str(r'ALVConsole.exe -e -t -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
print command
os.chdir('C:\Program Files\Anite\LogViewer\ALV2')
print os.getcwd()
print "This process detail: \n", execute(command)
Output is as follows :
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
C:\Program Files\Anite\LogViewer\ALV2
This process detail:
Error: failed to execute command: ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
Unhandled Exception: System.IO.IOException: The handle is invalid.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
at ALV.Console.CommandLineParametersHandler.ConsoleWriteLine(String message, Boolean isError)
at ALV.Console.CommandLineParametersHandler.InvokeActions()
at ALV.Console.Program.Main(String[] args)
When I copy the command line argument from the above output and run manually from cmd it works fine.
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
I am using Windows 7 and Python 2.7.13 for. Please suggest overcoming this issue.
EDIT:
I have also tried to pass command as a list s as per below code but the issue remains the same.
command = str(r'ALVConsole.exe -e --csv -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
s=shlex.split(command)
print s
print "This process detail: \n", execute(s)

Based on your error messages I think that this problem is with ALVConsole.exe, not your Python script.
When you redirect the output, ALVConsole.exe tries to do something to the console (like setting cursor position, or getting the size of the terminal) but fails like this.
Is there a flag to ALVConsole.exe that modifies the output to a machine-readable version? I wasn't able to find the documentation for this program.

Error from stderr - how to fix it?

I have a Python program (below) and when I run it I get the following error:
% python SSH_Prog.py
About to connect...
stderr: ["bash: -c: line 0: unexpected EOF while looking for matching `''\n", 'bash: -c: line 1: syntax error: unexpected end of file\n']
pwd: []
stderr: ['watch: no process found\n']
pwd: []
^CTraceback (most recent call last):
File "SSH_Prog.py", line 32, in <module>
time.sleep(3)
KeyboardInterrupt
I think it is to do with escape sequence probably, and the "\n" character from stdin, but I lack the experience to deal with it.
Here's the program:
import os
import sys
import time
import paramiko
#from ssh import SSHClient
# Define remote machine
host="<ip>"
user="<usrnm>"
passw="<passw>"
client = paramiko.SSHClient()
#client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# Try SSH connection, catch exception
#if not
print('About to connect...')
client.connect(host, username=user, password=passw)
# ForLoop to iterate through the interactions
for x in range(10):
xx = str(x)
# Commands to execute on local machine
f = os.popen3('tshark -i eth0 -f snmp -F pcapng -w ~/Desktop/traf_logs/n'+(xx))
# commands to execute on remote machine
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
g = os.popen3('snmpget -v 2c -c communitystring <ip> sysContact.0')
time.sleep(3)
stdin, stdout, stderr = client.exec_command('killall watch;')
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
ff = os.popen3('killall tshark')
# terminate connection
client.close()
exit(0)
Do you have any idea to fix it?
Regards.

Your first exec_command looks like this:
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
In other words, the first argument is:
"watch -n 0.1 'ps -p $(pgrep -d"
And your second argument is:
" -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";"
If you fire up bash in a terminal and type that first argument (without the quotes), followed by a newline and a ^D, it'll tell you this:
> -bash: unexpected EOF while looking for matching `''
-bash: syntax error: unexpected end of file
Which is exactly what you're getting back from Paramiko.
And the second error is just killall telling you that there is no process named watch, because your first command never started one.
If you just replace the "," with a space, that'll solve that problem… but without knowing why you thought you wanted a "," there, I'm not sure it'll do what you actually were intending to do.
I'm also not sure what the '\'' is supposed to do. Why do you want to triple-quote the arguments to awk, or why you're doing something so complicated when it ought to be equivalent to just { print $1 }, or why you're explicitly asking ps for multiple columns just to use awk to pick out the first one, or…