I found it here: https://netsec.ws/?p=337
echo os.system('/bin/bash')
os.system is from python but echo is not, I think... what language is this exactly?
If you looked at the echo tag, you'll see its description: "Simple function outputting text. Exists in script languages.", if you're running Linux or Mac and even Windows Command Prompt you can enter this:
$ echo Hello shell world
Output:
Hello shell world
echo is useful for testing shell glob expansion, echoing the values of environment variables and sending output to stdins of other programs, for example try this with Bash:
$ echo 'print("Spam" * 2)' | python -
Output:
SpamSpam
And this:
$ echo 'import os; os.system("echo SPAM")' | python -
Output:
SPAM
Related
I am using Click to create CLI tool.
I want to create a command that can be pipeable.
For example,
echo 'Hello world' | customcmd > hello.txt
where customcmd is a command which takes a stdin input and returns stdout.
Put a shebang at 1st line of the customcmd file:
#! /usr/bin/env python
and set the executable bit:
$ chmod a+x customcmd
Then you'll be good to go, e.g. $ echo Hello | ./customcmd
cf exec
I am running a python3 script which performs the following snippet on Debian 9:
os.environ["PA_DIR"] = "/home/myname/some_folder"
command_template = ("sudo java -Dconfig.file=$PA_DIR/google.conf "
"-jar ~/big/cromwell-42.jar run $PA_DIR/WholeGenomeGermlineSingleSample.wdl "
"-i {} -o $PA_DIR/au_options.json > FDP{}.log 2>&1")
command = command_template.format("test.json, "1")
os.system("screen -dm -S S{} bash -c '{}'".format("1", command))
The use of PA_DIR works as intended. When I tried it on command line:
PA_DIR="/home/myname/some_folder"
screen -dm -S S1 bash -c 'sudo java -Dconfig.file=$PA_DIR/google.conf -jar ~/big/cromwell-42.jar run $PA_DIR/WholeGenomeGermlineSingleSample.wdl -i test.json -o $PA_DIR/au_options.json > FDP1.log 2>&1'
it doesn't do variable substitution due to single quotes and I had to replace them with double quotes (it complains it cannot find the file /google.conf).
What is different when python runs it?
Thanks!
The Python os.system() invokes the underlying system function of the C library, which on POSIX systems is equivalent to do something like
sh -c "your_command and all its arguments"
So the command and all arguments are already surrounded by double-quotes, which does environment variable substitution. Any single quotes inside the string is irrelevant for the variable substitution.
You can test it easily. In a shell do something like
$ foo="bar"
$ echo "foo is '$foo'" # Will print foo is 'bar'
$ echo 'foo is "$foo"' # Will print foo is "$foo"
Waiting for your answer to daltonfury42, I'd bet the problem is, when running in a command line, you are not exporting the PA_DIR environment variable so it is not present in the second bash interpreter. And it behaves different beacuse of what Mihir answered.
If you run
PA_DIR=foo
you only declare a bash variable but it is not an environment variable. Then
bash -c "echo $PA_DIR"
this will output foo because your current interpreter interpolates $PA_DIR and then raises a second bash process with the command echo foo. But
bash -c 'echo $PA_DIR'
this prevents your bash interpreter from interpolating it so it raises a second bash process with the comand echo $PA_DIR. But in this second process the variable PA_DIR does not exist.
If you start your journey running
export PA_DIR=foo
this will become an environment variable that will be accessible to children processes, thus
bash -c 'echo $PA_DIR'
will output foo because the nested bash interpreter has access to the variable even if the parent bash interpreter did not interpolate it.
The same is true for any kind of children process. Try running
PA_DIR=foo
python3 -c 'import os; print(os.environ.get("PA_DIR"))'
python3 -c "import os; print(os.environ.get('PA_DIR'))"
export PA_DIR=foo
python3 -c 'import os; print(os.environ.get("PA_DIR"))'
python3 -c "import os; print(os.environ.get('PA_DIR'))"
in your shell. No quotes are involved here!
When you use the os.environ dictionary in a Python script, Python will export the variables for you. That's why you will see the variable interpolated by either
os.system("bash -c 'echo $PA_DIR'")
or
os.system('bash -c "echo $PA_DIR"')
But beware that in each case it is either the parent or either the children shell process who is interpolating the variable.
You must understand your process tree here:
/bin/bash # but it could be a zsh, fish, sh, ...
|- /usr/bin/python3 # presumably
|- /bin/sh # because os.system uses that
|- /bin/bash
If you want an environment variable to exist in the most nested process, you must export it anywhere in the upper tree. Or in that very process.
Consider this:
$ cat test.py
import sys
print "Doing something ..."
sys.exit(1)
$ python test.py
Doing something ...
$ echo $?
1
$ python test.py | tee log # used in Jenkins, but I need to capture rc of 1 as well
Doing something ...
$ echo $?
0
As you can see I am unable to capture the return code of 1 from the python script if I pipe the output to tee. Is there a way I can accomplish this ? This code is in Build->execute shell section in Jenkins.
Since I am unable to capture the rc of 1 the commands following the line continue to get executed, which I don't want to happen.
In bash you can use the PIPESTATUS array to get the exit status of each command in a pipeline:
python test.py | tee log
echo ${PIPESTATUS[0]}
Regarding "Since I am unable to capture the rc of 1 the commands following the line continue to get executed", you can add set -o errexit and set -o pipefail to the start of your script, it will then terminate directly if you get an error (even inside a piped command). Here a good resource with a more in-depth explanation.
I need to execute my python program from shell script so that the print commands in my python program will be exposed for reading from another program.
the shell script:
#!/bin/bash
if [ $# -lt 1 ] || [ $# -gt 2 ];then
echo "Usage: $0 APK <duration in seconds>"
exit 1;
fi
printf "$(python '/root/Desktop/DroidBox_4.1.1/scripts/droidbox.py' $1 $2 -c 'import sys; exec sys.stdin.read()')"
My python program should get the parametrs $1 and $2 but it doesn't recognize those as parametrs but taking -c and the command after it as the parametrs.
For answers like: getting the process input stream in my other project won't work for me. the only way that seems to be working is to use -c option and the command 'exec sys.stdin.read()'.
thank you.
It should work pretty well just the way you've written it. In a stripped down version here's what I get:
the (bash) test.sh script:
#!/bin/bash
python /tmp/test.py $1 $2
the (python) test.py script:
import sys
print "in python"
print sys.argv
and finally a shell session:
smassey#hacklabs:/tmp $ ./test.sh hello world
in python
['/tmp/test.py', 'hello', 'world']
As you can see, the bash script calls the python script which prints values to stdout so they're just as exposed as anything else directed to stdout:
smassey#hacklabs:/tmp $ ./test.sh hello world | grep python | tr 'a-z' 'A-Z'
IN PYTHON
The argparse module is very helpful as well. This allow you to add custom flags to your program (makes it more convenient to works with from a user's perspective as well).
Every now and then I need to get the answer to a calculation. As I usually have a terminal screen open that is a natural place for me to ask such mathematical questions.
The Python interactive shell is good for this purpose, provided you want to enter yet another shell only to have to exit out of it later on.
Sometimes though it is preferable to have the answer immediately available from the command line. Python has the -c command option that I found to be useful in processing a single command and returning the result. I wrote the following bash shell script to utilize it:
#!/bin/bash
# MHO 12-28-2014
#
# takes a equation from the command line, sends it to python and prints it
ARGS=0
#
if [ $# -eq 1 ]; then
ARGS=1
fi
#
if [ $ARGS -eq 0 ]; then
echo "pc - Python Command line calculator"
echo "ERROR: pc syntax is"
echo "pc EQUATION"
echo "Examples"
echo "pc 12.23+25.36 pc \"2+4+3*(55)\""
echo "Note: if calculating one single equation is not enough,"
echo "go elsewhere and do other things there."
echo "Enclose the equation in double quotes if doing anything fancy."
echo "m=math module ex. \"m.cos(55)\""
exit 1
fi
#
if [ $ARGS -eq 1 ]; then
eqn="$1"
python -c "import math; m=math; b=$eqn; print str(b)"
fi
#
Example Output
$ pc 1/3.0
0.333333333333
$ pc 56*(44)
2464
$ pc 56*(44)*3*(6*(4))
177408
$ pc "m.pi*(2**2)"
12.5663706144
Question, keeping in mind python -c option, is there any concise way to implicitly refer to the math module so that the last pc command might be formatted as pc "pi*(2**2)" ?
You can use
from math import *
to import all constants and functions from the math module into your global scope.
if [ $ARGS -eq 1 ]; then
eqn="$1"
python -c "from math import *; b=$eqn; print str(b)"
fi
$ pc "pi*(2**2)"
12.5663706144
Excellent! Thanks!