I am writing a bash script in which a small python script is embedded. I want to pass a variable from python to bash. After a few search I only found method based on os.environ.
I just cannot make it work. Here is my simple test.
#!/bin/bash
export myvar='first'
python - <<EOF
import os
os.environ["myvar"] = "second"
EOF
echo $myvar
I expected it to output second, however it still outputs first. What is wrong with my script? Also is there any way to pass variable without export?
summary
Thanks for all answers. Here is my summary.
A python script embedded inside bash will run as child process which by definition is not able to affect parent bash environment.
The solution is to pass assignment strings out from python and eval it subsequently in bash.
An example is
#!/bin/bash
a=0
b=0
assignment_string=$(python -<<EOF
var1=1
var2=2
print('a={};b={}'.format(var1,var2))
EOF
)
eval $assignment_string
echo $a
echo $b
Unless Python is used to do some kind of operation on the original data, there's no need to import anything. The answer could be as lame as:
myvar=$(python - <<< "print 'second'") ; echo "$myvar"
Suppose for some reason Python is needed to spit out a bunch of bash variables and assignments, or (cautiously) compose code on-the-fly. An eval method:
myvar=first
eval "$(python - <<< "print('myvar=second')" )"
echo "$myvar"
Complementing the useful Cyrus's comment in question, you just can't do it. Here is why,
Setting an environment variable sets it only for the current process and any child processes it launches. os.environ will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable.
You can pretty much do that with a shell script itself and just source it to reflect it on the current shell.
There are a few "dirty" ways of getting something like this done. Here is an example:
#!/bin/bash
myvar=$(python - <<EOF
print "second"
EOF
)
echo "$myvar"
The output of the python process is stored in a bash variable. It gets a bit messy if you want to return more complex stuff, though.
You can make python return val and pass it to bash:
pfile.py
print(100)
bfile.sh
var=$(python pfile.py)
echo "$var"
output: 100
Well, this may not be what you want but one option could be running the other batch commands in python using subprocess
import subprocess
x =400
subprocess.call(["echo", str(x)])
But this is more of a temporary work around. The other solutions are more along what you are looking for.
Hope I was able to help!
Related
I have written two python scripts A.py and B.py So B.py gets called in A.py like this:
config_object = {}
with open(config_filename) as data:
config_object = json.load(data, object_pairs_hook=OrderedDict)
command = './scripts/B.py --config-file={} --token-a={} --token-b={}'.format(promote_config_filename, config_object['username'], config_object['password'])
os.system(command)
In here config_object['password'] contains & in it. Say it is something like this S01S0lQb1T3&BRn2^Qt3
Now when this value get passed to B.py it gets password as S01S0lQb1T3 So after & whatever it is getting ignored.
How to solve this?
os.system runs a shell. You can escape arbitrary strings for the shell with shlex.quote() ... but a much superior solution is to use subprocess instead, like the os.system documentation also recommends.
subprocess.run(
['./scripts/B.py',
'--config-file={}'.format(promote_config_filename),
'--token-a={}'.format(config_object['username']),
'--token-b={}'.format(config_object['password'])])
Because there is no shell=True, the strings are now passed to the subprocess verbatim.
Perhaps see also Actual meaning of shell=True in subprocess
#tripleee has good suggestions. In terms of why this is happening, if you are running Linux/Unix at least, the & would start a background process. You can search "linux job control" for more info on that. The shortest (but not best) solution is to wrap your special characters in single or double quotes in the final command.
See this bash for a simple example:
$ echo foo&bar
[1] 20054
foo
Command 'bar' not found, but can be installed with:
sudo apt install bar
[1]+ Done echo foo
$ echo "foo&bar"
foo&bar
I've come across a situation where it would be convenient to use python within a bash script I'm writing. I call some executables within my script, then want to do a bit of light data processing with python, then carry on. It doesn't seem worth it to me to write a dedicated script for the processing.
So what I want to do is something like the following:
# do some stuff in bash script
# write some data into datafile.d
python_fragment= << EOF
f = open("datafile.d")
// do some stuff with opened file
print(result)
EOF
result=$(execute_python_fragment $python_fragment) # <- what I want to do
# do some stuff with result
Basically all I want to do is execute a string containing python code. I could of course just make another file containing the python code and execute that, but I'd prefer not to do so. I could do something like echo $python_fragment > temp_code_file, then execute temp_code_file, but that seems inelegant. I just want to execute the string directly, if that's possible.
What I want to do seems simple enough, but haven't figured it out or found the solution online.
Thanks!
You can run a python command direct from the command line with -c option
python -c 'from foo import hello; print (hello())'
Then with bash you could do something like
result=$(python -c '$python_fragment')
You only have to redirect that here-string/document to python
python <<< "print('Hello')"
or
python <<EOF
print('Hello')
EOF
and encapsulate that in a function
execute_python_fragment() {
python <<< "$1"
}
and now you can do your
result=$(execute_python_fragment "${python_fragment}")
You should also add some kind of error control, input sanitizing... it's up to you the level of security you need in this function.
If the string contains the exact python code, then this simple eval() function works.
Here's a really basic example:
>>> eval("print(2)")
2
Hope that helps.
maybe something like
result=$(echo $python_fragment | python3)
only problem is the heredoc assignment in the question doesn't work either. But https://stackoverflow.com/a/1167849 suggests a way to do it if that is what you want to do:
python_fragment=$(cat <<EOF
print('test message')
EOF
) ;
result=$(echo $python_fragment | python3)
echo result was $result
This question already has answers here:
Is it possible to change the Environment of a parent process in Python?
(4 answers)
Closed 4 years ago.
I have a bash script that looks like this:
python myPythonScript.py
python myOtherScript.py $VarFromFirstScript
and myPythonScript.py looks like this:
print("Running some code...")
VarFromFirstScript = someFunc()
print("Now I do other stuff")
The question is, how do I get the variable VarFromFirstScript back to the bash script that called myPythonScript.py.
I tried os.environ['VarFromFirstScript'] = VarFromFirstScript but this doesn't work (I assume this means that the python environment is a different env from the calling bash script).
you cannot propagate an environment variable to the parent process. But you can print the variable, and assign it back to the variable name from your shell:
VarFromFirstScript=$(python myOtherScript.py $VarFromFirstScript)
you must not print anything else in your code, or using stderr
sys.stderr.write("Running some code...\n")
VarFromFirstScript = someFunc()
sys.stdout.write(VarFromFirstScript)
an alternative would be to create a file with the variables to set, and make it parse by your shell (you could create a shell that the parent shell would source)
import shlex
with open("shell_to_source.sh","w") as f:
f.write("VarFromFirstScript={}\n".format(shlex.quote(VarFromFirstScript))
(shlex.quote allows to avoid code injection from python, courtesy Charles Duffy)
then after calling python:
source ./shell_to_source.sh
You can only pass environment variables from parent process to child.
When the child process is created the environment block is copied to the child - the child has a copy, so any changes in the child process only affects the child's copy (and any further children which it creates).
To communicate with the parent the simplest way is to use command substitution in bash where we capture stdout:
Bash script:
#!/bin/bash
var=$(python myPythonScript.py)
echo "Value in bash: $var"
Python script:
print("Hollow world!")
Sample run:
$ bash gash.sh
Value in bash: Hollow world!
You have other print statements in python, you will need to filter out to only the data you require, possibly by marking the data with a well-known prefix.
If you have many print statements in python then this solution is not scalable, so you might need to use process substitution, like this:
Bash script:
#!/bin/bash
while read -r line
do
if [[ $line = ++++* ]]
then
# Strip out the marker
var=${line#++++}
else
echo "$line"
fi
done < <(python myPythonScript.py)
echo "Value in bash: $var"
Python script:
def someFunc():
return "Hollow World"
print("Running some code...")
VarFromFirstScript = someFunc()
# Prefix our data with a well-known marker
print("++++" + VarFromFirstScript)
print("Now I do other stuff")
Sample Run:
$ bash gash.sh
Running some code...
Now I do other stuff
Value in bash: Hollow World
I would source your script, this is the most commonly used method. This executes the script under the current shell instead of loading another one. Because this uses same shell env variables you set will be accessible when it exits. . /path/to/script.sh or source /path/to/script.sh will both work, . works where source doesn't sometimes.
I am trying to convert this script to Python but am having problems with the BASH syntax ${variable}. This does something I do not understand as I have very little experience with BASH scripting. My question is basically,how can I do the dollar brackets variable thing in Python ${variable}. In my case the variable is a number from zero to two-hundred and fifty-five. Please see the BASH script below. I am trying to do the exact same thing in Python.
do
for code in {0..255};
do echo -e "\e[38;05;${code}m $code: Test";
done;
done
Here is my attempt to convert the code to a Python script. Unfortunately, I am still having to call BASH via the os.system() method or function. Please check out my script below. The script does not function the same way though, with changing the text in the BASH shell. The Python script below simply prints out the crazy text below and increments the numbers... :/
#! /usr/bin/env python
import os
def run():
for code in range(0, 255):
os.system('echo -e "\e[38;05;%dm %s Test"' % (code,str(code)))
run()
You can use print command and string formatting to evaluate your variable during the print.
BTW, you can use xrange rather than range in order not to generate all the numbers in your memory, but to yield it one-by-one (for large range)
You can use this:
import os
def run():
for code in range(0, 256):
print "\x1b[38;05;{code}m {code} Test".format(code=code)
run()
You’ll need to use string formatting.
Strangely enough, you already do this in your code (but using the old % syntax instead of the new .format method). I’m not even sure why you think you have to call echo since the string you pass to it is already the string that you are trying to get (if I understand your question right). Just use the print function to output the string.
try this:
#! /usr/bin/env python
import os
def run():
for code in range(0, 255):
os.system('/bin/echo -e "\e[38;05;%dm %s Test"' % (code,str(code)))
run()
the echo is usually both extrnal program and internal command of shell, which can make difference sometime
i'm calling a python script inside my bash script and I was wondering if there is a simple way to set my bash variables within my python script.
Example:
My bash script:
#!/bin/bash
someVar=""
python3 /some/folder/pythonScript.py
My python script:
anotherVar="HelloWorld"
Is there a way I can set my someVar to the value of anotherVar? I was thinking of printing properties in a file inside the python script and then read them from my bash script but maybe there is another way. Also I don't know and don't think it makes any difference but I can name both variable with the same name (someVar/someVar instead of someVar/anotherVar)
No, when you execute python, you start a new process, and every process has access only to their own memory. Imagine what would happen if a process could influence another processes memory! Even for parent/child processes like this, this would be a huge security problem.
You can make python print() something and use that, though:
#!/usr/bin/env python3
print('Hello!')
And in your shell script:
#!/usr/bin/env bash
someVar=$(python3 myscript.py)
echo "$someVar"
There are, of course, many others IPC techniques you could use, such as sockets, pipes, shared memory, etc... But without context, it's difficult to make a specific recommendation.
shlex.quote() in Python 3, or pipes.quote() in Python 2, can be used to generate code which can be evaled by the calling shell. Thus, if the following script:
#!/usr/bin/env python3
import sys, shlex
print('export foobar=%s' % (shlex.quote(sys.argv[1].upper())))
...is named setFoobar and invoked as:
eval "$(setFoobar argOne)"
...then the calling shell will have an environment variable set with the name foobar and the value argOne.