i'm calling a python script inside my bash script and I was wondering if there is a simple way to set my bash variables within my python script.
Example:
My bash script:
#!/bin/bash
someVar=""
python3 /some/folder/pythonScript.py
My python script:
anotherVar="HelloWorld"
Is there a way I can set my someVar to the value of anotherVar? I was thinking of printing properties in a file inside the python script and then read them from my bash script but maybe there is another way. Also I don't know and don't think it makes any difference but I can name both variable with the same name (someVar/someVar instead of someVar/anotherVar)
No, when you execute python, you start a new process, and every process has access only to their own memory. Imagine what would happen if a process could influence another processes memory! Even for parent/child processes like this, this would be a huge security problem.
You can make python print() something and use that, though:
#!/usr/bin/env python3
print('Hello!')
And in your shell script:
#!/usr/bin/env bash
someVar=$(python3 myscript.py)
echo "$someVar"
There are, of course, many others IPC techniques you could use, such as sockets, pipes, shared memory, etc... But without context, it's difficult to make a specific recommendation.
shlex.quote() in Python 3, or pipes.quote() in Python 2, can be used to generate code which can be evaled by the calling shell. Thus, if the following script:
#!/usr/bin/env python3
import sys, shlex
print('export foobar=%s' % (shlex.quote(sys.argv[1].upper())))
...is named setFoobar and invoked as:
eval "$(setFoobar argOne)"
...then the calling shell will have an environment variable set with the name foobar and the value argOne.
Related
This question already has answers here:
how to "source" file into python script
(8 answers)
Closed 3 years ago.
I am struggling to execute a shell script from a Python program. The actual issue is the script is a load profile script and runs manually as :
. /path/to/file
The program can't be run as sh script as the calling programs are loading some configuration file and so must need to be run as . /path/to/file
Please do guide how can I integrate the same in my Python script? I am using subprocess.Popen command to run the script and as said the only way it works is to run as . /path/to/file and so not giving the right result.
Without knowledge of the precise reason the script needs to be sourced, this is slightly speculative.
The fundamental problem is this: How do I get a source command to take effect outside the shell script?
Let's say your sourced file does something like
export fnord="value"
This cannot (usefully) be run in a subshell (as a normally executed script would) because the environment variable and its value will be lost when the script terminates. The solution is to source (aka .) this snippet from an already running shell; then the value stays in that shell's environment until that shell terminates.
But Python is not a shell, and there is no general way for Python to execute arbitrary shell script code, short of reimplementing the shell in Python. You can reimplement a small subset of the shell's functionality with something like
with open('/path/to/file') as shell_source:
lines = shell_source.readlines()
for line in lines:
if line.strip().startswith('export '):
var, value = line[7:].strip().split('=', 1)
if value.startswith('"'):
value = value.strip('"')
elif value.startswith("'"):
value = value.strip("'")
os.environ[var] = value
with some very strict restrictions (let's not say naïve assumptions) on the allowable shell script syntax in the file. But what if the file contained something else than a series of variable assignments, or the assignment used something other than trivial quoted strings in the values? (Even the export might or might not be there. Its significance is to make the variable visible to subprocesses of the current shell; maybe that is not wanted or required? Also export variable=value is not portable; proper Bourne shell script syntax would use variable=value; export variable or one of the many variations.)
If you know what exactly your Python script needs from the shell script, maybe do something like
r = subprocess.run('. /path/to/file; printf "%s\n" "$somevariable"',
shell=True, capture_output=True, text=True)
os.environ['somevariable'] = r.stdout.split('\n')[-2]
to source the entire script in a subshell, then print to standard output the part you actually need, and capture that from your Python script (and assign it to an environment variable if that's what you eventually need to accomplish).
I am writing a bash script in which a small python script is embedded. I want to pass a variable from python to bash. After a few search I only found method based on os.environ.
I just cannot make it work. Here is my simple test.
#!/bin/bash
export myvar='first'
python - <<EOF
import os
os.environ["myvar"] = "second"
EOF
echo $myvar
I expected it to output second, however it still outputs first. What is wrong with my script? Also is there any way to pass variable without export?
summary
Thanks for all answers. Here is my summary.
A python script embedded inside bash will run as child process which by definition is not able to affect parent bash environment.
The solution is to pass assignment strings out from python and eval it subsequently in bash.
An example is
#!/bin/bash
a=0
b=0
assignment_string=$(python -<<EOF
var1=1
var2=2
print('a={};b={}'.format(var1,var2))
EOF
)
eval $assignment_string
echo $a
echo $b
Unless Python is used to do some kind of operation on the original data, there's no need to import anything. The answer could be as lame as:
myvar=$(python - <<< "print 'second'") ; echo "$myvar"
Suppose for some reason Python is needed to spit out a bunch of bash variables and assignments, or (cautiously) compose code on-the-fly. An eval method:
myvar=first
eval "$(python - <<< "print('myvar=second')" )"
echo "$myvar"
Complementing the useful Cyrus's comment in question, you just can't do it. Here is why,
Setting an environment variable sets it only for the current process and any child processes it launches. os.environ will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable.
You can pretty much do that with a shell script itself and just source it to reflect it on the current shell.
There are a few "dirty" ways of getting something like this done. Here is an example:
#!/bin/bash
myvar=$(python - <<EOF
print "second"
EOF
)
echo "$myvar"
The output of the python process is stored in a bash variable. It gets a bit messy if you want to return more complex stuff, though.
You can make python return val and pass it to bash:
pfile.py
print(100)
bfile.sh
var=$(python pfile.py)
echo "$var"
output: 100
Well, this may not be what you want but one option could be running the other batch commands in python using subprocess
import subprocess
x =400
subprocess.call(["echo", str(x)])
But this is more of a temporary work around. The other solutions are more along what you are looking for.
Hope I was able to help!
I have a file with some environment variables that I want to use in a python script
The following works form the command line
$ source myFile.sh
$ python ./myScript.py
and from inside the python script I can access the variables like
import os
os.getenv('myvariable')
How can I source the shell script, then access the variables, from with the python script?
If you are saying backward environment propagation, sorry, you can't. It's a security issue. However, directly source environment from python is definitely valid. But it's more or less a manual process.
import subprocess as sp
SOURCE = 'your_file_path'
proc = sp.Popen(['bash', '-c', 'source {} && env'.format(SOURCE)], stdout=sp.PIPE)
source_env = {tup[0].strip(): tup[1].strip() for tup in map(lambda s: s.strip().split('=', 1), proc.stdout)}
Then you have everything you need in source_env.
If you need to write it back to your local environment (which is not recommended, since source_env keeps you clean):
import os
for k, v in source_env.items():
os.environ[k] = v
Another tiny attention needs to be paid here, is since I called bash here, you should expect the rules are applied here too. So if you want your variable to be seen, you will need to export them.
export VAR1='see me'
VAR2='but not me'
You can not load environmental variables in general from a bash or shell script, it is a different language. You will have to use bash to evaluate the file and then somehow print out the variables and then read them. see Forcing bash to expand variables in a string loaded from a file
This question already has answers here:
How to get the environment variables of a subprocess after it finishes running?
(5 answers)
Closed 9 years ago.
On Windows there's a 3rd party command line tool I would like to use in my python script. Let's say it's foobar.exe located under C:\Program Files (x86)\foobar. Foobar comes with an additional batch file init_env.bat that will set up the shell environment for foobar.exe to run.
I want to write a python script, that will first call init_env.bat once and then foobar.exe multiple times. However, all mechanisms I know of (subprocess, os.system and backticks) seem to spawn a new process for each execution. Therefore, calling init_env.bat is useless, because it does not change the environment of the process in which the python script runs and thus every subsequent call to foobar.exe fails, because it's environment is not set up.
Is it possible to call init_env.bat from python in a way that allows init_env.bat to alter the environment of the calling scripts process?
Is it possible to call init_env.bat from python in a way that allows
init_env.bat to alter the environment of the calling scripts
process?
Not easily, although, if the init_env.bat is really simple, you could attempt to parse it, and make the changes to os.environ yourself.
Otherwise it's much easier to spawn it in a sub-shell, followed by a call to set to output the new environment variables, and parse the output from that.
The following works for me...
init_env.bat
#echo off
set FOO=foo
set BAR=bar
foobar.bat
#echo off
echo FOO=%FOO%
echo BAR=%BAR%
main.py
import sys, os, subprocess
INIT_ENV_BAT = 'init_env.bat'
FOOBAR_EXE = 'foobar.bat'
def init_env():
vars = subprocess.check_output([INIT_ENV_BAT, '&&', 'set'], shell=True)
for var in vars.splitlines():
k, _, v = map(str.strip, var.strip().partition('='))
if k.startswith('?'):
continue
os.environ[k] = v
def main():
init_env()
subprocess.check_call(FOOBAR_EXE, shell=True)
subprocess.check_call(FOOBAR_EXE, shell=True)
if __name__ == '__main__':
main()
...for which python main.py outputs...
FOO=foo
BAR=bar
FOO=foo
BAR=bar
Note that I'm only using a batch file in place of your foobar.exe because I don't have a .exe file handy which can confirm the environment variables are set.
If you're using a .exe file, you can remove the shell=True clause from the lines subprocess.check_call(FOOBAR_EXE, shell=True).
I a trying to control the volume of mplayer from a python program. The mplayer program gets started from a bash script:
#!/bin/bash
mkfifo /home/administrator/files/mplayer-control.pipe
/usr/bin/mplayer -slave -input file=/home/administrator/files/mplayer-control.pipe /home/administrator/music/file.mp3
Then I have a GUI written in Python that is supposed to be able to control the volume of the instance of mplayer that is being played. I have tried the following:
os.system('echo "set_property volume $musicvol" > /home/administrator/files/mplayer-control.pipe')
That works if i substitute $musicvol with the numeric value instead, but that is unfortunately of no use. I need to be able to pass the variable.
I would also be able to solve it by invoking a bash script from the Python application, but I can not get that to work either:
subprocess.call("/home/administrator/files/setvolume.sh", executable="bash", shell=True)
You don't need to call os.system and invoke a shell to write that line to the FIFO from your Python script- you can just do:
new_volume = 50
with open("/home/administrator/files/mplayer-control.pipe","w") as fp:
fp.write("set_property volume %d\n" % (new_volume,))
It's not clear to me what you expect to happen in your original python, though - is musicvol set in the environment? If instead it's a Python variable that you want to insert into the string that you're passing, the easiest way is to use the string interpolation operator (%) as I've done in the example above.
In your example of using subprocess.call you don't need the executable or shell keyword arguments if setvolume.sh is executable and has a #! line - you could just do:
subprocess.call("/home/administrator/files/setvolume.sh")
However, it's better to just use open and write in Python as above, I think.