I have a shell script that calls various python scripts and passes an argument it got onto the python scripts. However, the python scripts are currently not getting the argument. when i go to sys.argv it instead shows something completely different. I have the code and sys.argv below:
Shell Script (called run_everything.sh) :
#!/bin/bash
source venv/bin/activate
python3 eval_otare_on_otare.py $1
then i have a line in eval_otare_on_otare.pt that prints the arguments passed:
print(sys.argv)
And I get the following list:
['eval_care_on_otare.py', 'run_everything.sh']
what can I do because sys.argv is clearly not getting what I want, I want it to return
['eval_care_on_otare.py', $1]
where $1 is the argument passed
If your activate script messes up the value of $1, save it first.
#!/bin/bash
arg=$1
source ./venv/bin/activate
python3 eval_otare_on_otare.py "$arg"
Tangentially note also When to wrap quotes around a shell variable?
I want to call from python a shell script which contain the running of another python function. I would like to use for that subprocess method. My code so far look like:
arguments = ["./my_shell.sh", path]
ret_val = subprocess.Popen(arguments, stdout=subprocess.PIPE)
while the script is the following:
#!/bin/sh
cd ...
python -c "from file import method;
method()"
How can I give in the directory (to cd) of the path that I pass as an argument in the shell file?
You can access your arguments as $1, $2, etc. So your cd command would simply be cd $1.
I have a python script...it gets called different then bash
./script.py 'arg1' 'arg2 stillarg2'
In the python script there is two arguments.
Now I want to call that with a bash script
#!/bin/bash
./script.py ${1} ${2} ${3}
I want to call the bash script by in this syntax
./script.sh arg1 arg2 stillarg2
In the Bash script there is 3 arguments and no quotes.
So is there a way that by using a bash script, could call the python script with arguments. Without quotes and 3 arguments?.
I've Tried:
#!/bin/bash
./script.py "'${1}'" "'${2} ${3}'"
Outcome is:
operation not support
You just want to combine $2 and $3 in the same argument, no?
#!/bin/bash
./script.py "${1}" "${2} ${3}"
This question is a simpler version of this question.
In simple terms, I have a custom bash function my_func defined on ~/.bash_profile and uses two more bash functions defined in the same environment.
Also, my_func accepts two arguments, let's say a and b. What my_func does is that it gets connected to a remote server and sends some files (these are determined by a and b).
If I type on the bash shell:
. my_func a b everything works fine and I get some print statements on the screen.
However, if I include:
subprocess.call(['#!/bin/bash . my_func a b'], shell=True) nothing seems to happen.
I tried to export all the bash functions that are used by my_func by including:
subprocess.call(['#!/bin/bash export -f my_func'], shell=True) and I did the same for the rest of the functions that are used by my_func.
EDIT:
If I use subprocess.call(['bash', '-c' ,'my_func a b], shell=True) the bash shell will change into bash-3.2$
You need to export the function before you start the python program:
export -f my_func
python foo.py
Well, the above example might not work if the system's default shell (/bin/sh) is not bash. To circumvent this, you may use subprocess call like this:
$ function foo() { echo "bar" ; }
$ export -f foo
$ cat foo.py
import subprocess
subprocess.call(['bash', '-c', 'foo'])
$ python foo.py
bar
Alternative:
I would put the function into a lib file, let's say:
# /usr/share/my_lib/my_lib.sh
function my_func() {
# do something
}
Then I would expose the function via a script in PATH:
#!/bin/bash
# /usr/local/bin/my_prog
source /usr/lib/my_lib/my_lib.sh
my_func "$1" "$2"
In Python you would just:
subprocess.call(['/usr/local/bin/my_prog', 'a', 'b'])
Btw: If you don't need that function somewhere else, you can just put it directly into /usr/local/bin/my_prog.
I was hoping to write a python script to create some appropriate environmental variables by running the script in whatever directory I'll be executing some simulation code, and I've read that I can't write a script to make these env vars persist in the mac os terminal. So two things:
Is this true?
and
It seems like it would be a useful things to do; why isn't it possible in general?
You can't do it from python, but some clever bash tricks can do something similar. The basic reasoning is this: environment variables exist in a per-process memory space. When a new process is created with fork() it inherits its parent's environment variables. When you set an environment variable in your shell (e.g. bash) like this:
export VAR="foo"
What you're doing is telling bash to set the variable VAR in its process space to "foo". When you run a program, bash uses fork() and then exec() to run the program, so anything you run from bash inherits the bash environment variables.
Now, suppose you want to create a bash command that sets some environment variable DATA with content from a file in your current directory called ".data". First, you need to have a command to get the data out of the file:
cat .data
That prints the data. Now, we want to create a bash command to set that data in an environment variable:
export DATA=`cat .data`
That command takes the contents of .data and puts it in the environment variable DATA. Now, if you put that inside an alias command, you have a bash command that sets your environment variable:
alias set-data="export DATA=`cat .data`"
You can put that alias command inside the .bashrc or .bash_profile files in your home directory to have that command available in any new bash shell you start.
One workaround is to output export commands, and have the parent shell evaluate this..
thescript.py:
import pipes
import random
r = random.randint(1,100)
print("export BLAHBLAH=%s" % (pipes.quote(str(r))))
..and the bash alias (the same can be done in most shells.. even tcsh!):
alias setblahblahenv="eval $(python thescript.py)"
Usage:
$ echo $BLAHBLAH
$ setblahblahenv
$ echo $BLAHBLAH
72
You can output any arbitrary shell code, including multiple commands like:
export BLAHBLAH=23 SECONDENVVAR='something else' && echo 'everything worked'
Just remember to be careful about escaping any dynamically created output (the pipes.quote module is good for this)
If you set environment variables within a python script (or any other script or program), it won't affect the parent shell.
Edit clarification:
So the answer to your question is yes, it is true.
You can however export from within a shell script and source it by using the dot invocation
in fooexport.sh
export FOO="bar"
at the command prompt
$ . ./fooexport.sh
$ echo $FOO
bar
It's not generally possible. The new process created for python cannot affect its parent process' environment. Neither can the parent affect the child, but the parent gets to setup the child's environment as part of new process creation.
Perhaps you can set them in .bashrc, .profile or the equivalent "runs on login" or "runs on every new terminal session" script in MacOS.
You can also have python start the simulation program with the desired environment. (use the env parameter to subprocess.Popen (http://docs.python.org/library/subprocess.html) )
import subprocess, os
os.chdir('/home/you/desired/directory')
subprocess.Popen(['desired_program_cmd', 'args', ...], env=dict(SOMEVAR='a_value') )
Or you could have python write out a shell script like this to a file with a .sh extension:
export SOMEVAR=a_value
cd /home/you/desired/directory
./desired_program_cmd
and then chmod +x it and run it from anywhere.
What I like to do is use /usr/bin/env in a shell script to "wrap" my command line when I find myself in similar situations:
#!/bin/bash
/usr/bin/env NAME1="VALUE1" NAME2="VALUE2" ${*}
So let's call this script "myappenv". I put it in my $HOME/bin directory which I have in my $PATH.
Now I can invoke any command using that environment by simply prepending "myappenv" as such:
myappenv dosometask -xyz
Other posted solutions work too, but this is my personal preference. One advantage is that the environment is transient, so if I'm working in the shell only the command I invoke is affected by the altered environment.
Modified version based on new comments
#!/bin/bash
/usr/bin/env G4WORKDIR=$PWD ${*}
You could wrap this all up in an alias too. I prefer the wrapper script approach since I tend to have other environment prep in there too, which makes it easier for me to maintain.
As answered by Benson, but the best hack-around is to create a simple bash function to preserve arguments:
upsert-env-var (){ eval $(python upsert_env_var.py $*); }
Your can do whatever you want in your python script with the arguments. To simply add a variable use something like:
var = sys.argv[1]
val = sys.argv[2]
if os.environ.get(var, None):
print "export %s=%s:%s" % (var, val, os.environ[var])
else:
print "export %s=%s" % (var, val)
Usage:
upsert-env-var VAR VAL
As others have pointed out, the reason this doesn't work is that environment variables live in a per-process memory spaces and thus die when the Python process exits.
They point out that a solution to this is to define an alias in .bashrc to do what you want such as this:
alias export_my_program="export MY_VAR=`my_program`"
However, there's another (a tad hacky) method which does not require you to modify .bachrc, nor requires you to have my_program in $PATH (or specify the full path to it in the alias). The idea is to run the program in Python if it is invoked normally (./my_program), but in Bash if it is sourced (source my_program). (Using source on a script does not spawn a new process and thus does not kill environment variables created within.) You can do that as follows:
my_program.py:
#!/usr/bin/env python3
_UNUSED_VAR=0
_UNUSED_VAR=0 \
<< _UNUSED_VAR
#=======================
# Bash code starts here
#=======================
'''
_UNUSED_VAR
export MY_VAR=`$(dirname $0)/my_program.py`
echo $MY_VAR
return
'''
#=========================
# Python code starts here
#=========================
print('Hello environment!')
Running this in Python (./my_program.py), the first 3 lines will not do anything useful and the triple-quotes will comment out the Bash code, allowing Python to run normally without any syntax errors from Bash.
Sourcing this in bash (source my_program.py), the heredoc (<< _UNUSED_VAR) is a hack used to "comment out" the first-triple quote, which would otherwise be a syntax error. The script returns before reaching the second triple-quote, avoiding another syntax error. The export assigns the result of running my_program.py in Python from the correct directory (given by $(dirname $0)) to the environment variable MY_VAR. echo $MY_VAR prints the result on the command-line.
Example usage:
$ source my_program.py
Hello environment!
$ echo $MY_VAR
Hello environment!
However, the script will still do everything it did before except exporting, the environment variable if run normally:
$ ./my_program.py
Hello environment!
$ echo $MY_VAR
<-- Empty line
As noted by other authors, the memory is thrown away when the Python process exits. But during the python process, you can edit the running environment. For example:
>>> os.environ["foo"] = "bar"
>>> import subprocess
>>> subprocess.call(["printenv", "foo"])
bar
0
>>> os.environ["foo"] = "foo"
>>> subprocess.call(["printenv", "foo"])
foo
0