I am new to ruby. Still learning it.
I have bash script and a ruby script, having independent responsibilities.
The bash script calls the ruby script with some command line arguments. The ruby script, after parsing these command line args will set/unset some variables local to it. These variables will have the same use/name in the parent bash script.
So, i need to have the modified values of the variables in the ruby script be reflected in the parent bash script also.
Couple of approaches I have thought of are:
1.) Export the variables to the environment in bash, then have ruby modify them by using the ENV[] hash. In this case, there wouldn't be any need to have local variables in ruby to store these values, ruby would directly modify the imported variables from bash, as they are environment variables. However, would these changed values of the environment be reflected in the bash script?
2.) After the ruby script returns to the parent bash script, call on some methods of ruby that will return the a variable's value to it. So for each variable, a method is called. Something like for python,
python -c 'import python_module; print python_method()'
Is this possible in ruby?
If none of the above two are possible, is there any other way to have the values of ruby variables be reflected back in the parent bash script?
Or, is there any scripting option other than ruby/python that can help me do this?
I Appreciate the time in reading this post and helping me out. :-)
The simplest mechanism would be similar to the one used by the ssh-agent program.
Have your program output a set of environment export statements like
export THIS=something
export THAT=something_else
and then have the calling script eval the result, as in
eval `my_program`
This will cause the shell to incorporate your environment variables. For example, here is a small Python program.
print "export THIS=something"
print "export THAT='something else'"
When I run it, it outputs the two export statements:
airhead:Python sholden$ python stackoflo.20140710-1.py
export THIS=something
export THAT='something else'
So when I eval the result the export statements are executed by the calling shell:
airhead:Python sholden$ eval `python stackoflo.20140710-1.py`
The variables its sets are now a part of the environment:
airhead:Python sholden$ echo $THIS
something
airhead:Python sholden$ echo $THAT
something else
Finally, here's proof that the values do go into the shell's environment: a subshell sees them.
airhead:Python sholden$ bash
bash-3.2$ echo $THIS
something
Related
i'm calling a python script inside my bash script and I was wondering if there is a simple way to set my bash variables within my python script.
Example:
My bash script:
#!/bin/bash
someVar=""
python3 /some/folder/pythonScript.py
My python script:
anotherVar="HelloWorld"
Is there a way I can set my someVar to the value of anotherVar? I was thinking of printing properties in a file inside the python script and then read them from my bash script but maybe there is another way. Also I don't know and don't think it makes any difference but I can name both variable with the same name (someVar/someVar instead of someVar/anotherVar)
No, when you execute python, you start a new process, and every process has access only to their own memory. Imagine what would happen if a process could influence another processes memory! Even for parent/child processes like this, this would be a huge security problem.
You can make python print() something and use that, though:
#!/usr/bin/env python3
print('Hello!')
And in your shell script:
#!/usr/bin/env bash
someVar=$(python3 myscript.py)
echo "$someVar"
There are, of course, many others IPC techniques you could use, such as sockets, pipes, shared memory, etc... But without context, it's difficult to make a specific recommendation.
shlex.quote() in Python 3, or pipes.quote() in Python 2, can be used to generate code which can be evaled by the calling shell. Thus, if the following script:
#!/usr/bin/env python3
import sys, shlex
print('export foobar=%s' % (shlex.quote(sys.argv[1].upper())))
...is named setFoobar and invoked as:
eval "$(setFoobar argOne)"
...then the calling shell will have an environment variable set with the name foobar and the value argOne.
Question
My default Python is 2.7, but I have a script that requires Python 3.4. I am trying to create a function in R that will:
Switch to Python 3.4
Run this script
Switch back to Python 2.7
Import results into R
To switch between Python versions, I use my cluster's "dotkit" system, like this:
use Python-2.7
use Python-3.4
"use" is a bash function that is imported in my .bashrc file. It sets all of my path variables (PATH, LIBRARY_PATH, LD_LIBRARY_PATH, CPATH, C_INCLUDE_PATH, etc). The problem is that when I try to call this function in R, I get the following error:
system('use Python-3.4')
sh: use: command not found
It seems like this is a problem with my PATH. I am using the correct shell:
system('echo $SHELL')
/bin/bash
My $PATH variable also looks good. However, when I create a script that essentially does the same thing:
load_py34.sh:
#!/bin/bash
source ~/.bashrc
use Python-3.4
and call this script through R, then it actually runs, but for some reason, it doesn't change my python version within R. (I have verified that this script works from the command line.)
> R
> system('python --version')
Python 2.7.1
> system('sh load_py34.sh')
Prepending: R-3.4 (ok)
> system('python --version')
Python 2.7.1
So I'm a little confused, but if anyone can help, I would really appreciate it.
Suggested fixes
When I combine them into a single command, I still have the same problem:
> system("sh load_py34.sh; python --version")
Prepending: Python-3.4 (already loaded)
Python 2.7.1
When I try calling bash directly, I still have a problem with the PATH:
> system("bash -c 'use Python-3.4; python --version'")
bash: use: command not found
Python 2.7.1
.bashrc is only loaded for interactive bash sessions.
"use" is a bash function that is imported in my .bashrc file. It sets
all of my path variables.
If set via export, the environment of the calling process will not be altered.
export [-fn] [name[=word]] ... The supplied names are marked for automatic export to the environment of subsequently executed commands. (https://man7.org/linux/man-pages/man1/bash.1.html)
Child processes do not normally have access to the parent process' environment. (This poses a problem because system() creates a sub-process.)
The source and . built-ins execute the commands in the current shell environment, hence why your script works.
Other commands (executables, non-shell-builtins) are executed by the fork-and-exec mechanism, whereby the executing shell process forks, creating a child process with an identical environment and state. This new child process is the process in which the command is executed. Changes to the environment of that process are not replicated to the parent's environment.
This means that you will not be able to rely on system('...') to modify the environment of the R process, or that of processes spawned by subsequent system() invocations.
In a single invocation to system(), you can construct a command-line that changes the environment of the spawned shell like so:
bash -c 'source ~/.bashrc; use Python-3.4; python --version'
Mind you, ~/.bashrc is not really the best place to put this functionality (might be subjective).
When you call system() it uses /bin/sh, not /bin/bash. sh doesn't read your .bashrc file when it starts up, so it does not know any of the functions you've defined there.
To use the function from your .bashrc, you must get bash to run it instead:
system("bash -c 'use Python-3.4; python --version'")
Edit: placement of closing single quote.
This question already has answers here:
Assign environment variables from bash script to current session from Python
(3 answers)
Closed 5 years ago.
I have a script called "script.sh," whose contents are:
#!/bin/sh
export A=5
I want to execute this script from within python (iPython actually) and read the variable 'A'.
import os
import subprocess
subprocess.call('./script.sh')
A=os.environ['A']
Unfortunately, this doesn't seem to work, giving me an error that A cannot be found. If I understand correctly, subprocess is actually running in a different shell than the one that os.environ queries. But then why can't I run something like:
subprocess.call('echo $A')
?
What should I change to make this work? In general, I just want to obtain the value of "A" from the script, preferably by executing the script through some form of shell (the actual script is quite long).
For some more info, the script will contain login credentials, so ideally I'd like a safe,minimalist way of accessing their values.
You need to source the script in the subshell (so it sets the variable in the same shell process), then echo the variable in that subshell:
a = subprocess.check_output('source ./script.sh; echo "$A"', shell=True)
Then you can read from the pipe to get the value of the variable.
The trick will be to spawn a new shell and tell it to both interpret your script's code, and can print out its environment in a way that Python can read it. A one-liner:
In [10]: subprocess.check_output(["bash", "-c", "source ./script.sh; env"])
Out[10]: '...\nA=5\n...'
What's happening: In general, environment variables are set at the beginning of a program, and any subprocesses can't modify their parent's environment; it's a sort of sandbox. But source is a bash builtin where bash says "instead of spawning script.sh as a new (sub-)subprocess which couldn't modify my environ, run the lines of code as myself (bash) and modify my environ accordingly for future commands". And env is tacked on so that bash prints the environment separated by newlines. check_output simply grabs that output and brings it back into Python.
(As a side note, that source command is what you use to update a shell to use a certain virtualenv: source my_project/bin/activate. Then the $PATH and other variables of your current shell are updated to use the virtualenv python and libraries for the rest of that session. You can't just say my_project/bin/activate since it would set them in a subshell, doing nothing :))
I want to set an environment variable with a Python script, influencing the shell I am starting the script in. Here is what I mean
python -c "import os;os.system('export TESTW=1')"
But the command
echo ${TESTW}
returns nothing. Also with the expression
python -c "import os;os.environ['TEST']='1'"
it does not work.
Is there another way to do this in the direct sense? Or is it better to write the variables in a file which I execute from 'outside' of the Python script?
You can influence environment via: putenv BUT it will not influence the caller environment, only environment of forked children.
It's really much better to setup environment before launching the python script.
I may propose such variant. You create a bash script and a python script. In bash script you call the python script with params. One param - one env variable. Eg:
#!/bin/bash
export TESTV1=$(python you_program.py testv1)
export TESTV2=$(python you_program.py testv2)
and you_program.py testv1 returns value just for one env variable.
I would strongly suggest using the solution proposed by chepner and Maxym (where the Python script provides the values and your shell exports the variables). If that is not an option for you, you could still use eval to execute what the python script writes in your current Bash process:
eval $( python -c "print('export TESTW=1')" )
Caution: eval is usually read "evil" in Bash programming. As a general rule of thumb, one should avoid "blindly" executing code that is not fully under one's control. That includes being generated by another program at runtime as in this case. See also Stack Overflow question Why should eval be avoided in Bash, and what should I use instead?.
I want to run a csh file from a python script,
example,
#!/usr/bin/python
import os
os.system("source path/to/file.csh")
and I want this file to run in the same shell as I am running the python script, because the file.csh script is settings some environment variables that I need.
Does anyone know how to do this in Python?
A child process cannot affect the environment of the parent process. The best you can do is to run your csh script in a separate process, get the environment variables that it defines, then set each environment variable in your python script.
Even with that, the python script won't be able to affect the shell in which you run the python script.
The common way to solve this (AFAIK) is to have your script emit shell commands to set the environment, then from the main shell you run the script and eval what you get back.
For more information you might want to check out this question: can a shell script set environment variables of the calling shell
You can kludge it this way:
#!/usr/bin/env python
# This is kludge.py
print "setenv VARNAME \"the value\""
In your case, you can have the file.sh print the setenv line.
Then from csh:
$ eval `./kludge.py`
$ echo $VARNAME
the value
This isn't clean, but it is the only way to have a child process effect the environment of its parent. This is only because the parent process is explicitly letting it happen with eval.