I have some instrument which requires environment variable which I want to set automatically from python code. So I tried several ways to make it happen, but none of them were successful.
Here are some examples:
I insert following code in my python script
import os
os.system("export ENV_VAR=/some_path")
I created bash script(env.sh) and run it from python:
#!/bin/bash
export ENV_VAR=some_path
#call it from python
os.system("source env.sh")
I also tried os.putenv() and os.environ*["ENV_VAR"] = "some_path"
Is it possible to set(export) environment variable using python, i.e
without directly exporting it to shell?
Setting an environment variable sets it only for the current process and any child processes it launches. So using os.system will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable. Setting it using os.putenv or os.environ has a similar effect; the environment variables are set for the Python process and any children of it.
I assume you are trying to have those variables set for the shell that you launch the script from, or globally. That can't work because the shell (or other process) is not a child of the Python script in which you are setting the variable.
You'll have better luck setting the variables in a shell script. If you then source that script (so that it runs in the current instance of the shell, rather than in a subshell) then they will remain set after the script ends.
As long as you start the "instrument" (a script I suppose) from the very same process it should work:
In [1]: os.putenv("VARIABLE", "123")
In [2]: os.system("echo $VARIABLE")
123
You can't change an environment variable of a different process or a parent process.
A shell function may do this. You need to print your export statement and eval that.
set_shell_env() {
output=$(python print_export_env.py $*)
eval $output
}
Depending on how you execute your instrument, you might be able to change environment specifically for the child process without affecting the parent. See documentation for os.spawn*e or subprocess.Popen which accept separate argument denoting child environment. For example, Replacing the os.spawn family in subprocess module documentation which provides both usages:
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Related
This question already has answers here:
Assign environment variables from bash script to current session from Python
(3 answers)
Closed 5 years ago.
I have a script called "script.sh," whose contents are:
#!/bin/sh
export A=5
I want to execute this script from within python (iPython actually) and read the variable 'A'.
import os
import subprocess
subprocess.call('./script.sh')
A=os.environ['A']
Unfortunately, this doesn't seem to work, giving me an error that A cannot be found. If I understand correctly, subprocess is actually running in a different shell than the one that os.environ queries. But then why can't I run something like:
subprocess.call('echo $A')
?
What should I change to make this work? In general, I just want to obtain the value of "A" from the script, preferably by executing the script through some form of shell (the actual script is quite long).
For some more info, the script will contain login credentials, so ideally I'd like a safe,minimalist way of accessing their values.
You need to source the script in the subshell (so it sets the variable in the same shell process), then echo the variable in that subshell:
a = subprocess.check_output('source ./script.sh; echo "$A"', shell=True)
Then you can read from the pipe to get the value of the variable.
The trick will be to spawn a new shell and tell it to both interpret your script's code, and can print out its environment in a way that Python can read it. A one-liner:
In [10]: subprocess.check_output(["bash", "-c", "source ./script.sh; env"])
Out[10]: '...\nA=5\n...'
What's happening: In general, environment variables are set at the beginning of a program, and any subprocesses can't modify their parent's environment; it's a sort of sandbox. But source is a bash builtin where bash says "instead of spawning script.sh as a new (sub-)subprocess which couldn't modify my environ, run the lines of code as myself (bash) and modify my environ accordingly for future commands". And env is tacked on so that bash prints the environment separated by newlines. check_output simply grabs that output and brings it back into Python.
(As a side note, that source command is what you use to update a shell to use a certain virtualenv: source my_project/bin/activate. Then the $PATH and other variables of your current shell are updated to use the virtualenv python and libraries for the rest of that session. You can't just say my_project/bin/activate since it would set them in a subshell, doing nothing :))
subprocess.run('export FOO=BAR', shell=True)
This simply doesn't work, and I have no idea why.
All I am trying to do I set an environment variable from my python (3.5.1) script, and when I run the above line, nothing happens. No errors are raised, and when I check the environment variable myself, it has not been set.
Other shell commands with subprocess.run() do work, such as ls and pwd, but not export.
.run() was added in Python 3.5 (in case you didn't recognise it), but I have also tried the above line with .call() and .Popen(), with no change in results.
I am aware that I can set environment variables in python with os.environ['FOO'] = "BAR", but I will be using shell commands a lot in my project, and I expect that I will need to string multiple commands together, which will make using export easier than os.environ.
My project will run on Linux, which is what my machine is running on.
It works fine; however, the variable setting only exists in the subprocess. You cannot affect the environment of the local process from a child.
os.environ is the correct solution, as it changes the environment of the local process, and those changes will be inherited by any process started with subprocess.run.
You can also use the env argument to run:
subprocess.run(["cmdname", "arg1", "arg number 2"], env=dict(FOO='BAR', **os.environ))
This runs the command in a modified environment that includes FOO=BAR without modifying the current environment.
I need to source the environment of child process. I have a c-shell script(really complicated) that sets many environment variables and I want to use them in the parent process. I doing something like this:
subprocess.call(['set_env_vars.csh; env>crazy_vars.log' shell=True])
In this way I am trying to get the env of child process but this method is not working as I think commands after semicolon are treated as separate processes.
A possible solution is that I created another c-shell script and put those two commands in there and then call that script in python but thats a dirty way.
Is there a way to make two commands as the part of same process.
Thanks
On my system (as with many others) the shell is bash, not csh, so explicitly invoking csh is a good idea. Also, you need to source, not execute set_env_vars.csh:
subprocess.call(['/bin/csh', '-c', 'source set_env_vars.csh; env > crazy_vars.log'])
In my build (I'm using Linux) I need to call a Python script and set some env variables. I need these variables to be set even after I exit the script. I am able to set it using os.environ within the script but whenever I exit the script and try to see if the env variable is set from the terminal (echo $myenv) - I get nothing.
I am new to Python and did quite a bit googling to figure this out. However, I am not quite sure if it's possible. I tried using the subprocess:
subprocess.call('setenv myenv 4s3', shell=True)
Also tried using os.system:
os.system("setenv myenv 4s3")
So far, I didn't succeed.
You cannot set environment variables from a child process and have them be visible in the parent process. Every process gets its own copy of the environment, and changes do not propagate upwards.
What you could do is have the Python script print the settings it wants to change and have the outside shell execute the appropriate commands.
Maybe if you find some equivalent function like c vfork for Python.
When you vfork, both processes share memory space so, you might overwrite environment variables in parent process from child process.
Warning: vfork has many security issues, and therefore not recommended. Just use it if you are desperate.
I want to run a csh file from a python script,
example,
#!/usr/bin/python
import os
os.system("source path/to/file.csh")
and I want this file to run in the same shell as I am running the python script, because the file.csh script is settings some environment variables that I need.
Does anyone know how to do this in Python?
A child process cannot affect the environment of the parent process. The best you can do is to run your csh script in a separate process, get the environment variables that it defines, then set each environment variable in your python script.
Even with that, the python script won't be able to affect the shell in which you run the python script.
The common way to solve this (AFAIK) is to have your script emit shell commands to set the environment, then from the main shell you run the script and eval what you get back.
For more information you might want to check out this question: can a shell script set environment variables of the calling shell
You can kludge it this way:
#!/usr/bin/env python
# This is kludge.py
print "setenv VARNAME \"the value\""
In your case, you can have the file.sh print the setenv line.
Then from csh:
$ eval `./kludge.py`
$ echo $VARNAME
the value
This isn't clean, but it is the only way to have a child process effect the environment of its parent. This is only because the parent process is explicitly letting it happen with eval.