In my build (I'm using Linux) I need to call a Python script and set some env variables. I need these variables to be set even after I exit the script. I am able to set it using os.environ within the script but whenever I exit the script and try to see if the env variable is set from the terminal (echo $myenv) - I get nothing.
I am new to Python and did quite a bit googling to figure this out. However, I am not quite sure if it's possible. I tried using the subprocess:
subprocess.call('setenv myenv 4s3', shell=True)
Also tried using os.system:
os.system("setenv myenv 4s3")
So far, I didn't succeed.
You cannot set environment variables from a child process and have them be visible in the parent process. Every process gets its own copy of the environment, and changes do not propagate upwards.
What you could do is have the Python script print the settings it wants to change and have the outside shell execute the appropriate commands.
Maybe if you find some equivalent function like c vfork for Python.
When you vfork, both processes share memory space so, you might overwrite environment variables in parent process from child process.
Warning: vfork has many security issues, and therefore not recommended. Just use it if you are desperate.
Related
From what I've read, any changes to the environment variables in a Python instance are only available within that instance, and disappear once the instance is closed. Is there any way to make them stick by committing them to the system?
The reason I need to do this is because at the studio where I work, tools like Maya rely heavily on environment variables to configure paths across multiple platforms.
My test code is
import os
os.environ['FAKE'] = 'C:\\'
Opening another instance of Python and requesting os.environ['FAKE'] yields a KeyError.
NOTE: Portability will be an issue, but the small API I'm writing will be able to check OS version and trigger different commands if necessary.
That said, I've gone the route of using the Windows registry technique and will simply write alternative methods that will call shell scripts on other platforms as they become requirements.
You can using SETX at the command-line.
By default these actions go on the USER env vars.
To set and modify SYSTEM vars use the /M flag
import os
env_var = "BUILD_NUMBER"
env_val = "3.1.3.3.7"
os.system("SETX {0} {1} /M".format(env_var,env_val))
make them stick by committing them to
the system?
I think you are a bit confused here. There is no 'system' environment. Each process has their own environment as part its memory. A process can only change its own environment. A process can set the initial environment for processes it creates.
If you really do think you need to set environment variables for the system you will need to look at changing them in the location they get initially loaded from like the registry on windows or your shell configuration file on Linux.
Under Windows it's possible for you to make changes to environment variables persistent via the registry with this recipe, though it seems like overkill.
To echo Brian's question, what are you trying to accomplish? There is probably an easier way.
Seems like there is simplier solution for Windows
import subprocess
subprocess.call(['setx', 'Hello', 'World!'], shell=True)
I don't believe you can do this; there are two work-arounds I can think of.
The os.putenv function sets the environment for processes you start with, i.e. os.system, popen, etc. Depending on what you're trying to do, perhaps you could have one master Python instance that sets the variable, and then spawns new instances.
You could run a shell script or batch file to set it for you, but that becomes much less portable. See this article:
http://code.activestate.com/recipes/159462/
Think about it this way.
You're not setting shell environment variables.
You're spawning a subshell with some given environment variable settings; this subshell runs your application with the modified environment.
According to this discussion, you cannot do it. What are you trying to accomplish?
You are forking a new process and cannot change the environment of the parent process as you cannot do if you start a new shell process from the shell
You might want to try Python Win32 Extensions, developed by Mark Hammond, which is included in the ActivePython (or can be installed separately). You can learn how to perform many Windows related tasks in Hammond's and Robinson's book.
Using PyWin32 to access windows COM objects, a Python program can use the Environment Property (a collection of environment variables) of the WScript.Shell object.
Try to use py-setenv that will allow you to set variable via registry
python -m pip install py-setenv
From within Python? No, it can't be done!
If you are not bound to Python, you should consider using shell scripts (sh, bash, etc). The "source" command allows you to run a script that modifies the environment and will "stick" like you want to the shell you "sourced" the script in. What's going on here is that the shell executes the script directly rather creating a sub-process to execute the script.
This will be quite portable - you can use cygwin on windows to do this.
In case someone might need this info. I realize this was asked 7 yrs ago, but even I forget how sometimes. .
Yes there is a way to make them "stick" in windows. Simply go control panel, system, advanced system settings,when the system properties window opens you should see an option (button) for Environment Variables. .The process for getting to this is a little different depending on what OS you're using (google it).
Choose that (click button), then the Environment Variables window will open. It has 2 split windows, the top one should be your "User Variables For yourusername". . .choose "new", then simply set the variable. For instance one of mine is "Database_Password = mypassword".
Then in your app you can access them like this: import os, os.environ.get('Database_Password'). You can do something like pass = os.environ.get('Database_Password').
We have a tool that comes with a shell script which sets up the environment variables necessary for running the tool. It's fairly convoluted chain of different scripts that determine a bunch of stuff and export/set the env.
We then need that environment every time we want to call the tool itself.
Ideally we would be able to do something like this:
completed_script = subprocess.run("the_settings_script.bat")
[...]
subprocess.run(["some", "other", "call"], env=completed_script.env)
That doesn't work obviously. Is there another nice way to get back the environment after running a subprocess? We could of course run the script in every subprocess.run() call before the actual tool call, but that is kind of inefficient.
No there is no portable way. In any modern OS the parent environment is passed to child processes, but in no way a child can change its parent environment. It used to be possible in good old MS/DOS and only with .com type programs because the address of the parent environment was stored at a well known address in the child process but I know no such tricks for Windows or any Unix-like system.
Here the best way it to setup the environment before starting the Python interpretor. That way the changed environmnent will be passed to all the subprocesses.
I have some instrument which requires environment variable which I want to set automatically from python code. So I tried several ways to make it happen, but none of them were successful.
Here are some examples:
I insert following code in my python script
import os
os.system("export ENV_VAR=/some_path")
I created bash script(env.sh) and run it from python:
#!/bin/bash
export ENV_VAR=some_path
#call it from python
os.system("source env.sh")
I also tried os.putenv() and os.environ*["ENV_VAR"] = "some_path"
Is it possible to set(export) environment variable using python, i.e
without directly exporting it to shell?
Setting an environment variable sets it only for the current process and any child processes it launches. So using os.system will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable. Setting it using os.putenv or os.environ has a similar effect; the environment variables are set for the Python process and any children of it.
I assume you are trying to have those variables set for the shell that you launch the script from, or globally. That can't work because the shell (or other process) is not a child of the Python script in which you are setting the variable.
You'll have better luck setting the variables in a shell script. If you then source that script (so that it runs in the current instance of the shell, rather than in a subshell) then they will remain set after the script ends.
As long as you start the "instrument" (a script I suppose) from the very same process it should work:
In [1]: os.putenv("VARIABLE", "123")
In [2]: os.system("echo $VARIABLE")
123
You can't change an environment variable of a different process or a parent process.
A shell function may do this. You need to print your export statement and eval that.
set_shell_env() {
output=$(python print_export_env.py $*)
eval $output
}
Depending on how you execute your instrument, you might be able to change environment specifically for the child process without affecting the parent. See documentation for os.spawn*e or subprocess.Popen which accept separate argument denoting child environment. For example, Replacing the os.spawn family in subprocess module documentation which provides both usages:
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
subprocess.run('export FOO=BAR', shell=True)
This simply doesn't work, and I have no idea why.
All I am trying to do I set an environment variable from my python (3.5.1) script, and when I run the above line, nothing happens. No errors are raised, and when I check the environment variable myself, it has not been set.
Other shell commands with subprocess.run() do work, such as ls and pwd, but not export.
.run() was added in Python 3.5 (in case you didn't recognise it), but I have also tried the above line with .call() and .Popen(), with no change in results.
I am aware that I can set environment variables in python with os.environ['FOO'] = "BAR", but I will be using shell commands a lot in my project, and I expect that I will need to string multiple commands together, which will make using export easier than os.environ.
My project will run on Linux, which is what my machine is running on.
It works fine; however, the variable setting only exists in the subprocess. You cannot affect the environment of the local process from a child.
os.environ is the correct solution, as it changes the environment of the local process, and those changes will be inherited by any process started with subprocess.run.
You can also use the env argument to run:
subprocess.run(["cmdname", "arg1", "arg number 2"], env=dict(FOO='BAR', **os.environ))
This runs the command in a modified environment that includes FOO=BAR without modifying the current environment.
I need to source the environment of child process. I have a c-shell script(really complicated) that sets many environment variables and I want to use them in the parent process. I doing something like this:
subprocess.call(['set_env_vars.csh; env>crazy_vars.log' shell=True])
In this way I am trying to get the env of child process but this method is not working as I think commands after semicolon are treated as separate processes.
A possible solution is that I created another c-shell script and put those two commands in there and then call that script in python but thats a dirty way.
Is there a way to make two commands as the part of same process.
Thanks
On my system (as with many others) the shell is bash, not csh, so explicitly invoking csh is a good idea. Also, you need to source, not execute set_env_vars.csh:
subprocess.call(['/bin/csh', '-c', 'source set_env_vars.csh; env > crazy_vars.log'])