From what I've read, any changes to the environment variables in a Python instance are only available within that instance, and disappear once the instance is closed. Is there any way to make them stick by committing them to the system?
The reason I need to do this is because at the studio where I work, tools like Maya rely heavily on environment variables to configure paths across multiple platforms.
My test code is
import os
os.environ['FAKE'] = 'C:\\'
Opening another instance of Python and requesting os.environ['FAKE'] yields a KeyError.
NOTE: Portability will be an issue, but the small API I'm writing will be able to check OS version and trigger different commands if necessary.
That said, I've gone the route of using the Windows registry technique and will simply write alternative methods that will call shell scripts on other platforms as they become requirements.
You can using SETX at the command-line.
By default these actions go on the USER env vars.
To set and modify SYSTEM vars use the /M flag
import os
env_var = "BUILD_NUMBER"
env_val = "3.1.3.3.7"
os.system("SETX {0} {1} /M".format(env_var,env_val))
make them stick by committing them to
the system?
I think you are a bit confused here. There is no 'system' environment. Each process has their own environment as part its memory. A process can only change its own environment. A process can set the initial environment for processes it creates.
If you really do think you need to set environment variables for the system you will need to look at changing them in the location they get initially loaded from like the registry on windows or your shell configuration file on Linux.
Under Windows it's possible for you to make changes to environment variables persistent via the registry with this recipe, though it seems like overkill.
To echo Brian's question, what are you trying to accomplish? There is probably an easier way.
Seems like there is simplier solution for Windows
import subprocess
subprocess.call(['setx', 'Hello', 'World!'], shell=True)
I don't believe you can do this; there are two work-arounds I can think of.
The os.putenv function sets the environment for processes you start with, i.e. os.system, popen, etc. Depending on what you're trying to do, perhaps you could have one master Python instance that sets the variable, and then spawns new instances.
You could run a shell script or batch file to set it for you, but that becomes much less portable. See this article:
http://code.activestate.com/recipes/159462/
Think about it this way.
You're not setting shell environment variables.
You're spawning a subshell with some given environment variable settings; this subshell runs your application with the modified environment.
According to this discussion, you cannot do it. What are you trying to accomplish?
You are forking a new process and cannot change the environment of the parent process as you cannot do if you start a new shell process from the shell
You might want to try Python Win32 Extensions, developed by Mark Hammond, which is included in the ActivePython (or can be installed separately). You can learn how to perform many Windows related tasks in Hammond's and Robinson's book.
Using PyWin32 to access windows COM objects, a Python program can use the Environment Property (a collection of environment variables) of the WScript.Shell object.
Try to use py-setenv that will allow you to set variable via registry
python -m pip install py-setenv
From within Python? No, it can't be done!
If you are not bound to Python, you should consider using shell scripts (sh, bash, etc). The "source" command allows you to run a script that modifies the environment and will "stick" like you want to the shell you "sourced" the script in. What's going on here is that the shell executes the script directly rather creating a sub-process to execute the script.
This will be quite portable - you can use cygwin on windows to do this.
In case someone might need this info. I realize this was asked 7 yrs ago, but even I forget how sometimes. .
Yes there is a way to make them "stick" in windows. Simply go control panel, system, advanced system settings,when the system properties window opens you should see an option (button) for Environment Variables. .The process for getting to this is a little different depending on what OS you're using (google it).
Choose that (click button), then the Environment Variables window will open. It has 2 split windows, the top one should be your "User Variables For yourusername". . .choose "new", then simply set the variable. For instance one of mine is "Database_Password = mypassword".
Then in your app you can access them like this: import os, os.environ.get('Database_Password'). You can do something like pass = os.environ.get('Database_Password').
i got a python script in my project. Depending on whether it was run manually or via Jenkins, I need to respond to it differently.
The only solution I came up with so far was to set an environment variable via Jenkins and check this var in the script.
Has anyone had a similar problem and could somehow solve it differently ?
If you only want to avoid setting an env variable, but still prefer using an existing env var you might want to use https://plugins.jenkins.io/envinject/#EnvInjectPlugin-BuildCauses which exposes following variables in the Env variables:
BUILD_CAUSE=MANUALTRIGGER
BUILD_CAUSE_MANUALTRIGGER=true in case of manual trigger. It internally checks which cause(s) triggered the build, which can be many like SCMTriggerCause, TimerTriggerCause etc.
I am trying to retrieve the parameters set from the jenkins build into my python script, but am having trouble. I understand the parameters set from here:
Are set as env variables and all I have to do in python is do:
# Env variables
UPDATE_DATA = os.environ.get('update_data')
ALL_BUILDS = os.environ.get('all_builds')
However I am getting None for those values. When I do an echo of those parameters in my jenkins script before my python script runs, I could see them being printed out correctly. However, for some reason python does not see them. If I go manually into a terminal and export a variable and run my python script, it works.. So I'm completely lost here.
Jenkins server is running on linux. Using python 2.7
You can use the boolean variable like this:
Output:
It seems like when I ran the python script in the Jenkins config (not inside a file within my project) like how #souravatta suggested, it found the env variable. So that means the env variable Jenkins is setting, is on a different instance somehow (even though they are on the same computer, same user). I just did a workaround where I wrote the env variables to a file and then just read that file in my python script.
I'm trying to remove all passwords and secrets from applications.
To do this, I'm using environment variables. I would like to avoid putting this specific stuff into ~/.bashrc or other built in files. It would be ideal to call a specific script that exports the desired variables.
The reason for this is because the environment variables contain passwords and I'm trying to isolate them outside of all code except the environment scripts that set environment variables.
To summarize the desired flow:
Launch Python app
App spawns process that sets new environment variables
App pulls values from previously set environment variables
App uses those values in further processing
contents of testpy.py
#!/usr/bin/env python
import os
import subprocess
env_file='/tmp/env_file'
db_pass_var='DBPASS'
#execute bash script that sets the env vars
subprocess.call([".", env_file], shell=True)
#try to get the variable set and print it
print os.getenv(db_pass, 'fail')
contents of /tmp/env_file
#!/usr/bin/env bash
export DBUSER="myuser"
export DBPASS="mypass"
I've tried various methods like subprocess.Popen commands I found on other stackoverflow threads but nothing seems to work.
Alternatives that I want to avoid:
Running the env_file script first, then launching the python code. The goal is to run it all from 1 application.
Putting the passwords into a config file. The goal is environment variables. Config files I would like to reserve to pointing to what variable to call, rather than setting the env vars themselves. This is because config files are checked into source control. The only thing I want to leave out of source control is the actual export environment variables themselves which contain all sensitive environment info.
There's an easier way. Just provide a wrapper script that sets the environment variables and then calls your main script.
It's convenient to use the envdir utility for this. If you can't use that for some reason, then set env vars directly using os.environ and spawn off your main script as a child process.
There are quite a few question for this topic but unfortunatly they didn't solve my problem.
I have a shell script whose purpose is only to set environment variables (let's call it env.sh). A second shell script is the main program script that sources env.sh and is using variables that are defined there. This works fine when both are bash-scripts.
I have the problem that I try to replace the main shell script with a python program. This python program does also depend on those environment variables that are set by env.sh.
What can I do to source env.sh within python before starting any routine that use those environmen variables?
I have to run bash -c source env.sh with the module subprocess. But if I understand correctly this does not work, because the variables are set in the child process of the calling python program and are therefore not available in the parent process. Is this correct?
A similar solution would have been to use bash -c source env.sh && env and read the output of env in python, iterate over the list and write into os.environ. But this approach would write every variable again, even if it was already defined. What if there are a lot of variables?
The last solution I could think of was to read and parse the env.sh file and set the variables I can find. One problem is that some exports in env.sh are nested, mening:
export SOMETHING=$FOO/Bar
This could become complicated to parse correctly.
Is there another way that I haven't found yet or didn't think of?
Some options:
https://github.com/mattseymour/python-env
https://github.com/rconradharris/envparse
Env variable sourcing is very common, I would not recommend rolling your own.