Say I do this in the terminal
TEST="abc"
A python script run after this (same session, variable is definitely still there) raises a KeyError as the key TEST doesn't exist. How do I access this environment variable?
import os
print os.environ["TEST"]
# bash
export TEST=abc
# sh
TEST=abc
export TEST
Make sure to export the variable. By default environment variables are not inherited by child processes. Marking them as exported tells the shell to pass them to its children.
In the terminal, do
export TEST="abc"
Related
I had no .bash_profile file so I created one and the only line in it is this:
YOUTUBE_API=someRandomString
In my .zshrc file the first line is this:
source ~/.bash_profile
And from the command line I can run this:
echo $YOUTUBE_API
Which gives me a correct output (my API key).
But when I try to do it in Python it returns None:
import os
print(os.environ.get('YOUTUBE_API'))
I'm running python version 3.9.4. Any idea why, and how I may fix it?
Thank you
You need
export YOUTUBE_API=someRandomString
Otherwise, the variable is available to the local shell, but not to any subprocesses.
It only depends on where you run your code from:
if the process the code runs in has the variable set, it shouldn't be None
if you run the code inside another process, you'll get None
Explanation:
Each process ID gets it's own environment, stored in /proc/<pid>/environ file, where its local non-exported variables get stored. this path gets deleted as soon as the process stops.
The only way to make this work is to either export the variable in the shell, before launch the program, or run YOUTUBE_API=someRandomString python <your python file>
I am trying to retrieve the parameters set from the jenkins build into my python script, but am having trouble. I understand the parameters set from here:
Are set as env variables and all I have to do in python is do:
# Env variables
UPDATE_DATA = os.environ.get('update_data')
ALL_BUILDS = os.environ.get('all_builds')
However I am getting None for those values. When I do an echo of those parameters in my jenkins script before my python script runs, I could see them being printed out correctly. However, for some reason python does not see them. If I go manually into a terminal and export a variable and run my python script, it works.. So I'm completely lost here.
Jenkins server is running on linux. Using python 2.7
You can use the boolean variable like this:
Output:
It seems like when I ran the python script in the Jenkins config (not inside a file within my project) like how #souravatta suggested, it found the env variable. So that means the env variable Jenkins is setting, is on a different instance somehow (even though they are on the same computer, same user). I just did a workaround where I wrote the env variables to a file and then just read that file in my python script.
I'm trying to remove all passwords and secrets from applications.
To do this, I'm using environment variables. I would like to avoid putting this specific stuff into ~/.bashrc or other built in files. It would be ideal to call a specific script that exports the desired variables.
The reason for this is because the environment variables contain passwords and I'm trying to isolate them outside of all code except the environment scripts that set environment variables.
To summarize the desired flow:
Launch Python app
App spawns process that sets new environment variables
App pulls values from previously set environment variables
App uses those values in further processing
contents of testpy.py
#!/usr/bin/env python
import os
import subprocess
env_file='/tmp/env_file'
db_pass_var='DBPASS'
#execute bash script that sets the env vars
subprocess.call([".", env_file], shell=True)
#try to get the variable set and print it
print os.getenv(db_pass, 'fail')
contents of /tmp/env_file
#!/usr/bin/env bash
export DBUSER="myuser"
export DBPASS="mypass"
I've tried various methods like subprocess.Popen commands I found on other stackoverflow threads but nothing seems to work.
Alternatives that I want to avoid:
Running the env_file script first, then launching the python code. The goal is to run it all from 1 application.
Putting the passwords into a config file. The goal is environment variables. Config files I would like to reserve to pointing to what variable to call, rather than setting the env vars themselves. This is because config files are checked into source control. The only thing I want to leave out of source control is the actual export environment variables themselves which contain all sensitive environment info.
There's an easier way. Just provide a wrapper script that sets the environment variables and then calls your main script.
It's convenient to use the envdir utility for this. If you can't use that for some reason, then set env vars directly using os.environ and spawn off your main script as a child process.
I have some instrument which requires environment variable which I want to set automatically from python code. So I tried several ways to make it happen, but none of them were successful.
Here are some examples:
I insert following code in my python script
import os
os.system("export ENV_VAR=/some_path")
I created bash script(env.sh) and run it from python:
#!/bin/bash
export ENV_VAR=some_path
#call it from python
os.system("source env.sh")
I also tried os.putenv() and os.environ*["ENV_VAR"] = "some_path"
Is it possible to set(export) environment variable using python, i.e
without directly exporting it to shell?
Setting an environment variable sets it only for the current process and any child processes it launches. So using os.system will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable. Setting it using os.putenv or os.environ has a similar effect; the environment variables are set for the Python process and any children of it.
I assume you are trying to have those variables set for the shell that you launch the script from, or globally. That can't work because the shell (or other process) is not a child of the Python script in which you are setting the variable.
You'll have better luck setting the variables in a shell script. If you then source that script (so that it runs in the current instance of the shell, rather than in a subshell) then they will remain set after the script ends.
As long as you start the "instrument" (a script I suppose) from the very same process it should work:
In [1]: os.putenv("VARIABLE", "123")
In [2]: os.system("echo $VARIABLE")
123
You can't change an environment variable of a different process or a parent process.
A shell function may do this. You need to print your export statement and eval that.
set_shell_env() {
output=$(python print_export_env.py $*)
eval $output
}
Depending on how you execute your instrument, you might be able to change environment specifically for the child process without affecting the parent. See documentation for os.spawn*e or subprocess.Popen which accept separate argument denoting child environment. For example, Replacing the os.spawn family in subprocess module documentation which provides both usages:
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
How do I set, temporarily, the PYTHONPATH environment variable just before executing a Python script?
In *nix, I can do this:
$ PYTHONPATH='.' python scripts/doit.py
In Windows, this syntax does not work, of course. What is the equivalent, though?
How temporarily? If you open a Windows console (cmd.exe), typing:
set PYTHONPATH=.
will change PYTHONPATH for that console only and any child processes created from it. Any python scripts run from this console will use the new PYTHONPATH value. Close the console and the change will be forgotten.
To set and restore an environment variable on Windows' command line requires an unfortunately "somewhat torturous" approach...:
SET SAVE=%PYTHONPATH%
SET PYTHONPATH=.
python scripts/doit.py
SET PYTHONPATH=%SAVE%
You could use a little auxiliary Python script to make it less painful, e.g.
import os
import sys
import subprocess
for i, a in enumerate(sys.argv[1:]):
if '=' not in a: break
name, _, value = a.partition('=')
os.environ[name] = value
sys.exit(subprocess.call(sys.argv[i:]))
to be called as, e.g.,
python withenv.py PYTHONPATH=. python scripts/doit.py
(I've coded it so it works for any subprocess, not just a Python script -- if you only care about Python scripts you could omit the second python in the cal and put 'python' in sys.argv[i-1] in the code, then use sys.argv[i-1:] as the argument for subprocess.call).
In Windows, you can set PYTHONPATH as an environment variable, which has a GUI front end. On most versions of Windows, you can launch by right click on My Computer and right click Properties.
You use SET on Windows:
SET PYTHONPATH=.
python scripts/doit.py
Windows can localize variables within the script. This will save having to set PYTHONPATH before running. It will also help when different scripts require different conflicting PYTHONPATHs.
setlocal
set PYTHONPATH=.
python.exe scripts\doit.py
endlocal
For more info, check MS documentation
setlocal
endlocal