I am aware that you can use os.environ['key'] to access an environment variable, but it appears that this function is localized to environment variables for the signed in user. I need to access the system environment variables. Is this possible?
You cannot do this using os.environ in python. You can, however, use os.system to run a commands such as:
sudo -Hiu $user env | grep $var
to access a variable from another user. This is a bit hacky, but it works. And your options are limited here.
Also -- why do you want to? You should have a process for setting the environment variables before this script is run at all, ideally.
Related
Airflow is returning an error when trying to run a DAG saying that it can't find an environment variable, which is odd because it's able to find 3 other environment variables that I'm storing as a Python variable. No issues with those variables at all.
I have all 4 variables in ~/.profile and have also done
export var1="varirable1"
export var2="varirable2"
export var3="varirable3"
export var4="varirable4"
Under what user does airflow run? I've done those export commands under sudo as well, so I thought they would be picked up by airflow when it runs the dag
Is it maybe because airflow uses non-login shell? Have you tried putting these lines in : ~/.bashrc instead of ~/.profile ?
As per this answer, the variables should be put in /etc/default/airflow (on Debian/Ubuntu) or /etc/sysconfig/airflow (on Centos/Redhat).
If you are just running a local instance you should be able to use environment variables like you expect. Remember that you need to set them in the shell that is running the webserver and scheduler though. If these are in your .profile, you may need to run source ~/.profile.
If I execute print os.environ without pkexec I get lots of useful system environment variables, however, once I execute with, most of them are gone. How can I get them back with using pkexec? I understand there's a workaround using sudo but haven't found one yet for pkexec
In short, you can't. From the pkexec man page:
The environment that PROGRAM will run in will be set to a minimal known and safe environment in order to avoid injecting code through LD_LIBRARY_PATH or similar mechanisms. In addition the PKEXEC_UID environment variable is set to the user id of the process invoking pkexec.
You can cause pkexec to retain some environmental variables (i.e. to allow X11 programs to work by retaining $DISPLAY and $XAUTHORITY) using the org.freedesktop.policykit.exec.allow_gui annotation. However, retaining all of the environmental variables appears to be a deliberate rejected decision.
If you only need to keep a few environment variables, you can always preserve them using env:
pkexec bash -c 'echo $PATH'
/usr/sbin:/usr/bin:/sbin:/bin:/root/bin
pkexec env PATH=$PATH bash -c 'echo $PATH'
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
I got a bunch of environment variables set with the launchctl command:
launchctl setenv TEST /Users/JohnDoe/Test
To get the value back, I can use:
launchctl getenv TEST
However, I can't access the value of TEST from Python, using os.getenv('TEST') (or even from Bash using echo $TEST). I do know how macOS manages environment variables (the difference between launchctl and Bash environment variables, etc.) so I understand why those commands don't return the value of TEST.
My question is: is there a way to access environment variables set with launchctl, without using subprocess? Using subprocess is not a no-go, I'd just rather avoid throwing lots of processes just to get environment variables :)
As I'm continuing to work in docker-machine and Django, I'm trying to make a setup script for my project that auto-detects platform and decides how to set up Docker and the required containers. Auto-detection works fine. One thing I can't figure out is how to automatically set the environment variables needed for docker-machine to work on Mac OS X. Currently, the script will just tell the user to manually set the environment variable using the command
eval $(docker-machine env dev)
where dev is the name of the VM. This prompt happens after initial setup is successfully completed. The user is told to do this because the following subprocess call does not actually set the environment variables:
subprocess.call('eval $(docker-machine env dev)', shell=True)
If an error occurs during creating the VM because the VM already exists, then I use subprocess to see if Docker is already installed:
check_docker = subprocess.check_call('docker run hello-world', shell=True)
If this call is successful, then the script tells the user that Docker was already installed and then prompts the user to manually set the environment variables to be able to start the containers needed for the Django server to run. I had originally thought that the script behaved correctly in this scenario, but it turns out that it only appeared that way because I had already set the environment variables manually. Of course, I see now that the docker run command needs the environment variables to be set in order to work, and since the environment variables never get set in the script, the docker run test doesn't work. So, how am I supposed to correctly set the environment variables from Python? It seems like using subprocess is resulting in the wrong environment getting these variables set. If I do something like
subprocess.call('setdockerenv.sh', shell=True)
where setdockerenv.sh has the correct eval command, then I run into the same problem, which I'm guessing is rooted in using subprocess. Would os have something to do this properly where subprocess can't? It's important that I do this in the Python script, or else having the user manually set the environment variables and then manually test to see if docker is installed defeats the purpose of having the script.
You cannot use subprocess to change the environment, since any changes it makes are local to that process. Instead, (as you found) you can change your current environment via os.environ, and that is inherited by any other processes you subsequently create.
I have a .sh file that locally sets some environment variables in my shell source my_env.sh.
#!/bin/sh
echo "Setting up local environment variables"
export MY_URL="http://domain.com"
echo "Done"
This only sets the variables for that session. Which means that my Python Celery and Supervisor apps which run under a different session cannot access them. I understand I can run them under a user, but I want to know if on Ubuntu using the same shell script above if I can set the variables so they are globally accessible to all applications regardless of users or session?
export your variables in the "/etc/profile".
NOTE: This will make it global for each shell sessions for any user. If you wish to set this variable for every session for a specific user, set it in that user's "~/.profile".
According to Ubuntu env variables doc the best way would be
A suitable file for environment variable settings that affect the system as a whole (rather than just a particular user) is /etc/environment
That's assuming you don't mind having them set for the whole machine.
If you are using bashrc or zshrc you can source the shell file which sets the environment variables across the sessions or precisely loads all the environment variables in each session.
source location/shell-script-sets-env.sh
The zshrc and bashrc is available in your $HOME directory. or ~/.zshrc and ~/.bashrc. The current shell can be looked via
echo $SHELL
Have a look at setting env permanently for adding it to the profile.