I have a bat file that sets some environment variables such as
#echo off
SET MY_ENV_VAR=C:\temp
I would like to run this bat file via Python and run other executables that depend on this environment variable bat sets. But even if the bat file runs, I cannot see the environment variables via Python
subprocess.call(['path_to_bat_file\file.bat'], shell = False)
print(os.environ['MY_ENV_VAR'])
I tried to set Shell to True and add other parameters I found on the internet but nothing was successfull. It gives KeyError on os.environ that MY_ENV_VAR is not found. When I run the bat file manually before running the python script, everything works as expected.
Any help is appreciated.
Thank you,
There is no way to change your environment from a child process. The end :)
But you can change the environment variable from within a script like,
import os
os.environ["MY_ENV_VAR"] = "C:\temp"
Related
I had no .bash_profile file so I created one and the only line in it is this:
YOUTUBE_API=someRandomString
In my .zshrc file the first line is this:
source ~/.bash_profile
And from the command line I can run this:
echo $YOUTUBE_API
Which gives me a correct output (my API key).
But when I try to do it in Python it returns None:
import os
print(os.environ.get('YOUTUBE_API'))
I'm running python version 3.9.4. Any idea why, and how I may fix it?
Thank you
You need
export YOUTUBE_API=someRandomString
Otherwise, the variable is available to the local shell, but not to any subprocesses.
It only depends on where you run your code from:
if the process the code runs in has the variable set, it shouldn't be None
if you run the code inside another process, you'll get None
Explanation:
Each process ID gets it's own environment, stored in /proc/<pid>/environ file, where its local non-exported variables get stored. this path gets deleted as soon as the process stops.
The only way to make this work is to either export the variable in the shell, before launch the program, or run YOUTUBE_API=someRandomString python <your python file>
I am trying to retrieve the parameters set from the jenkins build into my python script, but am having trouble. I understand the parameters set from here:
Are set as env variables and all I have to do in python is do:
# Env variables
UPDATE_DATA = os.environ.get('update_data')
ALL_BUILDS = os.environ.get('all_builds')
However I am getting None for those values. When I do an echo of those parameters in my jenkins script before my python script runs, I could see them being printed out correctly. However, for some reason python does not see them. If I go manually into a terminal and export a variable and run my python script, it works.. So I'm completely lost here.
Jenkins server is running on linux. Using python 2.7
You can use the boolean variable like this:
Output:
It seems like when I ran the python script in the Jenkins config (not inside a file within my project) like how #souravatta suggested, it found the env variable. So that means the env variable Jenkins is setting, is on a different instance somehow (even though they are on the same computer, same user). I just did a workaround where I wrote the env variables to a file and then just read that file in my python script.
I am not able to run a system call using PyCharm and can't figure out what variables or environment settings to change.
Given this simple script:
import os
cmd = 'ifconfig -a'
os.system(cmd)
...which runs fine at the command line in terminal, yields the following error:
sh: ifconfig: command not found
This is happening with really any process I'm trying to run such as CSVSQL, PSQL, etc.
I have tried: Displaying my python interpreter paths dispayed at the command line, I tried adding them to the PyCharm interpreter paths, to no avail.
There are several other threads out there describing similar problems, but there doesn't seem to be a good solution that I have come across.
I'm running Linux Mint 19, though this works on my Windows installation (PATH output is much different).
My apologies if this is really simple... Thank you!
Run printenv on both Python and terminal, and check the PATH variable. Use os.environ['PATH'] = 'My path' to set it to what you saw on the terminal.
For future issues (That I've run into):
A quick way to check if it's an exported environment variable is to run os.system("/bin/sh -c \"MYCMD\""), and then run the same "/bin/sh -c \"MYCMD\"" string in your terminal. If there's still a problem, then it must be an export (And this is the likely issue).
To resolve this, try printenv in both python and the terminal to see the list of exports. You should see a discrepancy. The format is simple as you can simple copy the output of the terminal's printenv (Which should be a series of declares), and paste it into python so python will get the same variables. Then your "/bin/sh CMD" calls should align.
The wrapped /bin/sh is in case they're running different shells or have different local variables. echo $SHELL can confirm this, at which point you can compare sets and printenvs and copy paste in the same way. Wrapped you only have to compare exports, as that's what get passed to child processes.
Looks like pycharm isn't getting the PATH from your profile or rc. Try giving the absolute path of the command.
import os
cmd = '/sbin/ifconfig -a'
os.system(cmd)
You can also verify your path using following.
print(os.environ['PATH'])
And use following to add your custom path to current env path.
os.environ['PATH'] += ':/sbin'
Running on Ubuntu, I have a python script, written by someone else, which simply runs like this:
python cool_script.py command line args
However, this script needs some environment variables set before it can run. So I created exports.sh located in the same dir as the python script, which looks something like:
export ENV1='foo'
export ENV2='bar'
export ENV3='baz'
so running:
$source exports.sh
$python cool_script.py command line args
works great. However, my case is a bit more complex. I need to run this on several environments, where the values of ENV1,ENV2,ENV3 will have to be different. I also want to do some stuff to the outputs of cool_script.py. So I wrote cool_script_wrapper.py, which looks like this:
# set environment variables
import os
exports_file = os.path.dirname(os.path.realpath(__file__)) + '/exports.sh'
os.system('source %s' % exports_file)
# run cool_script.py
os.system('python cool_script.py command line args')
# do some stuff to the output
...
I was planning to have different exports.sh scripts for each environment I need to run on, always keeping them in the same dir as the main script. Only problem is that this 'source' approach doesn't work. The environment variables are simply not available to cool_script.py.
Searching around Stack Overflow, I understand now that it's not supposed to work, but I didn't find any solution that suits my needs. I don't want to use os.environ with hard-coded values, and I don't want to parse the exports file. I guess I could make my wrapper a bash script rather than a python, but I hope there is a better solution. Any ideas?
The problem is in the source command that attempts to call a bash builtin.
This should work:
import os
os.environ['ENV1'] = "foo"
os.system("python cool_script.py command line args")
compare with this answer:
Calling the "source" command from subprocess.Popen
How do I set, temporarily, the PYTHONPATH environment variable just before executing a Python script?
In *nix, I can do this:
$ PYTHONPATH='.' python scripts/doit.py
In Windows, this syntax does not work, of course. What is the equivalent, though?
How temporarily? If you open a Windows console (cmd.exe), typing:
set PYTHONPATH=.
will change PYTHONPATH for that console only and any child processes created from it. Any python scripts run from this console will use the new PYTHONPATH value. Close the console and the change will be forgotten.
To set and restore an environment variable on Windows' command line requires an unfortunately "somewhat torturous" approach...:
SET SAVE=%PYTHONPATH%
SET PYTHONPATH=.
python scripts/doit.py
SET PYTHONPATH=%SAVE%
You could use a little auxiliary Python script to make it less painful, e.g.
import os
import sys
import subprocess
for i, a in enumerate(sys.argv[1:]):
if '=' not in a: break
name, _, value = a.partition('=')
os.environ[name] = value
sys.exit(subprocess.call(sys.argv[i:]))
to be called as, e.g.,
python withenv.py PYTHONPATH=. python scripts/doit.py
(I've coded it so it works for any subprocess, not just a Python script -- if you only care about Python scripts you could omit the second python in the cal and put 'python' in sys.argv[i-1] in the code, then use sys.argv[i-1:] as the argument for subprocess.call).
In Windows, you can set PYTHONPATH as an environment variable, which has a GUI front end. On most versions of Windows, you can launch by right click on My Computer and right click Properties.
You use SET on Windows:
SET PYTHONPATH=.
python scripts/doit.py
Windows can localize variables within the script. This will save having to set PYTHONPATH before running. It will also help when different scripts require different conflicting PYTHONPATHs.
setlocal
set PYTHONPATH=.
python.exe scripts\doit.py
endlocal
For more info, check MS documentation
setlocal
endlocal