I'm trying to remove all passwords and secrets from applications.
To do this, I'm using environment variables. I would like to avoid putting this specific stuff into ~/.bashrc or other built in files. It would be ideal to call a specific script that exports the desired variables.
The reason for this is because the environment variables contain passwords and I'm trying to isolate them outside of all code except the environment scripts that set environment variables.
To summarize the desired flow:
Launch Python app
App spawns process that sets new environment variables
App pulls values from previously set environment variables
App uses those values in further processing
contents of testpy.py
#!/usr/bin/env python
import os
import subprocess
env_file='/tmp/env_file'
db_pass_var='DBPASS'
#execute bash script that sets the env vars
subprocess.call([".", env_file], shell=True)
#try to get the variable set and print it
print os.getenv(db_pass, 'fail')
contents of /tmp/env_file
#!/usr/bin/env bash
export DBUSER="myuser"
export DBPASS="mypass"
I've tried various methods like subprocess.Popen commands I found on other stackoverflow threads but nothing seems to work.
Alternatives that I want to avoid:
Running the env_file script first, then launching the python code. The goal is to run it all from 1 application.
Putting the passwords into a config file. The goal is environment variables. Config files I would like to reserve to pointing to what variable to call, rather than setting the env vars themselves. This is because config files are checked into source control. The only thing I want to leave out of source control is the actual export environment variables themselves which contain all sensitive environment info.
There's an easier way. Just provide a wrapper script that sets the environment variables and then calls your main script.
It's convenient to use the envdir utility for this. If you can't use that for some reason, then set env vars directly using os.environ and spawn off your main script as a child process.
Related
I have a shell script which exports some credentials.
BHUKRK848D:~ ranip.hore$ cat cred.ksh
export ORACLE_UID_MD="abcd"
export ORACLE_PWD_MD="welcome"
I am trying to execute the shell script from a python script using subprocess and on trying to get the exported variables using os.environ from the python script, I am unable to retrieve them.
Below is the code snippet I am executing :
BHUKRK848D:~ ranip.hore$ cat demo.py
import os
import subprocess
subprocess.call("sh cred.ksh",shell=True)
print(os.environ.get("ORACLE_UID_MD",None))
BHUKRK848D:~ ranip.hore$ python demo.py
None
Is it due to the reason that the variables are exported in a different process and from script while fetching it is trying from some separate process? If i execute the export commands manually from terminal and run the python command it is able to fetch the credentials.
Environment variables are inherited downwards by subprocess, but it doesn't propagate upwards to the parent process. Setting environment variables in a subprocess don't affect the environment variables of the parent process.
You'll have to parse the environment variables from the file instead of executing it.
I have some instrument which requires environment variable which I want to set automatically from python code. So I tried several ways to make it happen, but none of them were successful.
Here are some examples:
I insert following code in my python script
import os
os.system("export ENV_VAR=/some_path")
I created bash script(env.sh) and run it from python:
#!/bin/bash
export ENV_VAR=some_path
#call it from python
os.system("source env.sh")
I also tried os.putenv() and os.environ*["ENV_VAR"] = "some_path"
Is it possible to set(export) environment variable using python, i.e
without directly exporting it to shell?
Setting an environment variable sets it only for the current process and any child processes it launches. So using os.system will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable. Setting it using os.putenv or os.environ has a similar effect; the environment variables are set for the Python process and any children of it.
I assume you are trying to have those variables set for the shell that you launch the script from, or globally. That can't work because the shell (or other process) is not a child of the Python script in which you are setting the variable.
You'll have better luck setting the variables in a shell script. If you then source that script (so that it runs in the current instance of the shell, rather than in a subshell) then they will remain set after the script ends.
As long as you start the "instrument" (a script I suppose) from the very same process it should work:
In [1]: os.putenv("VARIABLE", "123")
In [2]: os.system("echo $VARIABLE")
123
You can't change an environment variable of a different process or a parent process.
A shell function may do this. You need to print your export statement and eval that.
set_shell_env() {
output=$(python print_export_env.py $*)
eval $output
}
Depending on how you execute your instrument, you might be able to change environment specifically for the child process without affecting the parent. See documentation for os.spawn*e or subprocess.Popen which accept separate argument denoting child environment. For example, Replacing the os.spawn family in subprocess module documentation which provides both usages:
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
I want to write a python script to get list of "jenkins" variables and show them.
I wrote this, but it returned all environment variables instead of jenkins variables.
import os
print os.environ
How can I get them in python script?
Jenkins variables are environment variables. There isn't really anything that separates them.
You can use EnvInject plugin to clean the Jenkins environment of all OS environment variables that are inherited, this way you are left with "just Jenkins environment variables", but careful as this may break your tools as none of the paths will be set.
If you want to view just the Build Parameters, you can use the Jenkins API for each job, or parse the job's XML file
There is a specific set of built-in variable that a base Jenkins install adds to each execution step; here is a list from the reference:
https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project#Buildingasoftwareproject-below
As Slav says, these are just like any other environment variable, and if you want to access one you would just ask for it by name e.g., for $WORKSPACE, you would use os.environ['WORKSPACE']
You could enumerate the variables at the url above in a list ['WORKSPACE', ...] and print them that way...
I have an application that has environment variables which I cannot control and are modified from the "default" environment variables applications are given when started from cmd or explore (or whatever). My application allows me to run vbscripts, but these scripts take the parent application's environmental variables, which is okay. I would like to use the vbscript to start python with the "normal"/"default" environment variables most application get when started from cmd or explorer (or whatever). I've attempted to use the vbscript to call a bat file that runs python, however python is still maintaining the environment variables of its grand-grand-parent.
How can I get a vbscript to call application (python) without passing its "modified" environment variables to its child? I've also tried using start /i, but because the vbscript's parent modified its variables before the vbscript started, it won't reset to a clean environment.
My VBScript looks something like this:
sub run
dim wshShell
Set wshShell = CreateObject( "WScript.Shell" )
Dim cmd
cmd = ""
cmd = cmd & chr(34) ' double quote character'
cmd = cmd & "startPython.bat"
cmd = cmd & chr(34)
wshShell.run( cmd )
end sub
My application will start the run subroutine with modified undesirable environment variables.
The startPython.bat file looks like this:
start /i python "python3file.py" %*
rem pause
The batch file is not required, but seems like possible point that the environment chain could be broken, but it does not seem to.
In the end, I'd like to have the vbscript start python using the "default"/"normal" environment variables that new application started by the user via cmd or explorer would be given (I'm not picky about how old they are, or if they are the OS startup variables, or ones that are slightly modified during startup before any cmd or explorer, or user startup. These modifications are acceptable, however modifications made by my parent application should be gone.
(It's also important that the session environment variables of the parent DO NOT change, however after pages of stackoverflow and other readings, that seems to be difficult (and non-recommended) anyways.)
(I understand their isn't a "default" or "normal" set of environment variables, but there exists system level and user level, either of which would be acceptable in my situation, as long as I'm not using the session variables created by my parent application. Acceptable solutions would also include "stealing" a copy of the environment variables, but a fresh set of OS generated environment variables would be most preferred (either of the system level or user level type).)
An ideal solution would be limited to vbscript (optionally via bat files) to start python with clean set of environment variables.
EDIT Additional updates:
Based on Harry Johnston suggestions:
I tried using ShellExecute by modifying the VBScript to look something like:
dim objShell
set objShell = CreateObject("shell.application")
objShell.ShellExecute cmd , "" , "blahBlahBlah"
set objShell = nothing
Based on the msdn link suggestion, but I'm setting getting a modified environmental variable session.
(I also removed /i then start from the batch script file in case they were getting the parents session variables.)
You want the ShellExecute method of the shell.application object.
This causes the Windows shell (aka Windows Explorer, more or less) to open the specified application or file on your behalf. That means the newly launched application gets a fresh set of environment variables, just as if you'd launched it by double-clicking in Explorer.
Inheritance of environment variables is a built-in concept of Windows. However, anyone calling the CreateProcess() function can specify the environment variables for that process (lpEnvironment). Problem is: how to know what the default environment variables are?
The only workaround I see at this time, is to create a task in task scheduler and having the command executed by task scheduler instead of your own application. This approach assumes that task scheduler is started with an original set of environment variables. Commands you'll need for that:
schtasks /create ...
schtasks /run ...
schtasks /delete ...
Setting it all up is a bit tricky. I can't tell whether that works, but maybe you can give it a try.
I want to run a csh file from a python script,
example,
#!/usr/bin/python
import os
os.system("source path/to/file.csh")
and I want this file to run in the same shell as I am running the python script, because the file.csh script is settings some environment variables that I need.
Does anyone know how to do this in Python?
A child process cannot affect the environment of the parent process. The best you can do is to run your csh script in a separate process, get the environment variables that it defines, then set each environment variable in your python script.
Even with that, the python script won't be able to affect the shell in which you run the python script.
The common way to solve this (AFAIK) is to have your script emit shell commands to set the environment, then from the main shell you run the script and eval what you get back.
For more information you might want to check out this question: can a shell script set environment variables of the calling shell
You can kludge it this way:
#!/usr/bin/env python
# This is kludge.py
print "setenv VARNAME \"the value\""
In your case, you can have the file.sh print the setenv line.
Then from csh:
$ eval `./kludge.py`
$ echo $VARNAME
the value
This isn't clean, but it is the only way to have a child process effect the environment of its parent. This is only because the parent process is explicitly letting it happen with eval.