Develop Reference

%USERPROFILE% env variable for python

I am writing a script in Python 2.7.
It needs to be able to go whoever the current users profile in Windows.
This is the variable and function I currently have:
import os
desired_paths = os.path.expanduser('HOME'\"My Documents")
I do have doubts that this expanduser will work though. I tried looking for Windows Env Variables to in Python to hopefully find a list and know what to convert it to. Either such tool doesn't exist or I am just not using the right search terms since I am still pretty new and learning.

You can access environment variables via the os.environ mapping:
import os
print(os.environ['USERPROFILE'])
This will work in Windows. For another OS, you'd need the appropriate environment variable.
Also, the way to concatenate strings in Python is with + signs, so this:
os.path.expanduser('HOME'\"My Documents")
^^^^^^^^^^^^^^^^^^^^^
should probably be something else. But to concatenate paths you should be more careful, and probably want to use something like:
os.sep.join(<your path parts>)
# or
os.path.join(<your path parts>)
(There is a slight distinction between the two)
If you want the My Documents directory of the current user, you might try something like:
docs = os.path.join(os.environ['USERPROFILE'], "My Documents")
Alternatively, using expanduser:
docs = os.path.expanduser(os.sep.join(["~","My Documents"]))
Lastly, to see what environment variables are set, you can do something like:
print(os.environ.keys())
(In reference to finding a list of what environment vars are set)

Going by os.path.expanduser , using a ~ would seem more reliable than using 'HOME'.

bring DYLD_LIBRARY_PATH and/or LD_LIBRARY_PATH into python on OSX [duplicate]

I believe that running an external command with a slightly modified environment is a very common case. That's how I tend to do it:
import subprocess, os
my_env = os.environ
my_env["PATH"] = "/usr/sbin:/sbin:" + my_env["PATH"]
subprocess.Popen(my_command, env=my_env)
I've got a gut feeling that there's a better way; does it look alright?

I think os.environ.copy() is better if you don't intend to modify the os.environ for the current process:
import subprocess, os
my_env = os.environ.copy()
my_env["PATH"] = "/usr/sbin:/sbin:" + my_env["PATH"]
subprocess.Popen(my_command, env=my_env)

That depends on what the issue is. If it's to clone and modify the environment one solution could be:
subprocess.Popen(my_command, env=dict(os.environ, PATH="path"))
But that somewhat depends on that the replaced variables are valid python identifiers, which they most often are (how often do you run into environment variable names that are not alphanumeric+underscore or variables that starts with a number?).
Otherwise you'll could write something like:
subprocess.Popen(my_command, env=dict(os.environ,
**{"Not valid python name":"value"}))
In the very odd case (how often do you use control codes or non-ascii characters in environment variable names?) that the keys of the environment are bytes you can't (on python3) even use that construct.
As you can see the techniques (especially the first) used here benefits on the keys of the environment normally is valid python identifiers, and also known in advance (at coding time), the second approach has issues. In cases where that isn't the case you should probably look for another approach.

With Python 3.5 you could do it this way:
import os
import subprocess
my_env = {**os.environ, 'PATH': '/usr/sbin:/sbin:' + os.environ['PATH']}
subprocess.Popen(my_command, env=my_env)
Here we end up with a copy of os.environ and overridden PATH value.
It was made possible by PEP 448 (Additional Unpacking Generalizations).
Another example. If you have a default environment (i.e. os.environ), and a dict you want to override defaults with, you can express it like this:
my_env = {**os.environ, **dict_with_env_variables}

you might use my_env.get("PATH", '') instead of my_env["PATH"] in case PATH somehow not defined in the original environment, but other than that it looks fine.

To temporarily set an environment variable without having to copy the os.envrion object etc, I do this:
process = subprocess.Popen(['env', 'RSYNC_PASSWORD=foobar', 'rsync', \
'rsync://username#foobar.com::'], stdout=subprocess.PIPE)

The env parameter accepts a dictionary. You can simply take os.environ, add a key (your desired variable) (to a copy of the dict if you must) to that and use it as a parameter to Popen.

I know this has been answered for some time, but there are some points that some may want to know about using PYTHONPATH instead of PATH in their environment variable. I have outlined an explanation of running python scripts with cronjobs that deals with the modified environment in a different way (found here). Thought it would be of some good for those who, like me, needed just a bit more than this answer provided.

In certain circumstances you may want to only pass down the environment variables your subprocess needs, but I think you've got the right idea in general (that's how I do it too).

This could be a solution :
new_env = dict([(k,(':'.join([env[k], v]) if k in env else v)) for k,v in os.environ.items()])

How to set and retrieve environment variable in Python

I need to set a environment variables in the python, and I try the commands bellow
import os
basepath = os.putenv('CPATH','/Users/cat/doc')
basepath = os.getenv('CPATH','/Users/cat/doc')
And when I print the varible, they are not set:
print basepath
None
What i'm doing wrong?
Rephrasing the question, I would like to create a base path based on a evironment variable. I'm testing this:
os.environ["CPATH"] = "/Users/cat/doc"
print os.environ["CPATH"]
base_path=os.getenv('C_PATH')
And when I try to print the basepath:
print basepath
it always return
None

Try this one.
os.environ["CPATH"] = "/Users/cat/doc"
Python documentation is quite clear about it.
Calling putenv() directly does not change os.environ, so it’s better to modify os.environ.
Based on the documentation, the os.getenv() is available on most flavors of Unix and Windows. OS X is not listed.
Use rather the snippet below to retrieve the value.
value = os.environ.get('CPATH')

Use os.environ:
os.environ['CPATH'] = '/Users/cat/doc'
print os.environ['CPATH'] # /Users/cat/doc
print os.environ.get('CPATH') # /Users/cat/doc
See the above link for more details:
If the platform supports the putenv() function, this mapping may be
used to modify the environment as well as query the environment.
putenv() will be called automatically when the mapping is modified.
Note Calling putenv() directly does not change os.environ, so it’s
better to modify os.environ.

It's a good practice to restore the environment variables at function completion.
You may need something like the modified_environ context manager describe in this question to restore the environment variables.
Usage example:
with modified_environ(CPATH='/Users/cat/doc'):
call_my_function()

Using environment variables in Fabric

Assuming:
export TEST=/somewhere
I want to run the command /somewhere/program using:
with cd('$TEST'):
run('program')
However, this doesn't work because the $ gets escaped.
Is there a way to use an environment variable in a Fabric cd() call?

Following suggestion from #AndrewWalker, here is a more compact solution that worked for me (and to my knowledge, the result is the same):
with cd(run("echo $TEST")):
run("program")
But I decided to go for a (very slightly) more concise yet as readable solution:
run('cd $TEST && program')
This second solution, if I am correct, produces the same result.

You can capture the value by using echo
testdir = str(run("echo $TEST"))
with cd(testdir):
run("program")

Alternatively:
import os
def my_task():
with lcd(os.environ['TEST_PATH']):
local('pwd')
os.getenv('TEST_PATH') may also be used (with a default, optionally)
Hat tip: Send bash environment variable back to python fabric

How to get "canonical unix shell" for Python

According to the documentation for the subprocess module, its default shell is /bin/sh, but I have an ingrained, and probably irrational, aversion to hard-coding such constants.
Therefore, I much prefer to refer to some constant defined in subprocess. I have not been able to find any way to interrogate subprocess directly for this constant. The best I've managed is
def _getshpath():
return subprocess.check_output('which sh', shell=True).strip()
or
def _getshpath():
return subprocess.check_output('echo "$0"', shell=True).strip()
...both of which look pathetically fragile, since their validity ultimately depend on precisely the specific value I'm trying to determine in the first place. (I.e., if the value of this executable is not "/bin/sh", either definition could easily be nonsensical.)
What's best-practice for getting this path (without hard-coding it as "/bin/sh")?
Thanks!

Hard-coding it as /bin/sh is perfectly valid. If you look at the documentation for C's popen() you'll find it does this too. /bin/sh is, by construction, the system's standard shell.

You could try
>>> import os
>>> shell = os.environ['SHELL']
>>> print shell
'/bin/bash'
You can use this to set the executable argument of subprocess.Popen.