My Python script is trying to setup a Windows system, including the value of the environment variable PATH.
Windows appears to have two sets of places where it stores environment variables - "User variables" and "System variables". (To see what I mean in Win10, type "environment variables" into the Start menu, then click on the "Environment Variables..." button at the bottom right of that dialog.)
The PATH as seen by programs (example: os.environ in Python) appears to be a concatenation of the PATH system variable and the PATH user variable.
I'm trying to setup the User version of PATH, without affecting the System version.
My problem: When I use os.environ('PATH') = value, the PATH available to Python becomes the string in value only (what I meant for the User version of PATH), and the contents of the System PATH disappears. Then the rest of my Python script can't find the executables it needs.
I can think of several hacky workaounds, but what's the proper way to deal with this?
FWIW, here's what I use to set environment variables (including PATH) now:
def setEnvironmentVariable(name, value):
'''Set an environment variable in both current and permanent environment '''
os.environ[name] = value
os.system('set ' + name + ' = ' + value) # for this shell
os.system('setx ' + name + ' "' + value + '"') # for the permanent user environment
(Note os.system is using cmd.exe; I'm OK with using Powershell if that'll help.)
Related
I am looking to permanently modify the path variable inside windows from a python script. The script is a wrapper to help automate the installation of applications and I want to enable the API to add applications to the path.
So for example I want to install a program called micro which has an install path of C:\Users\USERNAME\Path\to\micro and then add that install path to my path variable so that I can just run micro in my terminal.
I've been made aware of 2 possible solutions, which both do not work:
1. Using os.environ
In python the os module let's you read environment variables, but not actually modify them. so for example:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
new_path = f"{os.environ['PATH']};{program_path}"
os.environ["PATH"] = new_path
This would update the path variable in the python script, but it does not actually modify it on the system which is what I want.
2. setx
I was made aware that it is possible to update your path using the setx command in windows, but for some reason on windows 10 this destroys your path variable.
The idea is that you can call the setx command from python and use it to update the path variable. You should be able to type setx path "%path%;C:\Users\USERNAME\Path\to\micro" and have it update correctly.
So for example, in python code that would be:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
subprocess.Popen(f'setx path "%path%;{program_path}"')
This should take the current path variable and append the program path to it, but instead it just wipes your entire path and replaces it with a literal %path% and then the program path.
So now my path looks like this:
%path%
C:\Users\USERNAME\Path\to\micro
Any ideas on how to get this to work would be appreciated.
Okay, so after a long (and disgusting) amount of research I found a solution. Here is the method I came up with for a cross-platform system of adding to PATH variable:
def add_to_path(program_path:str):
"""Takes in a path to a program and adds it to the system path"""
if os.name == "nt": # Windows systems
import winreg # Allows access to the windows registry
import ctypes # Allows interface with low-level C API's
with winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) as root: # Get the current user registry
with winreg.OpenKey(root, "Environment", 0, winreg.KEY_ALL_ACCESS) as key: # Go to the environment key
existing_path_value = winreg.EnumValue(key, 3)[1] # Grab the current path value
new_path_value = existing_path_value + program_path + ";" # Takes the current path value and appends the new program path
winreg.SetValueEx(key, "PATH", 0, winreg.REG_EXPAND_SZ, new_path_value) # Updated the path with the updated path
# Tell other processes to update their environment
HWND_BROADCAST = 0xFFFF
WM_SETTINGCHANGE = 0x1A
SMTO_ABORTIFHUNG = 0x0002
result = ctypes.c_long()
SendMessageTimeoutW = ctypes.windll.user32.SendMessageTimeoutW
SendMessageTimeoutW(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u"Environment", SMTO_ABORTIFHUNG, 5000, ctypes.byref(result),)
else: # If system is *nix
with open(f"{os.getenv('HOME')}/.bashrc", "a") as bash_file: # Open bashrc file
bash_file.write(f'\nexport PATH="{program_path}:$PATH"\n') # Add program path to Path variable
os.system(f". {os.getenv('HOME')}/.bashrc") # Update bash source
print(f"Added {program_path} to path, please restart shell for changes to take effect")
Neither are pretty, but it does actually work. You do need to restart running shells for it to take effect, but other than that it's perfect.
To retrieve the contents of the %PATH% variable on a Windows 10 Home edition machine, what values should be used for the key, subkey, and name parameters in the following Python script?
import winreg
def _get_reg_value(key, subkey, name):
"""Return registry value specified by key, subkey, and name.
Environment variables in values of type REG_EXPAND_SZ are expanded
if possible.
"""
key = _winreg.OpenKey(key, subkey)
try:
ret = _winreg.QueryValueEx(key, name)
except WindowsError:
return None
else:
key.Close()
if ret[1] == _winreg.REG_EXPAND_SZ:
return expandvars(ret[0])
else:
return ret[0]
Note that we are not using os.environ here because we need to interact with the permanent path values, not just with the runtime values that are exposed by os.environ.
The code example above is from this link . I am using it to study how to interact with the Windows Registry programmatically using Python.
Failed Attempt:
When I call the above function using the following syntax in a .py file run from Windows CMD, all that is returned is a blank line in Windows CMD, followed by a command prompt. As if nothing has happened.
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r'SYSTEM\CurrentControlSet\Control\Session Manager')
_get_reg_value(key, 'Environment', 'Path')
If Python's winreg library simply wraps the corresponding Windows APIs, then you have to open each component of the key. You can't say:
winreg.OpenKey(winreg::HKEY_LOCAL_MACHINE, "FOO\\BAR")
Instead, you first have to open FOO and then open BAR:
foo_key = winreg.OpenKey(winreg::HKEY_LOCAL_MACHINE, "FOO")
bar_key = winreg.OpenKey(foo_key, "BAR")
This is likely the root of your problem.
But there may be more to it. After making such a change, you're supposed to broadcast a WM_SETTINGCHANGE so that other processes (like the shell) know to invalidate their caches and reread the system settings. It's likely the PowerShell commandlet does that automatically.
There can be other issues, too. For example, if you're running a 32-bit Python on a 64-bit OS, you might run into Registry redirection, depending exactly on which part(s) of the hive you're trying to access.
To clarify, what you're calling the "permanent path values" is part of the system environment variables. If you have privileges (e.g., running as Administrator), you are correct that you can modify the path from the system environment block at those keys.
The system merges the user environment variables with the system environment variables. If PATH is defined in both environments, the final PATH is the concatenation of the system values followed by the user values. For other variables with both a system and a user definition (e.g., TMP), the user values are used.
Child processes inherit a copy of their parent process's environment. So, even with a WM_SETTINGCHANGE broadcast, they probably won't update their path to reflect changes you've made to the system environment block.
I've written a program to add directories to the PATH variable via the registry, either the HKCU(user) or HKLM(system) path, depending on an input option.
It works fine when using the User path.
However, when setting the path for the System, Windows acts as if the path variable is empty, e.g.
'notepad' is not recognized as an internal or external command....
However, echo %path% prints everything out appropriately, without any syntax errors. Similarly, if I view the variable in the System Properties GUI, it shows my full path appropriately, e.g.
%SystemRoot%\system32;%SystemRoot%;
Now, if I manually open that variable in the GUI, and add OR remove the trailing semicolon (i.e. make a noticeable but seemingly irrelevant change), then the path seems to work fine.
Yes, I am opening a new command window to check the path. Restarting the machine doesn't seem to do anything either.
Any ideas?
Code excerpt is here:
import _winreg as registry
#HKEY_LOCAL_MACHINE\
SYS_ENV_SUBPATH = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
#HKEY_CURRENT_USER\
USR_ENV_SUBPATH = r"Environment"
def update_reg_path_value(paths_to_add,privilege):
env_key = open_env_registry_key(privilege)
current_path = get_path_from_registry_or_create(env_key)
val_string = create_new_path_value(current_path, paths_to_add)
registry.SetValueEx(env_key,"Path",0,registry.REG_SZ,val_string)
def open_env_registry_key(privilege):
if privilege == 'system':
return registry.OpenKey(registry.HKEY_LOCAL_MACHINE,SYS_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
return registry.OpenKey(registry.HKEY_CURRENT_USER,USR_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
As in the comments, changing REG_SZ to REG_EXPAND_SZ did the trick, as variables using "%" weren't being recognized. This also works when no "%"s exist, so I use it for the user path as well rather than needing to switch between the two.
registry.SetValueEx(env_key,"Path",0,registry.REG_EXPAND_SZ,val_string)
I am trying to set a environmental variable permanently. but temporarily it is working.
if i run below program i got the variable path. after close it and open new terminal to find the variable path using the command printenv LD_LIBRARY_PATH nothing will print.
#!/usr/bin/python
import os
import subprocess
def setenv_var():
env_var = "LD_LIBRARY_PATH"
env_path = "/usr/local/lib"`enter code here`
os.environ[env_var] = env_path
process = subprocess.Popen('printenv ' + env_var, stdout=subprocess.PIPE, shell=True)
result = process.communicate()[0]
return result
if __name__ == '__main__':
print setenv_var()
please help me.
Here is what I use to set environment variables:
def setenv_var(env_file, set_this_env=True):
env_var = "LD_LIBRARY_PATH"
env_path = "/usr/local/lib"`enter code here`
# set environments opened later by appending to `source`-d file
with open(env_file, 'a') as f:
f.write(os.linesep + ("%s=%s" % (env_var, env_path)))
if set_this_end:
# set this environment
os.environ[env_var] = env_path
Now you only have to choose where to set it, that is the first argument in the function. I recommend the profile-specific file ~/.profile or if you're using bash which is pretty common ~/.bashrc
You can also set it globally by using a file like /etc/environment but you'll need to have permissions when you run this script (sudo python script.py).
Remember that environments are inherited from the parent process, and you can't have a child set up a parent process' environment.
When you set an environment variable, it only affects the currently running process (and, by extension, any children that are forked after the variable is set). If you are attempting to set an environment variable in your shell and you want that environment variable to always be set for your interactive shells, you need to set it in the startup scripts (eg .login, .bashrc, .profile) for your shell. Commands that you run are (initially) children of the shell from which you run them, so although they inherit the environment of the shell and can change their own environment, they cannot change the environment of your shell.
Whether you do an export from bash or you set your os.environ from Python, these only stay for the session or process's lifetime. If you want to set them permanent you will have to touch and add it to the respective shell's profile file.
For ex. If you are on bash, you could do:
with open("~/.bashrc", "a") as outfile: # 'a' stands for "append"
outfile.write("export LD_LIBRARY_PATH=/usr/local/lib")
Check this out for some insight as to which file to add this depending on the target shell. https://unix.stackexchange.com/questions/117467/how-to-permanently-set-environmental-variables
I see that if we change the HOME (linux) or USERPROFILE (windows) environmental variable and run a python script, it returns the new value as the user home when I try
os.environ['HOME']
os.exp
Is there any way to find the real user home directory without relying on the environmental variable?
edit:
Here is a way to find userhome in windows by reading in the registry,
http://mail.python.org/pipermail/python-win32/2008-January/006677.html
edit:
One way to find windows home using pywin32,
from win32com.shell import shell,shellcon
home = shell.SHGetFolderPath(0, shellcon.CSIDL_PROFILE, None, 0)
I think os.path.expanduser(path) could be helpful.
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
On Unix, an initial ~ is replaced by the environment variable HOME if it is set; otherwise the current user’s home directory is looked up in the password directory through the built-in module pwd. An initial ~user is looked up directly in the password directory.
On Windows, HOME and USERPROFILE will be used if set, otherwise a combination of HOMEPATH and HOMEDRIVE will be used. An initial ~user is handled by stripping the last directory component from the created user path derived above.
If the expansion fails or if the path does not begin with a tilde, the path is returned unchanged.
So you could just do:
os.path.expanduser('~user')
from pathlib import Path
str(Path.home())
works in Python 3.5 and above. Path.home() returns a Path object providing an API I find very useful.
I think os.path.expanduser(path) is the best answer to your question, but there's an alternative that may be worth mentioning in the Unix world: the pwd package. e.g.
import os, pwd
pwd.getpwuid(os.getuid()).pw_dir
For windows;
import os
homepath = os.path.expanduser(os.getenv('USERPROFILE'))
will give you a handle to current user's home directory and
filepath = os.path.expanduser(os.getenv('USERPROFILE'))+'\\Documents\\myfile.txt'
will give you a handle to below file;
C:\Users\urUserName\Documents\myfile.txt
home_folder = os.getenv('HOME')
This should work on Windows and Mac OS too, works well on Linux.
Really, a change in environment variable indicates that the home must be changed. So every program/script should have the new home in context; also the consequences are up to the person who changed it.
I would still stick with
home = os.getenv('USERPROFILE') or os.getenv('HOME')
what exactly is required?
I realize that this is an old question that has been answered but I thought I would add my two cents. The accepted answer was not working for me. I needed to find the user directory and I wanted it to work with and without sudo. In Linux, my user directory is "/home/someuser" but my root directory is "/root/". However, on my Mac, the user directory is "/Users/someuser". Here is what I ended up doing:
_USERNAME = os.getenv("SUDO_USER") or os.getenv("USER")
_HOME = os.path.expanduser('~'+_USERNAME)
This worked both with and without sudo on Mac and Linux.
get (translated) user folder names on Linux:
from gi.repository import GLib
docs = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOCUMENTS)
desktop = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DESKTOP)
pics = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PICTURES)
videos = GLib.get_user_special_dir(GLib.USER_DIRECTORY_VIDEOS)
music = GLib.get_user_special_dir(GLib.USER_DIRECTORY_MUSIC)
downloads = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOWNLOAD)
public = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PUBLIC_SHARE)
templates = GLib.get_user_special_dir(GLib.USER_DIRECTORY_TEMPLATES)
print(docs)
print(desktop)
print(pics)
print(videos)
print(music)
print(downloads)
print(public)
print(templates)
On Linux and other UNIXoids you can always take a peek in /etc/passwd. The home directory is the sixth colon-separated field in there. No idea on how to do better than the environment variable on Windows though. There'll be a system call for it, but if it's available from Python, ...