Is that possible to modify PATH environment variable, globally and permanently, in a platform-independent way using Python (distutils)?
Background
I have some application (a plugin for Serna XML Editor), and now I'm going to make an installer for it, probably using python distutils (setup.py). After the installation setup.py needs to modify the PATH environment variable to add the installation directory to its value.
A possible solution to achieve what I want would be to copy the executables to /usr/local/bin or somewhere else, but for MS Windows it is not obvious where to copy execs.
Any ideas?
As far as I know, distutils has no cross-platform utility for permanently changing environment variables. So you will have to write platform specific code.
In Windows, environment variables are stored in the registry. This is a sample code to read and set some of it's keys. I use only the standard librarie (no need to install pywin32!) to achieve that.
import _winreg as winreg
import ctypes
ENV_HTTP_PROXY = u'http://87.254.212.121:8080'
class Registry(object):
def __init__(self, key_location, key_path):
self.reg_key = winreg.OpenKey(key_location, key_path, 0, winreg.KEY_ALL_ACCESS)
def set_key(self, name, value):
try:
_, reg_type = winreg.QueryValueEx(self.reg_key, name)
except WindowsError:
# If the value does not exists yet, we (guess) use a string as the
# reg_type
reg_type = winreg.REG_SZ
winreg.SetValueEx(self.reg_key, name, 0, reg_type, value)
def delete_key(self, name):
try:
winreg.DeleteValue(self.reg_key, name)
except WindowsError:
# Ignores if the key value doesn't exists
pass
class EnvironmentVariables(Registry):
"""
Configures the HTTP_PROXY environment variable, it's used by the PIP proxy
"""
def __init__(self):
super(EnvironmentVariables, self).__init__(winreg.HKEY_LOCAL_MACHINE,
r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment')
def on(self):
self.set_key('HTTP_PROXY', ENV_HTTP_PROXY)
self.refresh()
def off(self):
self.delete_key('HTTP_PROXY')
self.refresh()
def refresh(self):
HWND_BROADCAST = 0xFFFF
WM_SETTINGCHANGE = 0x1A
SMTO_ABORTIFHUNG = 0x0002
result = ctypes.c_long()
SendMessageTimeoutW = ctypes.windll.user32.SendMessageTimeoutW
SendMessageTimeoutW(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u'Environment', SMTO_ABORTIFHUNG, 5000, ctypes.byref(result));
This is just a sample code for you to get started with, it only implements settings and deleting keys.
Make sure that you always calls the refresh method after changing a registry. This will tell Windows that something has changed and it will refresh the registry settings.
Here is the link for the full application that I wrote, its a proxy switcher for Windows:
https://bitbucket.org/canassa/switch-proxy/
Distutils doesn’t set environment variables. On Windows, this would imply mucking with the registry; on UNIX, it would require to find out the right shell configuration file (which is not trivial) and edit it, which is just not done in this culture: people are told to edit their $PATH or to use full paths to the programs.
Related
I am looking to permanently modify the path variable inside windows from a python script. The script is a wrapper to help automate the installation of applications and I want to enable the API to add applications to the path.
So for example I want to install a program called micro which has an install path of C:\Users\USERNAME\Path\to\micro and then add that install path to my path variable so that I can just run micro in my terminal.
I've been made aware of 2 possible solutions, which both do not work:
1. Using os.environ
In python the os module let's you read environment variables, but not actually modify them. so for example:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
new_path = f"{os.environ['PATH']};{program_path}"
os.environ["PATH"] = new_path
This would update the path variable in the python script, but it does not actually modify it on the system which is what I want.
2. setx
I was made aware that it is possible to update your path using the setx command in windows, but for some reason on windows 10 this destroys your path variable.
The idea is that you can call the setx command from python and use it to update the path variable. You should be able to type setx path "%path%;C:\Users\USERNAME\Path\to\micro" and have it update correctly.
So for example, in python code that would be:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
subprocess.Popen(f'setx path "%path%;{program_path}"')
This should take the current path variable and append the program path to it, but instead it just wipes your entire path and replaces it with a literal %path% and then the program path.
So now my path looks like this:
%path%
C:\Users\USERNAME\Path\to\micro
Any ideas on how to get this to work would be appreciated.
Okay, so after a long (and disgusting) amount of research I found a solution. Here is the method I came up with for a cross-platform system of adding to PATH variable:
def add_to_path(program_path:str):
"""Takes in a path to a program and adds it to the system path"""
if os.name == "nt": # Windows systems
import winreg # Allows access to the windows registry
import ctypes # Allows interface with low-level C API's
with winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) as root: # Get the current user registry
with winreg.OpenKey(root, "Environment", 0, winreg.KEY_ALL_ACCESS) as key: # Go to the environment key
existing_path_value = winreg.EnumValue(key, 3)[1] # Grab the current path value
new_path_value = existing_path_value + program_path + ";" # Takes the current path value and appends the new program path
winreg.SetValueEx(key, "PATH", 0, winreg.REG_EXPAND_SZ, new_path_value) # Updated the path with the updated path
# Tell other processes to update their environment
HWND_BROADCAST = 0xFFFF
WM_SETTINGCHANGE = 0x1A
SMTO_ABORTIFHUNG = 0x0002
result = ctypes.c_long()
SendMessageTimeoutW = ctypes.windll.user32.SendMessageTimeoutW
SendMessageTimeoutW(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u"Environment", SMTO_ABORTIFHUNG, 5000, ctypes.byref(result),)
else: # If system is *nix
with open(f"{os.getenv('HOME')}/.bashrc", "a") as bash_file: # Open bashrc file
bash_file.write(f'\nexport PATH="{program_path}:$PATH"\n') # Add program path to Path variable
os.system(f". {os.getenv('HOME')}/.bashrc") # Update bash source
print(f"Added {program_path} to path, please restart shell for changes to take effect")
Neither are pretty, but it does actually work. You do need to restart running shells for it to take effect, but other than that it's perfect.
To retrieve the contents of the %PATH% variable on a Windows 10 Home edition machine, what values should be used for the key, subkey, and name parameters in the following Python script?
import winreg
def _get_reg_value(key, subkey, name):
"""Return registry value specified by key, subkey, and name.
Environment variables in values of type REG_EXPAND_SZ are expanded
if possible.
"""
key = _winreg.OpenKey(key, subkey)
try:
ret = _winreg.QueryValueEx(key, name)
except WindowsError:
return None
else:
key.Close()
if ret[1] == _winreg.REG_EXPAND_SZ:
return expandvars(ret[0])
else:
return ret[0]
Note that we are not using os.environ here because we need to interact with the permanent path values, not just with the runtime values that are exposed by os.environ.
The code example above is from this link . I am using it to study how to interact with the Windows Registry programmatically using Python.
Failed Attempt:
When I call the above function using the following syntax in a .py file run from Windows CMD, all that is returned is a blank line in Windows CMD, followed by a command prompt. As if nothing has happened.
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r'SYSTEM\CurrentControlSet\Control\Session Manager')
_get_reg_value(key, 'Environment', 'Path')
If Python's winreg library simply wraps the corresponding Windows APIs, then you have to open each component of the key. You can't say:
winreg.OpenKey(winreg::HKEY_LOCAL_MACHINE, "FOO\\BAR")
Instead, you first have to open FOO and then open BAR:
foo_key = winreg.OpenKey(winreg::HKEY_LOCAL_MACHINE, "FOO")
bar_key = winreg.OpenKey(foo_key, "BAR")
This is likely the root of your problem.
But there may be more to it. After making such a change, you're supposed to broadcast a WM_SETTINGCHANGE so that other processes (like the shell) know to invalidate their caches and reread the system settings. It's likely the PowerShell commandlet does that automatically.
There can be other issues, too. For example, if you're running a 32-bit Python on a 64-bit OS, you might run into Registry redirection, depending exactly on which part(s) of the hive you're trying to access.
To clarify, what you're calling the "permanent path values" is part of the system environment variables. If you have privileges (e.g., running as Administrator), you are correct that you can modify the path from the system environment block at those keys.
The system merges the user environment variables with the system environment variables. If PATH is defined in both environments, the final PATH is the concatenation of the system values followed by the user values. For other variables with both a system and a user definition (e.g., TMP), the user values are used.
Child processes inherit a copy of their parent process's environment. So, even with a WM_SETTINGCHANGE broadcast, they probably won't update their path to reflect changes you've made to the system environment block.
I've written a program to add directories to the PATH variable via the registry, either the HKCU(user) or HKLM(system) path, depending on an input option.
It works fine when using the User path.
However, when setting the path for the System, Windows acts as if the path variable is empty, e.g.
'notepad' is not recognized as an internal or external command....
However, echo %path% prints everything out appropriately, without any syntax errors. Similarly, if I view the variable in the System Properties GUI, it shows my full path appropriately, e.g.
%SystemRoot%\system32;%SystemRoot%;
Now, if I manually open that variable in the GUI, and add OR remove the trailing semicolon (i.e. make a noticeable but seemingly irrelevant change), then the path seems to work fine.
Yes, I am opening a new command window to check the path. Restarting the machine doesn't seem to do anything either.
Any ideas?
Code excerpt is here:
import _winreg as registry
#HKEY_LOCAL_MACHINE\
SYS_ENV_SUBPATH = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
#HKEY_CURRENT_USER\
USR_ENV_SUBPATH = r"Environment"
def update_reg_path_value(paths_to_add,privilege):
env_key = open_env_registry_key(privilege)
current_path = get_path_from_registry_or_create(env_key)
val_string = create_new_path_value(current_path, paths_to_add)
registry.SetValueEx(env_key,"Path",0,registry.REG_SZ,val_string)
def open_env_registry_key(privilege):
if privilege == 'system':
return registry.OpenKey(registry.HKEY_LOCAL_MACHINE,SYS_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
return registry.OpenKey(registry.HKEY_CURRENT_USER,USR_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
As in the comments, changing REG_SZ to REG_EXPAND_SZ did the trick, as variables using "%" weren't being recognized. This also works when no "%"s exist, so I use it for the user path as well rather than needing to switch between the two.
registry.SetValueEx(env_key,"Path",0,registry.REG_EXPAND_SZ,val_string)
Is it possible to share ImageMagick files with others over a network drive (ex.: Z:), without having the users install ImageMagick?
ImageMagick works fine on my own machine as I used its binary installer and set up Windows environment variable ("MAGICK_HOME").
I tried using Python to auto setup the user's system environment variable, but even then when user types in command prompt:
convert c:\testA.psd c:\testB.png
it'll give an error:
convert.exe: no decode delegate for this image format 'PSD' # error/constitute.c/ReadImage/501.
convert.exe: no images defined 'c:\testB.png' # error/convert.c/ConvertImageCommand/3212.
and related error such as:
RegistryKeyLookupFailed 'CoderModulesPath'
Note
System: Windows 7, 64bit
ImageMagick version: 6.9.0-Q8
ImageMagick folder contains many files, include CORE_RL_*.dll,
dcraw.exe, ffmpeg.exe, convert.exe, compare.exe, etc. and "modules"
folder
Reference #1
http://www.imagemagick.org/discourse-server/viewtopic.php?t=20599
Some commands that help debug IM. I used
convert -list format
and it returns an empty list. So now I'm sure the user's IM isn't properly installed.
Reference #2
Packaging an application that uses the ImageMagick C API
I was searching for answers more along the line of 'manually install ImageMagick' (not through binary installer) so i can know exactly what Windows settings I have to configure through Python. Then the link above (posted by Alex) shows what I want. And now I realized I did not configure the environment variable "CoderModulesPath". Now I'm going to try it out...
After a long search... finally found a solution:
If you want to share your ImageMagick folder and files with other users - to save them the hassle of installation or for some other reasons - be sure to programatically configure their system's environment variables:
"MAGICK_HOME" = [path to the ImageMagick folder]
ex. z:\ImageMagick-6.9.0-Q8
"MAGICK_CODER_MODULE_PATH" = [path to the ImageMagick folder]\modules\coders
ex. z:\ImageMagick-6.9.0-Q8\modules\coders
in the ...modules\coders folders, there are DLLs that process different types of image files, and if you don't specify this path, IM can't find "decode delegate"
After the environment variables are set, you might want to inform the user to restart their machine in order for the variables to take effect.
Compatibility
If you are using API to access this ImageMagick library (ex. Python wand), then this might be a good way to go. The portable version of ImageMagick may not be compatible since the file structures are different from the non-portable.
Resources
Python class that retrieves and modifies Windows registry keys and values:
Reference: http://code.activestate.com/recipes/577621-manage-environment-variables-on-windows/
if sys.hexversion > 0x03000000:
import winreg
else:
import _winreg as winreg
class Win32Environment:
# Utility class to get/set windows environment variable
def __init__(self, scope):
assert scope in ('user', 'system')
self.scope = scope
if scope == 'user':
self.root = winreg.HKEY_CURRENT_USER
self.subkey = 'Environment'
else:
self.root = winreg.HKEY_LOCAL_MACHINE
self.subkey = r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment'
def getenv(self, name):
key = winreg.OpenKey(self.root, self.subkey, 0, winreg.KEY_READ)
try:
value, _ = winreg.QueryValueEx(key, name)
except WindowsError:
value = ''
winreg.CloseKey(key)
return value
def setenv(self, name, value):
# Note: for 'system' scope, you must run this as Administrator
key = winreg.OpenKey(self.root, self.subkey, 0, winreg.KEY_ALL_ACCESS)
winreg.SetValueEx(key, name, 0, winreg.REG_EXPAND_SZ, value)
winreg.CloseKey(key)
Your best chance to achieve that is to share the portable version of ImageMagick. Look up ImageMagick portable on Google, download the files and put them on a share.
I'm writing a SublimeText Plugin that uses a shelf for persistent storage of some data; the path of the shelf file is defined in the settings of the current project. I have a WindowCommand and a keybinding for it that initializes the class that holds the shelf, but I want to automatically execute this on startup if a suitable project is open.
Simply running the command when the plugin is loaded fails because sublime.active_window() is still uninitialized - my current workaround is to use set_timeout with a (hopefully) large enough timeout:
import sublime, sublime_plugin
_data = None
class MkshelfCommand(sublime_plugin.WindowCommand):
def run(self):
global _data
shelf_path = self.window.active_view().settings().get("shelf_path")
if shelf_path:
_data = MyClass(shelf_path)
sublime.set_timeout(lambda: sublime.active_window().run_command("mkshelf"), 1000)
This is obviously all kinds of bad; but I couldn't figure out how to do this more reliably as I could neither find a way to access the current projects settings without a view, nor a method to ensure the window exists before executing the command so I could get rid of the timeout (I've thought of abusing EventListener.on_activated, but this seems even more ugly than what I'm doing right now). Is there a better way to do this or should I just bite the bullet and stick with my current approach?
Instead of calling sublime.active_window() directly in you timeout callback, call a function that calls itself when active_window() is still None:
def runCommandWhenInitialized():
activeWindow = sublime.active_window()
if activeWindow is not None:
activeWindow.run_command("mkshelf")
else:
sublime.set_timeout(lambda: runCommandWhenInitialized(), 100)
sublime.set_timeout(lambda: runCommandWhenInitialized(), 100)
(I didn't actually run this code, but I think it should work.)
Try defining a module level method called plugin_loaded. It might
only work in Sublime 3. I don't know which one you're using.
def plugin_loaded():
pass
http://www.sublimetext.com/docs/3/api_reference.html