I am looking to permanently modify the path variable inside windows from a python script. The script is a wrapper to help automate the installation of applications and I want to enable the API to add applications to the path.
So for example I want to install a program called micro which has an install path of C:\Users\USERNAME\Path\to\micro and then add that install path to my path variable so that I can just run micro in my terminal.
I've been made aware of 2 possible solutions, which both do not work:
1. Using os.environ
In python the os module let's you read environment variables, but not actually modify them. so for example:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
new_path = f"{os.environ['PATH']};{program_path}"
os.environ["PATH"] = new_path
This would update the path variable in the python script, but it does not actually modify it on the system which is what I want.
2. setx
I was made aware that it is possible to update your path using the setx command in windows, but for some reason on windows 10 this destroys your path variable.
The idea is that you can call the setx command from python and use it to update the path variable. You should be able to type setx path "%path%;C:\Users\USERNAME\Path\to\micro" and have it update correctly.
So for example, in python code that would be:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
subprocess.Popen(f'setx path "%path%;{program_path}"')
This should take the current path variable and append the program path to it, but instead it just wipes your entire path and replaces it with a literal %path% and then the program path.
So now my path looks like this:
%path%
C:\Users\USERNAME\Path\to\micro
Any ideas on how to get this to work would be appreciated.
Okay, so after a long (and disgusting) amount of research I found a solution. Here is the method I came up with for a cross-platform system of adding to PATH variable:
def add_to_path(program_path:str):
"""Takes in a path to a program and adds it to the system path"""
if os.name == "nt": # Windows systems
import winreg # Allows access to the windows registry
import ctypes # Allows interface with low-level C API's
with winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) as root: # Get the current user registry
with winreg.OpenKey(root, "Environment", 0, winreg.KEY_ALL_ACCESS) as key: # Go to the environment key
existing_path_value = winreg.EnumValue(key, 3)[1] # Grab the current path value
new_path_value = existing_path_value + program_path + ";" # Takes the current path value and appends the new program path
winreg.SetValueEx(key, "PATH", 0, winreg.REG_EXPAND_SZ, new_path_value) # Updated the path with the updated path
# Tell other processes to update their environment
HWND_BROADCAST = 0xFFFF
WM_SETTINGCHANGE = 0x1A
SMTO_ABORTIFHUNG = 0x0002
result = ctypes.c_long()
SendMessageTimeoutW = ctypes.windll.user32.SendMessageTimeoutW
SendMessageTimeoutW(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u"Environment", SMTO_ABORTIFHUNG, 5000, ctypes.byref(result),)
else: # If system is *nix
with open(f"{os.getenv('HOME')}/.bashrc", "a") as bash_file: # Open bashrc file
bash_file.write(f'\nexport PATH="{program_path}:$PATH"\n') # Add program path to Path variable
os.system(f". {os.getenv('HOME')}/.bashrc") # Update bash source
print(f"Added {program_path} to path, please restart shell for changes to take effect")
Neither are pretty, but it does actually work. You do need to restart running shells for it to take effect, but other than that it's perfect.
Related
My Python script is trying to setup a Windows system, including the value of the environment variable PATH.
Windows appears to have two sets of places where it stores environment variables - "User variables" and "System variables". (To see what I mean in Win10, type "environment variables" into the Start menu, then click on the "Environment Variables..." button at the bottom right of that dialog.)
The PATH as seen by programs (example: os.environ in Python) appears to be a concatenation of the PATH system variable and the PATH user variable.
I'm trying to setup the User version of PATH, without affecting the System version.
My problem: When I use os.environ('PATH') = value, the PATH available to Python becomes the string in value only (what I meant for the User version of PATH), and the contents of the System PATH disappears. Then the rest of my Python script can't find the executables it needs.
I can think of several hacky workaounds, but what's the proper way to deal with this?
FWIW, here's what I use to set environment variables (including PATH) now:
def setEnvironmentVariable(name, value):
'''Set an environment variable in both current and permanent environment '''
os.environ[name] = value
os.system('set ' + name + ' = ' + value) # for this shell
os.system('setx ' + name + ' "' + value + '"') # for the permanent user environment
(Note os.system is using cmd.exe; I'm OK with using Powershell if that'll help.)
I have a file contains set of environment variables .
env_script.env:
export a=hjk
export b=jkjk
export c=kjjhh
export i=jkkl
..........
I want set these environment variables by reading from file .
how can i do this in python
Tried sample code:
pipe = subprocess.Popen([".%s;env", "/home/user/env_script.env"], stdout=subprocess.PIPE, shell=True)
output = pipe.communicate()[0]
env = dict((line.split("=", 1) for line in output.splitlines()))
os.environ.update(env)
Please give some suggestion
There's a great python library python-dotenv that allows you to have your variables exported to your environment from a .env file, or any file you want, which you can keep out of source control (i.e. add to .gitignore):
# to install
pip install -U python-dotenv
# your .env file
export MY_VAR_A=super-secret-value
export MY_VAR_B=other-very-secret-value
...
And you just load it in python when your start like:
# settings.py
from dotenv import load_dotenv
load_dotenv()
Then, you can access any variable later in your code:
from os import environ
my_value_a = environ.get('MY_VALUE_A')
print(my_value_a) # 'super-secret-value'
You don't need to use subprocess.
Read lines and split environment variable name, value and assign it to os.environ:
import os
with open('/home/user/env_script.env') as f:
for line in f:
if 'export' not in line:
continue
if line.startswith('#'):
continue
# Remove leading `export `
# then, split name / value pair
key, value = line.replace('export ', '', 1).strip().split('=', 1)
os.environ[key] = value
or using dict.update and generator expression:
with open('env_script.env') as f:
os.environ.update(
line.replace('export ', '', 1).strip().split('=', 1) for line in f
if 'export' in line
)
Alternatively, you can make a wrapper shell script, which sources the env_script.env, then execute the original python file.
#!/bin/bash
source /home/user/env_script.env
python /path/to/original_script.py
Modern operating systems do not allow a child process to change the environment of its parent. The environment can only be changed for the current process and its descendants. And a Python interpreter is a child of the calling shell.
That's the reason why source is not an external command but is interpreted directly by the shell to allow a change in its environment.
It used to be possible in the good old MS/DOS system with the .COM executable format. A .com executable file had a preamble of 256 (0x100) bytes among which was a pointer to the COMMAND.COM's environment string! So with low level memory functions, and after ensuring not overwriting anything past the environment, a command could change directly its parent environment.
It may still be possible in modern OS, but require cooperation from system. For example Windows can allow a process to get read/write access to the memory of another process, provided the appropriate permissions are set. But this is really a hacky way, and I would not dare doing this in Python.
TL/DR: if your requirement is to change the environment of the calling shell from a Python script, you have misunderstood your requirement.
But what is easy is to start a new shell with a modified environment:
import os
import subprocess
env = os.environ.copy() # get a copy of current environment
# modify the copy of environment at will using for example falsetru's answer
# here is just an example
env['AAA'] = 'BBB'
# and open a subshell with the modified environment
p = subprocess.Popen("/bin/sh", env = env)
p.wait()
I've written a program to add directories to the PATH variable via the registry, either the HKCU(user) or HKLM(system) path, depending on an input option.
It works fine when using the User path.
However, when setting the path for the System, Windows acts as if the path variable is empty, e.g.
'notepad' is not recognized as an internal or external command....
However, echo %path% prints everything out appropriately, without any syntax errors. Similarly, if I view the variable in the System Properties GUI, it shows my full path appropriately, e.g.
%SystemRoot%\system32;%SystemRoot%;
Now, if I manually open that variable in the GUI, and add OR remove the trailing semicolon (i.e. make a noticeable but seemingly irrelevant change), then the path seems to work fine.
Yes, I am opening a new command window to check the path. Restarting the machine doesn't seem to do anything either.
Any ideas?
Code excerpt is here:
import _winreg as registry
#HKEY_LOCAL_MACHINE\
SYS_ENV_SUBPATH = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
#HKEY_CURRENT_USER\
USR_ENV_SUBPATH = r"Environment"
def update_reg_path_value(paths_to_add,privilege):
env_key = open_env_registry_key(privilege)
current_path = get_path_from_registry_or_create(env_key)
val_string = create_new_path_value(current_path, paths_to_add)
registry.SetValueEx(env_key,"Path",0,registry.REG_SZ,val_string)
def open_env_registry_key(privilege):
if privilege == 'system':
return registry.OpenKey(registry.HKEY_LOCAL_MACHINE,SYS_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
return registry.OpenKey(registry.HKEY_CURRENT_USER,USR_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
As in the comments, changing REG_SZ to REG_EXPAND_SZ did the trick, as variables using "%" weren't being recognized. This also works when no "%"s exist, so I use it for the user path as well rather than needing to switch between the two.
registry.SetValueEx(env_key,"Path",0,registry.REG_EXPAND_SZ,val_string)
I am trying to set a environmental variable permanently. but temporarily it is working.
if i run below program i got the variable path. after close it and open new terminal to find the variable path using the command printenv LD_LIBRARY_PATH nothing will print.
#!/usr/bin/python
import os
import subprocess
def setenv_var():
env_var = "LD_LIBRARY_PATH"
env_path = "/usr/local/lib"`enter code here`
os.environ[env_var] = env_path
process = subprocess.Popen('printenv ' + env_var, stdout=subprocess.PIPE, shell=True)
result = process.communicate()[0]
return result
if __name__ == '__main__':
print setenv_var()
please help me.
Here is what I use to set environment variables:
def setenv_var(env_file, set_this_env=True):
env_var = "LD_LIBRARY_PATH"
env_path = "/usr/local/lib"`enter code here`
# set environments opened later by appending to `source`-d file
with open(env_file, 'a') as f:
f.write(os.linesep + ("%s=%s" % (env_var, env_path)))
if set_this_end:
# set this environment
os.environ[env_var] = env_path
Now you only have to choose where to set it, that is the first argument in the function. I recommend the profile-specific file ~/.profile or if you're using bash which is pretty common ~/.bashrc
You can also set it globally by using a file like /etc/environment but you'll need to have permissions when you run this script (sudo python script.py).
Remember that environments are inherited from the parent process, and you can't have a child set up a parent process' environment.
When you set an environment variable, it only affects the currently running process (and, by extension, any children that are forked after the variable is set). If you are attempting to set an environment variable in your shell and you want that environment variable to always be set for your interactive shells, you need to set it in the startup scripts (eg .login, .bashrc, .profile) for your shell. Commands that you run are (initially) children of the shell from which you run them, so although they inherit the environment of the shell and can change their own environment, they cannot change the environment of your shell.
Whether you do an export from bash or you set your os.environ from Python, these only stay for the session or process's lifetime. If you want to set them permanent you will have to touch and add it to the respective shell's profile file.
For ex. If you are on bash, you could do:
with open("~/.bashrc", "a") as outfile: # 'a' stands for "append"
outfile.write("export LD_LIBRARY_PATH=/usr/local/lib")
Check this out for some insight as to which file to add this depending on the target shell. https://unix.stackexchange.com/questions/117467/how-to-permanently-set-environmental-variables
I see that if we change the HOME (linux) or USERPROFILE (windows) environmental variable and run a python script, it returns the new value as the user home when I try
os.environ['HOME']
os.exp
Is there any way to find the real user home directory without relying on the environmental variable?
edit:
Here is a way to find userhome in windows by reading in the registry,
http://mail.python.org/pipermail/python-win32/2008-January/006677.html
edit:
One way to find windows home using pywin32,
from win32com.shell import shell,shellcon
home = shell.SHGetFolderPath(0, shellcon.CSIDL_PROFILE, None, 0)
I think os.path.expanduser(path) could be helpful.
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
On Unix, an initial ~ is replaced by the environment variable HOME if it is set; otherwise the current user’s home directory is looked up in the password directory through the built-in module pwd. An initial ~user is looked up directly in the password directory.
On Windows, HOME and USERPROFILE will be used if set, otherwise a combination of HOMEPATH and HOMEDRIVE will be used. An initial ~user is handled by stripping the last directory component from the created user path derived above.
If the expansion fails or if the path does not begin with a tilde, the path is returned unchanged.
So you could just do:
os.path.expanduser('~user')
from pathlib import Path
str(Path.home())
works in Python 3.5 and above. Path.home() returns a Path object providing an API I find very useful.
I think os.path.expanduser(path) is the best answer to your question, but there's an alternative that may be worth mentioning in the Unix world: the pwd package. e.g.
import os, pwd
pwd.getpwuid(os.getuid()).pw_dir
For windows;
import os
homepath = os.path.expanduser(os.getenv('USERPROFILE'))
will give you a handle to current user's home directory and
filepath = os.path.expanduser(os.getenv('USERPROFILE'))+'\\Documents\\myfile.txt'
will give you a handle to below file;
C:\Users\urUserName\Documents\myfile.txt
home_folder = os.getenv('HOME')
This should work on Windows and Mac OS too, works well on Linux.
Really, a change in environment variable indicates that the home must be changed. So every program/script should have the new home in context; also the consequences are up to the person who changed it.
I would still stick with
home = os.getenv('USERPROFILE') or os.getenv('HOME')
what exactly is required?
I realize that this is an old question that has been answered but I thought I would add my two cents. The accepted answer was not working for me. I needed to find the user directory and I wanted it to work with and without sudo. In Linux, my user directory is "/home/someuser" but my root directory is "/root/". However, on my Mac, the user directory is "/Users/someuser". Here is what I ended up doing:
_USERNAME = os.getenv("SUDO_USER") or os.getenv("USER")
_HOME = os.path.expanduser('~'+_USERNAME)
This worked both with and without sudo on Mac and Linux.
get (translated) user folder names on Linux:
from gi.repository import GLib
docs = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOCUMENTS)
desktop = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DESKTOP)
pics = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PICTURES)
videos = GLib.get_user_special_dir(GLib.USER_DIRECTORY_VIDEOS)
music = GLib.get_user_special_dir(GLib.USER_DIRECTORY_MUSIC)
downloads = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOWNLOAD)
public = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PUBLIC_SHARE)
templates = GLib.get_user_special_dir(GLib.USER_DIRECTORY_TEMPLATES)
print(docs)
print(desktop)
print(pics)
print(videos)
print(music)
print(downloads)
print(public)
print(templates)
On Linux and other UNIXoids you can always take a peek in /etc/passwd. The home directory is the sixth colon-separated field in there. No idea on how to do better than the environment variable on Windows though. There'll be a system call for it, but if it's available from Python, ...