Develop Reference

How to modify windows 10 path variable directly from a python script

I am looking to permanently modify the path variable inside windows from a python script. The script is a wrapper to help automate the installation of applications and I want to enable the API to add applications to the path.
So for example I want to install a program called micro which has an install path of C:\Users\USERNAME\Path\to\micro and then add that install path to my path variable so that I can just run micro in my terminal.
I've been made aware of 2 possible solutions, which both do not work:
1. Using os.environ
In python the os module let's you read environment variables, but not actually modify them. so for example:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
new_path = f"{os.environ['PATH']};{program_path}"
os.environ["PATH"] = new_path
This would update the path variable in the python script, but it does not actually modify it on the system which is what I want.
2. setx
I was made aware that it is possible to update your path using the setx command in windows, but for some reason on windows 10 this destroys your path variable.
The idea is that you can call the setx command from python and use it to update the path variable. You should be able to type setx path "%path%;C:\Users\USERNAME\Path\to\micro" and have it update correctly.
So for example, in python code that would be:
program_path = "C:\\Users\\USERNAME\\Path\\to\\micro"
subprocess.Popen(f'setx path "%path%;{program_path}"')
This should take the current path variable and append the program path to it, but instead it just wipes your entire path and replaces it with a literal %path% and then the program path.
So now my path looks like this:
%path%
C:\Users\USERNAME\Path\to\micro
Any ideas on how to get this to work would be appreciated.

Okay, so after a long (and disgusting) amount of research I found a solution. Here is the method I came up with for a cross-platform system of adding to PATH variable:
def add_to_path(program_path:str):
"""Takes in a path to a program and adds it to the system path"""
if os.name == "nt": # Windows systems
import winreg # Allows access to the windows registry
import ctypes # Allows interface with low-level C API's
with winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) as root: # Get the current user registry
with winreg.OpenKey(root, "Environment", 0, winreg.KEY_ALL_ACCESS) as key: # Go to the environment key
existing_path_value = winreg.EnumValue(key, 3)[1] # Grab the current path value
new_path_value = existing_path_value + program_path + ";" # Takes the current path value and appends the new program path
winreg.SetValueEx(key, "PATH", 0, winreg.REG_EXPAND_SZ, new_path_value) # Updated the path with the updated path
# Tell other processes to update their environment
HWND_BROADCAST = 0xFFFF
WM_SETTINGCHANGE = 0x1A
SMTO_ABORTIFHUNG = 0x0002
result = ctypes.c_long()
SendMessageTimeoutW = ctypes.windll.user32.SendMessageTimeoutW
SendMessageTimeoutW(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u"Environment", SMTO_ABORTIFHUNG, 5000, ctypes.byref(result),)
else: # If system is *nix
with open(f"{os.getenv('HOME')}/.bashrc", "a") as bash_file: # Open bashrc file
bash_file.write(f'\nexport PATH="{program_path}:$PATH"\n') # Add program path to Path variable
os.system(f". {os.getenv('HOME')}/.bashrc") # Update bash source
print(f"Added {program_path} to path, please restart shell for changes to take effect")
Neither are pretty, but it does actually work. You do need to restart running shells for it to take effect, but other than that it's perfect.

Methods to avoid hard-coding file paths in Python

Working with scientific data, specifically climate data, I am constantly hard-coding paths to data directories in my Python code. Even if I were to write the most extensible code in the world, the hard-coded file paths prevent it from ever being truly portable. I also feel like having information about the file system of your machine coded in your programs could be security issue.
What solutions are out there for handling the configuration of paths in Python to avoid having to code them out explicitly?

One of the solution rely on using configuration files.
You can store all your path in a json file like so :
{
"base_path" : "/home/bob/base_folder",
"low_temp_area_path" : "/home/bob/base/folder/low_temp"
}
and then in your python code, you could just do :
import json
with open("conf.json") as json_conf :
CONF = json.load(json_conf)
and then you can use your path (or any configuration variable you like) like so :
print "The base path is {}".format(CONF["base_path"])

First off its always good practise to add a main function to go with each class to test that class or functions in the file. Along with this you determine the current working directory. This becomes incredibly important when running python from a cron job or from a directory that is not the current working directory. No JSON files or environment variables are then needed and you will obtain interoperation across Mac, RHEL and Debian distributions.
This is how you do it, and it will work on windows also if you use '\' instead of '/' (if that is even necessary, in your case).
if "__main__" == __name__:
workingDirectory = os.path.realpath(sys.argv[0])
As you can see when you run your command, the working directory is calculated if you provide a full path or relative path, meaning it will work in a cron job automatically.
After that if you want to work with data that is stored in the current directory use:
fileName = os.path.join( workingDirectory, './sub-folder-of-current-directory/filename.csv' )
fp = open( fileName,'r')
or in the case of the above working directory (parallel to your project directory):
fileName = os.path.join( workingDirectory, '../folder-at-same-level-as-my-project/filename.csv' )
fp = open( fileName,'r')

I believe there are many ways around this, but here is what I would do:
Create a JSON config file with all the paths I need defined.
For even more portability, I'd have a default path where I look for this config file but also have a command line input to change it.

In my opinion passing arguments from command line would be best solution. You should take a look at argparse . This allows you to create nice way to handle arguments from the command line. for example:
myDataScript.py /home/userName/datasource1/

Setting windows system PATH in Registry via Python winreg

I've written a program to add directories to the PATH variable via the registry, either the HKCU(user) or HKLM(system) path, depending on an input option.
It works fine when using the User path.
However, when setting the path for the System, Windows acts as if the path variable is empty, e.g.
'notepad' is not recognized as an internal or external command....
However, echo %path% prints everything out appropriately, without any syntax errors. Similarly, if I view the variable in the System Properties GUI, it shows my full path appropriately, e.g.
%SystemRoot%\system32;%SystemRoot%;
Now, if I manually open that variable in the GUI, and add OR remove the trailing semicolon (i.e. make a noticeable but seemingly irrelevant change), then the path seems to work fine.
Yes, I am opening a new command window to check the path. Restarting the machine doesn't seem to do anything either.
Any ideas?
Code excerpt is here:
import _winreg as registry
#HKEY_LOCAL_MACHINE\
SYS_ENV_SUBPATH = r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
#HKEY_CURRENT_USER\
USR_ENV_SUBPATH = r"Environment"
def update_reg_path_value(paths_to_add,privilege):
env_key = open_env_registry_key(privilege)
current_path = get_path_from_registry_or_create(env_key)
val_string = create_new_path_value(current_path, paths_to_add)
registry.SetValueEx(env_key,"Path",0,registry.REG_SZ,val_string)
def open_env_registry_key(privilege):
if privilege == 'system':
return registry.OpenKey(registry.HKEY_LOCAL_MACHINE,SYS_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)
return registry.OpenKey(registry.HKEY_CURRENT_USER,USR_ENV_SUBPATH,
0,registry.KEY_ALL_ACCESS)

As in the comments, changing REG_SZ to REG_EXPAND_SZ did the trick, as variables using "%" weren't being recognized. This also works when no "%"s exist, so I use it for the user path as well rather than needing to switch between the two.
registry.SetValueEx(env_key,"Path",0,registry.REG_EXPAND_SZ,val_string)

How do I read/write files to an unknown user directory?

I'm attempting to read and write files from a User Directory, (C:\Users\USERNAME\Test Source) But I've been unsuccessful in finding any resources on how I can auto detect the name of the user, USERNAME in the above example, or anyway that I can have it read and write to the directory without knowledge off what a users name is.
Could anyone point me towards the right direction or methods for this, if it's even a logical request? I'm not sure how much difference, if any, it makes but this program is being written in Python 2.7.

The simplest way is this:
import os
print os.path.expanduser('~')
Append your folder to the path like so:
userdir = os.path.expanduser('~')
print os.path.join(userdir, 'Test Source')
Besides requiring the least lines of code, this method has the advantage of working under every OS (Linux, Windows XP / 7 / 8 / etc).

You can use in windows command line
echo %username%
or
whoami
for getting the username of the user who is currently logged in .
Store it in a variable and then append it to the path name.
You can also use
‘C:\users\%username%\file‘
directly .To check through whoami do
l=`whoami`
echo $l

Use the %userprofile% variable in your path if you're on Windows:
%userprofile%\Test Source\file.txt

Try:
>>> import getpass
>>> import os.path
>>> usename = getpass.getuser()
>>> mypath = os.path.join("C:\Users", username, "Test Source")

Python. Unchroot directory

I chrooted directory using following commands:
os.chroot("/mydir")
How to return to directory to previous - before chrooting?
Maybe it is possible to unchroot directory?
SOLUTION:
Thanks to Phihag. I found a solution. Simple example:
import os
os.mkdir('/tmp/new_dir')
dir1 = os.open('.', os.O_RDONLY)
dir2 = os.open('/tmp/new_dir', os.O_RDONLY)
os.getcwd() # we are in 'tmp'
os.chroot('/tmp/new_dir') # chrooting 'new_dir' directory
os.fchdir(dir2)
os.getcwd() # we are in chrooted directory, but path is '/'. It's OK.
os.fchdir(dir1)
os.getcwd() # we came back to not chrooted 'tmp' directory
os.close(dir1)
os.close(dir2)
More info

If you haven't changed your current working directory, you can simply call
os.chroot('../..') # Add '../' as needed
Of course, this requires the CAP_SYS_CHROOT capability (usually only given to root).
If you have changed your working directory, you can still escape, but it's harder:
os.mkdir('tmp')
os.chroot('tmp')
os.chdir('../../') # Add '../' as needed
os.chroot('.')
If chroot changes the current working directory, you can get around that by opening the directory, and using fchdir to go back.
Of course, if you intend to go out of a chroot in the course of a normal program (i.e. not a demonstration or security exploit), you should rethink your program. First of all, do you really need to escape the chroot? Why can't you just copy the required info into it beforehand?
Also, consider using a second process that stays outside of the chroot and answers to the requests of the chrooted one.