Develop Reference

How do I add a directory to system environment variable using python script [duplicate]

From what I've read, any changes to the environment variables in a Python instance are only available within that instance, and disappear once the instance is closed. Is there any way to make them stick by committing them to the system?
The reason I need to do this is because at the studio where I work, tools like Maya rely heavily on environment variables to configure paths across multiple platforms.
My test code is
import os
os.environ['FAKE'] = 'C:\\'
Opening another instance of Python and requesting os.environ['FAKE'] yields a KeyError.
NOTE: Portability will be an issue, but the small API I'm writing will be able to check OS version and trigger different commands if necessary.
That said, I've gone the route of using the Windows registry technique and will simply write alternative methods that will call shell scripts on other platforms as they become requirements.

You can using SETX at the command-line.
By default these actions go on the USER env vars.
To set and modify SYSTEM vars use the /M flag
import os
env_var = "BUILD_NUMBER"
env_val = "3.1.3.3.7"
os.system("SETX {0} {1} /M".format(env_var,env_val))

make them stick by committing them to
the system?
I think you are a bit confused here. There is no 'system' environment. Each process has their own environment as part its memory. A process can only change its own environment. A process can set the initial environment for processes it creates.
If you really do think you need to set environment variables for the system you will need to look at changing them in the location they get initially loaded from like the registry on windows or your shell configuration file on Linux.

Under Windows it's possible for you to make changes to environment variables persistent via the registry with this recipe, though it seems like overkill.
To echo Brian's question, what are you trying to accomplish? There is probably an easier way.

Seems like there is simplier solution for Windows
import subprocess
subprocess.call(['setx', 'Hello', 'World!'], shell=True)

I don't believe you can do this; there are two work-arounds I can think of.
The os.putenv function sets the environment for processes you start with, i.e. os.system, popen, etc. Depending on what you're trying to do, perhaps you could have one master Python instance that sets the variable, and then spawns new instances.
You could run a shell script or batch file to set it for you, but that becomes much less portable. See this article:
http://code.activestate.com/recipes/159462/

Think about it this way.
You're not setting shell environment variables.
You're spawning a subshell with some given environment variable settings; this subshell runs your application with the modified environment.

According to this discussion, you cannot do it. What are you trying to accomplish?

You are forking a new process and cannot change the environment of the parent process as you cannot do if you start a new shell process from the shell

You might want to try Python Win32 Extensions, developed by Mark Hammond, which is included in the ActivePython (or can be installed separately). You can learn how to perform many Windows related tasks in Hammond's and Robinson's book.
Using PyWin32 to access windows COM objects, a Python program can use the Environment Property (a collection of environment variables) of the WScript.Shell object.

Try to use py-setenv that will allow you to set variable via registry
python -m pip install py-setenv

From within Python? No, it can't be done!
If you are not bound to Python, you should consider using shell scripts (sh, bash, etc). The "source" command allows you to run a script that modifies the environment and will "stick" like you want to the shell you "sourced" the script in. What's going on here is that the shell executes the script directly rather creating a sub-process to execute the script.
This will be quite portable - you can use cygwin on windows to do this.

In case someone might need this info. I realize this was asked 7 yrs ago, but even I forget how sometimes. .
Yes there is a way to make them "stick" in windows. Simply go control panel, system, advanced system settings,when the system properties window opens you should see an option (button) for Environment Variables. .The process for getting to this is a little different depending on what OS you're using (google it).
Choose that (click button), then the Environment Variables window will open. It has 2 split windows, the top one should be your "User Variables For yourusername". . .choose "new", then simply set the variable. For instance one of mine is "Database_Password = mypassword".
Then in your app you can access them like this: import os, os.environ.get('Database_Password'). You can do something like pass = os.environ.get('Database_Password').

how to launch R in anaconda PowerShell prompt in windows [duplicate]

So I tried adding R to the path on windows 10 (that is supposedly easy).
System Properties -> Environment variables -> Edit -> new: copy and paste: "C:\Program Files\R\R-3.5.0\bin\x64"
Now the thing is, Powershell just refuses to start the R environment when I type in R. R.exe works apparently. Rgui works as well. Is R a reserved letter in powershell or something? It also seems to repeat the previous command sometimes but that doesn't really seem completely consistent either.
(I put this entry on top of the list of the path and restarted the pc already)

when entering get-alias r I got the following result, so yes "r" is already taken ...
CommandType Name Version Source
----------- ---- ------- ------
Alias r -> Invoke-History
PS: you could remove that alias with remove-item alias:\r from your current powershell session and test if "r" then starts "R.exe". if that works for you, you could edit your profile to remove the alias "r -> Invoke-History" from every new session.

To generalize Guenther Schmitz' helpful answer:
PowerShell has several types of commands, whose precedence is:
Alias
Function
Cmdlet
External application
Note that name resolution is always case-insensitive, so that both r and R refer to the same command.
That is, before R resolves to R.exe, it is not only an r alias that could get in the way, but potentially also a function or a cmdlet (though the latter case is hypothetical, because well-behaved cmdlets follow a <Verb>-<Noun> naming pattern).
Note that built-in aliases shadowing external programs, especially standard ones, is problematic, and in the context of PowerShell Core a discussion is underway about whether to remove all built-in aliases and make them opt-in only - see this GitHub issue.
To see what a given name resolves to, use the Get-Command cmdlet:
# See what R resolves to
Get-Command R
# See ALL commands that R *can* resolve to, with the EFFECTIVE one listed first:
Get-Command -All R
Choices for unambiguously targeting R.exe:
(As you already know) If its folder is in one of the folders listed in environment variable $env:PATH, append .exe - i.e., use the filename extension explicitly:
R.exe ...
Use R.exe's full path (note the need for & for invocation, because the path needs quoting):
& "C:\Program Files\R\R-3.5.0\bin\x64\R.exe" ...
(For the sake of completeness; this is the cumbersome equivalent of using just R.exe): Use Get-Command -Type Application to locate the executable:
& (Get-Command -Type Application R) ...
Alternatively, as stated in Guenther's answer, you could add Remove-Alias r to your PowerShell $PROFILE file in order to remove the built-in alias on session startup, which then allows you to just use r to start R.exe.

Run the following code in your console to install the R package. This code will automatically add R to your os PATH.
sudo apt-get install littler

Is there a more elegant way to refer to script's current directory?

I have dev environment and production for running various Python scripts used in crontab.
In dev environment, I usually test scripts from commandline in script directory such as "python myscript.py".
Now, some scripts load config from JSON files in the same or subdirectory of script directory.
In dev, I can therefore refer to file just like that:
printconf = Config('printing.json')
However, once the script is ready for production, it is put in crontab, and crontab calls scripts from root, therefore breaking the above line.
Additionally, dev and production are obviously in different places in the filesystem, so I can't even use absolute paths, because they won't be the same.
As described in How can I find script's directory with Python?, I can use various methods to find the file current directory. They mean, however, extra processing and I was wondering,
if any Python version might have (or might have planned) any additional built-in way to tell that the file must be found relatively to running script's directory? Something like __location__?
In essence, something that would work for file references like importing modules already does.
In addition, I have tried to add a global __location__ variable via sitecustomize.py, but that doesn't even work.
sitecustomize.py:
if '__file__' in globals():
import os
_location_ = os.path.join(os.getcwd(), os.path.dirname(__file__))
But that doesn't work either, because:
- __location__ is not passed to the script,
- and __file__ refers to sitecustomize.py

First and foremost, your Dev and Prod environments should not differ too much. This means that for consistency's sake you should use the same setup (filesystem, libraries etc.) for both. This way most of your issues will vanish.
If you do this, than it's safe to use a hardcoded path.
Other options (that inevitably involve some processing, but it should not be deal breakers):
Use os.getcwd() and __file__, as you already suggested.
Use a specific argument when running your scripts (e.g.: $SCRIPT_PATH/myscript.py PROD) and choose your paths based on this inside the script.
Use a dedicated config file just for script init, put it in the same place both on DEV and PROD (I would suggest in /etc/$PROJECT_NAME) and run the script with a specific argument (like mentioned above).
None of the above methods will prevent other issues related to consistency, which makes me underline that you should look into using Docker or Vagrant for your Dev setup (and/or Prod, if possible).

As I reviewed this question, it became clear that I provided probably too much of wrong context and too little of relevant one. As I tested the actual solution, I decided against rewriting the question, and instead put the right context in the answer.
Most of the problems were due to fact that I wanted to determine path of file B, relative to script A, when the actual CALL to use file B used a system-wide module C (called from script A) which was in completely different place, like this:
#module C (system-wide in /usr.../site-packages
import os
class Config()
...
def load(file_name):
# needs to be relative to calling script's dir, not module's!
real_file_name=os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])),file_name)
...
#script-A.py ( in /var/scripts/projectA )
from C import Config
conf=Config()
conf.load('file-in-projectA-dir.json')
# finds /var/scripts/projectA/file-in-projectA-dir.json
# regardless of cwd
So this works both for:
cd /
python3 /var/scripts/projectA/script-A.py
#and:
cd /var/scripts/projectA/
python3 script-A.py
#etc
As you can see, this only works due to fact that script-A is called from command line as first parameter. However, that is my main use case. Not sure, what could work otherwise.

How do you I add a module into my PYTHONPATH?

I am attempting to download a code from github which contains the library "ee" - Google Earth Engine. GitBash is giving me an error:
ModuleNotFoundError: No module named 'fcntl'
fcntl is a module within the library Google Earth Engine. I have Windows and it seems Linux is required. I was directed to add the module (fcntl) to the PYTHONPATH. Any other suggestions for this error would be helpful as well! The code I intend to add in PYTHONPATH is below.
def fcntl(fd, op, arg=0):
return 0
def ioctl(fd, op, arg=0, mutable_flag=True):
if mutable_flag:
return 0
else:
return ""
def flock(fd, op):
return
def lockf(fd, operation, length=0, start=0, whence=0):
return

First, this is probably not going to work for you.
You can't turn Windows to Linux just by adding modules to your Python library. The reason you don't have the fcntl module on your path is that fcntl isn't included on Windows. And the reason it isn't included on Windows is that the Windows OS doesn't support the syscalls that module wraps, or anything close enough to reasonably emulate those syscalls.
If you have code that requires fcntl, that code cannot run on Windows (unless you do some significant work to port it to not require fcntl in the first place).
If you have code that doesn't require fcntl but uses it anyway, or if you just need something for temporary development purposes so you can catch and fix file sharing errors while porting the code to not require fcntl, then you can use msoliman's dummy code, which I'll explain how to do below. But you seem to be expecting it to do magic, and it won't do that.
You may not be sure. Maybe you're using code that uses other code that uses other code that uses fcntl in some scenarios but not others, it may not actually need fcntl to do any of the things you're actually trying to do with it.
If you want to test that, you can take msoliman's dummy code, and change each function body to this:
raise RuntimeError('Oops, using fcntl!')
Then run the program and see if it fails with that error. If not, you don't actually need fcntl after all. (Or at least you don't need it for any of the things you tested—it's always possible that some other thing you need to do with the app that you didn't think to test will need it.)
If your code actually needs fcntl, and you don't want to/can't port that code to Windows code that uses Win32 API calls (or a cross-platform library like portalocker), then what you probably need to do is install Linux and run the program there.
There are multiple ways to run Linux on top of Windows, rather than instead of Windows. For example, you could install Docker for Windows and then build a linux docker container with the app. Or you could use VMWare Player to, in effect, run a Linux image as an application under Windows, and then do your work inside that image. And so on.
Finally, msoliman's "Place this module in your PYTHONPATH" is a little misleading.
What you actually need to do is get it into your sys.path. PYTHONPATH is just one way of doing that, and probably not the one you want here.
The options are:
Just put it in the same directory as your script. As the docs say, "As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter."
Put it in your user or system site packages, or some other directory that's already on your default sys.path. You can import sys; print(sys.path) to get a list of these directories. If you see something inside your home directory, that's a good place to put it; if not, look for something with site-packages in the name.
Put it in some other directory somewhere else, and set the PYTHONPATH environment variable to be the full path to directory. You can set an environment variable in the Windows cmd command prompt by writing SET PYTHONPATH C:\Path\To\Directory. This will only persist as long as the current command prompt window. If you want to set it permanently, there's a setting somewhere in Control Panel (it changes with each Windows version; Super User should have good up-to-date answers for each version) where you can set System and User environment variables. Any User environment variable will take effect in every new command prompt window you open from now on.

Python off a Flash Drive, how to start IDLE?

I was trying to run Python 3.3 off of my flash drive. I already tried Portable Python, but most of what it had wouldn't open, and it crashed my laptop.
So, I deleted it, and installed regular Python. Then, I wanted to start adding my favorite modules. And, I needed a way to start IDLE without the original shortcut.
To install modules, I added my Python install to my PATH variable.
To make all this easier, I made a batch file, using some code I found on this question.
So far, I have this. It also asks for the drive letter, because that changes from computer to computer.
#echo off
echo This only works if your Python install is added to the system-wide PATH variable
set /p Path="Enter the Drive Letter on this computer. No Symbols, Just the Letter, Capital"
cd %Path%:\Program Files\Python33
echo type this when python comes up...
echo import idlelib.PyShell
echo idlelib.PyShell.main()
echo.
echo.
echo.
echo.
python
It outputs this:
If you go on and follow the instructions and type what it says, it brings up IDLE.
I couldn't figure out how to get the batch file to actually type into the Python prompt, so I told it to tell the user to type what needed to be typed.
What I need to know is, how can I change the PATH variable from within the batch file. Also, how to I remove it when I'm done (this isn't as important, and could even be in a separate batch file).
Or, alternatively, is there a way just to shortcut to IDLE?
Also, is there a way to run .py files without the command line, with the Python install on my flash drive?
Thanks!

You can invoke Python with this command line:
python -c"import idlelib.PyShell;idlelib.PyShell.main()"
and it will bring-up IDLE shell without the need for the user to type anything.
Edit: BTW, Are you sure you really need to change the global path settings. Try to see if the following script can start Python the way you want. You must put it in the root of the USB drive where you have your Python installation.
#echo off
setlocal
set SCRIPT_DIR=%~dp0
:: Removes trailing backslash (for readability in the following)
set SCRIPT_DIR=%SCRIPT_DIR:~0,-1%
set PYTHON_HOME=%SCRIPT_DIR%\Program Files\Python33
set PATH=%PYTHON_HOME%;%PATH%
"%PYTHON_HOME%\python.exe" -c"import idlelib.PyShell;idlelib.PyShell.main()"
Edit: Every process has an associated environment, which is a set of name-value pairs called environment variables. When a process is started it gets a copy of the environment of its parent process. The global OS settings for environment variables are used for processes started directly from the OS (GUI or command line) shell. The set command in batch files sets or modifies an environment variable in the environment of the current process (not globally).
All the set commands you see in the above script change only the environment of the current process. These changes will be seen by the process created by the last line (python.exe) because it is a child process of the command shell (cmd.exe) process that is executing the batch-file.
The line
set PATH=%PYTHON_HOME%;%PATH%
prepends the content of the PYTHON_HOME variable to the PATH variable of the current process. For example, if PATH were c:\foo\bar;d:\aaa\bbb and PYTHON_HOME were c:\python then the new value of PATH will be c:\python;c:\foo\bar;d:\aaa\bbb

It can not be guaranteed this is possible unless you have high enough system privileges that you can change the global path. There is really no way around this on most computers that you do not own, which I imagine is the main purpose. In those cases when you have enough privileges (it is worth a try some systems still allow this for regular users but many others do not) you can use:
setx PATH "%path%;yourpath"
edit and ps:
You can figure out the drive letter without input, if you know the disk label, with something like this:
#echo off
set label=DRIVENAME
set cmd=WMIC logicaldisk WHERE volumename^^="%label%" GET caption
FOR /F "tokens=1" %%G IN ('%cmd% ^| find ":"')DO set pydrive=%%G
echo %pydrive%\pathtopython
rem do your stuff here
the idle started inside the batch will inherit path but other instances will not. Hard to test conclusively tough.
Explanation of the batch script above. The command wmic is short for windows management instrumentation commandline. One can use WMI to do many things one of them is to issue WQL (SQL for WMI) queries as if windows would be a a database. Databases contain many tables in this case the computer is instructed to fetch the table named logicaldisk. Table logicaldisk has 38 columns and one row for each disk connected to the system. This is way to much data for this purpose. So the data is filtered. WHERE causes the database only to spit out rows that contain some specific value in this case its only interested in rows where column volumename ins equal to DRIVENAME, likewise you could use serial number size or any other criteria. Lastly GET is used to limit the columns you get back as results since your only interested in the letter name of the drive that's what you ask. That is called a caption in the table so that what you ask.
Since the command is a bit long, so i put the command inside a variable (not the result) this shortens the for line so it fits stack overflow. Since = need to be escaped i need to use the escape sequence ^ too times so it still es capable in the for loop.
The for loop is used to capture the return value of the wmic command. Since the answer has many lines i filter for only the lines which contain a colon character. And put it to variable pydrive.