Develop Reference

python subprocess.run with working directory changed by cwd is not working

I am trying to use python subprocess to call an exe. The application usually takes the parameter file from the same directory as exe. However, as the python file is not located at the same directory as exe, the exe cannot find the parameter file when called by subprocess.run. Hence I specified the cwd when calling subprocess.run like below:
subprocess.run([cwd_exe, "--cal-cn-bv", cwd_cif, "Cs1"], cwd=r'd:\Study\softBV_mix\GitHub\projects\Coord\bin', capture_output=True)
However the subprocess still cannot find the dat file in
d:\Study\softBV_mix\GitHub\projects\Coord\bin
The error message appears as
CompletedProcess(args=['d:\Study\softBV_mix\GitHub\projects\Coord\bin/softBV0405.exe',
'--cal-cn-bv',
'd:\Study\softBV_mix\GitHub\projects\Coord\test/CsCl.cif',
'Cs1'], returncode=0, stdout=b'Warning: unable to find
d:\Study\softBV_mix\GitHub\projects\Coord\database_unitary.dat
where the database_unitary.dat is supposed to be in .../coord/bin/. The application works well if I call it from powershell or command prompt.

No one has answered my question but I kind of found the workaround myself though I am sure if I identify the root cause correctly.
In the end I imported os and make sure cwd is the recognised absolute addresss
import os
cwd = os.path.abspath("../bin")
This worked out.
So the expression
r'd:\Study\softBV_mix\GitHub\projects\Coord\bin'
make cause the issue. Hope some PRO can further clarify this.

linux command using python to go under a specific folder

Hello everyone : I have only one folder under my current directory and I want to go to it by running "cd $(ls)".
So I write this code
import os
os.system("cd $(ls)")
But this did not work for me . Anyone can help to write python syntax to go under the only available folder.
PS : the name of the folder is changeable that's why I want to use "cd $(ls)"

The OS module has some utility functions to achieve what you want to do.
os.listdir(): return a list with all files/directories inside the current working directory
os.chdir(path): changes your working directory
So, you could apply these like:
os.chdir(os.listdir()[0])

It is not obvious if by "go to it" you mean "change current directory for the remaining part of script code" or "change directory to be in after the script exits". If the later, you won't be able to do it - os.system starts a subshell, and changes of current directory in a subshell are not propagated to the parent shell. If the former, you should just use:
import glob, os
os.chdir(glob.glob('*')[0])

Use instead:
os.chdir(os.listdir('.')[0])
Although os.system("cd %(ls)) is correctly working in your shell it will not change the current working directory of your running python interpreter, because os.system() is using a separate shell instance that will be destroyed directly after the execution of the cd shell command.
Double check by executing os.getcwd() before and after (os.getcwd() returns the current working directory of your python interpreter).

How do I run external programs in Python without using the entire path?

Now, I'm working on a voice controlling assistant application.
When I say "open Google Chrome", it should open Chrome. I do not have any problems with speech recognition but I do have a problem with starting the program.
For example, the following code:
import os
os.system("chrome.exe")
Works fine for the notepad.exe or cmd.exe programs but when I give external programs like chrome, I need to give the entire path.
However, in C# we can only give the direct name like chrome.exe to run the program.
So, my problem is that, is there any ways to start external programs without giving the entire path like in C#?
Giving path to start the program would be a serious problem. Because when we move the program to another computer, we will face many code errors.

Try os.path.abspath
os.path.abspath("chrome.exe")
returns the following:
C:\Users\Yourname\chrome.exe

The PATH system variable governs this inside Python just as outside. If you want to modify it from Python, it's os.environ.
As an aside, a better solution is probably to use subprocess instead, as suggested in the os.system documentation.
The PATH manipulation will be the same regardless.
import os
import subprocess
os.environ['PATH'] = os.environ['PATH'] + r';c:\google\chrome\'
subprocess.run(['chrome.exe'])
... where obviously you need to add the directory which contains chrome.exe to your PATH.
Probably you will want to make sure the PATH is correct even before running Python in your scenario, though.

The first part of this answer is OS specific and doesn't solve it in code or python
Add the path where Chrome exists to your PATH variable (you'll likely need a restart), and then when you execute a command such as chrome.exe it will try to run it from that path location
UPDATE:
If you want to implement a search for the absolute path here are some good resources on path listing How do I list all files of a directory?
But again you'll likely need some educated guesses and will need to do some OS specific things to find it.
The Chrome binary is often installed in the same set of locations

I solved this problem by using C#. Like I already mentioned, we can directly call programs like "chrome.exe" in C#. I created a console application in C# which open Google Chrome via using bellow code. Then I complied that console application as .exe, after that I link that exe file with my python program.
C#
Process ExternalProcess = new Process();
ExternalProcess.StartInfo.FileName = "chrome.exe";
ExternalProcess.StartInfo.WindowStyle = ProcessWindowStyle.Maximized;
ExternalProcess.Start();
ExternalProcess.WaitForExit();
Python
import os
os.system('console_application.exe')

Is it possible to access the launching shortcut directory from a Python executalbe?

I currently have a Python scrip that runs through all Excel files in the current directory and generates a PDF report.
It works fine now but I don't want the users to be anywhere near frozen Python scripts. I created an MSI with cxFreeze which puts the EXE and scripts in the Program Files directory.
What I would like to be able to do is create a shortcut to this executable and pass the directory the shortcut was run from to the Python program so that can be set as the working directory. This would allow the user to move the shortcut to any folder of Excel files and generate a report there.
Does Windows send the location of a opened shortcut to the executable and is there a way to access it from Python?

When you launch a shortcut, Windows changes the working directory to the directory specified in the shortcut, in the Start in field. At this point, Windows has no memory of where the shortcut was stored.
You could change the Start in field to point to the directory that the shortcut is in. But you'd have to do that for every single shortcut, and never make a mistake.
The better approach is to use a script, rather than a shortcut. Place your actual Python script (which we'll call doit.py for sake of example) somewhere in your PYTHONPATH. Then create a single-line Python script that imports it:
import doit
Save it (but don't name it doit.py) and copy it to each directory from which you want to be able to invoke the main script. In doit.py you can use os.getcwd() to find out what directory you're being invoked from.
You could also do it with a batch file. This is a little more flexible in that you can specify the exact name of the script and which Python interpreter should be used, and don't need to store the script in a directory in PYTHONPATH. Also, you don't need to worry about the file's name clashing with the name of a Python module. Simply put this line in a file:
C:\path\to\your\python.exe C:\path\to\your\script.py
Save it as (e.g.) doit.bat and copy it into the directories from which you want to invoke it. As before, your Python script can call os.getcwd() to get the directory. Or you can write it so your Python script accepts it as the first argument, and write your batch file like:
C:\path\to\your\python.exe C:\path\to\your\script.py %cd%
Another thing you can do with the batch file approach is add a pause command to the end so that the user is asked to press a key after the script runs, giving them the opportunity to read any output generated by the script. You could even make this conditional so that it only happens if an error occurs (which requires returning a proper exit code from the script). I'll leave that as an exercise. :-)

Is there a problem with modifying the script to take the directory to process as a command line argument?
You could then configure the different shortcuts to pass in the appropriate directory.

Type the following into a batch file (i.e. script.bat):
python \absolute\path\to\your\script.py %~dp0
pause
Then add these imports at the top of your python file script.py (if not already included):
import os
import sys
And add this to the bottom of the python file (or combine it with a similar statement):
if __name__ == "__main__":
# set current working directory:
if len(sys.argv) > 1:
os.chdir(sys.argv[1])
main()
replace main() with whatever function you want to call or code you want to run.
The following is how I came to my answer:
I tried using kindall's answer and had the following issues:
The first suggestion of storing the script somewhere in PYTHONPATH could not be applied to my situation because my script will be used on a server and needs to be independent of the client computer's python environment (besides having the required pip installations).
I tried calling my python script from a Windows Batch File which could be moved to a different location. Instead of the batch file's location being used as the current working directory, it was C:\Windows.
I tried passing %cd% as an argument to my python script, then setting that to be my CWD. This still resulted in a CWD of C:\Windows.
After reviewing the comments, I tried Eryk Sun's suggestion of instead passing %~dp0 as an argument to the python script. This resulted in the CWD being correctly set to the batch file's location.
I hope this helps others facing similar difficulties.

How to go one or more levels using os.chdir with '..''s as arguments

How do I change path in Python using '..' to go down one or more levels as argument to os.chdir(). So that if I am on /home/usr/one and I want to go to 'home' directory a '../..' argument to chdir will do it. Do I wrap the argument in some other function?

As you say in your question, if you are in the directory /home/usr/one os.chdir('../../') will bring you to /home/.
You can confirm this by calling:
os.getcwd()
Before and after changing directories. This function will show you the current working directory. Also, there is no need to wrap the argument to chdir() in another function.
Edit:
Note that os.chdir() in a script will not change the directory you are in when you run the script from the terminal. In other words, if you are in /home/usr/one and run a script with python myscript.py, any directory changes made with os.chdir() within that script will not be refelected when the script finishes; you will still be in /home/usr/one.