Develop Reference

SVN commands not accepted via Python subprocess module

I can enter SVN commands successfully in the Windows command line, but when I try to pass them via subprocess with shell=True I get this error:
'svn' is not recognized as an internal or external command, operable program or batch file
When I omit the shell argument I get this:
WindowsError: [Error 2] The system cannot find the file specified
I'm running Python 2.7 on Windows 10, where I also have Python 3.8 installed. I've tried a variety of SVN commands, some complex and some simple, with a variety of arguments, both as a single string and a list of strings, both in IDLE and Spyder (console and script in each), and keep getting the same results. I am able to pass other types of Windows commands via subprocess, just not SVN. I've confirmed that the COMSPEC environment variable is correct. I've also tried moving the path to svn.exe all the way up in the PATH environment variable and rebooting. No dice.
Here's an example of what I'm trying to do:
import subprocess
my_cmd = ['svn', 'propget', 'svn:externals', '-R', '"https://the/rest/of/the/url"']
res = subprocess.check_output(my_cmd, shell=True)
print "the result of the svn command via subprocess is...\n{}".format(res)

After finding that the path to svn.exe was missing from the string returned by os.environ.get('PATH'), I added this path with the following line:
os.environ['PATH'] += r"C:\Program Files\TortoiseSVN\bin;"
And now my subsequent SVN commands are working via subprocess.
Thank you #John Gordon and #Maurice Meyer for the help!

Python subprocess.run('ls',shell=True) not working on windows

import subprocess
subprocess.call('ls', shell=True)
Output:
'ls' is not recognized as an internal or external command, operable program or batch file.

ls is not a Windows command. It works on Unixes. Its counterpart on Windows is dir.
Try iy out:
import subprocess
subprocess.call('dir', shell=True)
If, for some arcane reason, you have to call ls there is a bunch of ways to do so.
Firstly, Windows PowerShell supports calling ls, you just have to tell Python to execute it (the path below is valid on my system):
subprocess.call(r'c:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe ls', shell=True)
Alternatively, Windows 10 now supports an interoperability layer between Windows and Linux, which allows to use Linux's environment on Windows (called Windows Subsystem for Linux). If you have it installed, one of the way to use it is to precede Linux command with wsl:
subprocess.call('wsl ls', shell=True)
Lastly, the most universal way to list the directory would involve using the built-in Python functionality. E.g the following would give you the content of the current directory:
import os
os.listdir('.')

If your goal is to run (Git) Bash on Windows as your shell for subprocess.run() and friends, you'll need to pass in an executable= keyword argument to point to its location.
import subprocess
listing = subprocess.check_output(
'ls', shell=True, executable='c:/Program Files (x86)/Git/bin/bash.exe')
(where obviously you will need to make sure the file name is correct; I'm guessing wildly here).
As an aside, you should generally avoid ls in scripts. Python has good facilities for getting information about file size, ownership, etc via the pathlib module, or look in the os module if you need to support really old versions of Python (and of course glob.glob() for wildcard expansion, though ls is useless for that too).
On Linux (or Unix-like platforms generally) this is also necessary if you want to use Bash-specific shell syntax like arrays, process substitution, brace expansion, here strings, etc etc etc. The location of Bash is often /bin/bash for the system-installed version, though it's not entirely uncommon to have it in /usr/bin/bash, or somewhere like /usr/local/bin/bash or somewhere in /opt for a custom-installed version.
p = subprocess.run(
# Need bash for '{...}' and '<<<'
'cat /tmp/{one,two} - <<<"three"',
shell=True, executable='/bin/bash',
capture_output=True, check=True)
For merely running a single executable, you usually want to avoid shell=True though you'll need to specify the full path to the executable if it's not on your PATH.
subprocess.call(['/bin/ls'])
(where obviously with Git Bash on Windows, the actual directory where ls.exe is installed will be something a lot longer and more godforsaken).

Python Subprocess Popen Prepare Python Path under Linux

I am trying to prepare my pythonpath under Ubuntu via subprocess.Popen call, for another script. The call to Python estimateskeleton.py does work fine. However since it needs the python path to be prepared it doesn't work completely correct, since it can not find some other scripts which need to be imported. The export PYTHONPATH command did work with commands.getoutput. However with commands.getoutput the estimateskeleton script still doesn't work / can't find the other files which should be imported. My try to export PYTHONPATH via subprocess.Popen resulted in Error Number 2:
OSError: [Errno 2] No such file or directory
I couldn't find a proper solution with the search function. So I am hoping that one of the more advanced users of this board can help me
Best Regards
import subprocess as sub
import os
import commands
proc = sub.Popen(["export", "PYTHONPATH=\"${PYTHONPATH}:/media/sf_myFolder/Scripts/code/\""],
stdout=sub.PIPE,
stderr=sub.STDOUT)
print proc.communicate()[0]
proc2 = sub.Popen(["python", "estimateskeleton.py"],
stdout=sub.PIPE,
stderr=sub.STDOUT)
print proc2.communicate()[0]

your first Popen command would work without shell=True because export is a shell built-in.
However, that won't fix it, because the second process spawned by Popen is unaware of the previous variable set in a dead process.
So instead of running the first useless Popen, you could add your path to existing PYTHONPATH using os.putenv() like this:
os.putenv("PYTHONPATH",os.pathsep.join([os.getenv("PYTHONPATH",""),"/media/sf_myFolder/Scripts/code"]))
so your next python command is run with the added folder in PYTHONPATH

Python: Subprocess "call" function can't find path

I have a python program that needs to run a bash command on a server. The program works when I run a bash command in my local directory like this:
import subprocess
from subprocess import call
call(['bash', 'Script_Test.sh'])
However, after SSH'ing to a server and running a similar line of code below with a path on the server to a bash script, I get the error "No such file or directory"
call['bash', path]
This doesn't make sense for a number of reasons. I triple checked that the path is correct. I went on Putty, connected to the server on there, and ran a bash command with the same path and it worked, so it can't be the path. I also ran a number of tests to make sure I was SSH'd into the correct server, and I was. I thought there was a security issue on the server with me running bash, so I tried cat instead. Nope, still unable to locate the path.
I'm not super familiar with python subprocesses, so any pointers to anything I'm missing here with "call" would be very helpful.

Making Sure Your Script is Ready for Execution
Give your script a shebang line
First things first, it is important that you include a shebang line in your script on Unix-like systems. I recommend, for your script's portability, that you use #!/usr/bin/env bash.
A Note on File Extensions:
I would recommend that you remove the .sh extension from your script. If you are using the Bourne Again Shell (bash) to execute your script, then using the .sh extension is misleading. Simply put, the Bourne Shell (sh) is different than the Bourne Again Shell (bash) - so don't use a file extension that suggests you are using a different shell than you actually are!
It's not the end of the world if you don't do change your file extension - your script will still be executed as a bash script if you have the proper bash shebang line. Still, it is good practice to either use no file extension (Script_Test -- strongly preferred) or the .bash file extension (Script_Test.bash).
Give your script the proper file permissions
For your purposes, maybe it is only important to give the current user permissions to read and execute the script. In that case, use chmod u+x Script_Test.sh. What is important here is that the correct user (u+) / group (g+) has permissions to execute the script.
Make sure that your script's path is in the $PATH environment variable
Executing your bash script in a Python script
Once you've followed these steps, your Python script should work as you've called it in your question:
import subprocess
from subprocess import call
your_call = call("Test_Script.sh")
If you would rather not move your script into the $PATH environment variable, just make sure that you refer to the script's full path (which is the current directory, ./, in your case):
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh")
Lastly, if your script does not have a shebang line, you will need to specify an additional parameter in your call function:
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh", shell=True)
However, I would not recommend this last approach. See the Python 2.7.12 documentation for the subprocess package...
Warning: Using shell=True can be a security hazard. See the warning under Frequently Used Arguments for details.
Please check out #zenpoy's explanation to a similar StackOverflow question if you are still having problems.
Happy coding!

Windows can't find the file on subprocess.call()

I am getting the following error:
WindowsError: [Error 2] The system cannot find the file specified
My code is:
subprocess.call(["<<executable file found in PATH>>"])
Windows 7, 64 bit. Python 3.x latest, stable.
Any ideas?
Thanks,

When the command is a shell built-in, add a shell=True to the call.
E.g. for dir you would type:
import subprocess
subprocess.call('dir', shell=True)
To quote from the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.

On Windows, I believe the subprocess module doesn't look in the PATH unless you pass shell=True because it use CreateProcess() behind the scenes. However, shell=True can be a security risk if you're passing arguments that may come from outside your program. To make subprocess nonetheless able to find the correct executable, you can use shutil.which. Suppose the executable in your PATH is named frob:
subprocess.call([shutil.which('frob'), arg1, arg2])
(This works on Python 3.3 and above.)

On Windows you have to call through cmd.exe. As Apalala mentioned, Windows commands are implemented in cmd.exe not as separate executables.
e.g.
subprocess.call(['cmd', '/c', 'dir'])
/c tells cmd to run the follow command
This is safer than using shell=True, which allows shell injections.

If you are using powershell, then in it will be subprocess.call(['powershell','-command','dir']). Powershell supports a large portion of POSIX commands

After much head scratching, I discovered that running a file that is located in C:\Windows\System32\ while running a 32bit version of python on a 64bit machine is a potential issue, due to Windows trying to out-smart the process, and redirect calls to C:\Windows\System32 to C:\Windows\SysWOW64.
I found an example of how to fix this here:
http://code.activestate.com/recipes/578035-disable-file-system-redirector/

To quote from the documentation:
"Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions."
SO: instead of subprocess.call use subprocess.run for Python 3.5 and above

I met the same issue while I was calling a PHP. The reason is PHP isn't in PATH so the command PHP was not found. But PowerShell found it does exist in the current location and it suggests replacing the 'PHP' by the '.\PHP' if I trust this command. Then it runs well.