Develop Reference

Python subprocess can't call "ssh"

I'm working on using the subprocess module to send shell commands from Python, specifically, ssh. Below is a barebones sample:
import subprocess
sp = subprocess.run(["ssh"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"stdout: {sp.stdout.decode()} \n\nstderr: {sp.stderr.decode()}")
This should return the ssh command help from stdout, and nothing from stderr. However, I get:
stdout:
stderr: 'ssh' is not recognized as an internal or external command,
operable program or batch file.
I've tried other commands, like echo and cd, and those work fine. I am also able to use ssh when manually typing the command into the shell, but it fails when I try to do it through subprocess. The directory C:\Windows\System32\OpenSSH does exist on my computer (and it contains ssh.exe), but for some strange reason I'm unable to cd to it using subprocess.
If it matters, subprocess is using the command prompt, cmd.exe, as it seems to be the default.
Any help is appreciated. Thanks!
-- Edits with tests from comments --
Using the absolute path C:/Windows/System32/OpenSSH/ssh.exe does not work, and gives The system cannot find the path specified via stderr. The OpenSSH folder doesn't seem to be visible to Python through subprocess
os.environ[PATH] contains both C:/Windows/System32/ and C:/Windows/System32/OpenSSH/
Running it with shell=False (either with the absolute path or just with ssh) raises an error in Python: FileNotFoundError: [WinError 2] The system cannot find the file specified

You say C:\Windows\System32\OpenSSH\ssh.exe exists, but that it's not found when running from Python. This is likely a result of having a 32 bit version of Python installed, rather than a 64 bit version.
If the path exists elsewhere, but not for Python, that would tend to implicate the file system redirector. Python is probably seeing C:\Windows\SysWOW64 when you tell it to look in C:\Windows\System32. I'd recommend uninstalling whatever Python you have, and explicitly installing a 64 bit version, so it isn't affected by the redirector, and sees the "real" System32.

Python Subprocess Popen Prepare Python Path under Linux

I am trying to prepare my pythonpath under Ubuntu via subprocess.Popen call, for another script. The call to Python estimateskeleton.py does work fine. However since it needs the python path to be prepared it doesn't work completely correct, since it can not find some other scripts which need to be imported. The export PYTHONPATH command did work with commands.getoutput. However with commands.getoutput the estimateskeleton script still doesn't work / can't find the other files which should be imported. My try to export PYTHONPATH via subprocess.Popen resulted in Error Number 2:
OSError: [Errno 2] No such file or directory
I couldn't find a proper solution with the search function. So I am hoping that one of the more advanced users of this board can help me
Best Regards
import subprocess as sub
import os
import commands
proc = sub.Popen(["export", "PYTHONPATH=\"${PYTHONPATH}:/media/sf_myFolder/Scripts/code/\""],
stdout=sub.PIPE,
stderr=sub.STDOUT)
print proc.communicate()[0]
proc2 = sub.Popen(["python", "estimateskeleton.py"],
stdout=sub.PIPE,
stderr=sub.STDOUT)
print proc2.communicate()[0]

your first Popen command would work without shell=True because export is a shell built-in.
However, that won't fix it, because the second process spawned by Popen is unaware of the previous variable set in a dead process.
So instead of running the first useless Popen, you could add your path to existing PYTHONPATH using os.putenv() like this:
os.putenv("PYTHONPATH",os.pathsep.join([os.getenv("PYTHONPATH",""),"/media/sf_myFolder/Scripts/code"]))
so your next python command is run with the added folder in PYTHONPATH

Python subprocess does not work

I try to use a program called bwa in python with the subprocess module:
subprocess.call('bwa', shell=True)
Result is:
/bin/sh: bwa: command not found
However if I start a terminal and type the command, the program starts (i.e. I get the Usage dialogue).
The binaries of bwa are in a folder on my desktop. In .bash_profile I have modified the path variable so that everytime I type 'bwa' in the terminal, the program binaries are found in my folder.
My assumption is that the subprocess does not start a shell with my bash_profile settings ... but if that is correct why wouldn't it do so and what can I do about it?

Struggling with Python's subprocess.Popen running a Perl script

I have a Perl script that I need to run with Python, and I've been trying to use subprocess to do it, unsuccessfully. I'm able to get the command to run just fine on the command line, but subprocess isn't able to get it to work.
If I have a Perl script like this:
#!/usr/bin/perl
use strict;
use warnings;
my $name = shift;
print "Hello $name!\n";
I am able to successfully run the command on the command line like so
C:\current\path> perl test.pl world
>>>Hello world!
But when I try and invoke the same command with subprocess, I get this error
cmd = 'perl test.pl world'
pipe = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)
>>>"'perl' is not recognized as an internal or external command, operable program or batch file."
I've tried different combinations of creating cmd as a list
cmd = ['perl','test.pl','world']
and of giving subprocess the absolute path to Perl
cmd = ['C:\path\to\perl.exe','test.pl','world']
but nothing gets it to work. I'm able to get subprocess to play well without Perl
pipe = subprocess.Popen('dir',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()
>>>Volume in drive C is Windows7_OS....
I'm absolutely sure that Perl is in my PATH, because like I said I can invoke the command just fine in the command line. I've looked around at different posts that suggested to check os.environ["COMSPEC"], adding Perl to the subprocess env, and nothing has worked.
EDIT: I also can't get the command to work with any other subprocess methods: check_output() returns an empty byte string, and call() returns 1.
Any sort of fix or alternative solution would be immensely appreciated. I'm running Python 3.4 on Windows 7 64bit. I've tried 32- and 64-bit Python as well, to no avail.
UPDATE:
I've been able to get this to work on Mac, but with one difference: I can't have shell=True. Other than that, any subprocess function I want will work, including Popen, check_output, and call.
So I guess this is a Windows problem more than anything. I've tried not setting shell=True on my Windows machine, but I always get this error:
WindowsError: [Error 193] %1 is not a valid Win32 application
UPDATE 2:
I've also tried creating a .bat file that runs the command, but I get the same error as well I try to just call Perl.
pipe = subprocess.Popen('test.bat',shell=True,stdout=subprocess.PIPE)
>>>"'test.bat' is not recognized as an internal or external command, operable program or batch file."
Where test.bat has only one line:
echo Hello, World

you could try using os.system('command') instead. Apparently its not well regarded for some reason, or at least not as well regarded as subprocess though. Alternately, you could try some of the other subprocess methods, such as call, check_call, or check_output. I've had a similar problem with Popen in the past and switching to one of these methods helped.

I got the same error while trying to run a perl script from Python.
I managed to get it work by adding the full path of the perl.exe (with an extra escape character, as '\') to the command I was trying to run as below:
pipe = subprocess.Popen('C:\\Perl64\\bin\\perl myscrip.pl',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()

This works fine for me. I know it's been a while and things may have just fixed themselves.
I'm using activestate perl 5.8.9
and python 3.9.5 on Windows 10.
from subprocess import Popen, PIPE
proc = Popen(['perl', 'test.pl', 'world'], stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
if proc.returncode == 0:
print(stdout.decode())
else:
print(stderr.decode())

Windows can't find the file on subprocess.call()

I am getting the following error:
WindowsError: [Error 2] The system cannot find the file specified
My code is:
subprocess.call(["<<executable file found in PATH>>"])
Windows 7, 64 bit. Python 3.x latest, stable.
Any ideas?
Thanks,

When the command is a shell built-in, add a shell=True to the call.
E.g. for dir you would type:
import subprocess
subprocess.call('dir', shell=True)
To quote from the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.

On Windows, I believe the subprocess module doesn't look in the PATH unless you pass shell=True because it use CreateProcess() behind the scenes. However, shell=True can be a security risk if you're passing arguments that may come from outside your program. To make subprocess nonetheless able to find the correct executable, you can use shutil.which. Suppose the executable in your PATH is named frob:
subprocess.call([shutil.which('frob'), arg1, arg2])
(This works on Python 3.3 and above.)

On Windows you have to call through cmd.exe. As Apalala mentioned, Windows commands are implemented in cmd.exe not as separate executables.
e.g.
subprocess.call(['cmd', '/c', 'dir'])
/c tells cmd to run the follow command
This is safer than using shell=True, which allows shell injections.

If you are using powershell, then in it will be subprocess.call(['powershell','-command','dir']). Powershell supports a large portion of POSIX commands

After much head scratching, I discovered that running a file that is located in C:\Windows\System32\ while running a 32bit version of python on a 64bit machine is a potential issue, due to Windows trying to out-smart the process, and redirect calls to C:\Windows\System32 to C:\Windows\SysWOW64.
I found an example of how to fix this here:
http://code.activestate.com/recipes/578035-disable-file-system-redirector/

To quote from the documentation:
"Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions."
SO: instead of subprocess.call use subprocess.run for Python 3.5 and above

I met the same issue while I was calling a PHP. The reason is PHP isn't in PATH so the command PHP was not found. But PowerShell found it does exist in the current location and it suggests replacing the 'PHP' by the '.\PHP' if I trust this command. Then it runs well.