I am getting the following error:
WindowsError: [Error 2] The system cannot find the file specified
My code is:
subprocess.call(["<<executable file found in PATH>>"])
Windows 7, 64 bit. Python 3.x latest, stable.
Any ideas?
Thanks,
When the command is a shell built-in, add a shell=True to the call.
E.g. for dir you would type:
import subprocess
subprocess.call('dir', shell=True)
To quote from the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
On Windows, I believe the subprocess module doesn't look in the PATH unless you pass shell=True because it use CreateProcess() behind the scenes. However, shell=True can be a security risk if you're passing arguments that may come from outside your program. To make subprocess nonetheless able to find the correct executable, you can use shutil.which. Suppose the executable in your PATH is named frob:
subprocess.call([shutil.which('frob'), arg1, arg2])
(This works on Python 3.3 and above.)
On Windows you have to call through cmd.exe. As Apalala mentioned, Windows commands are implemented in cmd.exe not as separate executables.
e.g.
subprocess.call(['cmd', '/c', 'dir'])
/c tells cmd to run the follow command
This is safer than using shell=True, which allows shell injections.
If you are using powershell, then in it will be subprocess.call(['powershell','-command','dir']). Powershell supports a large portion of POSIX commands
After much head scratching, I discovered that running a file that is located in C:\Windows\System32\ while running a 32bit version of python on a 64bit machine is a potential issue, due to Windows trying to out-smart the process, and redirect calls to C:\Windows\System32 to C:\Windows\SysWOW64.
I found an example of how to fix this here:
http://code.activestate.com/recipes/578035-disable-file-system-redirector/
To quote from the documentation:
"Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions."
SO: instead of subprocess.call use subprocess.run for Python 3.5 and above
I met the same issue while I was calling a PHP. The reason is PHP isn't in PATH so the command PHP was not found. But PowerShell found it does exist in the current location and it suggests replacing the 'PHP' by the '.\PHP' if I trust this command. Then it runs well.
Related
I'm working on using the subprocess module to send shell commands from Python, specifically, ssh. Below is a barebones sample:
import subprocess
sp = subprocess.run(["ssh"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"stdout: {sp.stdout.decode()} \n\nstderr: {sp.stderr.decode()}")
This should return the ssh command help from stdout, and nothing from stderr. However, I get:
stdout:
stderr: 'ssh' is not recognized as an internal or external command,
operable program or batch file.
I've tried other commands, like echo and cd, and those work fine. I am also able to use ssh when manually typing the command into the shell, but it fails when I try to do it through subprocess. The directory C:\Windows\System32\OpenSSH does exist on my computer (and it contains ssh.exe), but for some strange reason I'm unable to cd to it using subprocess.
If it matters, subprocess is using the command prompt, cmd.exe, as it seems to be the default.
Any help is appreciated. Thanks!
-- Edits with tests from comments --
Using the absolute path C:/Windows/System32/OpenSSH/ssh.exe does not work, and gives The system cannot find the path specified via stderr. The OpenSSH folder doesn't seem to be visible to Python through subprocess
os.environ[PATH] contains both C:/Windows/System32/ and C:/Windows/System32/OpenSSH/
Running it with shell=False (either with the absolute path or just with ssh) raises an error in Python: FileNotFoundError: [WinError 2] The system cannot find the file specified
You say C:\Windows\System32\OpenSSH\ssh.exe exists, but that it's not found when running from Python. This is likely a result of having a 32 bit version of Python installed, rather than a 64 bit version.
If the path exists elsewhere, but not for Python, that would tend to implicate the file system redirector. Python is probably seeing C:\Windows\SysWOW64 when you tell it to look in C:\Windows\System32. I'd recommend uninstalling whatever Python you have, and explicitly installing a 64 bit version, so it isn't affected by the redirector, and sees the "real" System32.
import subprocess
subprocess.call('ls', shell=True)
Output:
'ls' is not recognized as an internal or external command, operable program or batch file.
ls is not a Windows command. It works on Unixes. Its counterpart on Windows is dir.
Try iy out:
import subprocess
subprocess.call('dir', shell=True)
If, for some arcane reason, you have to call ls there is a bunch of ways to do so.
Firstly, Windows PowerShell supports calling ls, you just have to tell Python to execute it (the path below is valid on my system):
subprocess.call(r'c:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe ls', shell=True)
Alternatively, Windows 10 now supports an interoperability layer between Windows and Linux, which allows to use Linux's environment on Windows (called Windows Subsystem for Linux). If you have it installed, one of the way to use it is to precede Linux command with wsl:
subprocess.call('wsl ls', shell=True)
Lastly, the most universal way to list the directory would involve using the built-in Python functionality. E.g the following would give you the content of the current directory:
import os
os.listdir('.')
If your goal is to run (Git) Bash on Windows as your shell for subprocess.run() and friends, you'll need to pass in an executable= keyword argument to point to its location.
import subprocess
listing = subprocess.check_output(
'ls', shell=True, executable='c:/Program Files (x86)/Git/bin/bash.exe')
(where obviously you will need to make sure the file name is correct; I'm guessing wildly here).
As an aside, you should generally avoid ls in scripts. Python has good facilities for getting information about file size, ownership, etc via the pathlib module, or look in the os module if you need to support really old versions of Python (and of course glob.glob() for wildcard expansion, though ls is useless for that too).
On Linux (or Unix-like platforms generally) this is also necessary if you want to use Bash-specific shell syntax like arrays, process substitution, brace expansion, here strings, etc etc etc. The location of Bash is often /bin/bash for the system-installed version, though it's not entirely uncommon to have it in /usr/bin/bash, or somewhere like /usr/local/bin/bash or somewhere in /opt for a custom-installed version.
p = subprocess.run(
# Need bash for '{...}' and '<<<'
'cat /tmp/{one,two} - <<<"three"',
shell=True, executable='/bin/bash',
capture_output=True, check=True)
For merely running a single executable, you usually want to avoid shell=True though you'll need to specify the full path to the executable if it's not on your PATH.
subprocess.call(['/bin/ls'])
(where obviously with Git Bash on Windows, the actual directory where ls.exe is installed will be something a lot longer and more godforsaken).
I have a python program that needs to run a bash command on a server. The program works when I run a bash command in my local directory like this:
import subprocess
from subprocess import call
call(['bash', 'Script_Test.sh'])
However, after SSH'ing to a server and running a similar line of code below with a path on the server to a bash script, I get the error "No such file or directory"
call['bash', path]
This doesn't make sense for a number of reasons. I triple checked that the path is correct. I went on Putty, connected to the server on there, and ran a bash command with the same path and it worked, so it can't be the path. I also ran a number of tests to make sure I was SSH'd into the correct server, and I was. I thought there was a security issue on the server with me running bash, so I tried cat instead. Nope, still unable to locate the path.
I'm not super familiar with python subprocesses, so any pointers to anything I'm missing here with "call" would be very helpful.
Making Sure Your Script is Ready for Execution
Give your script a shebang line
First things first, it is important that you include a shebang line in your script on Unix-like systems. I recommend, for your script's portability, that you use #!/usr/bin/env bash.
A Note on File Extensions:
I would recommend that you remove the .sh extension from your script. If you are using the Bourne Again Shell (bash) to execute your script, then using the .sh extension is misleading. Simply put, the Bourne Shell (sh) is different than the Bourne Again Shell (bash) - so don't use a file extension that suggests you are using a different shell than you actually are!
It's not the end of the world if you don't do change your file extension - your script will still be executed as a bash script if you have the proper bash shebang line. Still, it is good practice to either use no file extension (Script_Test -- strongly preferred) or the .bash file extension (Script_Test.bash).
Give your script the proper file permissions
For your purposes, maybe it is only important to give the current user permissions to read and execute the script. In that case, use chmod u+x Script_Test.sh. What is important here is that the correct user (u+) / group (g+) has permissions to execute the script.
Make sure that your script's path is in the $PATH environment variable
Executing your bash script in a Python script
Once you've followed these steps, your Python script should work as you've called it in your question:
import subprocess
from subprocess import call
your_call = call("Test_Script.sh")
If you would rather not move your script into the $PATH environment variable, just make sure that you refer to the script's full path (which is the current directory, ./, in your case):
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh")
Lastly, if your script does not have a shebang line, you will need to specify an additional parameter in your call function:
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh", shell=True)
However, I would not recommend this last approach. See the Python 2.7.12 documentation for the subprocess package...
Warning: Using shell=True can be a security hazard. See the warning under Frequently Used Arguments for details.
Please check out #zenpoy's explanation to a similar StackOverflow question if you are still having problems.
Happy coding!
I am trying to open a .txt file in Windows. The code is as follows:
subprocess.Popen("C:\folder\file.txt", shell=True)
This works perfectly fine. The default editor is automatically opened and the file is loaded, however, I have read somewhere before that invoking calls through the shell (cmd.exe in Windows) is less safe. How can I do the same without it. Simply setting shell=False is giving me the error:
OSError: [WinError 193] %1 is not a valid Win32 application
Now, I can try this as a workaround:
subprocess.Popen("notepad C:\folder\file.txt")
but this only works if notepad is available, hence loses its generality.
If you are using windows then there is the (non portable) os.startfile command which will take a file path and open it in the default application for that filetype.
In your case you would do:
import os
os.startfile("C:\folder\file.txt")
Note that this method won't work on Linux and Mac OSX, you'll have to have to use their own utilities for this. (open for OSX and xdg-open on Linux) and start them with subprocess.
The feature you try to use is a builtin thing of the windows cmd.exe. Therefore you need to set the shell=True parameter. The cmd.exe knows what to do with the file you hand in.
If you use shell=False, you try to start the file like a programm and hence nothing happens, since .txt files have no exe-header.
Read more about it in the documentation.
Reading further, you can find why using the shell=True parameter can be a security flaw. If you consider to assemble the parameters by user inputs, you should not use this, otherwise nothing speaks against it.
Anyway, I recommend using your second example, because it is explicit. You decide what program to start.
subprocess.Popen("notepad C:\folder\file.txt")
So as the title says, I'm having problem starting a new subprocess under Fedora. Now the situation is, I have a main python script from which I start a couple of other python processes using:
import subprocess
subprocess.Popen(['python', '-m', 'first_child.run', 'start'], shell=False)
Now this works fine on MacOS, debian and windows. On fedora if I run it from Aptana 3 IDE it also works, the only problem is when i try to run this main scrip from a terminal, where I get:
OSError: [Errno 2] No such file or directory
Do you have any ideea what can be the problem here?
Regards,
Bogdan
Sorry if this is something you've already thought of -- but the most common cause of OSError from calls to subprocess is that it cannot find the process
http://docs.python.org/library/subprocess.html#exceptions
Are you absolutely certain python is in your path?
I know you're probably going to point out you ran this script from the python executable -- but I thought I'd take a shot that perhaps you specified the full path to python when you ran it from the terminal.
For fun, right before the call to subprocess, you could dump your PATH
import os
print os.environ['PATH']
It's your current working directory. I don't think the problem is that it can't find python, the problem is that it can't find first_child.run.
Try printing os.getcwd() before you launch the subprocess, and see if it's different in the terminal vs. in the IDE.
On a side note, it's probably more reliable to use sys.executable as the python you use in your subprocess, as opposed to just saying python. For example, subprocess.Popen([sys.executable, '-m', 'first_child.run', 'start'], shell=False)