import subprocess
subprocess.call('ls', shell=True)
Output:
'ls' is not recognized as an internal or external command, operable program or batch file.
ls is not a Windows command. It works on Unixes. Its counterpart on Windows is dir.
Try iy out:
import subprocess
subprocess.call('dir', shell=True)
If, for some arcane reason, you have to call ls there is a bunch of ways to do so.
Firstly, Windows PowerShell supports calling ls, you just have to tell Python to execute it (the path below is valid on my system):
subprocess.call(r'c:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe ls', shell=True)
Alternatively, Windows 10 now supports an interoperability layer between Windows and Linux, which allows to use Linux's environment on Windows (called Windows Subsystem for Linux). If you have it installed, one of the way to use it is to precede Linux command with wsl:
subprocess.call('wsl ls', shell=True)
Lastly, the most universal way to list the directory would involve using the built-in Python functionality. E.g the following would give you the content of the current directory:
import os
os.listdir('.')
If your goal is to run (Git) Bash on Windows as your shell for subprocess.run() and friends, you'll need to pass in an executable= keyword argument to point to its location.
import subprocess
listing = subprocess.check_output(
'ls', shell=True, executable='c:/Program Files (x86)/Git/bin/bash.exe')
(where obviously you will need to make sure the file name is correct; I'm guessing wildly here).
As an aside, you should generally avoid ls in scripts. Python has good facilities for getting information about file size, ownership, etc via the pathlib module, or look in the os module if you need to support really old versions of Python (and of course glob.glob() for wildcard expansion, though ls is useless for that too).
On Linux (or Unix-like platforms generally) this is also necessary if you want to use Bash-specific shell syntax like arrays, process substitution, brace expansion, here strings, etc etc etc. The location of Bash is often /bin/bash for the system-installed version, though it's not entirely uncommon to have it in /usr/bin/bash, or somewhere like /usr/local/bin/bash or somewhere in /opt for a custom-installed version.
p = subprocess.run(
# Need bash for '{...}' and '<<<'
'cat /tmp/{one,two} - <<<"three"',
shell=True, executable='/bin/bash',
capture_output=True, check=True)
For merely running a single executable, you usually want to avoid shell=True though you'll need to specify the full path to the executable if it's not on your PATH.
subprocess.call(['/bin/ls'])
(where obviously with Git Bash on Windows, the actual directory where ls.exe is installed will be something a lot longer and more godforsaken).
Related
I'm working on using the subprocess module to send shell commands from Python, specifically, ssh. Below is a barebones sample:
import subprocess
sp = subprocess.run(["ssh"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"stdout: {sp.stdout.decode()} \n\nstderr: {sp.stderr.decode()}")
This should return the ssh command help from stdout, and nothing from stderr. However, I get:
stdout:
stderr: 'ssh' is not recognized as an internal or external command,
operable program or batch file.
I've tried other commands, like echo and cd, and those work fine. I am also able to use ssh when manually typing the command into the shell, but it fails when I try to do it through subprocess. The directory C:\Windows\System32\OpenSSH does exist on my computer (and it contains ssh.exe), but for some strange reason I'm unable to cd to it using subprocess.
If it matters, subprocess is using the command prompt, cmd.exe, as it seems to be the default.
Any help is appreciated. Thanks!
-- Edits with tests from comments --
Using the absolute path C:/Windows/System32/OpenSSH/ssh.exe does not work, and gives The system cannot find the path specified via stderr. The OpenSSH folder doesn't seem to be visible to Python through subprocess
os.environ[PATH] contains both C:/Windows/System32/ and C:/Windows/System32/OpenSSH/
Running it with shell=False (either with the absolute path or just with ssh) raises an error in Python: FileNotFoundError: [WinError 2] The system cannot find the file specified
You say C:\Windows\System32\OpenSSH\ssh.exe exists, but that it's not found when running from Python. This is likely a result of having a 32 bit version of Python installed, rather than a 64 bit version.
If the path exists elsewhere, but not for Python, that would tend to implicate the file system redirector. Python is probably seeing C:\Windows\SysWOW64 when you tell it to look in C:\Windows\System32. I'd recommend uninstalling whatever Python you have, and explicitly installing a 64 bit version, so it isn't affected by the redirector, and sees the "real" System32.
I need to run a python command inside a Popen. The problem is that the command NEEDS to run in python3 and I need it to be portable, which means that I can't really use the python3 alias for every situation...
I have computers where python is already the correct version, and others where the correct one is python3. I tried to insert #!/usr/bin python3 in the beginning of the file and then run as python but it didn't work.
I can't modify environment vars to change python3 to python. I would like to know if there is a way to check which one I need to use or a way to change the python3 to python ONLY inside the Popen command...
The Popen command I am trying to run is very simple and no I can't just import the file and use as a class... it needs to be ran through Popen. Also, virtualenv or similars are not an option.
subprocess.Popen(['python', 'main.py'], shell=True, universal_newlines=True)
The shebang -- the initial line showing which interpreter to use, such as #!/usr/bin/python or #!/usr/bin/python3 -- is only honored if you don't explicitly select an interpreter yourself: If you run python foo.py, the OS is invoking a specific Python interpreter and passing it foo.py as an argument (which it interprets as the name of a script it should run); whereas when you run ./foo.py, you're telling the OS itself to figure out which interpreter to use to run foo.py, which it does by looking at the shebang.
To leave it up to the operating system to select, just explicitly specify the name of your script:
subprocess.Popen(['./main.py'], universal_newlines=True)
I'm using canopy python. The shell has commands "pwd" and "cd".
It even does autocomplete.
Is there a command - not something I have to write and keep track of - a built-in command that just lists the files in the directory.
I've tried ls, ls(), dir, dir(), directory, directory(), files, files(), filelist, filelist().
I've searched their documentation for listing files. It shows how to get a list of files in a python program - that's great. I do that all the time. I don't want to do that every time I want to list the contents of the current directory.
Is there a built-in shell command for this?
You can try
import os
fileList = os.listdir(directory_path)
so that you have the list of files in the python program to manipulate.
If you want the list of files in the current working directory in the way that ls would give you you can do.
fileList = os.listdir(os.getcwd())
If you really want a short command to type over and over you can write
ls = lambda : os.listdir(os.getcwd())
and then it will run this command when you type
ls()
You can use the python built in module, os, to query a directory using the system method. Please see below.
import os
os.system("dir")
Update
Please keep in mind that my solution is OS specific (i.e. ls vs dir) so Jon's solution is preferable.
The premise of the question seems inconsistent. If you can do the ipython pwd or cd standard magic commands, then you should also be able to access the ipython standard alias ls. If you cannot, then it almost certainly because you've overwritten / shadowed the meaning of the ls alias.
It's crucial to distinguish between (1) a plain Python shell (opened by typing python at a Canopy command prompt), which does not provide pwd nor cd nor 'lscommands; and (2) *any* IPython shell, whether the Python panel in the Canopy GUI or anipython` terminal in the Canopy Command Prompt, which does provide those, and many other Ipython "magic" commands.
(For more about IPython magic commands, see http://ipython.readthedocs.io/en/stable/interactive/magics.html; these are the docs for IPython version 6, but they are mostly the same for IPython version 5 which Canopy uses in order to support both Python 2.7 and Python 3.5+.)
The two previous answers are always correct in that they will work in any Python program or shell (depending on OS). That is because they only use the Python standard library. However they are not analogous to the cd and pwd commands that you mention, which are specific to the IPython shell (not to programs running in an IPython shell).
IMO, there is very seldom a good reason for opening a plain Python shell. If you simply want to run a script file from a Command Prompt, then by all means python myscript.py. But if you want an interactive shell, IPython provides so much extra capability that it has been for many years the de facto standard for Python shells (which is why it is used for the Canopy GUI's Python shell).
Got a note back from Canopy Support).
There is an environment variable, COMSPEC that was set to:
C:\windows\system32\cmd.exe;
That semicolon at the end is wrong. This is a single directory and not a list of directories. I removed that semicolon and suddenly the 'ls' command works as I remember!
I have a python program that needs to run a bash command on a server. The program works when I run a bash command in my local directory like this:
import subprocess
from subprocess import call
call(['bash', 'Script_Test.sh'])
However, after SSH'ing to a server and running a similar line of code below with a path on the server to a bash script, I get the error "No such file or directory"
call['bash', path]
This doesn't make sense for a number of reasons. I triple checked that the path is correct. I went on Putty, connected to the server on there, and ran a bash command with the same path and it worked, so it can't be the path. I also ran a number of tests to make sure I was SSH'd into the correct server, and I was. I thought there was a security issue on the server with me running bash, so I tried cat instead. Nope, still unable to locate the path.
I'm not super familiar with python subprocesses, so any pointers to anything I'm missing here with "call" would be very helpful.
Making Sure Your Script is Ready for Execution
Give your script a shebang line
First things first, it is important that you include a shebang line in your script on Unix-like systems. I recommend, for your script's portability, that you use #!/usr/bin/env bash.
A Note on File Extensions:
I would recommend that you remove the .sh extension from your script. If you are using the Bourne Again Shell (bash) to execute your script, then using the .sh extension is misleading. Simply put, the Bourne Shell (sh) is different than the Bourne Again Shell (bash) - so don't use a file extension that suggests you are using a different shell than you actually are!
It's not the end of the world if you don't do change your file extension - your script will still be executed as a bash script if you have the proper bash shebang line. Still, it is good practice to either use no file extension (Script_Test -- strongly preferred) or the .bash file extension (Script_Test.bash).
Give your script the proper file permissions
For your purposes, maybe it is only important to give the current user permissions to read and execute the script. In that case, use chmod u+x Script_Test.sh. What is important here is that the correct user (u+) / group (g+) has permissions to execute the script.
Make sure that your script's path is in the $PATH environment variable
Executing your bash script in a Python script
Once you've followed these steps, your Python script should work as you've called it in your question:
import subprocess
from subprocess import call
your_call = call("Test_Script.sh")
If you would rather not move your script into the $PATH environment variable, just make sure that you refer to the script's full path (which is the current directory, ./, in your case):
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh")
Lastly, if your script does not have a shebang line, you will need to specify an additional parameter in your call function:
import subprocess
from subprocess import call
your_call = call("./Test_Script.sh", shell=True)
However, I would not recommend this last approach. See the Python 2.7.12 documentation for the subprocess package...
Warning: Using shell=True can be a security hazard. See the warning under Frequently Used Arguments for details.
Please check out #zenpoy's explanation to a similar StackOverflow question if you are still having problems.
Happy coding!
I am getting the following error:
WindowsError: [Error 2] The system cannot find the file specified
My code is:
subprocess.call(["<<executable file found in PATH>>"])
Windows 7, 64 bit. Python 3.x latest, stable.
Any ideas?
Thanks,
When the command is a shell built-in, add a shell=True to the call.
E.g. for dir you would type:
import subprocess
subprocess.call('dir', shell=True)
To quote from the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
On Windows, I believe the subprocess module doesn't look in the PATH unless you pass shell=True because it use CreateProcess() behind the scenes. However, shell=True can be a security risk if you're passing arguments that may come from outside your program. To make subprocess nonetheless able to find the correct executable, you can use shutil.which. Suppose the executable in your PATH is named frob:
subprocess.call([shutil.which('frob'), arg1, arg2])
(This works on Python 3.3 and above.)
On Windows you have to call through cmd.exe. As Apalala mentioned, Windows commands are implemented in cmd.exe not as separate executables.
e.g.
subprocess.call(['cmd', '/c', 'dir'])
/c tells cmd to run the follow command
This is safer than using shell=True, which allows shell injections.
If you are using powershell, then in it will be subprocess.call(['powershell','-command','dir']). Powershell supports a large portion of POSIX commands
After much head scratching, I discovered that running a file that is located in C:\Windows\System32\ while running a 32bit version of python on a 64bit machine is a potential issue, due to Windows trying to out-smart the process, and redirect calls to C:\Windows\System32 to C:\Windows\SysWOW64.
I found an example of how to fix this here:
http://code.activestate.com/recipes/578035-disable-file-system-redirector/
To quote from the documentation:
"Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions."
SO: instead of subprocess.call use subprocess.run for Python 3.5 and above
I met the same issue while I was calling a PHP. The reason is PHP isn't in PATH so the command PHP was not found. But PowerShell found it does exist in the current location and it suggests replacing the 'PHP' by the '.\PHP' if I trust this command. Then it runs well.