I try to use a program called bwa in python with the subprocess module:
subprocess.call('bwa', shell=True)
Result is:
/bin/sh: bwa: command not found
However if I start a terminal and type the command, the program starts (i.e. I get the Usage dialogue).
The binaries of bwa are in a folder on my desktop. In .bash_profile I have modified the path variable so that everytime I type 'bwa' in the terminal, the program binaries are found in my folder.
My assumption is that the subprocess does not start a shell with my bash_profile settings ... but if that is correct why wouldn't it do so and what can I do about it?
Related
I'm working on using the subprocess module to send shell commands from Python, specifically, ssh. Below is a barebones sample:
import subprocess
sp = subprocess.run(["ssh"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"stdout: {sp.stdout.decode()} \n\nstderr: {sp.stderr.decode()}")
This should return the ssh command help from stdout, and nothing from stderr. However, I get:
stdout:
stderr: 'ssh' is not recognized as an internal or external command,
operable program or batch file.
I've tried other commands, like echo and cd, and those work fine. I am also able to use ssh when manually typing the command into the shell, but it fails when I try to do it through subprocess. The directory C:\Windows\System32\OpenSSH does exist on my computer (and it contains ssh.exe), but for some strange reason I'm unable to cd to it using subprocess.
If it matters, subprocess is using the command prompt, cmd.exe, as it seems to be the default.
Any help is appreciated. Thanks!
-- Edits with tests from comments --
Using the absolute path C:/Windows/System32/OpenSSH/ssh.exe does not work, and gives The system cannot find the path specified via stderr. The OpenSSH folder doesn't seem to be visible to Python through subprocess
os.environ[PATH] contains both C:/Windows/System32/ and C:/Windows/System32/OpenSSH/
Running it with shell=False (either with the absolute path or just with ssh) raises an error in Python: FileNotFoundError: [WinError 2] The system cannot find the file specified
You say C:\Windows\System32\OpenSSH\ssh.exe exists, but that it's not found when running from Python. This is likely a result of having a 32 bit version of Python installed, rather than a 64 bit version.
If the path exists elsewhere, but not for Python, that would tend to implicate the file system redirector. Python is probably seeing C:\Windows\SysWOW64 when you tell it to look in C:\Windows\System32. I'd recommend uninstalling whatever Python you have, and explicitly installing a 64 bit version, so it isn't affected by the redirector, and sees the "real" System32.
I can enter SVN commands successfully in the Windows command line, but when I try to pass them via subprocess with shell=True I get this error:
'svn' is not recognized as an internal or external command, operable program or batch file
When I omit the shell argument I get this:
WindowsError: [Error 2] The system cannot find the file specified
I'm running Python 2.7 on Windows 10, where I also have Python 3.8 installed. I've tried a variety of SVN commands, some complex and some simple, with a variety of arguments, both as a single string and a list of strings, both in IDLE and Spyder (console and script in each), and keep getting the same results. I am able to pass other types of Windows commands via subprocess, just not SVN. I've confirmed that the COMSPEC environment variable is correct. I've also tried moving the path to svn.exe all the way up in the PATH environment variable and rebooting. No dice.
Here's an example of what I'm trying to do:
import subprocess
my_cmd = ['svn', 'propget', 'svn:externals', '-R', '"https://the/rest/of/the/url"']
res = subprocess.check_output(my_cmd, shell=True)
print "the result of the svn command via subprocess is...\n{}".format(res)
After finding that the path to svn.exe was missing from the string returned by os.environ.get('PATH'), I added this path with the following line:
os.environ['PATH'] += r"C:\Program Files\TortoiseSVN\bin;"
And now my subsequent SVN commands are working via subprocess.
Thank you #John Gordon and #Maurice Meyer for the help!
I am trying to use os.system from a .py file to open a new cmd, change its directory and then run separate commands in the new cmd. However after changing the directory, the command I enter using '&&' in the o.system runs in the first cmd and not the new one. I know I should use 'subprocess' module in place of 'os' but both have the same limitation. Here is a snippet of a similar code which I personally feel is easy to understand.
os.system('start /wait cmd /k cd C:/Users/FenilD/Desktop/Python/Python 2.7 && echo "hello"')
Here 'hello' prints in the old cmd from where I am running the file which contains the os.system command.
Any ideas how I can get the 'hello' to print in the new cmd?
I have notepad++ installed on win7, and have installed the notepad++ exec plugin. Whenever I execute the .py script I am running, i get the following error:
C:/Python27/python.exe ""
Process started >>>
C:\Python27\python.exe: can't find '__main__' module in ''
<<< Process finished.
================ READY ================
My index.py script is as follows:
text = "Hello World"
print text
The Notepad++ execute script (F6) is as follows:
C:/Python27/python.exe "$(C:/Python27/python.exe)"
This is the correct path to python.exe on my system. In the windows environmental variables, I have also added this to the PATH. Also, I have booted up the python.exe command prompt and have gotten the script to work by manually typing it in. I have done a directory search of C:/Python27, and the main.py file exists.
What is going on that is wrong here? Why can't python find the __ main __ module??
EDIT:
Nevermind this, I'm an idiot.
So I had misinterpreted the setup for the notepad++ exec (F6) script:
C:/Python27/python.exe "$(C:/Python27/python.exe)"
It should have been this instead:
C:/Python27/python.exe "$(FULL_CURRENT_PATH)"
Of course it couldn't find the main.py file because it wasn't going to FULL_CURRENT_PATH.
Carry on.
Basically, it's because the 1st argument in your script has been missinterpreted.
That is to say, with the code
C:/python27/python.exe "$(C:/python27/python.exe)"
You're doing something roughly equivalent
C:/python27/python.exe
What you need to use is:
C:/Python27/python.exe "$(FULL_CURRENT_PATH)"
Note: Don't substitute "$(FULL_CURRENT_PATH)" for something else - put that in verbatim - it's a predefined variable of the NppExec plugin. It may be considered shorthand for the full-path of the file in the currently open tab.
When I execute the script above (on a file called tmp.py, in my c:\xampp\htdocs\enhzflep folder), I get this displayed in the console:
C:/Python27/python.exe "C:\xampp\htdocs\enhzflep\tmp.py"
CreateProcess() failed with error code 2: The system cannot find the
file specified.
Which is just fine, since I don't have python installed on this machine. :)
EDIT: Just saw your edit. It seems I'm the greater fool!
I'm using os.system() to do Windows command line shell executions. I would like to change the Windows cmd current directory. Here's one way of doing it:
os.chdir('newPath')
But chdir() will also change the actual Python current working directory. I don't want to change the actual Python working directory because I want other parts of my script to run in the original current working directory. What I want to change is only the Windows cmd current working directory. In other words: I want os.system() commands to run in one current working directory (Windows cmd current working directory) while anything else should run in another current working directory (the actual Python current working directory).
Here's another try to change only the Windows cmd current directory:
os.system('cd newPath')
However, that obviously doesn't work since right after the execution of the cd newPath command the Windows cmd current directory is reset (because I won't use the same Windows command shell in the next call to os.system()).
Is it possible to have a separate current working directory for the Windows cmd shell? (separate from the actual current working directory).
The subprocess module is intended to replace os.system.
Among other things, it gives you subprocess.Popen(), which takes a cwd argument to specify the working directory for the spawned process (for exactly your situation).
See: http://docs.python.org/library/subprocess.html
Example usage replacing os.system:
p = subprocess.Popen("yourcmd" + " yourarg", shell=True, cwd="c:/your/path")
sts = os.waitpid(p.pid, 0)[1]
If it only has to work on Windows, one way might be:
os.system('start /d newPath cmd')
When you use os.system, you're not reusing the same command shell, but spawning a new one for each request. This means that you can't actually expect changes in it to propagate between invocations.
You could write a wrapper though, that will always change to the directory you want before launching the command.