I have a Python script that needs to execute an external program, but for some reason fails.
If I have the following script:
import os;
os.system("C:\\Temp\\a b c\\Notepad.exe");
raw_input();
Then it fails with the following error:
'C:\Temp\a' is not recognized as an internal or external command, operable program or batch file.
If I escape the program with quotes:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe"');
raw_input();
Then it works. However, if I add a parameter, it stops working again:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe" "C:\\test.txt"');
raw_input();
What is the right way to execute a program and wait for it to complete? I do not need to read output from it, as it is a visual program that does a job and then just exits, but I need to wait for it to complete.
Also note, moving the program to a non-spaced path is not an option either.
This does not work either:
import os;
os.system("'C:\\Temp\\a b c\\Notepad.exe'");
raw_input();
Note the swapped single/double quotes.
With or without a parameter to Notepad here, it fails with the error message
The filename, directory name, or volume label syntax is incorrect.
subprocess.call will avoid problems with having to deal with quoting conventions of various shells. It accepts a list, rather than a string, so arguments are more easily delimited. i.e.
import subprocess
subprocess.call(['C:\\Temp\\a b c\\Notepad.exe', 'C:\\test.txt'])
Here's a different way of doing it.
If you're using Windows the following acts like double-clicking the file in Explorer, or giving the file name as an argument to the DOS "start" command: the file is opened with whatever application (if any) its extension is associated with.
filepath = 'textfile.txt'
import os
os.startfile(filepath)
Example:
import os
os.startfile('textfile.txt')
This will open textfile.txt with Notepad if Notepad is associated with .txt files.
The outermost quotes are consumed by Python itself, and the Windows shell doesn't see it. As mentioned above, Windows only understands double-quotes.
Python will convert forward-slashed to backslashes on Windows, so you can use
os.system('"C://Temp/a b c/Notepad.exe"')
The ' is consumed by Python, which then passes "C://Temp/a b c/Notepad.exe" (as a Windows path, no double-backslashes needed) to CMD.EXE
At least in Windows 7 and Python 3.1, os.system in Windows wants the command line double-quoted if there are spaces in path to the command. For example:
TheCommand = '\"\"C:\\Temp\\a b c\\Notepad.exe\"\"'
os.system(TheCommand)
A real-world example that was stumping me was cloning a drive in VirtualBox. The subprocess.call solution above didn't work because of some access rights issue, but when I double-quoted the command, os.system became happy:
TheCommand = '\"\"C:\\Program Files\\Sun\\VirtualBox\\VBoxManage.exe\" ' \
+ ' clonehd \"' + OrigFile + '\" \"' + NewFile + '\"\"'
os.system(TheCommand)
For python >= 3.5 subprocess.run should be used in place of subprocess.call
https://docs.python.org/3/library/subprocess.html#older-high-level-api
import subprocess
subprocess.run(['notepad.exe', 'test.txt'])
import win32api # if active state python is installed or install pywin32 package seperately
try: win32api.WinExec('NOTEPAD.exe') # Works seamlessly
except: pass
I suspect it's the same problem as when you use shortcuts in Windows... Try this:
import os;
os.system("\"C:\\Temp\\a b c\\Notepad.exe\" C:\\test.txt");
For Python 3.7, use subprocess.call. Use raw string to simplify the Windows paths:
import subprocess
subprocess.call([r'C:\Temp\Example\Notepad.exe', 'C:\test.txt'])
Suppose we want to run your Django web server (in Linux) that there is space between your path (path='/home/<you>/<first-path-section> <second-path-section>'), so do the following:
import subprocess
args = ['{}/manage.py'.format('/home/<you>/<first-path-section> <second-path-section>'), 'runserver']
res = subprocess.Popen(args, stdout=subprocess.PIPE)
output, error_ = res.communicate()
if not error_:
print(output)
else:
print(error_)
[Note]:
Do not forget accessing permission: chmod 755 -R <'yor path'>
manage.py is exceutable: chmod +x manage.py
No need for sub-process, It can be simply achieved by
GitPath="C:\\Program Files\\Git\\git-bash.exe"# Application File Path in mycase its GITBASH
os.startfile(GitPath)
Normally, in Python shells I can press Tab twice to get a list of prompts.
On the other hand, in gdb's Python shell (pi or python-interactive command), there's only gdb-style completion.
Example session:
$ gdb -q
(gdb) pi
>>> gdb
<module 'gdb' from '/usr/share/gdb/python/gdb/__init__.py'>
>>> gdb.TabTab
... nothing ...
>>> show TabTab
Display all 148 possibilities? (y or n)
ada exec-direction record
agent exec-done-display remote
annotate exec-file-mismatch remoteaddresssize
[...]
Python auto complete should be at least like this.
$ python
Python 3.X.Y ...
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.TabTab
sys.abiflags sys.hash_info
sys.addaudithook( sys.hexversion
sys.api_version sys.implementation
[...]
How can I get the same/similar thing in gdb? in particular IPython shell with tab completion is fine.
Failed attempts:
This solution
import readline
import rlcompleter
readline.parse_and_bind("tab: complete")
makes the shell output a literal tab when Tab is pressed after sys. or similar.
At least it does work for identifier tab completion (aTabTab) does list some entries)
Looks like this is because of some interaction with gdb -- get_completer_delims get reset to some value every time, and if the code above is run then the tab completion outside gdb switches to "Python mode" as well.
Use background_zmq_ipython causes segmentation fault, because some gdb API (such as gdb.Value) cannot be read from outside the main thread.
Use IPython.embed() also make Tab output a literal tab character.
The official gdb documentation https://sourceware.org/gdb/current/onlinedocs/gdb/Completion.html doesn't mention anything about Python.
There are a few ways I've figured out.
I can't figure out any way to use the built-in readline library.
See the failed attempts in the question for more details.
Reset stdout and stderr before calling IPython.embed.
import sys
sys.stdout=sys.__stdout__
sys.stderr=sys.__stderr__
import IPython
IPython.embed(colors="neutral")
Remember to reset stdout and stderr afterwards to avoid possible issues.
Reference:
How to recognize whether a script is running on a tty?
IPython.embed() does not use terminal colors
IPython will only use tab-completion and color when all of stdin, stdout and stderr are tty devices.
By default gdb sys.stdout and sys.stderr are gdb wrappers (so that gdb can do "press enter to continue"
when pagination limit is exceeded)
Start a kernel, and start a console separately.
import IPython
IPython.embed_kernel()
Read the console output on how to connect, and how to exit the terminal from the remote console.
Using my other answer it's also possible to exit the terminal remotely programmatically.
Start a kernel (the complex way)
Read the source code of IPython to figure out how to start a kernel manually, and get the connection file path in the process.
import threading
import subprocess
import IPython
from ipykernel.kernelapp import IPKernelApp
import sys
app = IPKernelApp.instance()
app.initialize([])
app.kernel.user_module = sys.modules[__name__]
app.kernel.user_ns = locals()
app.shell.set_completer_frame()
def console_thread_run():
subprocess.run(["jupyter-console", "--no-confirm-exit", "--existing",
app.abs_connection_file
])
app.kernel.do_shutdown(restart=False)
console_thread=threading.Thread(target=console_thread_run)
console_thread.start()
app.start()
Start a kernel using background_zmq_ipython (accesses internal property, might break at any time).
The main difference is that sys.stdin, sys.stdout etc. are not affected. See background_zmq_ipython
documentation and ipython - Provide remote shell for Python script - Stack Overflow for more details.
import subprocess
import logging
import threading
from background_zmq_ipython import IPythonBackgroundKernelWrapper
kernel_wrapper = IPythonBackgroundKernelWrapper(
banner="", # default value is "Hello from background-zmq-ipython."
user_ns=globals(),
logger=logging.Logger("IPython", level=logging.INFO)
# no handler
# otherwise it will print "To connect another client to this IPython kernel" ...
)
kernel_wrapper.thread=threading.main_thread() # workaround for assertions
subprocess.Popen(["python", "-c",
(
"from jupyter_console.app import ZMQTerminalIPythonApp;"
"app = ZMQTerminalIPythonApp();"
"app.initialize();"
"app.shell.own_kernel=True;"
"app.start();"
),
"--no-confirm-exit",
"--existing", kernel_wrapper.connection_filename
])
kernel_wrapper._thread_loop()
Also show how to change the message "keeping kernel alive" to "Shutting down kernel".
How can I get subprocess.check_call to give me the raw binary output of a command, it seems to be encoding it incorrectly somewhere.
Details:
I have a command that returns text like this:
some output text “quote” ...
(Those quotes are unicode e2809d)
Here's how I'm calling the command:
f_output = SpooledTemporaryFile()
subprocess.check_call(cmd, shell=True, stdout=f_output)
f_output.seek(0)
output = f_output.read()
The problem is I get this:
>>> repr(output)
some output text ?quote? ...
>>> type(output)
<str>
(And if I call 'ord' the '?' I get 63.)
I'm on Python 2.7 on Linux.
Note: Running the same code on OSX works correctly to me. The problem is when I run it on a Linux server.
Wow, this was the weirdest issue ever but I've fixed it!
It turns out that the program it was calling (a java program) was returning different encoding depending on where it was called from!
Dev osx machine, returns the characters fine, Linux server from command line, returns them fine, called from a Django app, nope turns into "?"s.
To fix this I ended up adding this argument to the command:
-Dfile.encoding=utf-8
I got that idea here, and it seems to work. There's also a way to modify the Java program internally to do that.
Sorry I blamed Python! You guys had the right idea.
The redirection (stdout=file) happens at the file descriptor level. Python has nothing to do with what is written to the file if you see ? instead of “ in the file itself (not in a REPL).
If it work on OS X and it "doesn't work" on Linux server then the likely reason is the difference in the environment, check LC_ALL, LC_CTYPE, LANG envvars—python, /bin/sh (due to shell=True), and the cmd may use your locale encoding that is ASCII if the environment is not set (C, POSIX locale).
To get "raw binary" from a subprocess:
#!/usr/bin/env python
import subprocess
raw_binary = subprocess.check_output(['cmd', 'arg 1', 'arg 2'])
print(repr(raw_binary))
Note:
no shell=True—don't use it unless it is necessary
many programs may change their behavior if they detect that the output is not a tty, example.
I need to be able to open a document using its default application in Windows and Mac OS. Basically, I want to do the same thing that happens when you double-click on the document icon in Explorer or Finder. What is the best way to do this in Python?
Use the subprocess module available on Python 2.4+, not os.system(), so you don't have to deal with shell escaping.
import subprocess, os, platform
if platform.system() == 'Darwin': # macOS
subprocess.call(('open', filepath))
elif platform.system() == 'Windows': # Windows
os.startfile(filepath)
else: # linux variants
subprocess.call(('xdg-open', filepath))
The double parentheses are because subprocess.call() wants a sequence as its first argument, so we're using a tuple here. On Linux systems with Gnome there is also a gnome-open command that does the same thing, but xdg-open is the Free Desktop Foundation standard and works across Linux desktop environments.
open and start are command-interpreter things for Mac OS/X and Windows respectively, to do this.
To call them from Python, you can either use subprocess module or os.system().
Here are considerations on which package to use:
You can call them via os.system, which works, but...
Escaping: os.system only works with filenames that don't have any spaces or other shell metacharacters in the pathname (e.g. A:\abc\def\a.txt), or else these need to be escaped. There is shlex.quote for Unix-like systems, but nothing really standard for Windows. Maybe see also python, windows : parsing command lines with shlex
MacOS/X: os.system("open " + shlex.quote(filename))
Windows: os.system("start " + filename) where properly speaking filename should be escaped, too.
You can also call them via subprocess module, but...
For Python 2.7 and newer, simply use
subprocess.check_call(['open', filename])
In Python 3.5+ you can equivalently use the slightly more complex but also somewhat more versatile
subprocess.run(['open', filename], check=True)
If you need to be compatible all the way back to Python 2.4, you can use subprocess.call() and implement your own error checking:
try:
retcode = subprocess.call("open " + filename, shell=True)
if retcode < 0:
print >>sys.stderr, "Child was terminated by signal", -retcode
else:
print >>sys.stderr, "Child returned", retcode
except OSError, e:
print >>sys.stderr, "Execution failed:", e
Now, what are the advantages of using subprocess?
Security: In theory, this is more secure, but in fact we're needing to execute a command line one way or the other; in either environment, we need the environment and services to interpret, get paths, and so forth. In neither case are we executing arbitrary text, so it doesn't have an inherent "but you can type 'filename ; rm -rf /'" problem, and if the file name can be corrupted, using subprocess.call gives us little additional protection.
Error handling: It doesn't actually give us any more error detection, we're still depending on the retcode in either case; but the behavior to explicitly raise an exception in the case of an error will certainly help you notice if there is a failure (though in some scenarios, a traceback might not at all be more helpful than simply ignoring the error).
Spawns a (non-blocking) subprocess: We don't need to wait for the child process, since we're by problem statement starting a separate process.
To the objection "But subprocess is preferred." However, os.system() is not deprecated, and it's in some sense the simplest tool for this particular job. Conclusion: using os.system() is therefore also a correct answer.
A marked disadvantage is that the Windows start command requires you to pass in shell=True which negates most of the benefits of using subprocess.
I prefer:
os.startfile(path, 'open')
Note that this module supports filenames that have spaces in their folders and files e.g.
A:\abc\folder with spaces\file with-spaces.txt
(python docs) 'open' does not have to be added (it is the default). The docs specifically mention that this is like double-clicking on a file's icon in Windows Explorer.
This solution is windows only.
Just for completeness (it wasn't in the question), xdg-open will do the same on Linux.
import os
import subprocess
def click_on_file(filename):
'''Open document with default application in Python.'''
try:
os.startfile(filename)
except AttributeError:
subprocess.call(['open', filename])
If you have to use an heuristic method, you may consider webbrowser.
It's standard library and despite of its name it would also try to open files:
Note that on some platforms, trying to open a filename using this
function, may work and start the operating system’s associated
program. However, this is neither supported nor portable.
(Reference)
I tried this code and it worked fine in Windows 7 and Ubuntu Natty:
import webbrowser
webbrowser.open("path_to_file")
This code also works fine in Windows XP Professional, using Internet Explorer 8.
If you want to go the subprocess.call() way, it should look like this on Windows:
import subprocess
subprocess.call(('cmd', '/C', 'start', '', FILE_NAME))
You can't just use:
subprocess.call(('start', FILE_NAME))
because start is not an executable but a command of the cmd.exe program. This works:
subprocess.call(('cmd', '/C', 'start', FILE_NAME))
but only if there are no spaces in the FILE_NAME.
While subprocess.call method enquotes the parameters properly, the start command has a rather strange syntax, where:
start notes.txt
does something else than:
start "notes.txt"
The first quoted string should set the title of the window. To make it work with spaces, we have to do:
start "" "my notes.txt"
which is what the code on top does.
Start does not support long path names and white spaces. You have to convert it to 8.3 compatible paths.
import subprocess
import win32api
filename = "C:\\Documents and Settings\\user\\Desktop\file.avi"
filename_short = win32api.GetShortPathName(filename)
subprocess.Popen('start ' + filename_short, shell=True )
The file has to exist in order to work with the API call.
os.startfile(path, 'open') under Windows is good because when spaces exist in the directory, os.system('start', path_name) can't open the app correctly and when the i18n exist in the directory, os.system needs to change the unicode to the codec of the console in Windows.
I am pretty late to the lot, but here is a solution using the windows api. This always opens the associated application.
import ctypes
shell32 = ctypes.windll.shell32
file = 'somedocument.doc'
shell32.ShellExecuteA(0,"open",file,0,0,5)
A lot of magic constants. The first zero is the hwnd of the current program. Can be zero. The other two zeros are optional parameters (parameters and directory). 5 == SW_SHOW, it specifies how to execute the app.
Read the
ShellExecute API docs for more info.
Here is the answer from Nick, adjusted slightly for WSL:
import os
import sys
import logging
import subprocess
def get_platform():
if sys.platform == 'linux':
try:
proc_version = open('/proc/version').read()
if 'Microsoft' in proc_version:
return 'wsl'
except:
pass
return sys.platform
def open_with_default_app(filename):
platform = get_platform()
if platform == 'darwin':
subprocess.call(('open', filename))
elif platform in ['win64', 'win32']:
os.startfile(filename.replace('/','\\'))
elif platform == 'wsl':
subprocess.call('cmd.exe /C start'.split() + [filename])
else: # linux variants
subprocess.call(('xdg-open', filename))
If you want to specify the app to open the file with on Mac OS X, use this:
os.system("open -a [app name] [file name]")
On windows 8.1, below have worked while other given ways with subprocess.call fails with path has spaces in it.
subprocess.call('cmd /c start "" "any file path with spaces"')
By utilizing this and other's answers before, here's an inline code which works on multiple platforms.
import sys, os, subprocess
subprocess.call(('cmd /c start "" "'+ filepath +'"') if os.name is 'nt' else ('open' if sys.platform.startswith('darwin') else 'xdg-open', filepath))
On mac os you can call open:
import os
os.open("open myfile.txt")
This would open the file with TextEdit, or whatever app is set as default for this filetype.
I think you might want to open file in editor.
For Windows
subprocess.Popen(["notepad", filename])
For Linux
subprocess.Popen(["text-editor", filename])
I built a small library combining the best answers here for cross-platform support:
$ pip install universal-startfile
then launch a file or URL:
from startfile import startfile
startfile("~/Downloads/example.png")
startfile("http://example.com")
I was getting an error when calling my open file() function. I was following along with a guide but the guide was written in windows while I'm on Linux. So the os.statrfile method wasn't working for me. I was able to alleviate this problem by doing the following:
Import libraries
import sys, os, subprocess
import tkinter
import tkinter.filedioalog as fd
import tkinter.messagebox as mb
After the lib imports I then called the subprocess method for opening a file in unix based OS which is "xdg-open" and the file that will be opened.
def open_file():
file = fd.askopenfilename(title='Choose a file of any type', filetypes=[('All files', "*.*")])
subprocess.call(['xdg-open', file])
In my Python script, this line:
call("/Applications/BitRock\\ InstallBuilder\\ for\\ Qt\\ 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh")
will fail every time with the error OSError: [Errno 2] No such file or directory
However, if I write out the result of that string:
sys.stdout.write("/Applications/BitRock\\ InstallBuilder\\ for\\ Qt\\ 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh")
I get:
/Applications/BitRock\ InstallBuilder\ for\ Qt\ 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh
If I put that directly into the terminal, it works perfect.
What am I missing?
By default subprocess.call uses no shell (shell=False). Therefore, there's no need to escape the spaces. The space escaping is needed in shells (because shells need to know what's the binary's name and what the arguments). Therefore the following uses are all correct (and similar):
Without spawning a shell that spawns the subprocess (favorable):
from subprocess import call
call('/Applications/BitRock InstallBuilder for Qt 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh')
or again without shell (explicit shell=False and the usage of an argument list)
from subprocess import call
call(['/Applications/BitRock InstallBuilder for Qt 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh'],
shell=False)
But when subprocess is told to first of all swawn shell which then spawns the subprocess itself, the spaces have to be escaped, because it's a shell command:
from subprocess import call
call('/Applications/BitRock\\ InstallBuilder\\ for\\ Qt 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh',
shell=True)
An alternative would be to use a shell and quotes:
from subprocess import call
call('"/Applications/BitRock InstallBuilder for Qt 8.5.2/bin/Builder.app/Contents/MacOS/installbuilder.sh"',
shell=True)
I'd recommend not to use a shell whenever possible (mostly for security reasons), remember though you have to pass the arguments to the command as a list if you're not using a shell.
Either (without shell, favourable):
call(['/bin/echo', 'foo', 'bar'])
or with a shell
call('/bin/echo foo bar', shell=True)
(both calls have the same output (foo bar\n)