I am using Ubuntu and I am running a command using the subprocess module. I am trying to find the maximum number of days a password can be use.
import subprocess
pass_max = subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs')
print(pass_max)
After running this code, I receive the error no such file or directory. How am I able to find the maximum number of days a password can be use?
check_output expects the command as a list:
subprocess.check_output(['grep', 'PASS_MAX_DAYS', '/etc/login.defs'])
Alternative is to pass shell=True, taking into account the security considerations
subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs', shell=True)
grep PASS_MAX_DAYS /etc/login.defs is being interpreted as a single executable, which can't be found. Use an array to pass an executable with arguments.
subprocess.check_output(['grep', 'PASS_MAX_DAYS', '/etc/login.defs'])
the argument of the function check_output has to be a list, so just add split at the end of your command string
import subprocess
pass_max = subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs'.split())
print(pass_max)
That should work
try this -
shell_output
By using the above you should be able to overcome all the challenges that otherwise appear with a subprocess module
Usage -
print(shell_output("your shell command here"))
Iv'e been using the following shell command to read the image off a scanner named scanner_name and save it in a file named file_name
scanimage -d <scanner_name> --resolution=300 --format=tiff --mode=Color 2>&1 > <file_name>
This has worked fine for my purposes.
I'm now trying to embed this in a python script. What I need is to save the scanned image, as before, into a file and also capture any std output (say error messages) to a string
I've tried
scan_result = os.system('scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name))
But when I run this in a loop (with different scanners), there is an unreasonably long lag between scans and the images aren't saved until the next scan starts (the file is created as an empty file and is not filled until the next scanning command). All this with scan_result=0, i.e. indicating no error
The subprocess method run() has been suggested to me, and I have tried
with open(file_name, 'w') as scanfile:
input_params = '-d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name)
scan_result = subprocess.run(["scanimage", input_params], stdout=scanfile, shell=True)
but this saved the image in some kind of an unreadable file format
Any ideas as to what may be going wrong? Or what else I can try that will allow me to both save the file and check the success status?
subprocess.run() is definitely preferred over os.system() but neither of them as such provides support for running multiple jobs in parallel. You will need to use something like Python's multiprocessing library to run several tasks in parallel (or painfully reimplement it yourself on top of the basic subprocess.Popen() API).
You also have a basic misunderstanding about how to run subprocess.run(). You can pass in either a string and shell=True or a list of tokens and shell=False (or no shell keyword at all; False is the default).
with_shell = subprocess.run(
"scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} ".format(
scanner, file_name), shell=True)
with open(file_name) as write_handle:
no_shell = subprocess.run([
"scanimage", "-d", scanner, "--resolution=300", "--format=tiff",
"--mode=Color"], stdout=write_handle)
You'll notice that the latter does not support redirection (because that's a shell feature) but this is reasonably easy to implement in Python. (I took out the redirection of standard error -- you really want error messages to remain on stderr!)
If you have a larger working Python program this should not be awfully hard to integrate with a multiprocessing.Pool(). If this is a small isolated program, I would suggest you peel off the Python layer entirely and go with something like xargs or GNU parallel to run a capped number of parallel subprocesses.
I suspect the issue is you're opening the output file, and then running the subprocess.run() within it. This isn't necessary. The end result is, you're opening the file via Python, then having the command open the file again via the OS, and then closing the file via Python.
JUST run the subprocess, and let the scanimage 2>&1> filename command create the file (just as it would if you ran the scanimage at the command line directly.)
I think subprocess.check_output() is now the preferred method of capturing the output.
I.e.
from subprocess import check_output
# Command must be a list, with all parameters as separate list items
command = ['scanimage',
'-d{}'.format(scanner),
'--resolution=300',
'--format=tiff',
'--mode=Color',
'2>&1>{}'.format(file_name)]
scan_result = check_output(command)
print(scan_result)
However, (with both run and check_output) that shell=True is a big security risk ... especially if the input_params come into the Python script externally. People can pass in unwanted commands, and have them run in the shell with the permissions of the script.
Sometimes, the shell=True is necessary for the OS command to run properly, in which case the best recommendation is to use an actual Python module to interface with the scanner - versus having Python pass an OS command to the OS.
So I use a program DSI Studio and I am doing a lot of repetitive tasks that I would like to automate. It has a command line interface that works for me with the command
dsi_studio --action=trk -source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --
seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track5.trk
it does exactly what I want and outputs a file.
However when I try
import subprocess
subprocess.call("dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk", shell=True)
I get back return code 1. The same thing happens if I use subprocess.run. I have dinked around with different permutations to no avail. the only thing I have managed to get a 0 return code from is
subprocess.call('cd /d G:\Programs\dsi_studio_64', shell=True)
which I attempted because that is the directory I need to be in for the command to work in cmd. but even after doing that it still doesn't work. I'm pretty novice at python and I have spent a few days reading questions like mine but when I try to implement their solutions by template matching I have had no luck.
Each subprocess call makes its own shell, so your cd isn't actually affecting your later call, which is then breaking because you're in the wrong directory. Try
os.chdir("G:\Programs\dsi_studio_64")
subprocess.call("dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk", shell=True)
you can also do it with the cwd argument to call(), like
subprocess.call("your long command", cwd="directory")
You can use subprocess.Popen.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import subprocess
def run_process(exe):
'Define a function for running commands and capturing stdout line by line'
p = subprocess.Popen(exe.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return iter(p.stdout.readline, b'')
if __name__ == '__main__':
for line in run_process("G:\Programs\dsi_studio_64\dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk"):
print(line)
I need to be able to open a document using its default application in Windows and Mac OS. Basically, I want to do the same thing that happens when you double-click on the document icon in Explorer or Finder. What is the best way to do this in Python?
Use the subprocess module available on Python 2.4+, not os.system(), so you don't have to deal with shell escaping.
import subprocess, os, platform
if platform.system() == 'Darwin': # macOS
subprocess.call(('open', filepath))
elif platform.system() == 'Windows': # Windows
os.startfile(filepath)
else: # linux variants
subprocess.call(('xdg-open', filepath))
The double parentheses are because subprocess.call() wants a sequence as its first argument, so we're using a tuple here. On Linux systems with Gnome there is also a gnome-open command that does the same thing, but xdg-open is the Free Desktop Foundation standard and works across Linux desktop environments.
open and start are command-interpreter things for Mac OS/X and Windows respectively, to do this.
To call them from Python, you can either use subprocess module or os.system().
Here are considerations on which package to use:
You can call them via os.system, which works, but...
Escaping: os.system only works with filenames that don't have any spaces or other shell metacharacters in the pathname (e.g. A:\abc\def\a.txt), or else these need to be escaped. There is shlex.quote for Unix-like systems, but nothing really standard for Windows. Maybe see also python, windows : parsing command lines with shlex
MacOS/X: os.system("open " + shlex.quote(filename))
Windows: os.system("start " + filename) where properly speaking filename should be escaped, too.
You can also call them via subprocess module, but...
For Python 2.7 and newer, simply use
subprocess.check_call(['open', filename])
In Python 3.5+ you can equivalently use the slightly more complex but also somewhat more versatile
subprocess.run(['open', filename], check=True)
If you need to be compatible all the way back to Python 2.4, you can use subprocess.call() and implement your own error checking:
try:
retcode = subprocess.call("open " + filename, shell=True)
if retcode < 0:
print >>sys.stderr, "Child was terminated by signal", -retcode
else:
print >>sys.stderr, "Child returned", retcode
except OSError, e:
print >>sys.stderr, "Execution failed:", e
Now, what are the advantages of using subprocess?
Security: In theory, this is more secure, but in fact we're needing to execute a command line one way or the other; in either environment, we need the environment and services to interpret, get paths, and so forth. In neither case are we executing arbitrary text, so it doesn't have an inherent "but you can type 'filename ; rm -rf /'" problem, and if the file name can be corrupted, using subprocess.call gives us little additional protection.
Error handling: It doesn't actually give us any more error detection, we're still depending on the retcode in either case; but the behavior to explicitly raise an exception in the case of an error will certainly help you notice if there is a failure (though in some scenarios, a traceback might not at all be more helpful than simply ignoring the error).
Spawns a (non-blocking) subprocess: We don't need to wait for the child process, since we're by problem statement starting a separate process.
To the objection "But subprocess is preferred." However, os.system() is not deprecated, and it's in some sense the simplest tool for this particular job. Conclusion: using os.system() is therefore also a correct answer.
A marked disadvantage is that the Windows start command requires you to pass in shell=True which negates most of the benefits of using subprocess.
I prefer:
os.startfile(path, 'open')
Note that this module supports filenames that have spaces in their folders and files e.g.
A:\abc\folder with spaces\file with-spaces.txt
(python docs) 'open' does not have to be added (it is the default). The docs specifically mention that this is like double-clicking on a file's icon in Windows Explorer.
This solution is windows only.
Just for completeness (it wasn't in the question), xdg-open will do the same on Linux.
import os
import subprocess
def click_on_file(filename):
'''Open document with default application in Python.'''
try:
os.startfile(filename)
except AttributeError:
subprocess.call(['open', filename])
If you have to use an heuristic method, you may consider webbrowser.
It's standard library and despite of its name it would also try to open files:
Note that on some platforms, trying to open a filename using this
function, may work and start the operating system’s associated
program. However, this is neither supported nor portable.
(Reference)
I tried this code and it worked fine in Windows 7 and Ubuntu Natty:
import webbrowser
webbrowser.open("path_to_file")
This code also works fine in Windows XP Professional, using Internet Explorer 8.
If you want to go the subprocess.call() way, it should look like this on Windows:
import subprocess
subprocess.call(('cmd', '/C', 'start', '', FILE_NAME))
You can't just use:
subprocess.call(('start', FILE_NAME))
because start is not an executable but a command of the cmd.exe program. This works:
subprocess.call(('cmd', '/C', 'start', FILE_NAME))
but only if there are no spaces in the FILE_NAME.
While subprocess.call method enquotes the parameters properly, the start command has a rather strange syntax, where:
start notes.txt
does something else than:
start "notes.txt"
The first quoted string should set the title of the window. To make it work with spaces, we have to do:
start "" "my notes.txt"
which is what the code on top does.
Start does not support long path names and white spaces. You have to convert it to 8.3 compatible paths.
import subprocess
import win32api
filename = "C:\\Documents and Settings\\user\\Desktop\file.avi"
filename_short = win32api.GetShortPathName(filename)
subprocess.Popen('start ' + filename_short, shell=True )
The file has to exist in order to work with the API call.
os.startfile(path, 'open') under Windows is good because when spaces exist in the directory, os.system('start', path_name) can't open the app correctly and when the i18n exist in the directory, os.system needs to change the unicode to the codec of the console in Windows.
I am pretty late to the lot, but here is a solution using the windows api. This always opens the associated application.
import ctypes
shell32 = ctypes.windll.shell32
file = 'somedocument.doc'
shell32.ShellExecuteA(0,"open",file,0,0,5)
A lot of magic constants. The first zero is the hwnd of the current program. Can be zero. The other two zeros are optional parameters (parameters and directory). 5 == SW_SHOW, it specifies how to execute the app.
Read the
ShellExecute API docs for more info.
Here is the answer from Nick, adjusted slightly for WSL:
import os
import sys
import logging
import subprocess
def get_platform():
if sys.platform == 'linux':
try:
proc_version = open('/proc/version').read()
if 'Microsoft' in proc_version:
return 'wsl'
except:
pass
return sys.platform
def open_with_default_app(filename):
platform = get_platform()
if platform == 'darwin':
subprocess.call(('open', filename))
elif platform in ['win64', 'win32']:
os.startfile(filename.replace('/','\\'))
elif platform == 'wsl':
subprocess.call('cmd.exe /C start'.split() + [filename])
else: # linux variants
subprocess.call(('xdg-open', filename))
If you want to specify the app to open the file with on Mac OS X, use this:
os.system("open -a [app name] [file name]")
On windows 8.1, below have worked while other given ways with subprocess.call fails with path has spaces in it.
subprocess.call('cmd /c start "" "any file path with spaces"')
By utilizing this and other's answers before, here's an inline code which works on multiple platforms.
import sys, os, subprocess
subprocess.call(('cmd /c start "" "'+ filepath +'"') if os.name is 'nt' else ('open' if sys.platform.startswith('darwin') else 'xdg-open', filepath))
On mac os you can call open:
import os
os.open("open myfile.txt")
This would open the file with TextEdit, or whatever app is set as default for this filetype.
I think you might want to open file in editor.
For Windows
subprocess.Popen(["notepad", filename])
For Linux
subprocess.Popen(["text-editor", filename])
I built a small library combining the best answers here for cross-platform support:
$ pip install universal-startfile
then launch a file or URL:
from startfile import startfile
startfile("~/Downloads/example.png")
startfile("http://example.com")
I was getting an error when calling my open file() function. I was following along with a guide but the guide was written in windows while I'm on Linux. So the os.statrfile method wasn't working for me. I was able to alleviate this problem by doing the following:
Import libraries
import sys, os, subprocess
import tkinter
import tkinter.filedioalog as fd
import tkinter.messagebox as mb
After the lib imports I then called the subprocess method for opening a file in unix based OS which is "xdg-open" and the file that will be opened.
def open_file():
file = fd.askopenfilename(title='Choose a file of any type', filetypes=[('All files', "*.*")])
subprocess.call(['xdg-open', file])
I'm trying to copy thousands files to a remote server. These files are generated in real-time within the script. I'm working on a Windows system and need to copy the files to a Linux server (hence the escaping).
I currently have:
import os
os.system("winscp.exe /console /command \"option batch on\" \"option confirm off\" \"open user:pass#host\" \"put f1.txt /remote/dest/\"")
I'm using Python to generate the files but need a way to persist the remote connection so that I can copy each file, to the server, as it is generated (as opposed to creating a new connection each time). That way, I'll only need to change the field in the put option thus:
"put f2 /remote/dest"
"put f3 /remote/dest"
etc.
I needed to do this and found that code similar to this worked well:
from subprocess import Popen, PIPE
WINSCP = r'c:\<path to>\winscp.com'
class UploadFailed(Exception):
pass
def upload_files(host, user, passwd, files):
cmds = ['option batch abort', 'option confirm off']
cmds.append('open sftp://{user}:{passwd}#{host}/'.format(host=host, user=user, passwd=passwd))
cmds.append('put {} ./'.format(' '.join(files)))
cmds.append('exit\n')
with Popen(WINSCP, stdin=PIPE, stdout=PIPE, stderr=PIPE,
universal_newlines=True) as winscp: #might need shell = True here
stdout, stderr = winscp.communicate('\n'.join(cmds))
if winscp.returncode:
# WinSCP returns 0 for success, so upload failed
raise UploadFailed
This is simplified (and using Python 3), but you get the idea.
Instead of using an external program (winscp) you could also use an python ssh-library like pyssh.
You would have to start persistent WinSCP sub-process in Python and feed the put commands to its standard input continuously.
I do not have Python example for this, but there's an equivalent JScript example:
https://winscp.net/eng/docs/guide_automation_advanced#inout
or C# example:
https://winscp.net/eng/docs/guide_dotnet#input
Though using WinSCP .NET assembly via its COM interface for Python would be a way easier:
https://winscp.net/eng/docs/library