Develop Reference

How do I embed my shell scanning-script into a Python script?

Iv'e been using the following shell command to read the image off a scanner named scanner_name and save it in a file named file_name
scanimage -d <scanner_name> --resolution=300 --format=tiff --mode=Color 2>&1 > <file_name>
This has worked fine for my purposes.
I'm now trying to embed this in a python script. What I need is to save the scanned image, as before, into a file and also capture any std output (say error messages) to a string
I've tried
scan_result = os.system('scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name))
But when I run this in a loop (with different scanners), there is an unreasonably long lag between scans and the images aren't saved until the next scan starts (the file is created as an empty file and is not filled until the next scanning command). All this with scan_result=0, i.e. indicating no error
The subprocess method run() has been suggested to me, and I have tried
with open(file_name, 'w') as scanfile:
input_params = '-d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name)
scan_result = subprocess.run(["scanimage", input_params], stdout=scanfile, shell=True)
but this saved the image in some kind of an unreadable file format
Any ideas as to what may be going wrong? Or what else I can try that will allow me to both save the file and check the success status?

subprocess.run() is definitely preferred over os.system() but neither of them as such provides support for running multiple jobs in parallel. You will need to use something like Python's multiprocessing library to run several tasks in parallel (or painfully reimplement it yourself on top of the basic subprocess.Popen() API).
You also have a basic misunderstanding about how to run subprocess.run(). You can pass in either a string and shell=True or a list of tokens and shell=False (or no shell keyword at all; False is the default).
with_shell = subprocess.run(
"scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} ".format(
scanner, file_name), shell=True)
with open(file_name) as write_handle:
no_shell = subprocess.run([
"scanimage", "-d", scanner, "--resolution=300", "--format=tiff",
"--mode=Color"], stdout=write_handle)
You'll notice that the latter does not support redirection (because that's a shell feature) but this is reasonably easy to implement in Python. (I took out the redirection of standard error -- you really want error messages to remain on stderr!)
If you have a larger working Python program this should not be awfully hard to integrate with a multiprocessing.Pool(). If this is a small isolated program, I would suggest you peel off the Python layer entirely and go with something like xargs or GNU parallel to run a capped number of parallel subprocesses.

I suspect the issue is you're opening the output file, and then running the subprocess.run() within it. This isn't necessary. The end result is, you're opening the file via Python, then having the command open the file again via the OS, and then closing the file via Python.
JUST run the subprocess, and let the scanimage 2>&1> filename command create the file (just as it would if you ran the scanimage at the command line directly.)
I think subprocess.check_output() is now the preferred method of capturing the output.
I.e.
from subprocess import check_output
# Command must be a list, with all parameters as separate list items
command = ['scanimage',
'-d{}'.format(scanner),
'--resolution=300',
'--format=tiff',
'--mode=Color',
'2>&1>{}'.format(file_name)]
scan_result = check_output(command)
print(scan_result)
However, (with both run and check_output) that shell=True is a big security risk ... especially if the input_params come into the Python script externally. People can pass in unwanted commands, and have them run in the shell with the permissions of the script.
Sometimes, the shell=True is necessary for the OS command to run properly, in which case the best recommendation is to use an actual Python module to interface with the scanner - versus having Python pass an OS command to the OS.

subprocess.Popen and relative directories

I am writing a script to open notepad.exe using subprocess.Popen()
import subprocess
command = '%windir%\system32\\notepad.exe'
process = subprocess.Popen(command)
output = process.communicate()
print(output[0])
This throws a FileNotFoundError
Is it possible to change/add to the above code to make it work with relative paths?
I did try to run the script from C:\Windows> after moving it there, which again failed. Also set the shell=True, but failed as well.
Writing a similar script using os.popen() works ok with relative paths, regardless which directory the script is run from, but as far as I understand popen is not the way forward..
Early steps in the world of programming/Python. Any input much appreciated.

Use os.path.expandvars to expand %windir%:
command = os.path.expandvars('%windir%\\system32\\notepad.exe')
The result is a path that then can be passed to subprocess.Popen.
subprocess.Popen does not expand environment variables such as %windir%. The shell might but you really should not depend on shell=True to do that.

Pro tip: whenever you get an error asking the system to execute a command, print the command (and, if applicable, the current working directory). The results will often surprise you.
In your case, I suspect you're just missing a backslash. Use this instead:
command = '%windir%\\system32\\notepad.exe'
Before you make that change, try printing the value of command immediately after assignment. I think you'll find the leading "s" in "system" is missing, and that the mistake is obvious.
HTH.

You could use raw strings to avoid having to double-up your backslashes.
command = r'%windir%\system32\notepad.exe'

Command line from Python 3

I have 1 python 3 script. I need to use another script via command line. What function should i use?
I mean something like that:
res = execute('C:\python32\python Z:\home\192.168.0.15\www\start.pyw start=1 module=server > Z:\home\192.168.0.15\www\test.html')

Use the subprocess module. That gives you the most flexibility.

Check out the Process Management section of the os module
http://docs.python.org/3/library/os.html#module-os
os.popen will work well if you are interested in i/o with the process

This is a python program you want to start. It would be better to import the module, run the method you want and write the output to a file.
However, this would be how you can do it via shell execution:
from subprocess import *
command_stdout = Popen(['C:\python32\python', 'Z:\home\192.168.0.15\www\start.pyw', 'start=1', 'module=server'], stdout=PIPE).communicate()[0]
res = command_stdout.decode("utf-8")
fd = open('Z:\home\192.168.0.15\www\test.html',"w")
fd.write(res)

Execute cmd-like command in Python

I want to make a Python code that will open a program like cmd would, then export a .txt file from the file menu. The code looks like this for cmd:
c:\ESG\Statsvis.exe \\192.168.100.222\c\ESG\S1-424\2012\06\29\S1-42420120629.dsf /output=C:\Users\jessica.macleod\Desktop\outfile.txt /param=RMS Amplitude
In cmd, the above line does exactly what I want. What would be the equivalent for Python?

See subprocess.Popen, like this:
subprocess.Popen(["/bin/ls", "-l"]
Or, depending on what you want to get as result (stdout, return code), use subprocess.call, subprocess.call_check, or other snippets in this module.

Another way would be os.system().
import os
os.system("c:\\ESG\\Statsvis.exe \\192.16...0629.dsf /output=C:\\...\\outfile.txt ...")

If you want to have exact shell/cmd behavior, then set the shell argument to True in a suprocess.Popen() call. However, from the documentation:
Warning
Invoking the system shell with shell=True can be a security hazard if
combined with untrusted input. See the warning under Frequently Used
Arguments for details.

If you need the output of the command use subprocess:
import subprocess
out = subprocess.check_output("dir c:\ /AD", shell = True)

How can I launch an instance of an application using Python?

I am creating a Python script where it does a bunch of tasks and one of those tasks is to launch and open an instance of Excel. What is the ideal way of accomplishing that in my script?

While the Popen answers are reasonable for the general case, I would recommend win32api for this specific case, if you want to do something useful with it:
It goes something like this:
from win32com.client import Dispatch
xl = Dispatch('Excel.Application')
wb = xl.Workbooks.Open('C:\\Documents and Settings\\GradeBook.xls')
xl.Visible = True # optional: if you want to see the spreadsheet
Taken from a mailing list post but there are plenty of examples around.

or
os.system("start excel.exe <path/to/file>")
(presuming it's in the path, and you're on windows)
and also on Windows, just start <filename> works, too - if it's an associated extension already (as xls would be)

The subprocess module intends to replace several other, older modules and functions, such as:
os.system
os.spawn*
os.popen*
popen2.*
commands.*
.
import subprocess
process_one = subprocess.Popen(['gqview', '/home/toto/my_images'])
print process_one.pid

I like popen2 for the ability to monitor the process.
excelProcess = popen2.Popen4("start excel %s" % (excelFile))
status = excelProcess.wait()
https://docs.python.org/2/library/popen2.html
EDIT: be aware that calling wait() will block until the process returns. Depending on your script, this may not be your desired behavior.

As others have stated, I would suggest os.system. In case anyone is looking for a Mac-compatible solution, here is an example:
import os
os.system("open /Applications/Safari.app")

os.system("open file.xls")

I like os.startfile("path to file") as it opens the file as if you've double clicked to open.
I found that with os.system("start excel filename") it opened it like a file opened from the web and you had to enable editing.