Develop Reference

python invoking PV-Wave

'/usr/local/bin/wave' only accepts a filename as input, so I need to invoke the process, then "send in" the commands, and wait for the output file to be written. Then my process can proceed to read the output file. Here is my code that does not write to the output file:
hdfFile = "/archive/HDF/16023343.hdf"
pngFile = "/xrfc_calib/xrfc.130.png"
lpFile = os.environ['DOCUMENT_ROOT'] + pngFile
waveCmd = "hdfview, '" + hdfFile + "', outfile='" + lpFile + "', web, view='RASTER', /neg"
os.environ['WAVE_PATH'] = "/oudvmt/wave/pro:/dvmt/wave/pro"
wfile = subprocess.Popen ('/usr/local/bin/wave >&2', shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
wfile.stdin = "\#hdf_startup\n\#hdf_common\n" + waveCmd + "\nquit\n"

I found what I was missing. The change is to the last 2 lines. They are:
wfile = subprocess.Popen ('/usr/local/bin/wave', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
wfile.communicate("\#hdf_startup\n\#hdf_common\n" + waveCmd + "\nquit\n")
I needed to set "stdout" to avoid extra output from PV-Wave.
I needed to use "communicate" to wait for the process to complete.

Does Popen().stdout.close() return a value?

I have in the code I maintain:
app7z = dirs['mopy'].join('7z.exe').s # path to 7z.exe
command = '"%s" a "%s" -y -r "%s\\*"' % (app7z, dstFile.temp.s, srcDir.s)
ins = Popen(command, stdout=PIPE, startupinfo=startupinfo).stdout
#--Error checking and progress feedback
reCompressing = re.compile('Compressing\s+(.+)')
regMatch = reCompressing.match
reError = re.compile('Error: (.*)')
regErrMatch = reError.match
errorLine = []
for line in ins:
maCompressing = regMatch(line)
if len(errorLine) or regErrMatch(line):
errorLine.append(line)
if maCompressing:
# update progress
result = ins.close() # THIS
if result:
dstFile.temp.remove()
raise StateError(_("%s: Compression failed:\n%s") % (dstFile.s,
"\n".join(errorLine)))
(full code)
Does ins.close() return a non None value on failure ? My IDE (pycharm 3.4.2/4.5.2) warns me that it does not, but then not consistently.
I am on windows if this makes a difference - python 2.7.8

What do you think, close can return?
You probably want to use wait to get the exit code:
app7z = dirs['mopy'].join('7z.exe').s # path to 7z.exe
command = [app7z, 'a', dstFile.temp.s, "-y", "-r", os.path.join(src.Dir.s, '*')]
process = Popen(command, stdout=PIPE, startupinfo=startupinfo)
out = process.stdout
regMatch = re.compile('Compressing\s+(.+)').match
regErrMatch = re.compile('Error: (.*)').match
errorLine = []
for line in out:
maCompressing = regMatch(line)
if len(errorLine) or regErrMatch(line):
errorLine.append(line)
if maCompressing:
# update progress
result = process.wait()
if result:
dstFile.temp.remove()
raise StateError(_("%s: Compression failed:\n%s") % (dstFile.s,
"\n".join(errorLine)))

Python subprocess: capture output of ffmpeg and run regular expression against it

I have the following code
import subprocess
import re
from itertools import *
command = ['ffprobe', '-i', '/media/some_file.mp4']
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
text = p.stderr.read()
retcode = p.wait()
text = text.decode('utf-8')
p = re.compile("Duration(.*)")
num = 0 #for debugging
for line in iter(text.splitlines()):
print(str(num) + line) #for debugging
m = p.match(str(line))
if m != None:
print(m.group(1))
When I look at the output there is a line that says "Duration" on it, however it is not captured, print(m.group(1)) is never reached. If I change the text variable to a hardcoded string of "Duration blahblah" I get " blahblah", which is what I expect. It seems like the regex doesn't recognize the text coming back from stderr. How can I get the text into a format that the regex will recognize and match on?
I have come up with the following solution, should it help anyone else attempting to capture duration from ffmpeg using python
import subprocess
import re
command = ['ffprobe', '-i', '/media/some_file.mp4']
p = subprocess.Popen(command, stderr=subprocess.PIPE)
text = p.stderr.read()
retcode = p.wait()
text = text.decode('utf-8')
p = re.compile(".*Duration:\s([0-9:\.]*),", re.MULTILINE|re.DOTALL)
m = p.match(text)
print(m.group(1))

p = re.compile(r".*?Duration(.*)")
Try this.match starts from the begining while there may might be something before duration.

Initializing...Command 'sox' returned non-zero exit status 2

I have checked similar error message questions but have not found anything that quite fits my situation. I am trying to time align a .wav file with a .lab file using HTK, Prosodylab-aligner, and SoX.
Here is my input (using Prosodylab-aligner):
./align.py /path/to/files
All that comes up is this line of code:
Command 'sox' returned non-zero exit status 2
I looked up what this code means and apparently it means there is a command or keyword missing
I believe that the problem is in the align.py file but I am not sure where exactly.
Here is the area of the file that references SoX.
def _check_aud(self, wav_list, train=False):
"""
Check audio files, mixing down to mono and downsampling if
necessary. Writes copy_scp and the training or testing SCP files
"""
copy_scp = open(self.copy_scp, 'a')
check_scp = open(self.train_scp if train else self.test_scp, 'w')
i = 0
if self.has_sox:
for wav in wav_list:
head = os.path.splitext(os.path.split(wav)[1])[0]
mfc = os.path.join(self.aud_dir, head + '.mfc')
w = wave.open(wav, 'r')
pids = [] # pids
if (w.getframerate() != self.sr) or (w.getnchannels() > 1):
new_wav = os.path.join(self.aud_dir, head + '.wav')
pids.append(Popen(['sox', '-G', wav, '-b', '16',
new_wav, 'remix', '-',
'rate', str(self.sr),
'dither', '-s'], stderr=PIPE))
wav = new_wav
for pid in pids: # do a join
retcode = pid.wait()
if retcode != 0:
raise CalledProcessError(retcode, 'sox')
print >> copy_scp, '"{0}" "{1}"'.format(wav, mfc)
print >> check_scp, '"{0}"'.format(mfc)
w.close()
else:
for wav in wav_list:
head = os.path.splitext(wav)[0]
mfc = os.path.join(self.aud_dir, head + '.mfc')
w = wave.open(wav, 'r')
if (w.getframerate() != self.sr) or (w.getnchannels() != 1):
error('File {0} needs resampled but Sox not found ', w)
print >> copy_scp, '"{0}" "{1}"'.format(wav, mfc)
print >> check_scp, '"{0}"'.format(mfc)
w.close()
copy_scp.close()
check_scp.close()

Factor out the sox command line in the pids.append(Popen(...)) call into a variable like cmd, and print that before running it.
That should give you a command line that you can reproduce the problem with, possibly see a more descpriptive error message, and maybe narrow the problem down by tweaking the arguments.
# ...
new_wav = os.path.join(self.aud_dir, head + '.wav')
cmd = ['sox', '-G', wav, '-b', '16',
new_wav, 'remix', '-', 'rate',
str(self.sr), 'dither', '-s']
print "About to execute command:\n%s" % ' '.join(cmd)
pids.append(Popen(cmd, stderr=PIPE))
wav = new_wav
# ...

Execute Shell Script from python with arguments

I want to execute a shell script with 3 arguments from a python script. (as described here: Python: executing shell script with arguments(variable), but argument is not read in shell script)
Here is my code:
subprocess.call('/root/bin/xen-limit %s %s %s' % (str(dom),str(result),str('--nosave'),), shell=True)
variables dom and result are containing strings.
And here is the output:
/bin/sh: --nosave: not found
UPDATE:
That is the variable "result":
c1 = ['/bin/cat', '/etc/xen/%s.cfg' % (str(dom))]
p1 = subprocess.Popen(c1, stdout=subprocess.PIPE)
c2 = ['grep', 'limited']
p2 = subprocess.Popen(c2, stdin=p1.stdout, stdout=subprocess.PIPE)
c3 = ['cut', '-d=', '-f2']
p3 = subprocess.Popen(c3, stdin=p2.stdout, stdout=subprocess.PIPE)
c4 = ['tr', '-d', '\"']
p4 = subprocess.Popen(c4, stdin=p3.stdout, stdout=subprocess.PIPE)
result = p4.stdout.read()
After that, the variable result is containing a number with mbit (for example 16mbit)
And dom is a string like "myserver"

from subprocess import Popen, STDOUT, PIPE
print('Executing: /root/bin/xen-limit ' + str(dom) + ' ' + str(result) + ' --nosave')
handle = Popen('/root/bin/xen-limit ' + str(dom) + ' ' + str(result) + ' --nosave', shell=True, stdout=PIPE, stderr=STDOUT, stdin=PIPE)
print(handle.stdout.read())
If this doesn't work i honestly don't know what would.
This is the most basic but yet error describing way of opening a 3:d party application or script while still giving you the debug you need.

Why not you save --nosave to a variable and pass the variable in subprocess

It's simpler (and safer) to pass a list consisting of the command name and its arguments.
subprocess.call(['/root/bin/xen-limit]',
str(dom),
str(result),
str('--nosave')
])
str('--nosave') is a no-op, as '--nosave' is already a string. The same may be true for dom and result as well.