Develop Reference

Quotes without backslash in subprocess command line in python

I'm trying to use ffmpeg from python. The command I need to execute is:
ffmpeg -i test_file-1kB.mp4 -i test_file.mp4 -filter_complex psnr="stats_file=test_file.mp4-1kB.psnr" -f null -
However, my output that is getting passed to subprocess looks like it is escaping the double quotes with backslashes like so:
In[1]: print(subprocess.list2cmdline(psnr_args))
ffmpeg -i test_file-1kB.mp4 -i test_file.mp4 -filter_complex psnr=\"stats_file=test_file.mp4-1kB.psnr\" -f null -
To use subprocess, I build my command line arguments one at a time into a list and then pass the list to subprocess.
psnr_args = []
psnr_args.append("ffmpeg")
#add first input, the encoded video
psnr_args.append("-i")
psnr_args.append(full_output_file_name)
#add second input, the original video
psnr_args.append("-i")
psnr_args.append(video_file)
#Setup the psnr log file
psnr_args.append("-filter_complex")
psnr_args.append('psnr="stats_file=%s.psnr"' % vstats_abs_filename )
#Output the video to null
psnr_args.append("-f")
psnr_args.append("null")
psnr_args.append("-")
print(subprocess.list2cmdline(psnr_args))
run_info_psnr = subprocess.run(psnr_args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

After more fiddling, I found a solution that works in this case but may not work in all cases. If I use double quotes as the outer quotes and the single quotes as the inner quotes, the output to subprocess uses a single quote at the same location with no backslash. This is acceptable for ffmpeg. However, for others where double quotes are the only solution, it won't be a fix.
psnr_args.append("psnr='stats_file=%s.psnr'" % vstats_abs_filename )
Output to subprocess looks like this:
In[1]: print(subprocess.list2cmdline(psnr_args))
ffmpeg -i test_file-1kB.mp4 -i test_file.mp4 -filter_complex psnr='stats_file=test_file.mp4-1kB.psnr' -f null -

In shell, the argument:
psnr="stats_file=test_file.mp4-1kB.psnr"
Is absolutely identical to:
psnr=stats_file=test_file.mp4-1kB.psnr
The quotes are removed during the shell's own processing. They are not part of the command passed to ffmpeg, which doesn't expect or understand them. Because you're directly telling the Python subprocess module to invoke a literal argument vector, there's no shell involved, so shell syntax shouldn't be present.

This has something to do with ffmpeg AV filter chain syntax too. You need to run the command like xxxx -filter_complex "psnr='stats.txt'" xxxx. To get this, you should ensure the double quote that encapsulate the filter chain reaches inside. subproces expects a flat list as the first argument, where the command is the first entry. So ['ffmpeg', '-i', "t1.mp4", "-filter_compelx", '"psnr=\'stats.txt\'"', .... and so on ].

How to specify data type in subprocess in python?

After executing a command using subprocess gives out result in string type.
cmd = "ping -c 2 stackoverflow.com | grep -i icmp* |wc -l"
count = subprocess.check_output([cmd],shell=True)
count
'2\n'
type(count)
<type 'str'>
But i need this count in 'int' format,except typecasting it to "int(count)" ,how can i do this at subprocess command level ...is there any option to specify the data type in subprocess itself?

Command output is always a string buffer. Just cast it to int if that's what you need.

subprocess communicates via raw data pipes, which are sent and received as str (bytes in Python 3). No other data type is possible. You just need to convert the result with count = int(count) as you mention in your question.

Ffmpeg map and filter_complex subprocess - Python

I want to crop and re encode videos via ffmpeg from within python using subprocesses.
I managed starting a subprocess using a pure string command and shell=True but I want to build more complex commands and would prefer to use shell=False and passing a list of arguments.
So what works is this form (this is a simplified example, there will be multiple streams in the final version):
import subprocess as sp
sp.Popen('ffmpeg.exe -i Test.avi -filter_complex "[0:v]crop=1024:1024:0:0[out1]" -map [out1] out1.mp4', shell=True)
This script produces the expected cropped output video.
For a list of arguments, I tried:
FFMPEG_PATH = 'ffmpeg.exe'
aviP='Test.avi'
sp.Popen([FFMPEG_PATH,
'-i', aviP,
'-filter_complex', '[0:v]crop=1024:1024:0:0[out1]',
'-map', '[out1] out1.mp4'])
When I execute this second version, simply nothing happens. (no error, no output)
I suspect I am messing up something in the map command syntax?

I think I figured it out:
FFMPEG_PATH = 'ffmpeg.exe'
aviP='Test.avi'
sp.Popen([FFMPEG_PATH,
'-i', aviP,
'-filter_complex', '[0:v]crop=1024:1024:0:0[out1]',
'-map', '[out1]', 'out1.mp4'])
is the correct syntax

Send args to subprocess while using stdin

I'm trying to take a screenshot then run a command on that screenshot without saving to disk.
The actual command I want to run is visgrep image.png pattern.pat
visgrep must have two args: the image file and a .pat file.
Here is what I have so far.
p = subprocess.Popen(['import', '-crop', '305x42+1328+281', '-window', 'root', '-depth', '8', 'png:' ], stdout=subprocess.PIPE,)
cmd = ['visgrep']
subprocess.call(cmd, stdin=p.stdout)
Obviously this fails as visgrep must have two args.
So how can I do visgrep image.png pattern.pat but substituting 'image.png' with the output of ImageMagick's import?
Do I need to use xargs? Is there a better way to accomplish what I'm trying?

In linux you can use /dev/stdin as file name but it does not work all the times. If it does not work with visgrep, you must use a temporary file (which is not a shame).
PS. shouldn't png: be png:-?

According to this answer, changing the argument png: to png:- will cause the import command to output to standard out instead of a file. I am unfamiliar with visgrep, so I'm not sure how to tell it to read the source image from stdin.

From the ImageMagick documentation:
STDIN, STDOUT, and file descriptors
Unix and Windows permit the output of one command to be piped to the
input of another. ImageMagick permits image data to be read and
written from the standard streams STDIN (standard in) and STDOUT
(standard out), respectively, using a pseudo-filename of -. In this
example we pipe the output of convert to the display program:
$ convert logo: gif:- | display gif:-
The second explicit format "gif:" is optional in the preceding
example. The GIF image format has a unique signature within the image
so ImageMagick's display command can readily recognize the format as
GIF. The convert program also accepts STDIN as input in this way:
$ convert rose: gif:- | convert - -resize "200%" bigrose.jpg
You can use the same filename convention with the import command.
So, try:
p = subprocess.Popen(['import', '-crop', '305x42+1328+281',
'-window', 'root', '-depth', '8', 'png:-' ],
stdout=subprocess.PIPE,)

Getting output of system commands that use pipes (Python)

I'm trying to generate a random string using this command:
strings /dev/urandom | grep -o '[[:alnum:]]' | head -n 30 | tr -d '\n';
Works fine, but when I try to do subprocess.call(cmd,shell=True) it just gets stuck on the strings /dev/urandom command and spams my screen with grep: writing output: Broken pipe
What's causing this and how do I fix it?

No need for subprocess, observe:
>>> import base64
>>> r = open("/dev/urandom","r")
>>> base64.encodestring(r.read(22))[:30]
'3Ttlx6TT3siM8h+zKm+Q6lH1k+dTcg'
>>> r.close()
Also, stringsing and then greping alphanumeric characters from /dev/urandom is hugely inefficient and wastes a whole lot of randomness. On my desktop PC, the above python takes less than 10 ms to executed from bash, your strings ... oneliner takes 300-400...
For a pure python solution that works also on systems without /dev/urandom - and gives only alphanumeric characters (if you really don't want + or /):
import string
import random
''.join([random.choice(string.printable[:62]) for i in range(30)])

First of all, for what you're doing, it should be better to generate the string using python directly.
Anyway, when using subprocess, the correct way to pipe data from a process to another is by redirecting stdout and/or stderr to a subprocess.PIPE, and feed the new process' stdin with the previous process' stdout.