Develop Reference

Bash command not working in Python

I'm trying to execute the next bash command either from Python or Perl:
googlesamples-assistant-pushtotalk --credentials /home/jsti/.config/google-oauthlib-tool/credentials.json
--device-model-id 'pbx-assista' --device-id 'pbx' -i /tmp/google_audio1314_in.wav -o /tmp/google_audio1314_out.wav -v
Basically the idea is to send an audio to Google Assistant, after that, it should answer me the audio with another audio. I should receive an audio file as a response from Google Assistant but I don't receive it. There is no errors but the file does not arrive.
The command works properly if I execute it in the terminal.
Does anyone know what it is happening with this command?
This is the code:
#!/usr/bin/env python
import sys
from asterisk.agi import *
import subprocess
command = "googlesamples-assistant-pushtotalk"
oauth_dir = "/home/jsti/.config/google-oauthlib-tool/credentials.json"
audio_in = "/tmp/google_audio1314_in.wav"
audio_out = "google_audio1314_out.wav"
agi = AGI()
agi.verbose("python agi started")
callerId = agi.env['agi_callerid']
agi.verbose("call from %s" % callerId)
while True:
args = [command, '--credentials', oauth_dir, '--device-model-id', '"pbx-assista"', '--device-id', '"pbx"', '-i', audio_in, '-o', audio_out, '-v' ]
subprocess.Popen(args)

Get rid of the double quotes around "pbx-assista" and "pbx".
args = [command, '--credentials', oauth_dir, '--device-model-id', 'pbx-assista', '--device-id', 'pbx', '-i', audio_in, '-o', audio_out, '-v']

The code in use here doesn't actually wait to allow the subprocess to exit (and doesn't look at whether it succeeded or not, and so can't detect and report errors).
Change:
subprocess.Popen(args)
...to...
subprocess.check_call(args)
...or...
p = subprocess.Popen(args)
p.wait()
Also, you'll want to change '"pbx"' to just 'pbx'; the double quotes in the original bash version are syntactic, just like the single quotes in the Python version are -- you don't need literal quotes in addition to the syntactic ones. (Bash optionally allows syntactic quotes to be left out when they aren't needed to prevent unwanted expansion, make otherwise-syntactically-significant characters literal, or the like; with Python, they're always mandatory when defining a string)

Bash script will not run with subprocess in Python

For some reason, no matter how many variations I've tried, I can't seem to execute a bash script I've written. The command words 100% fine in Terminal, but when I try calling it with a subprocess, it returns nothing.
from os import listdir
import subprocess
computer_name = 'homedirectoryname'
moviefolder = '/Users/{}/Documents/Programming/Voicer/Movies'.format(computer_name)
string = 'The lion king'
for i in listdir(moviefolder):
title = i.split('.')
formatted_title = title[0].replace(' ', '\ ')
if string.lower() == title[0].lower():
command = 'vlc {}/{}.{}'.format(moviefolder, formatted_title, title[1])
subprocess.call(["/usr/local/bin",'-i','-c', command], stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
else:
continue
The bash executable file looks like this:
#/bin/bash
func() {
open -a /Applications/VLC.app/Contents/MacOS/VLC $1
}
Where have I gone wrong?

You should call open directly:
import os
import subprocess
computer_name = 'homedirectoryname'
moviefolder = '/Users/{}/Documents/Programming/Voicer/Movies'.format(computer_name)
string = 'The lion king'
for filename in os.listdir(moviefolder):
title = filename.split('.')
if string.lower() == title[0].lower():
subprocess.call(['open', '-a', '/Applications/VLC.app/Contents/MacOS/VLC', os.path.join(moviefolder, filename)])

Since you are using shell=True, the command must be a string:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
This means that the string must be formatted exactly as it would be
when typed at the shell prompt. This includes, for example, quoting or
backslash escaping filenames with spaces in them. If args is a
sequence, the first item specifies the command string, and any
additional items will be treated as additional arguments to the shell
itself. (docs)

Like you even mentioned in a comment, you get /usr/local/bin: is a directory when you properly capture the error from the shell (and take out the erroneous shell=True; or correspondingly refactor the command line to be suitable for this usage, i.e. pass a string instead of a list).
Just to spell this out, you are attempting to run the command /usr/local/bin with some options; but of course, it's not a valid command; so this fails.
The actual script you seem to want to run will declare a function and then exit, which results in the function's definition being lost again, because the subprocess which ran the shell in which this function declaration was executed has now terminated and released all its resources back to the system.
Perhaps you should take more than just a few steps back and explain what you actually want to accomplish; but really, that should be a new, separate question.
Assuming you are actually trying to run vlc, and guessing some other things, too, perhaps you actually want
subprocess.call(['vlc','{}/{}.{}'.format(moviefolder, formatted_title, title[1]),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
If your PATH is correct, you should not need to specify /usr/local/bin/ explicitly (and if your PATH is wrong, correct it in the code before, instead of hardcoding a directory for the executable you want to call).

/usr/local/bin is a directory. You can't run a directory as if it were a command.
Anyhow, there's no point to having /usr/local/bin anywhere in your command at all. Leave out the shell=True, and explicitly call vlc:
subprocess.call([
'vlc',
'{}/{}.{}'.format(moviefolder, formatted_title, title[1])
])

When shell=True is used in subprocess.call, if the command arguments is a sequence, then the first element of the sequence needs to be the command, and the rest are treated as argument(s) to the shell itself.
So, this should do:
subprocess.call(["/usr/local/bin/{}".format(command), '-i','-c'], shell=True, ...)
Otherwise, you can make the command a string.
Example:
In [20]: subprocess.call(["cat spamegg", "-i", "-c"], shell=True)
foobar

Python subprocess.check_output(args) fails, while args executed via Windows command line work OK

Some problems with python subprocess.check_output.
output = subprocess.check_output(args)
where my args is:
args = "C:\\DO\\bin\\Config.exe --ChCfg7 --LFE -b1152000 C:\\DO\\PCM\\1.wav C:\\DO\\PCM\\2.wav C:\\DO\\PCM\\3.wav C:\\DO\\PCM\\4.wav C:\\DO\\PCM\\5.wav C:\\DO\\PCM\6.wav --ModeBCast -oC:\\DO\\OUT\\outfile > C:\\DO\\OUT\\log.txt
This works when executed from standard windows command line, but doesn't work when executed via Python subprocess.check_output. In win cmd case there is output file produced and log.txt too, and python script produces out file with size 0, and no log.txt at all.

output = subprocess.check_output(args,shell=True)
Run this with shell=True

Use a list of args and redirect the output to a file:
import subprocess
args = ['C:/DO/bin/Config.exe', '--ChCfg7', '--LFE', '-b1152000', 'C:/DO/PCM/1.wav', 'C:/DO/PCM/2.wav', 'C:/DO/PCM/3.wav', 'C:/DO/PCM/4.wav', 'C:/DO/PCM/5.wav', 'C:/DO/PCM/6.wav', '--ModeBCast', '-oC:/DO/OUT/outfile']
with open("C:/DO/OUT/log.txt", "w") as f:
subprocess.check_call(args, stdout=f)
You can use shell=Truebut for security reasons generally it is not a very good idea and the same can be quite easily achieved using the code above and simply redirecting the output to the file.

> is a shell redirection operator. Either run the command in a shell or (better) as #Padraic Cunningham suggested emulate it in Python:
#!/usr/bin/env python
import subprocess
args = r"C:\DO\bin\Config.exe --ChCfg7 --LFE -b1152000".split()
args += [r'C:\DO\PCM\%d.wav' % i for i in range(1, 7)]
args += ["--ModeBCast", r"-oC:\DO\OUT\outfile"]
with open(r"C:\DO\OUT\log.txt", "wb", 0) as output_file:
subprocess.check_call(args, stdout=output_file)
The code uses raw string literals for Windows paths to avoid escaping backslashes.
There is usually no point to use shell=True on Windows unless you want to run a built-in command such as dir. If args is not constructed using input from an external source then security considerations do not apply. shell=True starts additional process (%COMSPEC%) and it changes how the executable is searched and it changes what characters should be escaped (what characters are metacharacters) — do not use shell=True unless necessary.

Python Subprocess Error in using "cp"

I was trying to use subprocess calls to perform a copy operation (code below):
import subprocess
pr1 = subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = True)
and I got an error saying:
cp: missing file operand
Try `cp --help' for more information.
When I try with shell=False , I get
cp: cannot stat `./testdir1/*': No such file or directory
How do I get around this problem?
I'm using RedHat Linux GNOME Deskop version 2.16.0 and bash shell and Python 2.6
P.S. I read the question posted in Problems with issuing cp command with Popen in Python, and it suggested using shell = True option, which is not working for me as I mentioned :(

When using shell=True, pass a string, not a list to subprocess.call:
subprocess.call('cp -r ./testdir1/* ./testdir2/', shell=True)
The docs say:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
This means that the string must be formatted exactly as it would be
when typed at the shell prompt. This includes, for example, quoting or
backslash escaping filenames with spaces in them. If args is a
sequence, the first item specifies the command string, and any
additional items will be treated as additional arguments to the shell
itself.
So (on Unix), when a list is passed to subprocess.Popen (or subprocess.call), the first element of the list is interpreted as the command, all the other elements in the list are interpreted as arguments for the shell. Since in your case you do not need to pass arguments to the shell, you can just pass a string as the first argument.

This is an old thread now, but I was just having the same problem.
The problem you were having with this call:
subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = False)
was that each of the parameters after the first one are quoted. So to the shell sees the command like this:
cp '-r' './testdir1/*' './testdir2/'
The problem with that is the wildcard character (*). The filesystem looks for a file with the literal name '*' in the testdir1 directory, which of course, is not there.
The solution is to make the call like the selected answer using the shell = True option and none of the parameters quoted.

I know that the option of shell=True may be tempting but it's always inadvisable due to security issues. Instead, you can use a combination of the subprocess and glob modules.
For Python 3.5 or higher:
import subprocess
import glob
subprocess.run(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])
For Python 3.4 or lower:
import subprocess
import glob
subprocess.call(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])

Python Subprocess Grep

I am trying to use the grep command in a python script using the subprocess module.
Here's what I have:
userid = 'foo12'
p = subprocess.Popen(['grep', "%s *.log"%userid], stdout=subprocess.PIPE)
And it returns nothing.
I am not entirely sure what I am doing wrong so can someone please explain. The current method that I am using that works is by adding the shell=true which makes it output the correct output but as the help pages have pointed out it is unsafe. I need help trying to make this work so that my script isn't unsafe.

I think you're running up against two problems:
This call:
p = subprocess.Popen(['grep', "%s *.log"%userid]...
will not work as expected without shell=True because the list of arguments are being passed directly to os.execvp, which requires each item to be a single string representing an argument. You've squished two separate arguments together into a single string (in other words, grep is interpreting "foo12 *.log" as the pattern to search, and not pattern+file list).
You can fix this by saying:
p = subprocess.Popen(['grep', userid, '*.log']...)
The second issue is that, again without shell=True, execvp doesn't know what you mean by *.log and passes it directly along to grep, without going through the shell's wildcard expansion mechanism. If you don't want to use shell=True, you can instead do something like:
import glob
args = ['grep', userid]
args.extend(glob.glob('*.log')
p = subprocess.Popen(args, ...)

Here are two tested pieces of code to model from:
>>> print subprocess.check_output(['grep', 'python', 'api_talk.txt'])
Discuss python API patterns
Limitations of python
Introspection in python
>>> print subprocess.check_output('grep python *.txt', shell=True)
Use the latter if you want the shell to do wildcard expansion for you. When shell is True, be sure to put the whole command in a single string rather than a list of separate fields.

I am assuming you want to grep for 'foo12' in all files that end with '.log', to get this to work with just subprocess you will need to change your code to the following:
userid = 'foo12'
p = subprocess.Popen('grep %s *.log' % userid, stdout=subprocess.PIPE, shell=True)
shell=True is necessary for the wildcard expansion, and when that option is set you need to provide a string command instead of a list.
Also, make sure when you are providing a list of arguments that each argument is a separate entry in the list, your initial code would have been equivalent to the following:
grep 'foo12 *.log'