Develop Reference

Bash script will not run with subprocess in Python

For some reason, no matter how many variations I've tried, I can't seem to execute a bash script I've written. The command words 100% fine in Terminal, but when I try calling it with a subprocess, it returns nothing.
from os import listdir
import subprocess
computer_name = 'homedirectoryname'
moviefolder = '/Users/{}/Documents/Programming/Voicer/Movies'.format(computer_name)
string = 'The lion king'
for i in listdir(moviefolder):
title = i.split('.')
formatted_title = title[0].replace(' ', '\ ')
if string.lower() == title[0].lower():
command = 'vlc {}/{}.{}'.format(moviefolder, formatted_title, title[1])
subprocess.call(["/usr/local/bin",'-i','-c', command], stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
else:
continue
The bash executable file looks like this:
#/bin/bash
func() {
open -a /Applications/VLC.app/Contents/MacOS/VLC $1
}
Where have I gone wrong?

You should call open directly:
import os
import subprocess
computer_name = 'homedirectoryname'
moviefolder = '/Users/{}/Documents/Programming/Voicer/Movies'.format(computer_name)
string = 'The lion king'
for filename in os.listdir(moviefolder):
title = filename.split('.')
if string.lower() == title[0].lower():
subprocess.call(['open', '-a', '/Applications/VLC.app/Contents/MacOS/VLC', os.path.join(moviefolder, filename)])

Since you are using shell=True, the command must be a string:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
This means that the string must be formatted exactly as it would be
when typed at the shell prompt. This includes, for example, quoting or
backslash escaping filenames with spaces in them. If args is a
sequence, the first item specifies the command string, and any
additional items will be treated as additional arguments to the shell
itself. (docs)

Like you even mentioned in a comment, you get /usr/local/bin: is a directory when you properly capture the error from the shell (and take out the erroneous shell=True; or correspondingly refactor the command line to be suitable for this usage, i.e. pass a string instead of a list).
Just to spell this out, you are attempting to run the command /usr/local/bin with some options; but of course, it's not a valid command; so this fails.
The actual script you seem to want to run will declare a function and then exit, which results in the function's definition being lost again, because the subprocess which ran the shell in which this function declaration was executed has now terminated and released all its resources back to the system.
Perhaps you should take more than just a few steps back and explain what you actually want to accomplish; but really, that should be a new, separate question.
Assuming you are actually trying to run vlc, and guessing some other things, too, perhaps you actually want
subprocess.call(['vlc','{}/{}.{}'.format(moviefolder, formatted_title, title[1]),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
If your PATH is correct, you should not need to specify /usr/local/bin/ explicitly (and if your PATH is wrong, correct it in the code before, instead of hardcoding a directory for the executable you want to call).

/usr/local/bin is a directory. You can't run a directory as if it were a command.
Anyhow, there's no point to having /usr/local/bin anywhere in your command at all. Leave out the shell=True, and explicitly call vlc:
subprocess.call([
'vlc',
'{}/{}.{}'.format(moviefolder, formatted_title, title[1])
])

When shell=True is used in subprocess.call, if the command arguments is a sequence, then the first element of the sequence needs to be the command, and the rest are treated as argument(s) to the shell itself.
So, this should do:
subprocess.call(["/usr/local/bin/{}".format(command), '-i','-c'], shell=True, ...)
Otherwise, you can make the command a string.
Example:
In [20]: subprocess.call(["cat spamegg", "-i", "-c"], shell=True)
foobar

How to handle ':' in command using os.popen

I am trying to call a program with:
os.popen("program -s:'*' -c:'A;B;C;'")
However, it seems that it was interpreted as shell command:
program -s '*' -c 'A;B;C;'
which result incorrect behavior.
Can somebody help me on how to hanle such situdations where ':' is inside shell commandline?

Don't use os.popen(), use the subprocess module instead:
import subprocess
result = subprocess.check_output(['program', "-s:'*'", "-c:'A;B;C;'"])
This returns the output of the program without running it through a shell, passing in the arguments directly without any additional parsing.

subprocess not working with change directory in python

I'm doing this simple thing
import subprocess
with cd("/home/myuserid"):
subprocess.call("ls ")
where cd is taken from here and it just does not work (same with any other path):
OSError: [Errno 2] No such file or directory

You have an extra space after ls, which is causing your issue. Remove that and it should work fine.
import subprocess
with cd("/home/myuserid"):
subprocess.call("ls")
When you use subprocess without shell=True, it interprets the entire string you pass as the command to execute. So it looks for a program literally called "ls " when you provide that extra space, which of course doesn't exist.
If you were to use shell=True, it would work fine even with the extra space, because a /bin/sh shell would be used to run the command, and the shell wouldn't care about the extra space. In general it's safer to use the default of shell=False, though, so I'd stick with that.

Python Subprocess Error in using "cp"

I was trying to use subprocess calls to perform a copy operation (code below):
import subprocess
pr1 = subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = True)
and I got an error saying:
cp: missing file operand
Try `cp --help' for more information.
When I try with shell=False , I get
cp: cannot stat `./testdir1/*': No such file or directory
How do I get around this problem?
I'm using RedHat Linux GNOME Deskop version 2.16.0 and bash shell and Python 2.6
P.S. I read the question posted in Problems with issuing cp command with Popen in Python, and it suggested using shell = True option, which is not working for me as I mentioned :(

When using shell=True, pass a string, not a list to subprocess.call:
subprocess.call('cp -r ./testdir1/* ./testdir2/', shell=True)
The docs say:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
This means that the string must be formatted exactly as it would be
when typed at the shell prompt. This includes, for example, quoting or
backslash escaping filenames with spaces in them. If args is a
sequence, the first item specifies the command string, and any
additional items will be treated as additional arguments to the shell
itself.
So (on Unix), when a list is passed to subprocess.Popen (or subprocess.call), the first element of the list is interpreted as the command, all the other elements in the list are interpreted as arguments for the shell. Since in your case you do not need to pass arguments to the shell, you can just pass a string as the first argument.

This is an old thread now, but I was just having the same problem.
The problem you were having with this call:
subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = False)
was that each of the parameters after the first one are quoted. So to the shell sees the command like this:
cp '-r' './testdir1/*' './testdir2/'
The problem with that is the wildcard character (*). The filesystem looks for a file with the literal name '*' in the testdir1 directory, which of course, is not there.
The solution is to make the call like the selected answer using the shell = True option and none of the parameters quoted.

I know that the option of shell=True may be tempting but it's always inadvisable due to security issues. Instead, you can use a combination of the subprocess and glob modules.
For Python 3.5 or higher:
import subprocess
import glob
subprocess.run(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])
For Python 3.4 or lower:
import subprocess
import glob
subprocess.call(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])

Pydev: Send stdout to a real (tty) terminal

Following up on a previous question ( subprocess: PyDev console vs. cmd.exe ), is there a way to change where PyDev sends stdout--namely to a tty terminal?
I've come across several instances where not having a tty terminal has been limiting. For the case of the subprocess module, I can use the CREATE_NEW_CONSOLE flag, but in other instances, such as in this question ( Print \r correctly in console ), the PyDev console doesn't seem to support using escape characters.
Any thoughts are appreciated.

I normally deal with issues like this through the logging module in the standard library, which is quite good, but I'll assume you have good reason for wanting this.
I would be surprised if the PyDev console supported full terminal emulation. At least under Helios on Windows, I've had no problem with Unicode display, but terminal escapes are another matter.
If you know specifically which terminal you want to use, you can run sleep 3600 in it and then do this in your test driver:
import sys
def redirect_terminal(ttypath):
term = open(ttypath, 'w+')
sys.stdout = term
sys.stderr = term
Trying this in the interactive interpreter, which will likely be a bit different from running it in PyDev, I get this in the initial terminal (note local echo and prompt still returned here):
>>> redirect_terminal('/dev/pts/0')
>>> dir()
>>> raise TypeError
>>>
and this in the /dev/pts/0 terminal:
['__builtins__', '__doc__', '__name__', '__package__', 'redirect_terminal', 'sys']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError
While I did not try any terminal escapes here, they are just byte sequences that get printed like any other, so they should be printed on the remote terminal.
I cannot manage to collect input from a different terminal in the interactive interpreter. When I try, input is still read from the initial terminal.

This is currently a limitation in Eclipse... (which PyDev inherits).
Aptana Studio does have a terminal view which could probably be used as a replacement, but there are no plans to do so for now.
Answering comment below, to create a new shell from a running Python program it's possible to use the code below:
import subprocess
import sys
import os
args = [sys.executable] + sys.argv
new_environ = os.environ.copy()
if hasattr(subprocess, 'CREATE_NEW_CONSOLE'):
popen = subprocess.Popen(args, env=new_environ, creationflags=subprocess.CREATE_NEW_CONSOLE)
exit_code = popen.wait()
else:
#On Linux, CREATE_NEW_CONSOLE is not available, thus, we use xterm itself.
args = ['xterm', '-e'] + args
popen = subprocess.Popen(args, env=new_environ)
popen.wait() #This exit code will always be 0 when xterm is executed.