Develop Reference

How to pass double quotes to subprocess without setting shell to true?

I need to use double quotes when passing a command to subprocess and can't use shell=true. The following is simplified version of the code for clarity:
def run_command(cmd):
if isinstance(cmd, str):
cmd = cmd.split()
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
output, error = proc.communicate()
if proc.returncode != 0:
raise subprocess.CalledProcessError(proc.returncode, cmd, error)
cmd = 'sh -c " sudo ls "'
run_command(cmd)
I tried the followings but still fails:
cmd = 'sh -c " sudo ls "' #sudo: Syntax error: Unterminated quoted string
cmd = 'sh -c "sudo ls"' #ls": Syntax error: Unterminated quoted string
cmd = 'sh -c sudo ls' #usage: sudo -h | -K | -k | -V

The use of a string input with .split() is going to get in your way here as split() will not respect your quotes.
Instead try Popen(['sh', '-c', 'sudo ls'])
Note that the double quotes are not present. This is because the entire string sudo ls (including space) is placed into one argv entry this way.

Python 2.4: Get the output of system commands into a non-quoted string

I want to write the output of a psql-command into an integer variable that I can use with some "if x larger than 0" statement in python, but I cannot get the value only.
My environment is limited. I cannot import the psycopg(2) and I cannot update so I have to work with the given commands of python 2.4.
See the following:
>>> os.popen('psql -d database -U datareader -t -A -c "select count(*) from mails"').read()
'151\n'
or
>>> commands.getoutput('psql -d database -U datareader -t -A -c "select count(*) from mails"')
'151'
Both outputs are at least singlequoted strings and I don't know how to use them as integers.
Another way is older os.system:
>>> os.system('psql -U database -d datareader -t -A -c "select count(*) from mails"')
151
0
No quotes, but the output of the command is the exit code and the commands output. Putting this into a variable puts only the exit code in, not the command output.
The latter one could go with a workaround by writing the output with appended into a file. This however, I would like to avoid.

The output from an external command is always going to be a string. Convert it to an integer manually:
output = commands.getoutput('psql -d database -U datareader -t -A -c "select count(*) from mails"')
count = int(output)

Subprocess call python script not producing Jasper Report but cmd execution does

When I run this at command line it generates my Jasper report correctly:
jasperstarter pr "C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml" -f pdf -t postgres -H localhost -n template_postgis_20 -u postgres -p postgres -P SiteID=123
However, if I then try to run it through python with the following code the report doesn't get created. Am I messing up the syntax somewhere?
import subprocess
from subprocess import call
subprocess.call(["cmd","/C","jasperstarter","pr","""C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml""","-f","pdf",
"-t","postgres","-H","localhost","-n","template_postgis_20","-u","postgres","-p","postgres",
"-P","SiteID=123"], shell=True)
EDIT:
Following the comments, I tried running this at cmd after typing python to bring up >>>:
jasperstarter pr "C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml" -f pdf -t postgres -H localhost -n template_postgis_20 -u postgres -p postgres -P SiteID=123
This time I got a syntax error at -u. I then tried reordering the parameters and the syntax error then occurred at the same character number, rather than at the -u. So is there a maximum line length when entering commands in python at cmd?

\a is a escape sequence that is same to \x07 (BEL). You should escape \ or use raw string literal to make \a represent \a literally.
>>> '\a' # not escaped
'\x07'
>>> '\\a' # escaped
'\\a'
>>> r'\a' # raw string literal
'\\a'
So, replace following:
"""C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml"""
with
"""C:\\users\\ant\\jaspersoftworkspace\\myreports\\carereport.jrxml"""
or
r"""C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml"""
UPDATE
Try following:
subprocess.call(r'jasperstarter pr "C:\users\ant\jaspersoftworkspace\myreports\carereport.jrxml" -f pdf -t postgres -H localhost -n template_postgis_20 -u postgres -p postgres -P SiteID=123', shell=True)

Bash not recognising python command

I have this command as part of a bash script
$(python -c "import urllib, sys; print urllib.unquote(sys.argv[0])", "h%23g")
But when I run it, I get this:
-bash: -c: command not found
As though bash has missed reading the python, and is thinking -c is the name of the command. Exactly the same happens when using backticks.
How can I make bash recognise the python?

the Python command is returning the string "-c" from your $(...) structure, which bash then tries to execute.
for example
python -c "import urllib, sys; print urllib.unquote(sys.argv[0])"
prints "-c", so you are essentially asking bash to interpret $(-c), for which the error is valid.

I think you want something like the following:
$(python -c "import urllib, sys; print urllib.unquote(sys.argv[1])" "h%23g")
This will result in h#g, if this is all you have on a line then it will also attempt to run a command called h#g, so I'm assuming you are actually using this as a part of a larger command.
The issue with your version is that sys.argv[0] is the -c from the command, and urllib.unquote('-c') will just return '-c'.
From the documentation on sys.argv:
If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'.
Combining that with info from the man page (emphasis mine):
-c command
Specify the command to execute (see next section). This terminates the option list (following options are passed as arguments to the command).
So, when you use -c, sys.argv[0] will be '-c', the argument provided to -c is the script so it will not be included in sys.argv, and any additional arguments are added to sys.argv starting at index 1.

python popen rsync with rsh option

I'm trying to execute a rsync command via subrocess & popen. Everything's ok until I don't put the rsh subcommand where things go wrong.
from subprocess import Popen
args = ['-avz', '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"', 'bond#localhost:/home/bond/Bureau', '/home/bond/data/user/bond/backups/']
p = Popen(['rsync'] + args, shell=False)
print p.wait()
#just printing generated command:
print ' '.join(['rsync']+args)
I've tried to escape the '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"' in many ways, but it seems that it's not the problem.
I'm getting the error
rsync: Failed to exec ssh -C -p 22 -i /home/bond/.ssh/test: No such file or directory (2)
If I copy/paste the same args that I output at the time, I'm getting a correct execution of the command.
Thanks.

What happens if you use '--rsh=ssh -C -p 22 -i /home/bond/.ssh/test' instead (I removed the double quotes).
I suspect that this should work. What happens when you cut/paste your line into the commandline is that your shell sees the double quotes and removes them but uses them to prevent -C -p etc. from being interpreted as separate arguments. when you call subprocess.Popen with a list, you've already partitioned the arguments without the help of the shell, so you no longer need the quotes to preserve where the arguments should be split.

Having the same problem, I googled this issue extensively. It would seem you simply cannot pass arguments to ssh with subprocess. Ultimately, I wrote a shell script to run the rsync command, which I could pass arguments to via subprocess.call(['rsyncscript', src, dest, sshkey]). The shell script was: /usr/bin/rsync -az -e "ssh -i $3" $1 $2
This fixed the problem.