I've got a simple python script in file 'bin/test':
#!/usr/bin/env python
import argparse
PROGRAM_NAME = "name"
PROGRAM_VERSION = "0.0.1"
PROGRAM_DESCRIPTION = "desc"
parser = argparse.ArgumentParser(prog=PROGRAM_NAME, description=PROGRAM_DESCRIPTION)
parser.add_argument('--version', action='version', version='%(prog)s ' + PROGRAM_VERSION)
args = parser.parse_args()
When I run it with the --version param, or --help, it prints everything OK:
$ bin/test --version
name 0.0.1
$ bin/test --help
usage: name [-h] [--version]
desc
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
When I run the file using subprocess.check_output, it doesn't get anything:
>>> subprocess.check_output(["bin/test", "--help"], stderr=subprocess.STDOUT, shell=True)
''
>>> subprocess.check_output(["bin/test", "--version"], stderr=subprocess.STDOUT, shell=True)
''
I'm using Ubuntu 11.10 with Python version:
python --version
Python 2.7.1+
I need to get the script output in tests. How should I do that?
If you're using shell=True, don't pass the program and its arguments as a list. This works:
subprocess.check_output("bin/test --help", stderr=subprocess.STDOUT, shell=True)
Edit: of course, leaving shell as False would have worked too.
Edit2: the documentation explains why
On Unix, with shell=True: If args is a string, it specifies the
command string to execute through the shell. This means that the
string must be formatted exactly as it would be when typed at the
shell prompt. This includes, for example, quoting or backslash
escaping filenames with spaces in them. If args is a sequence, the
first item specifies the command string, and any additional items will
be treated as additional arguments to the shell itself.
Related
Launching a hypothetical Python script with several arguments (with argparse) from the shell, let's say, python my_script.py --foo 7.4 --bar whatever, I want to capture this shell command from the own script and put it in a string variable.
Of course, I could save those arguments just after doing args = parser.parse_args() by converting that object into a dictionary: args_dict = vars(args). But I don't want that. What I want is to get the whole shell command python my_script.py --foo 7.4 --bar whatever as a string.
Is that possible? If not, could I still get the shell command and put it as a part of the output when saving it in a file by doing python my_script.py --foo 7.4 --bar whatever >> my_output.txt?
It's not perfectly accurate, but you can infer it like this:
import sys
print ("%s %s" % (sys.executable, " ".join(sys.argv) ) )
❯ python3 test.py --foo 7.4 --bar=whatever
/usr/bin/python3 test.py --foo 7.4 --bar=whatever
This will not catch pipes and the likes, only the command invoking python:
❯ python3 test.py | cat
/usr/bin/python3 test.py
If I run this command in ubuntu shell:
debconf-set-selections <<< 'postfix postfix/mailname string server.exmaple.com'
It runs successfully, but if I run it via python:
>>> from subprocess import run
>>> run("debconf-set-selections <<< 'postfix postfix/mailname string server.exmaple.com'", shell=True)
/bin/sh: 1: Syntax error: redirection unexpected
CompletedProcess(args="debconf-set-selections <<< 'postfix postfix/mailname string server.exmaple.com'", returncode=2)
I don't understand why python is trying to interpret whether there is redirection etc. How does one make the command successfully run so one can script installation of an application, e.g. postfix in this case via python (not a normal bash script)?
I have tried various forms with double and single quotes (as recommended in other posts), with no success.
subprocess uses /bin/sh as shell, and presumably your system's one does not support here-string (<<<), hence the error.
From subprocess source:
if shell:
# On Android the default shell is at '/system/bin/sh'.
unix_shell = ('/system/bin/sh' if
hasattr(sys, 'getandroidapilevel') else '/bin/sh')
You can run the command as an argument to any shell that supports here string e.g. bash:
run('bash -c "debconf-set-selections <<< \"postfix postfix/mailname string server.exmaple.com\""', shell=True)
Be careful with the quoting.
Or better you can stay POSIX and use echo and pipe to pass via STDIN:
run("echo 'postfix postfix/mailname string server.exmaple.com' | debconf-set-selections", shell=True)
When I call to pandoc with "-V args" like for example: '-V title="Wartość"'
in Python script, I get output without title..:(
Example:
Manually typed command to pandoca(in terminal):
/usr/bin/pandoc /home/user/program/content.md -V title="Wartość"
-V authors="Jerry" --output=/home/user/program/outputs/book_22.pdf
It works :)
output file:
pandoc output when use manually pandoc in the terminal
but when I run the same command in python(call to pandoc):
subprocess.call(['/usr/bin/pandoc', '/home/user/program/content.md', '-V title="Wartość", -V authors="Jerry" ', '--output=/home/user/program/outputs/book_33.pdf'])
output file: pandoc output when I call to him from python script
how to fix it?
Your assumption that you are running the "same command" in Python is incorrect. You have combined arguments into a single string when they should be separate.
subprocess.call(['/usr/bin/pandoc', '/home/user/program/content.md',
'-V', 'title="Wartość"', '-V', 'authors="Jerry"',
'--output=/home/user/program/outputs/book_33.pdf'])
An easy way to convert a command line to a list suitable for subprocess.call() is shlex.split().
In my python script myscript.py I use argparse to pass command-line arguments. When I want to display the help information about the input arguments, I just do:
$ python myscript.py --help
If instead I want to use ipython to run my script, the help message won't be displayed. Ipython will display its own help information:
$ ipython -- myscript.py -h
=========
IPython
=========
Tools for Interactive Computing in Python
=========================================
A Python shell with automatic history (input and output), dynamic object
introspection, easier configuration, command completion, access to the
system shell and more. IPython can also be embedded in running programs.
Usage
ipython [subcommand] [options] [files]
It's not so annoying, but is there a way around it?
You need to run your .py script inside the ipython. Something like that:
%run script.py -h
This is an IPython bug, corrected in https://github.com/ipython/ipython/pull/2663.
My 0.13 has this error; it is corrected in 0.13.2. The fix is in IPthyon/config/application.py Application.parse_command_line. This function looks for help and version flags (-h,-V) in sys.argv before passing things on to parse_known_args (hence the custom help formatting). In the corrected release, it checks sys.argv only up to the first --. Before it looked in the whole array.
earlier:
A fix for earlier releases is to define an alternate help flag in the script:
simple.py script:
import argparse, sys
print(sys.argv)
p = argparse.ArgumentParser(add_help=False) # turn off the regular -h
p.add_argument('-t')
p.add_argument('-a','--ayuda',action=argparse._HelpAction,help='alternate help')
print(p.parse_args())
Invoke with:
$ ./ipython3 -- simple.py -a
['/home/paul/mypy/argdev/simple.py', '-a']
usage: simple.py [-t T] [-a]
optional arguments:
-t T
-a, --ayuda alternate help
$ ./ipython3 -- simple.py -t test
['/home/paul/mypy/argdev/simple.py', '-t', 'test']
Namespace(t='test')
I am writing a small python script that needs to execute git commands from inside a given directory
The code is as follows:
import subprocess, os
pr = subprocess.Popen(['/usr/bin/git', 'status'],
cwd=os.path.dirname('/path/to/dir/'),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)
(out, error) = pr.communicate()
print out
But it shows git usage as the output.
If the command doesn't involve git for eg. ['ls'] then it shows the correct output.
Is there anything I am missing ?
python version - 2.6.6
Thanks.
subprocess.Popen:
On Unix, with shell=True: […] If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.
You don't want shell=True and also a list of arguments. Set shell=False.