Develop Reference

Python argparse proper formatting of choice arguments with flag

I'm trying to create well formatted help messages for 'choice' type command line arguments with Python's argparse. For the command I allow the name '--operation' and the alias '-o'. Currently, argparse is printing the list of options next to both in the help message.
Please note that this question is different to the question of formatting the help messages of the options (That problem has a good answer here by Anthon: Python argparse: How to insert newline in the help text?)
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-o', '--operation', help="operation to perform", type=str, choices=["create", "update", "delete"])
_StoreAction(option_strings=['-o', '--operation'], dest='operation', nargs=None, const=None, default=None, type=<class 'str'>, choices=['create', 'update', 'delete'], help='operation to perform', metavar=None)
>>> parser.print_help()
usage: [-h] [-o {create,update,delete}]
optional arguments:
-h, --help show this help message and exit
-o {create,update,delete}, --operation {create,update,delete}
operation to perform
>>>
My problem is this line:
-o {create,update,delete}, --operation {create,update,delete}
It's very clunky how the list of options is repeated twice. Especially since I will have lists that are even longer. It would be better I think to have this:
-o, --operation {create,update,delete}
This is assuming of course that there isn't some POSIX rule about how this has to work. I don't think there is.
How can I achieve the desired output? Or is there a good reason that I shouldn't be trying to?

This is quite a hack, but there doesn't appear to be a good place to hook into this.
Define your own formatter, which overrides (by basically copying) the _format_action_invocation method. The only change you'll make is to add the choices only to the last option string.
class MyHelpFormatter(HelpFormatter):
def _format_action_invocation(self, action):
if not action.option_strings:
default = self._get_default_metavar_for_positional(action)
metavar, = self._metavar_formatter(action, default)(1)
return metavar
else:
parts = []
# if the Optional doesn't take a value, format is:
# -s, --long
if action.nargs == 0:
parts.extend(action.option_strings)
# if the Optional takes a value, format is:
# -s ARGS, --long ARGS
else:
default = self._get_default_metavar_for_optional(action)
args_string = self._format_args(action, default)
for option_string in action.option_strings[:-1]:
parts.append('%s' % (option_string,))
parts.append('%s %s' % (action.option_strings[-1], args_string)
return ', '.join(parts)

how to set argparse to exclude optional argument by values of positional argument

the arg include an action field and optional switches that modify the behavior of the actions.
the argparse code is like the below:
parser=argparse.ArgumentParser()
parser.add-argument('action',metavar='action', choices=['analysis','report','update'],nargs='?', default='report')
parser.add-argument('-d',dest='delay',type=int, choices=range(1,10),default=1)
parser.add-argument('-v',dest='verbose',action='store-true',default=False)
parser.add-argument('-o',dest='offline',action='store-true',default=False)
parser.add-argument('-n',dest='names',required=False)
i want to make the switch option -o, -d, -v only available for action=report, while option -n only available for action=analysis.
i know there is a mutual group setting, but it is just set for the arguments not for the argument values!
btw, does argparse support combine options, like -vo...???

First: Yes, combined options like -vo are supported
Now to the action switching:
Adding a SubParser to your ArgumentParser is exactely what you want to do.
See Section 15.4.5.1. Sub-commands in the Documentation of argparse, everything is explained to some detail and with example there
EDIT:
I know refer to your comment below this post:
If you do not provide a subcommand parameter in the program call, the parser normally gives you a kind advice how to use the program and then exits with a "too few arguments error"
I rewrote your code to show this
test.py
import argparse
parser=argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
parser_analysis = subparsers.add_parser('analysis', help='analysis help text')
parser_analysis.add_argument('-n',dest='names',required=False)
parser_report = subparsers.add_parser('report', help='report help text')
parser_report.add_argument('-d',dest='delay',type=int, choices=range(1,10),default=1)
parser_report.add_argument('-v',dest='verbose',action='store_true',default=False)
parser_report.add_argument('-o',dest='offline',action='store_true',default=False)
parser_update = subparsers.add_parser('update', help='update help text')
parser.parse_args()
Now some calls of this test.py with different arguments:
$python test.py
usage: test.py [-h] {analysis,report,update} ...
test.py: error: too few arguments
$python test.py -h
usage: test.py [-h] {analysis,report,update} ...
positional arguments:
{analysis,report,update}
sub-command help
analysis analysis help text
report report help text
update update help text
optional arguments:
-h, --help show this help message and exit
$python test.py report -h
usage: test.py report [-h] [-d {1,2,3,4,5,6,7,8,9}] [-v] [-o]
optional arguments:
-h, --help show this help message and exit
-d {1,2,3,4,5,6,7,8,9}
-v
-o
So as I see it the only problem is that the program throws an error after calling python test.py without any subcommand. So I would do s.th. like this
try:
args=parser.parse_args()
except:
exit(0)
to avoid that the user sees that there was an unhandled error. You have then the same behaviour as i.e. the svn command.
If you want to handle this in the way that a default subcommand is executed, you whould have to do s.th. like mentioned in this post answer:
Argparse - How to Specify a Default Subcommand
import sys
#...your parser definitions
if (len(sys.argv) < 2):
args = parser.parse_args(['update'])
else:
args = parser.parse_args()
This would parse a command update if the argument list in sys.argv is smaller than 2. Why 2? Because the first argument in the argument list is always the program which you called, i.e. test.py
The question is, do you really want this behaviour? Because there is no need for calling test.py update if i can always call test.py, so users will probably get lazy and never use test.py update command. Also if you later want a different default behaviour like test.py starting an interactive mode, users which are by then used to calling test.py for updating will get confused or their scripts which use your program get broken.

Argparse: Way to include default values in '--help'?

Suppose I have the following argparse snippet:
diags.cmdln_parser.add_argument( '--scan-time',
action = 'store',
nargs = '?',
type = int,
default = 5,
help = "Wait SCAN-TIME seconds between status checks.")
Currently, --help returns:
usage: connection_check.py [-h]
[--version] [--scan-time [SCAN_TIME]]
Test the reliability/uptime of a connection.
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
I would prefer something like:
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
(Default = 5)
Peeking at the help formatter code revealed limited options. Is there a clever way to get argparse to print the default value for --scan-time in a similar fashion, or should I just subclass the help formatter?

Use the argparse.ArgumentDefaultsHelpFormatter formatter:
parser = argparse.ArgumentParser(
# ... other options ...
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
To quote the documentation:
The other formatter class available, ArgumentDefaultsHelpFormatter, will add information about the default value of each of the arguments.
Note that this only applies to arguments that have help text defined; with no help value for an argument, there is no help message to add information about the default value to.
The exact output for your scan-time option then becomes:
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
(default: 5)

Add '%(default)s' to the help parameter to control what is displayed.
parser.add_argument("--type", default="toto", choices=["toto","titi"],
help = "type (default: %(default)s)")
Notes:
It is %+ default in parenthesis + format characters (not to be confused with curly brackets {default} we find in format or f-string)
Don't forget to add the "specifier character" for the type representation at the end (i.e. s for strings, d for integers, f for floats, etc.)
You can also add the usual "printf" format specifiers (like number of digits for floats, leading zeros, etc.)
You can refer to printf documentation for more details.

Wrapper class
This is the most reliable and DRY approach I've found so far to both show defaults and use another formatter such as argparse.RawTextHelpFormatter at the same time:
#!/usr/bin/env python3
import argparse
class ArgumentParserWithDefaults(argparse.ArgumentParser):
def add_argument(self, *args, help=None, default=None, **kwargs):
if help is not None:
kwargs['help'] = help
if default is not None and args[0] != '-h':
kwargs['default'] = default
if help is not None:
kwargs['help'] += ' Default: {}'.format(default)
super().add_argument(*args, **kwargs)
parser = ArgumentParserWithDefaults(
formatter_class=argparse.RawTextHelpFormatter
)
parser.add_argument('-a', default=13, help='''my help
for a''')
parser.add_argument('-b', default=42, help='''my help
for b''')
parser.add_argument('--no-default', help='''my help
for no-default''')
parser.add_argument('--no-help', default=101)
parser.print_help()
print()
print(parser.parse_args())
Output:
usage: main.py [-h] [-a A] [-b B] [--no-default NO_DEFAULT]
[--no-help NO_HELP]
optional arguments:
-h, --help show this help message and exit
-a A my help
for a Default: 13
-b B my help
for b Default: 42
--no-default NO_DEFAULT
my help
for no-default
--no-help NO_HELP
Namespace(a=13, b=42, no_default=None, no_help=101)
ArgumentDefaultsHelpFormatter + RawTextHelpFormatter multiple inheritance
Multiple inheritance just works, but it does not seem to be public API:
#!/usr/bin/env python3
import argparse
class RawTextArgumentDefaultsHelpFormatter(
argparse.ArgumentDefaultsHelpFormatter,
argparse.RawTextHelpFormatter
):
pass
parser = argparse.ArgumentParser(
formatter_class=RawTextArgumentDefaultsHelpFormatter
)
parser.add_argument('-a', default=13, help='''my help
for a''')
parser.add_argument('-b', default=42, help='''my help
for b''')
parser.print_help()
Output:
usage: a.py [-h] [-a A] [-b B]
optional arguments:
-h, --help show this help message and exit
-a A my help
for a (default: 13)
-b B my help
for b (default: 42)
It just works works because as we can see trivially from the sources https://github.com/python/cpython/blob/v3.6.5/Lib/argparse.py#L648 that:
RawTextHelpFormatter implements _split_lines
ArgumentDefaultsHelpFormatter implements _get_help_string
so we can guess that they will work together just fine.
However, this does not seem to be public API, and neither are the methods of formatter_class, so I don't think there is a public API way to do it currently. argparse docstring says:
All other classes in this module are considered implementation details.
(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
considered public as object names -- the API of the formatter objects is
still considered an implementation detail.)
See also: Customize argparse help message
Tested on Python 3.6.5.

It is often useful to be able to automatically include the default values in the help output, but only those that were explicitly specified (with default=..). The methods already mentioned have some shortcomings in this respect:
The ArgumentDefaultsHelpFormatter method prints out (default: None) for every argument whose default was not explicitly specified, and (default: False) for 'flags' (action='store_true'). This clutters the help output. To avoid it, default=argparse.SUPPRESS needs to be manually added for each such argument.
The '%(default)s' method requires manually adding it to all the arguments' help strings that we do want printed in help.
Both methods end up needing manual intervention to print out only the "right" defaults.
One way to do this automatically is to augment the ArgumentDefaultsHelpFormatter to ignore the Nones and Falses default values:
class ExplicitDefaultsHelpFormatter(argparse.ArgumentDefaultsHelpFormatter):
def _get_help_string(self, action):
if action.default in (None, False):
return action.help
return super()._get_help_string(action)
Use it in place of ArgumentDefaultsHelpFormatter:
parser = argparse.ArgumentParser(
formatter_class=ExplicitDefaultsHelpFormatter
)
This will print only the explicitly set default values in the help output.
Note: if an argument's default was explicitly set as None or False, it won't be shown in help with this class; add %(default)s string to help for that argument if you want it in the help output.

Writing a help for python script

I am trying to make my python script very user friendly, so I like to write some sort of help for it. What is your advise for this? I could just put in some logic that if the user passed help as a paramater to the script, they get help. Is there a best practise or convention for this?

Use argparse.
For example, with test.py:
import argparse
parser=argparse.ArgumentParser(
description='''My Description. And what a lovely description it is. ''',
epilog="""All is well that ends well.""")
parser.add_argument('--foo', type=int, default=42, help='FOO!')
parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
args=parser.parse_args()
Running
% test.py -h
yields
usage: test.py [-h] [--foo FOO] [bar [bar ...]]
My Description. And what a lovely description it is.
positional arguments:
bar BAR!
optional arguments:
-h, --help show this help message and exit
--foo FOO FOO!
All is well that ends well.

Best practice is to use argparse to handle all your commandline arguments. It includes a default --help which you can customize to your likings.
Here's the simplest example:
import argparse
parser = argparse.ArgumentParser(description='This is my help')
args = parser.parse_args()
Which results in:
% python argparse_test.py -h
usage: argparse_test.py [-h]
This is my help
optional arguments:
-h, --help show this help message and exit
You can define all your arguments with argparse and set a help message for each one of them. The resulting filtered/validated arguments are returned by parser.parse_args().

An alternative to the built-in argparse is a 3rd-party package called Click which features "automatic help page generation" and "arbitrary nesting of commands" (which also produces nested help pages). Internally, it's based on argparse, but, for me, makes the creation of complex CLI more convenient using decorators.
Here's a sample code:
import click
#click.command()
#click.argument("things", nargs=-1)
#click.option("-t", show_default=True, default="int", help="Data type")
#click.option("-o", help="Output format")
def combine(things, t):
"""Combines things into a single element"""
pass
if __name__ == "__main__":
combine()
And the generated help page:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] [THINGS]...
Combines things into a single element
Options:
-t TEXT Data type [default: int]
-o TEXT Output format
--help Show this message and exit.
One of the nice things about it is that it uses the method docstrings as part of the help page, which is convenient because the docstring can now be used both for developer documentation and for script usage help.
You can also have nested command groups:
import click
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for add")
def add(numbers, e):
"""Adds numbers"""
print(f"This method should add {numbers}")
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for mul")
def mul(numbers, e):
"""Multiplies numbers"""
print(f"This method should multiply {numbers}")
#click.group()
def calc():
pass
calc.add_command(add)
calc.add_command(mul)
if __name__ == "__main__":
calc()
And it will produce nested help pages:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
add Adds numbers
mul Multiplies numbers
$ python myapp.py add --help
Usage: myapp.py add [OPTIONS] [NUMBERS]...
Adds numbers
Options:
-e TEXT Extra option for add
--help Show this message and exit.
$ python myapp.py mul --help
Usage: myapp.py mul [OPTIONS] [NUMBERS]...
Multiplies numbers
Options:
-e TEXT Extra option for mul
--help Show this message and exit.
For more information, see the Documenting Scripts section of the docs.

Can argparse in python 2.7 be told to require a minimum of TWO arguments?

My application is a specialized file comparison utility and obviously it does not make sense to compare only one file, so nargs='+' is not quite appropriate.
nargs=N only excepts a maximum of N arguments, but I need to accept an infinite number of arguments as long as there are at least two of them.

Short answer is you can't do that because nargs doesn't support something like '2+'.
Long answer is you can workaround that using something like this:
parser = argparse.ArgumentParser(usage='%(prog)s [-h] file file [file ...]')
parser.add_argument('file1', nargs=1, metavar='file')
parser.add_argument('file2', nargs='+', metavar='file', help=argparse.SUPPRESS)
namespace = parser.parse_args()
namespace.file = namespace.file1 + namespace.file2
The tricks that you need are:
Use usage to provide you own usage string to the parser
Use metavar to display an argument with a different name in the help string
Use SUPPRESS to avoid displaying help for one of the variables
Merge two different variables just adding a new attribute to the Namespace object that the parser returns
The example above produces the following help string:
usage: test.py [-h] file file [file ...]
positional arguments:
file
optional arguments:
-h, --help show this help message and exit
and will still fail when less than two arguments are passed:
$ python test.py arg
usage: test.py [-h] file file [file ...]
test.py: error: too few arguments

Couldn't you do something like this:
import argparse
parser = argparse.ArgumentParser(description = "Compare files")
parser.add_argument('first', help="the first file")
parser.add_argument('other', nargs='+', help="the other files")
args = parser.parse_args()
print args
When I run this with -h I get:
usage: script.py [-h] first other [other ...]
Compare files
positional arguments:
first the first file
other the other files
optional arguments:
-h, --help show this help message and exit
When I run it with only one argument, it won't work:
usage: script.py [-h] first other [other ...]
script.py: error: too few arguments
But two or more arguments is fine. With three arguments it prints:
Namespace(first='one', other=['two', 'three'])