Develop Reference

How to remove empty metavar square brackets from python argparse help command?

I dont want to show default metavar in the help page nor I need to provide anything, so hack is to add an empty string as a metavar value, but it produces an empty square bracket in the help page, can anyone help how to remove it?
required.add_argument(
'--configure',
type=str,
nargs='?',
metavar='',
const='default',
dest='configure',
default=argparse.SUPPRESS,
help='Configure a profile',
)
Output:
mycommand profile --help
usage: command profile [--help] [options]
optional arguments:
--list Shows list profiles
--configure [] Configure a profile
--delete [] Delete a profile
--show [] Show a profile

The use of [] is integral to using `nargs='?'. They mean "optional/not required".
In [626]: p=argparse.ArgumentParser()
In [627]: a = p.add_argument('--foo',nargs='?')
In [628]: p.print_help()
usage: ipython3 [-h] [--foo [FOO]]
optional arguments:
-h, --help show this help message and exit
--foo [FOO]
To change that you'll have to subclass the HelpFormatter, and change one of the nested methods.

How to use argsparse so the script options with the same affect will appear in the same 'help' line

I'm using argsparse to parse the options passed to my python scripts.
I want to enable passing '-a', and to allow passing '-b' with the same affect.
No problem, I'll call parser.add_argument() twice, with the same description:
parser.add_argument('-a', help='do something')
parser.add_argument('-b', help='do something')
But now when displaying the script help, I will see both, as such:
-a do something
-b do something
This is ugly.
I would prefer to have {-a, -b} or {-a|b).
I could not find in argsparse documentation any way around this (admittedly, not critical) issue.

You could try to pass both arguments to the same add_argument call:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', '-b', help='do something')
parser.parse_args(['--help'])
output
usage: test.py [-h] [-a A]
optional arguments:
-h, --help show this help message and exit
-a A, -b A do something

Stop parsing after subparser

In my CLI script I am using argparse to take in a few optional arguments and then a positional argument. The positional argument is used to determine a subparser to use which in turn runs a function that calls an external program that takes its own arguments. So, the command-line usage looks something like this:
myscript [OPTIONS] subcommand [SUBCOMMAND_OPTIONS]
Now my problem is that there are conflicts between my OPTIONS I've declared and the SUBCOMMAND_OPTIONS declared in the external program. The easy fix is to ensure I rename all conflicts in myscript but I can't do this for all options - most notably the "-h" option for help. Ideally I'd like argparse to stop parsing immediately after it encounters the subcommand and simply pass on the rest of the args to the external program.
So, the following invocation should show the help text for myscript:
myscript -h
While, in contrast the following should show the help text from the external program invoked by the "bar" subparser:
myscript --foo bar -h
Some more code to make the above clearer:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> subparsers = parser.add_subparsers()
>>> subparsers.add_parser("bar")
>>> parser.parse_known_args("--foo bar --test".split())
(Namespace(foo=True), ['--test'])
# cool - this is what I want, I'll just pass --test on to the external program
>>> parser.parse_known_args("--foo bar -h".split())
usage: bar [-h]
optional arguments:
-h, --help show this help message and exit
# unfortunately the above argparse help message is NOT what I wanted, instead I was looking for the result below:
(Namespace(foo=True), ['-h'])
>>> parser.parse_known_args("bar --test -- -h".split())
# this works, sort of, it requires educating the end-user to use the '--' parameter and I'd like to avoid that if possible.
(Namespace(foo=False), ['--test', '--', '-h'])

Your initial description was sufficiently close to subparsers that it takes some careful reading to identify what's wrong (for you).
From comments it sounds like the biggest fault is that the subparser captures the -h give you a help message, rather than passing it through to the extras. Subparsers, just like a main parser, takes a add_help=False parameter.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
sp=p.add_subparsers(dest='cmd')
sp1=sp.add_parser('cmd1') # with a subparser help
sp2=sp.add_parser('cmd2', add_help=False) # will ignore -h
producing
p.parse_known_args('-h'.split()) # top level help
p.parse_known_args('--bar xxx foo cmd1 -h'.split())
# usage: ipython foo cmd1 [-h]
# exit msg
p.parse_known_args('--bar xxx foo cmd2 -h'.split())
# (Namespace(bar='xxx', cmd='cmd2', foo='foo'), ['-h'])
p.parse_known_args('foo cmd2 test -o --more --bar xxx'.split())
# (Namespace(bar=None, cmd='cmd2', foo='foo'),
# ['test', '-o', '--more', '--bar', 'xxx'])
In a comment I mentioned a couple of nargs values, argparse.PARSER and argparse.REMAINDER. To the main parser, subparsers are just a positional with a PARSER nargs (and choices). It's a specialaction` type, which goes on to invoke another parser based on the 1st value.
REMAINDER is like the * nargs, except that it takes everything, even strings that look like flags. PARSER is like +, requiring at least one string.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
p.add_argument('rest', nargs=argparse.REMAINDER)
producing
In [32]: p.parse_args('--bar yyy foo'.split())
Out[32]: Namespace(bar='yyy', foo='foo', rest=[])
In [33]: p.parse_args('--bar yyy foo -h'.split())
Out[33]: Namespace(bar='yyy', foo='foo', rest=['-h'])
In [34]: p.parse_args('--bar yyy foo cmd2 test -o --more --bar xxx'.split())Out[34]: Namespace(bar='yyy', foo='foo', rest=['cmd2', 'test', '-o', '--more', '--bar', 'xxx'])
The REMAINDER note in the argparse docs is:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
and has an example similar to my last one.
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')

The normal sub-commands support in argparse does this just fine. Just note that when reusing argument names, you should specify a custom dest to make sure that your main commands’ values are not overwritten:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', action='store_true')
subparsers = parser.add_subparsers()
parser_foo = subparsers.add_parser('foo')
parser_foo.add_argument('--foo', dest='foo_foo')
parser_foo.add_argument('--bar', dest='foo_bar')
parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('--foo', dest='bar_foo')
Examples:
>>> parser.parse_args('-h'.split())
usage: [-h] [--foo] {foo,bar} ...
positional arguments:
{foo,bar}
optional arguments:
-h, --help show this help message and exit
--foo
>>> parser.parse_args('foo -h'.split())
usage: foo [-h] [--foo FOO_FOO] [--bar FOO_BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO_FOO
--bar FOO_BAR
>>> parser.parse_args('bar -h'.split())
usage: bar [-h] [--foo BAR_FOO]
optional arguments:
-h, --help show this help message and exit
--foo BAR_FOO
>>> parser.parse_args('--foo foo --foo test --bar baz'.split())
Namespace(foo=True, foo_bar='baz', foo_foo='test')

The arpgarse module has a subparser feature, which you can combine with argparse.REMAINDER to capture any arguments without declaring those explicitly.
Update:
If you want to have more control than argparse provides, it might be worthwile to look into the click package instead. It has special support for ignoring unknown options and preventing options like --help to be handled. Details at
http://click.pocoo.org/4/api/#click.Context.ignore_unknown_options

How to show help for all subparsers in argparse?

I have made a Python script that is doing a lot of actions, so it has many options, so I divided it to subparsers that also use parent parsers for common options grouping.
I want a help option that will show the help for all commands with their options, is it possible without overriding the format_help method?
I saw a similar question, but the grouping is not critical for me, I just want the options there.
For example:
general_group = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,add_help=False)
general_group.add_argument('--threads', action='store_true', default=False)
second_group = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,add_help=False)
second_group.add_argument('--sleep', action='store', default=60, type=int)
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
subparsers=parser.add_subparsers(dest='action')
subparsers.add_parser('Restart',parents=[general_group,second_group])
subparsers.add_parser('Start',parents=[general_group])
args = parser.parse_args()
In this case I would like that if someone runs ./script.py -h they'll see the threads option in the help.

The problem is that in the lines:
subparsers=parser.add_subparsers(dest='action')
subparsers.add_parser('Restart',parents=[general_group,second_group])
subparsers.add_parser('Start',parents=[general_group])
You are adding general_group as parent to the subparsers, so the main parser does not know about them, which results in ./script.py -h to not show --threads. If you plan to put it as parent of all the subparsers then you should put it as top parser parent:
parser = argparse.ArgumentParser(parents=[general_group])
subparsers=parser.add_subparsers(dest='action')
subparsers.add_parser('Restart',parents=[second_group])
subparsers.add_parser('Start')
Which results in:
$ python script.py -h
usage: script.py [-h] [--threads] {Restart,Start} ...
positional arguments:
{Restart,Start}
optional arguments:
-h, --help show this help message and exit
--threads
Note however that in this case the option is part only of the parent parser and not the subparsers, which means that the following:
$python script.py --threads Start
is correct, while:
$ python script.py Start --threads
usage: script.py [-h] [--threads] {Restart,Start} ...
script.py: error: unrecognized arguments: --threads
Because --threads is not "inherited" by the subparser. If you want to have --threads also in the subparser you must specify it in its parents argument:
parser = argparse.ArgumentParser(parents=[general_group])
subparsers=parser.add_subparsers(dest='action')
subparsers.add_parser('Restart',parents=[general_group, second_group])
subparsers.add_parser('Start', parents=[general_group])
This should do what you want:
$ python script.py -h
usage: script.py [-h] [--threads] {Restart,Start} ...
positional arguments:
{Restart,Start}
optional arguments:
-h, --help show this help message and exit
--threads
$ python script.py Start -h
usage: script.py Start [-h] [--threads]
optional arguments:
-h, --help show this help message and exit
--threads

Writing a help for python script

I am trying to make my python script very user friendly, so I like to write some sort of help for it. What is your advise for this? I could just put in some logic that if the user passed help as a paramater to the script, they get help. Is there a best practise or convention for this?

Use argparse.
For example, with test.py:
import argparse
parser=argparse.ArgumentParser(
description='''My Description. And what a lovely description it is. ''',
epilog="""All is well that ends well.""")
parser.add_argument('--foo', type=int, default=42, help='FOO!')
parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
args=parser.parse_args()
Running
% test.py -h
yields
usage: test.py [-h] [--foo FOO] [bar [bar ...]]
My Description. And what a lovely description it is.
positional arguments:
bar BAR!
optional arguments:
-h, --help show this help message and exit
--foo FOO FOO!
All is well that ends well.

Best practice is to use argparse to handle all your commandline arguments. It includes a default --help which you can customize to your likings.
Here's the simplest example:
import argparse
parser = argparse.ArgumentParser(description='This is my help')
args = parser.parse_args()
Which results in:
% python argparse_test.py -h
usage: argparse_test.py [-h]
This is my help
optional arguments:
-h, --help show this help message and exit
You can define all your arguments with argparse and set a help message for each one of them. The resulting filtered/validated arguments are returned by parser.parse_args().

An alternative to the built-in argparse is a 3rd-party package called Click which features "automatic help page generation" and "arbitrary nesting of commands" (which also produces nested help pages). Internally, it's based on argparse, but, for me, makes the creation of complex CLI more convenient using decorators.
Here's a sample code:
import click
#click.command()
#click.argument("things", nargs=-1)
#click.option("-t", show_default=True, default="int", help="Data type")
#click.option("-o", help="Output format")
def combine(things, t):
"""Combines things into a single element"""
pass
if __name__ == "__main__":
combine()
And the generated help page:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] [THINGS]...
Combines things into a single element
Options:
-t TEXT Data type [default: int]
-o TEXT Output format
--help Show this message and exit.
One of the nice things about it is that it uses the method docstrings as part of the help page, which is convenient because the docstring can now be used both for developer documentation and for script usage help.
You can also have nested command groups:
import click
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for add")
def add(numbers, e):
"""Adds numbers"""
print(f"This method should add {numbers}")
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for mul")
def mul(numbers, e):
"""Multiplies numbers"""
print(f"This method should multiply {numbers}")
#click.group()
def calc():
pass
calc.add_command(add)
calc.add_command(mul)
if __name__ == "__main__":
calc()
And it will produce nested help pages:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
add Adds numbers
mul Multiplies numbers
$ python myapp.py add --help
Usage: myapp.py add [OPTIONS] [NUMBERS]...
Adds numbers
Options:
-e TEXT Extra option for add
--help Show this message and exit.
$ python myapp.py mul --help
Usage: myapp.py mul [OPTIONS] [NUMBERS]...
Multiplies numbers
Options:
-e TEXT Extra option for mul
--help Show this message and exit.
For more information, see the Documenting Scripts section of the docs.