In my CLI script I am using argparse to take in a few optional arguments and then a positional argument. The positional argument is used to determine a subparser to use which in turn runs a function that calls an external program that takes its own arguments. So, the command-line usage looks something like this:
myscript [OPTIONS] subcommand [SUBCOMMAND_OPTIONS]
Now my problem is that there are conflicts between my OPTIONS I've declared and the SUBCOMMAND_OPTIONS declared in the external program. The easy fix is to ensure I rename all conflicts in myscript but I can't do this for all options - most notably the "-h" option for help. Ideally I'd like argparse to stop parsing immediately after it encounters the subcommand and simply pass on the rest of the args to the external program.
So, the following invocation should show the help text for myscript:
myscript -h
While, in contrast the following should show the help text from the external program invoked by the "bar" subparser:
myscript --foo bar -h
Some more code to make the above clearer:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> subparsers = parser.add_subparsers()
>>> subparsers.add_parser("bar")
>>> parser.parse_known_args("--foo bar --test".split())
(Namespace(foo=True), ['--test'])
# cool - this is what I want, I'll just pass --test on to the external program
>>> parser.parse_known_args("--foo bar -h".split())
usage: bar [-h]
optional arguments:
-h, --help show this help message and exit
# unfortunately the above argparse help message is NOT what I wanted, instead I was looking for the result below:
(Namespace(foo=True), ['-h'])
>>> parser.parse_known_args("bar --test -- -h".split())
# this works, sort of, it requires educating the end-user to use the '--' parameter and I'd like to avoid that if possible.
(Namespace(foo=False), ['--test', '--', '-h'])
Your initial description was sufficiently close to subparsers that it takes some careful reading to identify what's wrong (for you).
From comments it sounds like the biggest fault is that the subparser captures the -h give you a help message, rather than passing it through to the extras. Subparsers, just like a main parser, takes a add_help=False parameter.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
sp=p.add_subparsers(dest='cmd')
sp1=sp.add_parser('cmd1') # with a subparser help
sp2=sp.add_parser('cmd2', add_help=False) # will ignore -h
producing
p.parse_known_args('-h'.split()) # top level help
p.parse_known_args('--bar xxx foo cmd1 -h'.split())
# usage: ipython foo cmd1 [-h]
# exit msg
p.parse_known_args('--bar xxx foo cmd2 -h'.split())
# (Namespace(bar='xxx', cmd='cmd2', foo='foo'), ['-h'])
p.parse_known_args('foo cmd2 test -o --more --bar xxx'.split())
# (Namespace(bar=None, cmd='cmd2', foo='foo'),
# ['test', '-o', '--more', '--bar', 'xxx'])
In a comment I mentioned a couple of nargs values, argparse.PARSER and argparse.REMAINDER. To the main parser, subparsers are just a positional with a PARSER nargs (and choices). It's a specialaction` type, which goes on to invoke another parser based on the 1st value.
REMAINDER is like the * nargs, except that it takes everything, even strings that look like flags. PARSER is like +, requiring at least one string.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
p.add_argument('rest', nargs=argparse.REMAINDER)
producing
In [32]: p.parse_args('--bar yyy foo'.split())
Out[32]: Namespace(bar='yyy', foo='foo', rest=[])
In [33]: p.parse_args('--bar yyy foo -h'.split())
Out[33]: Namespace(bar='yyy', foo='foo', rest=['-h'])
In [34]: p.parse_args('--bar yyy foo cmd2 test -o --more --bar xxx'.split())Out[34]: Namespace(bar='yyy', foo='foo', rest=['cmd2', 'test', '-o', '--more', '--bar', 'xxx'])
The REMAINDER note in the argparse docs is:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
and has an example similar to my last one.
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
The normal sub-commands support in argparse does this just fine. Just note that when reusing argument names, you should specify a custom dest to make sure that your main commands’ values are not overwritten:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', action='store_true')
subparsers = parser.add_subparsers()
parser_foo = subparsers.add_parser('foo')
parser_foo.add_argument('--foo', dest='foo_foo')
parser_foo.add_argument('--bar', dest='foo_bar')
parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('--foo', dest='bar_foo')
Examples:
>>> parser.parse_args('-h'.split())
usage: [-h] [--foo] {foo,bar} ...
positional arguments:
{foo,bar}
optional arguments:
-h, --help show this help message and exit
--foo
>>> parser.parse_args('foo -h'.split())
usage: foo [-h] [--foo FOO_FOO] [--bar FOO_BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO_FOO
--bar FOO_BAR
>>> parser.parse_args('bar -h'.split())
usage: bar [-h] [--foo BAR_FOO]
optional arguments:
-h, --help show this help message and exit
--foo BAR_FOO
>>> parser.parse_args('--foo foo --foo test --bar baz'.split())
Namespace(foo=True, foo_bar='baz', foo_foo='test')
The arpgarse module has a subparser feature, which you can combine with argparse.REMAINDER to capture any arguments without declaring those explicitly.
Update:
If you want to have more control than argparse provides, it might be worthwile to look into the click package instead. It has special support for ignoring unknown options and preventing options like --help to be handled. Details at
http://click.pocoo.org/4/api/#click.Context.ignore_unknown_options
Related
What I need is:
pro [-a xxx | [-b yyy -c zzz]]
I tried this but does not work. Could someone help me out?
group= parser.add_argument_group('Model 2')
group_ex = group.add_mutually_exclusive_group()
group_ex.add_argument("-a", type=str, action = "store", default = "", help="test")
group_ex_2 = group_ex.add_argument_group("option 2")
group_ex_2.add_argument("-b", type=str, action = "store", default = "", help="test")
group_ex_2.add_argument("-c", type=str, action = "store", default = "", help="test")
Thanks!
add_mutually_exclusive_group doesn't make an entire group mutually exclusive. It makes options within the group mutually exclusive.
What you're looking for is subcommands. Instead of prog [ -a xxxx | [-b yyy -c zzz]], you'd have:
prog
command 1
-a: ...
command 2
-b: ...
-c: ...
To invoke with the first set of arguments:
prog command_1 -a xxxx
To invoke with the second set of arguments:
prog command_2 -b yyyy -c zzzz
You can also set the sub command arguments as positional.
prog command_1 xxxx
Kind of like git or svn:
git commit -am
git merge develop
Working Example
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='help for foo arg.')
subparsers = parser.add_subparsers(help='help for subcommand', dest="subcommand")
# create the parser for the "command_1" command
parser_a = subparsers.add_parser('command_1', help='command_1 help')
parser_a.add_argument('a', type=str, help='help for bar, positional')
# create the parser for the "command_2" command
parser_b = subparsers.add_parser('command_2', help='help for command_2')
parser_b.add_argument('-b', type=str, help='help for b')
parser_b.add_argument('-c', type=str, action='store', default='', help='test')
Test it
>>> parser.print_help()
usage: PROG [-h] [--foo] {command_1,command_2} ...
positional arguments:
{command_1,command_2}
help for subcommand
command_1 command_1 help
command_2 help for command_2
optional arguments:
-h, --help show this help message and exit
--foo help for foo arg.
>>>
>>> parser.parse_args(['command_1', 'working'])
Namespace(subcommand='command_1', a='working', foo=False)
>>> parser.parse_args(['command_1', 'wellness', '-b x'])
usage: PROG [-h] [--foo] {command_1,command_2} ...
PROG: error: unrecognized arguments: -b x
Good luck.
While Jonathan's answer is perfectly fine for complex options, there is a very simple solution which will work for the simple cases, e.g. 1 option excludes 2 other options like in
command [- a xxx | [ -b yyy | -c zzz ]]
or even as in the original question:
pro [-a xxx | [-b yyy -c zzz]]
Here is how I would do it:
parser = argparse.ArgumentParser()
# group 1
parser.add_argument("-q", "--query", help="query")
parser.add_argument("-f", "--fields", help="field names")
# group 2
parser.add_argument("-a", "--aggregation", help="aggregation")
I am using here options given to a command line wrapper for querying a mongodb. The collection instance can either call the method aggregate or the method find with to optional arguments query and fields, hence you see why the first two arguments are compatible and the last one isn't.
So now I run parser.parse_args() and check it's content:
args = parser.parse_args()
if args.aggregation and (args.query or args.fields):
print "-a and -q|-f are mutually exclusive ..."
sys.exit(2)
Of course, this little hack is only working for simple cases and it would become a nightmare to check all the possible options if you have many mutually exclusive options and groups. In that case you should break your options in to command groups like Jonathan suggested.
There is a python patch (in development) that would allow you to do this.
http://bugs.python.org/issue10984
The idea is to allow overlapping mutually exclusive groups. So usage might look like:
pro [-a xxx | -b yyy] [-a xxx | -c zzz]
Changing the argparse code so you can create two groups like this was the easy part. Changing the usage formatting code required writing a custom HelpFormatter.
In argparse, action groups don't affect the parsing. They are just a help formatting tool. In the help, mutually exclusive groups only affect the usage line. When parsing, the parser uses the mutually exclusive groups to construct a dictionary of potential conflicts (a can't occur with b or c, b can't occur with a, etc), and then raises an error if a conflict arises.
Without that argparse patch, I think your best choice is to test the namespace produced by parse_args yourself (e.g. if both a and b have nondefault values), and raise your own error. You could even use the parser's own error mechanism.
parser.error('custom error message')
If you don't want subparsers, this can currently be done with mutually exclusive groups, but fair warning, it involves accessing private variables so use it at your own risk. The idea is you want -a to be mutually exclusive with -b and -c, but -b and -c don't want to be mutually exclusive with each other
import argparse
p = argparse.ArgumentParser()
# first set up a mutually exclusive group for a and b
g1 = p.add_mutually_exclusive_group()
arg_a = g1.add_argument('-a') # save this _StoreAction for later
g1.add_argument('-b')
# now set up a second group for a and c
g2 = p.add_mutually_exclusive_group()
g2.add_argument('-c')
g2._group_actions.append(arg_a) # this is the magic/hack
Now we've got -a exclusive to both -c and -b.
a = p.parse_args(['-a', '1'])
# a.a = 1, a.b = None, a.c = None
a = p.parse_args(['-a', '1', '-b', '2'])
# usage: prog.py [-h] [-a A | -b B] [-c C]
# prog.py: error: argument -b: not allowed with argument -a
Note, it does mess up the help message, but you could probably override that, or just ignore it because you've got the functionality you want, which is probably more important anyway.
If you want to ensure if we're using any of b and c, we have to use both of them, then simply add the required=True keyword arg when instantiating the mutually exclusive groups.
Is there a way to include '-help' command to argparse help list?
I wish to have something like this on output, if i am typing '-help'.
optional arguments:
-h, -help, --help show this help message and exit
Thanks
As #Akaisteph7 suggested:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-help', action="help", help="second help :)")
parser.add_argument('-f', '--foo')
parser.print_help()
0945:~/mypy$ python3 stack57058526.py
usage: stack57058526.py [-h] [-help] [-f FOO]
optional arguments:
-h, --help show this help message and exit
-help second help :)
-f FOO, --foo FOO
Changing to:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('-h','--help','-help', action="help", help="replacement help")
0946:~/mypy$ python3 stack57058526.py
usage: stack57058526.py [-h] [-f FOO]
optional arguments:
-h, --help, -help replacement help
-f FOO, --foo FOO
Adding the '-help' flag to the default help requires modifying a couple of 'private' attributes:
parser = argparse.ArgumentParser()
parser._actions[0].option_strings += ['-help']
parser._option_string_actions['-help'] = parser._option_string_actions['-h']
0947:~/mypy$ python3 stack57058526.py
usage: stack57058526.py [-h] [-f FOO]
optional arguments:
-h, --help, -help show this help message and exit
-f FOO, --foo FOO
If you want to build this change into your local version of argparse, you could modify this block of code in the ArgumentParser.__init__ method:
if self.add_help:
self.add_argument(
default_prefix+'h', default_prefix*2+'help',
action='help', default=SUPPRESS,
help=_('show this help message and exit'))
Whether you change a local copy of argparse.py, or subclass ArgumentParser is up to you.
While this is possible to do, it is not recommended. Single dashes are only meant to be used with single letters. In general, you should follow recommendations as they are there for a reason.
If you really want to add it however, you can do it with:
parser.add_argument("-help", action="help")
argparse can work with any prefix characters.
For instance to support POSIX, Cmd.exe and PowerShell-type queries you could use:
p = ArgumentParser(prefix_chars="-/", add_help=False)
p.add_argument("-help", "--help", "-h", "/?", action="help")
Remember to use add_help=False or you'll have two help commands!
I wrote the following command-line parses by using argparse that make use of sub-commands.
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
sub_parser = parser.add_subparsers(title='sub commands',
help='sub commands help')
foo = sub_parser.add_parser('foo')
foo.add_argument(
'--a',
action='store',
default='1234',
help='A'
)
parser.parse_args(['foo', '--help'])
When I print the usage help for sub-command foo, I would expect that the default value of the argument --a is shown. But that doesn't happen.
Here the current output:
usage: test_args.py foo [-h] [--a A]
optional arguments:
-h, --help show this help message and exit
--a A A
Process finished with exit code 0
By calling sub-commands foo without argument a, the default value is used. So, why isn't also the default value shown inside the usage output? Is that a bug?
Or do you know how to achieve that?
As suggested by #hpauly, I found a workaround by using (default: %(default)s).
foo = sub_parser.add_parser('foo')
foo.add_argument(
'--a',
action='store',
default='1234',
help='A. (default: %(default)s)'
)
add_parser takes the same arguments as ArgumentParser, so you can just do formatter_class=argparse.ArgumentDefaultsHelpFormatter to set the default argument formatter for the subparser.
I'm using argsparse to parse the options passed to my python scripts.
I want to enable passing '-a', and to allow passing '-b' with the same affect.
No problem, I'll call parser.add_argument() twice, with the same description:
parser.add_argument('-a', help='do something')
parser.add_argument('-b', help='do something')
But now when displaying the script help, I will see both, as such:
-a do something
-b do something
This is ugly.
I would prefer to have {-a, -b} or {-a|b).
I could not find in argsparse documentation any way around this (admittedly, not critical) issue.
You could try to pass both arguments to the same add_argument call:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', '-b', help='do something')
parser.parse_args(['--help'])
output
usage: test.py [-h] [-a A]
optional arguments:
-h, --help show this help message and exit
-a A, -b A do something
What I need is:
pro [-a xxx | [-b yyy -c zzz]]
I tried this but does not work. Could someone help me out?
group= parser.add_argument_group('Model 2')
group_ex = group.add_mutually_exclusive_group()
group_ex.add_argument("-a", type=str, action = "store", default = "", help="test")
group_ex_2 = group_ex.add_argument_group("option 2")
group_ex_2.add_argument("-b", type=str, action = "store", default = "", help="test")
group_ex_2.add_argument("-c", type=str, action = "store", default = "", help="test")
Thanks!
add_mutually_exclusive_group doesn't make an entire group mutually exclusive. It makes options within the group mutually exclusive.
What you're looking for is subcommands. Instead of prog [ -a xxxx | [-b yyy -c zzz]], you'd have:
prog
command 1
-a: ...
command 2
-b: ...
-c: ...
To invoke with the first set of arguments:
prog command_1 -a xxxx
To invoke with the second set of arguments:
prog command_2 -b yyyy -c zzzz
You can also set the sub command arguments as positional.
prog command_1 xxxx
Kind of like git or svn:
git commit -am
git merge develop
Working Example
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='help for foo arg.')
subparsers = parser.add_subparsers(help='help for subcommand', dest="subcommand")
# create the parser for the "command_1" command
parser_a = subparsers.add_parser('command_1', help='command_1 help')
parser_a.add_argument('a', type=str, help='help for bar, positional')
# create the parser for the "command_2" command
parser_b = subparsers.add_parser('command_2', help='help for command_2')
parser_b.add_argument('-b', type=str, help='help for b')
parser_b.add_argument('-c', type=str, action='store', default='', help='test')
Test it
>>> parser.print_help()
usage: PROG [-h] [--foo] {command_1,command_2} ...
positional arguments:
{command_1,command_2}
help for subcommand
command_1 command_1 help
command_2 help for command_2
optional arguments:
-h, --help show this help message and exit
--foo help for foo arg.
>>>
>>> parser.parse_args(['command_1', 'working'])
Namespace(subcommand='command_1', a='working', foo=False)
>>> parser.parse_args(['command_1', 'wellness', '-b x'])
usage: PROG [-h] [--foo] {command_1,command_2} ...
PROG: error: unrecognized arguments: -b x
Good luck.
While Jonathan's answer is perfectly fine for complex options, there is a very simple solution which will work for the simple cases, e.g. 1 option excludes 2 other options like in
command [- a xxx | [ -b yyy | -c zzz ]]
or even as in the original question:
pro [-a xxx | [-b yyy -c zzz]]
Here is how I would do it:
parser = argparse.ArgumentParser()
# group 1
parser.add_argument("-q", "--query", help="query")
parser.add_argument("-f", "--fields", help="field names")
# group 2
parser.add_argument("-a", "--aggregation", help="aggregation")
I am using here options given to a command line wrapper for querying a mongodb. The collection instance can either call the method aggregate or the method find with to optional arguments query and fields, hence you see why the first two arguments are compatible and the last one isn't.
So now I run parser.parse_args() and check it's content:
args = parser.parse_args()
if args.aggregation and (args.query or args.fields):
print "-a and -q|-f are mutually exclusive ..."
sys.exit(2)
Of course, this little hack is only working for simple cases and it would become a nightmare to check all the possible options if you have many mutually exclusive options and groups. In that case you should break your options in to command groups like Jonathan suggested.
There is a python patch (in development) that would allow you to do this.
http://bugs.python.org/issue10984
The idea is to allow overlapping mutually exclusive groups. So usage might look like:
pro [-a xxx | -b yyy] [-a xxx | -c zzz]
Changing the argparse code so you can create two groups like this was the easy part. Changing the usage formatting code required writing a custom HelpFormatter.
In argparse, action groups don't affect the parsing. They are just a help formatting tool. In the help, mutually exclusive groups only affect the usage line. When parsing, the parser uses the mutually exclusive groups to construct a dictionary of potential conflicts (a can't occur with b or c, b can't occur with a, etc), and then raises an error if a conflict arises.
Without that argparse patch, I think your best choice is to test the namespace produced by parse_args yourself (e.g. if both a and b have nondefault values), and raise your own error. You could even use the parser's own error mechanism.
parser.error('custom error message')
If you don't want subparsers, this can currently be done with mutually exclusive groups, but fair warning, it involves accessing private variables so use it at your own risk. The idea is you want -a to be mutually exclusive with -b and -c, but -b and -c don't want to be mutually exclusive with each other
import argparse
p = argparse.ArgumentParser()
# first set up a mutually exclusive group for a and b
g1 = p.add_mutually_exclusive_group()
arg_a = g1.add_argument('-a') # save this _StoreAction for later
g1.add_argument('-b')
# now set up a second group for a and c
g2 = p.add_mutually_exclusive_group()
g2.add_argument('-c')
g2._group_actions.append(arg_a) # this is the magic/hack
Now we've got -a exclusive to both -c and -b.
a = p.parse_args(['-a', '1'])
# a.a = 1, a.b = None, a.c = None
a = p.parse_args(['-a', '1', '-b', '2'])
# usage: prog.py [-h] [-a A | -b B] [-c C]
# prog.py: error: argument -b: not allowed with argument -a
Note, it does mess up the help message, but you could probably override that, or just ignore it because you've got the functionality you want, which is probably more important anyway.
If you want to ensure if we're using any of b and c, we have to use both of them, then simply add the required=True keyword arg when instantiating the mutually exclusive groups.