Develop Reference

Python argparser mutually exclusive with same argument in multiple groups

I use argparser as a generic way to provide inputs on multiple different files, that are used to generate at the end a json, and sent to a database.
Have that said, I want to use multiple mutually exclusive groups, with the option to a flag being part of multiple different groups (as in the example below).
parser = argparser.argumentParser
group1 = parser.add_mutually_exclusive_group()
group2 = parser.add_mutually_exclusive_group(required=True)
group3 = parser.add_mutually_exclusive_group()
group1.add_argument('-a', type=int)
group1.add_argument('-d', type=int)
group2.add_argument('-z', type=int)
group2.add_argument('-x', type=int)
group3.add_argument('-a', type=int)
group3.add_argument('-z', type=int)
it means that -d and -z can go together (but -z or -x are mandatory), giving me the option to have -a -d -x OR -z -d
for some reason, the argparser thinks each one of the -a or -z flags are conflicting, so i have added the conflict_handler to 'resolve', but seems to have no effect

When you add an argument to a group, it is also added to the parser. Groups, both argument_group and mutually_exclusive_group are ways of defining some special actions (in help and testing), but they don't change the fundamental parsing.
So the arguments you try to define via group3 conflict with the arguments already defined via the other groups. I should also note that add_argument creates an argument Action object.
For a bug/issue I came up with a way of adding pre existing Actions to a new group. That is, a way of adding the -a and -z that were created earlier to group3. Actually I wrote it as a way defining a group with a list of existing Actions. That wasn't very hard to do. But displaying such a group required a major rewrite to the usage formatter.
https://bugs.python.org/issue10984
mutually_exclusive_group does 2 things - it modifies the usage - if possible. And it does the 'mutually-exclusive' test. Otherwise it does not modify the parsing. You could perform the same tests after parsing.
In your example, all arguments have a default of None. So after parsing, you could do:
if args.a is not None and args.z is not None:
parse.error('cannot use both -a and -z')
In the bug/issue I modified add_mutually_exclusive_group to effectively do:
group1 = parser.add_mutually_exclusive_group()
group2 = parser.add_mutually_exclusive_group(required=True)
a1 = group1.add_argument('-a', type=int) # hang onto the newly created Action
group1.add_argument('-d', type=int)
a2 = group2.add_argument('-z', type=int)
group2.add_argument('-x', type=int)
group3 = parser.add_mutually_exclusive_group()
group3._group_actions.append(a1) # add existing Action to group
group3._group_actions.append(a2)
#group3.add_argument('-a', type=int)
#group3.add_argument('-z', type=int)
That is, pointers to the existing Actions are added directly to the new group, without going through add_argument.
testing group3:
2347:~/mypy$ python3 stack47670008.py -z 3 -a3
usage: stack47670008.py [-h] [-a A | -d D] (-z Z | -x X)
stack47670008.py: error: argument -a: not allowed with argument -z
2347:~/mypy$ python3 stack47670008.py -z 3 -d3
Namespace(a=None, d=3, x=None, z=3)
2347:~/mypy$ python3 stack47670008.py -h
usage: stack47670008.py [-h] [-a A | -d D] (-z Z | -x X)
optional arguments:
-h, --help show this help message and exit
-a A
-d D
-z Z
-x X
group1 and group2 show up in the usage, but not group3.

I think what you're looking for is the sub_parsers option from argparse.
Kindly check the link from the python docs add_subparsers.

Stop parsing after subparser

In my CLI script I am using argparse to take in a few optional arguments and then a positional argument. The positional argument is used to determine a subparser to use which in turn runs a function that calls an external program that takes its own arguments. So, the command-line usage looks something like this:
myscript [OPTIONS] subcommand [SUBCOMMAND_OPTIONS]
Now my problem is that there are conflicts between my OPTIONS I've declared and the SUBCOMMAND_OPTIONS declared in the external program. The easy fix is to ensure I rename all conflicts in myscript but I can't do this for all options - most notably the "-h" option for help. Ideally I'd like argparse to stop parsing immediately after it encounters the subcommand and simply pass on the rest of the args to the external program.
So, the following invocation should show the help text for myscript:
myscript -h
While, in contrast the following should show the help text from the external program invoked by the "bar" subparser:
myscript --foo bar -h
Some more code to make the above clearer:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> subparsers = parser.add_subparsers()
>>> subparsers.add_parser("bar")
>>> parser.parse_known_args("--foo bar --test".split())
(Namespace(foo=True), ['--test'])
# cool - this is what I want, I'll just pass --test on to the external program
>>> parser.parse_known_args("--foo bar -h".split())
usage: bar [-h]
optional arguments:
-h, --help show this help message and exit
# unfortunately the above argparse help message is NOT what I wanted, instead I was looking for the result below:
(Namespace(foo=True), ['-h'])
>>> parser.parse_known_args("bar --test -- -h".split())
# this works, sort of, it requires educating the end-user to use the '--' parameter and I'd like to avoid that if possible.
(Namespace(foo=False), ['--test', '--', '-h'])

Your initial description was sufficiently close to subparsers that it takes some careful reading to identify what's wrong (for you).
From comments it sounds like the biggest fault is that the subparser captures the -h give you a help message, rather than passing it through to the extras. Subparsers, just like a main parser, takes a add_help=False parameter.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
sp=p.add_subparsers(dest='cmd')
sp1=sp.add_parser('cmd1') # with a subparser help
sp2=sp.add_parser('cmd2', add_help=False) # will ignore -h
producing
p.parse_known_args('-h'.split()) # top level help
p.parse_known_args('--bar xxx foo cmd1 -h'.split())
# usage: ipython foo cmd1 [-h]
# exit msg
p.parse_known_args('--bar xxx foo cmd2 -h'.split())
# (Namespace(bar='xxx', cmd='cmd2', foo='foo'), ['-h'])
p.parse_known_args('foo cmd2 test -o --more --bar xxx'.split())
# (Namespace(bar=None, cmd='cmd2', foo='foo'),
# ['test', '-o', '--more', '--bar', 'xxx'])
In a comment I mentioned a couple of nargs values, argparse.PARSER and argparse.REMAINDER. To the main parser, subparsers are just a positional with a PARSER nargs (and choices). It's a specialaction` type, which goes on to invoke another parser based on the 1st value.
REMAINDER is like the * nargs, except that it takes everything, even strings that look like flags. PARSER is like +, requiring at least one string.
p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
p.add_argument('rest', nargs=argparse.REMAINDER)
producing
In [32]: p.parse_args('--bar yyy foo'.split())
Out[32]: Namespace(bar='yyy', foo='foo', rest=[])
In [33]: p.parse_args('--bar yyy foo -h'.split())
Out[33]: Namespace(bar='yyy', foo='foo', rest=['-h'])
In [34]: p.parse_args('--bar yyy foo cmd2 test -o --more --bar xxx'.split())Out[34]: Namespace(bar='yyy', foo='foo', rest=['cmd2', 'test', '-o', '--more', '--bar', 'xxx'])
The REMAINDER note in the argparse docs is:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
and has an example similar to my last one.
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')

The normal sub-commands support in argparse does this just fine. Just note that when reusing argument names, you should specify a custom dest to make sure that your main commands’ values are not overwritten:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', action='store_true')
subparsers = parser.add_subparsers()
parser_foo = subparsers.add_parser('foo')
parser_foo.add_argument('--foo', dest='foo_foo')
parser_foo.add_argument('--bar', dest='foo_bar')
parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('--foo', dest='bar_foo')
Examples:
>>> parser.parse_args('-h'.split())
usage: [-h] [--foo] {foo,bar} ...
positional arguments:
{foo,bar}
optional arguments:
-h, --help show this help message and exit
--foo
>>> parser.parse_args('foo -h'.split())
usage: foo [-h] [--foo FOO_FOO] [--bar FOO_BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO_FOO
--bar FOO_BAR
>>> parser.parse_args('bar -h'.split())
usage: bar [-h] [--foo BAR_FOO]
optional arguments:
-h, --help show this help message and exit
--foo BAR_FOO
>>> parser.parse_args('--foo foo --foo test --bar baz'.split())
Namespace(foo=True, foo_bar='baz', foo_foo='test')

The arpgarse module has a subparser feature, which you can combine with argparse.REMAINDER to capture any arguments without declaring those explicitly.
Update:
If you want to have more control than argparse provides, it might be worthwile to look into the click package instead. It has special support for ignoring unknown options and preventing options like --help to be handled. Details at
http://click.pocoo.org/4/api/#click.Context.ignore_unknown_options

Parsing exclusive groups in Python

I have 2 group which are exclusive, you can define either arguments from group1 or group2 but group2 have to be exclusive within it's arguments too.
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
sub_exclusive_1.add_argument("-a")
sub_exclusive_1.add_argument("-b")
sub_exclusive_1.add_argument("-c")
sub_exclusive_1.add_argument("-d")
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
args = parser.parse_args()
The code have to terminate if [-a and -AA or -BB] or [-AA and -BB] have been defined but still have to work with [-a and/or -b],
The problem is that it's not terminating...
I found this thread and edited my code to
subparsers = parser.add_subparsers()
parser_a = subparsers.add_parser('command_1')
parser_a.add_argument("-a")
parser_a.add_argument("-b")
parser_a.add_argument("-c")
parser_a.add_argument("-d")
parser_b = subparsers.add_parser('command_2')
parser_b.add_argument("-AA")
parser_b.add_argument("-BB")
still does not work, traceback: main.py: error: too few arguments
What do i do wrong?
current workaround:
parser = argparse.ArgumentParser()
parser.add_argument("-a")
...
parser.add_argument("-AA")
args = parser.parse_args()
if (args.a or args.b or args.c or args.d) and (args.AA or args.BB) or (args.AA and args.BB):
raise SystemExit()

At the risk of repeating my answer from the earlier question, let's focus on your case
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
...
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
Despite similar names (and class nesting), the functionality of argument_groups and mutually_exclusive_groups is quite different. And the former does not nest meaningfully within the second.
An argument group is a tool to organize arguments in the help. It does not enter arguments 'as a group' into another group, and has NO effect on parsing or error checking.
If it did act as you want, what would the usage line look like?
With the subparser formulation the parser responds with:
prog command1 -a -b -c # ok
prog command1 -a -AA # error - not recognize -AA
prog command2 -AA -BB # ok
prog command2 -a -AA # error - -a not recognized
prog -AA # error - too few arg
The subparser mechanism is similar to
parser.add_argument('cmd', choices=['command1','command2']
The 'command1' string tells it - parser the reset of the strings using the '-a -b ...' group of arguments. It has to know which group you expect it to use.
Short of using the bug/issue patch that I worked on a while back, you need to do your own 'mutually-exclusive' testing after parsing. As long as you use the default default None, it is is easy to test whether an argument has been used or now (args.AA is not None).
https://stackoverflow.com/a/30337890/901925 is a recent example of doing post-parsing testing.

Python argparse mutual exclusive group

What I need is:
pro [-a xxx | [-b yyy -c zzz]]
I tried this but does not work. Could someone help me out?
group= parser.add_argument_group('Model 2')
group_ex = group.add_mutually_exclusive_group()
group_ex.add_argument("-a", type=str, action = "store", default = "", help="test")
group_ex_2 = group_ex.add_argument_group("option 2")
group_ex_2.add_argument("-b", type=str, action = "store", default = "", help="test")
group_ex_2.add_argument("-c", type=str, action = "store", default = "", help="test")
Thanks!

add_mutually_exclusive_group doesn't make an entire group mutually exclusive. It makes options within the group mutually exclusive.
What you're looking for is subcommands. Instead of prog [ -a xxxx | [-b yyy -c zzz]], you'd have:
prog
command 1
-a: ...
command 2
-b: ...
-c: ...
To invoke with the first set of arguments:
prog command_1 -a xxxx
To invoke with the second set of arguments:
prog command_2 -b yyyy -c zzzz
You can also set the sub command arguments as positional.
prog command_1 xxxx
Kind of like git or svn:
git commit -am
git merge develop
Working Example
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='help for foo arg.')
subparsers = parser.add_subparsers(help='help for subcommand', dest="subcommand")
# create the parser for the "command_1" command
parser_a = subparsers.add_parser('command_1', help='command_1 help')
parser_a.add_argument('a', type=str, help='help for bar, positional')
# create the parser for the "command_2" command
parser_b = subparsers.add_parser('command_2', help='help for command_2')
parser_b.add_argument('-b', type=str, help='help for b')
parser_b.add_argument('-c', type=str, action='store', default='', help='test')
Test it
>>> parser.print_help()
usage: PROG [-h] [--foo] {command_1,command_2} ...
positional arguments:
{command_1,command_2}
help for subcommand
command_1 command_1 help
command_2 help for command_2
optional arguments:
-h, --help show this help message and exit
--foo help for foo arg.
>>>
>>> parser.parse_args(['command_1', 'working'])
Namespace(subcommand='command_1', a='working', foo=False)
>>> parser.parse_args(['command_1', 'wellness', '-b x'])
usage: PROG [-h] [--foo] {command_1,command_2} ...
PROG: error: unrecognized arguments: -b x
Good luck.

While Jonathan's answer is perfectly fine for complex options, there is a very simple solution which will work for the simple cases, e.g. 1 option excludes 2 other options like in
command [- a xxx | [ -b yyy | -c zzz ]]
or even as in the original question:
pro [-a xxx | [-b yyy -c zzz]]
Here is how I would do it:
parser = argparse.ArgumentParser()
# group 1
parser.add_argument("-q", "--query", help="query")
parser.add_argument("-f", "--fields", help="field names")
# group 2
parser.add_argument("-a", "--aggregation", help="aggregation")
I am using here options given to a command line wrapper for querying a mongodb. The collection instance can either call the method aggregate or the method find with to optional arguments query and fields, hence you see why the first two arguments are compatible and the last one isn't.
So now I run parser.parse_args() and check it's content:
args = parser.parse_args()
if args.aggregation and (args.query or args.fields):
print "-a and -q|-f are mutually exclusive ..."
sys.exit(2)
Of course, this little hack is only working for simple cases and it would become a nightmare to check all the possible options if you have many mutually exclusive options and groups. In that case you should break your options in to command groups like Jonathan suggested.

There is a python patch (in development) that would allow you to do this.
http://bugs.python.org/issue10984
The idea is to allow overlapping mutually exclusive groups. So usage might look like:
pro [-a xxx | -b yyy] [-a xxx | -c zzz]
Changing the argparse code so you can create two groups like this was the easy part. Changing the usage formatting code required writing a custom HelpFormatter.
In argparse, action groups don't affect the parsing. They are just a help formatting tool. In the help, mutually exclusive groups only affect the usage line. When parsing, the parser uses the mutually exclusive groups to construct a dictionary of potential conflicts (a can't occur with b or c, b can't occur with a, etc), and then raises an error if a conflict arises.
Without that argparse patch, I think your best choice is to test the namespace produced by parse_args yourself (e.g. if both a and b have nondefault values), and raise your own error. You could even use the parser's own error mechanism.
parser.error('custom error message')

If you don't want subparsers, this can currently be done with mutually exclusive groups, but fair warning, it involves accessing private variables so use it at your own risk. The idea is you want -a to be mutually exclusive with -b and -c, but -b and -c don't want to be mutually exclusive with each other
import argparse
p = argparse.ArgumentParser()
# first set up a mutually exclusive group for a and b
g1 = p.add_mutually_exclusive_group()
arg_a = g1.add_argument('-a') # save this _StoreAction for later
g1.add_argument('-b')
# now set up a second group for a and c
g2 = p.add_mutually_exclusive_group()
g2.add_argument('-c')
g2._group_actions.append(arg_a) # this is the magic/hack
Now we've got -a exclusive to both -c and -b.
a = p.parse_args(['-a', '1'])
# a.a = 1, a.b = None, a.c = None
a = p.parse_args(['-a', '1', '-b', '2'])
# usage: prog.py [-h] [-a A | -b B] [-c C]
# prog.py: error: argument -b: not allowed with argument -a
Note, it does mess up the help message, but you could probably override that, or just ignore it because you've got the functionality you want, which is probably more important anyway.
If you want to ensure if we're using any of b and c, we have to use both of them, then simply add the required=True keyword arg when instantiating the mutually exclusive groups.

argparse, two arguments depend on each other

I would like to make the parser like cmd [-a xxx -b xxx] -c xxx -d xxx
When -a is used, I want -b to be used too. likewise, if -b is used, -a must be used too. It's ok both -a and -b are not used.
How do I do that? I have tried custom actions, but it does not go well.

A better design would be to have a single option that takes two arguments:
parser.add_argument('-a', nargs=2)
Then you either specify the option with 2 arguments, or you don't specify it at all.
$ script -a 1 2
or
$ script
A custom action (or postprocessing) can split the tuple args.a into two separate values args.a and args.b.

Argparse doesn't natively support this type of use.
The most effective thing to do is check and see if those types of conditions are met after parsing:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parser.add_argument('-c')
args = parser.parse_args()
required_together = ('b','c')
# args.b will be None if b is not provided
if not all([getattr(args,x) for x in required_together]):
raise RuntimeError("Cannot supply -c without -b")