Develop Reference

Python - argparse - (Singleton) Argument w/ optional parameter [duplicate]

I'm trying to implement the following argument dependency using the argparse module:
./prog [-h | [-v schema] file]
meaning the user must pass either -h or a file, if a file is passed the user can optionally pass -v schema.
That's what I have now but that doesn't seem to be working:
import argparse
parser = argparse.ArgumentParser()
mtx = parser.add_mutually_exclusive_group()
mtx.add_argument('-h', ...)
grp = mtx.add_argument_group()
grp.add_argument('-v', ...)
grp.add_argument('file', ...)
args = parser.parse_args()
It looks like you can't add an arg group to a mutex group or am I missing something?

If -h means the default help, then this is all you need (this help is already exclusive)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('file')
parser.add_argument('-s','--schema')
parser.parse_args('-h'.split()) # parser.print_help()
producing
usage: stack23951543.py [-h] [-s SCHEMA] file
...
If by -h you mean some other action, lets rename it -x. This would come close to what you describe
parser = argparse.ArgumentParser()
parser.add_argument('-s','--schema', default='meaningful default value')
mxg = parser.add_mutually_exclusive_group(required=True)
mxg.add_argument('-x','--xxx', action='store_true')
mxg.add_argument('file', nargs='?')
parser.parse_args('-h'.split())
usage is:
usage: stack23951543.py [-h] [-s SCHEMA] (-x | file)
Now -x or file is required (but not both). -s is optional in either case, but with a meaningful default, it doesn't matter if it is omitted. And if -x is given, you can just ignore the -s value.
If necessary you could test args after parsing, to confirm that if args.file is not None, then args.schema can't be either.
Earlier I wrote (maybe over thinking the question):
An argument_group cannot be added to a mutually_exclusive_group. The two kinds of groups have different purposes and functions. There are previous SO discussions of this (see 'related'), as well as a couple of relevant Python bug issues. If you want tests that go beyond a simple mutually exclusive group, you probably should do your own testing after parse_args. That may also require your own usage line.
An argument_group is just a means of grouping and labeling arguments in the help section.
A mutually_exclusive_group affects the usage formatting (if it can), and also runs tests during parse_args. The use of 'group' for both implies that they are more connected than they really are.
http://bugs.python.org/issue11588 asks for nested groups, and the ability to test for 'inclusivity' as well. I tried to make the case that 'groups' aren't general enough to express all the kinds of testing that users want. But it's one thing to generalize the testing mechanism, and quite another to come up with an intuitive API. Questions like this suggest that argparse does need some sort of 'nested group' syntax.

Python argparser mutually exclusive with same argument in multiple groups

I use argparser as a generic way to provide inputs on multiple different files, that are used to generate at the end a json, and sent to a database.
Have that said, I want to use multiple mutually exclusive groups, with the option to a flag being part of multiple different groups (as in the example below).
parser = argparser.argumentParser
group1 = parser.add_mutually_exclusive_group()
group2 = parser.add_mutually_exclusive_group(required=True)
group3 = parser.add_mutually_exclusive_group()
group1.add_argument('-a', type=int)
group1.add_argument('-d', type=int)
group2.add_argument('-z', type=int)
group2.add_argument('-x', type=int)
group3.add_argument('-a', type=int)
group3.add_argument('-z', type=int)
it means that -d and -z can go together (but -z or -x are mandatory), giving me the option to have -a -d -x OR -z -d
for some reason, the argparser thinks each one of the -a or -z flags are conflicting, so i have added the conflict_handler to 'resolve', but seems to have no effect

When you add an argument to a group, it is also added to the parser. Groups, both argument_group and mutually_exclusive_group are ways of defining some special actions (in help and testing), but they don't change the fundamental parsing.
So the arguments you try to define via group3 conflict with the arguments already defined via the other groups. I should also note that add_argument creates an argument Action object.
For a bug/issue I came up with a way of adding pre existing Actions to a new group. That is, a way of adding the -a and -z that were created earlier to group3. Actually I wrote it as a way defining a group with a list of existing Actions. That wasn't very hard to do. But displaying such a group required a major rewrite to the usage formatter.
https://bugs.python.org/issue10984
mutually_exclusive_group does 2 things - it modifies the usage - if possible. And it does the 'mutually-exclusive' test. Otherwise it does not modify the parsing. You could perform the same tests after parsing.
In your example, all arguments have a default of None. So after parsing, you could do:
if args.a is not None and args.z is not None:
parse.error('cannot use both -a and -z')
In the bug/issue I modified add_mutually_exclusive_group to effectively do:
group1 = parser.add_mutually_exclusive_group()
group2 = parser.add_mutually_exclusive_group(required=True)
a1 = group1.add_argument('-a', type=int) # hang onto the newly created Action
group1.add_argument('-d', type=int)
a2 = group2.add_argument('-z', type=int)
group2.add_argument('-x', type=int)
group3 = parser.add_mutually_exclusive_group()
group3._group_actions.append(a1) # add existing Action to group
group3._group_actions.append(a2)
#group3.add_argument('-a', type=int)
#group3.add_argument('-z', type=int)
That is, pointers to the existing Actions are added directly to the new group, without going through add_argument.
testing group3:
2347:~/mypy$ python3 stack47670008.py -z 3 -a3
usage: stack47670008.py [-h] [-a A | -d D] (-z Z | -x X)
stack47670008.py: error: argument -a: not allowed with argument -z
2347:~/mypy$ python3 stack47670008.py -z 3 -d3
Namespace(a=None, d=3, x=None, z=3)
2347:~/mypy$ python3 stack47670008.py -h
usage: stack47670008.py [-h] [-a A | -d D] (-z Z | -x X)
optional arguments:
-h, --help show this help message and exit
-a A
-d D
-z Z
-x X
group1 and group2 show up in the usage, but not group3.

I think what you're looking for is the sub_parsers option from argparse.
Kindly check the link from the python docs add_subparsers.

Customize help among groups in argparse

I'm using argparse and I have various groups which have set of its own options.
Now with the --help option I do not want to show all the options by default. Only a set of groups options are to be shown for --help.
Other group options should be shown based on other help options, as --help_1, --help_2:
For example:
--help' to show Group 2 and 3
--help_1' to show Group 11 and 12
--help_2' to show Group 22 and 23
I know that we can disable the default --help option with using add_help=False but how do I get to display only selected group specific helps.
We can get the list of groups from the parser using _action_groups attribute, but they do not expose any print_help() option as such.
My sample code:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('--help_a', action='store_true')
parser.add_argument('--help_b', action='store_true')
group1 = parser.add_argument_group("Feature 1")
group1.add_argument('--foo1')
group2 = parser.add_argument_group("Feature 2")
group2.add_argument('--foo2')
group3 = parser.add_argument_group("Feature 3")
group3.add_argument('--foo3')
# TODO: --help_a to only print "Feature 1" groups help
# and --help_b to print Feature 2 and 3's help.
EDIT: Using subparser and adding parsers(instead of group) will solve the above. But subparser doesn't fit in my case, as I am parsing it always, I only need to customize help to be displayed.

Here's the custom format_help approach:
import argparse
def format_help(self, groups=None):
# self == parser
formatter = self._get_formatter()
# usage
formatter.add_usage(self.usage, self._actions,
self._mutually_exclusive_groups)
# description
formatter.add_text(self.description)
if groups is None:
groups = self._action_groups
# positionals, optionals and user-defined groups
for action_group in groups:
formatter.start_section(action_group.title)
formatter.add_text(action_group.description)
formatter.add_arguments(action_group._group_actions)
formatter.end_section()
# epilog
formatter.add_text(self.epilog)
# determine help from format above
return formatter.format_help()
<your parser>
args = parser.parse_args()
# _action_groups[:2] are the default ones
if args.help_a:
print(format_help(parser, [parser._action_groups[2]]))
parser.exit()
if args.help_b:
print(format_help(parser, parser._action_groups[3:]))
parser.exit()
Sample runs
1444:~/mypy$ python stack40718566.py --help_a
usage: stack40718566.py [-h] [--help_a] [--help_b] [--foo1 FOO1] [--foo2 FOO2]
[--foo3 FOO3]
Feature 1:
--foo1 FOO1
1444:~/mypy$ python stack40718566.py --help_b
usage: stack40718566.py [-h] [--help_a] [--help_b] [--foo1 FOO1] [--foo2 FOO2]
[--foo3 FOO3]
Feature 2:
--foo2 FOO2
Feature 3:
--foo3 FOO3
So it's just like the default format_help, except it takes a groups parameter. It could even replace the default method in an ArgumentParser subclass.
We could also create a custom Help Action class that behaves like the standard help, except that it takes some sort of group_list parameter. But this post-parsing action is simpler to code and test.

I recommend against what you are trying to do.
You are solving a problem that isn't yours to solve. It is the job of your script to return usage information. It isn't your problem if that is a lot of text. The thing that you could do, you are doing: Put arguments into groups that make sense for the user. But the amount of text is not a problem of data structure but of data presentation.
Secondly, you would be following a convention nobody is using. There usually are
man command
command --help
command subcommand --help
Anything else would be confusing to first time users.
Also, if you have a lot of argument groups a person would always need to consult --help to find out which --help_* they would have to consult next. This can be frustrating to users when you could just present it in --help right away.
If you use multiple help pages, you would prevent the reuse of your help text. Searching, for example: Multiple pages cannot be searched without switching between them manually.
The right way to do is pass text through a paginator like less. This allows users to read the text page by page, search through it (press /) or save it to file:
command --help | less
For convenience some commands, like git log, even check if the output is an interactive terminal and automatically pass the output through less. This would mean
command --help > help.txt
saves the help to file, while
command --help
shows the help text in pagination, and searchable.
So my recommendation for you on both Windows and UNIX is
import os
import sys
import argparse
import subprocess
def less(data):
if sys.stdout.isatty():
if os.name == 'posix':
cmd = "less"
elif os.name == 'nt':
cmd = "more"
process = subprocess.Popen([cmd], stdin=subprocess.PIPE)
try:
process.stdin.write(data)
process.communicate()
except IOError:
pass
else:
print data
class MyArgumentParser(argparse.ArgumentParser):
def print_help(self, file=None):
less(self.format_help())
self.exit()
parser = MyArgumentParser(prog='PROG')
group1 = parser.add_argument_group("Feature 1")
group1.add_argument('--foo1')
group2 = parser.add_argument_group("Feature 2")
group2.add_argument('--foo2')
group3 = parser.add_argument_group("Feature 3")
group3.add_argument('--foo3')
# parse some argument lists
print parser.parse_args()

Parsing exclusive groups in Python

I have 2 group which are exclusive, you can define either arguments from group1 or group2 but group2 have to be exclusive within it's arguments too.
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
sub_exclusive_1.add_argument("-a")
sub_exclusive_1.add_argument("-b")
sub_exclusive_1.add_argument("-c")
sub_exclusive_1.add_argument("-d")
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
args = parser.parse_args()
The code have to terminate if [-a and -AA or -BB] or [-AA and -BB] have been defined but still have to work with [-a and/or -b],
The problem is that it's not terminating...
I found this thread and edited my code to
subparsers = parser.add_subparsers()
parser_a = subparsers.add_parser('command_1')
parser_a.add_argument("-a")
parser_a.add_argument("-b")
parser_a.add_argument("-c")
parser_a.add_argument("-d")
parser_b = subparsers.add_parser('command_2')
parser_b.add_argument("-AA")
parser_b.add_argument("-BB")
still does not work, traceback: main.py: error: too few arguments
What do i do wrong?
current workaround:
parser = argparse.ArgumentParser()
parser.add_argument("-a")
...
parser.add_argument("-AA")
args = parser.parse_args()
if (args.a or args.b or args.c or args.d) and (args.AA or args.BB) or (args.AA and args.BB):
raise SystemExit()

At the risk of repeating my answer from the earlier question, let's focus on your case
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
...
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
Despite similar names (and class nesting), the functionality of argument_groups and mutually_exclusive_groups is quite different. And the former does not nest meaningfully within the second.
An argument group is a tool to organize arguments in the help. It does not enter arguments 'as a group' into another group, and has NO effect on parsing or error checking.
If it did act as you want, what would the usage line look like?
With the subparser formulation the parser responds with:
prog command1 -a -b -c # ok
prog command1 -a -AA # error - not recognize -AA
prog command2 -AA -BB # ok
prog command2 -a -AA # error - -a not recognized
prog -AA # error - too few arg
The subparser mechanism is similar to
parser.add_argument('cmd', choices=['command1','command2']
The 'command1' string tells it - parser the reset of the strings using the '-a -b ...' group of arguments. It has to know which group you expect it to use.
Short of using the bug/issue patch that I worked on a while back, you need to do your own 'mutually-exclusive' testing after parsing. As long as you use the default default None, it is is easy to test whether an argument has been used or now (args.AA is not None).
https://stackoverflow.com/a/30337890/901925 is a recent example of doing post-parsing testing.

Mutual exclusion between argument groups

I'm trying to implement the following argument dependency using the argparse module:
./prog [-h | [-v schema] file]
meaning the user must pass either -h or a file, if a file is passed the user can optionally pass -v schema.
That's what I have now but that doesn't seem to be working:
import argparse
parser = argparse.ArgumentParser()
mtx = parser.add_mutually_exclusive_group()
mtx.add_argument('-h', ...)
grp = mtx.add_argument_group()
grp.add_argument('-v', ...)
grp.add_argument('file', ...)
args = parser.parse_args()
It looks like you can't add an arg group to a mutex group or am I missing something?

If -h means the default help, then this is all you need (this help is already exclusive)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('file')
parser.add_argument('-s','--schema')
parser.parse_args('-h'.split()) # parser.print_help()
producing
usage: stack23951543.py [-h] [-s SCHEMA] file
...
If by -h you mean some other action, lets rename it -x. This would come close to what you describe
parser = argparse.ArgumentParser()
parser.add_argument('-s','--schema', default='meaningful default value')
mxg = parser.add_mutually_exclusive_group(required=True)
mxg.add_argument('-x','--xxx', action='store_true')
mxg.add_argument('file', nargs='?')
parser.parse_args('-h'.split())
usage is:
usage: stack23951543.py [-h] [-s SCHEMA] (-x | file)
Now -x or file is required (but not both). -s is optional in either case, but with a meaningful default, it doesn't matter if it is omitted. And if -x is given, you can just ignore the -s value.
If necessary you could test args after parsing, to confirm that if args.file is not None, then args.schema can't be either.
Earlier I wrote (maybe over thinking the question):
An argument_group cannot be added to a mutually_exclusive_group. The two kinds of groups have different purposes and functions. There are previous SO discussions of this (see 'related'), as well as a couple of relevant Python bug issues. If you want tests that go beyond a simple mutually exclusive group, you probably should do your own testing after parse_args. That may also require your own usage line.
An argument_group is just a means of grouping and labeling arguments in the help section.
A mutually_exclusive_group affects the usage formatting (if it can), and also runs tests during parse_args. The use of 'group' for both implies that they are more connected than they really are.
http://bugs.python.org/issue11588 asks for nested groups, and the ability to test for 'inclusivity' as well. I tried to make the case that 'groups' aren't general enough to express all the kinds of testing that users want. But it's one thing to generalize the testing mechanism, and quite another to come up with an intuitive API. Questions like this suggest that argparse does need some sort of 'nested group' syntax.