I'm using argparse with several subparsers. I want my program to take options for verbosity anywhere in the args, including the subparser.
from argparse import ArgumentParser
p = ArgumentParser()
p.add_argument('--verbose', '-v', action='count')
sub = p.add_subparsers()
a = sub.add_parser('a')
print(p.parse_args())
By default, options for the main parser will throw an error if used for subparsers:
$ python tmp.py -v a
Namespace(verbose=1)
$ python tmp.py a -v
usage: tmp.py [-h] [--verbose] {a} ...
tmp.py: error: unrecognized arguments: -v
I looked into parent parsers, from this answer.
from argparse import ArgumentParser
parent = ArgumentParser(add_help=False)
parent.add_argument('--verbose', '-v', action='count')
main = ArgumentParser(parents=[parent])
sub = main.add_subparsers()
a = sub.add_parser('a', parents=[parent])
print(main.parse_args())
For some reason though, none of the shared flags work on the main parser.
$ python tmp2.py a -vvv
Namespace(verbose=3)
$ python tmp2.py -vvv a
Namespace(verbose=None)
Note that the main parser definitely has the appropriate arguments, because when I change it to main = ArgumentParser() I get error: unrecognized arguments: -v. What am I missing here?
First, a couple of general comments.
The main parser handles the input upto the subparser invocation, then the subparser is called and given the remaining argv. When it is done, it's namespace is merged back into the the main namespace.
The parents mechanism copies Actions from the parent by reference. So your main and subparsers share the same verbose Action object. That's been a problem when the subparser tries to set a different default or help. It may not be an issue here, but just keep it in mind.
Even without the parents mechanism, sharing a dest or options flag between main and subparser can be tricky. Should the default of the subparser Action be used? What if both are used? Does the subparser overwrite the main parser's actions?
Originally the main namespace was passed to the subparser, which it modified and returned. This was changed a while back (I can find the bug/issue if needed). Now the subparser starts with a default empty namespace, fills it. And these values are then merged into the main.
So in your linked SO question, be wary of older answers. argparse may have changed since then.
I think what's happening in your case is that the main and subparser verbose are counting separately. And when you get None it's the subparser's default that you see.
The __call__ for _Count_Action is:
def __call__(self, parser, namespace, values, option_string=None):
new_count = _ensure_value(namespace, self.dest, 0) + 1
setattr(namespace, self.dest, new_count)
I suspect that in older argparse when the namespace was shared, the count would have been cumulative, but I can't test it without recreating an older style subparser action class.
https://bugs.python.org/issue15327 - here the original developer suggests giving the two arguments different dest. That records the inputs from both main and sub. Your own code can then merge the results if needed.
https://bugs.python.org/issue27859 argparse - subparsers does not retain namespace. Here I suggest a way of recreating the older style.
https://bugs.python.org/issue9351 argparse set_defaults on subcommands should override top level set_defaults - this is the issue in 2014 that changed the namespace use.
My workaround for this behavior, which is very well described in #hpaulj's answer is to create a second parser that does not have subparsers but only the positional arguments that were first found.
The first parse_args, used with the first parser, will validate the positional arguments and flags, show an error message if needed or show the proper help.
The second parse_args, for the second parser, will correctly fill in the namespace.
Building on your example:
from argparse import ArgumentParser
parent = ArgumentParser(add_help=False)
parent.add_argument('--verbose', '-v', action='count')
main1 = ArgumentParser(parents=[parent])
sub = main1.add_subparsers()
# eg: tmp.py -vv a -v
a = sub.add_parser('a', parents=[parent])
a.set_defaults(which='a')
# eg: tmp.py -vv v -v --output toto
b = sub.add_parser('b', parents=[parent])
b.add_argument('--output', type=str)
b.set_defaults(which='b')
args = main1.parse_args()
print(args)
# parse a second time with another parser
main2 = ArgumentParser(parents=[parent])
if args.which == 'a':
main2.add_argument('a')
elif args.which == 'b':
main2.add_argument('b')
main2.add_argument('--output', type=str)
print(main2.parse_args())
Which gives:
$ ./tmp.py -vv a -v
Namespace(verbose=1, which='a')
Namespace(a='a', verbose=3)
$ ./tmp.py -vv b -v --output toto
Namespace(output='toto', verbose=1, which='b')
Namespace(b='b', output='toto', verbose=3)
$ ./tmp.py -vv a --output
usage: tmp.py [-h] [--verbose] {a,b} ...
tmp.py: error: unrecognized arguments: --output
I use this technique with multiple nested subparsers.
Related
I'm trying to implement the following argument dependency using the argparse module:
./prog [-h | [-v schema] file]
meaning the user must pass either -h or a file, if a file is passed the user can optionally pass -v schema.
That's what I have now but that doesn't seem to be working:
import argparse
parser = argparse.ArgumentParser()
mtx = parser.add_mutually_exclusive_group()
mtx.add_argument('-h', ...)
grp = mtx.add_argument_group()
grp.add_argument('-v', ...)
grp.add_argument('file', ...)
args = parser.parse_args()
It looks like you can't add an arg group to a mutex group or am I missing something?
If -h means the default help, then this is all you need (this help is already exclusive)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('file')
parser.add_argument('-s','--schema')
parser.parse_args('-h'.split()) # parser.print_help()
producing
usage: stack23951543.py [-h] [-s SCHEMA] file
...
If by -h you mean some other action, lets rename it -x. This would come close to what you describe
parser = argparse.ArgumentParser()
parser.add_argument('-s','--schema', default='meaningful default value')
mxg = parser.add_mutually_exclusive_group(required=True)
mxg.add_argument('-x','--xxx', action='store_true')
mxg.add_argument('file', nargs='?')
parser.parse_args('-h'.split())
usage is:
usage: stack23951543.py [-h] [-s SCHEMA] (-x | file)
Now -x or file is required (but not both). -s is optional in either case, but with a meaningful default, it doesn't matter if it is omitted. And if -x is given, you can just ignore the -s value.
If necessary you could test args after parsing, to confirm that if args.file is not None, then args.schema can't be either.
Earlier I wrote (maybe over thinking the question):
An argument_group cannot be added to a mutually_exclusive_group. The two kinds of groups have different purposes and functions. There are previous SO discussions of this (see 'related'), as well as a couple of relevant Python bug issues. If you want tests that go beyond a simple mutually exclusive group, you probably should do your own testing after parse_args. That may also require your own usage line.
An argument_group is just a means of grouping and labeling arguments in the help section.
A mutually_exclusive_group affects the usage formatting (if it can), and also runs tests during parse_args. The use of 'group' for both implies that they are more connected than they really are.
http://bugs.python.org/issue11588 asks for nested groups, and the ability to test for 'inclusivity' as well. I tried to make the case that 'groups' aren't general enough to express all the kinds of testing that users want. But it's one thing to generalize the testing mechanism, and quite another to come up with an intuitive API. Questions like this suggest that argparse does need some sort of 'nested group' syntax.
I want to implement an arg parser that allows me to run unittests as one of the sub commands, blindly passing the arguments on to unittest.main(). e.g.,
$ foo.py unittest [args to pass to unittest.main()]
along with other sub commands:
$ foo.py foo ...
$ foo.py bar ...
Following argparse's example, this works:
#!/usr/bin/python
import argparse
p = argparse.ArgumentParser(prog='PROG')
p.add_argument('-v', '--verbose', action='store_true')
sub = p.add_subparsers(dest='cmd')
foo = sub.add_parser('foo')
bar = sub.add_parser('bar')
unittest = sub.add_parser('unittest')
unittest.add_argument('command') # Need to add this to make it work.
unittest.add_argument('args', nargs=argparse.REMAINDER)
print(p.parse_args('unittest command -blah blah'.split()))
Output:
Namespace(args=['-blah', 'blah'], cmd='unittest', command='command', verbose=False)
But this doesn't. It seems to require a "normal" argument first:
#!/usr/bin/python
import argparse
p = argparse.ArgumentParser(prog='PROG')
p.add_argument('-v', '--verbose', action='store_true')
sub = p.add_subparsers(dest='cmd')
foo = sub.add_parser('foo')
bar = sub.add_parser('bar')
unittest = sub.add_parser('unittest')
unittest.add_argument('args', nargs=argparse.REMAINDER)
print(p.parse_args('unittest -blah blah'.split()))
Output:
$ /tmp/foo.py
usage: PROG [-h] [-v] {foo,bar,unittest} ...
PROG: error: unrecognized arguments: -blah
I can do print(p.parse_args('unittest -- -f -g'.split())), but requiring -- kind of defeats the purpose of argparse.REMAINDER.
Is there a way to get argparse to do what I want? Or do I just need to hand parse this case?
Python 2.7.5
Looks like the same issue discussed in http://bugs.python.org/issue17050, argparse.REMAINDER doesn't work as first argument
My deduction from 4 years ago still holds - the -blah is being classed as an optional's flag even before REMAINDER has a chance to act. '--' is parsed earlier, but ... is, in a sense just a generalization of '*'. And not a widely used one. For what it's worth the 'subparsers' Action has a nargs='+...' value (argparse.PARSER) - it's like REMAINDER except it requires at least one string, the 'cmd'.
The possible fix in http://bugs.python.org/issue9334 has not been acted on. So you either need to handle the '-blah' by itself, or use '--'. parse_known_args might also work in your case.
As noted, the existing behavior is bad. One workaround is to implement a simple
ArgumentParser subclass and use that for your subparser:
class SubcommandParser(argparse.ArgumentParser):
"""This subparser puts all remaining arguments in args attribute of namespace"""
def parse_known_args(self, args=None, namespace=None):
if namespace is None:
namespace = argparse.Namespace()
setattr(namespace, 'args', args)
return namespace, []
...
p.add_subparsers(dest='cmd', parser_class=SubcommandParser)
I have 2 group which are exclusive, you can define either arguments from group1 or group2 but group2 have to be exclusive within it's arguments too.
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
sub_exclusive_1.add_argument("-a")
sub_exclusive_1.add_argument("-b")
sub_exclusive_1.add_argument("-c")
sub_exclusive_1.add_argument("-d")
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
args = parser.parse_args()
The code have to terminate if [-a and -AA or -BB] or [-AA and -BB] have been defined but still have to work with [-a and/or -b],
The problem is that it's not terminating...
I found this thread and edited my code to
subparsers = parser.add_subparsers()
parser_a = subparsers.add_parser('command_1')
parser_a.add_argument("-a")
parser_a.add_argument("-b")
parser_a.add_argument("-c")
parser_a.add_argument("-d")
parser_b = subparsers.add_parser('command_2')
parser_b.add_argument("-AA")
parser_b.add_argument("-BB")
still does not work, traceback: main.py: error: too few arguments
What do i do wrong?
current workaround:
parser = argparse.ArgumentParser()
parser.add_argument("-a")
...
parser.add_argument("-AA")
args = parser.parse_args()
if (args.a or args.b or args.c or args.d) and (args.AA or args.BB) or (args.AA and args.BB):
raise SystemExit()
At the risk of repeating my answer from the earlier question, let's focus on your case
parser = argparse.ArgumentParser()
group_exclusive = parser.add_mutually_exclusive_group()
sub_exclusive_1 = group_exclusive.add_argument_group()
...
sub_exclusive_2 = group_exclusive.add_mutually_exclusive_group()
sub_exclusive_2.add_argument("-AA")
sub_exclusive_2.add_argument("-BB")
Despite similar names (and class nesting), the functionality of argument_groups and mutually_exclusive_groups is quite different. And the former does not nest meaningfully within the second.
An argument group is a tool to organize arguments in the help. It does not enter arguments 'as a group' into another group, and has NO effect on parsing or error checking.
If it did act as you want, what would the usage line look like?
With the subparser formulation the parser responds with:
prog command1 -a -b -c # ok
prog command1 -a -AA # error - not recognize -AA
prog command2 -AA -BB # ok
prog command2 -a -AA # error - -a not recognized
prog -AA # error - too few arg
The subparser mechanism is similar to
parser.add_argument('cmd', choices=['command1','command2']
The 'command1' string tells it - parser the reset of the strings using the '-a -b ...' group of arguments. It has to know which group you expect it to use.
Short of using the bug/issue patch that I worked on a while back, you need to do your own 'mutually-exclusive' testing after parsing. As long as you use the default default None, it is is easy to test whether an argument has been used or now (args.AA is not None).
https://stackoverflow.com/a/30337890/901925 is a recent example of doing post-parsing testing.
I'm trying to pass a list of arguments with argparse but the only way that I've found involves rewriting the option for each argument that I want to pass:
What I currently use:
main.py -t arg1 -a arg2
and I would like:
main.py -t arg1 arg2 ...
Here is my code:
parser.add_argument("-t", action='append', dest='table', default=[], help="")
Use nargs:
ArgumentParser objects usually associate a single command-line
argument with a single action to be taken. The nargs keyword argument
associates a different number of command-line arguments with a single
action.
For example, if nargs is set to '+'
Just like '*', all command-line args present are gathered into a list.
Additionally, an error message will be generated if there wasn’t at
least one command-line argument present.
So, your code would look like
parser.add_argument('-t', dest='table', help='', nargs='+')
That way -t arguments will be gathered into list automatically (you don't have to explicitly specify the action).
Being aware, you asked for argparse solution, I would like to present alternative solution using package docopt
Install it first:
$ pip install docopt
Write the code:
"""Usage:
main.py -a <arg>...
"""
if __name__ == "__main__":
from docopt import docopt
resargs = docopt(__doc__)
print resargs
Run it to show usage instrucitons:
$ python main.py
Usage:
main.py -a <arg>...
Call it with your parameters:
$ python main.py -a AA BB CC
{'-a': True,
'<arg>': ['AA', 'BB', 'CC']}
Btw. if you do not need the -a option, you shall directly allow passing the arguments. It makes usage simpler to the user.
Is there a Python module for doing gem/git-style command line arguments? What I mean by gem/git style is:
$ ./MyApp.py
The most commonly used MyApp commands are:
add Add file contents to the index
bisect Find by binary search the change that introduced a bug
branch List, create, or delete branches
checkout Checkout a branch or paths to the working tree
...
$ ./MyApp.py branch
* current-branch
master
With no arguments, the output tells you how you can proceed. And there is a special "help" command:
$ ./MyApp.py help branch
Which gets you deeper tips about the "branch" command.
Edit:
And by doing I mean it does the usage printing for you, exits with invalid input, runs your functions according to your CLI specification. Sort of a "URL mapper" for the command line.
Yes, argparse with add_subparsers().
It's all well explained in the Sub-commands section.
Copying one of the examples from there:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers()
>>> checkout = subparsers.add_parser('checkout', aliases=['co'])
>>> checkout.add_argument('foo')
>>> parser.parse_args(['checkout', 'bar'])
Namespace(foo='bar')
Edit: Unfortunately there's no self generated special help command, but you can get the verbose help message (that you seem to want) with -h or --help like one normally would after the command:
$ ./MyApp.py branch --help
By verbose I don't mean that is like a man page, it's like every other --help kind of help: listing all the arguments, etc...
Example:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers(description='Sub description')
>>> checkout = subparsers.add_parser('checkout', description='Checkout description')
>>> checkout.add_argument('foo', help='This is the foo help')
>>> parser.parse_args(['checkout', '--help'])
usage: checkout [-h] foo
Checkout description
positional arguments:
foo This is the foo help
optional arguments:
-h, --help show this help message and exit
If you need to, it should be easy to implement an help command that redirects to --help.
A reasonable hack to get the gem/git style "help" behavior (I just wrote this for what I'm working on anyway):
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='sub_commands')
parser_branch = subparsers.add_parser('branch', description='list of branches')
parser_help = subparsers.add_parser('help')
parser_help.add_argument('command', nargs="?", default=None)
# I can't find a legitimate way to set a default subparser in the docs
# If you know of one, please let me know!
if len(sys.argv) < 2:
sys.argv.append('--help')
parsed = parser.parse_args()
if parsed.sub_commands == "help":
if not parsed.command:
parser.parse_args(['--help'])
else:
parser.parse_args([parsed.command, '--help'])
argparse is definitely a step up from optparse and other python solutions I've come across. But IMO the gem/git style of handling args is just a more logical and safer way to do things so it's annoying that it's not supported.
I wanted to do something similar to git commands, where I would load a second script based off of one of the command line options, and have that script populate more command line options, and also have the help work.
I was able to do this by disabling the help option, parse known args, add more arguments, re-enable the help option, and then parse the rest of the arguments.
This is what I came up with.
import argparse
#Note add_help=False
arg_parser = argparse.ArgumentParser(description='Add more arguments after parsing.',add_help=False)
arg_parser.add_argument('MODE', default='default',type=str, help='What commands to use')
args = arg_parser.parse_known_args()[0]
if args.MODE == 'branch':
arg_parser.add_argument('-d', '--delete', default='Delete a branch')
arg_parser.add_argument('-m', '--move', default='move a branch')
elif args.MODE == 'clone' :
arg_parser.add_argument('--local', '-l')
arg_parser.add_argument('--shared')
#Finally re-enable the help option, and reparse the arguments
arg_parser.add_argument(
'-h', '--help',
action='help', default=argparse.SUPPRESS,
help=argparse._('show this help message and exit'))
args = arg_parser.parse_args()