I'm trying to create a python script that will execute another script, depending on the first positional parameter. Think along the lines of how git add behaves.
Problem is that ArgumentParser appears to want the positional sub-command to be listed... at the end. Which is pretty counter-intuitive. (When you want to list all files, you do ls -a [FILE positional], not -a ls [FILE positional], so why would it require scriptname [optionals] subcommand instead of scriptname subcommand [optionals] since 'subcommand' is the 'real' command?)
Toy example:
def get_arg_parser():
parser = argparse.ArgumentParser()
# set up subprocessors
subparser = parser.add_subparsers(required=True)
parser.add_argument('--verbose', action='store_const', const=True, default=False, help="Enable verbose output.")
subcommand1_subparser = subparser.add_parser('subcommand1')
subcommand1_subparser.add_argument('--foo1', type=float)
subcommand2_subparser = subparser.add_parser('subcommand2')
subcommand2_subparser.add_argument('--foo2', type=float)
return parser
if __name__ == "__main__":
if len(sys.argv) > 1:
get_arg_parser().parse_args()
# more
else:
get_arg_parser().print_help()
Problem is that if I try to run python toyexample.py subcommand1 --verbose, it complains about error: unrecognized arguments: --verbose. Meanwhile, python toyexample.py --verbose subcommand1 works, but it's requiring the optionals before the name of the command you're actually intending to run.
How do I override this?
Thanks to #hpaulj, I found a solution: simply add the shared arguments to both subparsers.
I put the parser.add_argument('--verbose', action='store_const', const=True, default=False, help="Enable verbose output.") line in a add_shared_args_to_parser to function, which I then call twice, passing the subparsers.
Net result is that the subparsers have some unfortunate (but not terrible) duplication and the main parser has nothing but subparsers.
Related
Positional parameters with nargs='+' or nargs=<integer> are required, meaning that argparse gives an error if the parameter is not specified at least once. However, no error is given if the user calls the program with the -h|--help option, regardless of whether the positional parameters were specified.
How can I implement custom options like -h that do not require positional parameters to be set, while still requiring positional parameters for all other options?
For example, if let's say my program has the following (or similar) usage text:
usage: PROG [-h] [-o] [-u] positional
How can I implement the following behaviour:
calling PROG with no options and no positionals is an error
calling PROG with -u and no positionals is an error
calling PROG with only positionals is allowed
calling PROG with -h or -o is allowed regardless of whatever else is specified
My code
This meets all requirements except the last one, namely -o still requires the positional parameter to be specified.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('positional', nargs=1)
parser.add_argument('-o', action='store_true')
parser.add_argument('-u', action='store_true')
args = parser.parse_args()
print(args)
Does argparse have a built-in way to say that positional is not required iff -o is specified, or must this behaviour be implemented manually?
Manual methods
Set nargs='?' or nargs='*' for the positional parameter (i.e. make it optional), and then add the following lines immediately after args = parser.parse_args():
if not args.o and args.positional is None:
parser.error("the following arguments are required: positional")
Or add these lines, courtesy of this answer:
if len(sys.argv) == 1:
# display help message when no args are passed.
parser.print_help()
sys.exit(1)
This simply prints the help message if no options or positional parameters were specified. This behaviour is consistent with many command line programs, e.g. git.
Of course, making positional optional will affect the usage message:
usage: PROG [-h] [-o] [-u] [positional]
If you really don't want this to happen you can override the usage method like this:
parser = argparse.ArgumentParser(usage="%(prog)s [-h] [-o] [-u] positional")
I have code:
parser = ArgumentParser()
parser.add_argument('--verbose', action='count', default=0, help='debug output')
subparsers = parser.add_subparsers(help='subparser')
parser1 = subparsers.add_parser('action', help='Do something')
parser1.add_argument('--start', action='store_true', help='start')
parser1.add_argument('--stop', action='store_true', help='stop')
parser2 = subparsers.add_parser('control', help='Control something')
parser2.add_argument('--input', action='store_true', help='start')
parser2.add_argument('--output', action='store_true', help='stop')
args = parser.parse_args()
Then I can run script:
script.py --verbose action --start
script.py --verbose control --output
but not
script.py action --start --verbose
script.py control --output --verbose
Can I transfer option --verbose to the end, without adding it to each group?
To elaborate on my comment:
argparse parses the input list (sys.argv[1:]) in order, matching the strings with the Actions (add_argument object). So if the command is
python foo.py --arg1=3 cmd --arg2=4
it tries to handle '--arg1', then 'cmd'. If 'cmd' matches a subparser name, it then delegates the parsing to that parser, giving the remaining strings to it. If the cmd subparser can handle --arg2, it returns that as an unrecognized argument.
The main parser does not resume parsing. Rather it just handles the unrecognized arguments as it normally would - raising an error if using parse_args, and returning them in the extras list if using parse_known_args.
So if you want to put --verbose at the end, you have define it as a subparser argument. Or do some further parsing after parse_known_args.
You are allowed to define --verbose at both levels, though sometimes such a definition can create conflicts (especially if defaults differ).
The parents mechanism can be used to reduce the amount of typing, though you could just as easily write your own utility functions.
I define a parser with a description, options, and an epilog. When I run the app with --help, it outputs help with the epilog as expected. However, I only want to see the epilog if --help is accompanied with --verbose. What is the proper way to achieve this with argparse?
# example code in file test
import argparse
parser = argparse.ArgumentParser( description='description', epilog='epilog' )
parser.add_argument('-v', '--verbose', action='store_true', help='verbose help')
parser.parse_args()
When I run test as follows
$ python test -h
it yields
usage: test [-h] [-v]
description
optional arguments:
-h, --help show this help message and exit
-v, --verbose verbose help
epilog
However, what I want to see is
usage: test [-h] [-v]
description
optional arguments:
-h, --help show this help message and exit
-v, --verbose verbose help
with the epilog shown only when I run
$ python test -h -v
Ick. The only way I know of doing this is by writing the help output by yourself:
import argparse
parser = argparse.ArgumentParser(
description='description',
add_help=False )
parser.add_argument(
'-h', '--help',
action=store_true,
dest='show_help')
parser.add_argument(
'-v', '--verbose',
action='store_true',
help='verbose help')
args = parser.parse_args()
if args.show_help:
if args.verbose:
print '%s\n%s' % (parser.format_help(), 'epilog')
else
parser.print_help()
sys.exit(0)
There's no provision in argparse for this. So you will have to write your own code to change the epilog before parsing, or perform your own help after parsing, or conceivably modifying the format_help method.
You can view and change the epilog attribute of the parser after creation.
parser = argparse.ArgumentParser(epilog='test')
print parser.epilog # should see 'test'
parser.epilog = None # or ''
One deleted answer suggested looking at sys.argv before parsing, and if the --verbose is present, modify the the epilog attribute. That may miss some ways of specifying the value (e.g. -hv), but it is relatively simple.
Acting on the --verbose during parsing is difficult. The parser will act on the -h as soon as it parses it, displaying the message and exiting. Thus any -v after -h will be missed.
Doing your own help after parsing is a viable option, if you turn off the regular help (thus preventing that print and exit action). You will know the final values of both help and verbose. But you will be responsible for your own exit.
Using the ideas suggested, here's what I came up with:
import argparse
parser = argparse.ArgumentParser( description='description', epilog='', add_help=False )
parser.add_argument('-h', '--help', action='store_true', help='show help')
parser.add_argument('-v', '--verbose', action='store_true', help='more help')
args = parser.parse_args()
if args.help:
if args.verbose:
parser.epilog += "epilog for %(prog)s"
else:
parser.epilog += "\nfor more help run '%(prog)s -h -v'"
parser.print_help()
parser.exit(0)
print 'the end'
The only difficulty I found with this approach is that it is no longer possible to add required options or positional arguments. A workaround for positional arguments is to use nargs='?' and do the checking manually.
I would suggest a different approach.
1 Build the parser as you have done right now.
do a pretty print on the parser and figure out how epilog is stored in an option. Or put a debug via pdb.set_trace() and use dirs and vars to look around.
i.e. figure out what the option data structure looks like with an epilog and without an epilog.
2 Instead of calling parser.parse_args() (standard use):
look at sys.argv yourself. If -h and -v leave the parser as is.
if -h but not -v, adjust your parser before calling it to look as if it had no epilog.
3 call with parser.parse_args()
You could even build 2 parsers, one with epilog, one without and dynamically decide which one to call depending on -v flag.
p.s. actually, you want to check
if "-h" in sys.argv and not "-v" in sys.argv
I also see the value of verbose help to add examples.
From Python 2.7 argparse printing help and Python 2.7 argument parser objects and comments above, I settled upon the following method:
import argparse
. . .
if __name__ == '__main__':
description_text = """
DESCRIPTION
This command ...
"""
epilog_text = """
After execution, the user can ...
"""
example_text = """
EXAMPLES
The following examples ...
"""
parser = argparse.ArgumentParser(
description=description_text,
epilog=epilog_text,
formatter_class=argparse.RawDescriptionHelpFormatter,
add_help=False)
parser.add_argument('-h', '--help', dest='help', action='store_true',
help='Show help and exit; see also --verbose')
parser.add_argument('--usage', dest='usage', action='store_true',
help='Show usage and exit')
. . .
parser.add_argument('-v', '--verbose', dest='verbose',
action='store_true',
help='Display additional help or logging')
arguments = parser.parse_args()
if arguments.usage:
print(parser.format_usage())
sys.exit(0)
if arguments.help:
help_string = parser.format_help()
if arguments.verbose:
help_string += example_text
print(help_string)
sys.exit(0)
. . .
The result is a flexible output which supports --help, --help --verbose and --usage controls for the command. Thanks to others above for the inspiration.
In Python's argparse, how do you implement top-level arguments while still using commands implemented as subparsers?
I'm trying to implement a --version argument to show the program's version number, but argparse is giving me error: too few arguments because I'm not specifying a sub-command for one of the subparsers.
My code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
'-v', '--version',
help='Show version.',
action='store_true',
default=False
)
subparsers = parser.add_subparsers(
dest="command",
)
list_parser = subparsers.add_parser('list')
parser.parse_args(['--version'])
the output:
usage: myscript.py [-h] [-v] {list} ...
myscript.py: error: too few arguments
If you only need version to work, you can do this:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
'-v', '--version',
action='version',
version='%(prog)s 1.0',
)
Subparsers won't bother any more; the special version action is processed and exits the script before the parser looks for subcommands.
The subparsers is a kind of positional argument. So normally that's required (just as though you'd specified add_argument('foo')).
skyline's suggestion works because action='version' is an action class that exits after displaying its information, just like the default -h.
There is bug/feature in the latest argparse that makes subparsers optional. Depending on how that is resolved, it may be possible in the future to give the add_subparsers command a required=False parameter. But the intended design is that subparsers will be required, unless a flagged argument (like '-h') short circuits the parsing.
I am trying to make a Python program that uses the argparse module to parse command-line options.
I want to make an optional argument that can either be named or positional. For example, I want myScript --username=batman to do the same thing as myScript batman. I also want myScript without a username to be valid. Is this possible? If so, how can it be done?
I tried various things similar to the code below without any success.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin")
group.add_argument("user-name", default="admin")
args = parser.parse_args()
EDIT: The above code throws an exception saying ValueError: mutually exclusive arguments must be optional.
I am using Python 2.7.2 on OS X 10.8.4.
EDIT: I tried Gabriel Jacobsohn's suggestion but I couldn't get it working correctly in all cases.
I tried this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin", nargs="?")
group.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)
and running myScript batman would print Namespace(user_name='batman'), but myScript -u batman and myScript --user-name=batman would print Namespace(user_name='admin').
I tried changing the name user-name to user_name in the 1st add_argument line and that sometimes resulted in both batman and admin in the namespace or an error, depending on how I ran the program.
I tried changing the name user_name to user-name in the 2nd add_argument line but that would print either Namespace(user-name='batman', user_name='admin') or Namespace(user-name='admin', user_name='batman'), depending on how I ran the program.
The way the ArgumentParser works, it always checks for any trailing positional arguments after it has parsed the optional arguments. So if you have a positional argument with the same name as an optional argument, and it doesn't appear anywhere on the command line, it's guaranteed to override the optional argument (either with its default value or None).
Frankly this seems like a bug to me, at least when used in a mutually exclusive group, since if you had specified the parameter explicitly it would have been an error.
That said, my suggested solution, is to give the positional argument a different name.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-u','--username')
group.add_argument('static_username',nargs='?',default='admin')
Then when parsing, you use the optional username if present, otherwise fallback to the positional static_username.
results = parser.parse_args()
username = results.username or results.static_username
I realise this isn't a particularly neat solution, but I don't think any of the answers will be.
Here is a solution that I think does everything you want:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
# Gather all extra args into a list named "args.extra"
parser.add_argument("extra", nargs='*')
args = parser.parse_args()
# Set args.user_name to first extra arg if it is not given and len(args.extra) > 0
if args.user_name == parser.get_default("user_name") and args.extra:
args.user_name = args.extra.pop(0)
print args
If you run myScript -u batman or myScript --user-name=batman, args.user_name is set to 'batman'. If you do myScript batman, args.user_name is again set to 'batman'. And finally, if you just do myScript, args.user_name is set to 'admin'.
Also, as an added bonus, you now have all of the extra arguments that were passed to the script stored in args.extra. Hope this helps.
Try to use the "nargs" parameter of the add_argument method.
This way it works for me.
Now you can add the username twice,
so you have to check it and raise an error, if you want.
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
parser.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)