I'm trying to create well formatted help messages for 'choice' type command line arguments with Python's argparse. For the command I allow the name '--operation' and the alias '-o'. Currently, argparse is printing the list of options next to both in the help message.
Please note that this question is different to the question of formatting the help messages of the options (That problem has a good answer here by Anthon: Python argparse: How to insert newline in the help text?)
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-o', '--operation', help="operation to perform", type=str, choices=["create", "update", "delete"])
_StoreAction(option_strings=['-o', '--operation'], dest='operation', nargs=None, const=None, default=None, type=<class 'str'>, choices=['create', 'update', 'delete'], help='operation to perform', metavar=None)
>>> parser.print_help()
usage: [-h] [-o {create,update,delete}]
optional arguments:
-h, --help show this help message and exit
-o {create,update,delete}, --operation {create,update,delete}
operation to perform
>>>
My problem is this line:
-o {create,update,delete}, --operation {create,update,delete}
It's very clunky how the list of options is repeated twice. Especially since I will have lists that are even longer. It would be better I think to have this:
-o, --operation {create,update,delete}
This is assuming of course that there isn't some POSIX rule about how this has to work. I don't think there is.
How can I achieve the desired output? Or is there a good reason that I shouldn't be trying to?
This is quite a hack, but there doesn't appear to be a good place to hook into this.
Define your own formatter, which overrides (by basically copying) the _format_action_invocation method. The only change you'll make is to add the choices only to the last option string.
class MyHelpFormatter(HelpFormatter):
def _format_action_invocation(self, action):
if not action.option_strings:
default = self._get_default_metavar_for_positional(action)
metavar, = self._metavar_formatter(action, default)(1)
return metavar
else:
parts = []
# if the Optional doesn't take a value, format is:
# -s, --long
if action.nargs == 0:
parts.extend(action.option_strings)
# if the Optional takes a value, format is:
# -s ARGS, --long ARGS
else:
default = self._get_default_metavar_for_optional(action)
args_string = self._format_args(action, default)
for option_string in action.option_strings[:-1]:
parts.append('%s' % (option_string,))
parts.append('%s %s' % (action.option_strings[-1], args_string)
return ', '.join(parts)
Related
I have created a pastebin terminal client in python. It can take some command line arguments, like -o to open a file, -n to set a paste name etc. It also has option -l which lists the pastes and allows you to delete or view pastes. The problem is that I don't know how to do it in a nice way (using argparse) - it should not allow to use -l with any other options.
I added a simple logic:
if args.name:
if args.list:
print('The -l should be used alone. Check "pb -h" for help.')
sys.exit()
Can it be done using just argparse?
I know about mutually exclusive groups, I even have one (to set paste privacy) but I haven't figured this one yet.
Full code is available here: https://github.com/lkorba/pbstc/blob/master/pb
I don't think you can use argparse to achieve your goal in "a nice way" as you say.
I see 2 options here:
1) The simpler solution as I get it would be to just check your arguments after parsing them. Nothing fancy just:
args = parser.parse_args()
if args.list is not None:
if not (args.name is None and args.open is None and
args.public is None and args.format is None and args.exp is None):
parser.error('Cannot use list with name, open, public, format or exp argument')
2) On the other hand you could revise a bit your program and use
subparsers like:
subparsers = parser.add_subparsers(title="commands", dest="command")
parser_a = subparsers.add_parser('list', help='list help')
parser_b = subparsers.add_parser('action', help='Any action here')
parser_b.add_argument('-f', action="store", help='Choose paste format/syntax: text=None, '
'mysql=MYSQL, perl=Perl, python=Python etc...')
parser_b.add_argument('-n', '--name', action="store")
parser_b.add_argument('-o', '--open', action="store", help='Open file')
...
args = parser.parse_args()
if args.command == 'list':
...
elif args.command == 'action':
...
So, for example if you pass list -n='Name' as arguments in the latter case you will get an error:
usage: subparser_example.py [-h] {list,action} ...
subparser_example.py: error: unrecognized arguments: -n='Name'
Of course you also get (as overhead) one extra parameter action here...
I am now starting exploring Python, and was testing how arguments can be passed to a script with "argparse".
The way I wrote a sample script was as following, where arguments passed through flags -i and -o are compulsory and flag -u is optional:
#!/usr/bin/python
import sys
import argparse
## set usage options and define arguments
usage = "usage: %prog [options]"
parser = argparse.ArgumentParser(usage)
parser.add_argument("-i", action="store", dest="input", help="input file")
parser.add_argument("-o", action="store", dest="output", help="output file")
parser.add_argument("-u", action="store_true", dest="isunfolded", default=False, help="optional flag")
args = parser.parse_args()
print len(sys.argv)
if len(sys.argv) < 2:
# parser.print_help()
print 'Incorrect number of params'
exit()
else:
print "Correct number of params: ", len(sys.argv)
Running this script:
> ./test-args.py -i a -o b
prints:
5
Correct number of params: 5
I understand the printing statement in the if conditional (5 is higher than 2), however, after reading the argparse documentation (https://docs.python.org/3/library/argparse.html) I still don't quite understand why -i and -o flags are counted as arguments. This behaviour seems to be quite different from e.g. perl Getopt::Std, which I'm more used to.
So, the question is what is the best way of parsing arguments in Python and to evaluate the presence of mandatory arguments (without using required=True)
It gives you 5 because sys.argv contains the raw input passed to python as arguments (the script name and 4 arguments).
You can see argparse as an abstraction for this, so once you use it, you can forget about sys.argv. In most cases it is better not to mix these two methods.
argparse its a nice way to handle arguments, I don't quite get why you don't want to use the required option when that's exactly the way to go. Another alternative is to parse the sys.argv yourself (regex maybe?) and drop argparse altogether.
There's a Python getopt which probably is similar to the Perl one (assuming both are modelled after the C/Unix version).
https://docs.python.org/2/library/getopt.html
In your code, sys.argv is a list of strings from the command line (as interpreted by the shell and interpreter). It is the raw input for any of the parsers ('getopt', 'optparse', 'argparse'). And it is available for your parsing as well. When learning it is a good idea to include a
print sys.argv
line. parser.parse_args() uses this list. sys.argv[0] is used as prog attribute (in the default usage), while sys.argv[1:] (the rest) is parsed according to the rules you define with add_argument. For testing I often like to use parse_args with a custom list of strings, e.g.
print parser.parse_args(['-i', 'input', '-o', 'output', '-u'])
With your definition I'd expect to see something like:
Namespace(input='input', output='output', isunfolded=True)
The parser returns an object (type argparse.Namespace), which has attributes defined by your arguments. Values are usually accessed with expressions like args.input, args.isunfolded. The docs also show how you easily express this as a dictionary.
By long standing UNIX conventions, arguments flagged by strings like '-i' are options, that is they are optional. argparse generalizes this concept by letting you specify a required=True parameter.
Other arguments are positionals. They are interpreted according to their order. And as such they are normally required. What argparse adds is the ability to define those positionals, such as type, nargs, etc. With nargs='?' they are optional. Many of the nargs values are similar to regular expression characters (e.g. +?*). In fact argparse uses a form a regular expression parsing to allocate strings among arguments.
I'd refine your arguments thus (taking advantage of various defaults)
a1 = parser.add_argument("-i", "--input", help="input file") # 'store' is the default
a2 = parser.add_argument("-o", "--output",help="output file") # use the --output as dest
a3 = parser.add_argument("-u", "--isunfolded", action="store_true", help="optional flag")
If input and output were required, I could change them to:
parser.add_argument("input", help="input file") # 'store' is the default
parser.add_argument("output",help="output file") # use the --output as dest
parser.add_argument("-u", "--isunfolded", action="store_true", help="optional flag")
Now input and output are positional arguments, as in test.py -u inputfile outputfile
By using a1 = parser... I can look at the object produced by this statement.
print a1
produces
_StoreAction(option_strings=['-i', '--input'], dest='input', nargs=None, const=None,
default=None, type=None, choices=None, help='input file', metavar=None)
This tells me that a1 is a _StoreAction object (a subclass of argparse.Action). It also displays a number (not all) of its attributes, ones that define its action. A positional, on the other hand, has values like these:
a2 = p.add_argument("output", help="output file")
_StoreAction(option_strings=[], dest='output', nargs=None, const=None,
default=None, type=None, choices=None, help='output file', metavar=None)
It may also be instructive to look at a1.required and a2.required, which are respectively False and True. required is an Action attribute that is not routinely displayed, but is, never the less accessible.
I've pulled all these test values from a parser defined in an interactive shell (Ipython). It's a great way to explore Python and modules like argparse.
After reading other related posts it seems that the best way to do this is as was suggested by #Rufflewind and inspect the args itself:
if not args.input or not args.output:
print 'Incorrect number of params'
exit()
else:
print "Correct number of params"
I was wondering if there was a way to use the metavar from argparse be grabbed from elsewhere. Example, there is a -f FILE option, and a -d DIR option. Can I make the help of -d grab the file.metavar or some such?
Maybe:
parser.add_argument("-f", "--file",
metavar = 'FILE',
action="store", dest="file_name", default="foo.bar",
help="Name of the {} to be loaded".format(metavar))
parser.add_argument("-d", "--dir",
metavar = 'DIR',
action = 'store', dest='file_dir', default=".",
help="Directory of {} if it is not current directory".format(option.file_name.metavar)
I know this code is wrong (string doesn't have metavar and options doesn't get set until the parser.parse_args() is run), but I have a few other times I want to just grab the metavar without having a bunch of:
meta_file = 'FILE'
meta_dir = 'DIR'
meta_inquisition = 'SURPRISE'
floating around.
Thanks.
EDIT: s/add_option/add_argument/
In argparse it is add_argument(). This method returns an Action object (actually a subclass of that depending the action parameter). You can access various parameters of that object, either to use, or even change. For example:
action1 = parser.add_argument(
"-f",
"--file",
metavar = "FILE",
dest="file_name",
default="foo.bar",
help="Name of the %(metavar)s to be loaded"
)
action2 = parser.add_argument(
"-d",
"--dir",
metavar="DIR",
dest="file_dir",
default=".",
help="Directory of %s if it is not current directory" % action1.metavar
)
print(action1.metavar) # read the metavar attribute
action2.metvar = "DIRECTORY" # change the metavar attribute
The help reads:
usage: ipython [-h] [-f FILE] [-d DIR]
optional arguments:
-h, --help show this help message and exit
-f FILE, --file FILE Name of the FILE to be loaded
-d DIR, --dir DIR Directory of FILE if it is not current directory
I removed action="store" since that is the default value (no big deal though).
I changed the help values to use %(metavar)s. This is used to incorporate various action attributes. Most commonly it is used for the default.
From the argparse docs:
The help strings can include various format specifiers to avoid repetition of things like the program name or the argument default. The available specifiers include the program name, %(prog)s and most keyword arguments to add_argument(), e.g. %(default)s, %(type)s, etc.:
I am using action1.metavar to place FILE in the help line for action2. It's not a common usage, but I don't see anything wrong with it.
Note that action1.metavar is used once when setting up the parser (to create the action2 help line), and then later when the help is formatted.
In [17]: action2.help
Out[17]: 'Directory of FILE if it is not current directory'
In [18]: action1.help
Out[18]: 'Name of the %(metavar)s to be loaded'
py3 style formatting can be use for action2:
help2 = "Directory of {} if it is not current directory".format(action2.metavar)
action2.help = help2
but py2 style has to be used for action1. Unless you did:
action1.help = "Name of the {} to be loaded".format(action1.metavar)
After creating both actions you even could use:
action1.help = "Name of the {} to be loaded from {}".format(action1.metavar, action2.metavar)
But that's just ordinary Python coding.
According to the docs for the help argument, the help text can contain format specifiers like %(default)s, %(type)s, etc. You can write %(metavar)s and it will be expanded to the value of the metavar (or None if it's not specified).
I don't think there's a way to grab another's argument metavar.
Suppose I have the following argparse snippet:
diags.cmdln_parser.add_argument( '--scan-time',
action = 'store',
nargs = '?',
type = int,
default = 5,
help = "Wait SCAN-TIME seconds between status checks.")
Currently, --help returns:
usage: connection_check.py [-h]
[--version] [--scan-time [SCAN_TIME]]
Test the reliability/uptime of a connection.
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
I would prefer something like:
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
(Default = 5)
Peeking at the help formatter code revealed limited options. Is there a clever way to get argparse to print the default value for --scan-time in a similar fashion, or should I just subclass the help formatter?
Use the argparse.ArgumentDefaultsHelpFormatter formatter:
parser = argparse.ArgumentParser(
# ... other options ...
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
To quote the documentation:
The other formatter class available, ArgumentDefaultsHelpFormatter, will add information about the default value of each of the arguments.
Note that this only applies to arguments that have help text defined; with no help value for an argument, there is no help message to add information about the default value to.
The exact output for your scan-time option then becomes:
--scan-time [SCAN_TIME]
Wait SCAN-TIME seconds between status checks.
(default: 5)
Add '%(default)s' to the help parameter to control what is displayed.
parser.add_argument("--type", default="toto", choices=["toto","titi"],
help = "type (default: %(default)s)")
Notes:
It is %+ default in parenthesis + format characters (not to be confused with curly brackets {default} we find in format or f-string)
Don't forget to add the "specifier character" for the type representation at the end (i.e. s for strings, d for integers, f for floats, etc.)
You can also add the usual "printf" format specifiers (like number of digits for floats, leading zeros, etc.)
You can refer to printf documentation for more details.
Wrapper class
This is the most reliable and DRY approach I've found so far to both show defaults and use another formatter such as argparse.RawTextHelpFormatter at the same time:
#!/usr/bin/env python3
import argparse
class ArgumentParserWithDefaults(argparse.ArgumentParser):
def add_argument(self, *args, help=None, default=None, **kwargs):
if help is not None:
kwargs['help'] = help
if default is not None and args[0] != '-h':
kwargs['default'] = default
if help is not None:
kwargs['help'] += ' Default: {}'.format(default)
super().add_argument(*args, **kwargs)
parser = ArgumentParserWithDefaults(
formatter_class=argparse.RawTextHelpFormatter
)
parser.add_argument('-a', default=13, help='''my help
for a''')
parser.add_argument('-b', default=42, help='''my help
for b''')
parser.add_argument('--no-default', help='''my help
for no-default''')
parser.add_argument('--no-help', default=101)
parser.print_help()
print()
print(parser.parse_args())
Output:
usage: main.py [-h] [-a A] [-b B] [--no-default NO_DEFAULT]
[--no-help NO_HELP]
optional arguments:
-h, --help show this help message and exit
-a A my help
for a Default: 13
-b B my help
for b Default: 42
--no-default NO_DEFAULT
my help
for no-default
--no-help NO_HELP
Namespace(a=13, b=42, no_default=None, no_help=101)
ArgumentDefaultsHelpFormatter + RawTextHelpFormatter multiple inheritance
Multiple inheritance just works, but it does not seem to be public API:
#!/usr/bin/env python3
import argparse
class RawTextArgumentDefaultsHelpFormatter(
argparse.ArgumentDefaultsHelpFormatter,
argparse.RawTextHelpFormatter
):
pass
parser = argparse.ArgumentParser(
formatter_class=RawTextArgumentDefaultsHelpFormatter
)
parser.add_argument('-a', default=13, help='''my help
for a''')
parser.add_argument('-b', default=42, help='''my help
for b''')
parser.print_help()
Output:
usage: a.py [-h] [-a A] [-b B]
optional arguments:
-h, --help show this help message and exit
-a A my help
for a (default: 13)
-b B my help
for b (default: 42)
It just works works because as we can see trivially from the sources https://github.com/python/cpython/blob/v3.6.5/Lib/argparse.py#L648 that:
RawTextHelpFormatter implements _split_lines
ArgumentDefaultsHelpFormatter implements _get_help_string
so we can guess that they will work together just fine.
However, this does not seem to be public API, and neither are the methods of formatter_class, so I don't think there is a public API way to do it currently. argparse docstring says:
All other classes in this module are considered implementation details.
(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
considered public as object names -- the API of the formatter objects is
still considered an implementation detail.)
See also: Customize argparse help message
Tested on Python 3.6.5.
It is often useful to be able to automatically include the default values in the help output, but only those that were explicitly specified (with default=..). The methods already mentioned have some shortcomings in this respect:
The ArgumentDefaultsHelpFormatter method prints out (default: None) for every argument whose default was not explicitly specified, and (default: False) for 'flags' (action='store_true'). This clutters the help output. To avoid it, default=argparse.SUPPRESS needs to be manually added for each such argument.
The '%(default)s' method requires manually adding it to all the arguments' help strings that we do want printed in help.
Both methods end up needing manual intervention to print out only the "right" defaults.
One way to do this automatically is to augment the ArgumentDefaultsHelpFormatter to ignore the Nones and Falses default values:
class ExplicitDefaultsHelpFormatter(argparse.ArgumentDefaultsHelpFormatter):
def _get_help_string(self, action):
if action.default in (None, False):
return action.help
return super()._get_help_string(action)
Use it in place of ArgumentDefaultsHelpFormatter:
parser = argparse.ArgumentParser(
formatter_class=ExplicitDefaultsHelpFormatter
)
This will print only the explicitly set default values in the help output.
Note: if an argument's default was explicitly set as None or False, it won't be shown in help with this class; add %(default)s string to help for that argument if you want it in the help output.
Is there a Python module for doing gem/git-style command line arguments? What I mean by gem/git style is:
$ ./MyApp.py
The most commonly used MyApp commands are:
add Add file contents to the index
bisect Find by binary search the change that introduced a bug
branch List, create, or delete branches
checkout Checkout a branch or paths to the working tree
...
$ ./MyApp.py branch
* current-branch
master
With no arguments, the output tells you how you can proceed. And there is a special "help" command:
$ ./MyApp.py help branch
Which gets you deeper tips about the "branch" command.
Edit:
And by doing I mean it does the usage printing for you, exits with invalid input, runs your functions according to your CLI specification. Sort of a "URL mapper" for the command line.
Yes, argparse with add_subparsers().
It's all well explained in the Sub-commands section.
Copying one of the examples from there:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers()
>>> checkout = subparsers.add_parser('checkout', aliases=['co'])
>>> checkout.add_argument('foo')
>>> parser.parse_args(['checkout', 'bar'])
Namespace(foo='bar')
Edit: Unfortunately there's no self generated special help command, but you can get the verbose help message (that you seem to want) with -h or --help like one normally would after the command:
$ ./MyApp.py branch --help
By verbose I don't mean that is like a man page, it's like every other --help kind of help: listing all the arguments, etc...
Example:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers(description='Sub description')
>>> checkout = subparsers.add_parser('checkout', description='Checkout description')
>>> checkout.add_argument('foo', help='This is the foo help')
>>> parser.parse_args(['checkout', '--help'])
usage: checkout [-h] foo
Checkout description
positional arguments:
foo This is the foo help
optional arguments:
-h, --help show this help message and exit
If you need to, it should be easy to implement an help command that redirects to --help.
A reasonable hack to get the gem/git style "help" behavior (I just wrote this for what I'm working on anyway):
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='sub_commands')
parser_branch = subparsers.add_parser('branch', description='list of branches')
parser_help = subparsers.add_parser('help')
parser_help.add_argument('command', nargs="?", default=None)
# I can't find a legitimate way to set a default subparser in the docs
# If you know of one, please let me know!
if len(sys.argv) < 2:
sys.argv.append('--help')
parsed = parser.parse_args()
if parsed.sub_commands == "help":
if not parsed.command:
parser.parse_args(['--help'])
else:
parser.parse_args([parsed.command, '--help'])
argparse is definitely a step up from optparse and other python solutions I've come across. But IMO the gem/git style of handling args is just a more logical and safer way to do things so it's annoying that it's not supported.
I wanted to do something similar to git commands, where I would load a second script based off of one of the command line options, and have that script populate more command line options, and also have the help work.
I was able to do this by disabling the help option, parse known args, add more arguments, re-enable the help option, and then parse the rest of the arguments.
This is what I came up with.
import argparse
#Note add_help=False
arg_parser = argparse.ArgumentParser(description='Add more arguments after parsing.',add_help=False)
arg_parser.add_argument('MODE', default='default',type=str, help='What commands to use')
args = arg_parser.parse_known_args()[0]
if args.MODE == 'branch':
arg_parser.add_argument('-d', '--delete', default='Delete a branch')
arg_parser.add_argument('-m', '--move', default='move a branch')
elif args.MODE == 'clone' :
arg_parser.add_argument('--local', '-l')
arg_parser.add_argument('--shared')
#Finally re-enable the help option, and reparse the arguments
arg_parser.add_argument(
'-h', '--help',
action='help', default=argparse.SUPPRESS,
help=argparse._('show this help message and exit'))
args = arg_parser.parse_args()