I am trying to make my python script very user friendly, so I like to write some sort of help for it. What is your advise for this? I could just put in some logic that if the user passed help as a paramater to the script, they get help. Is there a best practise or convention for this?
Use argparse.
For example, with test.py:
import argparse
parser=argparse.ArgumentParser(
description='''My Description. And what a lovely description it is. ''',
epilog="""All is well that ends well.""")
parser.add_argument('--foo', type=int, default=42, help='FOO!')
parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
args=parser.parse_args()
Running
% test.py -h
yields
usage: test.py [-h] [--foo FOO] [bar [bar ...]]
My Description. And what a lovely description it is.
positional arguments:
bar BAR!
optional arguments:
-h, --help show this help message and exit
--foo FOO FOO!
All is well that ends well.
Best practice is to use argparse to handle all your commandline arguments. It includes a default --help which you can customize to your likings.
Here's the simplest example:
import argparse
parser = argparse.ArgumentParser(description='This is my help')
args = parser.parse_args()
Which results in:
% python argparse_test.py -h
usage: argparse_test.py [-h]
This is my help
optional arguments:
-h, --help show this help message and exit
You can define all your arguments with argparse and set a help message for each one of them. The resulting filtered/validated arguments are returned by parser.parse_args().
An alternative to the built-in argparse is a 3rd-party package called Click which features "automatic help page generation" and "arbitrary nesting of commands" (which also produces nested help pages). Internally, it's based on argparse, but, for me, makes the creation of complex CLI more convenient using decorators.
Here's a sample code:
import click
#click.command()
#click.argument("things", nargs=-1)
#click.option("-t", show_default=True, default="int", help="Data type")
#click.option("-o", help="Output format")
def combine(things, t):
"""Combines things into a single element"""
pass
if __name__ == "__main__":
combine()
And the generated help page:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] [THINGS]...
Combines things into a single element
Options:
-t TEXT Data type [default: int]
-o TEXT Output format
--help Show this message and exit.
One of the nice things about it is that it uses the method docstrings as part of the help page, which is convenient because the docstring can now be used both for developer documentation and for script usage help.
You can also have nested command groups:
import click
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for add")
def add(numbers, e):
"""Adds numbers"""
print(f"This method should add {numbers}")
#click.command()
#click.argument("numbers", nargs=-1)
#click.option("-e", help="Extra option for mul")
def mul(numbers, e):
"""Multiplies numbers"""
print(f"This method should multiply {numbers}")
#click.group()
def calc():
pass
calc.add_command(add)
calc.add_command(mul)
if __name__ == "__main__":
calc()
And it will produce nested help pages:
$ python myapp.py --help
Usage: myapp.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
add Adds numbers
mul Multiplies numbers
$ python myapp.py add --help
Usage: myapp.py add [OPTIONS] [NUMBERS]...
Adds numbers
Options:
-e TEXT Extra option for add
--help Show this message and exit.
$ python myapp.py mul --help
Usage: myapp.py mul [OPTIONS] [NUMBERS]...
Multiplies numbers
Options:
-e TEXT Extra option for mul
--help Show this message and exit.
For more information, see the Documenting Scripts section of the docs.
Related
I've been using argparse for a Python program that can -process, -upload or both:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
The program is meaningless without at least one parameter. How can I configure argparse to force at least one parameter to be chosen?
UPDATE:
Following the comments: What's the Pythonic way to parametrize a program with at least one option?
if not (args.process or args.upload):
parser.error('No action requested, add -process or -upload')
args = vars(parser.parse_args())
if not any(args.values()):
parser.error('No arguments provided.')
I know this is old as dirt, but the way to require one option but forbid more than one (XOR) is like this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
print args
Output:
>opt.py
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: one of the arguments -process -upload is required
>opt.py -upload
Namespace(process=False, upload=True)
>opt.py -process
Namespace(process=True, upload=False)
>opt.py -upload -process
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: argument -process: not allowed with argument -upload
If not the 'or both' part (I have initially missed this) you could use something like this:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('--process', action='store_const', const='process', dest='mode')
parser.add_argument('--upload', action='store_const', const='upload', dest='mode')
args = parser.parse_args()
if not args.mode:
parser.error("One of --process or --upload must be given")
Though, probably it would be a better idea to use subcommands instead.
Requirements Review
use argparse (I will ignore this one)
allow one or two actions to be called (at least one required).
try to by Pythonic (I would rather call it "POSIX"-like)
There are also some implicit requirements when living on command line:
explain the usage to the user in a way which is easy to understand
options shall be optional
allow specifying flags and options
allow combining with other parameters (like file name or names).
Sample solution using docopt (file managelog.py):
"""Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Try to run it:
$ python managelog.py
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Show the help:
$ python managelog.py -h
Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> P managelog.py [options] upload -- <logfile>...
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
And use it:
$ python managelog.py -V -U user -P secret upload -- alfa.log beta.log
{'--': True,
'--pswd': 'secret',
'--user': 'user',
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': False,
'upload': True}
Short alternative short.py
There can be even shorter variant:
"""Manage logfiles
Usage:
short.py [options] (process|upload)... -- <logfile>...
short.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Usage looks like this:
$ python short.py -V process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 1,
'upload': 1}
Note, that instead of boolean values for "process" and "upload" keys there are counters.
It turns out, we cannot prevent duplication of these words:
$ python short.py -V process process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 2,
'upload': 1}
Conclusions
Designing good command line interface can be challenging sometime.
There are multiple aspects of command line based program:
good design of command line
selecting/using proper parser
argparse offers a lot, but restricts possible scenarios and can become very complex.
With docopt things go much shorter while preserving readability and offering high degree of flexibility. If you manage getting parsed arguments from dictionary and do some of conversions (to integer, opening files..) manually (or by other library called schema), you may find docopt good fit for command line parsing.
For http://bugs.python.org/issue11588 I am exploring ways of generalizing the mutually_exclusive_group concept to handle cases like this.
With this development argparse.py, https://github.com/hpaulj/argparse_issues/blob/nested/argparse.py
I am able to write:
parser = argparse.ArgumentParser(prog='PROG',
description='Log archiver arguments.')
group = parser.add_usage_group(kind='any', required=True,
title='possible actions (at least one is required)')
group.add_argument('-p', '--process', action='store_true')
group.add_argument('-u', '--upload', action='store_true')
args = parser.parse_args()
print(args)
which produces the following help:
usage: PROG [-h] (-p | -u)
Log archiver arguments.
optional arguments:
-h, --help show this help message and exit
possible actions (at least one is required):
-p, --process
-u, --upload
This accepts inputs like '-u', '-up', '--proc --up' etc.
It ends up running a test similar to https://stackoverflow.com/a/6723066/901925, though the error message needs to be clearer:
usage: PROG [-h] (-p | -u)
PROG: error: some of the arguments process upload is required
I wonder:
are the parameters kind='any', required=True clear enough (accept any of the group; at least one is required)?
is usage (-p | -u) clear? A required mutually_exclusive_group produces the same thing. Is there some alternative notation?
is using a group like this more intuitive than phihag's simple test?
The best way to do this is by using python inbuilt module add_mutually_exclusive_group.
parser = argparse.ArgumentParser(description='Log archiver arguments.')
group = parser.add_mutually_exclusive_group()
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
If you want only one argument to be selected by command line just use required=True as an argument for group
group = parser.add_mutually_exclusive_group(required=True)
If you require a python program to run with at least one parameter, add an argument that doesn't have the option prefix (- or -- by default) and set nargs=+ (Minimum of one argument required). The problem with this method I found is that if you do not specify the argument, argparse will generate a "too few arguments" error and not print out the help menu. If you don't need that functionality, here's how to do it in code:
import argparse
parser = argparse.ArgumentParser(description='Your program description')
parser.add_argument('command', nargs="+", help='describe what a command is')
args = parser.parse_args()
I think that when you add an argument with the option prefixes, nargs governs the entire argument parser and not just the option. (What I mean is, if you have an --option flag with nargs="+", then --option flag expects at least one argument. If you have option with nargs="+", it expects at least one argument overall.)
This achieves the purpose and this will also be relfected in the argparse autogenerated --help output, which is imho what most sane programmers want (also works with optional arguments):
parser.add_argument(
'commands',
nargs='+', # require at least 1
choices=['process', 'upload'], # restrict the choice
help='commands to execute'
)
Official docs on this:
https://docs.python.org/3/library/argparse.html#choices
Maybe use sub-parsers?
import argparse
parser = argparse.ArgumentParser(description='Log archiver arguments.')
subparsers = parser.add_subparsers(dest='subparser_name', help='sub-command help')
parser_process = subparsers.add_parser('process', help='Process logs')
parser_upload = subparsers.add_parser('upload', help='Upload logs')
args = parser.parse_args()
print("Subparser: ", args.subparser_name)
Now --help shows:
$ python /tmp/aaa.py --help
usage: aaa.py [-h] {process,upload} ...
Log archiver arguments.
positional arguments:
{process,upload} sub-command help
process Process logs
upload Upload logs
optional arguments:
-h, --help show this help message and exit
$ python /tmp/aaa.py
usage: aaa.py [-h] {process,upload} ...
aaa.py: error: too few arguments
$ python3 /tmp/aaa.py upload
Subparser: upload
You can add additional options to these sub-parsers as well. Also instead of using that dest='subparser_name' you can also bind functions to be directly called on given sub-command (see docs).
For cases like
parser.add_argument("--a")
parser.add_argument("--b")
We can use the following
if not args.a and not args.b:
parser.error("One of --a or --b must be present")
Use append_const to a list of actions and then check that the list is populated:
parser.add_argument('-process', dest=actions, const="process", action='append_const')
parser.add_argument('-upload', dest=actions, const="upload", action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
You can even specify the methods directly within the constants.
def upload:
...
parser.add_argument('-upload', dest=actions, const=upload, action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
else:
for action in args.actions:
action()
Using
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
Maybe try:
if len([False for arg in vars(args) if vars(args)[arg]]) == 0:
parsers.print_help()
exit(-1)
At least this is what I just used; hopefully this helps someone in the future!
I'm using Python's argparse, and I want there to be less indentation of the argument help text. This is what argparse is generating:
$ ./help.py -h
usage: help.py [-h] [--program-argument PROGRAM_ARGUMENT]
Description of program
optional arguments:
-h, --help show this help message and exit
--program-argument PROGRAM_ARGUMENT
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
I want it to generate something more like this:
$ ./help.py -h
usage: help.py [-h] [--program-argument PROGRAM_ARGUMENT]
Description of program
optional arguments:
-h, --help show this help message and exit
--program-argument PROGRAM_ARGUMENT
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
Is that achievable? This is my code:
#! /usr/bin/env python
import argparse
HELP_TEXT = """\
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
"""
if __name__ == '__main__':
argument_parser = argparse.ArgumentParser(
formatter_class=argparse.RawTextHelpFormatter,
description=('Description of program'))
argument_parser.add_argument(
'--program-argument',
help=HELP_TEXT
)
args, unknown = argument_parser.parse_known_args()
argparse formatters support several initialization values that can help control some formatting. They all derive from HelpFormatter, which has this __init__ method.
class HelpFormatter(object):
"""Formatter for generating usage messages and argument help strings.
Only the name of this class is considered a public API. All the methods
provided by the class are considered an implementation detail.
"""
def __init__(self,
prog,
indent_increment=2,
max_help_position=24,
width=None):
# stuff
The max_help_position is used when determining how far to indent help sub-messages, so you could try reducing it to something like 10 or 12 to get less indentation of your messages.
#!/usr/bin/env python
import argparse
HELP_TEXT = """\
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
"""
less_indent_formatter = lambda prog: argparse.RawTextHelpFormatter(prog, max_help_position=10)
if __name__ == '__main__':
argument_parser = argparse.ArgumentParser(
formatter_class=less_indent_formatter,
description=('Description of program'))
argument_parser.add_argument(
'--program-argument',
help=HELP_TEXT
)
args, unknown = argument_parser.parse_known_args()
This results in:
usage: help.py [-h] [--program-argument PROGRAM_ARGUMENT]
Description of program
optional arguments:
-h, --help
show this help message and exit
--program-argument PROGRAM_ARGUMENT
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
A value of 6 looks like this:
usage: help.py [-h] [--program-argument PROGRAM_ARGUMENT]
Description of program
optional arguments:
-h, --help
show this help message and exit
--program-argument PROGRAM_ARGUMENT
This is some help text about --program-argument. For example:
--program-argment "You can supply a string as the program argument"
I'm using argsparse to parse the options passed to my python scripts.
I want to enable passing '-a', and to allow passing '-b' with the same affect.
No problem, I'll call parser.add_argument() twice, with the same description:
parser.add_argument('-a', help='do something')
parser.add_argument('-b', help='do something')
But now when displaying the script help, I will see both, as such:
-a do something
-b do something
This is ugly.
I would prefer to have {-a, -b} or {-a|b).
I could not find in argsparse documentation any way around this (admittedly, not critical) issue.
You could try to pass both arguments to the same add_argument call:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', '-b', help='do something')
parser.parse_args(['--help'])
output
usage: test.py [-h] [-a A]
optional arguments:
-h, --help show this help message and exit
-a A, -b A do something
the arg include an action field and optional switches that modify the behavior of the actions.
the argparse code is like the below:
parser=argparse.ArgumentParser()
parser.add-argument('action',metavar='action', choices=['analysis','report','update'],nargs='?', default='report')
parser.add-argument('-d',dest='delay',type=int, choices=range(1,10),default=1)
parser.add-argument('-v',dest='verbose',action='store-true',default=False)
parser.add-argument('-o',dest='offline',action='store-true',default=False)
parser.add-argument('-n',dest='names',required=False)
i want to make the switch option -o, -d, -v only available for action=report, while option -n only available for action=analysis.
i know there is a mutual group setting, but it is just set for the arguments not for the argument values!
btw, does argparse support combine options, like -vo...???
First: Yes, combined options like -vo are supported
Now to the action switching:
Adding a SubParser to your ArgumentParser is exactely what you want to do.
See Section 15.4.5.1. Sub-commands in the Documentation of argparse, everything is explained to some detail and with example there
EDIT:
I know refer to your comment below this post:
If you do not provide a subcommand parameter in the program call, the parser normally gives you a kind advice how to use the program and then exits with a "too few arguments error"
I rewrote your code to show this
test.py
import argparse
parser=argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
parser_analysis = subparsers.add_parser('analysis', help='analysis help text')
parser_analysis.add_argument('-n',dest='names',required=False)
parser_report = subparsers.add_parser('report', help='report help text')
parser_report.add_argument('-d',dest='delay',type=int, choices=range(1,10),default=1)
parser_report.add_argument('-v',dest='verbose',action='store_true',default=False)
parser_report.add_argument('-o',dest='offline',action='store_true',default=False)
parser_update = subparsers.add_parser('update', help='update help text')
parser.parse_args()
Now some calls of this test.py with different arguments:
$python test.py
usage: test.py [-h] {analysis,report,update} ...
test.py: error: too few arguments
$python test.py -h
usage: test.py [-h] {analysis,report,update} ...
positional arguments:
{analysis,report,update}
sub-command help
analysis analysis help text
report report help text
update update help text
optional arguments:
-h, --help show this help message and exit
$python test.py report -h
usage: test.py report [-h] [-d {1,2,3,4,5,6,7,8,9}] [-v] [-o]
optional arguments:
-h, --help show this help message and exit
-d {1,2,3,4,5,6,7,8,9}
-v
-o
So as I see it the only problem is that the program throws an error after calling python test.py without any subcommand. So I would do s.th. like this
try:
args=parser.parse_args()
except:
exit(0)
to avoid that the user sees that there was an unhandled error. You have then the same behaviour as i.e. the svn command.
If you want to handle this in the way that a default subcommand is executed, you whould have to do s.th. like mentioned in this post answer:
Argparse - How to Specify a Default Subcommand
import sys
#...your parser definitions
if (len(sys.argv) < 2):
args = parser.parse_args(['update'])
else:
args = parser.parse_args()
This would parse a command update if the argument list in sys.argv is smaller than 2. Why 2? Because the first argument in the argument list is always the program which you called, i.e. test.py
The question is, do you really want this behaviour? Because there is no need for calling test.py update if i can always call test.py, so users will probably get lazy and never use test.py update command. Also if you later want a different default behaviour like test.py starting an interactive mode, users which are by then used to calling test.py for updating will get confused or their scripts which use your program get broken.
Python optparse works very good when script usage is something like this
%prog [options] [args]
But I need to write help for script with 1 required argument, so usage will be like this
%prog action [options] [args]
You can see something similar when you use Subversion - its usage string is
svn <subcommand> [options] [args]
So my question is: is it possible to prepare help for required argument with optparse in the manner of Subversion? As a result I want to see help like this:
Usage: python myscript.py action [options] [args]
Available actions:
foo
bar
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-v, --verbose Verbose mode. Output debug log to stdout.
I think a good solution for you is argparse, which has been proposed for inclusion in Python 2.7 and 3.2. It handles subcommands, I believe as you want, and the linked page includes a link to a page on porting your code from optparse.
See also the question command-line-arguments-in-python, into which someone edited a list of references that appears to include exactly the same thing you want:
Yes. You can set the usage string like this:
usage = "%prog action [options] [args]"
parser = OptionParser(usage=usage)
parser.add_option("-v", "--verbose",
action="store_true", dest="verbose", default=True,
help="make lots of noise [default]")
Prints the following:
Usage: action [options] [args]
Options:
-h, --help show this help message and exit
-v, --verbose make lots of noise [default]
This was copied almost verbatim from the docs.
Edit:
Based on your comment you could use the description to achieve something similar, though you can't put new-line characters in it.
parser.description = 'Available actions: foo, bar'
Will look like this:
Usage: action [options] [args]
Available actions: foo, bar
Options:
-h, --help show this help message and exit
-v, --verbose make lots of noise [default]
I've run into this problem as well. My solution was to declare commands in a list or tuple, format them into the usage parameter of OptionParser and then use the args list provided by the parser to determine if a command was provided or not, since it technically has to be args[0]. Eg:
self.commands = ('foo', 'bar' ...)
self.parser = <initialized instance of OptionParser>
(self.options, self.args) = parser.parse_args()
if len(self.args) == 0:
self.parser.error("Command required")
self.command = self.args[0]
if not self.command in self.commands:
self.parser.error("Command not recognized")
#... etc
This kinda gets you a command system that looks like Subversion's, but admittedly optparse could be better. I've heard the argparse module is supposed to make it into the stdlib, but with 2.7 being the last of the 2 series releases, I guess you'd have to wait for it to be incorporated into 3.x. Of course you can just install argparse, but that's a drag in some cases.