I've been using argparse for a Python program that can -process, -upload or both:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
The program is meaningless without at least one parameter. How can I configure argparse to force at least one parameter to be chosen?
UPDATE:
Following the comments: What's the Pythonic way to parametrize a program with at least one option?
if not (args.process or args.upload):
parser.error('No action requested, add -process or -upload')
args = vars(parser.parse_args())
if not any(args.values()):
parser.error('No arguments provided.')
I know this is old as dirt, but the way to require one option but forbid more than one (XOR) is like this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
print args
Output:
>opt.py
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: one of the arguments -process -upload is required
>opt.py -upload
Namespace(process=False, upload=True)
>opt.py -process
Namespace(process=True, upload=False)
>opt.py -upload -process
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: argument -process: not allowed with argument -upload
If not the 'or both' part (I have initially missed this) you could use something like this:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('--process', action='store_const', const='process', dest='mode')
parser.add_argument('--upload', action='store_const', const='upload', dest='mode')
args = parser.parse_args()
if not args.mode:
parser.error("One of --process or --upload must be given")
Though, probably it would be a better idea to use subcommands instead.
Requirements Review
use argparse (I will ignore this one)
allow one or two actions to be called (at least one required).
try to by Pythonic (I would rather call it "POSIX"-like)
There are also some implicit requirements when living on command line:
explain the usage to the user in a way which is easy to understand
options shall be optional
allow specifying flags and options
allow combining with other parameters (like file name or names).
Sample solution using docopt (file managelog.py):
"""Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Try to run it:
$ python managelog.py
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Show the help:
$ python managelog.py -h
Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> P managelog.py [options] upload -- <logfile>...
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
And use it:
$ python managelog.py -V -U user -P secret upload -- alfa.log beta.log
{'--': True,
'--pswd': 'secret',
'--user': 'user',
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': False,
'upload': True}
Short alternative short.py
There can be even shorter variant:
"""Manage logfiles
Usage:
short.py [options] (process|upload)... -- <logfile>...
short.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Usage looks like this:
$ python short.py -V process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 1,
'upload': 1}
Note, that instead of boolean values for "process" and "upload" keys there are counters.
It turns out, we cannot prevent duplication of these words:
$ python short.py -V process process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 2,
'upload': 1}
Conclusions
Designing good command line interface can be challenging sometime.
There are multiple aspects of command line based program:
good design of command line
selecting/using proper parser
argparse offers a lot, but restricts possible scenarios and can become very complex.
With docopt things go much shorter while preserving readability and offering high degree of flexibility. If you manage getting parsed arguments from dictionary and do some of conversions (to integer, opening files..) manually (or by other library called schema), you may find docopt good fit for command line parsing.
For http://bugs.python.org/issue11588 I am exploring ways of generalizing the mutually_exclusive_group concept to handle cases like this.
With this development argparse.py, https://github.com/hpaulj/argparse_issues/blob/nested/argparse.py
I am able to write:
parser = argparse.ArgumentParser(prog='PROG',
description='Log archiver arguments.')
group = parser.add_usage_group(kind='any', required=True,
title='possible actions (at least one is required)')
group.add_argument('-p', '--process', action='store_true')
group.add_argument('-u', '--upload', action='store_true')
args = parser.parse_args()
print(args)
which produces the following help:
usage: PROG [-h] (-p | -u)
Log archiver arguments.
optional arguments:
-h, --help show this help message and exit
possible actions (at least one is required):
-p, --process
-u, --upload
This accepts inputs like '-u', '-up', '--proc --up' etc.
It ends up running a test similar to https://stackoverflow.com/a/6723066/901925, though the error message needs to be clearer:
usage: PROG [-h] (-p | -u)
PROG: error: some of the arguments process upload is required
I wonder:
are the parameters kind='any', required=True clear enough (accept any of the group; at least one is required)?
is usage (-p | -u) clear? A required mutually_exclusive_group produces the same thing. Is there some alternative notation?
is using a group like this more intuitive than phihag's simple test?
The best way to do this is by using python inbuilt module add_mutually_exclusive_group.
parser = argparse.ArgumentParser(description='Log archiver arguments.')
group = parser.add_mutually_exclusive_group()
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
If you want only one argument to be selected by command line just use required=True as an argument for group
group = parser.add_mutually_exclusive_group(required=True)
If you require a python program to run with at least one parameter, add an argument that doesn't have the option prefix (- or -- by default) and set nargs=+ (Minimum of one argument required). The problem with this method I found is that if you do not specify the argument, argparse will generate a "too few arguments" error and not print out the help menu. If you don't need that functionality, here's how to do it in code:
import argparse
parser = argparse.ArgumentParser(description='Your program description')
parser.add_argument('command', nargs="+", help='describe what a command is')
args = parser.parse_args()
I think that when you add an argument with the option prefixes, nargs governs the entire argument parser and not just the option. (What I mean is, if you have an --option flag with nargs="+", then --option flag expects at least one argument. If you have option with nargs="+", it expects at least one argument overall.)
This achieves the purpose and this will also be relfected in the argparse autogenerated --help output, which is imho what most sane programmers want (also works with optional arguments):
parser.add_argument(
'commands',
nargs='+', # require at least 1
choices=['process', 'upload'], # restrict the choice
help='commands to execute'
)
Official docs on this:
https://docs.python.org/3/library/argparse.html#choices
Maybe use sub-parsers?
import argparse
parser = argparse.ArgumentParser(description='Log archiver arguments.')
subparsers = parser.add_subparsers(dest='subparser_name', help='sub-command help')
parser_process = subparsers.add_parser('process', help='Process logs')
parser_upload = subparsers.add_parser('upload', help='Upload logs')
args = parser.parse_args()
print("Subparser: ", args.subparser_name)
Now --help shows:
$ python /tmp/aaa.py --help
usage: aaa.py [-h] {process,upload} ...
Log archiver arguments.
positional arguments:
{process,upload} sub-command help
process Process logs
upload Upload logs
optional arguments:
-h, --help show this help message and exit
$ python /tmp/aaa.py
usage: aaa.py [-h] {process,upload} ...
aaa.py: error: too few arguments
$ python3 /tmp/aaa.py upload
Subparser: upload
You can add additional options to these sub-parsers as well. Also instead of using that dest='subparser_name' you can also bind functions to be directly called on given sub-command (see docs).
For cases like
parser.add_argument("--a")
parser.add_argument("--b")
We can use the following
if not args.a and not args.b:
parser.error("One of --a or --b must be present")
Use append_const to a list of actions and then check that the list is populated:
parser.add_argument('-process', dest=actions, const="process", action='append_const')
parser.add_argument('-upload', dest=actions, const="upload", action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
You can even specify the methods directly within the constants.
def upload:
...
parser.add_argument('-upload', dest=actions, const=upload, action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
else:
for action in args.actions:
action()
Using
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
Maybe try:
if len([False for arg in vars(args) if vars(args)[arg]]) == 0:
parsers.print_help()
exit(-1)
At least this is what I just used; hopefully this helps someone in the future!
I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?
With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.
The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
I had a perfectly fine and working piece of code, which used argparse. I've been using it for months for work without any issue. Below is an excrept.
import argparse
import sys
import math
import random
# Setup command line arguments
parser = argparse.ArgumentParser(description='RF Profile Generator', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('-o', dest='OutputFile', help='Output filename', required=True, type=argparse.FileType('w', encoding='utf-8'))
parser.add_argument('-a', dest='APPositions', nargs='+', type=float, help='Trackside AP position(s)', required=True)
parser.add_argument('-l', dest='TotalTrackLength', type=float, help='Length of the track (m)', required=True)
parser.add_argument('-p', dest='AttenuatorAddr', nargs='+', help='Programmable attenuator IP address(es)', required=True)
input("A")
# parse the command line arguments
# the parsed values will be stored in the corresponding variables defined by 'dest'
args = parser.parse_args()
input("B")
#More code to follow
However, I ran the code today with the following arguments (which i have been doing all along):
rf.py -o OutputFile -a 10 20 30 40 -l 600 -p 10.0.1.55
and the console output tells me
"error: the following arguments are required: -o, -a, -l, -p"
I do not understand why this has stopped working. To troubleshoot, i added 2 input commands, but the code never reaches input("B")
Could someone please advise me on a possible reason for this to happen. The version of python is 3.4.1, and has not been updated between the last time i ran this code and now.
Thank you very much
Since required=True is present on all the calls to add_argument, my conclusion is that either
somebody else has changed your program, or
this is the first time you have tried to run it without providing those arguments
I do, however, find it odd that running your program with the --help option gives the following output:
usage: so16.py [-h] -o OUTPUTFILE -a APPOSITIONS [APPOSITIONS ...] -l
TOTALTRACKLENGTH -p ATTENUATORADDR [ATTENUATORADDR ...]
RF Profile Generator
optional arguments:
-h, --help show this help message and exit
-o OUTPUTFILE Output filename (default: None)
-a APPOSITIONS [APPOSITIONS ...]
Trackside AP position(s) (default: None)
-l TOTALTRACKLENGTH Length of the track (m) (default: None)
-p ATTENUATORADDR [ATTENUATORADDR ...]
Programmable attenuator IP address(es) (default: None)
I'm puzzled as to why the message implies that required arguments are optional.
I have code:
parser = ArgumentParser()
parser.add_argument('--verbose', action='count', default=0, help='debug output')
subparsers = parser.add_subparsers(help='subparser')
parser1 = subparsers.add_parser('action', help='Do something')
parser1.add_argument('--start', action='store_true', help='start')
parser1.add_argument('--stop', action='store_true', help='stop')
parser2 = subparsers.add_parser('control', help='Control something')
parser2.add_argument('--input', action='store_true', help='start')
parser2.add_argument('--output', action='store_true', help='stop')
args = parser.parse_args()
Then I can run script:
script.py --verbose action --start
script.py --verbose control --output
but not
script.py action --start --verbose
script.py control --output --verbose
Can I transfer option --verbose to the end, without adding it to each group?
To elaborate on my comment:
argparse parses the input list (sys.argv[1:]) in order, matching the strings with the Actions (add_argument object). So if the command is
python foo.py --arg1=3 cmd --arg2=4
it tries to handle '--arg1', then 'cmd'. If 'cmd' matches a subparser name, it then delegates the parsing to that parser, giving the remaining strings to it. If the cmd subparser can handle --arg2, it returns that as an unrecognized argument.
The main parser does not resume parsing. Rather it just handles the unrecognized arguments as it normally would - raising an error if using parse_args, and returning them in the extras list if using parse_known_args.
So if you want to put --verbose at the end, you have define it as a subparser argument. Or do some further parsing after parse_known_args.
You are allowed to define --verbose at both levels, though sometimes such a definition can create conflicts (especially if defaults differ).
The parents mechanism can be used to reduce the amount of typing, though you could just as easily write your own utility functions.
My application is a specialized file comparison utility and obviously it does not make sense to compare only one file, so nargs='+' is not quite appropriate.
nargs=N only excepts a maximum of N arguments, but I need to accept an infinite number of arguments as long as there are at least two of them.
Short answer is you can't do that because nargs doesn't support something like '2+'.
Long answer is you can workaround that using something like this:
parser = argparse.ArgumentParser(usage='%(prog)s [-h] file file [file ...]')
parser.add_argument('file1', nargs=1, metavar='file')
parser.add_argument('file2', nargs='+', metavar='file', help=argparse.SUPPRESS)
namespace = parser.parse_args()
namespace.file = namespace.file1 + namespace.file2
The tricks that you need are:
Use usage to provide you own usage string to the parser
Use metavar to display an argument with a different name in the help string
Use SUPPRESS to avoid displaying help for one of the variables
Merge two different variables just adding a new attribute to the Namespace object that the parser returns
The example above produces the following help string:
usage: test.py [-h] file file [file ...]
positional arguments:
file
optional arguments:
-h, --help show this help message and exit
and will still fail when less than two arguments are passed:
$ python test.py arg
usage: test.py [-h] file file [file ...]
test.py: error: too few arguments
Couldn't you do something like this:
import argparse
parser = argparse.ArgumentParser(description = "Compare files")
parser.add_argument('first', help="the first file")
parser.add_argument('other', nargs='+', help="the other files")
args = parser.parse_args()
print args
When I run this with -h I get:
usage: script.py [-h] first other [other ...]
Compare files
positional arguments:
first the first file
other the other files
optional arguments:
-h, --help show this help message and exit
When I run it with only one argument, it won't work:
usage: script.py [-h] first other [other ...]
script.py: error: too few arguments
But two or more arguments is fine. With three arguments it prints:
Namespace(first='one', other=['two', 'three'])