I am using subparsers to implement the following function:
import argparse
parser = argparse.ArgumentParser(prog='dj')
parser.add_argument('--log','-l',type=str, help='log file')
parser.add_argument('--parser','-p',type=bool, nargs='?', const=True, default=False, help='wheher parsing a dj file or not')
subparsers = parser.add_subparsers(help='execute specified command')
###############################################################################
parser_rt = subparsers.add_parser('run_test', help='kick off a case')
parser_rt.add_argument('--test', '-t', type=str, help='test case name')
parser_rt.add_argument('--config','-c', type=str, help='config name')
parser_rt.set_defaults(func=run_test)
everything is fine when kicking off the command:
dj -p -l dj.log run_test -t testa -c configa
or
dj -l dj.log run_test -t testa -c configa
However, issue was popping up when running the command:
dj -l dj.log -p run_test -t testa -c configa
Here is error:
usage: dj [-h] [--log log] [--parser [PARSER]]
{run_test}...
dj: error: invalid choice: 'testa' (choose from 'run_test')
It seems that the issue is related to the position of -p.
By defining -p as nargs='?', const=True, default=False it still consumes the next argument, just not doing anything with it...
When it has run_test after it, it is consumed as an argument and not as a subparser and then -t is not defined either...
To overcome this, simply use the idiomatic way for defining "flag" arguments which is either store_true or store_false as the action. So to fix your parser, change to:
parser.add_argument('--parser','-p', action='store_true', help='wheher parsing a dj file or not')
When you ran -p -l there was no issue because argparse knows to differentiate arguments using the -. So in this case -p had no value and -l started a new argument. run_test was taken as a value before argparse checked it for a subparser.
In your code, -p might take an argument. Is that intended or do you rather want just a flag?
parser.add_argument('--parser','-p', action="store_true", help='whether parsing a dj file or not')
When you run dj -l dj.log -p run_test -t testa -c configa the parser thinks that run_test is an argument to -p.
Related
If I have the arguments '-a', '-b', '-c', '-d', with the add_mutually_exclusive_group() function my program will have to use just one of them. Is there a way to combine that, so that the program will accept only either '-a 999 -b 999' or '-c 999 -d 999'?
Edit: adding a simple program for more clarity:
>>> parser = argparse.ArgumentParser()
>>> group = parser.add_mutually_exclusive_group()
>>> group.add_argument('-a')
>>> group.add_argument('-b')
>>> group.add_argument('-c')
>>> group.add_argument('-d')
Then only ./app.py -a | ./app.py -b | ./app.py -c | ./app.py -d can be called. Is it possible to have argparse group the exclusion groups, so that only ./app.py -a .. -b .. | ./app.py -c .. -d .. be called?
EDIT: Never mind. Because argparse makes the horrible choice of having to create an option when invoking group.add_argument. That wouldn't be my design choice. If you're desperate for this feature, you can try doing it with ConflictsOptionParser:
# exclusivegroups.py
import conflictsparse
parser = conflictsparse.ConflictsOptionParser()
a_opt = parser.add_option('-a')
b_opt = parser.add_option('-b')
c_opt = parser.add_option('-c')
d_opt = parser.add_option('-d')
import itertools
compatible_opts1 = (a_opt, b_opt)
compatible_opts2 = (c_opt, d_opt)
exclusives = itertools.product(compatible_opts1, compatible_opts2)
for exclusive_grp in exclusives:
parser.register_conflict(exclusive_grp)
opts, args = parser.parse_args()
print "opts: ", opts
print "args: ", args
Thus when we invoke it, we can see we get the desired effect.
$ python exclusivegroups.py -a 1 -b 2
opts: {'a': '1', 'c': None, 'b': '2', 'd': None}
args: []
$ python exclusivegroups.py -c 3 -d 2
opts: {'a': None, 'c': '3', 'b': None, 'd': '2'}
args: []
$ python exclusivegroups.py -a 1 -b 2 -c 3
Usage: exclusivegroups.py [options]
exclusivegroups.py: error: -b, -c are incompatible options.
The warning message doesn't inform you that both '-a' and '-b' are incompatible with '-c', however a more appropriate error message could be crafted. Older, wrong answer below.
OLDER EDIT: [This edit is wrong, although wouldn't it be just a perfect world if argparse worked this way?] My previous answer actually was incorrect, you should be able to do this with argparse by specifying one group per mutually exclusive options. We can even use itertools to generalize the process. And make it so we don't have to type out all the combinations explicitly:
import itertools
compatible_opts1 = ('-a', '-b')
compatible_opts2 = ('-c', '-d')
exclusives = itertools.product(compatible_opts1, compatible_opts2)
for exclusive_grp in exclusives:
group = parser.add_mutually_exclusive_group()
group.add_argument(exclusive_grp[0])
group.add_argument(exclusive_grp[1])
Just stumbled on this problem myself. From my reading of the argparse docs, there doesn't seem to be a simple way to achieve that within argparse. I considered using parse_known_args, but that soon amounts to writing a special-purpose version of argparse ;-)
Perhaps a bug report is in order. In the meanwhile, if you're willing to make your user do a tiny bit extra typing, you can fake it with subgroups (like how git and svn's arguments work), e.g.
subparsers = parser.add_subparsers()
p_ab = subparsers.add_parser('ab')
p_ab.add_argument(...)
p_cd = subparsers.add_parser('cd')
p_cd.add_argument(...)
Not ideal, but at least it gives you the good from argparse without too much ugly hackery. I ended up doing away with the switches and just using the subparser operations with required subarguments.
The argparse enhancement request referenced in #hpaulj's comment is still open after more than nine years, so I figured other people might benefit from the workaround I just discovered. Based on this comment in the enhancement request, I found I was able to add an option to two different mutually-exclusive groups using this syntax:
#!/usr/bin/env python
import argparse
import os
import sys
def parse_args():
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
parser.add_argument("-d", "--device", help="Path to UART device", default="./ttyS0")
mutex_group1 = parser.add_mutually_exclusive_group()
mutex_group2 = parser.add_mutually_exclusive_group()
mutex_group1.add_argument(
"-o",
"--output-file",
help="Name of output CSV file",
default="sensor_data_sent.csv",
)
input_file_action = mutex_group1.add_argument(
"-i", "--input-file", type=argparse.FileType("r"), help="Name of input CSV file"
)
# See: https://bugs.python.org/issue10984#msg219660
mutex_group2._group_actions.append(input_file_action)
mutex_group2.add_argument(
"-t",
"--time",
type=int,
help="How long to run, in seconds (-1 = loop forever)",
default=-1,
)
# Add missing ']' to usage message
usage = parser.format_usage()
usage = usage.replace('usage: ', '')
usage = usage.replace(']\n', ']]\n')
parser.usage = usage
return parser.parse_args()
if __name__ == "__main__":
args = parse_args()
print("Args parsed successfully...")
sys.exit(0)
This works well enough for my purposes:
$ ./fake_sensor.py -i input.csv -o output.csv
usage: fake_sensor.py [-h] [-d DEVICE] [-o OUTPUT_FILE | [-i INPUT_FILE | -t TIME]]
fake_sensor.py: error: argument -o/--output-file: not allowed with argument -i/--input-file
$ ./fake_sensor.py -i input.csv -t 30
usage: fake_sensor.py [-h] [-d DEVICE] [-o OUTPUT_FILE | [-i INPUT_FILE | -t TIME]]
fake_sensor.py: error: argument -t/--time: not allowed with argument -i/--input-file
$ ./fake_sensor.py -i input.csv
Args parsed successfully...
$ ./fake_sensor.py -o output.csv
Args parsed successfully...
$ ./fake_sensor.py -o output.csv -t 30
Args parsed successfully...
Accessing private members of argparse is, of course, rather brittle, so I probably wouldn't use this approach in production code. Also, an astute reader may notice that the usage message is misleading, since it implies that -o and -i can be used together when they cannot(!) However, I'm using this script for testing only, so I'm not overly concerned. (Fixing the usage message 'for real' would, I think, require much more time than I can spare for this task, but please comment if you know a clever hack for this.)
Subparsers?
Similar to unhammer's answer, but with more user control. Note: I have not actually tested this method, but it should work in theory and with the capabilities of python.
You can create two parsers, one for each of the two groups, and use conditionals to do the mutually exclusive part. Essentially using argparse for only part of the argument parsing. Using this method, you can go beyond the limitations of unhammer's answer as well.
# Python 3
import argparse
try:
parser = argparse.ArgumentParser()
parser.add_argument('-a')
parser.add_argument('-b')
args = parser.parse_args
except argparse.ArgumentError:
parser = argparse.ArgumentParser()
parser.add_argument('-c')
parser.add_argument('-d')
args = parser.parse_args
I want to write a python code in which, based on some arguments passed from command line I want to make positional argument optional.
For example,
My python program is test.py, and with it I can give --init, --snap, --check options. Now if I have given --snap and --check option, then file name is compulsory i.e.
test.py --snap file1
but if I have given --init option then it should not take any other arguments. i.e. in this case file name is optional:
test.py --init
How to implement this condition
If you are ok with changing the command line you are passing a little bit, then a set of subparsers should work.
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
init_parser = subparsers.add_parser('init', help='do the init stuff')
snap_parser = subparsers.add_parser('snap', help='do the snap stuff')
snap_parser.add_argument('--file', '-f', required=True)
check_parser = subparsers.add_parser('check', help='do the check stuff')
check_parser.add_argument('--file', '-f', required=True)
args = parser.parse_args()
print args
And then the output...
> python foobar.py init
Namespace()
> python foobar.py check
usage: foobar.py check [-h] --file FILE
foobar.py check: error: argument --file/-f is required
> python foobar.py check --file foobar.txt
Namespace(file='foobar.txt')
General help:
> python foobar.py --help
usage: foobar.py [-h] {init,snap,check} ...
positional arguments:
{init,snap,check} sub-command help
init do the init stuff
snap do the snap stuff
check do the check stuff
optional arguments:
-h, --help show this help message and exit
And specific sub-command help
> python foobar.py snap -h
usage: foobar.py snap [-h] --file FILE
optional arguments:
-h, --help show this help message and exit
--file FILE, -f FILE
Your other option is to use nargs as #1.618 has already mentioned.
argparse allows you to specify that certain args have their own args, like so:
parser.add_argument("--snap", nargs=1)
or use a + to allow for an arbitrary number of "subargs"
After you call parse_args(), the values will be in a list:
filename = args.snap[0]
How do I define an arbitrary string as an optional argument using argparse?
Example:
[user#host]$ ./script.py FOOBAR -a -b
Running "script.py"...
You set the option "-a"
You set the option "-b"
You passed the string "FOOBAR"
Ideally, I'd like the position of the arguments to not matter. i.e:
./script.py -a FOOBAR -b == ./script.py -a -b FOOBAR == ./script.py FOOBAR -a -b
In BASH, I can accomplish this while using getopts. After handling all desired switches in a case loop, I'd have a line that reads shift $((OPTIND-1)), and from there I can access all remaining arguments using the standard $1, $2, $3, etc...
Does something like that exisit for argparse?
To get exactly what you're looking for, the trick is to use parse_known_args() instead of parse_args():
#!/bin/env python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', action="store_true")
parser.add_argument('-b', action="store_true")
opts = parser.parse_known_args()
# Print info about flags
if opts[0].a: print('You set the option "-a"')
if opts[0].b: print('You set the option "-b"')
# Collect remainder (opts[1] is a list (possibly empty) of all remaining args)
if opts[1]: print('You passed the strings %s' % opts[1])
EDIT:
The above code displays the following help information:
./clargs.py -h
usage: clargs_old.py [-h] [-a] [-b]
optional arguments:
-h, --help show this help message and exit
-a
-b
If you want to inform the user about the optional arbitrary argument, the only solution I can think of is to subclass ArgumentParser and write it in yourself.
For example:
#!/bin/env python
import os
import argparse
class MyParser(argparse.ArgumentParser):
def format_help(self):
help = super(MyParser, self).format_help()
helplines = help.splitlines()
helplines[0] += ' [FOO]'
helplines.append(' FOO some description of FOO')
helplines.append('') # Just a trick to force a linesep at the end
return os.linesep.join(helplines)
parser = MyParser()
parser.add_argument('-a', action="store_true")
parser.add_argument('-b', action="store_true")
opts = parser.parse_known_args()
# Print info about flags
if opts[0].a: print('You set the option "-a"')
if opts[0].b: print('You set the option "-b"')
# Collect remainder
if opts[1]: print('You passed the strings %s' % opts[1])
Which displays the following help information:
./clargs.py -h
usage: clargs.py [-h] [-a] [-b] [FOO]
optional arguments:
-h, --help show this help message and exit
-a
-b
FOO some description of FOO
Note the addition of the [FOO] in the "usage" line and the FOO in the help under "optional arguments".
I would like use argparse to parse the arguments that it knows and then leave the rest untouched. For example I want to be able to run
performance -o output other_script.py -a opt1 -b opt2
Which uses the -o option and leaves the rest untouched.
The module profiler.py does a similar thing with optparse, but since I'm using argparse I'm doing:
def parse_arguments():
parser = new_argument_parser('show the performance of the given run script')
parser.add_argument('-o', '--output', default='profiled.prof')
return parser.parse_known_args()
def main():
progname = sys.argv[1]
ns, other_args = parse_arguments()
sys.argv[:] = other_args
Which also seems to work, but what happens if also other_script.py also has a -o flag?
Is there in general a better way to solve this problem?
You could also add a positional argument to your parser with nargs=argparse.REMAINDER, to capture the script and its options:
# In script 'performance'...
p = argparse.ArgumentParser()
p.add_argument("-o")
p.add_argument("command", nargs=argparse.REMAINDER)
args = p.parse_args()
print args
Running the above minimal script...
$ performance -o output other_script.py -a opt1 -b opt2
Namespace(command=['performance', '-a', 'opt1', '-b', 'opt2'], o='output')
argparse will stop to parse argument until EOF or --. If you want to have argument without beeing parsed by argparse, you can write::
python [PYTHONOPTS] yourfile.py [YOURFILEOPT] -- [ANYTHINGELSE]
If I have the arguments '-a', '-b', '-c', '-d', with the add_mutually_exclusive_group() function my program will have to use just one of them. Is there a way to combine that, so that the program will accept only either '-a 999 -b 999' or '-c 999 -d 999'?
Edit: adding a simple program for more clarity:
>>> parser = argparse.ArgumentParser()
>>> group = parser.add_mutually_exclusive_group()
>>> group.add_argument('-a')
>>> group.add_argument('-b')
>>> group.add_argument('-c')
>>> group.add_argument('-d')
Then only ./app.py -a | ./app.py -b | ./app.py -c | ./app.py -d can be called. Is it possible to have argparse group the exclusion groups, so that only ./app.py -a .. -b .. | ./app.py -c .. -d .. be called?
EDIT: Never mind. Because argparse makes the horrible choice of having to create an option when invoking group.add_argument. That wouldn't be my design choice. If you're desperate for this feature, you can try doing it with ConflictsOptionParser:
# exclusivegroups.py
import conflictsparse
parser = conflictsparse.ConflictsOptionParser()
a_opt = parser.add_option('-a')
b_opt = parser.add_option('-b')
c_opt = parser.add_option('-c')
d_opt = parser.add_option('-d')
import itertools
compatible_opts1 = (a_opt, b_opt)
compatible_opts2 = (c_opt, d_opt)
exclusives = itertools.product(compatible_opts1, compatible_opts2)
for exclusive_grp in exclusives:
parser.register_conflict(exclusive_grp)
opts, args = parser.parse_args()
print "opts: ", opts
print "args: ", args
Thus when we invoke it, we can see we get the desired effect.
$ python exclusivegroups.py -a 1 -b 2
opts: {'a': '1', 'c': None, 'b': '2', 'd': None}
args: []
$ python exclusivegroups.py -c 3 -d 2
opts: {'a': None, 'c': '3', 'b': None, 'd': '2'}
args: []
$ python exclusivegroups.py -a 1 -b 2 -c 3
Usage: exclusivegroups.py [options]
exclusivegroups.py: error: -b, -c are incompatible options.
The warning message doesn't inform you that both '-a' and '-b' are incompatible with '-c', however a more appropriate error message could be crafted. Older, wrong answer below.
OLDER EDIT: [This edit is wrong, although wouldn't it be just a perfect world if argparse worked this way?] My previous answer actually was incorrect, you should be able to do this with argparse by specifying one group per mutually exclusive options. We can even use itertools to generalize the process. And make it so we don't have to type out all the combinations explicitly:
import itertools
compatible_opts1 = ('-a', '-b')
compatible_opts2 = ('-c', '-d')
exclusives = itertools.product(compatible_opts1, compatible_opts2)
for exclusive_grp in exclusives:
group = parser.add_mutually_exclusive_group()
group.add_argument(exclusive_grp[0])
group.add_argument(exclusive_grp[1])
Just stumbled on this problem myself. From my reading of the argparse docs, there doesn't seem to be a simple way to achieve that within argparse. I considered using parse_known_args, but that soon amounts to writing a special-purpose version of argparse ;-)
Perhaps a bug report is in order. In the meanwhile, if you're willing to make your user do a tiny bit extra typing, you can fake it with subgroups (like how git and svn's arguments work), e.g.
subparsers = parser.add_subparsers()
p_ab = subparsers.add_parser('ab')
p_ab.add_argument(...)
p_cd = subparsers.add_parser('cd')
p_cd.add_argument(...)
Not ideal, but at least it gives you the good from argparse without too much ugly hackery. I ended up doing away with the switches and just using the subparser operations with required subarguments.
The argparse enhancement request referenced in #hpaulj's comment is still open after more than nine years, so I figured other people might benefit from the workaround I just discovered. Based on this comment in the enhancement request, I found I was able to add an option to two different mutually-exclusive groups using this syntax:
#!/usr/bin/env python
import argparse
import os
import sys
def parse_args():
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
parser.add_argument("-d", "--device", help="Path to UART device", default="./ttyS0")
mutex_group1 = parser.add_mutually_exclusive_group()
mutex_group2 = parser.add_mutually_exclusive_group()
mutex_group1.add_argument(
"-o",
"--output-file",
help="Name of output CSV file",
default="sensor_data_sent.csv",
)
input_file_action = mutex_group1.add_argument(
"-i", "--input-file", type=argparse.FileType("r"), help="Name of input CSV file"
)
# See: https://bugs.python.org/issue10984#msg219660
mutex_group2._group_actions.append(input_file_action)
mutex_group2.add_argument(
"-t",
"--time",
type=int,
help="How long to run, in seconds (-1 = loop forever)",
default=-1,
)
# Add missing ']' to usage message
usage = parser.format_usage()
usage = usage.replace('usage: ', '')
usage = usage.replace(']\n', ']]\n')
parser.usage = usage
return parser.parse_args()
if __name__ == "__main__":
args = parse_args()
print("Args parsed successfully...")
sys.exit(0)
This works well enough for my purposes:
$ ./fake_sensor.py -i input.csv -o output.csv
usage: fake_sensor.py [-h] [-d DEVICE] [-o OUTPUT_FILE | [-i INPUT_FILE | -t TIME]]
fake_sensor.py: error: argument -o/--output-file: not allowed with argument -i/--input-file
$ ./fake_sensor.py -i input.csv -t 30
usage: fake_sensor.py [-h] [-d DEVICE] [-o OUTPUT_FILE | [-i INPUT_FILE | -t TIME]]
fake_sensor.py: error: argument -t/--time: not allowed with argument -i/--input-file
$ ./fake_sensor.py -i input.csv
Args parsed successfully...
$ ./fake_sensor.py -o output.csv
Args parsed successfully...
$ ./fake_sensor.py -o output.csv -t 30
Args parsed successfully...
Accessing private members of argparse is, of course, rather brittle, so I probably wouldn't use this approach in production code. Also, an astute reader may notice that the usage message is misleading, since it implies that -o and -i can be used together when they cannot(!) However, I'm using this script for testing only, so I'm not overly concerned. (Fixing the usage message 'for real' would, I think, require much more time than I can spare for this task, but please comment if you know a clever hack for this.)
Subparsers?
Similar to unhammer's answer, but with more user control. Note: I have not actually tested this method, but it should work in theory and with the capabilities of python.
You can create two parsers, one for each of the two groups, and use conditionals to do the mutually exclusive part. Essentially using argparse for only part of the argument parsing. Using this method, you can go beyond the limitations of unhammer's answer as well.
# Python 3
import argparse
try:
parser = argparse.ArgumentParser()
parser.add_argument('-a')
parser.add_argument('-b')
args = parser.parse_args
except argparse.ArgumentError:
parser = argparse.ArgumentParser()
parser.add_argument('-c')
parser.add_argument('-d')
args = parser.parse_args