I was learning to handle command line arguments in Python with argparse. While not mutually exclusive arguments can be passed as variables, it is not clear to me how to do the same for mutually exclusive arguments. In the following example, I'd like to print out all the arguments. First 2 is easy. However the third one is tricky, because '-a' and '-b' have different names of destination. Therefore the last 2 lines cannot exist in the code at the same time.
#/usr/bin/env python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-t' dest='thread', help='gtdownload thread', default=4, type=int)
parser.add_argument('-n' dest='number', help='number of downloads', default=1, type=int)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-a', dest='analysis', help='analysis ID')
group.add_argument('-b', dest='barcode', help='barcode')
args = parser.parser_args()
print args.thread
print args.number
#???? how to print out mutually exclusive argument
print args.analysis
print args.barcode
Most of the tutorials about add_mutually_exlusive_group out there stop at parser.parser_args() and never say what to do with the mutually exclusive arguments afterwards. But it is very important to know how exactly can the mutually exclusive arguments be passed to the rest of the code.
if args.analysis is not None:
print args.analysis
if args.barcode is not None:
print args.barcode
By putting -a and -b in the group, all you are telling the parser is to raise an error if you use both options in the command line.
Print args, and you will see that both attributes are present in the Namespace. The group just ensures that one will have its default value (None). The other will have the value you gave in the command line. Otherwise those attributes are just like the other ones.
print args # a very useful statement when debugging argparse
(The group also affects the usage display).
You'd have to use default=argparse.SUPPRESS to keep an attribute out of the Namespace (unless given in the commandline).
Related
I have done as much research as possible but I haven't found the best way to make certain cmdline arguments necessary only under certain conditions, in this case only if other arguments have been given. Here's what I want to do at a very basic level:
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required=False) # only required if --argument is given
p.add_argument('-b', required=False) # only required if --argument is given
From what I have seen, other people seem to just add their own check at the end:
if args.argument and (args.a is None or args.b is None):
# raise argparse error here
Is there a way to do this natively within the argparse package?
I've been searching for a simple answer to this kind of question for some time. All you need to do is check if '--argument' is in sys.argv, so basically for your code sample you could just do:
import argparse
import sys
if __name__ == '__main__':
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required='--argument' in sys.argv) #only required if --argument is given
p.add_argument('-b', required='--argument' in sys.argv) #only required if --argument is given
args = p.parse_args()
This way required receives either True or False depending on whether the user as used --argument. Already tested it, seems to work and guarantees that -a and -b have an independent behavior between each other.
You can implement a check by providing a custom action for --argument, which will take an additional keyword argument to specify which other action(s) should become required if --argument is used.
import argparse
class CondAction(argparse.Action):
def __init__(self, option_strings, dest, nargs=None, **kwargs):
x = kwargs.pop('to_be_required', [])
super(CondAction, self).__init__(option_strings, dest, **kwargs)
self.make_required = x
def __call__(self, parser, namespace, values, option_string=None):
for x in self.make_required:
x.required = True
try:
return super(CondAction, self).__call__(parser, namespace, values, option_string)
except NotImplementedError:
pass
p = argparse.ArgumentParser()
x = p.add_argument("--a")
p.add_argument("--argument", action=CondAction, to_be_required=[x])
The exact definition of CondAction will depend on what, exactly, --argument should do. But, for example, if --argument is a regular, take-one-argument-and-save-it type of action, then just inheriting from argparse._StoreAction should be sufficient.
In the example parser, we save a reference to the --a option inside the --argument option, and when --argument is seen on the command line, it sets the required flag on --a to True. Once all the options are processed, argparse verifies that any option marked as required has been set.
Your post parsing test is fine, especially if testing for defaults with is None suits your needs.
http://bugs.python.org/issue11588 'Add "necessarily inclusive" groups to argparse' looks into implementing tests like this using the groups mechanism (a generalization of mutuall_exclusive_groups).
I've written a set of UsageGroups that implement tests like xor (mutually exclusive), and, or, and not. I thought those where comprehensive, but I haven't been able to express your case in terms of those operations. (looks like I need nand - not and, see below)
This script uses a custom Test class, that essentially implements your post-parsing test. seen_actions is a list of Actions that the parse has seen.
class Test(argparse.UsageGroup):
def _add_test(self):
self.usage = '(if --argument then -a and -b are required)'
def testfn(parser, seen_actions, *vargs, **kwargs):
"custom error"
actions = self._group_actions
if actions[0] in seen_actions:
if actions[1] not in seen_actions or actions[2] not in seen_actions:
msg = '%s - 2nd and 3rd required with 1st'
self.raise_error(parser, msg)
return True
self.testfn = testfn
self.dest = 'Test'
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind=Test)
g1.add_argument('--argument')
g1.add_argument('-a')
g1.add_argument('-b')
print(p.parse_args())
Sample output is:
1646:~/mypy/argdev/usage_groups$ python3 issue25626109.py --arg=1 -a1
usage: issue25626109.py [-h] [--argument ARGUMENT] [-a A] [-b B]
(if --argument then -a and -b are required)
issue25626109.py: error: group Test: argument, a, b - 2nd and 3rd required with 1st
usage and error messages still need work. And it doesn't do anything that post-parsing test can't.
Your test raises an error if (argument & (!a or !b)). Conversely, what is allowed is !(argument & (!a or !b)) = !(argument & !(a and b)). By adding a nand test to my UsageGroup classes, I can implement your case as:
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
arg = g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b')
The usage is (using !() to mark a 'nand' test):
usage: issue25626109.py [-h] !(--arg C & !(-a A & -b B))
I think this is the shortest and clearest way of expressing this problem using general purpose usage groups.
In my tests, inputs that parse successfully are:
''
'-a1'
'-a1 -b2'
'--arg=3 -a1 -b2'
Ones that are supposed to raise errors are:
'--arg=3'
'--arg=3 -a1'
'--arg=3 -b2'
For arguments I've come up with a quick-n-dirty solution like this.
Assumptions: (1) '--help' should display help and not complain about required argument and (2) we're parsing sys.argv
p = argparse.ArgumentParser(...)
p.add_argument('-required', ..., required = '--help' not in sys.argv )
This can easily be modified to match a specific setting.
For required positionals (which will become unrequired if e.g. '--help' is given on the command line) I've come up with the following: [positionals do not allow for a required=... keyword arg!]
p.add_argument('pattern', ..., narg = '+' if '--help' not in sys.argv else '*' )
basically this turns the number of required occurrences of 'pattern' on the command line from one-or-more into zero-or-more in case '--help' is specified.
Here is a simple and clean solution with these advantages:
No ambiguity and loss of functionality caused by oversimplified parsing using the in sys.argv test.
No need to implement a special argparse.Action or argparse.UsageGroup class.
Simple usage even for multiple and complex deciding arguments.
I noticed just one considerable drawback (which some may find desirable): The help text changes according to the state of the deciding arguments.
The idea is to use argparse twice:
Parse the deciding arguments instead of the oversimplified use of the in sys.argv test. For this we use a short parser not showing help and the method .parse_known_args() which ignores unknown arguments.
Parse everything normally while reusing the parser from the first step as a parent and having the results from the first parser available.
import argparse
# First parse the deciding arguments.
deciding_args_parser = argparse.ArgumentParser(add_help=False)
deciding_args_parser.add_argument(
'--argument', required=False, action='store_true')
deciding_args, _ = deciding_args_parser.parse_known_args()
# Create the main parser with the knowledge of the deciding arguments.
parser = argparse.ArgumentParser(
description='...', parents=[deciding_args_parser])
parser.add_argument('-a', required=deciding_args.argument)
parser.add_argument('-b', required=deciding_args.argument)
arguments = parser.parse_args()
print(arguments)
Until http://bugs.python.org/issue11588 is solved, I'd just use nargs:
p = argparse.ArgumentParser(description='...')
p.add_argument('--arguments', required=False, nargs=2, metavar=('A', 'B'))
This way, if anybody supplies --arguments, it will have 2 values.
Maybe its CLI result is less readable, but code is much smaller. You can fix that with good docs/help.
This is really the same as #Mira 's answer but I wanted to show it for the case where when an option is given that an extra arg is required:
For instance, if --option foo is given then some args are also required that are not required if --option bar is given:
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--option', required=True,
help='foo and bar need different args')
if 'foo' in sys.argv:
parser.add_argument('--foo_opt1', required=True,
help='--option foo requires "--foo_opt1"')
parser.add_argument('--foo_opt2', required=True,
help='--option foo requires "--foo_opt2"')
...
if 'bar' in sys.argv:
parser.add_argument('--bar_opt', required=True,
help='--option bar requires "--bar_opt"')
...
It's not perfect - for instance proggy --option foo --foo_opt1 bar is ambiguous but for what I needed to do its ok.
Add additional simple "pre"parser to check --argument, but use parse_known_args() .
pre = argparse.ArgumentParser()
pre.add_argument('--argument', required=False, action='store_true', default=False)
args_pre=pre.parse_known_args()
p = argparse.ArgumentParser()
p.add_argument('--argument', required=False)
p.add_argument('-a', required=args_pre.argument)
p.add_argument('-b', required=not args_pre.argument)
I have this very simple ArgumentParser instance, with an optional positional argument and an option, which writes a constant to the same destination:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', action='store_const', dest='path', const='<all>')
parser.add_argument('path', nargs='?')
# Prints None instead of '<all>'
print(parser.parse_args(['-a']).path)
But no matter what, parsing the command line ['-a'] does not yield a Namespace instance with path set to that constant. Instead, the default from the positional argument is used.
What am I doing wrong?
My use case is that the user should be able to specify a path (actually a list of paths). This list of paths defaults to the current working directory. But instead of using that default, -a can be passed, which should result in some configured root directory to be used. The full code for this part of the argument parser is this:
all_sentinel = object()
parser = argparse.ArgumentParser()
paths_group = parser.add_mutually_exclusive_group()
paths_group.add_argument('-a', action='store_const', dest='paths', const=all_sentinel)
paths_group.add_argument('paths', nargs='*', default=['.'])
A positional with nargs='?' has some special handling of its default (here None).
Normally defaults are assigned to the Namespace at the start of parsing, and overwritten by the actions, such as the optional.
Because an empty list of values satisfies the nargs, that positional is always 'seen'. But rather than assign [] or some other 'blank' to it, the parser assigns the default. So the positional's default overwrites the value set by '-a'.
nargs='*' gets the same kind of special handling.
I suspect that if you had another positional argument before the '-a', that you wouldn't see this effect. The '?*' positional would be processed before the '-a', and not overwrite its value.
Optionals are only processed if the flag occurs. Positionals are always processed, regardless of the nargs. The 'optional' positionals are processed, but with some extra handling of the defaults. But when they are processed relative to flagged arguments can vary.
That's some tricky behavior that I'm aware of simply because I've studied the code in detail, and answered a lot questions here and on the Python bug/issues.
Sharing the dest often does work, but that's more by default than design. It's the result of other design choices. argparse makes no promises in that regard. So if it isn't reliable, don't use it.
I'm trying to disable same argument occurences within one command line, using argparse
./python3 --argument1=something --argument2 --argument1=something_else
which means this should raise an error, because value of argument1 is overriden, by default, argparse just overrides the value and continues like nothing happened... Is there any smart way how to disable this behaviour?
I don't think there is a native way to do it using argparse, but fortunately, argparse offers methods to report custom errors. The most elegant way is probably to define a custom action that checks for duplicates (and exits if there are).
class UniqueStore(argparse.Action):
def __call__(self, parser, namespace, values, option_string):
if getattr(namespace, self.dest, self.default) is not self.default:
parser.error(option_string + " appears several times.")
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--foo', action=UniqueStore)
args = parser.parse_args()
(Read the docs about cutom actions)
Another way is to use the append action and count the len of the list.
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--foo', action='append')
args = parser.parse_args()
if len(args.foo) > 1:
parser.error("--foo appears several times.")
There's no built in test or constraint. A positional argument will be handled only once, but the flagged (or optional) ones can, as you say, be repeated. This lets you collect multiple occurrences with append or count actions.
The override action is acceptable to most people. Why might your user use the option more than once? Why should the first be preferred over the last?
A custom Action may be the best choice. It could raise an error if the namespace[dest] already has a non-default value. Or this Action could add some other 'repeat' flag to the namespace.
In python argparse, is it possible to declare an argument which is just single-valued, instead a list, but allows being specified multiple times, the latest one overrides the earlier ones?
The use case is, I am writing a Python program that reads command line argument from a ~/.xxxrc file, where .xxxrc file has an command line argument per line. I want to allow user override the value in ~/.xxxrc file through command line. My plan is to implicitly adds an #~/.xxxrc to the beginning of argv before passing it to argparse, so that argparse will reads in the file. Now the problem turns into my question above.
Is it possible to achieve this effect? If not, is there any alternative solution to my use case.
The argparse module does that by default. Take a look at this example, where we specify the same flag twice, and the last one wins:
import argparse
parser = argparse.ArgumentParser(description='example')
parser.add_argument('-a', '--add')
options = parser.parse_args(['-a', 'foo', '-a', 'bar'])
print 'add = {}'.format(options.add) # output: add = bar
Yes, you can just create a custom action with an nargs='*' or nargs='+'.
Something like this should work:
class GetLastAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if values:
setattr(namespace, self.dest, values[-1])
parser = argparse.ArgumentParser()
parser.add_argument('-a', nargs='+', action=GetLastAction)
I would like to implement logic like this in argparse:
If argument A is selected, the user cannot select arguments B or C.
B and C can both be selected
It looks like add_mutually_exclusive_group is what I would want for this, but it looks like you can only choose one option from a mutually exclusive group, so I cannot put all three in a mutually exclusive group.
Is there a way to do this in argparse?
You could not really do it with argparse, however you can do it after argparse has run.
Here is an example:
parser = argparse.ArgumentParser()
# group 1
parser.add_argument("-q", "--query", help="query", required=False)
parser.add_argument("-f", "--fields", help="field names", required=False)
# group 2
parser.add_argument("-a", "--aggregation", help="aggregation",
required=False)
I am using here options given to a command line wrapper for querying a mongodb. The collection instance can either call the method aggregate or the method find with to optional arguments query and fields, hence you see why the first two arguments are compatible and the last one isn't.
So now I run parser.parse_args() and check it's content:
args = parser().parse_args()
print args.aggregation
if args.aggregation and (args.query or args.fields):
print "-a and -q|-f are mutually exclusive ..."
sys.exit(2)
Of course, this little hack is only working for simple cases and it would become a nightmare to check all the possible options if you have many mutually exclusive options and groups. In that case you should break your options in to command groups. For that, you should follow the suggestion here Python argparse mutual exclusive group.
Docopt could work in this instance.
It uses pipes for mutually exclusive elements.
my_program (aggregate | find [-q | -f])
You could negate the meaning of A and then use an subparser. The subparsers allows you to specify that "If and only if A is selected, user can select B or C."
http://docs.python.org/2/library/argparse.html