Related
I have done as much research as possible but I haven't found the best way to make certain cmdline arguments necessary only under certain conditions, in this case only if other arguments have been given. Here's what I want to do at a very basic level:
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required=False) # only required if --argument is given
p.add_argument('-b', required=False) # only required if --argument is given
From what I have seen, other people seem to just add their own check at the end:
if args.argument and (args.a is None or args.b is None):
# raise argparse error here
Is there a way to do this natively within the argparse package?
I've been searching for a simple answer to this kind of question for some time. All you need to do is check if '--argument' is in sys.argv, so basically for your code sample you could just do:
import argparse
import sys
if __name__ == '__main__':
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required='--argument' in sys.argv) #only required if --argument is given
p.add_argument('-b', required='--argument' in sys.argv) #only required if --argument is given
args = p.parse_args()
This way required receives either True or False depending on whether the user as used --argument. Already tested it, seems to work and guarantees that -a and -b have an independent behavior between each other.
You can implement a check by providing a custom action for --argument, which will take an additional keyword argument to specify which other action(s) should become required if --argument is used.
import argparse
class CondAction(argparse.Action):
def __init__(self, option_strings, dest, nargs=None, **kwargs):
x = kwargs.pop('to_be_required', [])
super(CondAction, self).__init__(option_strings, dest, **kwargs)
self.make_required = x
def __call__(self, parser, namespace, values, option_string=None):
for x in self.make_required:
x.required = True
try:
return super(CondAction, self).__call__(parser, namespace, values, option_string)
except NotImplementedError:
pass
p = argparse.ArgumentParser()
x = p.add_argument("--a")
p.add_argument("--argument", action=CondAction, to_be_required=[x])
The exact definition of CondAction will depend on what, exactly, --argument should do. But, for example, if --argument is a regular, take-one-argument-and-save-it type of action, then just inheriting from argparse._StoreAction should be sufficient.
In the example parser, we save a reference to the --a option inside the --argument option, and when --argument is seen on the command line, it sets the required flag on --a to True. Once all the options are processed, argparse verifies that any option marked as required has been set.
Your post parsing test is fine, especially if testing for defaults with is None suits your needs.
http://bugs.python.org/issue11588 'Add "necessarily inclusive" groups to argparse' looks into implementing tests like this using the groups mechanism (a generalization of mutuall_exclusive_groups).
I've written a set of UsageGroups that implement tests like xor (mutually exclusive), and, or, and not. I thought those where comprehensive, but I haven't been able to express your case in terms of those operations. (looks like I need nand - not and, see below)
This script uses a custom Test class, that essentially implements your post-parsing test. seen_actions is a list of Actions that the parse has seen.
class Test(argparse.UsageGroup):
def _add_test(self):
self.usage = '(if --argument then -a and -b are required)'
def testfn(parser, seen_actions, *vargs, **kwargs):
"custom error"
actions = self._group_actions
if actions[0] in seen_actions:
if actions[1] not in seen_actions or actions[2] not in seen_actions:
msg = '%s - 2nd and 3rd required with 1st'
self.raise_error(parser, msg)
return True
self.testfn = testfn
self.dest = 'Test'
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind=Test)
g1.add_argument('--argument')
g1.add_argument('-a')
g1.add_argument('-b')
print(p.parse_args())
Sample output is:
1646:~/mypy/argdev/usage_groups$ python3 issue25626109.py --arg=1 -a1
usage: issue25626109.py [-h] [--argument ARGUMENT] [-a A] [-b B]
(if --argument then -a and -b are required)
issue25626109.py: error: group Test: argument, a, b - 2nd and 3rd required with 1st
usage and error messages still need work. And it doesn't do anything that post-parsing test can't.
Your test raises an error if (argument & (!a or !b)). Conversely, what is allowed is !(argument & (!a or !b)) = !(argument & !(a and b)). By adding a nand test to my UsageGroup classes, I can implement your case as:
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
arg = g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b')
The usage is (using !() to mark a 'nand' test):
usage: issue25626109.py [-h] !(--arg C & !(-a A & -b B))
I think this is the shortest and clearest way of expressing this problem using general purpose usage groups.
In my tests, inputs that parse successfully are:
''
'-a1'
'-a1 -b2'
'--arg=3 -a1 -b2'
Ones that are supposed to raise errors are:
'--arg=3'
'--arg=3 -a1'
'--arg=3 -b2'
For arguments I've come up with a quick-n-dirty solution like this.
Assumptions: (1) '--help' should display help and not complain about required argument and (2) we're parsing sys.argv
p = argparse.ArgumentParser(...)
p.add_argument('-required', ..., required = '--help' not in sys.argv )
This can easily be modified to match a specific setting.
For required positionals (which will become unrequired if e.g. '--help' is given on the command line) I've come up with the following: [positionals do not allow for a required=... keyword arg!]
p.add_argument('pattern', ..., narg = '+' if '--help' not in sys.argv else '*' )
basically this turns the number of required occurrences of 'pattern' on the command line from one-or-more into zero-or-more in case '--help' is specified.
Here is a simple and clean solution with these advantages:
No ambiguity and loss of functionality caused by oversimplified parsing using the in sys.argv test.
No need to implement a special argparse.Action or argparse.UsageGroup class.
Simple usage even for multiple and complex deciding arguments.
I noticed just one considerable drawback (which some may find desirable): The help text changes according to the state of the deciding arguments.
The idea is to use argparse twice:
Parse the deciding arguments instead of the oversimplified use of the in sys.argv test. For this we use a short parser not showing help and the method .parse_known_args() which ignores unknown arguments.
Parse everything normally while reusing the parser from the first step as a parent and having the results from the first parser available.
import argparse
# First parse the deciding arguments.
deciding_args_parser = argparse.ArgumentParser(add_help=False)
deciding_args_parser.add_argument(
'--argument', required=False, action='store_true')
deciding_args, _ = deciding_args_parser.parse_known_args()
# Create the main parser with the knowledge of the deciding arguments.
parser = argparse.ArgumentParser(
description='...', parents=[deciding_args_parser])
parser.add_argument('-a', required=deciding_args.argument)
parser.add_argument('-b', required=deciding_args.argument)
arguments = parser.parse_args()
print(arguments)
Until http://bugs.python.org/issue11588 is solved, I'd just use nargs:
p = argparse.ArgumentParser(description='...')
p.add_argument('--arguments', required=False, nargs=2, metavar=('A', 'B'))
This way, if anybody supplies --arguments, it will have 2 values.
Maybe its CLI result is less readable, but code is much smaller. You can fix that with good docs/help.
This is really the same as #Mira 's answer but I wanted to show it for the case where when an option is given that an extra arg is required:
For instance, if --option foo is given then some args are also required that are not required if --option bar is given:
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--option', required=True,
help='foo and bar need different args')
if 'foo' in sys.argv:
parser.add_argument('--foo_opt1', required=True,
help='--option foo requires "--foo_opt1"')
parser.add_argument('--foo_opt2', required=True,
help='--option foo requires "--foo_opt2"')
...
if 'bar' in sys.argv:
parser.add_argument('--bar_opt', required=True,
help='--option bar requires "--bar_opt"')
...
It's not perfect - for instance proggy --option foo --foo_opt1 bar is ambiguous but for what I needed to do its ok.
Add additional simple "pre"parser to check --argument, but use parse_known_args() .
pre = argparse.ArgumentParser()
pre.add_argument('--argument', required=False, action='store_true', default=False)
args_pre=pre.parse_known_args()
p = argparse.ArgumentParser()
p.add_argument('--argument', required=False)
p.add_argument('-a', required=args_pre.argument)
p.add_argument('-b', required=not args_pre.argument)
Right now, I have a script that can accept command line arguments using argparse. For example, like this:
#foo.py
def function_with_args(optional_args=None):
parser = argparse.ArgumentParser()
# add some arguments
args = parser.parse_args(optional_args)
# do something with args
However, I'd like to be able to use this function with a dictionary instead, for example with something like this:
def function_using_dict(**kwargs):
# define parser and add some arguments
args = parser.parse_dict_args(kwargs)
# everything else is the same
Note that I have a lot of arguments with default values in argparse which I'd like to use, so the following wouldn't work:
def function_no_default_args(**kwargs):
args = kwargs # not using default values that we add to the parser!
argparse.Namespace is a relatively simple object subclass, with most of its code devoted to displaying the attributes (as print(args) shows). Internally parse_args uses get_attr and set_attr to access the namespace, minimizing the assumptions about attributes names.
When using subparsers, the subparser starts with a 'blank' namespace, and uses the following code to copy its values to the main namespace.
# In case this subparser defines new defaults, we parse them
# in a new namespace object and then update the original
# namespace for the relevant parts.
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
for key, value in vars(subnamespace).items():
setattr(namespace, key, value)
Originally the main namespace was passed to the subparser, eg. parser.parse_known_args(arg_strings, namespace), but the current version lets the subparser defaults take priority.
Handling defaults is a bit complicated. If you don't have any required arguments then
args = parser.parse_args([])
will set all the defaults. Or you could look at the start of parse.parse_known_args to see how defaults are inserted into the namespace at the start of parsing. Just beware that there's an added step at the end of parsing that runs remaining defaults through their respective type functions.
If you are trying to convert the result of parse_args into a dict, you can probably just do this:
kwargs = vars(args)
After your comment, I thought about it. Going to go with your existing function.
#foo.py
def function_with_args_and_default_kwargs(optional_args=None, **kwargs):
parser = argparse.ArgumentParser()
# add some arguments
# add the other arguments
for k, v in kwargs.items():
parser.add_argument('--' + k, default=v)
args = parser.parse_args(optional_args)
# do something with args
I have an argparse program with multiple subparsers. IF the value of a top-level argument is not specified, I would like it to depend on the subparser selection. This can be partially accomplished by using set_defaults() on each subparser. However, the passed value does NOT act like a true default in that it takes precedence even when the top-level argument is explicitly provided.
Am I missing something in the argparse module that would allow the subparser-specific defaults to yield to the top-level parser? Below is a toy example which demonstrates my desired use case
import argparse
# Top-level parser
parser = argparse.ArgumentParser()
parser.add_argument('-t', '--time', type=str)
# Add subparsers with options
actions = parser.add_subparsers(dest='action', metavar='action')
actions.required = True
eat = actions.add_parser('eat')
eat.add_argument('-f', '--food', type=str, default='pizza')
order = actions.add_parser('order')
order.add_argument('-i', '--item', type=int, required=True)
# Attempt to set main parser's --time depending on subparser selection
eat.set_defaults(time='now')
order.set_defaults(time='later')
# Tests that work as expected
print(parser.parse_args(['order', '--item', 0])) # Namespace(action='order', item=0, time='later')
print(parser.parse_args(['eat'])) # Namespace(action='eat', food='pizza', time='now')
# Tests that DO NOT work
# Actual result: the --time flag of `parser` is ignored in preference of each subparser's default
# Desired result: explicit usage of --time should override the subparser defaults like the comments below
# Namespace(action='order', item=0, time='before')
# Namespace(action='eat', food='pizza', time='after')
print(parser.parse_args(['--time', 'before', 'order', '--item', 0]))
print(parser.parse_args(['--time', 'after', 'eat']))
It seems that the sub-parser args, overwrite the Namespace
entries created from the top-level parser, because they have
the same name. So the Namespace ends up with only one 'time'
entry (instead of one for the top-level parser, and one for
the sub-parser); its value is the last value written to it:
that of the sub-parser.
Alternatively, maybe only use the arg in the main parser,
and alter it's default value (in the main parser),
before using a specific subparser?
import argparse
parserMain = argparse.ArgumentParser()
parserMain.add_argument('--arg', type=str)
subparsers = parserMain.add_subparsers()
parserX = subparsers.add_parser('x')
# This would overwrite the value of 'arg' in the Namespace:
#parserX.set_defaults(arg='defaultValForX')
# So, set default in main parser, before using sub-parser:
parserMain.set_defaults(arg='defaultValForX')
print(parserMain.parse_args(['x']))
print(parserMain.parse_args(['--arg', 'explicitVal', 'x']))
parserY = subparsers.add_parser('y')
# This would overwrite the value of 'arg' in the Namespace:
#parserY.set_defaults(arg='defaultValForY')
# So, set default in main parser, before using sub-parser:
parserMain.set_defaults(arg='defaultValForY')
print(parserMain.parse_args(['y']))
print(parserMain.parse_args(['--arg', 'explicitVal', 'y']))
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--pattern", help="Pattern file")
args = parser.parse_args()
Now is it possible to get back the string "--pattern" from args?
I need the string so that I can construct a cmd list to pass to Popen like Popen(['some_other_program', args.pattern.option_string, args.pattern], ...) without repeating it (and having to maintain it in two places) (Popen(['some_other_prog', '--pattern', args.pattern], ...)).
I need to create a wrapper for another program. Some of the args need to be passed to the wrapped program (via Popen) and some are required by the wrapper.
Is there a better method than the following example?
pass_1 = '--to-be-passed'
parser = argparse.ArgumentParser()
parser.add_argument("-p", pass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
...
process = Popen(['wrapped_program', pass_1, args.pass_1], ...)
...
This method of keeping the args in variables is not very good as:
Maintaining short options along with long options becomes difficult.
Popen if called in another function requires passing these variables(or a dict of them) to the function. This seems redundant as args passed to it should be sufficient.
Add a dest to your add_argument call.
parser.add_argmument("p", "--pattern", dest="pattern", help="your help text")
args = parser.parse_args()
args = vars(args)
The you can reference the pattern with args["pattern"] .
There doesn't seem to be an easy way to get the original option strings from the result of a parser.parse_args(), but you can get them from the parser object. You just need to peek into its __dict__, in order to retrieve the parser settings after it's created. In your case you want the _option_string_actions field. Unfortunately this doesn't seem officially supported, as I couldn't find a ArgumentParser method dedicated to this, so YMMV. On Python 3:
Demo:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=int, default=1000, help='intensity of bar')
parser.add_argument('--bar', '-b', type=str, default='bla', help='whatever')
store_action_dict=vars(parser)['_option_string_actions']
print(store_action_dict.keys()) # dict_keys(['--help', '-b', '-f', '-h', '--foo', '--bar'])
The deleted answers and comments indicate there is some confusion as to what you want. So I'll add to that confusion.
Normally the parser does not record the option string. However it is provided to the Action __call__ method. So a custom Action class could save it. The FooAction custom class example in the argparse docs illustrates this.
If I define this action subclass:
In [324]: class PassThru(argparse._StoreAction):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, [values, option_string])
In [324]: p.add_argument('-o','--other',action=PassThru)
The option string is recorded along with the value ('-o' or '--other'):
In [322]: p.parse_args('-p test -o teseting'.split())
Out[322]: Namespace(other=['teseting', '-o'], pass_me_on='test')
In [323]: p.parse_args('-p test --other teseting'.split())
Out[323]: Namespace(other=['teseting', '--other'], pass_me_on='test')
Obviously the option_string and value could be recorded in a different order, in a dictionary, as seperate attributes in the Namespace, etc.
There are other ways of passing options to another program, particularly if the wrapping parser does not need to handle them itself.
argparse gets the arguments from sys.argv[1:], and does not change it. So even if your parser uses some of the arguments, you could pass that list on to popen (all or in part).
The argparse docs has an example, under nargs=REMAINDER, of parsing some arguments for itself, and collecting the rest to pass to another program. This is their example:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
So you could call popen with something like
plist = ['wrapped_program']
plist.extend(args.args)
popen(plist, ...)
Using parse.parse_known_args can also be used to collect unparsed words into an 'extras' list. That section of the docs talks about passing those strings on to another program (just as you are doing). In contrast with the REMAINDER case, the extra stuff does not have to be last.
These work, of course, only if this parser doesn't need --pattern for itself. If it parses it, then it won't appear appear in the REMAINDER or extras. In that case you will have to add it back to the list that you give popen
I would tweak your parser thus:
pass_1 = 'passed' # without the -- or internal -
dpass_` = '--'+pass_
parser = argparse.ArgumentParser()
parser.add_argument("-p", dpass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
process = Popen(['wrapped_program', dpass_1, getattr(args, pass_1)], ...)
another option:
parser = argparse.ArgumentParser()
pass_action = parser.add_argument("-p", '--pass-me-on', help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
If you print pass_action (in a shell) you'll get something like:
_StoreAction(option_strings=['-p', '--pass-me-on'], dest='pass_me_on', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
So you could pull the --name and dest from that object, thus:
process = Popen(['wrapped_program', pass_action.option_strings[-1], getattr(args, pass_action.dest), ...], ...)
You have to look in sys.argv to see which option_string was used (the long, short or other). The parser does not record that anywhere.
Note '--pass-me-on' produced dest='pass_me_on'. The conversion of - to _ can complicate deriving one string from the other.
If you have a dest string, you have to use getattr to pull it from the args namespace, or use vars(args)[dest] (dictionary access).
Another issue. If --patten has nargs='+', its value will be a list, as opposed to a string. You'd have to careful when merging that into thepopen` argument list.
I'm trying to disable same argument occurences within one command line, using argparse
./python3 --argument1=something --argument2 --argument1=something_else
which means this should raise an error, because value of argument1 is overriden, by default, argparse just overrides the value and continues like nothing happened... Is there any smart way how to disable this behaviour?
I don't think there is a native way to do it using argparse, but fortunately, argparse offers methods to report custom errors. The most elegant way is probably to define a custom action that checks for duplicates (and exits if there are).
class UniqueStore(argparse.Action):
def __call__(self, parser, namespace, values, option_string):
if getattr(namespace, self.dest, self.default) is not self.default:
parser.error(option_string + " appears several times.")
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--foo', action=UniqueStore)
args = parser.parse_args()
(Read the docs about cutom actions)
Another way is to use the append action and count the len of the list.
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--foo', action='append')
args = parser.parse_args()
if len(args.foo) > 1:
parser.error("--foo appears several times.")
There's no built in test or constraint. A positional argument will be handled only once, but the flagged (or optional) ones can, as you say, be repeated. This lets you collect multiple occurrences with append or count actions.
The override action is acceptable to most people. Why might your user use the option more than once? Why should the first be preferred over the last?
A custom Action may be the best choice. It could raise an error if the namespace[dest] already has a non-default value. Or this Action could add some other 'repeat' flag to the namespace.