I am trying to use argparse to process several optional arguments. Each of the arguments will have a single optional argument as well. For example I have a script called runner.py. I want to call runner.py --functionals --capacity --performance and I want it to use the const values I have set. This part is working. I also want to be able to specify arguments such as --functionals test1 --performance test2 and --capacity test3. Instead of const, now I except the arguments to have the given values. for ex. functionals should be test1, performance test2 etc. What results in the latter case is I get: -c: error: argument --performance: not allowed with argument --functionals
Code for the parser looks like:
def get_parser():
parser = argparse.ArgumentParser(add_help=False)
required_arguments = parser.add_argument_group(title = "required arguments")
test_arguments = parser.add_mutually_exclusive_group()
test_arguments.add_argument(
'--capacity',
nargs='?',
)
test_arguments.add_argument(
'--functionals',
nargs='?',
)
test_arguments.add_argument(
'--performance',
nargs='?',
)
return parser
My mistake was that I was using a mutually exclusive group. I should have been using an regular argument group.
Related
I have an argparse that is given a string:
def f():
return 'dummy2'
p = argparse.ArgumentParser()
p.add_argument('--a', nargs=1, type=str)
p.add_argument('--b', nargs='?', const=f(), default=f())
p.parse_args('--a dummy'.split())
The parser namespace is Namespace(a=['dummy'], b='dummy2').
How can I make the argument for a be stored as a string and not as a list of strings?
It's simple, just skip the argument for nargs. Try this:
p = argparse.ArgumentParser()
p.add_argument('--a', type=str)
p.add_argument('--b', nargs='?', const=f(), default=f())
I believe this is what you expected:
p.parse_args('--a dummy'.split())
=> Namespace(a='dummy', b='dummy2')
Quoting the docs:
ArgumentParser objects usually associate a single command-line argument with a single action to be taken. The nargs keyword argument associates a different number of command-line arguments with a single action. The supported values are:
N (an integer). N arguments from the command line will be gathered together into a list ... Note that nargs=1 produces a list of one item. This is different from the default, in which the item is produced by itself.
We're trying to build a wrapper script over a command line tool we're using. We would like to set some tool arguments based on options in our wrapper scripts. We would also like to have the possibility to pass native arguments to the command line tool directly as they are written on the command line.
Here is what we came up with:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('positional')
parser.add_argument('-f', '--foo', action='store_true')
parser.add_argument('-b', '--bar', action='store_true')
parser.add_argument('native_arg', nargs='*')
args = parser.parse_args()
print (args)
positional is mandatory. Based on the options -f and -b we would add some extra options to our tool call. Anything that is left afterwards (if anything) should be treated as a native tool argument and given to the tool directly. Calling our script with -h produces the following usage:
usage: test.py [-h] [-f] [-b] positional [native_arg [native_arg ...]]
The trick is that these native arguments are themselves options for the tool and contain leading dashes, for example -native0 and -native1. We already know about the trick with the double dash to stop argparse from looking for more options. The following call:
./test.py pos -- -native0 -native1
produces the expected parsed arguments:
Namespace(bar=False, foo=False, native_arg=['-native0', '-native1'], positional='pos')
Trying to add an option after the first positional argument doesn't work, though. More specifically, the following call:
./test.py pos --foo -- -native0 -native1
produces the following output:
usage: [...shortened...]
test.py: error: unrecognized arguments: -- -native0 -native1
Putting the optional arguments before the positionals:
./test.py --foo pos -- -native0 -native1
seems to work, as the following is printed:
Namespace(bar=False, foo=True, native_arg=['-native0', '-native1'], positional='pos')
Even stranger, changing the value of nargs for native_arg to '+' works in all the above situations (with the caveat, of course, that at least one native_arg is expected).
Are we doing something wrong in our Python code or is this some kind of argparse bug?
argparse does have a hard time when you mix non-required positional arguments with optional arguments (see https://stackoverflow.com/a/47208725/1399279 for details into the bug report). Rather than suggesting a way to solve this issue, I am going to present an alternative approach.
You should check out the parse_known_args method, which was created for the situation you describe (i.e. passing options to a wrapped tool).
In [1]: import argparse
In [2]: parser = argparse.ArgumentParser()
In [3]: parser.add_argument('positional')
In [4]: parser.add_argument('-f', '--foo', action='store_true')
In [5]: parser.add_argument('-b', '--bar', action='store_true')
In [6]: parser.parse_known_args(['pos', '--foo', '-native0', '-native1'])
Out[6]: (Namespace(bar=False, foo=True, positional='pos'), ['-native0', '-native1'])
Unlike parse_args, the output of parse_known_args is a two-element tuple. The first element is the Namespace instance you would expect to get from parse_args, and it contains all the attributes defined by calls to add_argument. The second element is a list of all the arguments not known to the parser.
I personally prefer this method because the user does not need to remember any tricks about how to call your program, or which option order does not result in errors.
This is a known issue (https://bugs.python.org/issue15112, argparse: nargs='*' positional argument doesn't accept any items if preceded by an option and another positional)
The parsing alternates handling positionals and optionals. When dealing with positionals it tries to handle as many as the input strings require. But an ? or * positional is satisfied with [], an empty list of strings. + on the other hand requires at least one string
./test.py pos --foo -- -native0 -native1
The parser gives 'pos' to positional, and [] to native-arg. Then it gives '--foo' to its optional. There aren't anymore positionals left to hand the remaining strings, so it raises the error.
The allocation of input strings is done with a stylized form of regex string matching. Imagine matching a pattern that looks like AA?.
To correct this, parser would have to look ahead, and delay handling native-arg. We've suggested patches but they aren't in production.
#SethMMorton's suggestion of using parse_known_args is a good one.
Earlier parsers (e.g. Optparse) handle all the flagged arguments, but return the rest, the positionals, as a undifferentiated list. It's up to the user to split that list. argparse has added the ability to name and parse positionals, but the algorithm works best with fixed nargs, and gets flaky with too many variable nargs.
What's the best way to set a group argument as the default when no arguments are called.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("--a", action="store_true") #call when no arguments are provided
group.add_argument("--b", action="store_true")
group.add_argument("--c", action="store_true")
Let's call my program argparse_ex.py. I want argparse.py (with no arguments) and argparse.py --a to return the same output.
I would just add a simple test after parsing
if not any([args.a, args.b, args.c]):
args.a=True
This is simpler than any attempt to make parse_args to do this. The parser will parse all arguments independently - and in any order. So you really can't tell until the parsing is all done whether any of the options has been selected or not.
I have created the following argument parser in my python code.
parser = argparse.ArgumentParser()
parser.add_argument('projectPath')
parser.add_argument('-project')
parser.add_argument('-release')
parser.add_argument('--test', default=False, action='store_true')
args = parser.parse_args()
and I'm executing my program the following way.
myProgram.py /path/to/file -project super --test
it works fine if I use the sysntax above with
args = parser.parse_args()
However if I take and use the sys.argv as input
args = parser.parse_args(sys.argv)
The parser is suddenly picky about the order of the arguments and I get the unrecognized argument error.
usage: fbu.py [-h] [-project PROJECT] [-release RELEASE] [--test] projectPath
fbu.py: error: unrecognized arguments: /path/to/file
As I can see from the error and also using the -h argument. The path argument must be last and the error makes sense in the last example.
But why does it not care about the order in the first example ?
EDIT: I'm using python version 3.4.3
sys.argv contains the script name as the first item, i.e. myProgram.py. That argument takes the spot of projectPath. Now there's one additional positional argument /path/to/file, which can't be matched to any arguments, hence the error.
Calling parse_args without arguments ArgumentParser is clever enough to omit the script name from being parsed. But when explicitly passing an array of arguments, it can't do that and will parse everything.
As you can see from looking at the source code for parse_known_args (which is called by parse_args):
if args is None:
# args default to the system args
args = _sys.argv[1:]
When you don't provide the arguments explicitly, Python removes the first item from .argv (which is the name of the script). If you pass the arguments manually, you must do this yourself:
parser.parse_args(sys.argv[1:])
This isn't explicitly covered in the documentation, but note that this section doesn't include a script name when calling parse_args manually:
Beyond sys.argv
Sometimes it may be useful to have an ArgumentParser parse arguments
other than those of sys.argv. This can be accomplished by passing a
list of strings to parse_args(). This is useful for testing at the
interactive prompt:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument(
... 'integers', metavar='int', type=int, choices=xrange(10),
... nargs='+', help='an integer in the range 0..9')
>>> parser.add_argument(
... '--sum', dest='accumulate', action='store_const', const=sum,
... default=max, help='sum the integers (default: find the max)')
>>> parser.parse_args(['1', '2', '3', '4'])
Namespace(accumulate=<built-in function max>, integers=[1, 2, 3, 4])
>>> parser.parse_args('1 2 3 4 --sum'.split())
Namespace(accumulate=<built-in function sum>, integers=[1, 2, 3, 4])
The advantage of passing the arguments manually is that it makes it easier to test the parsing functionality, as you can pass in a list of appropriate arguments rather than trying to patch sys.argv.
I have a small program that uses argparse and a positional argument. I'm trying to allow that argument to be set by using an environment variable, but are not getting it to work.
I have seen this post: Setting options from environment variables when using argparse which mentions the same problem, but not for positional arguments.
This is the code so far:
import argparse
import os
class EnvDefault(argparse.Action):
def __init__(self, envvar, required=True, default=None, **kwargs):
if not default and envvar:
if envvar in os.environ:
default = os.environ[envvar]
if required and default:
required = False
super(EnvDefault, self).__init__(default=default, required=required, **kwargs)
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, values)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('testvar', help="Test variable", action=EnvDefault, envvar='TEST_VAR')
parser.add_argument('--othervar', help="Other variable", action='store_true')
args = parser.parse_args()
if not args.testvar: exit(parser.print_usage())
print args.testvar
Which returns this:
$ TEST_VAR="bla" ./test.py
usage: test.py [-h] [--othervar] testvar
test.py: error: too few arguments
You need to make positional argument optional, try nargs='?':
...
parser.add_argument('testvar', help="Test variable", action=EnvDefault,
envvar='TEST_VAR', nargs='?')
...
Note that output changes slightly:
$ python test.py
usage: test.py [-h] [--othervar] [testvar]
Note: There's one side effect - it doesn't return error, even if env variable is not set.
The too few error message indicates that you have slightly older version of Python/argparse. Here's the code that generates the message. It occurs at the end of parsing:
# if we didn't use all the Positional objects, there were too few
# arg strings supplied.
if positionals:
self.error(_('too few arguments'))
# make sure all required actions were present
for action in self._actions:
if action.required:
if action not in seen_actions:
name = _get_action_name(action)
self.error(_('argument %s is required') % name)
positionals starts a list of all the positional arguments, which are removed as they get matched up with argument strings. So it is testing if any were not matched.
Note that the test of the required attribute occurs after this positional test, so changing it, as your Action does, does not help.
The only way to make a positional optional is with nargs - ? or *. An empty list of strings matches those, so such a positional is always consumed.
Check the docs for these. The const parameter might be useful.
The latest version drops that if positionals test, using only the required test to generate a list of arguments that were not used. Your special Action might work in that code.