Develop Reference

Python: command-line arguments --foo and --no-foo

For parsing boolean command-line options using Python's built-in argparse package, I am aware of this question and its several answers: Parsing boolean values with argparse.
Several of the answers (correctly, IMO) point out that the most common and straightforward idiom for boolean options (from the caller's point of view) is to accept both --foo and --no-foo options, which sets some value in the program to True or False, respectively.
However, all the answers I can find don't actually accomplish the task correctly, it seems to me. They seem to generally fall short on one of the following:
A suitable default can be set (True, False, or None).
Help text given for program.py --help is correct and helpful, including showing what the default is.
Either of (I don't really care which, but both are sometimes desirable):
An argument --foo can be overridden by a later argument --no-foo and vice versa;
--foo and --no-foo are incompatible and mutually exclusive.
What I'm wondering is whether this is even possible at all using argparse.
Here's the closest I've come, based on answers by #mgilson and #fnkr:
def add_bool_arg(parser, name, help_true, help_false, default=None, exclusive=True):
if exclusive:
group = parser.add_mutually_exclusive_group(required=False)
else:
group = parser
group.add_argument('--' + name, dest=name, action='store_true', help=help_true)
group.add_argument('--no-' + name, dest=name, action='store_false', help=help_false)
parser.set_defaults(**{name: default})
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
add_bool_arg(parser, 'foo', "Do foo", "Don't foo", exclusive=True)
add_bool_arg(parser, 'bar', "Do bar", "Don't bar", default=True, exclusive=False)
That does most things well, but the help-text is confusing:
usage: argtest.py [-h] [--foo | --no-foo] [--bar] [--no-bar]
optional arguments:
-h, --help show this help message and exit
--foo Do foo (default: None)
--no-foo Don't foo (default: None)
--bar Do bar (default: True)
--no-bar Don't bar (default: True)
A better help text would be something like this:
usage: argtest.py [-h] [--foo | --no-foo] [--bar] [--no-bar]
optional arguments:
-h, --help show this help message and exit
--foo --no-foo Whether to foo (default: None)
--bar --no-bar Whether to bar (default: True)
But I don't see a way to accomplish that, since "--*" and "--no-*" must always be declared as separate arguments (right?).
In addition to the suggestions at the SO question mentioned above, I've also tried creating a custom action using techniques shown in this other SO question: Python argparse custom actions with additional arguments passed . These fail immediately saying either "error: argument --foo: expected one argument", or (if I set nargs=0) "ValueError: nargs for store actions must be > 0". From poking into the argparse source, it looks like this is because actions other than the pre-defined 'store_const', 'store_true', 'append', etc. must use the _StoreAction class, which requires an argument.
Is there some other way to accomplish this? If someone has a combination of ideas I haven't thought of yet, please let me know!
(BTW- I'm creating this new question, rather than trying to add to the first question above, because the original question above was actually asking for a method to handle --foo TRUE and --foo FALSE arguments, which is different and IMO less commonly seen.)

One of the answers in your linked question, specifically the one by Robert T. McGibbon, includes a code snippet from an enhancement request that was never accepted into the standard argparse. It works fairly well, though, if you discount one annoyance. Here is my reproduction, with a few small modifications, as a stand-alone module with a little bit of pydoc string added, and an example of its usage:
import argparse
import re
class FlagAction(argparse.Action):
"""
GNU style --foo/--no-foo flag action for argparse
(via http://bugs.python.org/issue8538 and
https://stackoverflow.com/a/26618391/1256452).
This provides a GNU style flag action for argparse. Use
as, e.g., parser.add_argument('--foo', action=FlagAction).
The destination will default to 'foo' and the default value
if neither --foo or --no-foo are specified will be None
(so that you can tell if one or the other was given).
"""
def __init__(self, option_strings, dest, default=None,
required=False, help=None, metavar=None,
positive_prefixes=['--'], negative_prefixes=['--no-']):
self.positive_strings = set()
# self.negative_strings = set()
# Order of strings is important: the first one is the only
# one that will be shown in the short usage message! (This
# is an annoying little flaw.)
strings = []
for string in option_strings:
assert re.match(r'--[a-z]+', string, re.IGNORECASE)
suffix = string[2:]
for positive_prefix in positive_prefixes:
s = positive_prefix + suffix
self.positive_strings.add(s)
strings.append(s)
for negative_prefix in negative_prefixes:
s = negative_prefix + suffix
# self.negative_strings.add(s)
strings.append(s)
super(FlagAction, self).__init__(option_strings=strings, dest=dest,
nargs=0, default=default,
required=required, help=help,
metavar=metavar)
def __call__(self, parser, namespace, values, option_string=None):
if option_string in self.positive_strings:
setattr(namespace, self.dest, True)
else:
setattr(namespace, self.dest, False)
if __name__ == '__main__':
p = argparse.ArgumentParser()
p.add_argument('-a', '--arg', help='example')
p.add_argument('--foo', action=FlagAction, help='the boolean thing')
args = p.parse_args()
print(args)
(this code works in Python 2 and 3 both).
Here is the thing in action:
$ python flag_action.py -h
usage: flag_action.py [-h] [-a ARG] [--foo]
optional arguments:
-h, --help show this help message and exit
-a ARG, --arg ARG example
--foo, --no-foo the boolean thing
Note that the initial usage message does not mention the --no-foo option. There is no easy way to correct this other than to use the group method that you dislike.
$ python flag_action.py -a something --foo
Namespace(arg='something', foo=True)
$ python flag_action.py --no-foo
Namespace(arg=None, foo=False)

How to grab all defaults from `argparse`?

I have an argument parser like so:
def args_parser():
parser = argparse.ArgumentParser(description='myparser')
parser.add_argument('--k', type=int, default=100, help = 'numbers')
return parser
Now, all I want to do is just get the default mappings, like so:
args = args_parser()
And then I want args['k'] to return 100, since that is the default. How do I make this happen?

This should do that for you. I think you should be able to find this in the Python docs on ArgumentParser as well:
vars(args.parse_args())['k']

You need to call the .parse_args() function on your parser object in order to actually return the command-line arguments parsed and assigned to attributes as defined in your .add_argument() statement. So something like:
parser = args_parser()
args = parser.parse_args()
k_value = args.k
The 'args' variable returned by .parse_args() will have attributes assigned corresponding to the names you assign to it via the .add_argument() call, although if there is no default and the user does not provide a given argument its value will be 'None'.

def args_parser():
parser = argparse.ArgumentParser(description='myparser')
parser.add_argument('--k', type=int, default=100, help = 'numbers')
return parser
parser = args_parser():
args = parser.parse_args([])
print(args) # see all values
print(args.k) # see just the k attribute
parse_args() parses the sys.argv as provided in the commandline. parse_args([]) parses an empty list, same as if you had called the script without arguments. I regularly test a parser setup with calls like this.
With your argument definition the [] works fine and shows the default k value. With other defintions, [] might produce an error. For example if you included
parser.add_argument('infile', help='positional file name')
now a string is required. For a simple positional like that a default is not needed nor does it make sense. But other parameter combinations can make use of a default.
I could show you how to access the default as set in the add_argument command, but that involves some undocumented features. In addition there are several ways of defining defaults, so the topic can get complicated.
Anyways simply running the parse_args with an empty list is often enough.
You can also make it display the default in the help line with a little addition:
parser.add_argument('--k', type=int, default=100, help = 'numbers, %(default)s')
For example, in an interactive ipython session:
In [73]: parser=argparse.ArgumentParser()
In [74]: parser.add_argument('--k', type=int, default=100, help = 'numbers, %(default)s')
Out[74]: _StoreAction(option_strings=['--k'], dest='k', nargs=None,
const=None, default=100, type=<class 'int'>, choices=None,
help='numbers, %(default)s', metavar=None)
In [75]: parser.print_help()
usage: ipython3 [-h] [--k K]
optional arguments:
-h, --help show this help message and exit
--k K numbers, 100
In [76]: parser.parse_args([]) # displays args
Out[76]: Namespace(k=100)
In [78]: parser.parse_args(['--k','200'])
Out[78]: Namespace(k=200)

Why is there a difference when calling argparse.parse_args() or .parse_args(sys.argv)

I have created the following argument parser in my python code.
parser = argparse.ArgumentParser()
parser.add_argument('projectPath')
parser.add_argument('-project')
parser.add_argument('-release')
parser.add_argument('--test', default=False, action='store_true')
args = parser.parse_args()
and I'm executing my program the following way.
myProgram.py /path/to/file -project super --test
it works fine if I use the sysntax above with
args = parser.parse_args()
However if I take and use the sys.argv as input
args = parser.parse_args(sys.argv)
The parser is suddenly picky about the order of the arguments and I get the unrecognized argument error.
usage: fbu.py [-h] [-project PROJECT] [-release RELEASE] [--test] projectPath
fbu.py: error: unrecognized arguments: /path/to/file
As I can see from the error and also using the -h argument. The path argument must be last and the error makes sense in the last example.
But why does it not care about the order in the first example ?
EDIT: I'm using python version 3.4.3

sys.argv contains the script name as the first item, i.e. myProgram.py. That argument takes the spot of projectPath. Now there's one additional positional argument /path/to/file, which can't be matched to any arguments, hence the error.
Calling parse_args without arguments ArgumentParser is clever enough to omit the script name from being parsed. But when explicitly passing an array of arguments, it can't do that and will parse everything.

As you can see from looking at the source code for parse_known_args (which is called by parse_args):
if args is None:
# args default to the system args
args = _sys.argv[1:]
When you don't provide the arguments explicitly, Python removes the first item from .argv (which is the name of the script). If you pass the arguments manually, you must do this yourself:
parser.parse_args(sys.argv[1:])
This isn't explicitly covered in the documentation, but note that this section doesn't include a script name when calling parse_args manually:
Beyond sys.argv
Sometimes it may be useful to have an ArgumentParser parse arguments
other than those of sys.argv. This can be accomplished by passing a
list of strings to parse_args(). This is useful for testing at the
interactive prompt:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument(
... 'integers', metavar='int', type=int, choices=xrange(10),
... nargs='+', help='an integer in the range 0..9')
>>> parser.add_argument(
... '--sum', dest='accumulate', action='store_const', const=sum,
... default=max, help='sum the integers (default: find the max)')
>>> parser.parse_args(['1', '2', '3', '4'])
Namespace(accumulate=<built-in function max>, integers=[1, 2, 3, 4])
>>> parser.parse_args('1 2 3 4 --sum'.split())
Namespace(accumulate=<built-in function sum>, integers=[1, 2, 3, 4])
The advantage of passing the arguments manually is that it makes it easier to test the parsing functionality, as you can pass in a list of appropriate arguments rather than trying to patch sys.argv.

How to use a Python keyword as an argument name for argparse?

lambda has a keyword function in Python:
f = lambda x: x**2 + 2*x - 5
What if I want to use it as a variable name? Is there an escape sequence or another way?
You may ask why I don't use another name. This is because I'd like to use argparse:
parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
parser.add_argument("-l","--lambda",help="Defines the quantity called lambda", type=float)
args = parser.parse_args()
print args.lambda # syntax error!
Script called with --help option gives:
...
optional arguments
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
Because of that, I would like to stay with lambda as the variable name. Solutions may be argparse-related as well.

You can use dynamic attribute access to access that specific attribute still:
print getattr(args, 'lambda')
Better still, tell argparse to use a different attribute name:
parser.add_argument("-l", "--lambda",
help="Defines the quantity called lambda",
type=float, dest='lambda_', metavar='LAMBDA')
Here the dest argument tells argparse to use lambda_ as the attribute name:
print args.lambda_
The help text still will show the argument as --lambda, of course; I set metavar explicitly as it otherwise would use dest in uppercase (so with the underscore):
>>> import argparse
>>> parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
>>> parser.add_argument("-l", "--lambda",
... help="Defines the quantity called lambda",
... type=float, dest='lambda_', metavar='LAMBDA')
_StoreAction(option_strings=['-l', '--lambda'], dest='lambda_', nargs=None, const=None, default=None, type=<type 'float'>, choices=None, help='Defines the quantity called lambda', metavar='LAMBDA')
>>> parser.print_help()
usage: Calculate something with a quantity commonly called lambda.
[-h] [-l LAMBDA]
optional arguments:
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
>>> args = parser.parse_args(['--lambda', '4.2'])
>>> args.lambda_
4.2

There is an argparse-specific way of dealing with this. From the documentation:
If you prefer to have dict-like view of the attributes, you can use
the standard Python idiom, vars().
Therefore, you should be able to write:
print vars(args)["lambda"] # No keyword used, no syntax error.

argparse provides destination functionality for arguments if the long option name is not the desired attribute name for the argument.
For instance:
parser = argparse.ArgumentParser()
parser.add_argument("--lambda", dest="function")
args = parser.parse_args()
print(args.function)

Can Python's optparse display the default value of an option?

Is there a way to make Python's optparse print the default value of an option or flag when showing the help with --help?

Try using the %default string placeholder:
# This example taken from http://docs.python.org/library/optparse.html#generating-help
parser.add_option("-m", "--mode",
default="intermediate",
help="interaction mode: novice, intermediate, "
"or expert [default: %default]")

And if you want to add default values automatically to all options that you have specified, you can do the following:
for option in parser.option_list:
if option.default != ("NO", "DEFAULT"):
option.help += (" " if option.help else "") + "[default: %default]"

And if you need programmatic access to the default values, you can get to them via the defaults attribute of the parser (it's a dict)

The comments to your question already indicate there's another way to parse arguments called argparse. It's been introduced in Python 3.2. It actually deprecates optparse but is used similarly.
argpass comes with different formatting classes and for instance argparse.ArgumentDefaultsHelpFormatter will also print the default values without you having to manipulate the help string manually.
ArgumentParser objects allow the help formatting to be customized by
specifying an alternate formatting class. Currently, there are four
such classes:
class argparse.RawDescriptionHelpFormatter
class argparse.RawTextHelpFormatter
class argparse.ArgumentDefaultsHelpFormatter
class argparse.MetavarTypeHelpFormatter
An example from the python docs:
>>> parser = argparse.ArgumentParser(
... prog='PROG',
... formatter_class=argparse.ArgumentDefaultsHelpFormatter)
>>> parser.add_argument('--foo', type=int, default=42, help='FOO!')
>>> parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
>>> parser.print_help()
usage: PROG [-h] [--foo FOO] [bar [bar ...]]
positional arguments:
bar BAR! (default: [1, 2, 3])
optional arguments:
-h, --help show this help message and exit
--foo FOO FOO! (default: 42)
see argparse formatting classes

Add argparse.ArgumentDefaultsHelpFormatter to your parser
import argparse
parser = argparse.ArgumentParser(
description='Your application description',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
from documentation:
ArgumentDefaultsHelpFormatter automatically adds information about
default values to each of the argument help messages:
Blockquote