Develop Reference

python argparse default with nargs wont work [duplicate]

This question already has answers here:
Argparse optional argument with different default if specified without a value
(2 answers)
Closed last month.
Here is my code:
from argparse import ArgumentParser, RawTextHelpFormatter
example_text = "test"
parser = ArgumentParser(description='my script.',
epilog=example_text,
formatter_class=RawTextHelpFormatter)
parser.add_argument('host', type=str, default="10.10.10.10",
help="Device IP address or Hostname.")
parser.add_argument('-j','--json_output', type=str, default="s", nargs='?',choices=["s", "l"],
help="Print GET statement in json form.")
#mutally exclusive required settings supplying the key
settingsgroup = parser.add_mutually_exclusive_group(required=True)
settingsgroup.add_argument('-k', '--key', type=str,
help="the api-key to use. WARNING take care when using this, the key specified will be in the user's history.")
settingsgroup.add_argument('--config', type=str,
help="yaml config file. All parameters can be placed in the yaml file. Parameters provided from form command line will take priority.")
args = parser.parse_args()
print(args.json_output)
my output:
None
Everything I am reading online says this should work, but it doesn't. Why?

You could use the const= parameter. const stores its value when the option is present but have no values
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-j', '--json-output', nargs='?', choices=['s', 'l'], const='s')
args = parser.parse_args()
However design wise, it might be better to use the following:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--output-type', choices=['json-s', 'json-l', 'normal'], default='normal')
args = parser.parse_args()

argparse - Modify the 'required' state of a subcommand's parent arguments

I'm trying to build a CLI in Python and I have an argument (--arg) that I want to reuse across multiple subcommands (req and opt).
But one subcommand (req) will have to require --arg and the other (opt) doesn't. How do I resolve this without having to make two versions of the same argument?
import argparse
arg_1 = argparse.ArgumentParser(add_help=False)
arg_1.add_argument('-a', '--arg', required=True,
help='reusable argument')
parser = argparse.ArgumentParser()
subp = parser.add_subparsers()
cmd_require = subp.add_parser('req', parents=[arg_1],
help='this subcommand requires --arg')
cmd_optional = subp.add_parser('opt', parents=[arg_1],
help='this subcommand doesn\'t require --arg')

I don't know any 'native' argparse feature that does that. However, I thought of 2 different approaches to solve your problem.
Validate args in a separate function -
Sometimes CLI application get complicated and by adding a validator function you can 'complete' the missing argparse features you wish for.
import argparse
arg_1 = argparse.ArgumentParser(add_help=False)
arg_1.add_argument('-a', '--arg', required=False,
help='reusable argument')
parser = argparse.ArgumentParser()
subp = parser.add_subparsers(dest='sub_parser')
cmd_require = subp.add_parser('req', parents=[arg_1],
help='this subcommand requires --arg')
cmd_optional = subp.add_parser('opt', parents=[arg_1],
help='this subcommand doesn\'t require --arg')
def validate_args(args):
print(args)
if args.sub_parser == 'req' and not args.arg:
print("Invalid usage! using 'req' requires 'arg'")
exit(1)
if __name__ == '__main__':
args = parser.parse_args()
validate_args(args)
Note:
I used dest for the subparser in order to later identify the chosen
subparser.
Using argparse, if an optional argument is not passed it will be 'None'
"prepared argument" -
Although argparse doesn't support an argument object - you could 'prepare' an argument by unpacking a dict and a tuple (*args, **kwargs)
import argparse
arg_name = ('-a', '--arg')
arg_dict = {'help': 'reusable argument'}
parser = argparse.ArgumentParser()
subp = parser.add_subparsers()
cmd_require = subp.add_parser('req',
help='this subcommand requires --arg')
cmd_optional = subp.add_parser('opt',
help='this subcommand doesn\'t require --arg')
cmd_optional.add_argument(*arg_name, **arg_dict, required=False)
cmd_require.add_argument(*arg_name, **arg_dict, required=True)
if __name__ == '__main__':
args = parser.parse_args()
validate_args(args)
I like the first approach better.
Hope you find that useful

import argparse
arg_1 = argparse.ArgumentParser(add_help=False)
foobar = arg_1.add_argument('-a', '--arg', required=True,
help='reusable argument')
arg_1 is a parser object. When you use the add_argument command, it creates an Action object and adds it to the args_1._actions list. I just saved a reference to that in the foobar variable.
parser = argparse.ArgumentParser()
subp = parser.add_subparsers()
The parents mechanism adds the args_1._actions list to the cmd_require._actions list. So foobar will appear in both subparsers. It's copy by reference, which is common in python.
cmd_require = subp.add_parser('req', parents=[arg_1],
help='this subcommand requires --arg')
cmd_optional = subp.add_parser('opt', parents=[arg_1],
help='this subcommand doesn\'t require --arg')
foobar.required=False will turn off at attribute, but will do so for both parsers. I've seen this issue come up when people wanted to assign different default attributes.
The parents mechanism just a shortcut, that occasionally is useful, but not always. It doesn't do anything special; just saves a bit of typing. There are plenty other ways of defining an Action with the same flags in both subparsers.
typing it twice (horror of horrors!)
copy-n-paste
writing a utility function to add arguments to subparsers (see Larry Wall's Three Virtues)

nargs depending on another setting?

I'm trying to write a program that supports arbitrary bitwise opertions: AND, OR, NOT and COUNT for bitmaps. The usage is that you run program.py --and f1.bit f2.bit and it prints you the result to the stdout.
The problem is that I'd like the parser to handle all the caveats. Specifically, I'd like the nargs to depend on the mode that's set - if it's set to COUNT or NOT, exactly one file is expected, if it's set to OR or AND, expect exactly two. Here's some (non-working) example code:
#!/usr/bin/env python
import argparse
def main(mode, fnames):
pass
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-O', '--or',
nargs=2,
action='store_const', const='or'
)
args = parser.parse_args()
import pprint
pprint.pprint(args.__dict__)
#main(**args.__dict__)
And the error I'm getting:
Traceback (most recent call last):
File "bitmaptool.py", line 12, in <module>
action='store_const', const='or'
File "/usr/lib/python3.7/argparse.py", line 1362, in add_argument
action = action_class(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'nargs'
Commenting out nargs helps, as does leaving nargs out but commenting out action - but I want both. Do I need to implement it manually or is there a trick or another library that would let me get there?
EDIT I wanted to clarify what I'm looking for by showing what code I needed to write manually for the thing to work:
if __name__ == '__main__':
parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS)
parser.add_argument('-O', '--or', nargs=2)
parser.add_argument('-A', '--and', nargs=2)
parser.add_argument('-M', '--minus', nargs=2)
parser.add_argument('-C', '--count', nargs=1)
parser.add_argument('-N', '--not', nargs=1)
parser.add_argument('-o', '--output', default='/dev/stdout')
args = parser.parse_args().__dict__
mode = None
files = []
for current_mode in ['or', 'and', 'not', 'count']:
if current_mode in args:
if mode is not None:
sys.exit('ERROR: more than one mode was specified')
mode = current_mode
files = args[mode]
if mode is None:
sys.stderr.write('ERROR: no mode was specified\n\n')
parser.print_help()
sys.exit(1)
import pprint
pprint.pprint(args)
Is there a more elegant way to get there?

store_const never gets arguments, it literally stores what you stated as const or None. Because it's a constant, not a variable. From the argparse's action documentation, ephasis mine:
'store_const' - This stores the value specified by the const keyword argument. The 'store_const' action is most commonly used with optional arguments that specify some sort of flag.
You should change the action to something that will actually store the filenames passed. As per argparse's nargs documentation and example, you actually don't need to specify action at all, default (action='store') will suffice.
Example from documentation:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', nargs=2) #this line
>>> parser.add_argument('bar', nargs=1)
>>> parser.parse_args('c --foo a b'.split())
Namespace(bar=['c'], foo=['a', 'b'])
EDIT for the edited version of the question - mutually exclusive group will make sure only one argument (from that group, of course) is specified:
if __name__ == '__main__':
parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS)
parser.add_argument('-o', '--output', default='/dev/stdout')
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-O', '--or', nargs=2)
group.add_argument('-A', '--and', nargs=2)
group.add_argument('-M', '--minus', nargs=2)
group.add_argument('-C', '--count', nargs=1)
group.add_argument('-N', '--not', nargs=1)
args = parser.parse_args().__dict__
import pprint
pprint.pprint(args)

Iterating over accepted args of argparse

I cant see m to figure out how to iterate over the accepted args of argparse. I get I can iterate over the parsed_args result, but what I want is to iterate over the arguments the parser is configured with ( ie with optparse you can iterate over the args ).
for example:
parser = argparse.ArgumentParser( prog = 'myapp' )
parser.add_argument( '--a', .. )
parser.add_argument( '--b', ...)
parser.add_argument( '--c', ... )
for arg in parser.args():
print arg
would result in
--a
--b
--c

You'll probably want to getattr from the args:
args = parser.parse_args()
for arg in vars(args):
print arg, getattr(args, arg)
Result:
a None
c None
b None

If you want to list the optionals you can do it this way:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar')
parser.add_argument('--baz')
for option in parser._optionals._actions:
print(option.option_strings)
I don't see a practical reason to iterate over them however. You can always see the options via --help.

A bit late to the game here, but I found a way to do this without reading from private variables by using a custom help formatter that collects the arguments it is asked to format.
The following program will print ['-h', '--help', '--a', '--b', '--c']
import argparse
class ArgCollector(argparse.HelpFormatter):
# Will store the arguments in a class variable since argparse uses a class
# name, not an instance of a class
args = []
def add_argument(self, action):
# Just remember the options
self.args.extend(action.option_strings)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--a')
parser.add_argument('--b')
parser.add_argument('--c')
# Install our new help formatter, use it, then restore the original
# formatter
original_formatter_class = parser.formatter_class
parser.formatter_class = ArgCollector
parser.format_help()
parser.formatter_class = original_formatter_class
# Print the args that argparse would accept
print(ArgCollector.args)
if __name__ == '__main__':
main()

Call function based on argparse

I'm new to python and currently playing with it.
I have a script which does some API Calls to an appliance. I would like to extend the functionality and call different functions based on the arguments given when calling the script.
Currently I have the following:
parser = argparse.ArgumentParser()
parser.add_argument("--showtop20", help="list top 20 by app",
action="store_true")
parser.add_argument("--listapps", help="list all available apps",
action="store_true")
args = parser.parse_args()
I also have a
def showtop20():
.....
and
def listapps():
....
How can I call the function (and only this) based on the argument given?
I don't want to run
if args.showtop20:
#code here
if args.listapps:
#code here
as I want to move the different functions to a module later on keeping the main executable file clean and tidy.

Since it seems like you want to run one, and only one, function depending on the arguments given, I would suggest you use a mandatory positional argument ./prog command, instead of optional arguments (./prog --command1 or ./prog --command2).
so, something like this should do it:
FUNCTION_MAP = {'top20' : my_top20_func,
'listapps' : my_listapps_func }
parser.add_argument('command', choices=FUNCTION_MAP.keys())
args = parser.parse_args()
func = FUNCTION_MAP[args.command]
func()

At least from what you have described, --showtop20 and --listapps sound more like sub-commands than options. Assuming this is the case, we can use subparsers to achieve your desired result. Here is a proof of concept:
import argparse
import sys
def showtop20():
print('running showtop20')
def listapps():
print('running listapps')
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Create a showtop20 subcommand
parser_showtop20 = subparsers.add_parser('showtop20', help='list top 20 by app')
parser_showtop20.set_defaults(func=showtop20)
# Create a listapps subcommand
parser_listapps = subparsers.add_parser('listapps', help='list all available apps')
parser_listapps.set_defaults(func=listapps)
# Print usage message if no args are supplied.
# NOTE: Python 2 will error 'too few arguments' if no subcommand is supplied.
# No such error occurs in Python 3, which makes it feasible to check
# whether a subcommand was provided (displaying a help message if not).
# argparse internals vary significantly over the major versions, so it's
# much easier to just override the args passed to it.
if len(sys.argv) <= 1:
sys.argv.append('--help')
options = parser.parse_args()
# Run the appropriate function (in this case showtop20 or listapps)
options.func()
# If you add command-line options, consider passing them to the function,
# e.g. `options.func(options)`

There are lots of ways of skinning this cat. Here's one using action='store_const' (inspired by the documented subparser example):
p=argparse.ArgumentParser()
p.add_argument('--cmd1', action='store_const', const=lambda:'cmd1', dest='cmd')
p.add_argument('--cmd2', action='store_const', const=lambda:'cmd2', dest='cmd')
args = p.parse_args(['--cmd1'])
# Out[21]: Namespace(cmd=<function <lambda> at 0x9abf994>)
p.parse_args(['--cmd2']).cmd()
# Out[19]: 'cmd2'
p.parse_args(['--cmd1']).cmd()
# Out[20]: 'cmd1'
With a shared dest, each action puts its function (const) in the same Namespace attribute. The function is invoked by args.cmd().
And as in the documented subparsers example, those functions could be written so as to use other values from Namespace.
args = parse_args()
args.cmd(args)
For sake of comparison, here's the equivalent subparsers case:
p = argparse.ArgumentParser()
sp = p.add_subparsers(dest='cmdstr')
sp1 = sp.add_parser('cmd1')
sp1.set_defaults(cmd=lambda:'cmd1')
sp2 = sp.add_parser('cmd2')
sp2.set_defaults(cmd=lambda:'cmd2')
p.parse_args(['cmd1']).cmd()
# Out[25]: 'cmd1'
As illustrated in the documentation, subparsers lets you define different parameter arguments for each of the commands.
And of course all of these add argument or parser statements could be created in a loop over some list or dictionary that pairs a key with a function.
Another important consideration - what kind of usage and help do you want? The different approaches generate very different help messages.

If your functions are "simple enough" take adventage of type parameter https://docs.python.org/2.7/library/argparse.html#type
type= can take any callable that takes a single string argument and
returns the converted value:
In your example (even if you don't need a converted value):
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
This simple script:
import argparse
def showtop20(dummy):
print "{0}\n".format(dummy) * 5
parser = argparse.ArgumentParser()
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
args = parser.parse_args()
Will give:
# ./test.py --listapps test
test
test
test
test
test
test

Instead of using your code as your_script --showtop20, make it into a sub-command your_script showtop20 and use the click library instead of argparse. You define functions that are the name of your subcommand and use decorators to specify the arguments:
import click
#click.group()
#click.option('--debug/--no-debug', default=False)
def cli(debug):
print(f'Debug mode is {"on" if debug else "off"}')
#cli.command() # #cli, not #click!
def showtop20():
# ...
#cli.command()
def listapps():
# ...
See https://click.palletsprojects.com/en/master/commands/

# based on parser input to invoke either regression/classification plus other params
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--target", type=str)
parser.add_argument("--type", type=str)
parser.add_argument("--deviceType", type=str)
args = parser.parse_args()
df = pd.read_csv(args.path)
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]
if args.type == "classification":
classify = AutoML(df, args.target, args.type, args.deviceType)
classify.class_dist()
classify.classification()
elif args.type == "regression":
reg = AutoML(df, args.target, args.type, args.deviceType)
reg.regression()
else:
ValueError("Invalid argument passed")
# Values passed as : python app.py --path C:\Users\Abhishek\Downloads\adult.csv --target income --type classification --deviceType GPU

You can evaluate using evalwhether your argument value is callable:
import argparse
def list_showtop20():
print("Calling from showtop20")
def list_apps():
print("Calling from listapps")
my_funcs = [x for x in dir() if x.startswith('list_')]
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--function", required=True,
choices=my_funcs,
help="function to call", metavar="")
args = parser.parse_args()
eval(args.function)()