Using argparse, is there any way to update the "choices" option of an argument after it was added to the parser ? Argparse documentation doesn't yield much about updating the choices
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
parser.add_argument('arg1', choices=choices_list)
# The list of choices now changes
choices_list = ['A', 'C', 'D']
# Some code to update 'arg1' choices option ?
parser.???
I tried using 'parser.add_argument' with the new 'choices_list', but it creates duplicate arguments.
Using Python 3.7
The argument itself has a choices attribute, but it's easiest if you save a reference to the argument instead of trying to retrieve it from the parser itself. (Otherwise, you have to scan through the private attribute parser._actions and try to identify which one you need.)
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
arg1 = parser.add_argument('arg1', choices=choices_list)
arg1.choices = ['A', 'B', 'D']
Looking at https://github.com/python/typeshed/blob/master/stdlib/2and3/argparse.pyi
# undocumented
class _ActionsContainer:
def add_argument(self,
...
**kwargs: Any) -> Action: ...
def add_argument_group(self, *args: Any, **kwargs: Any) -> _ArgumentGroup: ...
I don't use pycharm, but I'm puzzled as to why it would complain about add_argument, but not about add_argument_group.
The argparse docs doesn't mention the returned Action object, but that's because users don't usually need to access it. But when tested interactively it's pretty obvious:
In [93]: import argparse
In [94]: parser = argparse.ArgumentParser()
In [95]: parser.add_argument('--foo', choices=['one','two'])
Out[95]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)
In [96]: _.choices
Out[96]: ['one', 'two']
Previous SO answers have pointed out that the Actions are also available in 'hidden' _actions list. But in Python, that '_' is just an informal convention; the interpreter doesn't enforce privacy.
In [98]: parser._actions
Out[98]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)]
What's displayed is the string representation of the Action object. Those are the most commonly used attributes.
The documentation for argparse is not a formal API reference; it's too incomplete for that. It's more of a advanced how-to document, more involved than a tutorial, but not as complete as a formal specification.
Related
Let's say you have an argument parser like below, with a usual workflow that parses CLI args via parser.parse_args().
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
However, I also may want to bypass argument parsing and supply args directly via a dictonary, which may not contain optional arguments. All missing arguments should be supplied by the parser defaults.
I.e., desirable scenario:
mydict={"arg1": "Supplied value"} # only supplied non-optional args
args = somehow_resolve_this(parser, mydict)
# this should now work
args.arg1
# Supplied value
print(args.arg2)
# Some default value
An equivalent question would be: How can I obtain all optional argument names and default values from the parser?
You might convert your dict into argparse.Namespace and then feed it into .parse_args as follows
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
namespace = argparse.Namespace(**{"arg1":"value"})
parsed = parser.parse_args(namespace=namespace)
print(parsed.arg1) # value
print(parsed.arg2) # some default value
Explanation: convert dict into kwargs for argparse.Namespace using unpacking ** then feed it into parser.parse_args
As long as none of the arguments is required (positional or flagged), we can get the default values with:
In [3]: args = parser.parse_args([])
In [4]: args
Out[4]: Namespace(arg1=None, arg2='some default value')
In [5]: vars(args)
Out[5]: {'arg1': None, 'arg2': 'some default value'}
And we can 'update' that with your dict:
In [6]: mydict = {"arg1": "Supplied value"}
In [7]: vars(args).update(mydict)
In [8]: args
Out[8]: Namespace(arg1='Supplied value', arg2='some default value')
Using the idea of creating a namespace and passing that to the parser:
In [17]: ns = argparse.Namespace(**mydict)
In [18]: ns
Out[18]: Namespace(arg1='Supplied value')
In [19]: parser.parse_args([], namespace=ns)
Out[19]: Namespace(arg1='Supplied value', arg2='some default value')
Here I supplied the [] argv. That could be omitted if you still want to read the users input. This use of a namespace parameter in effect sets/replaces all the defaults. (this could fail, though, if you are using subparsers).
What if there are required arguments? Defaults don't matter with required arguments.
Another way to get the defaults, is to extract them from the actions list:
In [20]: parser._actions
Out[20]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['-arg1'], dest='arg1', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None),
_StoreAction(option_strings=['-arg2'], dest='arg2', nargs=None, const=None, default='some default value', type=None, choices=None, help=None, metavar=None)]
In [21]: {a.dest: a.default for a in parser._actions}
Out[21]: {'help': '==SUPPRESS==', 'arg1': None, 'arg2': 'some default value'}
Is there a way to check whether an argument with a given name has been added to an argparse instance? For instance, I'd expect something like this to be available:
argToCheck= 'my_argument'
if (argToCheck in parser.known_arguments): # "known_arguments" isn't a thing, but it should be?
# do some magic
else:
# do some different magic
I'd strongly suspect that all of the added arguments are the keys to a dict buried somewhere in the argparse, possibly even one that's intentionally exposed... but I haven't been able to find them...
Background...
I have an argparse parser defined with about a dozen optional arguments:
def jumpParser():
parser = argparse.ArgumentParser()
parser.add_argument("-i", "--input_file", action="store", type=str, required=True, help="input video file")
parser.add_argument("-o", "--output_file", action="store", type=str, required=True, help="output video file")
parser.add_argument("-s", "--stamp", action="store_true", help="enable frame stamping")
# and so on...
return parser
Separately, I have some code that parses an excel worksheet to call a function with a different set of arguments for each row of the XLS, with the column headers being the argument names.
Ideally, I'd like for the user to be able to have columns with headers that are NOT strictly known arguments to a specific parser, so it just skips those columns and calls the function with the arguments from that row which ARE known to a specific parser ...
You can access arguments that have been added to a parser with parser._actions.
[action.dest for action in parser._actions]
This will give you a list of
['help', 'input_file', 'output_file', 'stamp']
Your parser with a few tweaks. Note that add_argument returns an Action object, which can be ignored, or assigned to a variable:
In [19]: import argparse
In [20]: parser = argparse.ArgumentParser()
...: a1 = parser.add_argument("-i", "--input_file", action="store", type=str, help="input video file")
...: a2 = parser.add_argument("-o", "--output_file", action="store", type=str, help="output video file")
...: a3 = parser.add_argument("-s", "--stamp", action="store_true", help="enable frame stamping")
...: a4 = parser.add_argument("foobar")
What the repr of an Action looks like. These are the main attributes, but not all.
In [21]: a1
Out[21]: _StoreAction(option_strings=['-i', '--input_file'], dest='input_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='input video file', metavar=None)
In [22]: a3
Out[22]: _StoreTrueAction(option_strings=['-s', '--stamp'], dest='stamp', nargs=0, const=True, default=False, type=None, choices=None, help='enable frame stamping', metavar=None)
In [23]: a4
Out[23]: _StoreAction(option_strings=[], dest='foobar', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
The parser itself is an object, with methods and attributes.
In [24]: parser._actions
Out[24]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['-i', '--input_file'], dest='input_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='input video file', metavar=None),
_StoreAction(option_strings=['-o', '--output_file'], dest='output_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='output video file', metavar=None),
_StoreTrueAction(option_strings=['-s', '--stamp'], dest='stamp', nargs=0, const=True, default=False, type=None, choices=None, help='enable frame stamping', metavar=None),
_StoreAction(option_strings=[], dest='foobar', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)]
We can test the parser with appropriate list of strings, simulating the values submitted via the commandline.
In [25]: args = parser.parse_args([])
usage: ipython3 [-h] [-i INPUT_FILE] [-o OUTPUT_FILE] [-s] foobar
ipython3: error: the following arguments are required: foobar
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
With enough required arguments, the resulting Namespace shows all the defined arguments (unless some are SUPRESSED):
In [26]: args = parser.parse_args(['xxx'])
In [27]: args
Out[27]: Namespace(foobar='xxx', input_file=None, output_file=None, stamp=False)
We can look at this namespace object as a dictionary, e.g.
In [28]: vars(args)
Out[28]: {'input_file': None, 'output_file': None, 'stamp': False, 'foobar': 'xxx'}
In [29]: list(vars(args).keys())
Out[29]: ['input_file', 'output_file', 'stamp', 'foobar']
Those keys correspond to the argument dest attributes.
I'm using a subparser/subcommand that has an alias.
I'm using the dest option for the subparser to store the name of the subcommand so I can get it later.
Currently if the subcommand's name is reallyLongName and the alias is r (say) then the dest option stores either reallyLongName or r exactly - whatever I typed in gets stored. This is annoying because I now have to check for the name of the command or any of its aliases in order to identify the command.
Is there a way to get argparse to store the subcommand's name in the dest field in some sort of single, canonical text string?
For example, given the following code:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='command', help='sub-command help')
parser_ag = subparsers.add_parser( 'mySubcommand',
aliases=['m'],
help='Subcommand help')
print(parser.parse_args('mySubcommand'.split()))
print(parser.parse_args('m'.split()))
the following output is produced:
Namespace(command='mySubcommand')
Namespace(command='m')
Desired result: command has a single, canonical value for both, for example:
Namespace(command='mySubcommand')
Namespace(command='mySubcommand')
There was a Python bug/issue requesting this - saving the 'base' name, rather than the alias. You can't change that without changing argparse.py code. I think the change would limited to the Action subclass that handles subparsers. https://bugs.python.org/issue36664
But I point out that there's simpler way of handling this. Just use set_defaults as documented near the end of the https://docs.python.org/3/library/argparse.html#sub-commands section. There
parser_foo.set_defaults(func=foo)
is used to set a subparser specific function, but it could just as well be used to set the 'base' name.
parser_foo.set_defaults(name='theIncrediblyLongAlias')
This was surprisingly difficult to dig out. When you add a subparser, it gets stored in the parents ._actions attribute. From there it is just digging through attributes to get what you need. Below I create dictionaries to reference the subparser arguments by the dest name, and then added a function that lets us remap the inputted arguments to the primary argument name.
from collections import defaultdict
def get_subparser_aliases(parser, dest):
out = defaultdict(list)
prog_str = parser.prog
dest_dict = {a.dest: a for a in parser._actions}
try:
choices = dest_dict.get(dest).choices
except AttributeError:
raise AttributeError(f'The parser "{parser}" has no subparser with a `dest` of "{dest}"')
for k, v in choices.items():
clean_v = v.prog.replace(prog_str, '', 1).strip()
out[clean_v].append(k)
return dict(out)
def remap_args(args, mapping, dest):
setattr(args, dest, mapping.get(getattr(args, dest)))
return args
Using your example, we can remap the parse args using:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='command', help='sub-command help')
parser_ag = subparsers.add_parser('mySubcommand',
aliases=['m'],
help='Subcommand help')
args = parser.parse_args('m'.split())
mapping = get_subparser_aliases(parser, 'command')
remap_args(args, mapping, 'command')
print(args)
# prints:
Namespace(command='mySubcommand')
Here is an example of it at work with multiple subparser levels.. We have a parser with an optional argument and a subparser. The subparser has 3 possible arguments, the last of which invoke another subparser (a sub-subparser), with 2 possible arguments.
You can examine either the top level parser or the first level subparser to see alias mappings.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--someoption', '-s', action='store_true')
subparser1 = parser.add_subparsers(help='sub-command help', dest='sub1')
parser_r = subparser1.add_parser('reallyLongName', aliases=['r'])
parser_r.add_argument('foo', type=int, help='foo help')
parser_s = subparser1.add_parser('otherReallyLong', aliases=['L'])
parser_s.add_argument('bar', choices='abc', help='bar help')
parser_z = subparser1.add_parser('otherOptions', aliases=['oo'])
subparser2 = parser_z.add_subparsers(help='sub-sub-command help', dest='sub2')
parser_x = subparser2.add_parser('xxx', aliases=['x'])
parser_x.add_argument('fizz', type=float, help='fizz help')
parser_y = subparser2.add_parser('yyy', aliases=['y'])
parser_y.add_argument('blip', help='blip help')
get_subparser_aliases(parser, 'sub1')
# returns:
{'reallyLongName': ['reallyLongName', 'r'],
'otherReallyLong': ['otherReallyLong', 'L'],
'otherOptions': ['otherOptions', 'oo']}
get_subparser_aliases(parser_z, 'sub2')
# returns:
{'xxx': ['xxx', 'x'], 'yyy': ['yyy', 'y']}
Using this with the function above, we can remap the collected args to their longer names.
args = parser.parse_args('-s oo x 1.23'.split())
print(args)
# prints:
Namespace(fizz=1.23, someoption=True, sub1='oo', sub2='x')
for p, dest in zip((parser, parser_z), ('sub1', 'sub2')):
mapping = get_subparser_aliases(p, dest)
remap_args(args, mapping, dest)
print(args)
# prints:
Namespace(fizz=1.23, someoption=True, sub1='otherOptions', sub2='xxx')
I tried using argparse to learn how it works to parse a given list:
parser = argparse.ArgumentParser()
parser.add_argument('--ls', nargs='*', type=str, default = [])
Out[92]: _StoreAction(option_strings=['--ls'], dest='ls', nargs='*', const=None, default=[], type=<type 'str'>, choices=None, help=None, metavar=None)
args = parser.parse_args("--ls 'tomato' 'jug' 'andes'".split())
args
Out[94]: Namespace(ls=["'tomato'", "'jug'", "'andes'"])
args.ls
Out[96]: ["'tomato'", "'jug'", "'ande'"]
args.ls[0]
Out[97]: "'tomato'"
eval(args.ls[0])
Out[98]: 'tomato'
Q1: The above works but Is there a better way to access values in the list?
Then I tried it with dictionary to parse a dictionary given:
dict_parser = argparse.ArgumentParser()
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
Out[104]: _StoreAction(option_strings=['--dict'], dest='dict', nargs='*', const=None, default={}, type=<type 'dict'>, choices=None, help=None, metavar=None)
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
usage: -c [-h] [--dict [DICT [DICT ...]]]
-c: error: unrecognized arguments: - - d i c t { ' n a m e ' : ' m a n' , ' a d d r e s s ' : ' k a c' , ' t a g s ' : ' d o n ' }
To exit: use 'exit', 'quit', or Ctrl-D.
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
And that doesn't work.
Q2: How does the above work for dictionary?
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
How do I do that?
I referred to http://parezcoydigo.wordpress.com/2012/08/04/from-argparse-to-dictionary-in-python-2-7/
...and found assigning everything under a dictionary is pretty useful. Could somebody simplify this task so as to parse multiple datatypes in the arguments?
parser.add_argument('--ls', nargs='*', type=str, default = [])
Q1: The above works but Is there a better way to access values in the list?
As I often consider "simpler is better", and this is a really simple way to do things, I'd say there is no better way. But still, there are other ways.
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
Q2: How does the above work for dictionary?
I'd advice you to parse your string using json:
>>> class FromJSON():
... def __init__(self, string):
... self.string = string
... def decode(self):
... return json.loads(self.string)
...
>>> dict_parser.add_argument('--dict',type=FromJSON)
>>> arg2 = dict_parser.parse_args(['--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
though JSON is a lot like python, it is not python. It is picky about quoting (values are surrounded by double quotes only) and about (having no) trailing commas. But at least it is safe against code injection! And of course you shall surround your parameter with quotes
You may have anoter solution to give a dict as parameters, it would be to get rid of the brackets:
>>> parser.add_argument('--dict', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--dict', 'name:man', 'address:kac', 'tags:don'])
>>> args.dict = dict([arg.split(':') for arg in args.dict])
>>> print args.dict
{'tags': 'don', 'name': 'man', 'address': 'kac'}
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--dict', type=FromJSON, default=FromJSON("{}"))
>>> parser.add_argument('--ls', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--ls', 'tomato', 'jug', 'andes', '--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
>>> args.ls
['tomato', 'jug', 'andes']
>>> args.dict
<__main__.FromJSON instance at 0x7f932dd20c20>
>>> args.dict.decode()
{u'tags': u'don', u'name': u'man', u'address': u'kac'}
About FromJSON(), it can be improved so it is something like:
class JsonToDict():
def __call__(self, string):
return json.loads(string)
that you could use as follows:
dict_parser.add_argument('--dict',type=JsonToDict())
HTH
import ast
dict_parser.add_argument('--dict', nargs='*',type=ast.literal_eval,default={})
args = dict_parser.parse_args(["--dict", "{'name':'man', 'address': 'kac', 'tags':'don'}"])
I currently have 2 functions in my .py script.
#1 connects to the database and does some processing.
#2 does some other processing on files
Currently before I run the script, I have to manually comment/uncomment the function I want to run in my main if statement block.
How can I use argparse, so it asks me which function to run when I run my script?
It is possible to tell ArgumentParser objects about the function or object that has your desired behavior directly, by means of action='store_const' and const=<stuff> pairs in an add_argument() call, or with a set_defaults() call (the latter is most useful when you're using sub-parsers). If you do that, you can look up your function on the parsed_args object you get back from the parsing, instead of say, looking it up in the global namespace.
As a little example:
import argparse
def foo(parsed_args):
print "woop is {0!r}".format(getattr(parsed_args, 'woop'))
def bar(parsed_args):
print "moop is {0!r}".format(getattr(parsed_args, 'moop'))
parser = argparse.ArgumentParser()
parser.add_argument('--foo', dest='action', action='store_const', const=foo)
parser.add_argument('--bar', dest='action', action='store_const', const=bar)
parser.add_argument('--woop')
parser.add_argument('--moop')
parsed_args = parser.parse_args()
if parsed_args.action is None:
parser.parse_args(['-h'])
parsed_args.action(parsed_args)
And then you can call it like:
% python /tmp/junk.py --foo
woop is None
% python /tmp/junk.py --foo --woop 8 --moop 17
woop is '8'
% python /tmp/junk.py --bar --woop 8 --moop 17
moop is '17'
If it's just a flag of run A or B, then a simple "store_true" argument should be fine.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
_StoreTrueAction(option_strings=['--run_a_not_b'], dest='run_a_not_b', nargs=0, const=True, default=False, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args('--run_a_not_b')
>>> parsed_args = parser.parse_args('--run_a_not_b'.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run a
>>> parsed_args = parser.parse_args(''.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run b
Or if you want to actually pass in the name of the function to call, you can do it this (somewhat hackish) way:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--func_to_run', type=str)
_StoreAction(option_strings=['--func_to_run'], dest='func_to_run', nargs=None, const=None, default=None, type=<type 'str'>, choices=None, help=None, metavar=None)
>>> parsed_args = parser.parse_args('--func_to_run my_other_func'.split())
>>> parsed_args.func_to_run
'my_other_func'
>>> f = globals()[parsed_args.func_to_run]
<function my_other_func at 0x011F6670>
>>> f()
edit : to handle an integer argument, you would simply specify the type
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
>>> parser.add_argument('--func_arg', type=int)
>>> parsed_args = parser.parse_args('--run_a_not_b --arg 42'.split())
>>> parsed_args = parser.parse_args('--run_a_not_b --func_arg 42'.split())
>>> parsed_args
Namespace(func_arg=42, run_a_not_b=True)
So, you can simply get parsed_args.func_arg for the value if you choose in this example.
You might consider using fabric for this.