Let's say you have an argument parser like below, with a usual workflow that parses CLI args via parser.parse_args().
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
However, I also may want to bypass argument parsing and supply args directly via a dictonary, which may not contain optional arguments. All missing arguments should be supplied by the parser defaults.
I.e., desirable scenario:
mydict={"arg1": "Supplied value"} # only supplied non-optional args
args = somehow_resolve_this(parser, mydict)
# this should now work
args.arg1
# Supplied value
print(args.arg2)
# Some default value
An equivalent question would be: How can I obtain all optional argument names and default values from the parser?
You might convert your dict into argparse.Namespace and then feed it into .parse_args as follows
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
namespace = argparse.Namespace(**{"arg1":"value"})
parsed = parser.parse_args(namespace=namespace)
print(parsed.arg1) # value
print(parsed.arg2) # some default value
Explanation: convert dict into kwargs for argparse.Namespace using unpacking ** then feed it into parser.parse_args
As long as none of the arguments is required (positional or flagged), we can get the default values with:
In [3]: args = parser.parse_args([])
In [4]: args
Out[4]: Namespace(arg1=None, arg2='some default value')
In [5]: vars(args)
Out[5]: {'arg1': None, 'arg2': 'some default value'}
And we can 'update' that with your dict:
In [6]: mydict = {"arg1": "Supplied value"}
In [7]: vars(args).update(mydict)
In [8]: args
Out[8]: Namespace(arg1='Supplied value', arg2='some default value')
Using the idea of creating a namespace and passing that to the parser:
In [17]: ns = argparse.Namespace(**mydict)
In [18]: ns
Out[18]: Namespace(arg1='Supplied value')
In [19]: parser.parse_args([], namespace=ns)
Out[19]: Namespace(arg1='Supplied value', arg2='some default value')
Here I supplied the [] argv. That could be omitted if you still want to read the users input. This use of a namespace parameter in effect sets/replaces all the defaults. (this could fail, though, if you are using subparsers).
What if there are required arguments? Defaults don't matter with required arguments.
Another way to get the defaults, is to extract them from the actions list:
In [20]: parser._actions
Out[20]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['-arg1'], dest='arg1', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None),
_StoreAction(option_strings=['-arg2'], dest='arg2', nargs=None, const=None, default='some default value', type=None, choices=None, help=None, metavar=None)]
In [21]: {a.dest: a.default for a in parser._actions}
Out[21]: {'help': '==SUPPRESS==', 'arg1': None, 'arg2': 'some default value'}
Related
Is there a way to check whether an argument with a given name has been added to an argparse instance? For instance, I'd expect something like this to be available:
argToCheck= 'my_argument'
if (argToCheck in parser.known_arguments): # "known_arguments" isn't a thing, but it should be?
# do some magic
else:
# do some different magic
I'd strongly suspect that all of the added arguments are the keys to a dict buried somewhere in the argparse, possibly even one that's intentionally exposed... but I haven't been able to find them...
Background...
I have an argparse parser defined with about a dozen optional arguments:
def jumpParser():
parser = argparse.ArgumentParser()
parser.add_argument("-i", "--input_file", action="store", type=str, required=True, help="input video file")
parser.add_argument("-o", "--output_file", action="store", type=str, required=True, help="output video file")
parser.add_argument("-s", "--stamp", action="store_true", help="enable frame stamping")
# and so on...
return parser
Separately, I have some code that parses an excel worksheet to call a function with a different set of arguments for each row of the XLS, with the column headers being the argument names.
Ideally, I'd like for the user to be able to have columns with headers that are NOT strictly known arguments to a specific parser, so it just skips those columns and calls the function with the arguments from that row which ARE known to a specific parser ...
You can access arguments that have been added to a parser with parser._actions.
[action.dest for action in parser._actions]
This will give you a list of
['help', 'input_file', 'output_file', 'stamp']
Your parser with a few tweaks. Note that add_argument returns an Action object, which can be ignored, or assigned to a variable:
In [19]: import argparse
In [20]: parser = argparse.ArgumentParser()
...: a1 = parser.add_argument("-i", "--input_file", action="store", type=str, help="input video file")
...: a2 = parser.add_argument("-o", "--output_file", action="store", type=str, help="output video file")
...: a3 = parser.add_argument("-s", "--stamp", action="store_true", help="enable frame stamping")
...: a4 = parser.add_argument("foobar")
What the repr of an Action looks like. These are the main attributes, but not all.
In [21]: a1
Out[21]: _StoreAction(option_strings=['-i', '--input_file'], dest='input_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='input video file', metavar=None)
In [22]: a3
Out[22]: _StoreTrueAction(option_strings=['-s', '--stamp'], dest='stamp', nargs=0, const=True, default=False, type=None, choices=None, help='enable frame stamping', metavar=None)
In [23]: a4
Out[23]: _StoreAction(option_strings=[], dest='foobar', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
The parser itself is an object, with methods and attributes.
In [24]: parser._actions
Out[24]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['-i', '--input_file'], dest='input_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='input video file', metavar=None),
_StoreAction(option_strings=['-o', '--output_file'], dest='output_file', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, help='output video file', metavar=None),
_StoreTrueAction(option_strings=['-s', '--stamp'], dest='stamp', nargs=0, const=True, default=False, type=None, choices=None, help='enable frame stamping', metavar=None),
_StoreAction(option_strings=[], dest='foobar', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)]
We can test the parser with appropriate list of strings, simulating the values submitted via the commandline.
In [25]: args = parser.parse_args([])
usage: ipython3 [-h] [-i INPUT_FILE] [-o OUTPUT_FILE] [-s] foobar
ipython3: error: the following arguments are required: foobar
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
With enough required arguments, the resulting Namespace shows all the defined arguments (unless some are SUPRESSED):
In [26]: args = parser.parse_args(['xxx'])
In [27]: args
Out[27]: Namespace(foobar='xxx', input_file=None, output_file=None, stamp=False)
We can look at this namespace object as a dictionary, e.g.
In [28]: vars(args)
Out[28]: {'input_file': None, 'output_file': None, 'stamp': False, 'foobar': 'xxx'}
In [29]: list(vars(args).keys())
Out[29]: ['input_file', 'output_file', 'stamp', 'foobar']
Those keys correspond to the argument dest attributes.
Using argparse, is there any way to update the "choices" option of an argument after it was added to the parser ? Argparse documentation doesn't yield much about updating the choices
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
parser.add_argument('arg1', choices=choices_list)
# The list of choices now changes
choices_list = ['A', 'C', 'D']
# Some code to update 'arg1' choices option ?
parser.???
I tried using 'parser.add_argument' with the new 'choices_list', but it creates duplicate arguments.
Using Python 3.7
The argument itself has a choices attribute, but it's easiest if you save a reference to the argument instead of trying to retrieve it from the parser itself. (Otherwise, you have to scan through the private attribute parser._actions and try to identify which one you need.)
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
arg1 = parser.add_argument('arg1', choices=choices_list)
arg1.choices = ['A', 'B', 'D']
Looking at https://github.com/python/typeshed/blob/master/stdlib/2and3/argparse.pyi
# undocumented
class _ActionsContainer:
def add_argument(self,
...
**kwargs: Any) -> Action: ...
def add_argument_group(self, *args: Any, **kwargs: Any) -> _ArgumentGroup: ...
I don't use pycharm, but I'm puzzled as to why it would complain about add_argument, but not about add_argument_group.
The argparse docs doesn't mention the returned Action object, but that's because users don't usually need to access it. But when tested interactively it's pretty obvious:
In [93]: import argparse
In [94]: parser = argparse.ArgumentParser()
In [95]: parser.add_argument('--foo', choices=['one','two'])
Out[95]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)
In [96]: _.choices
Out[96]: ['one', 'two']
Previous SO answers have pointed out that the Actions are also available in 'hidden' _actions list. But in Python, that '_' is just an informal convention; the interpreter doesn't enforce privacy.
In [98]: parser._actions
Out[98]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)]
What's displayed is the string representation of the Action object. Those are the most commonly used attributes.
The documentation for argparse is not a formal API reference; it's too incomplete for that. It's more of a advanced how-to document, more involved than a tutorial, but not as complete as a formal specification.
When module optparse is used, then I can get all default values for all command line arguments like this:
import optparse
if __name__ == '__main__':
parser = optparse.OptionParser(usage='pokus --help')
parser.add_option("-d", "--debug", action='store_true', dest="debug",
default=False, help='Enabling debugging.')
options, args = parser.parse_args()
print(parser.defaults)
Since optparse is deprecated it is wise to rewrite your code to use argparse module. However I can't find any way how to get all default values of all command line arguments added to parser object:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(usage='pokus --help')
parser.add_argument("-d", "--debug", action='store_true', dest='debug',
default=False, help='Enabling debugging.')
args = parser.parse_args()
# <---- How to get default values for all arguments here?
# Not: vars(args)
I want to get all default values when I run program with (./app.py -d) or without any command line argument (./app.py).
I found solution:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(usage='pokus --help')
parser.add_argument("-d", "--debug", action='store_true', dest='debug',
default=False, help='Enabling debugging.')
parser.add_argument("-e", "--example", action='store', dest='example',
default="", help='Example of argument.')
# Arguments from command line and default values
args = vars(parser.parse_args())
# Only default values
defaults = vars(parser.parse_args([]))
Then you can compare args and defaults values and distinguish between default values and values from command line.
If you do not want to parse an empty input string, you can use the method get_default in the parser object:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(usage='pokus --help')
parser.add_argument("-d", "--debug", action='store_true', dest='debug',
default=False, help='Enabling debugging.')
args = parser.parse_args()
# To get a single default:
d_default = parser.get_default('d')
# To get all defaults:
all_defaults = {}
for key in vars(args):
all_defaults[key] = parser.get_default(key)
# Edit: Adding an alternative one-liner (using dict comprehension):
all_defaults = {key: parser.get_default(key) for key in vars(args)}
Somewhat late to the party, but this is a function (with bonus unittest) that I've used in a couple of cases to get hold of the default arguments without having to parse first (parsing first can be annoying if you have required arguments that aren't available yet)
def get_argparse_defaults(parser):
defaults = {}
for action in parser._actions:
if not action.required and action.dest != "help":
defaults[action.dest] = action.default
return defaults
def get_argparse_required(parser):
required = []
for action in parser._actions:
if action.required:
required.append(action.dest)
return required
parser = argparse.ArgumentParser()
optional_defaults_dict = get_argparse_defaults(parser)
required_list = get_argparse_required(parser)
class TestDefaultArgs(unittest.TestCase):
def test_get_args(self):
parser = argparse.ArgumentParser()
parser.add_argument('positional_arg')
parser.add_argument('--required_option', required=True)
parser.add_argument('--optional_with_default', required=False, default="default_value")
parser.add_argument('--optional_without_default', required=False)
required_args = get_argparse_required(parser)
self.assertEqual(['positional_arg', 'required_option'], required_args)
default_args = get_argparse_defaults(parser)
self.assertEqual({'optional_with_default': 'default_value',
'optional_without_default': None},
default_args)
For your information, here's the code, at the start of parsing that initializes the defaults:
def parse_known_args(...):
....
# add any action defaults that aren't present
for action in self._actions:
if action.dest is not SUPPRESS:
if not hasattr(namespace, action.dest):
if action.default is not SUPPRESS:
setattr(namespace, action.dest, action.default)
# add any parser defaults that aren't present
for dest in self._defaults:
if not hasattr(namespace, dest):
setattr(namespace, dest, self._defaults[dest])
...
So it loops through the parser._actions list, collecting the action.default attribute. (An action is a Action class object that was created by the parser.add_argument method.). It also checks self._defaults. This is the dictionary modified by a parse.set_defaults method. That can be used to set defaults that aren't linked directly to an action.
After parsing the command line, default strings in the namespace may be evaluated (with the action.type), turning, for example a default='1' into an integer 1.
Handling of defaults in argparse isn't trivial. Your parse_args([]) probably is simplest, provided the parser is ok with that (i.e. doesn't have any required arguments).
I don't know now optparse sets the defaults attribute. There is a non-trival method, optparse.OptionParser.get_default_values.
For the above example:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(usage='pokus --help')
parser.add_argument("-d", "--debug", action='store_true', dest='debug',
default=False, help='Enabling debugging.')
A. To get all the values with their defaults in a tuple format:
In[1]: args = parser.parse_known_args()[0]
In[2]: args._get_kwargs()
Out[1]: [('debug', False)]
to access to each item:
In[3]: args.debug
Out[2]: False
B. To get the values and their default as dictionary format
In[4]: dict_args = parser.parse_known_args()[0].__dict__
In[5]: dict_args
Out[3]: {'debug': False}
And to access each key:
In[6]: dict_args['debug']
Out[4]: False
Or print them iteratively:
In[7]: for key in dict_args:
... print('value for %s is: %s'% (key, dict_args[key]))
Out[5]: value for debug is: False
I tried using argparse to learn how it works to parse a given list:
parser = argparse.ArgumentParser()
parser.add_argument('--ls', nargs='*', type=str, default = [])
Out[92]: _StoreAction(option_strings=['--ls'], dest='ls', nargs='*', const=None, default=[], type=<type 'str'>, choices=None, help=None, metavar=None)
args = parser.parse_args("--ls 'tomato' 'jug' 'andes'".split())
args
Out[94]: Namespace(ls=["'tomato'", "'jug'", "'andes'"])
args.ls
Out[96]: ["'tomato'", "'jug'", "'ande'"]
args.ls[0]
Out[97]: "'tomato'"
eval(args.ls[0])
Out[98]: 'tomato'
Q1: The above works but Is there a better way to access values in the list?
Then I tried it with dictionary to parse a dictionary given:
dict_parser = argparse.ArgumentParser()
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
Out[104]: _StoreAction(option_strings=['--dict'], dest='dict', nargs='*', const=None, default={}, type=<type 'dict'>, choices=None, help=None, metavar=None)
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
usage: -c [-h] [--dict [DICT [DICT ...]]]
-c: error: unrecognized arguments: - - d i c t { ' n a m e ' : ' m a n' , ' a d d r e s s ' : ' k a c' , ' t a g s ' : ' d o n ' }
To exit: use 'exit', 'quit', or Ctrl-D.
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
And that doesn't work.
Q2: How does the above work for dictionary?
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
How do I do that?
I referred to http://parezcoydigo.wordpress.com/2012/08/04/from-argparse-to-dictionary-in-python-2-7/
...and found assigning everything under a dictionary is pretty useful. Could somebody simplify this task so as to parse multiple datatypes in the arguments?
parser.add_argument('--ls', nargs='*', type=str, default = [])
Q1: The above works but Is there a better way to access values in the list?
As I often consider "simpler is better", and this is a really simple way to do things, I'd say there is no better way. But still, there are other ways.
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
Q2: How does the above work for dictionary?
I'd advice you to parse your string using json:
>>> class FromJSON():
... def __init__(self, string):
... self.string = string
... def decode(self):
... return json.loads(self.string)
...
>>> dict_parser.add_argument('--dict',type=FromJSON)
>>> arg2 = dict_parser.parse_args(['--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
though JSON is a lot like python, it is not python. It is picky about quoting (values are surrounded by double quotes only) and about (having no) trailing commas. But at least it is safe against code injection! And of course you shall surround your parameter with quotes
You may have anoter solution to give a dict as parameters, it would be to get rid of the brackets:
>>> parser.add_argument('--dict', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--dict', 'name:man', 'address:kac', 'tags:don'])
>>> args.dict = dict([arg.split(':') for arg in args.dict])
>>> print args.dict
{'tags': 'don', 'name': 'man', 'address': 'kac'}
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--dict', type=FromJSON, default=FromJSON("{}"))
>>> parser.add_argument('--ls', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--ls', 'tomato', 'jug', 'andes', '--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
>>> args.ls
['tomato', 'jug', 'andes']
>>> args.dict
<__main__.FromJSON instance at 0x7f932dd20c20>
>>> args.dict.decode()
{u'tags': u'don', u'name': u'man', u'address': u'kac'}
About FromJSON(), it can be improved so it is something like:
class JsonToDict():
def __call__(self, string):
return json.loads(string)
that you could use as follows:
dict_parser.add_argument('--dict',type=JsonToDict())
HTH
import ast
dict_parser.add_argument('--dict', nargs='*',type=ast.literal_eval,default={})
args = dict_parser.parse_args(["--dict", "{'name':'man', 'address': 'kac', 'tags':'don'}"])
I currently have 2 functions in my .py script.
#1 connects to the database and does some processing.
#2 does some other processing on files
Currently before I run the script, I have to manually comment/uncomment the function I want to run in my main if statement block.
How can I use argparse, so it asks me which function to run when I run my script?
It is possible to tell ArgumentParser objects about the function or object that has your desired behavior directly, by means of action='store_const' and const=<stuff> pairs in an add_argument() call, or with a set_defaults() call (the latter is most useful when you're using sub-parsers). If you do that, you can look up your function on the parsed_args object you get back from the parsing, instead of say, looking it up in the global namespace.
As a little example:
import argparse
def foo(parsed_args):
print "woop is {0!r}".format(getattr(parsed_args, 'woop'))
def bar(parsed_args):
print "moop is {0!r}".format(getattr(parsed_args, 'moop'))
parser = argparse.ArgumentParser()
parser.add_argument('--foo', dest='action', action='store_const', const=foo)
parser.add_argument('--bar', dest='action', action='store_const', const=bar)
parser.add_argument('--woop')
parser.add_argument('--moop')
parsed_args = parser.parse_args()
if parsed_args.action is None:
parser.parse_args(['-h'])
parsed_args.action(parsed_args)
And then you can call it like:
% python /tmp/junk.py --foo
woop is None
% python /tmp/junk.py --foo --woop 8 --moop 17
woop is '8'
% python /tmp/junk.py --bar --woop 8 --moop 17
moop is '17'
If it's just a flag of run A or B, then a simple "store_true" argument should be fine.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
_StoreTrueAction(option_strings=['--run_a_not_b'], dest='run_a_not_b', nargs=0, const=True, default=False, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args('--run_a_not_b')
>>> parsed_args = parser.parse_args('--run_a_not_b'.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run a
>>> parsed_args = parser.parse_args(''.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run b
Or if you want to actually pass in the name of the function to call, you can do it this (somewhat hackish) way:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--func_to_run', type=str)
_StoreAction(option_strings=['--func_to_run'], dest='func_to_run', nargs=None, const=None, default=None, type=<type 'str'>, choices=None, help=None, metavar=None)
>>> parsed_args = parser.parse_args('--func_to_run my_other_func'.split())
>>> parsed_args.func_to_run
'my_other_func'
>>> f = globals()[parsed_args.func_to_run]
<function my_other_func at 0x011F6670>
>>> f()
edit : to handle an integer argument, you would simply specify the type
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
>>> parser.add_argument('--func_arg', type=int)
>>> parsed_args = parser.parse_args('--run_a_not_b --arg 42'.split())
>>> parsed_args = parser.parse_args('--run_a_not_b --func_arg 42'.split())
>>> parsed_args
Namespace(func_arg=42, run_a_not_b=True)
So, you can simply get parsed_args.func_arg for the value if you choose in this example.
You might consider using fabric for this.