I would like to get subset of parsed arguments and send them to another function in python. I found this argument_group idea but I couldn't find to reach argument groups. This is what I want to try to do:
import argparse
def some_function(args2):
x = args2.bar_this
print(x)
def main():
parser = argparse.ArgumentParser(description='Simple example')
parser.add_argument('--name', help='Who to greet', default='World')
# Create two argument groups
foo_group = parser.add_argument_group(title='Foo options')
bar_group = parser.add_argument_group(title='Bar options')
# Add arguments to those groups
foo_group.add_argument('--bar_this')
foo_group.add_argument('--bar_that')
bar_group.add_argument('--foo_this')
bar_group.add_argument('--foo_that')
args = parser.parse_args()
# How can I get the foo_group arguments for example only ?
args2 = args.foo_group
some_function(args2)
I don't know whether a simpler solution exists, but you can create a custom "namespace" object selecting only the keys arguments you need from the parsed arguments.
args2 = argparse.Namespace(**{k: v for k, v in args._get_kwargs()
if k.startswith("foo_")})
You can customize the if clause to your needs and possibly change the argument names k, e.g. removing the foo_ prefix.
Question: What is the intended / official way of accessing possible arguments from an existing argparse.ArgumentParser object?
Example: Let's assume the following context:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
Here I'd like to get the following list of allowed arguments:
['-h', '--foo', '--help', '-f']
I found the following workaround which does the trick for me
parser._option_string_actions.keys()
But I'm not happy with it, as it involves accessing a _-member that is not officially documented. Whats the correct alternative for this task?
I don't think there is a "better" way to achieve what you want.
If you really don't want to use the _option_string_actions attribute, you could process the parser.format_usage() to retrieve the options, but doing this, you will get only the short options names.
If you want both short and long options names, you could process the parser.format_help() instead.
This process can be done with a very simple regular expression: -+\w+
import re
OPTION_RE = re.compile(r"-+\w+")
PARSER_HELP = """usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO, -f FOO a random options
--bar BAR, -b BAR a more random option
"""
options = set(OPTION_RE.findall(PARSER_HELP))
print(options)
# set(['-f', '-b', '--bar', '-h', '--help', '--foo'])
Or you could first make a dictionnary which contains the argument parser configuration and then build the argmuent parser from it. Such a dictionnary could have the option names as key and the option configuration as value. Doing this, you can access the options list via the dictionnary keys flattened with itertools.chain:
import argparse
import itertools
parser_config = {
('--foo', '-f'): {"help": "a random options", "type": str},
('--bar', '-b'): {"help": "a more random option", "type": int, "default": 0}
}
parser = argparse.ArgumentParser()
for option, config in parser_config.items():
parser.add_argument(*option, **config)
print(parser.format_help())
# usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
#
# optional arguments:
# -h, --help show this help message and exit
# --foo FOO, -f FOO a random options
# --bar BAR, -b BAR a more random option
print(list(itertools.chain(*parser_config.keys())))
# ['--foo', '-f', '--bar', '-b']
This last way is what I would do, if I was reluctant to use _option_string_actions.
This started as a joke answer, but I've learned something since - so I'll post it.
Assume, we know the maximum length of an option allowed. Here is a nice answer to the question in this situation:
from itertools import combinations
def parsable(option):
try:
return len(parser.parse_known_args(option.split())[1]) != 2
except:
return False
def test(tester, option):
return any([tester(str(option) + ' ' + str(v)) for v in ['0', '0.0']])
def allowed_options(parser, max_len=3, min_len=1):
acceptable = []
for l in range(min_len, max_len + 1):
for option in combinations([c for c in [chr(i) for i in range(33, 127)] if c != '-'], l):
option = ''.join(option)
acceptable += [p + option for p in ['-', '--'] if test(parsable, p + option)]
return acceptable
Of course this is very pedantic as the question doesn't require any specific runtime. So I'll ignore that here. I'll also disregard, that the above version produces a mess of output because one can get rid of it easily.
But more importantly, this method detected the following interesting argparse "features":
In in the OP example, argparse would also allow --fo. This has to be a bug.
But further, in the OP example again, argparse would also allow -fo (ie. setting foo to o without space or anything). This is documented and intended, but I didn't know it.
Because of this, a correct solution is a bit longer and would look something like this (only parsable changes, I'll omit the other methods):
def parsable(option):
try:
default = vars(parser.parse_known_args(['--' + '0' * 200])[0])
parsed, remaining = parser.parse_known_args(option.split())
if len(remaining) == 2:
return False
parsed = vars(parsed)
for k in parsed.keys():
try:
if k in default and default[k] != parsed[k] and float(parsed[k]) != 0.0:
return False # Filter '-fx' cases where '-f' is the argument and 'x' the value.
except:
return False
return True
except:
return False
Summary: Besides all the restrictions (runtime and fixed maximum option length), this is the only answer that correctly respects the real parser behavior - however buggy it may even be. So here you are, a perfect answer that is absolutely useless.
I have to agree with Tryph's answer.
Not pretty, but you can retrieve them from parser.format_help():
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
goal = parser._option_string_actions.keys()
def get_allowed_arguments(parser):
lines = parser.format_help().split('\n')
line_index = 0
number_of_lines = len(lines)
found_optional_arguments = False
# skip the first lines until the section 'optional arguments'
while line_index < number_of_lines:
if lines[line_index] == 'optional arguments:':
found_optional_arguments = True
line_index += 1
break
line_index += 1
result_list = []
if found_optional_arguments:
while line_index < number_of_lines:
arg_list = get_arguments_from_line(lines[line_index])
if len(arg_list) == 0:
break
result_list += arg_list
line_index += 1
return result_list
def get_arguments_from_line(line):
if line[:2] != ' ':
return []
arg_list = []
i = 2
N = len(line)
inside_arg = False
arg_start = 2
while i < N:
if line[i] == '-' and not inside_arg:
arg_start = i
inside_arg = True
elif line[i] in [',',' '] and inside_arg:
arg_list.append(line[arg_start:i+1])
inside_arg = False
i += 1
return arg_list
answer = get_allowed_arguments(parser)
There's probably a regular expressions alternative to the above mess...
First a note on the argparse docs - it's basically a how-to-use document, not a formal API. The standard for what argparse does is the code itself, the unit tests (test/test_argparse.py), and a paralyzing concern for backward compatibility.
There's no 'official' way of accessing allowed arguments, because users usually don't need to know that (other than reading the help/usage).
But let me illustrate with a simple parser in an iteractive session:
In [247]: parser=argparse.ArgumentParser()
In [248]: a = parser.add_argument('pos')
In [249]: b = parser.add_argument('-f','--foo')
add_argument returns the Action object that it created. This isn't documented, but obvious to any one who has created a parser interactively.
The parser object has a repr method, that displays major parameters. But it has many more attributes, which you can see with vars(parser), or parser.<tab> in Ipython.
In [250]: parser
Out[250]: ArgumentParser(prog='ipython3', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
The Actions too have repr; the Action subclass is determined by the action parameter.
In [251]: a
Out[251]: _StoreAction(option_strings=[], dest='pos', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [252]: b
Out[252]: _StoreAction(option_strings=['-f', '--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
vars(a) etc can be used to see all attributes.
A key parser attribute is _actions, a list of all defined Actions. This is the basis for all parsing. Note it includes the help action that was created automatically. Look at option_strings; that determines whether the Action is positional or optional.
In [253]: parser._actions
Out[253]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=[], dest='pos',....),
_StoreAction(option_strings=['-f', '--foo'], dest='foo', ...)]
_option_string_actions is a dictionary, mapping from option_strings to Actions (the same objects that appear in _actions). References to those Action objects appear all over the place in argparse code.
In [255]: parser._option_string_actions
Out[255]:
{'--foo': _StoreAction(option_strings=['-f', '--foo'],....),
'--help': _HelpAction(option_strings=['-h', '--help'],...),
'-f': _StoreAction(option_strings=['-f', '--foo'], dest='foo',...),
'-h': _HelpAction(option_strings=['-h', '--help'], ....)}
In [256]: list(parser._option_string_actions.keys())
Out[256]: ['-f', '--help', '-h', '--foo']
Note that there is a key for each - string, long or short; but there's nothing for pos, the positional has an empty option_strings parameter.
If that list of keys is what you want, use it, and don't worry about the _. It does not have a 'public' alias.
I can understand parsing the help to discover the same; but that's a lot of work to just avoid using a 'private' attribute. If you worry about the undocumented attribute being changed, you should also worry about the help format being changed. That isn't part of the docs either.
help layout is controlled by parser.format_help. The usage is created from information in self._actions. Help lines from information in
for action_group in self._action_groups:
formatter.add_arguments(action_group._group_actions)
(you don't want to get into action groups do you?).
There is another way of getting the option_strings - collect them from the _actions:
In [258]: [a.option_strings for a in parser._actions]
Out[258]: [['-h', '--help'], [], ['-f', '--foo']]
===================
Delving in to code details a bit:
parser.add_argument creates an Action, and then passes it to parser._add_action. This is the method the populates both .actions and action.option_strings.
self._actions.append(action)
for option_string in action.option_strings:
self._option_string_actions[option_string] = action
If I have an optional argument with optional argument value, is there a way to validate if the argument is set when the value is not given?
For instance:
parser = argparse.ArgumentParser()
parser.add_argument('--abc', nargs='?')
args = parser.parse_args()
Would correctly give me:
optional arguments:
--abc [ABC]
How do I distinguish between 1 and 2 below?
'' => args.abc is None
'--abc' => args.abc is still None
'--abc something' => args.abc is something
...
Update:
Found a trick to solve this problem: you can use "nargs='*'" instead of "nargs='?'". This way #1 would return None, and #2 would return an empty list. The downside is this will allow multiple values for the arguments to be accepted too; so you'd need to add a check for it if appropriate.
Alternatively you can also set a default value for the argument; see answer from chepner and Anand S Kumar.
With nargs='?', you can supply both a default and const.
In [791]: parser=argparse.ArgumentParser()
In [792]: parser.add_argument('--abc', nargs='?', default='default', const='const')
If the argument is not given it uses the default:
In [793]: parser.parse_args([])
Out[793]: Namespace(abc='default')
If given, but without an argument string, it uses the const:
In [794]: parser.parse_args(['--abc'])
Out[794]: Namespace(abc='const')
Otherwise it uses the argument string:
In [795]: parser.parse_args(['--abc','test'])
Out[795]: Namespace(abc='test')
In [796]: parser.print_help()
usage: ipython3 [-h] [--abc [ABC]]
optional arguments:
-h, --help show this help message and exit
--abc [ABC]
Use a different default value for the option. Compare
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--abc', nargs='?', default="default")
>>> parser.parse_args()
Namespace(abc='default')
>>> parser.parse_args(['--abc'])
Namespace(abc=None)
>>> parser.parse_args(['--abc', 'value'])
Namespace(abc='value')
I'm not sure how you would provide a different value for when --abc is used without an argument, short of using a custom action instead of the nargs argument.
Not sure if this is the standard way, but you can set default argument to something , and then that value would be used in case --abc is not in the argument list.
Example code -
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--abc', nargs='?', default="-1")
args = parser.parse_args()
print(args)
Result -
>python a.py
Namespace(abc='-1')
>python a.py --abc
Namespace(abc=None)
>python a.py --abc something
Namespace(abc='something')
I'm using this to have a command-line arg for multiprocessing. Specifying --multi uses all cores and given an arg specifies a number of cores, e.g., --multi 4 for four cores.
parser.add_argument("-mp", "--multi", type=int, nargs="*", help=multi_text)
Parsing logic is then:
if (args.multi == None):
num_cores = 1
elif (args.multi == []):
num_cores = multiprocessing.cpu_count()
elif (len(args.multi) == 1):
num_cores = args.multi[0]
else:
print("Invalid specification of core usage.")
sys.exit(1)
Just wondering if it is possible to use both an optional argument in the same function as multiple arguments. I've looked around and I feel as if I just have the vocabulary wrong or something. Example:
def pprint(x, sub = False, *Headers):
pass
Can I call it still using the multiple headers without having to always put True or False in for sub? I feel like it's a no because Headers wouldn't know where it begins. I'd like to explicitly state that sub = True otherwise it defaults to False.
In Python 3, use:
def pprint(x, *headers, sub=False):
pass
putting the keyword arguments after the positionals. This syntax will not work in Python 2.
Demo:
>>> def pprint(x, *headers, sub=False):
... print(x, headers, sub)
...
>>> pprint('foo', 'bar', 'baz', sub=True)
foo ('bar', 'baz') True
>>> pprint('foo', 'bar', 'baz')
foo ('bar', 'baz') False
You must specify a different value for sub using a keyword argument when calling the pprint() function defined here.
I want to say yes because lots of matplotlib (for example) methods have something similar to this...
For example,
matplotlib.pyplot.xcorr(x, y, normed=True, detrend=<function detrend_none at 0x2523ed8>, usevlines=True, maxlags=10, hold=None, **kwargs)
When I'm using this I can specify any of the keyword arguments by saying maxlags=20 for example. You do have to specify all the non-keyworded arguments (so x in your case) before the keyword arguments.
Just do the following:
def pprint(x, **kwargs):
sub = kwargs.get('sub', False)
headers = kwargs.get('headers', [])
I have a Python script that reads a file containing a command line invocation of some other tool. I'd like to modify the options of this invocation before calling the tool. For example, I might transform:
my_util --input file1.txt --option1 red --option2 blue
...to this:
my_util --input file1_001.txt --option1 red --option3 green
(More accurately, I'd be working on the arguments as lists.)
I figured that using the argparse module would be the easiest way to do this: I could parse the args, change, add or remove the options as I need to, and then reconstruct the command line.
But how do I do the last step? Given the Namespace object returned by parse_args(), can I easily reconstruct a list of command line options, such as could be passed to subprocess.Popen()?
A Namespace object is just a simple object subclass, so you can get the values out as a dict with vars:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> args = parser.parse_args(['--foo', 'BAR'])
>>> vars(args)
{'foo': 'BAR'}
Or you can assign to a class directly and get the arguments out as class variables:
>>> class C(object):
... pass
...
>>> c = C()
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> parser.parse_args(args=['--foo', 'BAR'], namespace=c)
>>> c.foo
'BAR'
It would be fairly easy to use either of these structures to test/replace arguments and pass the results to Popen.
I know this is an old question, but I've just encountered the same problem. I realized that all I need is a way to iterate over the Action objects. Unfortunately, the internal list is not exposed by ArgParser itself. However, these objects are returned by add_argument(), so I can construct my own list. Well, putting actions.append() around each call looked like too much typing to me, so I store all options in a tuple:
def add_argument(*args, **kwargs):
return (args, kwargs)
parser = argparse.ArgumentParser()
options = (
add_argument('--verbose', action='store_true'),
add_argument('--author'),
add_argument('--subject', required=True),
add_argument('--cache', nargs='?'),
add_argument('files', nargs=''),
)
actions = []
for (args, kwargs) in options:
actions.append(parser.add_argument(*args, **kwargs))
args = parser.parse_args()
At this point, the options are parsed in args, and all argparse.Action objects are stored in the actions list. I can then iterate over this list and reconstruct the options like this:
cmdline = []
for action in actions:
value = getattr(args, action.dest)
if action.required or value != action.default:
if action.option_strings:
cmdline.append(action.option_strings[0])
if action.nargs is None:
cmdline.append(value)
elif action.nargs == '?':
if value != action.const:
cmdline.append(value)
elif action.nargs != 0:
cmdline += value
In my specific case, I also wanted to remove some options from the command line. To do that I simply added them separately with a call to parser.add_argument() and not through the options tuple.