I want to input args that not configured in argparse:
parser = argparse.ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help', dest="character", required=False)
subparsers.required = False
base_subparser = argparse.ArgumentParser(add_help=False)
# define common shared arguments
base_subparser.add_argument('--disable', choices=['false', 'true'])
base_subparser.add_argument('--foo', choices=['false', 'true'])
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help', parents=[base_subparser])
parser_a.add_argument('--bar', choices='ABC', help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help', parents=[base_subparser])
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
argcomplete.autocomplete(parser)
args = parser.parse_known_args()
print(args)
I use the parse_known_args() which use a list to store the args not configured in argparse. However, when I use ./prog.py key = val, it shows argument character: invalid choice: 'key=val' (choose from 'a', 'b'). So I have to choose 'a' or 'b', how can I input the args not configured in argparse without choose one of the subparsers.
The error you see is the same as produced by a '?' positional with choices:
In [25]: import argparse
In [26]: parser = argparse.ArgumentParser()
In [27]: parser.add_argument('foo', nargs='?', choices=['a','b'])
'foo' is optional:
In [28]: parser.parse_known_args([])
Out[28]: (Namespace(foo=None), [])
In [29]: parser.parse_known_args(['a'])
Out[29]: (Namespace(foo='a'), [])
but any string is parsed as a possible 'foo' value:
In [30]: parser.parse_known_args(['c'])
usage: ipykernel_launcher.py [-h] [{a,b}]
ipykernel_launcher.py: error: argument foo: invalid choice: 'c' (choose from 'a', 'b')
providing a proper choice first, allows it to treat 'c' as an extra:
In [31]: parser.parse_known_args(['a','c'])
Out[31]: (Namespace(foo='a'), ['c'])
Or if the string looks like a optional's flag:
In [32]: parser.parse_known_args(['-c'])
Out[32]: (Namespace(foo=None), ['-c'])
Another possibility is to go ahead and name a subparser, possibly a dummy one, and provide the extra. The subparser will be the one that actually puts that string in the 'unknowns' category.
In [40]: parser = argparse.ArgumentParser()
In [41]: subp = parser.add_subparsers(dest='cmd')
In [44]: p1 = subp.add_parser('a')
In [45]: parser.parse_known_args(['a','c'])
Out[45]: (Namespace(cmd='a'), ['c'])
In [46]: parser.parse_known_args([]) # not-required is the default
Out[46]: (Namespace(cmd=None), [])
Keep in mind that the main parser does not "know" anything about subparsers, except is a positional. It's doing its normal allocating strings to actions. But once it calls a subparser, that parser has full control over the parsing. Once it's done it passes the namespace back to the main, but the main doesn't do any more parsing - it just wraps things up and exits (with results or error).
Since subp is a positional with a special Action subclass, _SubParsersAction, I was thinking it might be possible to create a flagged argument with that class
parser.add_argument('--foo', action=argparse._SubParsersAction)
but there's more going on in add_subparsers, so it isn't a trivial addition. This is a purely speculative idea.
I have a function that looks like this:
import argparse
import sys
def execute():
parser = argparse.ArgumentParser()
if (total_args := len(sys.argv)) == 1:
do_stuff()
if total_args == 2:
first = sys.argv[1]
do_stuff2()
if total_args == 3:
first, second = sys.argv[1:3]
do_stuff3()
if total_args > 3:
first, second = sys.argv[1:3]
del sys.argv[1:3]
add_args(parser)
parser.parse_args()
do_stuff4()
Which should have a test function test_execute that will try different given args, the question: is there a clean way to do it without manually modifying sys.argv using sys.argv.extend(some_test_args) and delete the args later?
Note: I can't use argparse optional positional arguments by setting nargs=? in parser.add_argument() because the first 2 arguments are optional and each case (1, 2, 3, > 3 arguments) executes different functions. To understand further, please check the example below ...
parser = argparse.ArgumentParser()
parser.add_argument('arg1', nargs='?')
parser.add_argument('arg2', nargs='?')
args = parser.parse_known_args()
print(args)
which if called like the following, will result to the wrong variable saved in the second position:
>>> python my_script.py --unknown-arg 999
Will print:
(Namespace(arg1='999', arg2=None), ['--unknown-arg'])
which is totally not what I need. I'm expecting arg1 to have a None value. The reason sometimes there will be unknown arguments is that argparse does not support parsing arguments by specifying a group. Let's say I have argument group A and argument group B and I need to parse only group A, I can't do parser.parse_group('A') I will have to create parser_a = argparse.ArgumentParser() and add group A arguments and parse them and repeat for parser_b.
Therefore the best solution I have so far is using sys.argv despite the fact this is inconvenient for testing. Also adding all options without grouping, will create another problem because group B arguments depend on values parsed from group A.
One workaround is to specify using --unknown-arg=999 but this will create inconsistencies in the documentation and usage of the script and is also not what I need.
Could you pass in sys.argv into execute()?
Something like this:
import argparse
import sys
def execute(argv):
parser = argparse.ArgumentParser()
if (total_args := len(argv)) == 1:
do_stuff()
if total_args == 2:
first = argv[1]
do_stuff2()
if total_args == 3:
first, second = argv[1:3]
do_stuff3()
if total_args > 3:
first, second = argv[1:3]
del argv[1:3]
add_args(parser)
parser.parse_args(argv)
do_stuff4()
if __name__ == "__main__":
execute(sys.argv)
In your tests you could then do something along the lines of:
def test_execute():
test_argv = ["some", "args", "list"]
execute(test_argv)
# assert something
I am trying to sort out how to pass the boolean value False from the command line to the argparser. My origninal code looked like:
import argparse
parser = argparse.ArgumentParser(allow_abbrev=True)
parser.add_argument('-data', default=True, type=bool, help='uses the history file')
args = parser.parse_args(sys.argv[1:])
From the command line, I typed: python myscript.py -data False
Also variations with single & double quotes around False. When I examine the contents of the args namespace, args.data is always True.
So I changed the argument definition from bool to str, with a string "True" default as shown below:
parser.add_argument('-data', default="True", type=str, help='uses the history file')
I then added some massaging of the args to get the boolean value I really wanted:
if re.search("f", args.data, re.I):
args.data = False
else:
args.data = True
This workaround does work. Is there a better way to do this?
You can use the store_false or store_true parameter to add_argument (see the argparse documentation). For example, if you want the default to be True then you could add an argument with action='store_false':
parser.add_argument('--no-data', action='store_false', help="don't use the history file")
Then args.no_data will be False if you run python command.py --no-data and True if you run python command.py without the --no-data argument.
def my_bool(s):
return s != 'False'
parser.add_argument('-data',default=True,type=my_bool)
If an argument is meant to be a boolean, you should simply use store_true or store_false, so that the presence of the option itself would result in a True or False value, and the absence of the option would result in a False or True value, respectively.
From argparse's documentation:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> parser.add_argument('--bar', action='store_false')
>>> parser.add_argument('--baz', action='store_false')
>>> parser.parse_args('--foo --bar'.split())
Namespace(foo=True, bar=False, baz=True)
Question: What is the intended / official way of accessing possible arguments from an existing argparse.ArgumentParser object?
Example: Let's assume the following context:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
Here I'd like to get the following list of allowed arguments:
['-h', '--foo', '--help', '-f']
I found the following workaround which does the trick for me
parser._option_string_actions.keys()
But I'm not happy with it, as it involves accessing a _-member that is not officially documented. Whats the correct alternative for this task?
I don't think there is a "better" way to achieve what you want.
If you really don't want to use the _option_string_actions attribute, you could process the parser.format_usage() to retrieve the options, but doing this, you will get only the short options names.
If you want both short and long options names, you could process the parser.format_help() instead.
This process can be done with a very simple regular expression: -+\w+
import re
OPTION_RE = re.compile(r"-+\w+")
PARSER_HELP = """usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO, -f FOO a random options
--bar BAR, -b BAR a more random option
"""
options = set(OPTION_RE.findall(PARSER_HELP))
print(options)
# set(['-f', '-b', '--bar', '-h', '--help', '--foo'])
Or you could first make a dictionnary which contains the argument parser configuration and then build the argmuent parser from it. Such a dictionnary could have the option names as key and the option configuration as value. Doing this, you can access the options list via the dictionnary keys flattened with itertools.chain:
import argparse
import itertools
parser_config = {
('--foo', '-f'): {"help": "a random options", "type": str},
('--bar', '-b'): {"help": "a more random option", "type": int, "default": 0}
}
parser = argparse.ArgumentParser()
for option, config in parser_config.items():
parser.add_argument(*option, **config)
print(parser.format_help())
# usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
#
# optional arguments:
# -h, --help show this help message and exit
# --foo FOO, -f FOO a random options
# --bar BAR, -b BAR a more random option
print(list(itertools.chain(*parser_config.keys())))
# ['--foo', '-f', '--bar', '-b']
This last way is what I would do, if I was reluctant to use _option_string_actions.
This started as a joke answer, but I've learned something since - so I'll post it.
Assume, we know the maximum length of an option allowed. Here is a nice answer to the question in this situation:
from itertools import combinations
def parsable(option):
try:
return len(parser.parse_known_args(option.split())[1]) != 2
except:
return False
def test(tester, option):
return any([tester(str(option) + ' ' + str(v)) for v in ['0', '0.0']])
def allowed_options(parser, max_len=3, min_len=1):
acceptable = []
for l in range(min_len, max_len + 1):
for option in combinations([c for c in [chr(i) for i in range(33, 127)] if c != '-'], l):
option = ''.join(option)
acceptable += [p + option for p in ['-', '--'] if test(parsable, p + option)]
return acceptable
Of course this is very pedantic as the question doesn't require any specific runtime. So I'll ignore that here. I'll also disregard, that the above version produces a mess of output because one can get rid of it easily.
But more importantly, this method detected the following interesting argparse "features":
In in the OP example, argparse would also allow --fo. This has to be a bug.
But further, in the OP example again, argparse would also allow -fo (ie. setting foo to o without space or anything). This is documented and intended, but I didn't know it.
Because of this, a correct solution is a bit longer and would look something like this (only parsable changes, I'll omit the other methods):
def parsable(option):
try:
default = vars(parser.parse_known_args(['--' + '0' * 200])[0])
parsed, remaining = parser.parse_known_args(option.split())
if len(remaining) == 2:
return False
parsed = vars(parsed)
for k in parsed.keys():
try:
if k in default and default[k] != parsed[k] and float(parsed[k]) != 0.0:
return False # Filter '-fx' cases where '-f' is the argument and 'x' the value.
except:
return False
return True
except:
return False
Summary: Besides all the restrictions (runtime and fixed maximum option length), this is the only answer that correctly respects the real parser behavior - however buggy it may even be. So here you are, a perfect answer that is absolutely useless.
I have to agree with Tryph's answer.
Not pretty, but you can retrieve them from parser.format_help():
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
goal = parser._option_string_actions.keys()
def get_allowed_arguments(parser):
lines = parser.format_help().split('\n')
line_index = 0
number_of_lines = len(lines)
found_optional_arguments = False
# skip the first lines until the section 'optional arguments'
while line_index < number_of_lines:
if lines[line_index] == 'optional arguments:':
found_optional_arguments = True
line_index += 1
break
line_index += 1
result_list = []
if found_optional_arguments:
while line_index < number_of_lines:
arg_list = get_arguments_from_line(lines[line_index])
if len(arg_list) == 0:
break
result_list += arg_list
line_index += 1
return result_list
def get_arguments_from_line(line):
if line[:2] != ' ':
return []
arg_list = []
i = 2
N = len(line)
inside_arg = False
arg_start = 2
while i < N:
if line[i] == '-' and not inside_arg:
arg_start = i
inside_arg = True
elif line[i] in [',',' '] and inside_arg:
arg_list.append(line[arg_start:i+1])
inside_arg = False
i += 1
return arg_list
answer = get_allowed_arguments(parser)
There's probably a regular expressions alternative to the above mess...
First a note on the argparse docs - it's basically a how-to-use document, not a formal API. The standard for what argparse does is the code itself, the unit tests (test/test_argparse.py), and a paralyzing concern for backward compatibility.
There's no 'official' way of accessing allowed arguments, because users usually don't need to know that (other than reading the help/usage).
But let me illustrate with a simple parser in an iteractive session:
In [247]: parser=argparse.ArgumentParser()
In [248]: a = parser.add_argument('pos')
In [249]: b = parser.add_argument('-f','--foo')
add_argument returns the Action object that it created. This isn't documented, but obvious to any one who has created a parser interactively.
The parser object has a repr method, that displays major parameters. But it has many more attributes, which you can see with vars(parser), or parser.<tab> in Ipython.
In [250]: parser
Out[250]: ArgumentParser(prog='ipython3', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
The Actions too have repr; the Action subclass is determined by the action parameter.
In [251]: a
Out[251]: _StoreAction(option_strings=[], dest='pos', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [252]: b
Out[252]: _StoreAction(option_strings=['-f', '--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
vars(a) etc can be used to see all attributes.
A key parser attribute is _actions, a list of all defined Actions. This is the basis for all parsing. Note it includes the help action that was created automatically. Look at option_strings; that determines whether the Action is positional or optional.
In [253]: parser._actions
Out[253]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=[], dest='pos',....),
_StoreAction(option_strings=['-f', '--foo'], dest='foo', ...)]
_option_string_actions is a dictionary, mapping from option_strings to Actions (the same objects that appear in _actions). References to those Action objects appear all over the place in argparse code.
In [255]: parser._option_string_actions
Out[255]:
{'--foo': _StoreAction(option_strings=['-f', '--foo'],....),
'--help': _HelpAction(option_strings=['-h', '--help'],...),
'-f': _StoreAction(option_strings=['-f', '--foo'], dest='foo',...),
'-h': _HelpAction(option_strings=['-h', '--help'], ....)}
In [256]: list(parser._option_string_actions.keys())
Out[256]: ['-f', '--help', '-h', '--foo']
Note that there is a key for each - string, long or short; but there's nothing for pos, the positional has an empty option_strings parameter.
If that list of keys is what you want, use it, and don't worry about the _. It does not have a 'public' alias.
I can understand parsing the help to discover the same; but that's a lot of work to just avoid using a 'private' attribute. If you worry about the undocumented attribute being changed, you should also worry about the help format being changed. That isn't part of the docs either.
help layout is controlled by parser.format_help. The usage is created from information in self._actions. Help lines from information in
for action_group in self._action_groups:
formatter.add_arguments(action_group._group_actions)
(you don't want to get into action groups do you?).
There is another way of getting the option_strings - collect them from the _actions:
In [258]: [a.option_strings for a in parser._actions]
Out[258]: [['-h', '--help'], [], ['-f', '--foo']]
===================
Delving in to code details a bit:
parser.add_argument creates an Action, and then passes it to parser._add_action. This is the method the populates both .actions and action.option_strings.
self._actions.append(action)
for option_string in action.option_strings:
self._option_string_actions[option_string] = action
I'd like to have my python script take variable number of arguments depending on a particular choice. For example:
python run.py foo
python run.py bar X Y
where choosing the option bar requires two additional arguments, say integer inputs, but foo requires no additional arguments.
import argparse
argparser = argparse.ArgumentParser()
# Allow user to choose test to run
argparser.add_argument("test", choices=['foo', 'bar'], help="You may choose foo or bar.")
...
But how can I specify additional arguments required by bar? P.S. I'm working with Python 2.7, so if a solution requires Python 3, it won't be much help in my case.
You should use subparsers:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(title='subcommands')
parser_foo = subparsers.add_parser('foo')
parser_foo.set_defaults(target='foo')
parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('more')
parser_bar.set_defaults(target='bar')
Usage:
>>> parser.parse_args(['foo'])
Namespace(target='foo')
>>> parser.parse_args(['bar', '123'])
Namespace(target='bar', more='123')
Note that you could set the target to e.g. a function and call it directly. Here's some sample code that does this (extracted from Cactus' CLI, but that's a rather common pattern):
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(title = 'subcommands')
parser_create = subparsers.add_parser('create')
parser_create.add_argument('path')
parser_create.add_argument('-s', '--skeleton')
parser_create.set_defaults(target=create)
parser_build = subparsers.add_parser('build')
parser_build.set_defaults(target = build)
args = parser.parse_args()
args.target(**{k: v for k, v in vars(args).items() if k != 'target'})