I want to input args that not configured in argparse:
parser = argparse.ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help', dest="character", required=False)
subparsers.required = False
base_subparser = argparse.ArgumentParser(add_help=False)
# define common shared arguments
base_subparser.add_argument('--disable', choices=['false', 'true'])
base_subparser.add_argument('--foo', choices=['false', 'true'])
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help', parents=[base_subparser])
parser_a.add_argument('--bar', choices='ABC', help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help', parents=[base_subparser])
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
argcomplete.autocomplete(parser)
args = parser.parse_known_args()
print(args)
I use the parse_known_args() which use a list to store the args not configured in argparse. However, when I use ./prog.py key = val, it shows argument character: invalid choice: 'key=val' (choose from 'a', 'b'). So I have to choose 'a' or 'b', how can I input the args not configured in argparse without choose one of the subparsers.
The error you see is the same as produced by a '?' positional with choices:
In [25]: import argparse
In [26]: parser = argparse.ArgumentParser()
In [27]: parser.add_argument('foo', nargs='?', choices=['a','b'])
'foo' is optional:
In [28]: parser.parse_known_args([])
Out[28]: (Namespace(foo=None), [])
In [29]: parser.parse_known_args(['a'])
Out[29]: (Namespace(foo='a'), [])
but any string is parsed as a possible 'foo' value:
In [30]: parser.parse_known_args(['c'])
usage: ipykernel_launcher.py [-h] [{a,b}]
ipykernel_launcher.py: error: argument foo: invalid choice: 'c' (choose from 'a', 'b')
providing a proper choice first, allows it to treat 'c' as an extra:
In [31]: parser.parse_known_args(['a','c'])
Out[31]: (Namespace(foo='a'), ['c'])
Or if the string looks like a optional's flag:
In [32]: parser.parse_known_args(['-c'])
Out[32]: (Namespace(foo=None), ['-c'])
Another possibility is to go ahead and name a subparser, possibly a dummy one, and provide the extra. The subparser will be the one that actually puts that string in the 'unknowns' category.
In [40]: parser = argparse.ArgumentParser()
In [41]: subp = parser.add_subparsers(dest='cmd')
In [44]: p1 = subp.add_parser('a')
In [45]: parser.parse_known_args(['a','c'])
Out[45]: (Namespace(cmd='a'), ['c'])
In [46]: parser.parse_known_args([]) # not-required is the default
Out[46]: (Namespace(cmd=None), [])
Keep in mind that the main parser does not "know" anything about subparsers, except is a positional. It's doing its normal allocating strings to actions. But once it calls a subparser, that parser has full control over the parsing. Once it's done it passes the namespace back to the main, but the main doesn't do any more parsing - it just wraps things up and exits (with results or error).
Since subp is a positional with a special Action subclass, _SubParsersAction, I was thinking it might be possible to create a flagged argument with that class
parser.add_argument('--foo', action=argparse._SubParsersAction)
but there's more going on in add_subparsers, so it isn't a trivial addition. This is a purely speculative idea.
Related
I am trying to build a command line parser that will be able to share between arguments the values passed in order to avoid having to type them multiple times. Said otherwise, I would like the namespaces of both argument to be identical:
import argparse
class PrintAction(argparse.Action):
def __init__(self, option_strings, dest, **kwargs):
super(PrintAction, self).__init__(option_strings, dest, **kwargs)
def __call__(self, parser, namespace, values, option_string=None):
for val in values:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action=PrintAction)
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
I would then like a solution that stores in both cases ['a', 'b', 'c'] in bar but also that in the case that --foo is provided, then a, b and c would be printed.
For now, what I get is foo prints only a and bar stores only b and c in case 1 and the correct result in case 2.
You need to make --foo a boolean flag. Now it's a string parameter, because you did not state otherwise. Set action to store_true for the boolean flag effect.
The final solution would look like:
def print_args(args):
if args.foo:
for val in args.bar:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action='store_true')
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
Then calling print_args(args) in the first case will print a, b and c and in the second case, it won't.
You can't (readily) trick the argparse into reusing argv strings. The parser allocates values to the Actions.
The default nargs is None, which means, use the the next string as an argument.
parser.add_argument('--foo')
would set foo='a', and bar=['b','c'].
In your Action, values will be ['a'], which you print. In optparse each option gets the remaining argv list, which it can consume as it wants. In argparse it only gets the values that its nargs demands.
You could specify in the __init__ that the nargs=0, and then print from sys.argv. Eqivalently, as #9000 suggests, make it a store_true and print after parsing. Look at the code for the store_true Action class.
Another option is to give both foo and bar a *, and have foo both print and save to the bar dest. Then foo would consume all following strings. But, if bar doesn't have anything to save, it might write [] to the namespace.
In any case, the best you can do is fake the repeated use.
Another idea is to use 2 different parsers with parse_known_args. Parsers don't mess with the sys.argv, so it can read and parsed multiple times.
Question: What is the intended / official way of accessing possible arguments from an existing argparse.ArgumentParser object?
Example: Let's assume the following context:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
Here I'd like to get the following list of allowed arguments:
['-h', '--foo', '--help', '-f']
I found the following workaround which does the trick for me
parser._option_string_actions.keys()
But I'm not happy with it, as it involves accessing a _-member that is not officially documented. Whats the correct alternative for this task?
I don't think there is a "better" way to achieve what you want.
If you really don't want to use the _option_string_actions attribute, you could process the parser.format_usage() to retrieve the options, but doing this, you will get only the short options names.
If you want both short and long options names, you could process the parser.format_help() instead.
This process can be done with a very simple regular expression: -+\w+
import re
OPTION_RE = re.compile(r"-+\w+")
PARSER_HELP = """usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO, -f FOO a random options
--bar BAR, -b BAR a more random option
"""
options = set(OPTION_RE.findall(PARSER_HELP))
print(options)
# set(['-f', '-b', '--bar', '-h', '--help', '--foo'])
Or you could first make a dictionnary which contains the argument parser configuration and then build the argmuent parser from it. Such a dictionnary could have the option names as key and the option configuration as value. Doing this, you can access the options list via the dictionnary keys flattened with itertools.chain:
import argparse
import itertools
parser_config = {
('--foo', '-f'): {"help": "a random options", "type": str},
('--bar', '-b'): {"help": "a more random option", "type": int, "default": 0}
}
parser = argparse.ArgumentParser()
for option, config in parser_config.items():
parser.add_argument(*option, **config)
print(parser.format_help())
# usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
#
# optional arguments:
# -h, --help show this help message and exit
# --foo FOO, -f FOO a random options
# --bar BAR, -b BAR a more random option
print(list(itertools.chain(*parser_config.keys())))
# ['--foo', '-f', '--bar', '-b']
This last way is what I would do, if I was reluctant to use _option_string_actions.
This started as a joke answer, but I've learned something since - so I'll post it.
Assume, we know the maximum length of an option allowed. Here is a nice answer to the question in this situation:
from itertools import combinations
def parsable(option):
try:
return len(parser.parse_known_args(option.split())[1]) != 2
except:
return False
def test(tester, option):
return any([tester(str(option) + ' ' + str(v)) for v in ['0', '0.0']])
def allowed_options(parser, max_len=3, min_len=1):
acceptable = []
for l in range(min_len, max_len + 1):
for option in combinations([c for c in [chr(i) for i in range(33, 127)] if c != '-'], l):
option = ''.join(option)
acceptable += [p + option for p in ['-', '--'] if test(parsable, p + option)]
return acceptable
Of course this is very pedantic as the question doesn't require any specific runtime. So I'll ignore that here. I'll also disregard, that the above version produces a mess of output because one can get rid of it easily.
But more importantly, this method detected the following interesting argparse "features":
In in the OP example, argparse would also allow --fo. This has to be a bug.
But further, in the OP example again, argparse would also allow -fo (ie. setting foo to o without space or anything). This is documented and intended, but I didn't know it.
Because of this, a correct solution is a bit longer and would look something like this (only parsable changes, I'll omit the other methods):
def parsable(option):
try:
default = vars(parser.parse_known_args(['--' + '0' * 200])[0])
parsed, remaining = parser.parse_known_args(option.split())
if len(remaining) == 2:
return False
parsed = vars(parsed)
for k in parsed.keys():
try:
if k in default and default[k] != parsed[k] and float(parsed[k]) != 0.0:
return False # Filter '-fx' cases where '-f' is the argument and 'x' the value.
except:
return False
return True
except:
return False
Summary: Besides all the restrictions (runtime and fixed maximum option length), this is the only answer that correctly respects the real parser behavior - however buggy it may even be. So here you are, a perfect answer that is absolutely useless.
I have to agree with Tryph's answer.
Not pretty, but you can retrieve them from parser.format_help():
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
goal = parser._option_string_actions.keys()
def get_allowed_arguments(parser):
lines = parser.format_help().split('\n')
line_index = 0
number_of_lines = len(lines)
found_optional_arguments = False
# skip the first lines until the section 'optional arguments'
while line_index < number_of_lines:
if lines[line_index] == 'optional arguments:':
found_optional_arguments = True
line_index += 1
break
line_index += 1
result_list = []
if found_optional_arguments:
while line_index < number_of_lines:
arg_list = get_arguments_from_line(lines[line_index])
if len(arg_list) == 0:
break
result_list += arg_list
line_index += 1
return result_list
def get_arguments_from_line(line):
if line[:2] != ' ':
return []
arg_list = []
i = 2
N = len(line)
inside_arg = False
arg_start = 2
while i < N:
if line[i] == '-' and not inside_arg:
arg_start = i
inside_arg = True
elif line[i] in [',',' '] and inside_arg:
arg_list.append(line[arg_start:i+1])
inside_arg = False
i += 1
return arg_list
answer = get_allowed_arguments(parser)
There's probably a regular expressions alternative to the above mess...
First a note on the argparse docs - it's basically a how-to-use document, not a formal API. The standard for what argparse does is the code itself, the unit tests (test/test_argparse.py), and a paralyzing concern for backward compatibility.
There's no 'official' way of accessing allowed arguments, because users usually don't need to know that (other than reading the help/usage).
But let me illustrate with a simple parser in an iteractive session:
In [247]: parser=argparse.ArgumentParser()
In [248]: a = parser.add_argument('pos')
In [249]: b = parser.add_argument('-f','--foo')
add_argument returns the Action object that it created. This isn't documented, but obvious to any one who has created a parser interactively.
The parser object has a repr method, that displays major parameters. But it has many more attributes, which you can see with vars(parser), or parser.<tab> in Ipython.
In [250]: parser
Out[250]: ArgumentParser(prog='ipython3', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
The Actions too have repr; the Action subclass is determined by the action parameter.
In [251]: a
Out[251]: _StoreAction(option_strings=[], dest='pos', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [252]: b
Out[252]: _StoreAction(option_strings=['-f', '--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
vars(a) etc can be used to see all attributes.
A key parser attribute is _actions, a list of all defined Actions. This is the basis for all parsing. Note it includes the help action that was created automatically. Look at option_strings; that determines whether the Action is positional or optional.
In [253]: parser._actions
Out[253]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=[], dest='pos',....),
_StoreAction(option_strings=['-f', '--foo'], dest='foo', ...)]
_option_string_actions is a dictionary, mapping from option_strings to Actions (the same objects that appear in _actions). References to those Action objects appear all over the place in argparse code.
In [255]: parser._option_string_actions
Out[255]:
{'--foo': _StoreAction(option_strings=['-f', '--foo'],....),
'--help': _HelpAction(option_strings=['-h', '--help'],...),
'-f': _StoreAction(option_strings=['-f', '--foo'], dest='foo',...),
'-h': _HelpAction(option_strings=['-h', '--help'], ....)}
In [256]: list(parser._option_string_actions.keys())
Out[256]: ['-f', '--help', '-h', '--foo']
Note that there is a key for each - string, long or short; but there's nothing for pos, the positional has an empty option_strings parameter.
If that list of keys is what you want, use it, and don't worry about the _. It does not have a 'public' alias.
I can understand parsing the help to discover the same; but that's a lot of work to just avoid using a 'private' attribute. If you worry about the undocumented attribute being changed, you should also worry about the help format being changed. That isn't part of the docs either.
help layout is controlled by parser.format_help. The usage is created from information in self._actions. Help lines from information in
for action_group in self._action_groups:
formatter.add_arguments(action_group._group_actions)
(you don't want to get into action groups do you?).
There is another way of getting the option_strings - collect them from the _actions:
In [258]: [a.option_strings for a in parser._actions]
Out[258]: [['-h', '--help'], [], ['-f', '--foo']]
===================
Delving in to code details a bit:
parser.add_argument creates an Action, and then passes it to parser._add_action. This is the method the populates both .actions and action.option_strings.
self._actions.append(action)
for option_string in action.option_strings:
self._option_string_actions[option_string] = action
If I have an argparser.ArgumentParser, plus a namespace returned from parser.parse_args, is there a simple way to convert the namespace back into a list of argv such that they could be passed back to the program? Essentially, is there an inverse function of parse_args?
An example scenario:
parser = argparse.ArgumentParser()
parser.add_argument('--example', type=int, default=0)
args = parser.parse_args(argv)
args.example *= 2
new_argv = parser.generate_argv(args)
So if I call:
python my_program.py --example 1
I would want back:
new_argv = ['--example', '2']
I think this has been asked before, though I'm not sure of a good search term.
argparse - Build back command line (found by searching on argparse and sys.argv)
Before we get too far into this question, lets be clear. args=parser.parse_args() is the same as args=parser.parse_args(sys.argv[1:]). But I can imagine cases where you like to know what sys.argv[1:] would produce some arbitrary args. Maybe for testing, maybe for driving someone else's code.
There isn't any code in argparse that does this. But for a limited set of cases you could take information from the defined Actions, and create a plausible sys.argv.
In [432]: parser = argparse.ArgumentParser()
In [433]: parser.add_argument('--example', type=int, default=0)
Out[433]: _StoreAction(option_strings=['--example'], dest='example', nargs=None, const=None, default=0, type=<type 'int'>, choices=None, help=None, metavar=None)
The list of defined Actions:
In [435]: parser._actions
Out[435]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['--example'], dest='example', nargs=None, const=None, default=0, type=<type 'int'>, choices=None, help=None, metavar=None)]
select the one of interest, and look at some of its attributes:
In [436]: a1=parser._actions[-1]
In [437]: a1.type
Out[437]: int
In [438]: a1.default
Out[438]: 0
Now generate args:
In [439]: args=parser.parse_args(['--example','1'])
In [440]: args
Out[440]: Namespace(example=1)
In [441]: args.example *= 2
A simple example of creating a list using the new args and information from the Action. Obviously the working code needs to deduce which action to use. For the most common types str() is enough.
In [442]: if args.example != a1.default:
.....: print(['--example',str(args.example)])
.....:
['--example', '2']
Or I could play with the metavar attribute, and the usage formatter:
In [445]: a1.metavar=str(args.example)
In [446]: parser.print_usage()
usage: ipython2.7 [-h] [--example 2]
I ended up in a situation needing this, and here was my solution. It assumes that you know what positional arguments are expected. Definitely not foolproof, but it works for me.
def simulate_args_from_namespace(n, positional=[]):
""" check an argparse namespace against a module's get_args method.
Ideally, there would be something built in to argparse, but no such luck.
This tries to reconstruct the arg list that argparse.parse_args would expect
"""
arg_list = [[k, v] for k, v in sorted(vars(n).items())]
argparse_formatted_list = []
for l in arg_list:
#### deal with flag arguments (store true/false)
if l[1] == True:
argparse_formatted_list.append("--{}".format(l[0]))
elif l[1] == False or l[1] is None:
pass # dont add this arg
# add positional argments
elif l[0] in positional:
argparse_formatted_list.append(str(l[0]))
# add the named arguments
else:
argparse_formatted_list.append("--{}".format(l[0]))
argparse_formatted_list.append(str(l[1]))
return argparse_formatted_list
delete args from parser.parse_args(args)
args[1] == example
args[2] == string number
args.example == int number
from sys import argv
parser = argparse.ArgumentParser()
parser.add_argument('--example', type=int, default=0 )
args = parser.parse_args()
print ([argv[1],args.example*2])
then
python my_program.py --example 1
>>>['--example', 2]
If I have an optional argument with optional argument value, is there a way to validate if the argument is set when the value is not given?
For instance:
parser = argparse.ArgumentParser()
parser.add_argument('--abc', nargs='?')
args = parser.parse_args()
Would correctly give me:
optional arguments:
--abc [ABC]
How do I distinguish between 1 and 2 below?
'' => args.abc is None
'--abc' => args.abc is still None
'--abc something' => args.abc is something
...
Update:
Found a trick to solve this problem: you can use "nargs='*'" instead of "nargs='?'". This way #1 would return None, and #2 would return an empty list. The downside is this will allow multiple values for the arguments to be accepted too; so you'd need to add a check for it if appropriate.
Alternatively you can also set a default value for the argument; see answer from chepner and Anand S Kumar.
With nargs='?', you can supply both a default and const.
In [791]: parser=argparse.ArgumentParser()
In [792]: parser.add_argument('--abc', nargs='?', default='default', const='const')
If the argument is not given it uses the default:
In [793]: parser.parse_args([])
Out[793]: Namespace(abc='default')
If given, but without an argument string, it uses the const:
In [794]: parser.parse_args(['--abc'])
Out[794]: Namespace(abc='const')
Otherwise it uses the argument string:
In [795]: parser.parse_args(['--abc','test'])
Out[795]: Namespace(abc='test')
In [796]: parser.print_help()
usage: ipython3 [-h] [--abc [ABC]]
optional arguments:
-h, --help show this help message and exit
--abc [ABC]
Use a different default value for the option. Compare
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--abc', nargs='?', default="default")
>>> parser.parse_args()
Namespace(abc='default')
>>> parser.parse_args(['--abc'])
Namespace(abc=None)
>>> parser.parse_args(['--abc', 'value'])
Namespace(abc='value')
I'm not sure how you would provide a different value for when --abc is used without an argument, short of using a custom action instead of the nargs argument.
Not sure if this is the standard way, but you can set default argument to something , and then that value would be used in case --abc is not in the argument list.
Example code -
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--abc', nargs='?', default="-1")
args = parser.parse_args()
print(args)
Result -
>python a.py
Namespace(abc='-1')
>python a.py --abc
Namespace(abc=None)
>python a.py --abc something
Namespace(abc='something')
I'm using this to have a command-line arg for multiprocessing. Specifying --multi uses all cores and given an arg specifies a number of cores, e.g., --multi 4 for four cores.
parser.add_argument("-mp", "--multi", type=int, nargs="*", help=multi_text)
Parsing logic is then:
if (args.multi == None):
num_cores = 1
elif (args.multi == []):
num_cores = multiprocessing.cpu_count()
elif (len(args.multi) == 1):
num_cores = args.multi[0]
else:
print("Invalid specification of core usage.")
sys.exit(1)
I'd like to have my python script take variable number of arguments depending on a particular choice. For example:
python run.py foo
python run.py bar X Y
where choosing the option bar requires two additional arguments, say integer inputs, but foo requires no additional arguments.
import argparse
argparser = argparse.ArgumentParser()
# Allow user to choose test to run
argparser.add_argument("test", choices=['foo', 'bar'], help="You may choose foo or bar.")
...
But how can I specify additional arguments required by bar? P.S. I'm working with Python 2.7, so if a solution requires Python 3, it won't be much help in my case.
You should use subparsers:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(title='subcommands')
parser_foo = subparsers.add_parser('foo')
parser_foo.set_defaults(target='foo')
parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('more')
parser_bar.set_defaults(target='bar')
Usage:
>>> parser.parse_args(['foo'])
Namespace(target='foo')
>>> parser.parse_args(['bar', '123'])
Namespace(target='bar', more='123')
Note that you could set the target to e.g. a function and call it directly. Here's some sample code that does this (extracted from Cactus' CLI, but that's a rather common pattern):
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(title = 'subcommands')
parser_create = subparsers.add_parser('create')
parser_create.add_argument('path')
parser_create.add_argument('-s', '--skeleton')
parser_create.set_defaults(target=create)
parser_build = subparsers.add_parser('build')
parser_build.set_defaults(target = build)
args = parser.parse_args()
args.target(**{k: v for k, v in vars(args).items() if k != 'target'})