Develop Reference

How to make optional subparser in python3?

I want to input args that not configured in argparse:
parser = argparse.ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help', dest="character", required=False)
subparsers.required = False
base_subparser = argparse.ArgumentParser(add_help=False)
# define common shared arguments
base_subparser.add_argument('--disable', choices=['false', 'true'])
base_subparser.add_argument('--foo', choices=['false', 'true'])
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help', parents=[base_subparser])
parser_a.add_argument('--bar', choices='ABC', help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help', parents=[base_subparser])
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
argcomplete.autocomplete(parser)
args = parser.parse_known_args()
print(args)
I use the parse_known_args() which use a list to store the args not configured in argparse. However, when I use ./prog.py key = val, it shows argument character: invalid choice: 'key=val' (choose from 'a', 'b'). So I have to choose 'a' or 'b', how can I input the args not configured in argparse without choose one of the subparsers.

The error you see is the same as produced by a '?' positional with choices:
In [25]: import argparse
In [26]: parser = argparse.ArgumentParser()
In [27]: parser.add_argument('foo', nargs='?', choices=['a','b'])
'foo' is optional:
In [28]: parser.parse_known_args([])
Out[28]: (Namespace(foo=None), [])
In [29]: parser.parse_known_args(['a'])
Out[29]: (Namespace(foo='a'), [])
but any string is parsed as a possible 'foo' value:
In [30]: parser.parse_known_args(['c'])
usage: ipykernel_launcher.py [-h] [{a,b}]
ipykernel_launcher.py: error: argument foo: invalid choice: 'c' (choose from 'a', 'b')
providing a proper choice first, allows it to treat 'c' as an extra:
In [31]: parser.parse_known_args(['a','c'])
Out[31]: (Namespace(foo='a'), ['c'])
Or if the string looks like a optional's flag:
In [32]: parser.parse_known_args(['-c'])
Out[32]: (Namespace(foo=None), ['-c'])
Another possibility is to go ahead and name a subparser, possibly a dummy one, and provide the extra. The subparser will be the one that actually puts that string in the 'unknowns' category.
In [40]: parser = argparse.ArgumentParser()
In [41]: subp = parser.add_subparsers(dest='cmd')
In [44]: p1 = subp.add_parser('a')
In [45]: parser.parse_known_args(['a','c'])
Out[45]: (Namespace(cmd='a'), ['c'])
In [46]: parser.parse_known_args([]) # not-required is the default
Out[46]: (Namespace(cmd=None), [])
Keep in mind that the main parser does not "know" anything about subparsers, except is a positional. It's doing its normal allocating strings to actions. But once it calls a subparser, that parser has full control over the parsing. Once it's done it passes the namespace back to the main, but the main doesn't do any more parsing - it just wraps things up and exits (with results or error).
Since subp is a positional with a special Action subclass, _SubParsersAction, I was thinking it might be possible to create a flagged argument with that class
parser.add_argument('--foo', action=argparse._SubParsersAction)
but there's more going on in add_subparsers, so it isn't a trivial addition. This is a purely speculative idea.

argparse funneling positional arguments into multiple lists

I would like to be able to support positional command line arguments that go into different lists based on prior predicates.
For example, a command like:
mycommand one two three
would yield args like:
main_dest = ['one','two','three']
other_dest = []
but a command like:
mycommand one --other two three --main four five
would yield args like:
main_dest = ['one','four','five']
other_dest = ['two','three']
Conceptually what I'd like is an action that modifies the dest of the positional argument reader.

As a first try this set of Actions seems to do the trick:
In [73]: parser = argparse.ArgumentParser()
In [74]: parser.add_argument('main', nargs='*');
In [75]: parser.add_argument('other', nargs='*');
In [76]: parser.add_argument('--main', action='append');
In [77]: parser.add_argument('--other', action='append');
In [78]: parser.print_usage()
usage: ipython3 [-h] [--main MAIN] [--other OTHER]
[main [main ...]] [other [other ...]]
In [79]: parser.parse_args('one two three'.split())
Out[79]: Namespace(main=['one', 'two', 'three'], other=[])
In [80]: parser.parse_args('one --other two --main three'.split())
Out[80]: Namespace(main=['one', 'three'], other=['two'])
74 and 76 both have main as their dest. I use append for the flagged ones so they don't overwrite positional values. But despite what the usage shows, positionals will only work at the start. If placed a the end they'll overwrite flagged values. And the 'other' positional will never get values - so I should have omitted it.
So it is possible to play games like this, but I'm not sure it's robust, or any easier for your users.
argparse: flatten the result of action='append'

How to use optional positional arguments with nargs='*' arguments in argparse?

As shown in the following code, I want to have an optional positional argument files, I want to specify a default value for it, when paths are passed in, use specified path.
But because --bar can have multiple arguments, the path passed in didn't go into args.files, how do I fix that? Thanks!
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar', nargs='*')
parser.add_argument('files', nargs='?')
cmd = '--foo a --bar b c d '
print parser.parse_args(cmd.split())
# Namespace(bar=['b', 'c', 'd'], files=None, foo='a')
cmd = '--foo a --bar b c d /path/to/file1'
print parser.parse_args(cmd.split())
# Namespace(bar=['b', 'c', 'd', '/path/to/file1'], files=None, foo='a')

Your argument spec is inherently ambiguous (since --bar can take infinite arguments, there is no good way to tell when it ends, particularly since files is optional), so it requires user disambiguation. Specifically, argparse can be told "this is the end of the switches section, all subsequent argument are positional" by putting -- before the positional only section. If you do:
cmd = '--foo a --bar b c d -- /path/to/file1'
print parser.parse_args(cmd.split())
You should get:
Namespace(bar=['b', 'c', 'd'], files='/path/to/file1', foo='a')
(Tested on Py3, but should apply to Py2 as well)
Alternatively, the user can pass the positional argument anywhere it's unambiguous by avoiding putting positional arguments after --bar e.g.:
cmd = '/path/to/file1 --foo a --bar b c d'
or
cmd = '--foo a /path/to/file1 --bar b c d'
Lastly, you could avoid using nargs='*' for switches, given the ambiguity it introduces. Instead, define --bar to be accepted multiple times with a single value per switch, accumulating all uses to a list:
parser.add_argument('--bar', action='append')
then you pass --bar multiple times to supply multiple values one at a time, instead of passing it once with many values:
cmd = '--foo a --bar b --bar c --bar d /path/to/file1'

Accept exactly 0, 1 or 2 positional arguments [duplicate]

This question already has an answer here:
Python argparse: Is there a way to specify a range in nargs?
(1 answer)
Closed 6 years ago.
I'm using argparse package to parse command line arguments. Now I want to accept exactly 0, 1 or 2 strings and put them into a list.
My current approach looks like:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('strs', nargs='*')
print(parser.parse_args([]).strs)
print(parser.parse_args(['AAA']).strs)
print(parser.parse_args(['AAA', 'BBB']).strs)
print(parser.parse_args(['AAA', 'BBB', 'CCC']).strs)
For the 1st, 2nd and 3rd parse_args() the results are expected. For the last one my current approach is if len(parse_args.strs) > 2: raise. Is there any better approach that could let argparse check for me?

There isn't a means in argparse to accept a range of nargs, just values like 1,2,'+','*'.
You could write a custom Action class that would check the number of values that the nargs='*' passes it. But I think it would be just as easy to perform that test after parsing.
I could find a bug/issue that explores adding a range nargs option if you really want to get into that.
If you didn't have other positionals you could define 3 positionals, 2 of them with ?. You might even be able to combine them into one list with a common name and 'append' action. I'll explore that.
In [573]: p=argparse.ArgumentParser()
In [574]: p.add_argument('str',action='append')
In [575]: p.add_argument('str',nargs='?',action='append')
In [576]: p.add_argument('str',nargs='?',action='append')
In [577]: p.print_help()
usage: ipython3 [-h] str [str] [str]
positional arguments:
str
str
str
optional arguments:
-h, --help show this help message and exit
In [578]: p.parse_args('1 2 3'.split())
Out[578]: Namespace(str=['1', '2', '3'])
In [579]: p.parse_args('1 3'.split())
Out[579]: Namespace(str=['1', '3', None])
In [580]: p.parse_args('1'.split())
Out[580]: Namespace(str=['1', None, None])
Got the default Nones. I can correct that with
p.add_argument('str', nargs='?', action='append', default=argparse.SUPPRESS)
In [586]: p.parse_args('1 2'.split())
Out[586]: Namespace(str=['1', '2'])
In [588]: p.parse_args('1 2 3 4'.split())
usage: ipython3 [-h] str [str] [str]
ipython3: error: unrecognized arguments: 4
(oops, this is coded for 1-3 argument, but you get the idea).

Using the same option multiple times in Python's argparse

I'm trying to write a script that accepts multiple input sources and does something to each one. Something like this
./my_script.py \
-i input1_url input1_name input1_other_var \
-i input2_url input2_name input2_other_var \
-i input3_url input3_name
# notice inputX_other_var is optional
But I can't quite figure out how to do this using argparse. It seems that it's set up so that each option flag can only be used once. I know how to associate multiple arguments with a single option (nargs='*' or nargs='+'), but that still won't let me use the -i flag multiple times. How do I go about accomplishing this?
Just to be clear, what I would like in the end is a list of lists of strings. So
[["input1_url", "input1_name", "input1_other"],
["input2_url", "input2_name", "input2_other"],
["input3_url", "input3_name"]]

Here's a parser that handles a repeated 2 argument optional - with names defined in the metavar:
parser=argparse.ArgumentParser()
parser.add_argument('-i','--input',action='append',nargs=2,
metavar=('url','name'),help='help:')
In [295]: parser.print_help()
usage: ipython2.7 [-h] [-i url name]
optional arguments:
-h, --help show this help message and exit
-i url name, --input url name
help:
In [296]: parser.parse_args('-i one two -i three four'.split())
Out[296]: Namespace(input=[['one', 'two'], ['three', 'four']])
This does not handle the 2 or 3 argument case (though I wrote a patch some time ago for a Python bug/issue that would handle such a range).
How about a separate argument definition with nargs=3 and metavar=('url','name','other')?
The tuple metavar can also be used with nargs='+' and nargs='*'; the 2 strings are used as [-u A [B ...]] or [-u [A [B ...]]].

This is simple; just add both action='append' and nargs='*' (or '+').
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-i', action='append', nargs='+')
args = parser.parse_args()
Then when you run it, you get
In [32]: run test.py -i input1_url input1_name input1_other_var -i input2_url i
...: nput2_name input2_other_var -i input3_url input3_name
In [33]: args.i
Out[33]:
[['input1_url', 'input1_name', 'input1_other_var'],
['input2_url', 'input2_name', 'input2_other_var'],
['input3_url', 'input3_name']]

-i should be configured to accept 3 arguments and to use the append action.
>>> p = argparse.ArgumentParser()
>>> p.add_argument("-i", nargs=3, action='append')
_AppendAction(...)
>>> p.parse_args("-i a b c -i d e f -i g h i".split())
Namespace(i=[['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']])
To handle an optional value, you might try using a simple custom type. In this case, the argument to -i is a single comma-delimited string, with the number of splits limited to 2. You would need to post-process the values to ensure there are at least two values specified.
>>> p.add_argument("-i", type=lambda x: x.split(",", 2), action='append')
>>> print p.parse_args("-i a,b,c -i d,e -i g,h,i,j".split())
Namespace(i=[['a', 'b', 'c'], ['d', 'e'], ['g', 'h', 'i,j']])
For more control, define a custom action. This one extends the built-in _AppendAction (used by action='append'), but just does some range checking on the number of arguments given to -i.
class TwoOrThree(argparse._AppendAction):
def __call__(self, parser, namespace, values, option_string=None):
if not (2 <= len(values) <= 3):
raise argparse.ArgumentError(self, "%s takes 2 or 3 values, %d given" % (option_string, len(values)))
super(TwoOrThree, self).__call__(parser, namespace, values, option_string)
p.add_argument("-i", nargs='+', action=TwoOrThree)

Adding with Other in this Thread.
If you use action='append' in add_argument() then you will get arguments in list(s) within a list every time you add the option.
As you liked:
[
["input1_url", "input1_name", "input1_other"],
["input2_url", "input2_name", "input2_other"],
["input3_url", "input3_name"]
]
But if anyone wants those arguments in the same list[], then use action='extend' instead of  action='append' in your code. This will give you those arguments in a single list.
[
"input1_url",
"input1_name",
"input1_other",
"input2_url",
"input2_name",
"input2_other",
"input3_url",
"input3_name"
]