Develop Reference

How to pass and parse a list of strings from command line with argparse.ArgumentParser in Python?

I want to pass a list of names into my program written in Python from console. For instance, I would like to use a way similar to this (I know it shouldn't work because of bash):
$ python myprog.py -n name1 name2
So, I tried this code:
# myprog.py
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('-n', '--names-list', default=[])
args = parser.parse_args()
print(args.names_list) # I need ['name1', 'name2'] here
That led to the error:
usage: myprog.py [-h] [-n NAMES_LIST]
myprog.py: error: unrecognized arguments: name2
I know I could pass the names with quotes "name1 name2" and split it in my code args.names_list.split(). But I'm curious, is there a better way to pass the list of strings via argparse module.
Any ideas would be appreciated.
Thanks!

You need to define --names-list to take an arbitrary number of arguments.
parser.add_argument('-n', '--names-list', nargs='+', default=[])
Note that options with arbitrary number of arguments don't typically play well with positional arguments, though:
# Is this 4 arguments to -n, or
# 3 arguments and a single positional argument, or ...
myprog.py -n a b c d

You need to use nargs:
parser.add_argument('-n', '--names-list', nargs="*")
https://docs.python.org/3/library/argparse.html#nargs

parser.add_argument('-n', '--names-list', default=[], nargs='+')

How to use top-level arguments with subparsers in argparse

In Python's argparse, how do you implement top-level arguments while still using commands implemented as subparsers?
I'm trying to implement a --version argument to show the program's version number, but argparse is giving me error: too few arguments because I'm not specifying a sub-command for one of the subparsers.
My code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
'-v', '--version',
help='Show version.',
action='store_true',
default=False
)
subparsers = parser.add_subparsers(
dest="command",
)
list_parser = subparsers.add_parser('list')
parser.parse_args(['--version'])
the output:
usage: myscript.py [-h] [-v] {list} ...
myscript.py: error: too few arguments

If you only need version to work, you can do this:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
'-v', '--version',
action='version',
version='%(prog)s 1.0',
)
Subparsers won't bother any more; the special version action is processed and exits the script before the parser looks for subcommands.

The subparsers is a kind of positional argument. So normally that's required (just as though you'd specified add_argument('foo')).
skyline's suggestion works because action='version' is an action class that exits after displaying its information, just like the default -h.
There is bug/feature in the latest argparse that makes subparsers optional. Depending on how that is resolved, it may be possible in the future to give the add_subparsers command a required=False parameter. But the intended design is that subparsers will be required, unless a flagged argument (like '-h') short circuits the parsing.

Parsing multiple subparsers, but with global arguments

I have read quite a few questions and answers on how to define, parse and run multiple subparsers to run sth like
tool.py func_a -a 12 func_b -b 15 input.txt output.txt
^-- main parser args
^--------------- subparser b
^---------------------------- subparser a
They usually suggest something like:
def func_a(args):
pass
def func_b(args):
pass
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Define subparsers
subp_a = subparsers.add_parser(func_a.__name__)
subp_a.set_defaults(func=func_a)
subp_a.add_argument('-a')
subp_b = subparsers.add_parser(func_b.__name__)
subp_b.set_defaults(func=func_b)
subp_b.add_argument('-b')
# Define global parameters
parser.add_argument('input', type=argparse.FileType('r'))
parser.add_argument('output', type=argparse.FileType('wb', 0))
# Parse and run through all arguments
rest = sys.argv[1:]
while rest:
args, rest = parser.parse_known_args(rest)
args.func(args)
However this implementation has one problem: The arguments input and output are defined for the main parser and should only be used once. However, each call of parser.parse_known_args(rest) expects the values to be set and consequently removes it from rest.
This means the first call to parse_known_args retrieves the values and each subsequent call fails due to missing arguments.
Is there a solution to overcome this without manually copying the values into the rest list?

Yes, the first parse_known_args consumes the filenames along with the 1st subparser level.
You could define 2 parsers, one with the 'global' positionals, one without. First parse with the 'with' parser, hanging onto its args (for the filenames). Then work your way down the subparser stack with the 'without' parser.
Here's a simpler example:
parser = argparse.ArgumentParser()
parser.add_argument('foo')
subp = parser.add_subparsers(dest='cmd')
cmd1 = subp.add_parser('cmd1')
cmd1.add_argument('-a')
cmd2 = subp.add_parser('cmd2')
# parser.add_argument('bar') # confusing location
parser1 = argparse.ArgumentParser()
subp = parser1.add_subparsers(dest='cmd')
cmd1 = subp.add_parser('cmd1', parents=[cmd1], add_help=False)
cmd2 = subp.add_parser('cmd2')
# Parse and run through all arguments
args, rest = parser.parse_known_args()
print args, rest
while rest:
args, rest = parser1.parse_known_args(rest)
print args, rest
I put the parser foo before subp because it creates less confusion there. A positional at the end (bar) may be confused with arguments for one of the subparsers, or one of the nested subparsers - especially when generating error messages.
To get an idea of how parse_args will allocate argument strings to the various positionals, try a script like:
parser = argparse.ArgumentParser()
parser.add_argument('foo')
parser.add_argument('cmd', nargs=argparse.PARSER, choices=['cmd1','cmd2'])
parser.add_argument('bar')
print parser.parse_args()
To parser, the subparsers argument looks just a like a positional that takes at least one string (kind of like '+'), and the 1st must match choices. It will receive all the 'remaining' strings, consistent with the demands of the other positionals.

How to handle CLI subcommands with argparse

I need to implement a command line interface in which the program accepts subcommands.
For example, if the program is called “foo”, the CLI would look like
foo cmd1 <cmd1-options>
foo cmd2
foo cmd3 <cmd3-options>
cmd1 and cmd3 must be used with at least one of their options and the three cmd* arguments are always exclusive.
I am trying to use subparsers in argparse, but with no success for the moment. The problem is with cmd2, that has no arguments:
if I try to add the subparser entry with no arguments, the namespace returned by parse_args will not contain any information telling me that this option was selected (see the example below).
if I try to add cmd2 as an argument to the parser (not the subparser), then argparse will expect that the cmd2 argument will be followed by any of the subparsers arguments.
Is there a simple way to achieve this with argparse? The use case should be quite common…
Here follows what I have attempted so far that is closer to what I need:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='Functions')
parser_1 = subparsers.add_parser('cmd1', help='...')
parser_1.add_argument('cmd1_option1', type=str, help='...')
parser_2 = subparsers.add_parser(cmd2, help='...')
parser_3 = subparsers.add_parser('cmd3', help='...')
parser_3.add_argument('cmd3_options', type=int, help='...')
args = parser.parse_args()

First of all subparsers are never inserted in the namespace. In the example you posted if you try to run the script as:
$python3 test_args.py cmd1 1
Namespace(cmd1_option1='1')
where test_args.py contain the code you provided (with the import argparse at the beginning and print(args) at the end).
Note that there is no mention to cmd1 only to its argument. This is by design.
As pointed out in the comments you can add that information passing the dest argument to the add_subparsers call.
The usual way to handle these circumstances is to use the set_defaults method of the subparsers:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='Functions')
parser_1 = subparsers.add_parser('cmd1', help='...')
parser_1.add_argument('cmd1_option1', type=str, help='...')
parser_1.set_defaults(parser1=True)
parser_2 = subparsers.add_parser('cmd2', help='...')
parser_2.set_defaults(parser2=True)
parser_3 = subparsers.add_parser('cmd3', help='...')
parser_3.add_argument('cmd3_options', type=int, help='...')
parser_3.set_defaults(parser_3=True)
args = parser.parse_args()
print(args)
Which results in:
$python3 test_args.py cmd1 1
Namespace(cmd1_option1='1', parser1=True)
$python3 test_args.py cmd2
Namespace(parser2=True)
In general different subparser will, most of the time, handle the arguments in completely different ways. The usual pattern is to have different functions to run the different commands and use set_defaults to set a func attribute. When you parse the arguments you simply call that callable:
subparsers = parser.add_subparsers()
parser_1 = subparsers.add_parser(...)
parser_1.set_defaults(func=do_command_one)
parser_k = subparsers.add_parser(...)
parser_k.set_defaults(func=do_command_k)
args = parser.parse_args()
if args.func:
args.func(args)

The subparser identity can be added to the main Namespace if the add_subparsers command is given a dest.
From the documentation:
However, if it is necessary to check the name of the subparser that was invoked, the dest keyword argument to the add_subparsers() call will work:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers(dest='subparser_name')
>>> subparser1 = subparsers.add_parser('1')
>>> subparser1.add_argument('-x')
>>> subparser2 = subparsers.add_parser('2')
>>> subparser2.add_argument('y')
>>> parser.parse_args(['2', 'frobble'])
Namespace(subparser_name='2', y='frobble')
By default the dest is argparse.SUPPRESS, which keeps subparsers from adding the name to the namespace.

Creating mutually inclusive positional arguments with argparse

I'm trying to build a command line interface with Python's argparse module. I want two positional arguments where one depends on the other (mutually inclusive). Here is what I want:
prog [arg1 [arg2]]
Here's what I have so far:
prog [arg1] [arg2]
Which is produced by:
parser = argparse.ArgumentParser()
parser.add_argument('arg1', nargs='?')
parser.add_argument('arg2', nargs='?')
How do I get from there to having a mutually inclusive arg2?

Module argparse doesn't have options for creating mutually inclusive arguments.
However it's simple to write it by yourself.
Start with adding both arguments as optional:
parser.add_argument('arg1', nargs='?')
parser.add_argument('arg2', nargs='?')
After parsing arguments check if arg1 is set and arg2 is not:
args = parser.parse_args()
if args.arg1 and not args.arg2:
(this may be more tricky if you change default value from None for not used arguments to something different)
Then use parser.error() function to display normal argparse error message:
parser.error('the following arguments are required: arg2')
Finally change usage: message to show that arg2 depends on arg1:
parser = argparse.ArgumentParser(usage='%(prog)s [arg1 [arg2]]')
A complete script:
import argparse
parser = argparse.ArgumentParser(usage='%(prog)s [arg1 [arg2]]')
parser.add_argument('arg1', nargs='?')
parser.add_argument('arg2', nargs='?')
args = parser.parse_args()
if args.arg1 and not args.arg2:
parser.error('the following arguments are required: arg2')

You can do something similar to this using sub_parsers.
Here are the docs and examples:
http://docs.python.org/2/library/argparse.html#sub-commands