Develop Reference

Customizing optional arguments to include their own positional argument(s)

I would like to have a script which takes command-line arguments including flag options which take positional arguments themselves. I might expect the command line input to look something like
$ ./script.py [-o <file1> <file2> ...] inputfile
The official argparse documentation most similarly talks about
parser.add_argument("-v", "--verbosity", type=int, help="increase output verbosity")
args = parser.parse_args()
where the user inputs a single positional sub-argument (perhaps out of a set of choices) following the -v flag. This positional sub-argument is then stored in args.verbosity.
Thus it appears the flag's argument needs to be included in the same add_argument() line. Can you declare any special name for this sub-argument's variable (say, args.outputfile1)? Can the flag take more than one sub-argument? Can you adjust how the sub-variable looks in the help menu? By default it is something like-o OUTPUT, --output OUTPUT Save output data to a file OUTPUT. Can we change it to read -o <SomethingElse>?
Is there any more documentation which discusses this aspect?

With a definition like:
parser.add_argument('-o','--output', dest='output_file_name', nargs='+')
you can give a commandline like:
$ ./script.py -o file1 file2
and get args.output_file_name equal to ['file1','file2']. The '+' means one or more arguments (other nargs values are documented).
But
$ ./script.py -o file1 file2 an_input_file
where 'an_input_file' goes to a positional argument is harder to achieve. '*' is greedy, taking everything, leaving nothing for the positional. It's better to define another optional
parser.add_argument('-i','--input')
$ ./script.py -o file1 file2 -i an_input_file
If the '-o' is defined as an 'append' Action, you can use:
$ ./script.py -o file1 -o file2 -i an_input_file
In general you get the best control by using optionals. Positionals, because they 'parse' by position, not value, are harder to use in fancy combinations.
The metavar parameter lets you change the help display.

Your proposed interface is pretty unusual. It's more common to have people specify the same option multiple times, because tt's hard to distinguish ./script.py [-o file1 file2 ...] input file from ./script.py [-o file1] file2 inputfile. That might not be an issue yet, but as you add options or god-forbid arguments, your unusual design will become an issue.
I would recommend doing one of the following solutions:
1. Repeat the option flag
./script.py -o file1 -o file2 inputfile
2. Make your option a boolean flag
Change your API so -o indicates that all arguments except the last one are output files:
./script.py -o output1 output2 ... inputfileislast

argparser conditional fields depending in options

Currently I have a CLI tool that I'm building, and I want to give it this form similar to this.
usage: PROG SUBPARSER [-h]
(-l {optionA,optionB,optionC} | -s TERM [-a, [-b, [-c]]])
What I'm doing is I have a main cli module, which will import all the argument_parser function from all the modules I want to expose to the user, and it dynamically adds them to the main parser as sub parsers
the python code bellow it's a bit from a function that adds the parser to the main parser, being parser an object from type ArgumentParser which could be a root parser, or a sub parser. (I do this to each module, so they have their methods exposed as CLI).
Now what I'm trying to do in this particular case, is to have a command let's say PROG with a first argument SUBPARSER that has two (possibly more) mutually exclusive sequences of arguments (without creating a new sub parsers), saying I have two functions, search and list so
search and list could have common arguments (which will be assing to to the sub parser not the group) but there is also flags and arguments that are for use exclusively with --list or --search, in order to build commands like
PROG SUBARSER --list optionA -a -o -b
PROG SUBARSER --list optionA -a -o
PROG SUBARSER --list optionA -a -b
PROG SUBARSER --list optionA -a
PROG SUBARSER --list optionA
PROG SUBARSER --search TERM -a -k
PROG SUBARSER --search TERM -c
PROG SUBARSER --search TERM
I tried adding nested groups, with mutually exclusive and regular groups to the parser, but It doesn't allow me to (or at least I haven't found the way), to have mutually exclusive groups with multiple arguments, not just one flag or attribute.
This is what I have so far, that doesn't crash and actually runs usefully.
usage: PROG SUBPARSER [-h]
[-l {all,draft,staged,publish,build,not-build} | -s SEARCH]
def argument_parser(parser):
"""Argument parser for SUBPARSER subgroup"""
group = parser.add_mutually_exclusive_group(required=False)
group.add_argument('-l','--list',
choices=status_criterias,
help='List the content specified',
default='all'
)
group.add_argument('-s','--search',
help='Search by title from all the content')
Please don't mind the help strings.
Any help?

Python argparse: Leading dash in argument

I'm using Python's argparse module to parse command line arguments. Consider the following simplified example,
# File test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store')
parser.add_argument('-a', action='append')
args = parser.parse_args()
print(args)
which can successfully be called like
python test.py -s foo -a bar -a baz
A single argument is required after -s and after each -a, which may contain spaces if we use quotation. If however an argument starts with a dash (-) and does not contain any spaces, the code crashes:
python test.py -s -begins-with-dash -a bar -a baz
error: argument -s: expected one argument
I get that it interprets -begins-with-dash as the beginning of a new option, which is illegal as -s has not yet received its required argument. It's also pretty clear though that no option with the name -begins-with-dash has been defined, and so it should not interpret it as an option in the first place. How can I make argparse accept arguments with one or more leading dashes?

You can force argparse to interpret an argument as a value by including an equals sign:
python test.py -s=-begins-with-dash -a bar -a baz
Namespace(a=['bar', 'baz'], s='-begins-with-dash')

If you are instead trying to provide multiple values to one argument:
parser.add_argument('-a', action='append', nargs=argparse.REMAINDER)
will grab everything after -a on the command line and shove it in a.
python test.py -toe -a bar fi -fo -fum -s fee -foo
usage: test.py [-h] [-s S] [-a ...]
test.py: error: unrecognized arguments: -toe
python test.py -a bar fi -fo -fum -s fee -foo
Namespace(a=[['bar', 'fi', '-fo', '-fum', '-s', 'fee', '-foo']], s=None)
Note that even though -s is a recognized argument, argparse.REMAINDER adds it to the list of args found by -a since it is after -a on the command line

Argparse parse only before positional argument

How can I have argparse only parse commands that come after a positional argument?
Aka if I have the command pythonfile.py -d dir -e test pos_cmd_1 -d
How can I have it so that the first -d is parsed by argparse, and anything after the positional command is parsed by that command itself (read pos_cmd_1 -d as a single argument basically)
So that the argument list would be
pythonfile.py
-d dir
-e test
pos_cmd_1 -d -s -etc
So anything before the positional command would be optional. And anything after the positional command would be part of the positional command itself.
Edit: When trying to run the command with double dashes, it tells me that the arguments that come after aren't recognized.
pythonfile.py -d testdir -e test -- command -d -s
It says -d -s are unrecognized arguments instead of bundling them with the command.

you can achieve this just by slightly changing your command line to
pythonfile.py -d dir -e test -- pos_cmd_1 -d
by adding --, you tell argparse to stop looking for options. So all remaining arguments are set in the positional argument list instead.
An alternative is quoting the rest of arguments:
pythonfile.py -d dir -e test "pos_cmd_1 -d"
and (because it creates just one positional argument) use argument parser again on the splitted string (not ideal if you want to pass quoted strings in those args)
The advantages of those approaches is that they're natively supported by argparse, getopt and also that is a standard mechanism that won't surprise the users of your command.
if you want to stick to your approach, maybe you could pre-process argument list to insert the double dash by detecting 2 non-option arguments in a row:
args = "-d dir -e test pos_cmd_1 -d".split()
oldarg=""
for i,a in enumerate(args):
if oldarg and oldarg[0]!='-' and a[0]!='-':
args.insert(i,'--')
break
oldarg = a
args is now: ['-d', 'dir', '-e', 'test', '--', 'pos_cmd_1', '-d']

With the simple parser:
In [2]: p = argparse.ArgumentParser()
In [3]: p.add_argument('-d');
In [4]: p.add_argument('-e');
In [5]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E]
ipython3: error: argument -d: expected one argument
It tries to parse the last '-d' and hits an error. parse_known_args doesn't help.
With strings other than '-d' and '-e' parse_known_args works:
In [7]: p.parse_known_args('-d dir -e test pos_cmd_1 -s'.split())
Out[7]: (Namespace(d='dir', e='test'), ['pos_cmd_1', '-s'])
A positional with a REMAINDER nargs appears to work:
In [8]: a1 = p.add_argument('rest', nargs='...') # argparse.REMAINDER
In [9]: p.parse_args('-d dir -e test pos_cmd_1 -s'.split())
Out[9]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-s'])
In [10]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
Out[10]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-d'])
REMAINDER is supposed to work much like the '--', capturing input for use by another parser or command.
It can have problems if it's expected to catch the whole commandline, as in:
In [12]: p.parse_args('-s pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E] ...
ipython3: error: unrecognized arguments: -s
https://docs.python.org/3/library/argparse.html#nargs

accessing multiple files using argparse

I'm trying to read number of files using argparse:
parser.add_argument(
'-f',
'--text-file',
metavar='IN FILE',
type=argparse.FileType('r'),
nargs='*')
...
...
args = parser.parse_args()
print args
when more than one files are passed as command line arguments, only last file appears into args:
python example.py -o x.xml -s sss -c ccc -t "hello world" --report_failure -f ex.1 -f ex.2
Namespace(outputfile=<open file 'x.xml', mode 'w' at 0x028AD4F0>, report_failure=True, test_case='ccc', test_suite='sss', text='hello world', text_file=[<open file 'ex.2', mode 'r' at 0x028AD5A0>])
What I did wrong and how to access all files I passed from the command line?
Note: I'm using python 2.7.6 on Windows.

The complication occurs because you're passing multiple arguments to the same parameter -f and each argument replaces the argument before it. What would work in this case is:
python example.py -o x.xml -s sss -c ccc -t "hello world"
--report_failure -f ex.1 ex.2
This will collect ex.1 and ex.2 into a list, which is what I assume you want to do.
as a reference here is the docs on nargs:
'*'. All command-line arguments present are gathered into a list. Note
that it generally doesn’t make much sense to have more than one
positional argument with nargs='', but multiple optional arguments
with nargs='' is possible.
https://docs.python.org/3/library/argparse.html#nargs