Develop Reference

How to declare an optional argument to the parser which disables compulsory argument?

The title may be confusing but I can't think of a better explanation. Basically, I have a program which operates on some input file with a bunch of optional arguments. The input file is compulsory for my program. So I wrote a parser like this:
test.py
from argparse import ArgumentParser
parser = ArgumentParser(prog="prog")
parser.add_argument("file", help="input file")
parser.add_argument("--debug", action="store_true", help="enable debug messages")
parser.parse_args()
I'm not showing all the options for simplicity. Now, I want to add a feature to this program which don't require input file and runs on its own. Let's call this option as a. Option a takes unknown number of arguments and --debug or other options are not valid for option a. So the following would be valid runs for my program:
$ python3 test.py input.txt
$ python3 test.py input.txt --debug
$ python3 test.py -a 1 2 3
And these would be invalid:
$ python3 test.py input.txt -a 1 2 3
$ python3 test.py -a 1 2 3 --debug
When I add this line to my code
parser.add_argument("-a", nargs="+", help="help option a")
It accepts option a but it says "file is required" for understandable reasons. So how should I modify my code to achieve my goal?
P.S: I also tried using subparser but I couldn't manage to make it work as I want probably because my lack of knowledge.

Customizing optional arguments to include their own positional argument(s)

I would like to have a script which takes command-line arguments including flag options which take positional arguments themselves. I might expect the command line input to look something like
$ ./script.py [-o <file1> <file2> ...] inputfile
The official argparse documentation most similarly talks about
parser.add_argument("-v", "--verbosity", type=int, help="increase output verbosity")
args = parser.parse_args()
where the user inputs a single positional sub-argument (perhaps out of a set of choices) following the -v flag. This positional sub-argument is then stored in args.verbosity.
Thus it appears the flag's argument needs to be included in the same add_argument() line. Can you declare any special name for this sub-argument's variable (say, args.outputfile1)? Can the flag take more than one sub-argument? Can you adjust how the sub-variable looks in the help menu? By default it is something like-o OUTPUT, --output OUTPUT Save output data to a file OUTPUT. Can we change it to read -o <SomethingElse>?
Is there any more documentation which discusses this aspect?

With a definition like:
parser.add_argument('-o','--output', dest='output_file_name', nargs='+')
you can give a commandline like:
$ ./script.py -o file1 file2
and get args.output_file_name equal to ['file1','file2']. The '+' means one or more arguments (other nargs values are documented).
But
$ ./script.py -o file1 file2 an_input_file
where 'an_input_file' goes to a positional argument is harder to achieve. '*' is greedy, taking everything, leaving nothing for the positional. It's better to define another optional
parser.add_argument('-i','--input')
$ ./script.py -o file1 file2 -i an_input_file
If the '-o' is defined as an 'append' Action, you can use:
$ ./script.py -o file1 -o file2 -i an_input_file
In general you get the best control by using optionals. Positionals, because they 'parse' by position, not value, are harder to use in fancy combinations.
The metavar parameter lets you change the help display.

Your proposed interface is pretty unusual. It's more common to have people specify the same option multiple times, because tt's hard to distinguish ./script.py [-o file1 file2 ...] input file from ./script.py [-o file1] file2 inputfile. That might not be an issue yet, but as you add options or god-forbid arguments, your unusual design will become an issue.
I would recommend doing one of the following solutions:
1. Repeat the option flag
./script.py -o file1 -o file2 inputfile
2. Make your option a boolean flag
Change your API so -o indicates that all arguments except the last one are output files:
./script.py -o output1 output2 ... inputfileislast

Python argparse: Leading dash in argument

I'm using Python's argparse module to parse command line arguments. Consider the following simplified example,
# File test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store')
parser.add_argument('-a', action='append')
args = parser.parse_args()
print(args)
which can successfully be called like
python test.py -s foo -a bar -a baz
A single argument is required after -s and after each -a, which may contain spaces if we use quotation. If however an argument starts with a dash (-) and does not contain any spaces, the code crashes:
python test.py -s -begins-with-dash -a bar -a baz
error: argument -s: expected one argument
I get that it interprets -begins-with-dash as the beginning of a new option, which is illegal as -s has not yet received its required argument. It's also pretty clear though that no option with the name -begins-with-dash has been defined, and so it should not interpret it as an option in the first place. How can I make argparse accept arguments with one or more leading dashes?

You can force argparse to interpret an argument as a value by including an equals sign:
python test.py -s=-begins-with-dash -a bar -a baz
Namespace(a=['bar', 'baz'], s='-begins-with-dash')

If you are instead trying to provide multiple values to one argument:
parser.add_argument('-a', action='append', nargs=argparse.REMAINDER)
will grab everything after -a on the command line and shove it in a.
python test.py -toe -a bar fi -fo -fum -s fee -foo
usage: test.py [-h] [-s S] [-a ...]
test.py: error: unrecognized arguments: -toe
python test.py -a bar fi -fo -fum -s fee -foo
Namespace(a=[['bar', 'fi', '-fo', '-fum', '-s', 'fee', '-foo']], s=None)
Note that even though -s is a recognized argument, argparse.REMAINDER adds it to the list of args found by -a since it is after -a on the command line

Configure argparse to accept quoted arguments

I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?

With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.

The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')

argparse, two arguments depend on each other

I would like to make the parser like cmd [-a xxx -b xxx] -c xxx -d xxx
When -a is used, I want -b to be used too. likewise, if -b is used, -a must be used too. It's ok both -a and -b are not used.
How do I do that? I have tried custom actions, but it does not go well.

A better design would be to have a single option that takes two arguments:
parser.add_argument('-a', nargs=2)
Then you either specify the option with 2 arguments, or you don't specify it at all.
$ script -a 1 2
or
$ script
A custom action (or postprocessing) can split the tuple args.a into two separate values args.a and args.b.

Argparse doesn't natively support this type of use.
The most effective thing to do is check and see if those types of conditions are met after parsing:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parser.add_argument('-c')
args = parser.parse_args()
required_together = ('b','c')
# args.b will be None if b is not provided
if not all([getattr(args,x) for x in required_together]):
raise RuntimeError("Cannot supply -c without -b")