I'm trying to pass a list of arguments with argparse but the only way that I've found involves rewriting the option for each argument that I want to pass:
What I currently use:
main.py -t arg1 -a arg2
and I would like:
main.py -t arg1 arg2 ...
Here is my code:
parser.add_argument("-t", action='append', dest='table', default=[], help="")
Use nargs:
ArgumentParser objects usually associate a single command-line
argument with a single action to be taken. The nargs keyword argument
associates a different number of command-line arguments with a single
action.
For example, if nargs is set to '+'
Just like '*', all command-line args present are gathered into a list.
Additionally, an error message will be generated if there wasn’t at
least one command-line argument present.
So, your code would look like
parser.add_argument('-t', dest='table', help='', nargs='+')
That way -t arguments will be gathered into list automatically (you don't have to explicitly specify the action).
Being aware, you asked for argparse solution, I would like to present alternative solution using package docopt
Install it first:
$ pip install docopt
Write the code:
"""Usage:
main.py -a <arg>...
"""
if __name__ == "__main__":
from docopt import docopt
resargs = docopt(__doc__)
print resargs
Run it to show usage instrucitons:
$ python main.py
Usage:
main.py -a <arg>...
Call it with your parameters:
$ python main.py -a AA BB CC
{'-a': True,
'<arg>': ['AA', 'BB', 'CC']}
Btw. if you do not need the -a option, you shall directly allow passing the arguments. It makes usage simpler to the user.
Related
I am having some trouble with argparse. My goal is to have the user select one and only one option (-a, -b, -c, etc.) and then the arguments for that option. I'm using subparsers to do this:
parser_iq = subparsers.add_parser('iq', help='iq help')
parser_iq.add_argument("-iq", "--index_query", nargs="+", action=required_length(1,6),type=organize_args, help="Choose an index to query for. Start-date, end-date, "\
"csv, json, stdout are all optional")
This is just one of the subparsers I plan to have.
Problem: When running this in the command line:
python3.6 main.py iq "index_name_here"
I get the error that "index_name_here" is unrecognized. I am parsing it like this:
args = parser.parse_args()
I found some problems similar to mine, but they were passing in sys.argv into parse_args(), which was their issue.
How can I make it so that argparse will recognize the arguments passed? Also, is there a way to have only one option passed in at a time? For example:
Correct:
main.py option1 arg1 arg2
Wrong:
main.py option1 option2 arg1 arg2
Thank you!
You have to pass the value like python3.6 main.py -iq "index_name_here" (i.e., use -iq, not iq).
As far as making mutually exclusive arguments, subparsers is, from what I understand, the way to go, but I can't give much in the way of guidance on how to proceed on that.
Edit:
In response to your comment, does the following work:
python3.6 main.py iq -iq "index_name_here"
?
I'm using argparse with several subparsers. I want my program to take options for verbosity anywhere in the args, including the subparser.
from argparse import ArgumentParser
p = ArgumentParser()
p.add_argument('--verbose', '-v', action='count')
sub = p.add_subparsers()
a = sub.add_parser('a')
print(p.parse_args())
By default, options for the main parser will throw an error if used for subparsers:
$ python tmp.py -v a
Namespace(verbose=1)
$ python tmp.py a -v
usage: tmp.py [-h] [--verbose] {a} ...
tmp.py: error: unrecognized arguments: -v
I looked into parent parsers, from this answer.
from argparse import ArgumentParser
parent = ArgumentParser(add_help=False)
parent.add_argument('--verbose', '-v', action='count')
main = ArgumentParser(parents=[parent])
sub = main.add_subparsers()
a = sub.add_parser('a', parents=[parent])
print(main.parse_args())
For some reason though, none of the shared flags work on the main parser.
$ python tmp2.py a -vvv
Namespace(verbose=3)
$ python tmp2.py -vvv a
Namespace(verbose=None)
Note that the main parser definitely has the appropriate arguments, because when I change it to main = ArgumentParser() I get error: unrecognized arguments: -v. What am I missing here?
First, a couple of general comments.
The main parser handles the input upto the subparser invocation, then the subparser is called and given the remaining argv. When it is done, it's namespace is merged back into the the main namespace.
The parents mechanism copies Actions from the parent by reference. So your main and subparsers share the same verbose Action object. That's been a problem when the subparser tries to set a different default or help. It may not be an issue here, but just keep it in mind.
Even without the parents mechanism, sharing a dest or options flag between main and subparser can be tricky. Should the default of the subparser Action be used? What if both are used? Does the subparser overwrite the main parser's actions?
Originally the main namespace was passed to the subparser, which it modified and returned. This was changed a while back (I can find the bug/issue if needed). Now the subparser starts with a default empty namespace, fills it. And these values are then merged into the main.
So in your linked SO question, be wary of older answers. argparse may have changed since then.
I think what's happening in your case is that the main and subparser verbose are counting separately. And when you get None it's the subparser's default that you see.
The __call__ for _Count_Action is:
def __call__(self, parser, namespace, values, option_string=None):
new_count = _ensure_value(namespace, self.dest, 0) + 1
setattr(namespace, self.dest, new_count)
I suspect that in older argparse when the namespace was shared, the count would have been cumulative, but I can't test it without recreating an older style subparser action class.
https://bugs.python.org/issue15327 - here the original developer suggests giving the two arguments different dest. That records the inputs from both main and sub. Your own code can then merge the results if needed.
https://bugs.python.org/issue27859 argparse - subparsers does not retain namespace. Here I suggest a way of recreating the older style.
https://bugs.python.org/issue9351 argparse set_defaults on subcommands should override top level set_defaults - this is the issue in 2014 that changed the namespace use.
My workaround for this behavior, which is very well described in #hpaulj's answer is to create a second parser that does not have subparsers but only the positional arguments that were first found.
The first parse_args, used with the first parser, will validate the positional arguments and flags, show an error message if needed or show the proper help.
The second parse_args, for the second parser, will correctly fill in the namespace.
Building on your example:
from argparse import ArgumentParser
parent = ArgumentParser(add_help=False)
parent.add_argument('--verbose', '-v', action='count')
main1 = ArgumentParser(parents=[parent])
sub = main1.add_subparsers()
# eg: tmp.py -vv a -v
a = sub.add_parser('a', parents=[parent])
a.set_defaults(which='a')
# eg: tmp.py -vv v -v --output toto
b = sub.add_parser('b', parents=[parent])
b.add_argument('--output', type=str)
b.set_defaults(which='b')
args = main1.parse_args()
print(args)
# parse a second time with another parser
main2 = ArgumentParser(parents=[parent])
if args.which == 'a':
main2.add_argument('a')
elif args.which == 'b':
main2.add_argument('b')
main2.add_argument('--output', type=str)
print(main2.parse_args())
Which gives:
$ ./tmp.py -vv a -v
Namespace(verbose=1, which='a')
Namespace(a='a', verbose=3)
$ ./tmp.py -vv b -v --output toto
Namespace(output='toto', verbose=1, which='b')
Namespace(b='b', output='toto', verbose=3)
$ ./tmp.py -vv a --output
usage: tmp.py [-h] [--verbose] {a,b} ...
tmp.py: error: unrecognized arguments: --output
I use this technique with multiple nested subparsers.
I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?
With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.
The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
I would like to make the parser like cmd [-a xxx -b xxx] -c xxx -d xxx
When -a is used, I want -b to be used too. likewise, if -b is used, -a must be used too. It's ok both -a and -b are not used.
How do I do that? I have tried custom actions, but it does not go well.
A better design would be to have a single option that takes two arguments:
parser.add_argument('-a', nargs=2)
Then you either specify the option with 2 arguments, or you don't specify it at all.
$ script -a 1 2
or
$ script
A custom action (or postprocessing) can split the tuple args.a into two separate values args.a and args.b.
Argparse doesn't natively support this type of use.
The most effective thing to do is check and see if those types of conditions are met after parsing:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parser.add_argument('-c')
args = parser.parse_args()
required_together = ('b','c')
# args.b will be None if b is not provided
if not all([getattr(args,x) for x in required_together]):
raise RuntimeError("Cannot supply -c without -b")
I've a Python app like this:
$ app.py -a arg1 -a arg2 -b file1.b -b file2.b
Depending on the option, in app.py, I use two different functions to process the input files. As each option can have several file arguments, I would like something like this:
$ app.py -a arg1 arg2 -b *.b
After searching on the net, I find these three modules: getopt, argparse, optparse
And I wrote a simple one for my app:
optionDict= {'-a':aFunction, '-b':bFunction}
for arg in sys.argv[1:]:
if arg in optionDict: # arg is an option here.
funcName = optionDict[arg]
else: # arg is not an option, then it's a fileName. Deal with this file with the function found according to previous arg.
funcName(arg)
My quesion: Are there any other modules or better ways to do this?
Since 2012 Python has an easy, powerful and very cool module for argument parsing called docopt. It works with Python from 2.5 to 3.3 and needs no installation. Here is the code for your particular case:
'''app.py
Usage:
app.py -a ARG ... -b FILE ...
app.py (-h | -v)
'''
from docopt import docopt
args = docopt(__doc__, version='1.0')
# now do something based on the value of args dict
So this is it: one line of code plus your doc string which is essential. I told you it's cool -- didn't I ;-)
If you found argparse to be too hard to get started with because of the complexity, then you would probably like plac which is available on PyPi. It is a wrapper around argparse which hides the complexity, and it takes care of a lot of boilerplate argument checking for you.
It works for both Python 2 and 3, and has a decorator that allows Python 2 users to get access to a nice Python 3 feature for function parameters. And you can even nest commands in one another so that you could have -arguments (beginning with dash) and .arguments (beginning with dot) on one command line.
Here are the docs http://plac.googlecode.com/hg/doc/plac.html
I think that argparse is the way to go. It does a great job, it is easy to use and in the standard library since 2.7. Take a look a the nargs argument. From the docs
Fixed elements to consume
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs=2)
parser.add_argument('bar', nargs=1)
parser.parse_args('c --foo a b'.split())
Namespace(bar=['c'], foo=['a', 'b'])
One arg will be consumed from the command line if possible, and produced as a single item. If no command-line arg is present, the value from default will be produced
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs='?', const='c', default='d')
parser.add_argument('bar', nargs='?', default='d')
parser.parse_args('XX --foo YY'.split())
Namespace(bar='XX', foo='YY')
parser.parse_args('XX --foo'.split())
Namespace(bar='XX', foo='c')
parser.parse_args(''.split())
Namespace(bar='d', foo='d')
All command-line args present are gathered into a list.
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs='*')
parser.add_argument('--bar', nargs='*')
parser.add_argument('baz', nargs='*')
parser.parse_args('a b --foo x y --bar 1 2'.split())
Namespace(bar=['1', '2'], baz=['a', 'b'], foo=['x', 'y'])