I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?
With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.
The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
Related
I'm using Python's argparse module to parse command line arguments. Consider the following simplified example,
# File test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store')
parser.add_argument('-a', action='append')
args = parser.parse_args()
print(args)
which can successfully be called like
python test.py -s foo -a bar -a baz
A single argument is required after -s and after each -a, which may contain spaces if we use quotation. If however an argument starts with a dash (-) and does not contain any spaces, the code crashes:
python test.py -s -begins-with-dash -a bar -a baz
error: argument -s: expected one argument
I get that it interprets -begins-with-dash as the beginning of a new option, which is illegal as -s has not yet received its required argument. It's also pretty clear though that no option with the name -begins-with-dash has been defined, and so it should not interpret it as an option in the first place. How can I make argparse accept arguments with one or more leading dashes?
You can force argparse to interpret an argument as a value by including an equals sign:
python test.py -s=-begins-with-dash -a bar -a baz
Namespace(a=['bar', 'baz'], s='-begins-with-dash')
If you are instead trying to provide multiple values to one argument:
parser.add_argument('-a', action='append', nargs=argparse.REMAINDER)
will grab everything after -a on the command line and shove it in a.
python test.py -toe -a bar fi -fo -fum -s fee -foo
usage: test.py [-h] [-s S] [-a ...]
test.py: error: unrecognized arguments: -toe
python test.py -a bar fi -fo -fum -s fee -foo
Namespace(a=[['bar', 'fi', '-fo', '-fum', '-s', 'fee', '-foo']], s=None)
Note that even though -s is a recognized argument, argparse.REMAINDER adds it to the list of args found by -a since it is after -a on the command line
python test.py --arg -foo -bar
test.py: error: argument --arg: expected at least one argument
python test.py --arg -8
['-8']
How do I allow -non_number to work with argparse?
Is there a way to disable short arguments?
Call it like this:
python test.py --arg='-foo'
To allow specifying multiple:
parser.add_argument('--arg', action='append')
# call like python test.py --arg=-foo --arg=-bar
I think you're looking for the nargs parameter to argparser.
parser.add_argument('--arg', nargs='?')
At the moment, --arg is interpreting the value '-8' as the input, whereas it thinks '-f' (with parameters 'oo') is a new argument.
Alternatively, you could use action='store_true', which will represent the presence or absence of the argument with a boolean.
parser.add_argument('--arg', action='store_true')
https://docs.python.org/3/library/argparse.html#nargs
I have code:
parser = ArgumentParser()
parser.add_argument('--verbose', action='count', default=0, help='debug output')
subparsers = parser.add_subparsers(help='subparser')
parser1 = subparsers.add_parser('action', help='Do something')
parser1.add_argument('--start', action='store_true', help='start')
parser1.add_argument('--stop', action='store_true', help='stop')
parser2 = subparsers.add_parser('control', help='Control something')
parser2.add_argument('--input', action='store_true', help='start')
parser2.add_argument('--output', action='store_true', help='stop')
args = parser.parse_args()
Then I can run script:
script.py --verbose action --start
script.py --verbose control --output
but not
script.py action --start --verbose
script.py control --output --verbose
Can I transfer option --verbose to the end, without adding it to each group?
To elaborate on my comment:
argparse parses the input list (sys.argv[1:]) in order, matching the strings with the Actions (add_argument object). So if the command is
python foo.py --arg1=3 cmd --arg2=4
it tries to handle '--arg1', then 'cmd'. If 'cmd' matches a subparser name, it then delegates the parsing to that parser, giving the remaining strings to it. If the cmd subparser can handle --arg2, it returns that as an unrecognized argument.
The main parser does not resume parsing. Rather it just handles the unrecognized arguments as it normally would - raising an error if using parse_args, and returning them in the extras list if using parse_known_args.
So if you want to put --verbose at the end, you have define it as a subparser argument. Or do some further parsing after parse_known_args.
You are allowed to define --verbose at both levels, though sometimes such a definition can create conflicts (especially if defaults differ).
The parents mechanism can be used to reduce the amount of typing, though you could just as easily write your own utility functions.
I have the following code for reading the arguments from a file and process them using argparse, but I am getting an error, why is this the case?
import argparse
from ConfigParser import ConfigParser
import shlex
parser = argparse.ArgumentParser(description='Short sample app',
fromfile_prefix_chars='#')
parser.add_argument('--abool', action="store_true", default=False)
parser.add_argument('--bunit', action="store", dest="bunit",type=int)
parser.add_argument('--cpath', action="store", dest="c", type=str)
print parser.parse_args(['#argparse_fromfile_prefix_chars.txt']) #name of the file is argparse_fromfile_prefix_chars.txt
Error:
usage: -c [-h] [--abool] [--bunit BUNIT] [--cpath C]
-c: error: unrecognized arguments: --bunit 289 --cpath /path/to/file.txt
To exit: use 'exit', 'quit', or Ctrl-D.
Contents of the file argparse_fromfile_prefix_chars.txt
--abool
--bunit 289
--cpath /path/to/file.txt
argparse expects arguments from files to be one per line. Meaning the whole line is one quoted argument. So your current args file is interpreted as
python a.py '--abool' '--bunit 289' '--cpath /path/to/file.txt'
which causes the error. Instead, your args file should look like this
--abool
--bunit
289
--cpath
/path/to/file.txt
The documentation for fromfile_prefix_chars states:
Arguments read from a file must by default be one per line (but see
also convert_arg_line_to_args()) and are treated as if they were in
the same place as the original file referencing argument on the
command line.
Note that one argument does not mean one option followed by all its arguments. It means a command line argument. Currently the whole lines are interpreted as if they were a single argument.
In other words your file should look like:
--abool
--bunit
289
--cpath
/path/to/file.txt
Alternatively you can override the convert_arg_line_to_args() method to parse the file in an other way. The documentation already provides an implementation that parses white-space separated arguments instead of line-separated arguments:
def convert_arg_line_to_args(self, arg_line):
# consider using shlex.split() instead of arg_line.split()
for arg in arg_line.split():
if not arg.strip():
continue
yield arg
I believe you can either subclass ArgumentParser and reimplement this method, or, probably, even setting the attribute on an ArgumentParser instance should work.
For some reason the default implementation of convert_arg_line_to_args doesn't work properly:
$echo '--abool
--bunit
289
--cpath
/here/is/a/path
' > file.txt
$cat test_argparse.py
import argparse
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('--abool', action='store_true')
parser.add_argument('--bunit', type=int)
parser.add_argument('--cpath')
print(parser.parse_args(['#file.txt']))
$python test_argparse.py
usage: test_argparse.py [-h] [--abool] [--bunit BUNIT] [--cpath CPATH]
test_argparse.py: error: unrecognized arguments:
However if you use the implementation above it works:
$cat test_argparse.py
import argparse
def convert_arg_line_to_args(arg_line):
for arg in arg_line.split():
if not arg.strip():
continue
yield arg.strip()
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('--abool', action='store_true')
parser.add_argument('--bunit', type=int)
parser.add_argument('--cpath')
parser.convert_arg_line_to_args = convert_arg_line_to_args
print(parser.parse_args(['#file.txt']))
$python test_argparse.py
Namespace(abool=True, bunit=289, cpath='/here/is/a/path')
An other workaround is to use the --option=argument syntax:
--abool
--bunit=289
--cpath=/the/path/to/file.txt
However this will not work when an option has more than one argument. In such a case you have to use a different implementation of convert_arg_line_to_args.
Trying to debug, it seems like the convert_line_arg_to_args gets called with an empty string which gets added to the arguments, and the empty string is considered an argument (which isn't defined).
The problem is that there are two newlines at the end of the file.
In fact if you create the file without this double newline at the end, it works:
$echo -n '--abool
--bunit
289
--cpath
/here/is/a/path
' > file.txt
$python test_argparse.py
Namespace(abool=True, bunit=289, cpath='/here/is/a/path')
(echo -n doesn't add a newline at the end of the output).
I'm trying to pass a list of arguments with argparse but the only way that I've found involves rewriting the option for each argument that I want to pass:
What I currently use:
main.py -t arg1 -a arg2
and I would like:
main.py -t arg1 arg2 ...
Here is my code:
parser.add_argument("-t", action='append', dest='table', default=[], help="")
Use nargs:
ArgumentParser objects usually associate a single command-line
argument with a single action to be taken. The nargs keyword argument
associates a different number of command-line arguments with a single
action.
For example, if nargs is set to '+'
Just like '*', all command-line args present are gathered into a list.
Additionally, an error message will be generated if there wasn’t at
least one command-line argument present.
So, your code would look like
parser.add_argument('-t', dest='table', help='', nargs='+')
That way -t arguments will be gathered into list automatically (you don't have to explicitly specify the action).
Being aware, you asked for argparse solution, I would like to present alternative solution using package docopt
Install it first:
$ pip install docopt
Write the code:
"""Usage:
main.py -a <arg>...
"""
if __name__ == "__main__":
from docopt import docopt
resargs = docopt(__doc__)
print resargs
Run it to show usage instrucitons:
$ python main.py
Usage:
main.py -a <arg>...
Call it with your parameters:
$ python main.py -a AA BB CC
{'-a': True,
'<arg>': ['AA', 'BB', 'CC']}
Btw. if you do not need the -a option, you shall directly allow passing the arguments. It makes usage simpler to the user.