I'm using Python's argparse module to parse command line arguments. Consider the following simplified example,
# File test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store')
parser.add_argument('-a', action='append')
args = parser.parse_args()
print(args)
which can successfully be called like
python test.py -s foo -a bar -a baz
A single argument is required after -s and after each -a, which may contain spaces if we use quotation. If however an argument starts with a dash (-) and does not contain any spaces, the code crashes:
python test.py -s -begins-with-dash -a bar -a baz
error: argument -s: expected one argument
I get that it interprets -begins-with-dash as the beginning of a new option, which is illegal as -s has not yet received its required argument. It's also pretty clear though that no option with the name -begins-with-dash has been defined, and so it should not interpret it as an option in the first place. How can I make argparse accept arguments with one or more leading dashes?
You can force argparse to interpret an argument as a value by including an equals sign:
python test.py -s=-begins-with-dash -a bar -a baz
Namespace(a=['bar', 'baz'], s='-begins-with-dash')
If you are instead trying to provide multiple values to one argument:
parser.add_argument('-a', action='append', nargs=argparse.REMAINDER)
will grab everything after -a on the command line and shove it in a.
python test.py -toe -a bar fi -fo -fum -s fee -foo
usage: test.py [-h] [-s S] [-a ...]
test.py: error: unrecognized arguments: -toe
python test.py -a bar fi -fo -fum -s fee -foo
Namespace(a=[['bar', 'fi', '-fo', '-fum', '-s', 'fee', '-foo']], s=None)
Note that even though -s is a recognized argument, argparse.REMAINDER adds it to the list of args found by -a since it is after -a on the command line
Related
The title may be confusing but I can't think of a better explanation. Basically, I have a program which operates on some input file with a bunch of optional arguments. The input file is compulsory for my program. So I wrote a parser like this:
test.py
from argparse import ArgumentParser
parser = ArgumentParser(prog="prog")
parser.add_argument("file", help="input file")
parser.add_argument("--debug", action="store_true", help="enable debug messages")
parser.parse_args()
I'm not showing all the options for simplicity. Now, I want to add a feature to this program which don't require input file and runs on its own. Let's call this option as a. Option a takes unknown number of arguments and --debug or other options are not valid for option a. So the following would be valid runs for my program:
$ python3 test.py input.txt
$ python3 test.py input.txt --debug
$ python3 test.py -a 1 2 3
And these would be invalid:
$ python3 test.py input.txt -a 1 2 3
$ python3 test.py -a 1 2 3 --debug
When I add this line to my code
parser.add_argument("-a", nargs="+", help="help option a")
It accepts option a but it says "file is required" for understandable reasons. So how should I modify my code to achieve my goal?
P.S: I also tried using subparser but I couldn't manage to make it work as I want probably because my lack of knowledge.
How can I have argparse only parse commands that come after a positional argument?
Aka if I have the command pythonfile.py -d dir -e test pos_cmd_1 -d
How can I have it so that the first -d is parsed by argparse, and anything after the positional command is parsed by that command itself (read pos_cmd_1 -d as a single argument basically)
So that the argument list would be
pythonfile.py
-d dir
-e test
pos_cmd_1 -d -s -etc
So anything before the positional command would be optional. And anything after the positional command would be part of the positional command itself.
Edit: When trying to run the command with double dashes, it tells me that the arguments that come after aren't recognized.
pythonfile.py -d testdir -e test -- command -d -s
It says -d -s are unrecognized arguments instead of bundling them with the command.
you can achieve this just by slightly changing your command line to
pythonfile.py -d dir -e test -- pos_cmd_1 -d
by adding --, you tell argparse to stop looking for options. So all remaining arguments are set in the positional argument list instead.
An alternative is quoting the rest of arguments:
pythonfile.py -d dir -e test "pos_cmd_1 -d"
and (because it creates just one positional argument) use argument parser again on the splitted string (not ideal if you want to pass quoted strings in those args)
The advantages of those approaches is that they're natively supported by argparse, getopt and also that is a standard mechanism that won't surprise the users of your command.
if you want to stick to your approach, maybe you could pre-process argument list to insert the double dash by detecting 2 non-option arguments in a row:
args = "-d dir -e test pos_cmd_1 -d".split()
oldarg=""
for i,a in enumerate(args):
if oldarg and oldarg[0]!='-' and a[0]!='-':
args.insert(i,'--')
break
oldarg = a
args is now: ['-d', 'dir', '-e', 'test', '--', 'pos_cmd_1', '-d']
With the simple parser:
In [2]: p = argparse.ArgumentParser()
In [3]: p.add_argument('-d');
In [4]: p.add_argument('-e');
In [5]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E]
ipython3: error: argument -d: expected one argument
It tries to parse the last '-d' and hits an error. parse_known_args doesn't help.
With strings other than '-d' and '-e' parse_known_args works:
In [7]: p.parse_known_args('-d dir -e test pos_cmd_1 -s'.split())
Out[7]: (Namespace(d='dir', e='test'), ['pos_cmd_1', '-s'])
A positional with a REMAINDER nargs appears to work:
In [8]: a1 = p.add_argument('rest', nargs='...') # argparse.REMAINDER
In [9]: p.parse_args('-d dir -e test pos_cmd_1 -s'.split())
Out[9]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-s'])
In [10]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
Out[10]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-d'])
REMAINDER is supposed to work much like the '--', capturing input for use by another parser or command.
It can have problems if it's expected to catch the whole commandline, as in:
In [12]: p.parse_args('-s pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E] ...
ipython3: error: unrecognized arguments: -s
https://docs.python.org/3/library/argparse.html#nargs
$ script.py status
$ script.py terminate
$ script.py tail /tmp/some.log
As you can see the script can perform 3 tasks. The last task requires an additional argument (path of the file).
I want to use add_argument only once like below.
parser.add_argument("command")
And then check what command was requested by user and create conditionals based upon the same. If the command is tail I need to access the next argument (file path)
You might have to create a sub-parser for each command. This way it is extendable if those other commands also need arguments at some point. Something like this:
import argparse
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='global optional argument')
subparsers = parser.add_subparsers(dest="command", help='sub-command help')
# create the parser for the "status" command
parser_status = subparsers.add_parser('status', help='status help')
# create the parser for the "tail" command
parser_tail = subparsers.add_parser('tail', help='tail help')
parser_tail.add_argument('path', help='path to log')
print parser.parse_args()
The dest keyword of the add_subparsers ensures that you can still get the command name afterwards, as explained here and in the documentation.
Example usage:
$ python script.py status
Namespace(command='status', foo=False)
$ python script.py tail /tmp/some.log
Namespace(command='tail', foo=False, path='/tmp/some.log')
Note that any global argument needs to come before the command:
$ python script.py tail /tmp/some.log --foo
usage: PROG [-h] [--foo] {status,tail} ...
PROG: error: unrecognized arguments: --foo
$ python script.py --foo tail /tmp/some.log
Namespace(command='tail', foo=True, path='/tmp/some.log')
I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?
With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.
The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
I would like to make the parser like cmd [-a xxx -b xxx] -c xxx -d xxx
When -a is used, I want -b to be used too. likewise, if -b is used, -a must be used too. It's ok both -a and -b are not used.
How do I do that? I have tried custom actions, but it does not go well.
A better design would be to have a single option that takes two arguments:
parser.add_argument('-a', nargs=2)
Then you either specify the option with 2 arguments, or you don't specify it at all.
$ script -a 1 2
or
$ script
A custom action (or postprocessing) can split the tuple args.a into two separate values args.a and args.b.
Argparse doesn't natively support this type of use.
The most effective thing to do is check and see if those types of conditions are met after parsing:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parser.add_argument('-c')
args = parser.parse_args()
required_together = ('b','c')
# args.b will be None if b is not provided
if not all([getattr(args,x) for x in required_together]):
raise RuntimeError("Cannot supply -c without -b")