How can I have argparse only parse commands that come after a positional argument?
Aka if I have the command pythonfile.py -d dir -e test pos_cmd_1 -d
How can I have it so that the first -d is parsed by argparse, and anything after the positional command is parsed by that command itself (read pos_cmd_1 -d as a single argument basically)
So that the argument list would be
pythonfile.py
-d dir
-e test
pos_cmd_1 -d -s -etc
So anything before the positional command would be optional. And anything after the positional command would be part of the positional command itself.
Edit: When trying to run the command with double dashes, it tells me that the arguments that come after aren't recognized.
pythonfile.py -d testdir -e test -- command -d -s
It says -d -s are unrecognized arguments instead of bundling them with the command.
you can achieve this just by slightly changing your command line to
pythonfile.py -d dir -e test -- pos_cmd_1 -d
by adding --, you tell argparse to stop looking for options. So all remaining arguments are set in the positional argument list instead.
An alternative is quoting the rest of arguments:
pythonfile.py -d dir -e test "pos_cmd_1 -d"
and (because it creates just one positional argument) use argument parser again on the splitted string (not ideal if you want to pass quoted strings in those args)
The advantages of those approaches is that they're natively supported by argparse, getopt and also that is a standard mechanism that won't surprise the users of your command.
if you want to stick to your approach, maybe you could pre-process argument list to insert the double dash by detecting 2 non-option arguments in a row:
args = "-d dir -e test pos_cmd_1 -d".split()
oldarg=""
for i,a in enumerate(args):
if oldarg and oldarg[0]!='-' and a[0]!='-':
args.insert(i,'--')
break
oldarg = a
args is now: ['-d', 'dir', '-e', 'test', '--', 'pos_cmd_1', '-d']
With the simple parser:
In [2]: p = argparse.ArgumentParser()
In [3]: p.add_argument('-d');
In [4]: p.add_argument('-e');
In [5]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E]
ipython3: error: argument -d: expected one argument
It tries to parse the last '-d' and hits an error. parse_known_args doesn't help.
With strings other than '-d' and '-e' parse_known_args works:
In [7]: p.parse_known_args('-d dir -e test pos_cmd_1 -s'.split())
Out[7]: (Namespace(d='dir', e='test'), ['pos_cmd_1', '-s'])
A positional with a REMAINDER nargs appears to work:
In [8]: a1 = p.add_argument('rest', nargs='...') # argparse.REMAINDER
In [9]: p.parse_args('-d dir -e test pos_cmd_1 -s'.split())
Out[9]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-s'])
In [10]: p.parse_args('-d dir -e test pos_cmd_1 -d'.split())
Out[10]: Namespace(d='dir', e='test', rest=['pos_cmd_1', '-d'])
REMAINDER is supposed to work much like the '--', capturing input for use by another parser or command.
It can have problems if it's expected to catch the whole commandline, as in:
In [12]: p.parse_args('-s pos_cmd_1 -d'.split())
usage: ipython3 [-h] [-d D] [-e E] ...
ipython3: error: unrecognized arguments: -s
https://docs.python.org/3/library/argparse.html#nargs
Related
I'm trying to implement a python3 CLI using the docopt package. I'm trying to get my program accept multiple positional input files and, optionally a list of output files.
A MWE for my docstring is:
__doc__ = """
Usage:
test.py [INPUT...] [-o OUTPUT...] [-t TEST]
Options:
-o OUTPUT..., --output OUTPUT... #One output file for each INPUT file [default: DEFAULT]
-t TEST, --test TEST #A test option
"""
For example a programm call as
test.py FILE_A FILE_B -o OUTFILE_A OUTFILE B -t true
Should return a dict:
{'--output': ['OUTFILE_A', 'OUTFILE_B'],
'--test': 'true',
'INPUT': ['FILE_A', 'FILE_B']}
but for some reason it is always appended to the INPUT arguments:
{'--output': ['OUTFILE_A'],
'--test': 'true',
'INPUT': ['FILE_A', 'FILE_B', 'OUTFILE_B']}
Options in Docopt unfortunately can only take one argument, so [-o OUTPUT...] will not work. The remaining elements will as you state be interpreted as additional arguments.
One way around this is to move the ellipsis outside the square brackets:
Usage:
test.py [INPUT...] [-o OUTPUT]... [-t TEST]
and use it like this:
test.py FILE_A FILE_B -o OUTFILE_A -o OUTFILE_B
It doesn't look as good, but it works.
I'm using Python's argparse module to parse command line arguments. Consider the following simplified example,
# File test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store')
parser.add_argument('-a', action='append')
args = parser.parse_args()
print(args)
which can successfully be called like
python test.py -s foo -a bar -a baz
A single argument is required after -s and after each -a, which may contain spaces if we use quotation. If however an argument starts with a dash (-) and does not contain any spaces, the code crashes:
python test.py -s -begins-with-dash -a bar -a baz
error: argument -s: expected one argument
I get that it interprets -begins-with-dash as the beginning of a new option, which is illegal as -s has not yet received its required argument. It's also pretty clear though that no option with the name -begins-with-dash has been defined, and so it should not interpret it as an option in the first place. How can I make argparse accept arguments with one or more leading dashes?
You can force argparse to interpret an argument as a value by including an equals sign:
python test.py -s=-begins-with-dash -a bar -a baz
Namespace(a=['bar', 'baz'], s='-begins-with-dash')
If you are instead trying to provide multiple values to one argument:
parser.add_argument('-a', action='append', nargs=argparse.REMAINDER)
will grab everything after -a on the command line and shove it in a.
python test.py -toe -a bar fi -fo -fum -s fee -foo
usage: test.py [-h] [-s S] [-a ...]
test.py: error: unrecognized arguments: -toe
python test.py -a bar fi -fo -fum -s fee -foo
Namespace(a=[['bar', 'fi', '-fo', '-fum', '-s', 'fee', '-foo']], s=None)
Note that even though -s is a recognized argument, argparse.REMAINDER adds it to the list of args found by -a since it is after -a on the command line
parser.add_argument('-i', required=True) # One directory path
parser.add_argument('-d', required=True) # Output database path
parser.add_argument('-t', required=True) # DDL-script path
parser.add_argument('-c -i', required=True) # Another directory path
I run .py script like this:
python.exe s.py -c -i D:\Temp\dir1 -d D:\Temp\out.db -t D:\Temp\ddl.sql -i D:\Temp\dir2
and get error:
usage: s.py [-h] -i I -d D -t T -c -i C _I
s.py: error: argument -c -i: expected one argument
How can i use it without rename argument names??
Look at the sys.argv[1:] list. I expect it will be
['-c', '-i', 'D:\Temp\dir1', '-d', 'D:\Temp\out.db', '-t', 'D:\Temp\ddl.sql', '-i', 'D:\Temp\dir2']
Note that the '-c -i' are split. Because abbreviations are allowed, -c is accepted as short for that '-c -i' flag. But that string is followed by '-i' and 'D:...'. That's 2 arguments, not just one. Hence the error.
Yes you can quote the "-c -i" so the shell doesn't split it, but even that doesn't work cleanly. I don't see any point to specifying a flag like that. It doesn't build on the previously define '-i' Action.
In [113]: parser.parse_args(['-c -i', 'D:\Temp\dir1', '-d', 'D:\Temp\out.db',
'-t', 'D:\Temp\ddl.sql', '-i', 'D:\Temp\dir2'])
Out[113]: Namespace(d='D:\\Temp\\out.db', i='D:\\Temp\\dir2',
t='D:\\Temp\\ddl.sql', **{'c _i': 'D:\\Temp\\dir1'})
In [122]: getattr(Out[113],'c _i')
Out[122]: 'D:\\Temp\\dir1'
To elaborate on user2357112's comment, this "compound name" is not a thing. Off the top of my head I can't name a single flag of a single program that does this.
Even if you manage to implement this (nothing is impossible, although argparse will be of little help), this design decision would be highly eyebrow-raising to users of your program.
I am writing a program which, among other things, allows the user to specify through an argument a module to load (and then use to perform actions). I am trying to set up a way to easily pass arguments through to this inner module, and I was attempting to use ArgParse's action='append' to have it build a list of arguments that I would then pass through.
Here is a basic layout of the arguments that I am using
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
required=True)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
However - if I then try to run this program with python my_program -M module:class -A "-f filename" (where I would like to pass through the -f filename to my module) it seems to be parsing the -f as its own argument (and I get the error my_program: error: argument -A/--module_args: expected one argument
Any ideas?
With:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-M', '--module',
help="Module to run on changed files - should be in format MODULE:CLASS\n\
Specified class must have function with the signature run(src, dest)\
and return 0 upon success",
)
parser.add_argument('-A', '--module_args',
help="Arg to be passed through to the specified module",
action='append',
default=[])
import sys
print(sys.argv)
print(parser.parse_args())
I get:
1028:~/mypy$ python stack45146728.py -M module:class -A "-f filename"
['stack45146728.py', '-M', 'module:class', '-A', '-f filename']
Namespace(module='module:class', module_args=['-f filename'])
This is using a linux shell. The quoted string remains one string, as seen in the sys.argv, and is properly interpreted as an argument to -A.
Without the quotes the -f is separate and interpreted as a flag.
1028:~/mypy$ python stack45146728.py -M module:class -A -f filename
['stack45146728.py', '-M', 'module:class', '-A', '-f', 'filename']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
Are you using windows or some other OS/shell that doesn't handle quotes the same way?
In Argparse `append` not working as expected
you asked about a slightly different command line:
1032:~/mypy$ python stack45146728.py -A "-k filepath" -A "-t"
['stack45146728.py', '-A', '-k filepath', '-A', '-t']
usage: stack45146728.py [-h] [-M MODULE] [-A MODULE_ARGS]
stack45146728.py: error: argument -A/--module_args: expected one argument
As I already noted -k filepath is passed through as one string. Because of the space, argparse does not interpret that as a flag. But it does interpret the bare '-t' as a flag.
There was a bug/issue about the possibility of interpreting undefined '-xxx' strings as arguments instead of flags. I'd have to look that up to see whether anything made it into to production.
Details of how strings are categorized as flag or argument can be found in argparse.ArgumentParser._parse_optional method. It contains a comment:
# if it contains a space, it was meant to be a positional
if ' ' in arg_string:
return None
http://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) is an old and long bug/issue on the topic.
The solution is to accept arbitrary arguments - there's an example in argparse's doc here:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
I'm trying to read number of files using argparse:
parser.add_argument(
'-f',
'--text-file',
metavar='IN FILE',
type=argparse.FileType('r'),
nargs='*')
...
...
args = parser.parse_args()
print args
when more than one files are passed as command line arguments, only last file appears into args:
python example.py -o x.xml -s sss -c ccc -t "hello world" --report_failure -f ex.1 -f ex.2
Namespace(outputfile=<open file 'x.xml', mode 'w' at 0x028AD4F0>, report_failure=True, test_case='ccc', test_suite='sss', text='hello world', text_file=[<open file 'ex.2', mode 'r' at 0x028AD5A0>])
What I did wrong and how to access all files I passed from the command line?
Note: I'm using python 2.7.6 on Windows.
The complication occurs because you're passing multiple arguments to the same parameter -f and each argument replaces the argument before it. What would work in this case is:
python example.py -o x.xml -s sss -c ccc -t "hello world"
--report_failure -f ex.1 ex.2
This will collect ex.1 and ex.2 into a list, which is what I assume you want to do.
as a reference here is the docs on nargs:
'*'. All command-line arguments present are gathered into a list. Note
that it generally doesn’t make much sense to have more than one
positional argument with nargs='', but multiple optional arguments
with nargs='' is possible.
https://docs.python.org/3/library/argparse.html#nargs