I am trying to build a command line parser that will be able to share between arguments the values passed in order to avoid having to type them multiple times. Said otherwise, I would like the namespaces of both argument to be identical:
import argparse
class PrintAction(argparse.Action):
def __init__(self, option_strings, dest, **kwargs):
super(PrintAction, self).__init__(option_strings, dest, **kwargs)
def __call__(self, parser, namespace, values, option_string=None):
for val in values:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action=PrintAction)
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
I would then like a solution that stores in both cases ['a', 'b', 'c'] in bar but also that in the case that --foo is provided, then a, b and c would be printed.
For now, what I get is foo prints only a and bar stores only b and c in case 1 and the correct result in case 2.
You need to make --foo a boolean flag. Now it's a string parameter, because you did not state otherwise. Set action to store_true for the boolean flag effect.
The final solution would look like:
def print_args(args):
if args.foo:
for val in args.bar:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action='store_true')
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
Then calling print_args(args) in the first case will print a, b and c and in the second case, it won't.
You can't (readily) trick the argparse into reusing argv strings. The parser allocates values to the Actions.
The default nargs is None, which means, use the the next string as an argument.
parser.add_argument('--foo')
would set foo='a', and bar=['b','c'].
In your Action, values will be ['a'], which you print. In optparse each option gets the remaining argv list, which it can consume as it wants. In argparse it only gets the values that its nargs demands.
You could specify in the __init__ that the nargs=0, and then print from sys.argv. Eqivalently, as #9000 suggests, make it a store_true and print after parsing. Look at the code for the store_true Action class.
Another option is to give both foo and bar a *, and have foo both print and save to the bar dest. Then foo would consume all following strings. But, if bar doesn't have anything to save, it might write [] to the namespace.
In any case, the best you can do is fake the repeated use.
Another idea is to use 2 different parsers with parse_known_args. Parsers don't mess with the sys.argv, so it can read and parsed multiple times.
Related
I want to input args that not configured in argparse:
parser = argparse.ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help', dest="character", required=False)
subparsers.required = False
base_subparser = argparse.ArgumentParser(add_help=False)
# define common shared arguments
base_subparser.add_argument('--disable', choices=['false', 'true'])
base_subparser.add_argument('--foo', choices=['false', 'true'])
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help', parents=[base_subparser])
parser_a.add_argument('--bar', choices='ABC', help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help', parents=[base_subparser])
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
argcomplete.autocomplete(parser)
args = parser.parse_known_args()
print(args)
I use the parse_known_args() which use a list to store the args not configured in argparse. However, when I use ./prog.py key = val, it shows argument character: invalid choice: 'key=val' (choose from 'a', 'b'). So I have to choose 'a' or 'b', how can I input the args not configured in argparse without choose one of the subparsers.
The error you see is the same as produced by a '?' positional with choices:
In [25]: import argparse
In [26]: parser = argparse.ArgumentParser()
In [27]: parser.add_argument('foo', nargs='?', choices=['a','b'])
'foo' is optional:
In [28]: parser.parse_known_args([])
Out[28]: (Namespace(foo=None), [])
In [29]: parser.parse_known_args(['a'])
Out[29]: (Namespace(foo='a'), [])
but any string is parsed as a possible 'foo' value:
In [30]: parser.parse_known_args(['c'])
usage: ipykernel_launcher.py [-h] [{a,b}]
ipykernel_launcher.py: error: argument foo: invalid choice: 'c' (choose from 'a', 'b')
providing a proper choice first, allows it to treat 'c' as an extra:
In [31]: parser.parse_known_args(['a','c'])
Out[31]: (Namespace(foo='a'), ['c'])
Or if the string looks like a optional's flag:
In [32]: parser.parse_known_args(['-c'])
Out[32]: (Namespace(foo=None), ['-c'])
Another possibility is to go ahead and name a subparser, possibly a dummy one, and provide the extra. The subparser will be the one that actually puts that string in the 'unknowns' category.
In [40]: parser = argparse.ArgumentParser()
In [41]: subp = parser.add_subparsers(dest='cmd')
In [44]: p1 = subp.add_parser('a')
In [45]: parser.parse_known_args(['a','c'])
Out[45]: (Namespace(cmd='a'), ['c'])
In [46]: parser.parse_known_args([]) # not-required is the default
Out[46]: (Namespace(cmd=None), [])
Keep in mind that the main parser does not "know" anything about subparsers, except is a positional. It's doing its normal allocating strings to actions. But once it calls a subparser, that parser has full control over the parsing. Once it's done it passes the namespace back to the main, but the main doesn't do any more parsing - it just wraps things up and exits (with results or error).
Since subp is a positional with a special Action subclass, _SubParsersAction, I was thinking it might be possible to create a flagged argument with that class
parser.add_argument('--foo', action=argparse._SubParsersAction)
but there's more going on in add_subparsers, so it isn't a trivial addition. This is a purely speculative idea.
Firstly, I apologize if my title is misleading/ unclear as I am really not sure what is the best way to put it.
I have 3 optional arguments, which uses action='store_true'. Let's keep the argument flags to -va, -vb, -vc
var_list = ['a', 'b', 'c']
if args.va:
run_this_func(var_list[0])
if args.vb:
run_this_func(var_list[1])
if args.vc:
run_this_func(var_list[2])
if not args.higha and not args.highb and not args.highmem:
for var in var_list:
run_this_func(var)
if args.va and args.vb:
run_this_func(var_list[:-1])
if args.vb and args.vc:
run_this_func(var_list[1:])
if args.vc and args.va:
run_this_func(var_list[0], var_list[2])
How can I code in more efficient way? The above method that I had utilized while it may work, seems more like a roundabout way to get things going...
Initially I am thinking of using tuple such that it will be something such as input = (args.va, args.vb, args.vc) so that it may return me eg. (True, False, False)... Not sure if that is ideal though.
Any advice?
I think you can use tuple too.
var_list = ['a', 'b', 'c']
input = (args.va, args.vb, args.vc)
vars = [item for index, item in enumerate(var_list) if args_tuple[index]]
run_this_func(*vars)
I think this is a case where argparse.add_argument's nargs and choices keywords can be used to get the list that you want (some subset of [a,b,c]) without having to filter based on multiple arguments.
I would recommend doing something like this:
parser = argparse.ArgumentParser()
parser.add_argument('-v', choices=['a','b','c'], nargs='+')
args = parser.parse_args()
This says: create an optional argument -v that can take one or multiple values but each of those values must be one of ['a','b','c']
Then you can pass it arguments from the command line like this (for example):
$ python my_file.py -v a c
which means that args will look like: Namespace(v=['a', 'c'])
and args.v looks like ['a', 'c'] which is the list you were looking for (and without any filtering!)
You can then call your function as:
run_this_func(args.v)
As shown in the following code, I want to have an optional positional argument files, I want to specify a default value for it, when paths are passed in, use specified path.
But because --bar can have multiple arguments, the path passed in didn't go into args.files, how do I fix that? Thanks!
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar', nargs='*')
parser.add_argument('files', nargs='?')
cmd = '--foo a --bar b c d '
print parser.parse_args(cmd.split())
# Namespace(bar=['b', 'c', 'd'], files=None, foo='a')
cmd = '--foo a --bar b c d /path/to/file1'
print parser.parse_args(cmd.split())
# Namespace(bar=['b', 'c', 'd', '/path/to/file1'], files=None, foo='a')
Your argument spec is inherently ambiguous (since --bar can take infinite arguments, there is no good way to tell when it ends, particularly since files is optional), so it requires user disambiguation. Specifically, argparse can be told "this is the end of the switches section, all subsequent argument are positional" by putting -- before the positional only section. If you do:
cmd = '--foo a --bar b c d -- /path/to/file1'
print parser.parse_args(cmd.split())
You should get:
Namespace(bar=['b', 'c', 'd'], files='/path/to/file1', foo='a')
(Tested on Py3, but should apply to Py2 as well)
Alternatively, the user can pass the positional argument anywhere it's unambiguous by avoiding putting positional arguments after --bar e.g.:
cmd = '/path/to/file1 --foo a --bar b c d'
or
cmd = '--foo a /path/to/file1 --bar b c d'
Lastly, you could avoid using nargs='*' for switches, given the ambiguity it introduces. Instead, define --bar to be accepted multiple times with a single value per switch, accumulating all uses to a list:
parser.add_argument('--bar', action='append')
then you pass --bar multiple times to supply multiple values one at a time, instead of passing it once with many values:
cmd = '--foo a --bar b --bar c --bar d /path/to/file1'
I'm trying to write a script that accepts multiple input sources and does something to each one. Something like this
./my_script.py \
-i input1_url input1_name input1_other_var \
-i input2_url input2_name input2_other_var \
-i input3_url input3_name
# notice inputX_other_var is optional
But I can't quite figure out how to do this using argparse. It seems that it's set up so that each option flag can only be used once. I know how to associate multiple arguments with a single option (nargs='*' or nargs='+'), but that still won't let me use the -i flag multiple times. How do I go about accomplishing this?
Just to be clear, what I would like in the end is a list of lists of strings. So
[["input1_url", "input1_name", "input1_other"],
["input2_url", "input2_name", "input2_other"],
["input3_url", "input3_name"]]
Here's a parser that handles a repeated 2 argument optional - with names defined in the metavar:
parser=argparse.ArgumentParser()
parser.add_argument('-i','--input',action='append',nargs=2,
metavar=('url','name'),help='help:')
In [295]: parser.print_help()
usage: ipython2.7 [-h] [-i url name]
optional arguments:
-h, --help show this help message and exit
-i url name, --input url name
help:
In [296]: parser.parse_args('-i one two -i three four'.split())
Out[296]: Namespace(input=[['one', 'two'], ['three', 'four']])
This does not handle the 2 or 3 argument case (though I wrote a patch some time ago for a Python bug/issue that would handle such a range).
How about a separate argument definition with nargs=3 and metavar=('url','name','other')?
The tuple metavar can also be used with nargs='+' and nargs='*'; the 2 strings are used as [-u A [B ...]] or [-u [A [B ...]]].
This is simple; just add both action='append' and nargs='*' (or '+').
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-i', action='append', nargs='+')
args = parser.parse_args()
Then when you run it, you get
In [32]: run test.py -i input1_url input1_name input1_other_var -i input2_url i
...: nput2_name input2_other_var -i input3_url input3_name
In [33]: args.i
Out[33]:
[['input1_url', 'input1_name', 'input1_other_var'],
['input2_url', 'input2_name', 'input2_other_var'],
['input3_url', 'input3_name']]
-i should be configured to accept 3 arguments and to use the append action.
>>> p = argparse.ArgumentParser()
>>> p.add_argument("-i", nargs=3, action='append')
_AppendAction(...)
>>> p.parse_args("-i a b c -i d e f -i g h i".split())
Namespace(i=[['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']])
To handle an optional value, you might try using a simple custom type. In this case, the argument to -i is a single comma-delimited string, with the number of splits limited to 2. You would need to post-process the values to ensure there are at least two values specified.
>>> p.add_argument("-i", type=lambda x: x.split(",", 2), action='append')
>>> print p.parse_args("-i a,b,c -i d,e -i g,h,i,j".split())
Namespace(i=[['a', 'b', 'c'], ['d', 'e'], ['g', 'h', 'i,j']])
For more control, define a custom action. This one extends the built-in _AppendAction (used by action='append'), but just does some range checking on the number of arguments given to -i.
class TwoOrThree(argparse._AppendAction):
def __call__(self, parser, namespace, values, option_string=None):
if not (2 <= len(values) <= 3):
raise argparse.ArgumentError(self, "%s takes 2 or 3 values, %d given" % (option_string, len(values)))
super(TwoOrThree, self).__call__(parser, namespace, values, option_string)
p.add_argument("-i", nargs='+', action=TwoOrThree)
Adding with Other in this Thread.
If you use action='append' in add_argument() then you will get arguments in list(s) within a list every time you add the option.
As you liked:
[
["input1_url", "input1_name", "input1_other"],
["input2_url", "input2_name", "input2_other"],
["input3_url", "input3_name"]
]
But if anyone wants those arguments in the same list[], then use action='extend' instead of action='append' in your code. This will give you those arguments in a single list.
[
"input1_url",
"input1_name",
"input1_other",
"input2_url",
"input2_name",
"input2_other",
"input3_url",
"input3_name"
]
I have a Python script that reads a file containing a command line invocation of some other tool. I'd like to modify the options of this invocation before calling the tool. For example, I might transform:
my_util --input file1.txt --option1 red --option2 blue
...to this:
my_util --input file1_001.txt --option1 red --option3 green
(More accurately, I'd be working on the arguments as lists.)
I figured that using the argparse module would be the easiest way to do this: I could parse the args, change, add or remove the options as I need to, and then reconstruct the command line.
But how do I do the last step? Given the Namespace object returned by parse_args(), can I easily reconstruct a list of command line options, such as could be passed to subprocess.Popen()?
A Namespace object is just a simple object subclass, so you can get the values out as a dict with vars:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> args = parser.parse_args(['--foo', 'BAR'])
>>> vars(args)
{'foo': 'BAR'}
Or you can assign to a class directly and get the arguments out as class variables:
>>> class C(object):
... pass
...
>>> c = C()
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> parser.parse_args(args=['--foo', 'BAR'], namespace=c)
>>> c.foo
'BAR'
It would be fairly easy to use either of these structures to test/replace arguments and pass the results to Popen.
I know this is an old question, but I've just encountered the same problem. I realized that all I need is a way to iterate over the Action objects. Unfortunately, the internal list is not exposed by ArgParser itself. However, these objects are returned by add_argument(), so I can construct my own list. Well, putting actions.append() around each call looked like too much typing to me, so I store all options in a tuple:
def add_argument(*args, **kwargs):
return (args, kwargs)
parser = argparse.ArgumentParser()
options = (
add_argument('--verbose', action='store_true'),
add_argument('--author'),
add_argument('--subject', required=True),
add_argument('--cache', nargs='?'),
add_argument('files', nargs=''),
)
actions = []
for (args, kwargs) in options:
actions.append(parser.add_argument(*args, **kwargs))
args = parser.parse_args()
At this point, the options are parsed in args, and all argparse.Action objects are stored in the actions list. I can then iterate over this list and reconstruct the options like this:
cmdline = []
for action in actions:
value = getattr(args, action.dest)
if action.required or value != action.default:
if action.option_strings:
cmdline.append(action.option_strings[0])
if action.nargs is None:
cmdline.append(value)
elif action.nargs == '?':
if value != action.const:
cmdline.append(value)
elif action.nargs != 0:
cmdline += value
In my specific case, I also wanted to remove some options from the command line. To do that I simply added them separately with a call to parser.add_argument() and not through the options tuple.