I have this code
parser = argparse.ArgumentParser()
parser.add_argument('--input')
parser.add_argument('--min')
parser.add_argument('--max')
args = parser.parse_args()
How can I know the order of arguments ? For example, if I type:
python somefile.py --min min --input file.csv
How can I know position of --min is before --input and user didn't type --max?
I think it might be tricky to obtain that information from parser itself, but it can be easily obtained from sys.argv:
def get_arg_index(args: list, name: str):
return next((i for i, v in enumerate(args) if v.startswith(name)), None)
print(get_arg_index(sys.argv, "--min"))
print(get_arg_index(sys.argv, "--max"))
By design argparse is supposed to handle the optionals in any order. They are parsed in the order that they are provided in the command line, but it does not keep any record of that order. positionals without a keyword are handled in the order that they are defined by the add_argument definitions.
All attributes are added to the args Namespace with their defaults at the start of parsing. And it does this in the add_argument order. In newer Pythons this order is preserved in the args.__dict__ (which vars(args) also shows).
But in print(args), the attributes are displayed in sorted order. usage also preserves the definition order for optionals.
So for a sample parser:
In [11]: p.print_usage()
usage: ipython3 [-h] [--foo FOO] [--test TEST] [--aaa AAA] bar
In [12]: args=p.parse_args(['xxx'])
In [13]: args
Out[13]: Namespace(aaa=None, bar='xxx', foo=None, test=None) # sorted
In [14]: vars(args)
Out[14]: {'foo': None, 'bar': 'xxx', 'test': None, 'aaa': None} # definition
In [15]: [a.dest for a in p._actions]
Out[15]: ['help', 'foo', 'bar', 'test', 'aaa'] # definition order
So recording the order of occurrence of the arguments in the command line is, with just argparse, awkward. In a sense it goes against the spirit of optionals.
Alternatives:
work directly with the sys.argv
customize the action class
customize the Namespace class
One other trick comes to mind:
If the default is argparse.SUPPRESS, the default is not added to args. Then I think the var(args) order will be the order in which values are parsed and added.
In [16]: for a in p._actions: a.default=argparse.SUPPRESS
In [24]: args=p.parse_args(['--test', 'ttt','xxx','--foo=3'])
In [25]: args
Out[25]: Namespace(bar='xxx', foo='3', test='ttt')
In [26]: vars(args)
Out[26]: {'test': 'ttt', 'bar': 'xxx', 'foo': '3'}
Users may repeat optionals, with repeats over writing previous values.
Related
I would like to be able to support positional command line arguments that go into different lists based on prior predicates.
For example, a command like:
mycommand one two three
would yield args like:
main_dest = ['one','two','three']
other_dest = []
but a command like:
mycommand one --other two three --main four five
would yield args like:
main_dest = ['one','four','five']
other_dest = ['two','three']
Conceptually what I'd like is an action that modifies the dest of the positional argument reader.
As a first try this set of Actions seems to do the trick:
In [73]: parser = argparse.ArgumentParser()
In [74]: parser.add_argument('main', nargs='*');
In [75]: parser.add_argument('other', nargs='*');
In [76]: parser.add_argument('--main', action='append');
In [77]: parser.add_argument('--other', action='append');
In [78]: parser.print_usage()
usage: ipython3 [-h] [--main MAIN] [--other OTHER]
[main [main ...]] [other [other ...]]
In [79]: parser.parse_args('one two three'.split())
Out[79]: Namespace(main=['one', 'two', 'three'], other=[])
In [80]: parser.parse_args('one --other two --main three'.split())
Out[80]: Namespace(main=['one', 'three'], other=['two'])
74 and 76 both have main as their dest. I use append for the flagged ones so they don't overwrite positional values. But despite what the usage shows, positionals will only work at the start. If placed a the end they'll overwrite flagged values. And the 'other' positional will never get values - so I should have omitted it.
So it is possible to play games like this, but I'm not sure it's robust, or any easier for your users.
argparse: flatten the result of action='append'
I am trying to build a command line parser that will be able to share between arguments the values passed in order to avoid having to type them multiple times. Said otherwise, I would like the namespaces of both argument to be identical:
import argparse
class PrintAction(argparse.Action):
def __init__(self, option_strings, dest, **kwargs):
super(PrintAction, self).__init__(option_strings, dest, **kwargs)
def __call__(self, parser, namespace, values, option_string=None):
for val in values:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action=PrintAction)
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
I would then like a solution that stores in both cases ['a', 'b', 'c'] in bar but also that in the case that --foo is provided, then a, b and c would be printed.
For now, what I get is foo prints only a and bar stores only b and c in case 1 and the correct result in case 2.
You need to make --foo a boolean flag. Now it's a string parameter, because you did not state otherwise. Set action to store_true for the boolean flag effect.
The final solution would look like:
def print_args(args):
if args.foo:
for val in args.bar:
print(val)
parser = argparse.ArgumentParser(description='A foo that foos and a bar that bars')
parser.add_argument('--foo', action='store_true')
parser.add_argument('bar', nargs='+')
args = parser.parse_args(['--foo', 'a', 'b', 'c']) # Case 1
args = parser.parse_args(['a', 'b', 'c']) # Case 2
Then calling print_args(args) in the first case will print a, b and c and in the second case, it won't.
You can't (readily) trick the argparse into reusing argv strings. The parser allocates values to the Actions.
The default nargs is None, which means, use the the next string as an argument.
parser.add_argument('--foo')
would set foo='a', and bar=['b','c'].
In your Action, values will be ['a'], which you print. In optparse each option gets the remaining argv list, which it can consume as it wants. In argparse it only gets the values that its nargs demands.
You could specify in the __init__ that the nargs=0, and then print from sys.argv. Eqivalently, as #9000 suggests, make it a store_true and print after parsing. Look at the code for the store_true Action class.
Another option is to give both foo and bar a *, and have foo both print and save to the bar dest. Then foo would consume all following strings. But, if bar doesn't have anything to save, it might write [] to the namespace.
In any case, the best you can do is fake the repeated use.
Another idea is to use 2 different parsers with parse_known_args. Parsers don't mess with the sys.argv, so it can read and parsed multiple times.
I have an argparser, with two subsets of arguments, inputs and parameters. My command looks something like
program -input1 <input1> -input2 <input2> -p1 <param1> -p2 <param2>
I'd love to
args = parser.parse_args()
params = vars(args.params)
instead of
params = {"p1": args.p1, "p2": args.p2, etc...}
Is there an elegant way to do something like:
parser.add_argument("-p1", dest='p1', part_of=params)
Subparsers don't seem like they're made for this. Or are they?
Argparse has argument groups, but they seem like they're just for the help text.
Thanks!
This sounds like a variation on the - how do I accept arbitrary 'key=value' pairs? It's been asked in various ways over the years, with various answers.
In a recent one:
Parsing "python foo.py -DVAR1=9 -DVAR2=Off" with argparse
my solution was to split -DVAR1=9 into ('VAR1',9) and append that to the D attribute. That uses a custom type.
Using variable arg names with argparse - this processes the sys.argv before passing it to the parser.
python argparse store --foo=bar as args.key='foo', args.value='bar'
suggests a custom Action class.
I think we've also suggested a custom Namespace class.
The builtin mechanisms for grouping values are nargs and append action. Together you can get attributes which are lists of lists.
JSON strings can also be used to input complex data structures.
class MyAction(argparse._StoreAction):
def __call__(self, parser, namespace, values, option_string=None):
print('storing', option_string)
arg = getattr(namespace, self.dest)
if arg is None:
arg = {}
arg[option_string] = values
setattr(namespace, self.dest, arg)
In [135]: p=argparse.ArgumentParser()
In [136]: p.add_argument('--p1',action=MyAction,dest='p');
In [137]: p.add_argument('--p2',action=MyAction,dest='p');
In [138]: p.parse_args('--p1 one --p2 two'.split())
storing --p1
storing --p2
Out[138]: Namespace(p={'--p2': 'two', '--p1': 'one'})
In [139]: _.p
Out[139]: {'--p1': 'one', '--p2': 'two'}
Obviously this could be refined in various ways - trimming the keys to 'p1' or even '1', saving to a list, or nested Namespace or other custom structure, etc.
This approach still requires that you define an add_argument for each '--pn` variation.
An alternative is keep argparse simple, producing a namespace like:
In [141]: argparse.Namespace(p1='one', p2='two', input1=1, input2=3)
Out[141]: Namespace(input1=1, input2=3, p1='one', p2='two')
and then do your own grouping afterwards.
In [142]: args=argparse.Namespace(p1='one', p2='two', input1=1, input2=3)
In [143]: {key:getattr(args,key) for key in ['p1','p2']}
Out[143]: {'p1': 'one', 'p2': 'two'}
I have a Python script that reads a file containing a command line invocation of some other tool. I'd like to modify the options of this invocation before calling the tool. For example, I might transform:
my_util --input file1.txt --option1 red --option2 blue
...to this:
my_util --input file1_001.txt --option1 red --option3 green
(More accurately, I'd be working on the arguments as lists.)
I figured that using the argparse module would be the easiest way to do this: I could parse the args, change, add or remove the options as I need to, and then reconstruct the command line.
But how do I do the last step? Given the Namespace object returned by parse_args(), can I easily reconstruct a list of command line options, such as could be passed to subprocess.Popen()?
A Namespace object is just a simple object subclass, so you can get the values out as a dict with vars:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> args = parser.parse_args(['--foo', 'BAR'])
>>> vars(args)
{'foo': 'BAR'}
Or you can assign to a class directly and get the arguments out as class variables:
>>> class C(object):
... pass
...
>>> c = C()
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> parser.parse_args(args=['--foo', 'BAR'], namespace=c)
>>> c.foo
'BAR'
It would be fairly easy to use either of these structures to test/replace arguments and pass the results to Popen.
I know this is an old question, but I've just encountered the same problem. I realized that all I need is a way to iterate over the Action objects. Unfortunately, the internal list is not exposed by ArgParser itself. However, these objects are returned by add_argument(), so I can construct my own list. Well, putting actions.append() around each call looked like too much typing to me, so I store all options in a tuple:
def add_argument(*args, **kwargs):
return (args, kwargs)
parser = argparse.ArgumentParser()
options = (
add_argument('--verbose', action='store_true'),
add_argument('--author'),
add_argument('--subject', required=True),
add_argument('--cache', nargs='?'),
add_argument('files', nargs=''),
)
actions = []
for (args, kwargs) in options:
actions.append(parser.add_argument(*args, **kwargs))
args = parser.parse_args()
At this point, the options are parsed in args, and all argparse.Action objects are stored in the actions list. I can then iterate over this list and reconstruct the options like this:
cmdline = []
for action in actions:
value = getattr(args, action.dest)
if action.required or value != action.default:
if action.option_strings:
cmdline.append(action.option_strings[0])
if action.nargs is None:
cmdline.append(value)
elif action.nargs == '?':
if value != action.const:
cmdline.append(value)
elif action.nargs != 0:
cmdline += value
In my specific case, I also wanted to remove some options from the command line. To do that I simply added them separately with a call to parser.add_argument() and not through the options tuple.
I'm looking for a nicer way to assign a set with the conent of a list if such list is not empty, otherwise another list should be used.
If it is possible I'd like a nicer way to write this (or an argument to why this is the nicest way):
if args.onlyTheseServers:
only = set(args.onlyTheseServers)
else:
only = set(availableServers)
only = set(args.onlyTheseServers or availableServers)
Looking at your previous question, I'd say that what you're really looking for is a way to assign a default value to a missing parameter using argparse. In that case you should just use default as follows:
parser.add_argument('-o', '--only', default=default_servers, ...)
This way, when the -o/--only option isn't passed, the namespace will have the default value correctly set.
args.onlyTheseServers seems a variable coming from argparse.
If that's your case you should check the default argument and the set_default() method.
Here's an example:
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', nargs='*', default=['1', '2', '3'])
>>> args = parser.parse_args()
>>> args.foo
['1', '2', '3']
>>> args = parser.parse_args(['--foo', 'a', 'b'])
>>> args.foo
['a', 'b']
Not really much better, but at least a bit shorter:
only = set(availableServers if args.onlyTheseServers is None
else args.onlyTheseServers)
You can also do
only = set(args.onlyTheseServers or availableServers)
It works slightly different as it does not test for None, but only if the argument is true-is - in this case it should work though.
Call me crazy buy I like this better
only = set(args.onlyTheseServers) if args.onlyTheseServers is not None else set(availableServers)