Python: Parse multiple datatypes using argparse - python

I tried using argparse to learn how it works to parse a given list:
parser = argparse.ArgumentParser()
parser.add_argument('--ls', nargs='*', type=str, default = [])
Out[92]: _StoreAction(option_strings=['--ls'], dest='ls', nargs='*', const=None, default=[], type=<type 'str'>, choices=None, help=None, metavar=None)
args = parser.parse_args("--ls 'tomato' 'jug' 'andes'".split())
args
Out[94]: Namespace(ls=["'tomato'", "'jug'", "'andes'"])
args.ls
Out[96]: ["'tomato'", "'jug'", "'ande'"]
args.ls[0]
Out[97]: "'tomato'"
eval(args.ls[0])
Out[98]: 'tomato'
Q1: The above works but Is there a better way to access values in the list?
Then I tried it with dictionary to parse a dictionary given:
dict_parser = argparse.ArgumentParser()
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
Out[104]: _StoreAction(option_strings=['--dict'], dest='dict', nargs='*', const=None, default={}, type=<type 'dict'>, choices=None, help=None, metavar=None)
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
usage: -c [-h] [--dict [DICT [DICT ...]]]
-c: error: unrecognized arguments: - - d i c t { ' n a m e ' : ' m a n' , ' a d d r e s s ' : ' k a c' , ' t a g s ' : ' d o n ' }
To exit: use 'exit', 'quit', or Ctrl-D.
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
And that doesn't work.
Q2: How does the above work for dictionary?
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
How do I do that?
I referred to http://parezcoydigo.wordpress.com/2012/08/04/from-argparse-to-dictionary-in-python-2-7/
...and found assigning everything under a dictionary is pretty useful. Could somebody simplify this task so as to parse multiple datatypes in the arguments?

parser.add_argument('--ls', nargs='*', type=str, default = [])
Q1: The above works but Is there a better way to access values in the list?
As I often consider "simpler is better", and this is a really simple way to do things, I'd say there is no better way. But still, there are other ways.
dict_parser.add_argument('--dict', nargs='*',type=dict,default={})
arg2 = dict_parser.parse_args("--dict {'name':'man', 'address': 'kac', 'tags':'don'}")
Q2: How does the above work for dictionary?
I'd advice you to parse your string using json:
>>> class FromJSON():
... def __init__(self, string):
... self.string = string
... def decode(self):
... return json.loads(self.string)
...
>>> dict_parser.add_argument('--dict',type=FromJSON)
>>> arg2 = dict_parser.parse_args(['--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
though JSON is a lot like python, it is not python. It is picky about quoting (values are surrounded by double quotes only) and about (having no) trailing commas. But at least it is safe against code injection! And of course you shall surround your parameter with quotes
You may have anoter solution to give a dict as parameters, it would be to get rid of the brackets:
>>> parser.add_argument('--dict', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--dict', 'name:man', 'address:kac', 'tags:don'])
>>> args.dict = dict([arg.split(':') for arg in args.dict])
>>> print args.dict
{'tags': 'don', 'name': 'man', 'address': 'kac'}
Q3: Now I want
python my.py --ls tomato jug andes --dict {'name':'man', 'address': 'kac', 'tags':'don'}
to be parsed
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--dict', type=FromJSON, default=FromJSON("{}"))
>>> parser.add_argument('--ls', nargs='*', type=str, default = [])
>>> args = parser.parse_args(['--ls', 'tomato', 'jug', 'andes', '--dict', '{"name":"man", "address": "kac", "tags":"don"}'])
>>> args.ls
['tomato', 'jug', 'andes']
>>> args.dict
<__main__.FromJSON instance at 0x7f932dd20c20>
>>> args.dict.decode()
{u'tags': u'don', u'name': u'man', u'address': u'kac'}
About FromJSON(), it can be improved so it is something like:
class JsonToDict():
def __call__(self, string):
return json.loads(string)
that you could use as follows:
dict_parser.add_argument('--dict',type=JsonToDict())
HTH

import ast
dict_parser.add_argument('--dict', nargs='*',type=ast.literal_eval,default={})
args = dict_parser.parse_args(["--dict", "{'name':'man', 'address': 'kac', 'tags':'don'}"])

Related

Pass keyword arguments to argparse preserving defaults

Let's say you have an argument parser like below, with a usual workflow that parses CLI args via parser.parse_args().
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
However, I also may want to bypass argument parsing and supply args directly via a dictonary, which may not contain optional arguments. All missing arguments should be supplied by the parser defaults.
I.e., desirable scenario:
mydict={"arg1": "Supplied value"} # only supplied non-optional args
args = somehow_resolve_this(parser, mydict)
# this should now work
args.arg1
# Supplied value
print(args.arg2)
# Some default value
An equivalent question would be: How can I obtain all optional argument names and default values from the parser?
You might convert your dict into argparse.Namespace and then feed it into .parse_args as follows
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-arg1")
parser.add_argument("-arg2", default="some default value")
namespace = argparse.Namespace(**{"arg1":"value"})
parsed = parser.parse_args(namespace=namespace)
print(parsed.arg1) # value
print(parsed.arg2) # some default value
Explanation: convert dict into kwargs for argparse.Namespace using unpacking ** then feed it into parser.parse_args
As long as none of the arguments is required (positional or flagged), we can get the default values with:
In [3]: args = parser.parse_args([])
In [4]: args
Out[4]: Namespace(arg1=None, arg2='some default value')
In [5]: vars(args)
Out[5]: {'arg1': None, 'arg2': 'some default value'}
And we can 'update' that with your dict:
In [6]: mydict = {"arg1": "Supplied value"}
In [7]: vars(args).update(mydict)
In [8]: args
Out[8]: Namespace(arg1='Supplied value', arg2='some default value')
Using the idea of creating a namespace and passing that to the parser:
In [17]: ns = argparse.Namespace(**mydict)
In [18]: ns
Out[18]: Namespace(arg1='Supplied value')
In [19]: parser.parse_args([], namespace=ns)
Out[19]: Namespace(arg1='Supplied value', arg2='some default value')
Here I supplied the [] argv. That could be omitted if you still want to read the users input. This use of a namespace parameter in effect sets/replaces all the defaults. (this could fail, though, if you are using subparsers).
What if there are required arguments? Defaults don't matter with required arguments.
Another way to get the defaults, is to extract them from the actions list:
In [20]: parser._actions
Out[20]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['-arg1'], dest='arg1', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None),
_StoreAction(option_strings=['-arg2'], dest='arg2', nargs=None, const=None, default='some default value', type=None, choices=None, help=None, metavar=None)]
In [21]: {a.dest: a.default for a in parser._actions}
Out[21]: {'help': '==SUPPRESS==', 'arg1': None, 'arg2': 'some default value'}

how to get argparser arguments name?

I am trying to parse arguments passed from command line.I am passing 15 arguments at all. I am trying to group those by giving them same destination () I need to group those. Now when I print input i get lists f.e [mylogo.png, otherlogo.png] and so on. How I could get a result similar to {destination:'value1','value2'} . I know I could do it manually but It's not a solution in my case..
parser = argparse.ArgumentParser(prog='Moodle automation', add_help= False,
description=description(), usage='nana nanan nana')
parser.add_argument('-logo', '--set_logo',
help='',
dest='branding',
type=str,
action='append')
parser.add_argument('-c_logo', '--set_compact_logo',
help='',
dest='branding',
type=str,
action='append'
)
web_status.add_argument('-wn', '--web_new',
help=" ",
dest='web_state',
action="append")
web_status.add_argument('-wo', '--web_old',
help="",
dest="web_state",
action="append")
args = parser.parse_args()
branding_details = args.branding
print(branding_details)
in case input:
program.py -logo mylogo.png -c_logo custom_logo.png
I get output ['mylogo.png', 'custom_logo.png']
Here is a complete minimal example, where we can give several logo and compact_logo with nargs='*'. The result contains lists of arguments.
cli represents an example string you would to pass to the program.
import argparse
parser = argparse.ArgumentParser(prog='Moodle automation', add_help= False,
description='', usage='nana nanan nana')
parser.add_argument('-l', '--set_logo',
help='',
dest='logo',
nargs='*',
type=str)
parser.add_argument('-c', '--set_compact_logo',
help='',
dest='compact_logo',
nargs='*',
type=str)
cli = '-l mylogo.png -c custom_logo_1.png custom_logo_2.png'
args = parser.parse_args(cli.split())
print("List of arguments")
print(args.logo)
print(args.compact_logo)
print("Create a dict of key values for arguments")
dict_key_args = {key: value for key, value in args._get_kwargs()}
# Create the following data structure
# {'compact_logo': ['custom_logo_1.png', 'custom_logo_2.png'],
# 'logo': ['mylogo.png']}
If you just print(vars(args)), it will give output like this. vars() is always a handy function if you are dealing with object names.
{'branding': ['mylogo.png', 'custom_logo.png']}
It is just matter of removing those square brackets [] from output and if you are using multiple destinations, you can iterate over the args dictionary to get desired output.
args_dict = vars(args)
for k, v in args_dict.items():
print("{", k, ":", str(v).strip('[]'), "}")
output:
{ branding : 'mylogo.png', 'custom_logo.png' }
OR even better formatting with
args_dict = vars(args)
for k, v in args_dict.items():
print('{{{0}: {1}}}'.format(k, str(v).strip('[]')))
output:
{branding: 'mylogo.png', 'custom_logo.png'}

argparse update choices of an argument

Using argparse, is there any way to update the "choices" option of an argument after it was added to the parser ? Argparse documentation doesn't yield much about updating the choices
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
parser.add_argument('arg1', choices=choices_list)
# The list of choices now changes
choices_list = ['A', 'C', 'D']
# Some code to update 'arg1' choices option ?
parser.???
I tried using 'parser.add_argument' with the new 'choices_list', but it creates duplicate arguments.
Using Python 3.7
The argument itself has a choices attribute, but it's easiest if you save a reference to the argument instead of trying to retrieve it from the parser itself. (Otherwise, you have to scan through the private attribute parser._actions and try to identify which one you need.)
import argparse
parser = argparse.ArgumentParser()
choices_list = ['A', 'B']
arg1 = parser.add_argument('arg1', choices=choices_list)
arg1.choices = ['A', 'B', 'D']
Looking at https://github.com/python/typeshed/blob/master/stdlib/2and3/argparse.pyi
# undocumented
class _ActionsContainer:
def add_argument(self,
...
**kwargs: Any) -> Action: ...
def add_argument_group(self, *args: Any, **kwargs: Any) -> _ArgumentGroup: ...
I don't use pycharm, but I'm puzzled as to why it would complain about add_argument, but not about add_argument_group.
The argparse docs doesn't mention the returned Action object, but that's because users don't usually need to access it. But when tested interactively it's pretty obvious:
In [93]: import argparse
In [94]: parser = argparse.ArgumentParser()
In [95]: parser.add_argument('--foo', choices=['one','two'])
Out[95]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)
In [96]: _.choices
Out[96]: ['one', 'two']
Previous SO answers have pointed out that the Actions are also available in 'hidden' _actions list. But in Python, that '_' is just an informal convention; the interpreter doesn't enforce privacy.
In [98]: parser._actions
Out[98]:
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
_StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=['one', 'two'], help=None, metavar=None)]
What's displayed is the string representation of the Action object. Those are the most commonly used attributes.
The documentation for argparse is not a formal API reference; it's too incomplete for that. It's more of a advanced how-to document, more involved than a tutorial, but not as complete as a formal specification.

How to parse multiple nested sub-commands using python argparse?

I am implementing a command line program which has interface like this:
cmd [GLOBAL_OPTIONS] {command [COMMAND_OPTS]} [{command [COMMAND_OPTS]} ...]
I have gone through the argparse documentation. I can implement GLOBAL_OPTIONS as optional argument using add_argument in argparse. And the {command [COMMAND_OPTS]} using Sub-commands.
From the documentation it seems I can have only one sub-command. But as you can see I have to implement one or more sub-commands. What is the best way to parse such command line arguments useing argparse?
I came up with the same qustion, and it seems i have got a better answer.
The solution is we shall not simply nest subparser with another subparser, but we can add subparser following with a parser following another subparser.
Code tell you how:
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument('--user', '-u',
default=getpass.getuser(),
help='username')
parent_parser.add_argument('--debug', default=False, required=False,
action='store_true', dest="debug", help='debug flag')
main_parser = argparse.ArgumentParser()
service_subparsers = main_parser.add_subparsers(title="service",
dest="service_command")
service_parser = service_subparsers.add_parser("first", help="first",
parents=[parent_parser])
action_subparser = service_parser.add_subparsers(title="action",
dest="action_command")
action_parser = action_subparser.add_parser("second", help="second",
parents=[parent_parser])
args = main_parser.parse_args()
#mgilson has a nice answer to this question. But problem with splitting sys.argv myself is that i lose all the nice help message Argparse generates for the user. So i ended up doing this:
import argparse
## This function takes the 'extra' attribute from global namespace and re-parses it to create separate namespaces for all other chained commands.
def parse_extra (parser, namespace):
namespaces = []
extra = namespace.extra
while extra:
n = parser.parse_args(extra)
extra = n.extra
namespaces.append(n)
return namespaces
argparser=argparse.ArgumentParser()
subparsers = argparser.add_subparsers(help='sub-command help', dest='subparser_name')
parser_a = subparsers.add_parser('command_a', help = "command_a help")
## Setup options for parser_a
## Add nargs="*" for zero or more other commands
argparser.add_argument('extra', nargs = "*", help = 'Other commands')
## Do similar stuff for other sub-parsers
Now after first parse all chained commands are stored in extra. I reparse it while it is not empty to get all the chained commands and create separate namespaces for them. And i get nicer usage string that argparse generates.
parse_known_args returns a Namespace and a list of unknown strings. This is similar to the extra in the checked answer.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
sub = parser.add_subparsers()
for i in range(1,4):
sp = sub.add_parser('cmd%i'%i)
sp.add_argument('--foo%i'%i) # optionals have to be distinct
rest = '--foo 0 cmd2 --foo2 2 cmd3 --foo3 3 cmd1 --foo1 1'.split() # or sys.argv
args = argparse.Namespace()
while rest:
args,rest = parser.parse_known_args(rest,namespace=args)
print args, rest
produces:
Namespace(foo='0', foo2='2') ['cmd3', '--foo3', '3', 'cmd1', '--foo1', '1']
Namespace(foo='0', foo2='2', foo3='3') ['cmd1', '--foo1', '1']
Namespace(foo='0', foo1='1', foo2='2', foo3='3') []
An alternative loop would give each subparser its own namespace. This allows overlap in positionals names.
argslist = []
while rest:
args,rest = parser.parse_known_args(rest)
argslist.append(args)
The solution provide by #Vikas fails for subcommand-specific optional arguments, but the approach is valid. Here is an improved version:
import argparse
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help', dest='subparser_name')
# create the parser for the "command_a" command
parser_a = subparsers.add_parser('command_a', help='command_a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "command_b" command
parser_b = subparsers.add_parser('command_b', help='command_b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# parse some argument lists
argv = ['--foo', 'command_a', '12', 'command_b', '--baz', 'Z']
while argv:
print(argv)
options, argv = parser.parse_known_args(argv)
print(options)
if not options.subparser_name:
break
This uses parse_known_args instead of parse_args. parse_args aborts as soon as a argument unknown to the current subparser is encountered, parse_known_args returns them as a second value in the returned tuple. In this approach, the remaining arguments are fed again to the parser. So for each command, a new Namespace is created.
Note that in this basic example, all global options are added to the first options Namespace only, not to the subsequent Namespaces.
This approach works fine for most situations, but has three important limitations:
It is not possible to use the same optional argument for different subcommands, like myprog.py command_a --foo=bar command_b --foo=bar.
It is not possible to use any variable length positional arguments with subcommands (nargs='?' or nargs='+' or nargs='*').
Any known argument is parsed, without 'breaking' at the new command. E.g. in PROG --foo command_b command_a --baz Z 12 with the above code, --baz Z will be consumed by command_b, not by command_a.
These limitations are a direct limitation of argparse. Here is a simple example that shows the limitations of argparse -even when using a single subcommand-:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('spam', nargs='?')
subparsers = parser.add_subparsers(help='sub-command help', dest='subparser_name')
# create the parser for the "command_a" command
parser_a = subparsers.add_parser('command_a', help='command_a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "command_b" command
parser_b = subparsers.add_parser('command_b', help='command_b help')
options = parser.parse_args('command_a 42'.split())
print(options)
This will raise the error: argument subparser_name: invalid choice: '42' (choose from 'command_a', 'command_b').
The cause is that the internal method argparse.ArgParser._parse_known_args() it is too greedy and assumes that command_a is the value of the optional spam argument. In particular, when 'splitting' up optional and positional arguments, _parse_known_args() does not look at the names of the arugments (like command_a or command_b), but merely where they occur in the argument list. It also assumes that any subcommand will consume all remaining arguments.
This limitation of argparse also prevents a proper implementation of multi-command subparsers. This unfortunately means that a proper implementation requires a full rewrite of the argparse.ArgParser._parse_known_args() method, which is 200+ lines of code.
Given these limitation, it may be an options to simply revert to a single multiple-choice argument instead of subcommands:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--bar', type=int, help='bar help')
parser.add_argument('commands', nargs='*', metavar='COMMAND',
choices=['command_a', 'command_b'])
options = parser.parse_args('--bar 2 command_a command_b'.split())
print(options)
#options = parser.parse_args(['--help'])
It is even possible to list the different commands in the usage information, see my answer https://stackoverflow.com/a/49999185/428542
You can always split up the command-line yourself (split sys.argv on your command names), and then only pass the portion corresponding to the particular command to parse_args -- You can even use the same Namespace using the namespace keyword if you want.
Grouping the commandline is easy with itertools.groupby:
import sys
import itertools
import argparse
mycommands=['cmd1','cmd2','cmd3']
def groupargs(arg,currentarg=[None]):
if(arg in mycommands):currentarg[0]=arg
return currentarg[0]
commandlines=[list(args) for cmd,args in intertools.groupby(sys.argv,groupargs)]
#setup parser here...
parser=argparse.ArgumentParser()
#...
namespace=argparse.Namespace()
for cmdline in commandlines:
parser.parse_args(cmdline,namespace=namespace)
#Now do something with namespace...
untested
Improving on the answer by #mgilson, I wrote a small parsing method which splits argv into parts and puts values of arguments of commands into hierarchy of namespaces:
import sys
import argparse
def parse_args(parser, commands):
# Divide argv by commands
split_argv = [[]]
for c in sys.argv[1:]:
if c in commands.choices:
split_argv.append([c])
else:
split_argv[-1].append(c)
# Initialize namespace
args = argparse.Namespace()
for c in commands.choices:
setattr(args, c, None)
# Parse each command
parser.parse_args(split_argv[0], namespace=args) # Without command
for argv in split_argv[1:]: # Commands
n = argparse.Namespace()
setattr(args, argv[0], n)
parser.parse_args(argv, namespace=n)
return args
parser = argparse.ArgumentParser()
commands = parser.add_subparsers(title='sub-commands')
cmd1_parser = commands.add_parser('cmd1')
cmd1_parser.add_argument('--foo')
cmd2_parser = commands.add_parser('cmd2')
cmd2_parser.add_argument('--foo')
cmd2_parser = commands.add_parser('cmd3')
cmd2_parser.add_argument('--foo')
args = parse_args(parser, commands)
print(args)
It behaves properly, providing nice argparse help:
For ./test.py --help:
usage: test.py [-h] {cmd1,cmd2,cmd3} ...
optional arguments:
-h, --help show this help message and exit
sub-commands:
{cmd1,cmd2,cmd3}
For ./test.py cmd1 --help:
usage: test.py cmd1 [-h] [--foo FOO]
optional arguments:
-h, --help show this help message and exit
--foo FOO
And creates a hierarchy of namespaces containing the argument values:
./test.py cmd1 --foo 3 cmd3 --foo 4
Namespace(cmd1=Namespace(foo='3'), cmd2=None, cmd3=Namespace(foo='4'))
You could try arghandler. This is an extension to argparse with explicit support for subcommands.
Built a full Python 2/3 example with subparsers, parse_known_args and parse_args (running on IDEone):
from __future__ import print_function
from argparse import ArgumentParser
from random import randint
def main():
parser = get_parser()
input_sum_cmd = ['sum_cmd', '--sum']
input_min_cmd = ['min_cmd', '--min']
args, rest = parser.parse_known_args(
# `sum`
input_sum_cmd +
['-a', str(randint(21, 30)),
'-b', str(randint(51, 80))] +
# `min`
input_min_cmd +
['-y', str(float(randint(64, 79))),
'-z', str(float(randint(91, 120)) + .5)]
)
print('args:\t ', args,
'\nrest:\t ', rest, '\n', sep='')
sum_cmd_result = args.sm((args.a, args.b))
print(
'a:\t\t {:02d}\n'.format(args.a),
'b:\t\t {:02d}\n'.format(args.b),
'sum_cmd: {:02d}\n'.format(sum_cmd_result), sep='')
assert rest[0] == 'min_cmd'
args = parser.parse_args(rest)
min_cmd_result = args.mn((args.y, args.z))
print(
'y:\t\t {:05.2f}\n'.format(args.y),
'z:\t\t {:05.2f}\n'.format(args.z),
'min_cmd: {:05.2f}'.format(min_cmd_result), sep='')
def get_parser():
# create the top-level parser
parser = ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "sum" command
parser_a = subparsers.add_parser('sum_cmd', help='sum some integers')
parser_a.add_argument('-a', type=int,
help='an integer for the accumulator')
parser_a.add_argument('-b', type=int,
help='an integer for the accumulator')
parser_a.add_argument('--sum', dest='sm', action='store_const',
const=sum, default=max,
help='sum the integers (default: find the max)')
# create the parser for the "min" command
parser_b = subparsers.add_parser('min_cmd', help='min some integers')
parser_b.add_argument('-y', type=float,
help='an float for the accumulator')
parser_b.add_argument('-z', type=float,
help='an float for the accumulator')
parser_b.add_argument('--min', dest='mn', action='store_const',
const=min, default=0,
help='smallest integer (default: 0)')
return parser
if __name__ == '__main__':
main()
I had more or less the same requirements: Being able to set global arguments and being able to chain commands and execute them in order of command line.
I ended up with the following code. I did use some parts of the code from this and other threads.
# argtest.py
import sys
import argparse
def init_args():
def parse_args_into_namespaces(parser, commands):
'''
Split all command arguments (without prefix, like --) in
own namespaces. Each command accepts extra options for
configuration.
Example: `add 2 mul 5 --repeat 3` could be used to a sequencial
addition of 2, then multiply with 5 repeated 3 times.
'''
class OrderNamespace(argparse.Namespace):
'''
Add `command_order` attribute - a list of command
in order on the command line. This allows sequencial
processing of arguments.
'''
globals = None
def __init__(self, **kwargs):
self.command_order = []
super(OrderNamespace, self).__init__(**kwargs)
def __setattr__(self, attr, value):
attr = attr.replace('-', '_')
if value and attr not in self.command_order:
self.command_order.append(attr)
super(OrderNamespace, self).__setattr__(attr, value)
# Divide argv by commands
split_argv = [[]]
for c in sys.argv[1:]:
if c in commands.choices:
split_argv.append([c])
else:
split_argv[-1].append(c)
# Globals arguments without commands
args = OrderNamespace()
cmd, args_raw = 'globals', split_argv.pop(0)
args_parsed = parser.parse_args(args_raw, namespace=OrderNamespace())
setattr(args, cmd, args_parsed)
# Split all commands to separate namespace
pos = 0
while len(split_argv):
pos += 1
cmd, *args_raw = split_argv.pop(0)
assert cmd[0].isalpha(), 'Command must start with a letter.'
args_parsed = commands.choices[cmd].parse_args(args_raw, namespace=OrderNamespace())
setattr(args, f'{cmd}~{pos}', args_parsed)
return args
#
# Supported commands and options
#
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--print', action='store_true')
commands = parser.add_subparsers(title='Operation chain')
cmd1_parser = commands.add_parser('add', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
cmd1_parser.add_argument('add', help='Add this number.', type=float)
cmd1_parser.add_argument('-r', '--repeat', help='Repeat this operation N times.',
default=1, type=int)
cmd2_parser = commands.add_parser('mult', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
cmd2_parser.add_argument('mult', help='Multiply with this number.', type=float)
cmd2_parser.add_argument('-r', '--repeat', help='Repeat this operation N times.',
default=1, type=int)
args = parse_args_into_namespaces(parser, commands)
return args
#
# DEMO
#
args = init_args()
# print('Parsed arguments:')
# for cmd in args.command_order:
# namespace = getattr(args, cmd)
# for option_name in namespace.command_order:
# option_value = getattr(namespace, option_name)
# print((cmd, option_name, option_value))
print('Execution:')
result = 0
for cmd in args.command_order:
namespace = getattr(args, cmd)
cmd_name, cmd_position = cmd.split('~') if cmd.find('~') > -1 else (cmd, 0)
if cmd_name == 'globals':
pass
elif cmd_name == 'add':
for r in range(namespace.repeat):
if args.globals.print:
print(f'+ {namespace.add}')
result = result + namespace.add
elif cmd_name == 'mult':
for r in range(namespace.repeat):
if args.globals.print:
print(f'* {namespace.mult}')
result = result * namespace.mult
else:
raise NotImplementedError(f'Namespace `{cmd}` is not implemented.')
print(10*'-')
print(result)
Below an example:
$ python argstest.py --print add 1 -r 2 mult 5 add 3 mult -r 5 5
Execution:
+ 1.0
+ 1.0
* 5.0
+ 3.0
* 5.0
* 5.0
* 5.0
* 5.0
* 5.0
----------
40625.0
Another package which supports parallel parsers is "declarative_parser".
import argparse
from declarative_parser import Parser, Argument
supported_formats = ['png', 'jpeg', 'gif']
class InputParser(Parser):
path = Argument(type=argparse.FileType('rb'), optional=False)
format = Argument(default='png', choices=supported_formats)
class OutputParser(Parser):
format = Argument(default='jpeg', choices=supported_formats)
class ImageConverter(Parser):
description = 'This app converts images'
verbose = Argument(action='store_true')
input = InputParser()
output = OutputParser()
parser = ImageConverter()
commands = '--verbose input image.jpeg --format jpeg output --format gif'.split()
namespace = parser.parse_args(commands)
and namespace becomes:
Namespace(
input=Namespace(format='jpeg', path=<_io.BufferedReader name='image.jpeg'>),
output=Namespace(format='gif'),
verbose=True
)
Disclaimer: I am the author. Requires Python 3.6. To install use:
pip3 install declarative_parser
Here is the documentation and here is the repo on GitHub.
In order to parse the sub commands, I used the following (referred from argparse.py code). It parses the sub parser arguments and retains the help for both. Nothing additional passed there.
args, _ = parser.parse_known_args()
you can use the package optparse
import optparse
parser = optparse.OptionParser()
parser.add_option("-f", dest="filename", help="corpus filename")
parser.add_option("--alpha", dest="alpha", type="float", help="parameter alpha", default=0.5)
(options, args) = parser.parse_args()
fname = options.filename
alpha = options.alpha

Use argparse to run 1 of 2 functions in my script

I currently have 2 functions in my .py script.
#1 connects to the database and does some processing.
#2 does some other processing on files
Currently before I run the script, I have to manually comment/uncomment the function I want to run in my main if statement block.
How can I use argparse, so it asks me which function to run when I run my script?
It is possible to tell ArgumentParser objects about the function or object that has your desired behavior directly, by means of action='store_const' and const=<stuff> pairs in an add_argument() call, or with a set_defaults() call (the latter is most useful when you're using sub-parsers). If you do that, you can look up your function on the parsed_args object you get back from the parsing, instead of say, looking it up in the global namespace.
As a little example:
import argparse
def foo(parsed_args):
print "woop is {0!r}".format(getattr(parsed_args, 'woop'))
def bar(parsed_args):
print "moop is {0!r}".format(getattr(parsed_args, 'moop'))
parser = argparse.ArgumentParser()
parser.add_argument('--foo', dest='action', action='store_const', const=foo)
parser.add_argument('--bar', dest='action', action='store_const', const=bar)
parser.add_argument('--woop')
parser.add_argument('--moop')
parsed_args = parser.parse_args()
if parsed_args.action is None:
parser.parse_args(['-h'])
parsed_args.action(parsed_args)
And then you can call it like:
% python /tmp/junk.py --foo
woop is None
% python /tmp/junk.py --foo --woop 8 --moop 17
woop is '8'
% python /tmp/junk.py --bar --woop 8 --moop 17
moop is '17'
If it's just a flag of run A or B, then a simple "store_true" argument should be fine.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
_StoreTrueAction(option_strings=['--run_a_not_b'], dest='run_a_not_b', nargs=0, const=True, default=False, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args('--run_a_not_b')
>>> parsed_args = parser.parse_args('--run_a_not_b'.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run a
>>> parsed_args = parser.parse_args(''.split())
>>> if parsed_args.run_a_not_b:
print "run a"
else:
print "run b"
run b
Or if you want to actually pass in the name of the function to call, you can do it this (somewhat hackish) way:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--func_to_run', type=str)
_StoreAction(option_strings=['--func_to_run'], dest='func_to_run', nargs=None, const=None, default=None, type=<type 'str'>, choices=None, help=None, metavar=None)
>>> parsed_args = parser.parse_args('--func_to_run my_other_func'.split())
>>> parsed_args.func_to_run
'my_other_func'
>>> f = globals()[parsed_args.func_to_run]
<function my_other_func at 0x011F6670>
>>> f()
edit : to handle an integer argument, you would simply specify the type
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--run_a_not_b', action='store_true')
>>> parser.add_argument('--func_arg', type=int)
>>> parsed_args = parser.parse_args('--run_a_not_b --arg 42'.split())
>>> parsed_args = parser.parse_args('--run_a_not_b --func_arg 42'.split())
>>> parsed_args
Namespace(func_arg=42, run_a_not_b=True)
So, you can simply get parsed_args.func_arg for the value if you choose in this example.
You might consider using fabric for this.

Categories

Resources