Is there a way to use the argparse module hooked in as the interpreter for every prompt in an interface inheriting from cmd?
I'd like for my cmd interface to interpret the typical line parameter in the same way one would interpret the options and arguments passed in at runtime on the bash shell, using optional arguments with - as well as positional arguments.
Well, one way to do that is to override cmd's default method and use it to parse the line with argparse, because all commands without do_ method in your cmd.Cmd subclass will fall through to use the default method. Note the additional _ before do_test to avoid it being used as cmd's command.
import argparse
import cmd
import shlex
class TestCLI(cmd.Cmd):
def __init__(self, **kwargs):
cmd.Cmd.__init__(self, **kwargs)
self.parser = argparse.ArgumentParser()
subparsers = self.parser.add_subparsers()
test_parser = subparsers.add_parser("test")
test_parser.add_argument("--foo", default="Hello")
test_parser.add_argument("--bar", default="World")
test_parser.set_defaults(func=self._do_test)
def _do_test(self, args):
print args.foo, args.bar
def default(self, line):
args = self.parser.parse_args(shlex.split(line))
if hasattr(args, 'func'):
args.func(args)
else:
cmd.Cmd.default(self, line)
test = TestCLI()
test.cmdloop()
argparse does a sys.exit if it encounters unknown commands, so you would need to override or monkey patch your ArgumentParser's error method to raise an exception instead of exiting and handle that in the default method, in order to stay in cmd's command loop.
I would suggest you look into cliff which allows you to write commands that can automatically be used both as argparse and cmd commands, which is pretty neat. It also supports loading commands from setuptools entry points, which allows you to distribute commands as plugins to your app. Note however, that cliff uses cmd2, which is cmd's more powerful cousin, but you can replace it cmd as cmd2 was developed as a drop-in replacement for cmd.
The straight forward way would be to create an argparse parser, and parse line.split() within your function, expecting SystemExit in case invalid arguments are supplied (parse_args() calls sys.exit() when it finds invalid arguments).
class TestInterface(cmd.Cmd):
__test1_parser = argparse.ArgumentParser(prog="test1")
__test1_parser.add_argument('--bar', help="bar help")
def help_test1(self): self.__test1_parser.print_help()
def do_test1(self, line):
try:
parsed = self.__test1_parser.parse_args(line.split())
except SystemExit:
return
print("Test1...")
print(parsed)
If invalid arguments are passed, parse_args() will print errors, and the program will return to the interface without exiting.
(Cmd) test1 --unk
usage: test1 [-h] [--bar BAR]
test1: error: unrecognized arguments: --unk
(Cmd)
Everything else should work the same as a regular argparse use case, also maintaining all of cmd's functionality (help messages, function listing, etc.)
Source: https://groups.google.com/forum/#!topic/argparse-users/7QRPlG97cak
Another way, which simplifies the setup above, is using the decorator below:
class ArgparseCmdWrapper:
def __init__(self, parser):
"""Init decorator with an argparse parser to be used in parsing cmd-line options"""
self.parser = parser
self.help_msg = ""
def __call__(self, f):
"""Decorate 'f' to parse 'line' and pass options to decorated function"""
if not self.parser: # If no parser was passed to the decorator, get it from 'f'
self.parser = f(None, None, None, True)
def wrapped_f(*args):
line = args[1].split()
try:
parsed = self.parser.parse_args(line)
except SystemExit:
return
f(*args, parsed=parsed)
wrapped_f.__doc__ = self.__get_help(self.parser)
return wrapped_f
#staticmethod
def __get_help(parser):
"""Get and return help message from 'parser.print_help()'"""
f = tempfile.SpooledTemporaryFile(max_size=2048)
parser.print_help(file=f)
f.seek(0)
return f.read().rstrip()
It makes defining additional commands simpler, where they take an extra parsed parameter that contains the result of a successful parse_args(). If there are any invalid arguments the function is never entered, everything being handled by the decorator.
__test2_parser = argparse.ArgumentParser(prog="test2")
__test2_parser.add_argument('--foo', help="foo help")
#WrapperCmdLineArgParser(parser=__test2_parser)
def do_test2(self, line, parsed):
print("Test2...")
print(parsed)
Everything works as the original example, including argparse generated help messages - without the need to define a help_command() function.
Source: https://codereview.stackexchange.com/questions/134333/using-argparse-module-within-cmd-interface
Related
Ok, so lets say I have this part of the code (it is not perfect). I want this scenario - I am detecting the cloud name ones as an argument in __init__ so all other modules and scripts will run on that cloud, or if I want to run the specific python file from the terminal I can detect the cloud on which I want it to be runned like this python my_script.py cloud1 What would be the best way to do this?
The following script does work when I am running it from the terminal with the argument, but if not it gives this error
usage: To check what is the cloud name config_parser.py: error: too few arguments'
here is a code
class CredentialsCP:
def __init__(self, cloud_name=None):
self.config = ConfigParser.ConfigParser()
self.cloud_name = cloud_name
self.config_file_pass = os.path.expanduser('~/PycharmProjects/ui/config.cfg')
self.parser = ArgumentParser(usage='To check what is the cloud name')
self.parser.add_argument('cloud')
self.args = self.parser.parse_args()
if self.args:
self.cloud_name = self.args.cloud
if self.cloud_name is None:
self.cloud_name = 'cloud1'
I have a function that shows a url of the cloud, that how it is callable
ArgumentParser has provision for optional arguments, but the default for positional (non flag) arguments is required. Here you could use:
self.parser = ArgumentParser(usage='To check what is the cloud name')
self.parser.add_argument('cloud', nargs='?', default='cloud1') # optional argument with default value
self.args = self.parser.parse_args()
self.cloud_name = self.args.cloud
Possible improvements:
In this code, parser and args are member attributes of the class. If they are not used outside the __init__ method, they could just be locals:
self.parser = ArgumentParser(usage='To check what is the cloud name')
parser.add_argument('cloud', nargs='?', default='cloud1') # optional argument with default value
args = .parser.parse_args()
self.cloud_name = args.cloud
For only identifying a single optional argument, argparse may be overkill. sys.argv could be enough:
...
self.config_file_pass = os.path.expanduser('~/PycharmProjects/ui/config.cfg')
self.cloud_name = sys.argv[1] if len(sys.argv) > 1 else 'cloud1'
My program "program.py" has the form:
if __name__=='__main__':
args = parse_args()
main_function(args)
However, if I import program.py as a module and run program.main_function, how can I pass the parsed arguments structure as an argument to the main_function?
Here is the definition of the parse_args()
def parse_args():
parser=argparse.ArgumentParser()
parser.add_argument(...)
args=parser.parse_args()
return args
If we are talking about argparse.parse_args from the standard library, just pass the list of arguments explicitly.
For example, if you call your program from the command line with these arguments:
program --verbose --mode=3 file1 file2
the shell splits the command line into five words, the program name and its four arguments. These are stored in sys.argv.
To achieve the same effect directly from Python:
args = parse_args(['--verbose', '--mode=3', 'file1' , 'file2'])
main_function(args)
UPDATE - parse_args modification:
def parse_args(arglist=None):
parser=argparse.ArgumentParser()
parser.add_argument(...)
args=parser.parse_args(arglist)
return args
I have quite a big program which has a CLI interaction based on argparse, with several sub parsers. The list of supported choices for the subparsers arguments are determined based on DB queries, parsing different xml files, making different calculations etc, so it is quite IO intensive and time consuming.
The problem is that argparse seems to fetch choices for all sub parser when I run the script, which adds a considerable and annoying startup delay.
Is there a way to make argparse only fetch and validate choices for the currently used sub parser?
One solution could be to move all the validation logic deeper inside the code but that would mean quite a lot of work which I would like to avoid, if possible.
Thank you
To delay the fetching of choices, you could parse the command-line in two stages: In the first stage, you find only the subparser, and in the second stage, the subparser is used to parse the rest of the arguments:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('subparser', choices=['foo','bar'])
def foo_parser():
parser = argparse.ArgumentParser()
parser.add_argument('fooval', choices='123')
return parser
def bar_parser():
parser = argparse.ArgumentParser()
parser.add_argument('barval', choices='ABC')
return parser
dispatch = {'foo':foo_parser, 'bar':bar_parser}
args, unknown = parser.parse_known_args()
args = dispatch[args.subparser]().parse_args(unknown)
print(args)
It could be used like this:
% script.py foo 2
Namespace(fooval='2')
% script.py bar A
Namespace(barval='A')
Note that the top-level help message will be less friendly, since it can only tell you about the subparser choices:
% script.py -h
usage: script.py [-h] {foo,bar}
...
To find information about the choices in each subparser, the user would have to select the subparser and pass the -h to it:
% script.py bar -- -h
usage: script.py [-h] {A,B,C}
All arguments after the -- are considered non-options (to script.py) and are thus parsed by the bar_parser.
Here's a quick and dirty example of a 'lazy' choices. In this case choices are a range of integers. I think a case that requires expensive DB lookups could implemented in a similar fashion.
# argparse with lazy choices
class LazyChoice(object):
# large range
def __init__(self, argmax):
self.argmax=argmax
def __contains__(self, item):
# a 'lazy' test that does not enumerate all choices
return item<=self.argmax
def __iter__(self):
# iterable for display in error message
# use is in:
# tup = value, ', '.join(map(repr, action.choices))
# metavar bypasses this when formatting help/usage
return iter(['integers less than %s'%self.argmax])
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--regular','-r',choices=['one','two'])
larg = parser.add_argument('--lazy','-l', choices=LazyChoice(10))
larg.type = int
print parser.parse_args()
Implementing the testing part (__contains__) is easy. The help/usage can be customized with help and metavar attributes. Customizing the error message is harder. http://bugs.python.org/issue16468 discusses alternatives when choices are not iterable. (also on long list choices: http://bugs.python.org/issue16418)
I've also shown how the type can be changed after the initial setup. That doesn't solve the problem of setting type based on subparser choice. But it isn't hard to write a custom type, one that does some sort of Db lookup. All a type function needs to do is take a string, return the correct converted value, and raise ValueError if there's a problem.
I have solved the issue by creating a simple ArgumentParser subclass:
import argparse
class ArgumentParser(argparse.ArgumentParser):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.lazy_init = None
def parse_known_args(self, args=None, namespace=None):
if self.lazy_init is not None:
self.lazy_init()
self.lazy_init = None
return super().parse_known_args(args, namespace)
Then I can use it as following:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='command', title='commands', parser_class=ArgumentParser)
subparsers.required = True
subparser = subparsers.add_parser(
'do-something', help="do something",
description="Do something great.",
)
def lazy_init():
from my_database import data
subparser.add_argument(
'-o', '--option', choices=data.expensive_fetch(), action='save',
)
subparser.lazy_init = lazy_init
This will really initialize a sub-parser only when parent parser tries to parse arguments for the sub-parser. So if you do program -h it will not initialize the sub-parser, but if you do program do-something -h it will.
This is a script that tests the idea of delaying the creation of a subparser until it is actually needed. In theory it might save start up time, by only creating the subparser that's actually needed.
I use the nargs=argparse.PARSER to replicate the subparser behavior in the main parser. help behavior is similar.
# lazy subparsers test
# lazy behaves much like a regular subparser case, but only creates one subparser
# for N=5 time differences do not rise above the noise
import argparse
def regular(N):
parser = argparse.ArgumentParser()
sp = parser.add_subparsers(dest='cmd')
for i in range(N):
spp = sp.add_parser('cmd%s'%i)
spp.set_defaults(func='cmd%s'%(10*i))
spp.add_argument('-f','--foo')
spp.add_argument('pos', nargs='*')
return parser
def lazy(N):
parser = argparse.ArgumentParser()
sp = parser.add_argument('cmd', nargs=argparse.PARSER, choices=[])
for i in range(N):
sp.choices.append('cmd%s'%i)
return parser
def subpar(cmd):
cmd, argv = cmd[0], cmd[1:]
parser = argparse.ArgumentParser(prog=cmd)
parser.add_argument('-f','--foo')
parser.add_argument('pos', nargs='*')
parser.set_defaults(func=cmd)
args = parser.parse_args(argv)
return args
N = 5
mode = True #False
argv = 'cmd1 -f1 a b c'.split()
if mode:
args = regular(N).parse_args(argv)
print(args)
else:
args = lazy(N).parse_args(argv)
print(args)
if isinstance(args.cmd, list):
sargs = subpar(args.cmd)
print(sargs)
test runs with different values of mode (and N=5)
1004:~/mypy$ time python3 stack44315696.py
Namespace(cmd='cmd1', foo='1', func='cmd10', pos=['a', 'b', 'c'])
real 0m0.052s
user 0m0.044s
sys 0m0.008s
1011:~/mypy$ time python3 stack44315696.py
Namespace(cmd=['cmd1', '-f1', 'a', 'b', 'c'])
Namespace(foo='1', func='cmd1', pos=['a', 'b', 'c'])
real 0m0.051s
user 0m0.048s
sys 0m0.000s
N has to be much larger to start seeing a effect.
I am trying to make use of an argument handler that I wrote using argparse from within another python script. I would like to call it by passing it a list of arguments. Here is a simple example:
def argHandler(argv):
import argparse
parser = argparse.ArgumentParser(description='Test argument parser')
parser.add_argument('foo', action='store',type=str)
parser.add_argument('bar', action='store',type=str)
parser.add_argument('-nee','--knightssaynee',action='store',type=str)
args = parser.parse_args()
return args.foo, args.bar, args.nee
if __name__=='__main__':
argList = ['arg1','arg2','-nee','arg3']
print argHandler(argList)
This returns a:
usage: scratch.py [-h] [-nee KNIGHTSSAYNEE] foo bar
scratch.py: error: too few arguments
It seems to me that the function that I define should take a list of arguments and flags, and return a namespace. Am I wrong?
You need to pass those arguments to the parser.parse_args() method:
args = parser.parse_args(argv)
From the ArgumentParser.parse_args() documentation:
ArgumentParser.parse_args(args=None, namespace=None)
[...]
By default, the argument strings are taken from sys.argv [...]
Note the args argument there. You may want to make the argv argument to your argHandler() function default to None as well; that way you don't have to pass in an argument and end up with the same default None value:
def argHandler(argv=None):
another way i get input to program is from json file with key value pair and using json load library to load contents of file as json object.
I'm writing a wrapper around the ssh command line client. After the first positional argument that's part of command, all further options should also be treated as positional arguments.
Under optparse, I believe this would be done with disable_interspersed_args.
Presently I have something like this:
parser = argparse.ArgumentParser()
parser.add_argument('--parallel', default=False, action='store_true')
# maybe allow no command? this would ssh interactively into each machine...
parser.add_argument('command', nargs='+')
args = parser.parse_args()
But if options are passed as part of the command (such as my_wrapper ls -l), they're instead interpreted by ArgumentParser as unknown options. error: unrecognized arguments: -l
If I use parse_known_args(), the options may be taken out of order.
p = argparse.ArgumentParser()
p.add_argument('-a', action='store_true')
p.add_argument('command', nargs='+')
print(p.parse_known_args())
$ python3 bah.py -b ls -l -a
(Namespace(a=True, command=['ls']), ['-b', '-l'])
Here you can see that -b's position before ls has been lost, and -a has been parsed out from the command, which is not desired.
How can I:
Prevent arguments from being parsed after a certain point?
Disable parsing of interspersed arguments?
Allow arguments with a prefix to be consumed as positional arguments?
I had the same problem. I found the solution on the argparse bug tracker: http://code.google.com/p/argparse/issues/detail?id=52
The solution is simple: replace nargs='+' (or '*') with nargs=argparse.REMAINDER. This special value is not documented, but it does what you want.
I think your best bet to start solving these issues is to try out -- after all your optional args. -- is a pseudo-arg that tells ArgumentParser that everything after is a positional argument. Docs are here
As for prevent arguments from being parsed after a certain point, you can pass part of argv to parse_args. That combined with some introspection can be used to limit what is parsed.
What #dcolish suggested is the universal approach. Here is a sample implementation which also supports the standard -- separator, but its usage is not required for correct parsing.
Result:
# ./parse-pos.py -h
usage: parse-pos.py [-h] [-qa] [-qb] COMMAND [ARGS...]
# ./parse-pos.py -qa ls -q -h aa /bb
try_argv = ['-qa', 'ls']
cmd_rest_argv = ['-q', '-h', 'aa', '/bb']
parsed_args = Namespace(command='ls', qa=True, qb=False)
The code:
#!/usr/bin/python3
import argparse
import sys
from pprint import pprint
class CustomParserError(Exception):
pass
class CustomArgumentParser(argparse.ArgumentParser):
def error(self, message):
raise CustomParserError(message)
def original_error(self, message):
super().error(message)
def parse_argv():
parser = CustomArgumentParser(description='Example')
parser.add_argument('command', metavar='COMMAND [ARGS...]', help='the command to be executed')
parser.add_argument('-qa', action='store_true') # "ambiguous option" if you specify just "-q"
parser.add_argument('-qb', action='store_true') # "ambiguous option" if you specify just "-q"
def parse_until_positional(parser, _sys_argv = None):
if _sys_argv is None:
_sys_argv = sys.argv[1:] # skip the program name
for i in range(0, len(_sys_argv) + 1):
try_argv = _sys_argv[0:i]
try:
parsed_args = parser.parse_args(try_argv)
except CustomParserError as ex:
if len(try_argv) == len(_sys_argv):
# this is our last try and we still couldn't parse anything
parser.original_error(str(ex)) # sys.exit()
continue
# if we are here, we parsed our known optional & dash-prefixed parameters and the COMMAND
cmd_rest_argv = _sys_argv[i:]
break
return (parsed_args, cmd_rest_argv, try_argv)
(parsed_args, cmd_rest_argv, try_argv) = parse_until_positional(parser)
# debug
pprint(try_argv)
pprint(cmd_rest_argv)
pprint(parsed_args)
return (parsed_args, cmd_rest_argv)
def main():
parse_argv()
main()
Another option is to use parse_known_args, which stops parsing when an unknown argument is encountered.