I am trying to make use of an argument handler that I wrote using argparse from within another python script. I would like to call it by passing it a list of arguments. Here is a simple example:
def argHandler(argv):
import argparse
parser = argparse.ArgumentParser(description='Test argument parser')
parser.add_argument('foo', action='store',type=str)
parser.add_argument('bar', action='store',type=str)
parser.add_argument('-nee','--knightssaynee',action='store',type=str)
args = parser.parse_args()
return args.foo, args.bar, args.nee
if __name__=='__main__':
argList = ['arg1','arg2','-nee','arg3']
print argHandler(argList)
This returns a:
usage: scratch.py [-h] [-nee KNIGHTSSAYNEE] foo bar
scratch.py: error: too few arguments
It seems to me that the function that I define should take a list of arguments and flags, and return a namespace. Am I wrong?
You need to pass those arguments to the parser.parse_args() method:
args = parser.parse_args(argv)
From the ArgumentParser.parse_args() documentation:
ArgumentParser.parse_args(args=None, namespace=None)
[...]
By default, the argument strings are taken from sys.argv [...]
Note the args argument there. You may want to make the argv argument to your argHandler() function default to None as well; that way you don't have to pass in an argument and end up with the same default None value:
def argHandler(argv=None):
another way i get input to program is from json file with key value pair and using json load library to load contents of file as json object.
Related
I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script
If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()
You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?
You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2
My program "program.py" has the form:
if __name__=='__main__':
args = parse_args()
main_function(args)
However, if I import program.py as a module and run program.main_function, how can I pass the parsed arguments structure as an argument to the main_function?
Here is the definition of the parse_args()
def parse_args():
parser=argparse.ArgumentParser()
parser.add_argument(...)
args=parser.parse_args()
return args
If we are talking about argparse.parse_args from the standard library, just pass the list of arguments explicitly.
For example, if you call your program from the command line with these arguments:
program --verbose --mode=3 file1 file2
the shell splits the command line into five words, the program name and its four arguments. These are stored in sys.argv.
To achieve the same effect directly from Python:
args = parse_args(['--verbose', '--mode=3', 'file1' , 'file2'])
main_function(args)
UPDATE - parse_args modification:
def parse_args(arglist=None):
parser=argparse.ArgumentParser()
parser.add_argument(...)
args=parser.parse_args(arglist)
return args
Is there a way to use argparse with any list of strings, instead of only with sys.argv?
Here's my problem: I have a program which looks something like this:
# This file is program1.py
import argparse
def main(argv):
parser = argparse.ArgumentParser()
# Do some argument parsing
if __name__ == '__main__':
main(sys.argv)
This works fine when this program is called straight from the command line. However, I have another python script which runs batch versions of this script with different commandline arguments, which I'm using like this:
import program1
arguments = ['arg1', 'arg2', 'arg3']
program1.main(arguments)
I still want to be able to parse the arguments, but argparse automatically defaults to using sys.argv instead of the arguments that I give it. Is there a way to pass in the argument list instead of using sys.argv?
You can pass a list of strings to parse_args:
parser.parse_args(['--foo', 'FOO'])
Just change the script to default to sys.argv[1:] and parse arguments omitting the first one (which is the name of the invoked command)
import argparse,sys
def main(argv=sys.argv[1:]):
parser = argparse.ArgumentParser()
parser.add_argument("--level", type=int)
args = parser.parse_args(argv)
if __name__ == '__main__':
main()
Or, if you cannot omit the first argument:
import argparse,sys
def main(args=None):
# if None passed, uses sys.argv[1:], else use custom args
parser = argparse.ArgumentParser()
parser.add_argument("--level", type=int)
args = parser.parse_args(args)
# Do some argument parsing
if __name__ == '__main__':
main()
Last one: if you cannot change the called program, you can still do something
Let's suppose the program you cannot change is called argtest.py (I added a call to print arguments)
Then just change the local argv value of the argtest.sys module:
import argtest
argtest.sys.argv=["dummy","foo","bar"]
argtest.main()
output:
['dummy', 'foo', 'bar']
Python argparse now has a parameter nargs for add_argument (https://docs.python/3/library/argparse.html).
It allows us to have as many arguments as we want for a named parameter (here, alist)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--alist", nargs="*")
args = parser.parse_args()
print(args.alist)
All command line values that follow --alist are added to a list.
Example:
$ python3 argparse-01.py --alist fred barney pebbles "bamm bamm"
['fred', 'barney', 'pebbles', 'bamm bamm']
As you see, it is allowed to quote the arguments, but not necessary unless you need to protect a space.
I'm new to python and currently playing with it.
I have a script which does some API Calls to an appliance. I would like to extend the functionality and call different functions based on the arguments given when calling the script.
Currently I have the following:
parser = argparse.ArgumentParser()
parser.add_argument("--showtop20", help="list top 20 by app",
action="store_true")
parser.add_argument("--listapps", help="list all available apps",
action="store_true")
args = parser.parse_args()
I also have a
def showtop20():
.....
and
def listapps():
....
How can I call the function (and only this) based on the argument given?
I don't want to run
if args.showtop20:
#code here
if args.listapps:
#code here
as I want to move the different functions to a module later on keeping the main executable file clean and tidy.
Since it seems like you want to run one, and only one, function depending on the arguments given, I would suggest you use a mandatory positional argument ./prog command, instead of optional arguments (./prog --command1 or ./prog --command2).
so, something like this should do it:
FUNCTION_MAP = {'top20' : my_top20_func,
'listapps' : my_listapps_func }
parser.add_argument('command', choices=FUNCTION_MAP.keys())
args = parser.parse_args()
func = FUNCTION_MAP[args.command]
func()
At least from what you have described, --showtop20 and --listapps sound more like sub-commands than options. Assuming this is the case, we can use subparsers to achieve your desired result. Here is a proof of concept:
import argparse
import sys
def showtop20():
print('running showtop20')
def listapps():
print('running listapps')
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Create a showtop20 subcommand
parser_showtop20 = subparsers.add_parser('showtop20', help='list top 20 by app')
parser_showtop20.set_defaults(func=showtop20)
# Create a listapps subcommand
parser_listapps = subparsers.add_parser('listapps', help='list all available apps')
parser_listapps.set_defaults(func=listapps)
# Print usage message if no args are supplied.
# NOTE: Python 2 will error 'too few arguments' if no subcommand is supplied.
# No such error occurs in Python 3, which makes it feasible to check
# whether a subcommand was provided (displaying a help message if not).
# argparse internals vary significantly over the major versions, so it's
# much easier to just override the args passed to it.
if len(sys.argv) <= 1:
sys.argv.append('--help')
options = parser.parse_args()
# Run the appropriate function (in this case showtop20 or listapps)
options.func()
# If you add command-line options, consider passing them to the function,
# e.g. `options.func(options)`
There are lots of ways of skinning this cat. Here's one using action='store_const' (inspired by the documented subparser example):
p=argparse.ArgumentParser()
p.add_argument('--cmd1', action='store_const', const=lambda:'cmd1', dest='cmd')
p.add_argument('--cmd2', action='store_const', const=lambda:'cmd2', dest='cmd')
args = p.parse_args(['--cmd1'])
# Out[21]: Namespace(cmd=<function <lambda> at 0x9abf994>)
p.parse_args(['--cmd2']).cmd()
# Out[19]: 'cmd2'
p.parse_args(['--cmd1']).cmd()
# Out[20]: 'cmd1'
With a shared dest, each action puts its function (const) in the same Namespace attribute. The function is invoked by args.cmd().
And as in the documented subparsers example, those functions could be written so as to use other values from Namespace.
args = parse_args()
args.cmd(args)
For sake of comparison, here's the equivalent subparsers case:
p = argparse.ArgumentParser()
sp = p.add_subparsers(dest='cmdstr')
sp1 = sp.add_parser('cmd1')
sp1.set_defaults(cmd=lambda:'cmd1')
sp2 = sp.add_parser('cmd2')
sp2.set_defaults(cmd=lambda:'cmd2')
p.parse_args(['cmd1']).cmd()
# Out[25]: 'cmd1'
As illustrated in the documentation, subparsers lets you define different parameter arguments for each of the commands.
And of course all of these add argument or parser statements could be created in a loop over some list or dictionary that pairs a key with a function.
Another important consideration - what kind of usage and help do you want? The different approaches generate very different help messages.
If your functions are "simple enough" take adventage of type parameter https://docs.python.org/2.7/library/argparse.html#type
type= can take any callable that takes a single string argument and
returns the converted value:
In your example (even if you don't need a converted value):
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
This simple script:
import argparse
def showtop20(dummy):
print "{0}\n".format(dummy) * 5
parser = argparse.ArgumentParser()
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
args = parser.parse_args()
Will give:
# ./test.py --listapps test
test
test
test
test
test
test
Instead of using your code as your_script --showtop20, make it into a sub-command your_script showtop20 and use the click library instead of argparse. You define functions that are the name of your subcommand and use decorators to specify the arguments:
import click
#click.group()
#click.option('--debug/--no-debug', default=False)
def cli(debug):
print(f'Debug mode is {"on" if debug else "off"}')
#cli.command() # #cli, not #click!
def showtop20():
# ...
#cli.command()
def listapps():
# ...
See https://click.palletsprojects.com/en/master/commands/
# based on parser input to invoke either regression/classification plus other params
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--target", type=str)
parser.add_argument("--type", type=str)
parser.add_argument("--deviceType", type=str)
args = parser.parse_args()
df = pd.read_csv(args.path)
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]
if args.type == "classification":
classify = AutoML(df, args.target, args.type, args.deviceType)
classify.class_dist()
classify.classification()
elif args.type == "regression":
reg = AutoML(df, args.target, args.type, args.deviceType)
reg.regression()
else:
ValueError("Invalid argument passed")
# Values passed as : python app.py --path C:\Users\Abhishek\Downloads\adult.csv --target income --type classification --deviceType GPU
You can evaluate using evalwhether your argument value is callable:
import argparse
def list_showtop20():
print("Calling from showtop20")
def list_apps():
print("Calling from listapps")
my_funcs = [x for x in dir() if x.startswith('list_')]
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--function", required=True,
choices=my_funcs,
help="function to call", metavar="")
args = parser.parse_args()
eval(args.function)()
Is there a way to use the argparse module hooked in as the interpreter for every prompt in an interface inheriting from cmd?
I'd like for my cmd interface to interpret the typical line parameter in the same way one would interpret the options and arguments passed in at runtime on the bash shell, using optional arguments with - as well as positional arguments.
Well, one way to do that is to override cmd's default method and use it to parse the line with argparse, because all commands without do_ method in your cmd.Cmd subclass will fall through to use the default method. Note the additional _ before do_test to avoid it being used as cmd's command.
import argparse
import cmd
import shlex
class TestCLI(cmd.Cmd):
def __init__(self, **kwargs):
cmd.Cmd.__init__(self, **kwargs)
self.parser = argparse.ArgumentParser()
subparsers = self.parser.add_subparsers()
test_parser = subparsers.add_parser("test")
test_parser.add_argument("--foo", default="Hello")
test_parser.add_argument("--bar", default="World")
test_parser.set_defaults(func=self._do_test)
def _do_test(self, args):
print args.foo, args.bar
def default(self, line):
args = self.parser.parse_args(shlex.split(line))
if hasattr(args, 'func'):
args.func(args)
else:
cmd.Cmd.default(self, line)
test = TestCLI()
test.cmdloop()
argparse does a sys.exit if it encounters unknown commands, so you would need to override or monkey patch your ArgumentParser's error method to raise an exception instead of exiting and handle that in the default method, in order to stay in cmd's command loop.
I would suggest you look into cliff which allows you to write commands that can automatically be used both as argparse and cmd commands, which is pretty neat. It also supports loading commands from setuptools entry points, which allows you to distribute commands as plugins to your app. Note however, that cliff uses cmd2, which is cmd's more powerful cousin, but you can replace it cmd as cmd2 was developed as a drop-in replacement for cmd.
The straight forward way would be to create an argparse parser, and parse line.split() within your function, expecting SystemExit in case invalid arguments are supplied (parse_args() calls sys.exit() when it finds invalid arguments).
class TestInterface(cmd.Cmd):
__test1_parser = argparse.ArgumentParser(prog="test1")
__test1_parser.add_argument('--bar', help="bar help")
def help_test1(self): self.__test1_parser.print_help()
def do_test1(self, line):
try:
parsed = self.__test1_parser.parse_args(line.split())
except SystemExit:
return
print("Test1...")
print(parsed)
If invalid arguments are passed, parse_args() will print errors, and the program will return to the interface without exiting.
(Cmd) test1 --unk
usage: test1 [-h] [--bar BAR]
test1: error: unrecognized arguments: --unk
(Cmd)
Everything else should work the same as a regular argparse use case, also maintaining all of cmd's functionality (help messages, function listing, etc.)
Source: https://groups.google.com/forum/#!topic/argparse-users/7QRPlG97cak
Another way, which simplifies the setup above, is using the decorator below:
class ArgparseCmdWrapper:
def __init__(self, parser):
"""Init decorator with an argparse parser to be used in parsing cmd-line options"""
self.parser = parser
self.help_msg = ""
def __call__(self, f):
"""Decorate 'f' to parse 'line' and pass options to decorated function"""
if not self.parser: # If no parser was passed to the decorator, get it from 'f'
self.parser = f(None, None, None, True)
def wrapped_f(*args):
line = args[1].split()
try:
parsed = self.parser.parse_args(line)
except SystemExit:
return
f(*args, parsed=parsed)
wrapped_f.__doc__ = self.__get_help(self.parser)
return wrapped_f
#staticmethod
def __get_help(parser):
"""Get and return help message from 'parser.print_help()'"""
f = tempfile.SpooledTemporaryFile(max_size=2048)
parser.print_help(file=f)
f.seek(0)
return f.read().rstrip()
It makes defining additional commands simpler, where they take an extra parsed parameter that contains the result of a successful parse_args(). If there are any invalid arguments the function is never entered, everything being handled by the decorator.
__test2_parser = argparse.ArgumentParser(prog="test2")
__test2_parser.add_argument('--foo', help="foo help")
#WrapperCmdLineArgParser(parser=__test2_parser)
def do_test2(self, line, parsed):
print("Test2...")
print(parsed)
Everything works as the original example, including argparse generated help messages - without the need to define a help_command() function.
Source: https://codereview.stackexchange.com/questions/134333/using-argparse-module-within-cmd-interface