Background
I am trying to write a python script that contains multiple functions like this:
import sys
def util1(x, y):
assert(x is not None)
assert(y is not None)
#does something
def util2(x, y):
assert(x is not None)
assert(y is not None)
#does something
def util3(x, y):
assert(x is not None)
assert(y is not None)
#does something
I need to be able to call any method command line:
python3 myscript.py util1 arg1 arg2
or
python3 myscript.py util3 arg1 arg2
Problem
I don't know the proper way to grab the command line args and pass them to the methods. I found a way to grab the first arg... but I would like a way to say "pass all arg to function x" if this is possible.
What I've tried So far
So far, I at the bottom of my script, I added the following logic:
if __name__ == '__main__':
globals()[sys.argv[1]]()
and so now, when I try to run my script, I get the following response:
lab-1:/var/www/localhost/htdocs/widgets# python3 myscript.py utils1 1 99999
Traceback (most recent call last):
File "myscript.py", line 62, in <module>
globals()[sys.argv[1]]()
TypeError: util1() missing 2 required positional arguments: 'x' and 'y'
I've also tried the following:
globals()[*sys.argv[1:]]()
globals()[*sys.argv[1]:[2]]()
But that doesn't work. I'm getting errors like "TypeError: unhashable type: 'list'
If you can point me in the right direction, I'd appreciate it.
Thanks.
EDIT 1
Based on the recommendation here to review a similar post, I changed my logic to include the argparse library. So now I have the following:
parser = argparse.ArgumentParser(description='This is the description of my program')
parser.add_argument('-lc','--lower_create', type=int, help='lower range value for util1')
parser.add_argument('-uc','--upper_create', type=int, help='upper range value for util1')
parser.add_argument('-lr','--lower_reserve', type=int, help='lower range value for util3')
parser.add_argument('-ur','--upper_reserve', type=int, help='upper range value for util3')
args = parser.parse_args()
#if __name__ == '__main__':
# globals()[sys.argv[1]](sys.argv[2], sys.argv[3])
What's not clear is how I "link" these arguments with a specific function.
So let's say I need -lc and -uc for util1. How can I make that association?
and then for example associate -lr and -ur with util3?
Thank you
You need to pass the arguments to the function when you call it. The naive way to do this would be like this: globals()[sys.argv[1]](sys.argv[2], sys.argv[3]) although you'll probably want to do some extra checking to make sure the arguments exist, as well as the function being called.
That is a nice question.
Try like this.
import sys
def util1(x, y):
print('This is "util1" with the following arguments: "'+x+'" and "'+y+'"')
#does something
def util2(x, y):
print('This is "util2" with the following arguments: "'+x+'" and "'+y+'"')
#does something
def util3(x, y):
print('This is "util3" with the following arguments: "'+x+'" and "'+y+'"')
#does something
locals()[sys.argv[1]](sys.argv[2] , sys.argv[3])
Then calling it like this, works great for me. Just tried it on my test machine.
python file.py util1 arg1 arg2
You could do this quite neatly with click, e.g.
#click.command()
#click.argument('x')
#click.argument('y')
def util1(x, y):
#does something
You can also use varargs, so you don't have to specify every argument:
#click.command()
#click.argument('args', nargs=-1)
def util2(args):
#does something, args is a list
Click also supports different arguments types, validation, etc.
Related
Let's say I have a main.py and a subfile.py.
main.py
import subfile #Writen by myself. I want to share argparse with it
import argparse
parser = argparse.ArgumentParser(description='test',formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--a', type=int, default=0, help='test')
args = parser.parse_args()
args.a=0
subfile.fun()
args.a=1
subfile.fun()
args.a=0
subfile.fun()
2.subfile.py
xxxxxsome operation here to use args in main.pyxxxx
# In tensorflow wrapped argparse, I just need to define the same ArgumentParser to use args defined in main.py.
# But it's not suitable for "argparse", which would cause conflicts.
def fun():
if args.a==0:
print('yes')
else:
print('no')
In main.py I use argparse to pass arguments, and main.py use subfile.py.
Now, I want to use args define in main.py, how can I do?
(The codes are an simplified example described the problem, which can't be solved by reorganized code and has to be solved by using args in subfile.py. )
Thanks for your help! I have been torture by some bugs for the whole day and your answer would help me out. Appreciated!
edit:
Actually I know how to pass a paramter to a funciton. But as I said above, I writed the example code just for the key solution: is how to use the library "argparse" across python file, which is supported in a similar library in tensorflow "FLAG".
You can add another file, let's call it arguments.py for example.
Then include at the top of both main.py and subfile.py the following line:
from arguments import args
You then move all the code relating to parsing the command line arguments to arguments.py.
This way both the modules in main.py and subfile.py have access to the same args object.
def
When you define a function, if you need parameters, you can add them between the parentesis.
This is the model:
def fun(par: type) -> ReturnedType:
pass
For example:
def fun(a: int) -> None:
if a == 0:
print("Yes")
else:
print("No")
Then, when calling subfile.fun() add the argument like this:
subfile.fun(a = a)
or like this:
subfile.fun(a)
return
Remember your function doesn't return anything.
If you want to assign to a variable "yes" or "no" you can do like this:
def fun(a: int) -> str:
if a == 0:
return "Yes"
else:
return "No"
If you want to return only "yes" or "no", you might like to return a boolean instead:
def fun(a: int) -> bool:
if a == 0:
return True
else:
return False
You can also write the if statement in one line:
print('Yes') if (a == 0) else print('No)
import
To import subfile.py the file should be in the same directory.
Otherwise you have to do like this:
from sys import path
# Imagine you need C:\\Python\MyFolder\subfile.py
path.append(r"C:\\Python\MyFolder")
import subfile
I have a use case where I have a main python script with many command line arguments, I need to break it's functionality into multiple smaller scripts, a few command-line arguments will be common to more than one smaller scripts. I want to reduce code duplicacy. I tried to use decorators to register each argument to one or more scripts, but am not able to get around an error. Another caveat I have is I want to set default values for shared argument according to which script is being run. This is what I have currently
argument_parser.py
import argparse
import functools
import itertools
from scripts import Scripts
from collections import defaultdict
_args_register = defaultdict(list)
def argument(scope):
"""
Decorator to add argument to argument registry
:param scope: The module name to register current argument function to can also be a list of modules
:return: The decorated function after after adding it to registry
"""
def register(func):
if isinstance(scope, Scripts):
_args_register[scope].append(func)
elif isinstance(scope, list) and Scripts.ALL in scope:
_args_register[Scripts.ALL].append(func)
else:
for module in scope:
_args_register[module].append(func)
return func
return register
class ArgumentHandler:
def __init__(self, script, parser=None):
self._parser = parser or argparse.ArgumentParser(description=__doc__)
assert script in Scripts
self._script = script
#argument(scope=Scripts.ALL)
def common_arg(self):
self._parser.add_arg("--common-arg",
default=self._script,
help="An arg common to all scripts")
#argument(scope=[Scripts.TRAIN, Scripts.TEST])
def train_test_arg(self):
self._parser.add_arg("--train-test-arg",
default=self._script,
help=f"An arg common to train-test scripts added in argument handler"
)
def parse_args(self):
for argument in itertools.chain(_args_register[Scripts.ALL],
_args_register[self._script]):
argument()
_args = self._parser.parse_args()
return _args
One of the smaller scripts train.py
"""
A Train script to abstract away training tasks
"""
import argparse
from argument_parser import ArgumentHandler
from scripts import Scripts
current = Scripts.TRAIN
parser = argparse.ArgumentParser(description=__doc__)
def get_args() -> argparse.Namespace:
parser.add_argument('--train-arg',
default='blah',
help='a train argumrnt set in the train script')
args_handler = ArgumentHandler(parser=parser, script=current)
return args_handler.parse_args()
if __name__ == '__main__':
print(get_args())
When I run train.py I get the following error
File "../argument_parser.py", line 68, in parse_args
argument()
TypeError: common_arg() missing 1 required positional argument: 'self'
Process finished with exit code 1
I think this is because decorators are run at import time, but am not sure, is there any work around this? or any other better way to reduce code duplicacy? Any help will be highly appreciated. Thanks!
Say I have a list of strings containing arguments and options, with argparse, I’m able to parse this list using the parse_args function into an object, as follows:
import argparse
extra_params = [‘—sum’, ‘7’, ‘-1’, ‘42’]
parser=argparse.ArgumentParser(description=“argparse docs example”)
parser.add_argument(‘integers’, metavar=‘N’, type=int, nargs=‘+’,
help=‘an integer for the accumulator’)
parser.add_argument(‘—sum’, dest=‘accumulate’, action=‘store_const’,
const=sum, default=max,
help=‘sum the integers (default: find the max)’)
parsed_object=parser.parse_args(extra_params)
Here, argparse has parsed a provided iterable of strings. Can one use click to also parse a provided iterable of strings?
I’ve searched through the API documentation for click and it appears that there’s a parse_args function within the *Command set of classes but don’t see anything in the docs around how I can do this. I’ve tried instantiating BaseCommand as well as Command but not sure how to get parse_args working without a correct context.
For broader context, this question is a result of having built a launcher application that end users use as a scaffold to launch their own applications. Here, the launcher consumes a number of arguments for which click decorators work perfectly. Unknown arguments can be handled as shown in the documentation here. This launcher then calls an end-user provided callable with these unparsed parameters. Click leaves unparsed parameters as a tuple of strings. How would the end-user, in this situation, be able to use Click to parse the argument's they're interested in? Here's a snippet to illustrate the issue:
import click
from typing import Tuple
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
child_function(unprocessed_args)
def child_function(unprocessed_args: Tuple[str, ...]) -> None:
# How do I get Click to parse the provided args for me?
pass
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
Running this from the command line:
python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')
For the calling function not knowing anything about parameters for the child function, you can try this:
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])
click.get_current_context().invoke(child_function, **unprocessed_args)
#click.command(context_settings={"ignore_unknown_options": True})
#click.option('-p', '--prog-arg')
def child_function(prog_arg: str, **kwargs) -> None:
# How do I get Click to parse the provided args for me?
print(f"Child function passed: {prog_arg}")
# all remaining unknown options are in **kwargs
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
However, note that:
unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])
This assumes you can only have one value per option. The alternative is to call your script by passing in options like below, splitting the string on = and doing whatever pre-formatting you deem necessary.
--prog-arg=<Your-desired-values>
Try something like this:
import click
#click.command()
#click.option('--count', default=1, help='number of greetings')
#click.option('--test', default='test_was_not_provided', help='test option')
#click.argument('name')
def hello(*args, **kwargs):
click.echo(f"Hello World! {kwargs['name']} {kwargs['count']}")
if __name__ == '__main__':
hello()
run with something like: python main.py haha --test this_is_a_test --count=40
Reviewing the comments and my ensuing edit, made me think that simply applying the click decorators to the child function may work. Indeed it seems to but I don't entirely know why.
import click
from typing import Tuple
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
child_function(unprocessed_args)
#click.command()
#click.option('--prog-arg')
def child_function(prog_arg: str) -> None:
# How do I get Click to parse the provided args for me?
print(f"Child function passed: {prog_arg}")
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')
Child function passed: 10
I'm trying to access to the "resources" folder with the ArgumentParser.
This code and the "resources" folder are in the same folder...
Just to try to run the code, I've put a print function in the predict function. However this error occurs:
predict.py: error: the following arguments are required: resources_path
How can I fix it?
from argparse import ArgumentParser
def parse_args():
parser = ArgumentParser()
parser.add_argument("resources_path", help='/resources')
return parser.parse_args()
def predict(resources_path):
print(resources_path)
pass
if __name__ == '__main__':
args = parse_args()
predict(args.resources_path)
I am guessing from your error message that you are trying to call your program like this:
python predict.py
The argument parser by default gets the arguments from sys.argv, i.e. the command line. You'll have to pass it yourself like this:
python predict.py resources
It's possible that you want the resources argument to default to ./resources if you don't pass anything. (And I further assume you want ./resources, not /resources.) There's a keyword argument for that:
....
parser.add_argument('resources_path', default='./resources')
...
I'm developing a UI in python for maya, and I'm trying to do a function that performs an action when a frameLayout expands, in this case I have several "frameLayout" objects and I would like to use a single function "fl_expand", instead of one for each object
def fl_expand(*args):
print args
with frameLayout("layout1", collapsable=True, expandCommand=fl_expand):
...
with frameLayout("layout2", collapsable=True, expandCommand=fl_expand):
...
but I don't know how to pass the instance name as argument to the function, I tried:
with frameLayout("layout1", collapsable=True, expandCommand=fl_expand("layout1")):
...
But of course I get this error:
# Error: TypeError: file /usr/autodesk/maya2018/lib/python2.7/site-packages/pymel/internal/pmcmds.py line 134: Invalid arguments for flag 'ec'. Expected string or function, got NoneType #
Currently, you have something like that:
def fl_expand(*args):
print(args)
def frameLayout(name, collapsable, expandCommand):
print("frameLayout: " + name)
expandCommand('foo')
frameLayout("layout1", collapsable=True, expandCommand=fl_expand)
What you want is to call the fl_expand function with the first argument already filled with the name of the layout. To do that, you can use a partiel function. See the documentation for functools.partial.
You can try:
import functools
frameLayout("layout1", collapsable=True, expandCommand=functools.partial(fl_expand, "layout1"))
Of course, it could be laborious if you have a lot of calls like that. You can also define your own frameLayout function:
def myFrameLayout(name, collapsable, expandCommand):
cmd = functools.partial(fl_expand, name)
return frameLayout(name, collapsable, cmd)
myFrameLayout("layout2", collapsable=True, expandCommand=fl_expand)