I am trying to introduce some automation to a script I'm writing, and I'm having some trouble with calling a function that has parameters from another module. Here's the scenario:
I have two modules: test.py and Strategies.py. I have code that generates a list of all the functions in Strategies.py. From that list, I am using getattr to execute each function.
What I'm having problems with is that some of my functions have parameters. I am getting the following error with a function that has an 'x' argument:
TypeError: buy_test_function() missing 1 required positional argument: 'x'
To make this as clear as possible, here's the relevant code:
call_method = strategy_names[0][y]
call_method = getattr(Strategies, call_method)()
I know the first line above is working fine. I also know that it's the empty parentheses at the end of the second line that's causing the problem. The magic I need is finding a way to dynamically read each function's required arguments and execute the function with the necessary arguments in the parentheses.
I've tried to use inspect.signature(), but it keeps telling me the object is not callable.
I have to believe Python has an elegant solution to this, but I've had little luck on Google. Any assistance is greatly appreciated.
Thank you!
Assuming the functions in Strategies are not class methods, and you've annotated the function types in the signatures, you can construct default instances of the class types you specified for the arguments and pass them in as params:
from inspect import signature
call_method = strategy_names[0][y]
call_method = getattr(Strategies, call_method)
sig = signature(call_method)
call_method = getattr(Strategies, call_method)(*[param.annotation() for param in sig.parameters.values()])
See for reference:
>>> def test(x:int):
... return x*2
>>> sig = signature(test)
>>> sig.parameters['x'].annotation
<class 'int'>
>>> sig.parameters['x'].annotation()
0
>>> test(*[param.annotation() for param in sig.parameters.values()])
0
I will also note that if you can define the values you want to use in your call methods ahead of time for a given method and your function names in Strategies are unique, you can prebuild a dictionary that maps the function name to the args you want to use:
args = {'test':[1]}
test(*args[test.__name__])
Related
For this function
def eat_dog(name, should_digest=True):
print "ate dog named %s. Digested, too? %" % (name, str(should_digest))
I want to, external to the function, read its arguments and any default values attached. So for this specific example, I want to know that name has no default value (i.e. that it is a required argument) and that True is the default value for should_digest.
I'm aware of inspect.getargspec(), which does give me information about arguments and default values, but I see no connection between the two:
ArgSpec(args=['name', 'should_digest'], varargs=None, keywords=None, defaults=(True,))
From this output how can I tell that True (in the defaults tuple) is the default value for should_digest?
Additionally, I'm aware of the "ask for forgiveness" model of approaching a problem, but unfortunately output from that error won't tell me the name of the missing argument:
>>> eat_dog()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: eat_dog() takes at least 1 argument (0 given)
To give context (why I want to do this), I'm exposing functions in a module over a JSON API. If the caller omits certain function arguments, I want to return a specific error that names the specific function argument that was omitted. If a client omits an argument, but there's a default provided in the function signature, I want to use that default.
Python3.x
In a python3.x world, you should probably use a Signature object:
import inspect
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default
for k, v in signature.parameters.items()
if v.default is not inspect.Parameter.empty
}
Python2.x (old answer)
The args/defaults can be combined as:
import inspect
a = inspect.getargspec(eat_dog)
zip(a.args[-len(a.defaults):],a.defaults)
Here a.args[-len(a.defaults):] are the arguments with defaults values and obviously a.defaults are the corresponding default values.
You could even pass the output of zip to the dict constructor and create a mapping suitable for keyword unpacking.
looking at the docs, this solution will only work on python2.6 or newer since I assume that inspect.getargspec returns a named tuple. Earlier versions returned a regular tuple, but it would be very easy to modify accordingly. Here's a version which works with older (and newer) versions:
import inspect
def get_default_args(func):
"""
returns a dictionary of arg_name:default_values for the input function
"""
args, varargs, keywords, defaults = inspect.getargspec(func)
return dict(zip(args[-len(defaults):], defaults))
Come to think of it:
return dict(zip(reversed(args), reversed(defaults)))
would also work and may be more intuitive to some people.
Depending on exactly what you need, you might not need the inspect module since you can check the __defaults__ attribute of the function:
>>> eat_dog.__defaults__
(True,)
>>> eat_dog.__code__.co_argcount
2
>>> eat_dog.__code__.co_varnames
('name', 'should_digest')
>>>
>>> eat_dog.__kwdefaults__
>>> eat_dog.__code__.co_kwonlyargcount
0
You can use inspect module with its getargspec function:
inspect.getargspec(func)
Get the names and default values of a Python function’s arguments. A tuple of four things is returned: (args, varargs, keywords, defaults). args is a list of the argument names (it may contain nested lists). varargs and keywords are the names of the * and ** arguments or None. defaults is a tuple of default argument values or None if there are no default arguments; if this tuple has n elements, they correspond to the last n elements listed in args.
See mgilson's answer for exact code on how to retrieve argument names and their default values.
To those looking for a version to grab a specific default parameter with mgilson's answer.
value = signature(my_func).parameters['param_name'].default
Here's a full working version, done in Python 3.8.2
from inspect import signature
def my_func(a, b, c, param_name='apple'):
pass
value = signature(my_func).parameters['param_name'].default
print(value == 'apple') # True
to take care of keyword-only args (and because defaults and kwonlydefaults can be None):
spec = inspect.getfullargspec(func)
defaults = dict(zip(spec.args[::-1], (spec.defaults or ())[::-1]))
defaults.update(spec.kwonlydefaults or {})
You can get this via some of the __dunder__ vars as mentioned by other posts. Putting that into a simple helper function can get you a dictionary of default values.
.__code__.co_varnames: A tuple of all input variables
.__defaults__: A tuple of the default values
It is worth noting that this tuple only incudes the default provided variables which must always be positioned last in the function arguments
You can use these two items to match the last n variables in the .__code__.co_varnames with all the items in the .__defaults__
EDIT Thanks to #griloHBG - Added if statement to prevent exceptions when no defaults are specified.
def my_fn(a, b=2, c='a'):
pass
def get_defaults(fn):
if fn.__defaults__==None:
return {}
return dict(zip(
fn.__code__.co_varnames[-len(fn.__defaults__):],
fn.__defaults__
))
print(get_defaults(my_fn))
Should give:
{'b': 2, 'c': 'a'}
In python, all the arguments with default value come after the arguments without default value. So the mapping should start from the end till you exhaust the default value list. Hence the logic:
dict(zip(reversed(args), reversed(defaults)))
gives the correctly mapped defaults.
I'm attempting to thread a function call in my Python catastr^H^H^H^H^H^Hreation, and I've read up on how to use the threading.Thread() call. My function takes a simple string argument, so theoretically it should be as easy as:
thread = threading.Thread(target = my_func, args = (string_var, ))
bearing in mind that the args() needs to be a tuple. Got it. However, it appears as though I'm still doing something wrong because I continually get the barffage from Python:
TypeError: my_func() takes 1 positional argument but 2 were given
I'm a bit stumped here. Any guidance?
Thanks!
please provide some code for us to help you.
But before you do your post could be a possible duplicate of this post.
Seems the issue is that because it's a method (thanks gribvirus74 for the idea) and I'm attempting to thread it, it won't inherit the self. And that appears to be the issue. I moved the function outside of the class and called it with the Thread(). Works fine now.
If it's a method, then you can write the following code (assuming the class name is SomeClass and it has a method called foo with one argument):
x = SomeClass()
thread = threading.Thread(target=SomeClass.foo, args=(x, 'your method argument'))
I'm currently learning curses in python, and I found this piece of code online that is confusing me.
import curses
def draw_menu(stdscr):
# do stuff
# if you want more code just let me know
def main():
curses.wrapper(draw_menu)
if __name__ == "__main__":
main()
When I run this I don't get the expected missing 1 required positional argument error, since there is no parameter being passed in the curses.wrapper(draw_menu) line. Is this a curses thing? Any help is greatly appreciated.
A function is a datatype, just as much as strings, integers, and so on.
def my_function(txt):
print(txt)
here type(my_function) # => <class 'function'>
You invoke the code inside the function when you call it with parenthesis : my_function('hello') # => prints hello
Until then you can perfectly pass a function as an argument to another function.
And that last one can call the one you passed giving it some parameters.
Like in your case, I'd guess that curses.wrapper() creates a screen interface that it passes as argument your draw_menu() function.
And you can probably use that screen object to build your curse app.
See this : Python function as a function argument?
There's a big difference between curses.wrapper(draw_menu) and curses.wrapper(draw_menu()). curses.wrapper(draw_menu) calls curses.wrapper and passes the function draw_menu into it as an argument. In contrast, curses.wrapper(draw_menu()) would call draw_menu and pass its return value into curses.wrapper.
curses.wrapper will call the function you pass it. From that link:
Initialize curses and call another callable object, func, which should be the rest of your curses-using application.
E.g., it will call draw_menu when curses is completely initialized.
Here is the signature for curses.wrapper from here.
curses.wrapper(func, /, *args, **kwargs)
It says that you need to give curses.wrapper a function reference argument followed by zero or more arguments and keyword arguments. Your code satisfies those requirements.
Python allows function signatures like this to enable developers a lot of flexibility regarding what can be passed in by the caller.
I'm trying to use the ast module in Python to parse input code, but am struggling with a lot of the syntax of how to do so. For instance, I have the following code as a testing environment:
import ast
class NodeVisitor(ast.NodeVisitor):
def visit_Call(self, node):
for each in node.args:
print(ast.literal_eval(each))
self.generic_visit(node)
line = "circuit = QubitCircuit(3, True)"
tree = ast.parse(line)
print("VISITOR")
visitor = NodeVisitor()
visitor.visit(tree)
Output:
VISITOR
3
True
In this instance, and please correct me if I'm wrong, the visit_Call will be used if it's a function call? So I can get each argument, however there's no guarantee it will work like this as there are different arguments available to be provided. I understand that node.args is providing my arguments, but I'm not sure how to do things with them?
I guess what I'm asking is how do I check what the arguments are and do different things with them? I'd like to check, perhaps, that the first argument is an Int, and if so, run processInt(parameter) as an example.
The value each in your loop in the method will be assigned to the AST node for each of the arguments in each function call you visit. There are lots of different types of AST nodes, so by checking which kind you have, you may be able to learn things about the argument being passed in.
Note however that the AST is about syntax, not values. So if the function call was foo(bar), it's just going to tell you that the argument is a variable named bar, not what the value of that variable is (which it does not know). If the function call was foo(bar(baz)), it's going to show you that the argument is another function call. If you only need to handle calls with literals as their arguments, then you're probably going to be OK, you'll just look instances of AST.Num and similar.
If you want to check if the first argument is a number and process it if it is, you can do something like:
def visit_Call(self, node):
first_arg = node.args[0]
if isinstance(first_arg, ast.Num):
processInt(first_arg.n)
else:
pass # Do you want to do something on a bad argument? Raise an exception maybe?
I have already found various answers to this question (eg. lambda function acessing outside variable) and all point to the same hack, namely (eg.) lambda n=i : n*2 with i a variable in the external scope of lambda (hoping I'm not misusing the term scope). However, this is not working and given that all answers I found are generally from couple of years ago, I thought that maybe this has been deprecated and only worked with older versions of python. Does anybody have an idea or suggestion on how to solve this?
SORRY, forgot the MWE
from inspect import getargspec
params = ['a','b']
def test(*args):
return args[0]*args[1]
func = lambda p=params : test(p)
I expected the signature of func to be ['a','b'] but if I try
func(3,2)
I get a Type error (TypeError: <lambda>() takes at most 1 argument (2 given) )
and it's true signature (from getargspec(func)[0] ) is ['p']
In my real code the thing is more complicated. Shortly:
def fit(self, **kwargs):
settings = self.synch()
freepars = self.loglike.get_args()
func = lambda p=freeparams : self.loglike(p)
minuit = Minuit(func,**settings)
I need lambda because it's the only way I could think to create inplace a function object depending on a non-hardcoded list of variables (extracted via a method get_params() of the instance self.loglike). So func has to have the correct signature, to match the info inside the dict settings
The inspector gives ['p'] as argument of func, not the list of parameters which should go in loglike. Hope you can easily spot my mistake. Thank you
There's no way to do exactly what you want. The syntax you're trying to use to set the signature of the function you're creating doesn't do what you want. It instead sets a default value for the argument you've defined. Python's function syntax allows you to define a function that accepts an arbitrary number of arguments, but it doesn't let you define a function with argument names in a variable.
What you can do is accept *args (or **kwargs) and then do some processing on the value to match it up with a list of argument names. Here's an example where I turn positional arguments in a specific order into keyword arguments to be passed on to another function:
arg_names = ['a', 'b']
def foo(*args):
if len(args) != len(arg_names):
raise ValueError("wrong number of arguments passed to foo")
args_by_name = dict(zip(arg_names, args))
some_other_function(**args_by_name)
This example isn't terribly useful, but you could do more sophisticated processing on the args_by_name dict (e.g. combining it with another dict), which might be relevant to your actual use case.