Description
Say I have the following function, which makes a call to another function:
def f1(arg1, arg2, arg3):
f2(...)
The arguments of f1 and f2 are the same, or f2 might look like this:
def f2(**kwargs)
pass # whatever
The client code is going to define and call f1, and it is required that the signature of f1 explicitly defines all arguments, and thus no **kwargs is allowed for f1.
So, to make a call to f2 from inside f1 I have to do this:
def f1(arg1, arg2, arg3):
f2(arg1, arg2, arg3)
Question
Is there a way I can pass arguments to f2 without explicitly writing them? Ideally, I think it should look like this:
def f1(arg1, arg2, arg3):
kwargs = <Some Magic Here>
f2(**kwargs)
Any magic?
UPDATE
Possible Solution
Is there a way I can combine locals() and inspect.getargspec() to aggregate my **kwargs?
In General
Well, you can create kwargs as a dictionary of all the arguments that f2() accepts and pass it. Though I do not see any benefit from that, using -
def f1(arg1, arg2, arg3):
f2(arg1=arg1, arg2=arg2, arg3=arg3)
Looks fine to me , and would be easier than building the dictionary and calling it as **kwargs.
Anyway the way to do it is -
>>> def a(a,b,c):
... kwargs = {'a':a , 'b':b , 'c':c}
... d(**kwargs)
...
>>> def d(**kwargs):
... print(kwargs)
...
>>> a(1,2,3)
{'c': 3, 'a': 1, 'b': 2}
For your use case
The problem is that f1 is going to be defined by the client, the processing of argument is common for all, so I want to hide the processing details, so that the client passes all the arguments to the implementation. Furthermore, I want to ease the definition and automatically pass all arguments and not specify them explicitly.
locals() inside a function returns you the copy of the local variables in the function at that time as a dictionary. If as in your question if the definition of f1() and f2() are same you can use locals() , by calling it at the start of the function before any other code. Example -
>>> def a(a,b,c):
... lcl = locals()
... print(lcl)
... d(**lcl)
... e = 123
... print(locals())
...
>>> def d(**kwargs):
... print(kwargs)
...
>>> a(1,2,3)
{'c': 3, 'a': 1, 'b': 2}
{'c': 3, 'a': 1, 'b': 2}
{'c': 3, 'a': 1, 'e': 123, 'lcl': {...}, 'b': 2}
What you want here is passing the arguments of the parent function to an enclosing function.So fist I must say that you can not use local() because local is contain the local variables which is contain the argument and all local variables that you have define in your function buddy, but if you just want to get all the arguments when they are dynamic and like your example, I suggest to use *args in parents function :
def f1(*args):
But there is a point here, since you want to use **kwargs and it is for collecting the arbitrarily keyword arguments you need to initial names of your argument.
For example you can use a dict comprehension like following :
def f1(*args):
kwargs = {'arg{}'.format(i):j for i,j in enumerate(args,1)}
f2(**kwargs)
Also if you are sure that there is no local variable in your function buddy specially before defining the enclosing function you can use locals :
Example :
>>> globsl_a=8
>>> def fun(arg1,arg2,arg3):
... print locals()
...
>>> fun(5,[3,4],'text')
{'arg1': 5, 'arg2': [3, 4], 'arg3': 'text'}
>>>
Here is how I solved my problem based on Anand S Kumar's answer as well as Alex Martelli's answer to another question:
def f2():
kwargs = inspect.getouterframes(inspect.currentframe(), 2)[1][0].f_locals
print(kwargs)
def f1(arg1, arg2, arg3)
f2() # this should be called right at the first line
>>> f1(1, 2, 3)
{'arg1': 1, 'arg2': 2, 'arg3': 3}
Related
Hi I have a multiparameter function where only one parameter is missing in a kwargs and I want to be able to input the missing parameter without knowing what parameter it is:
def function(a,b,c): #multiparameter function
print('a=',a)
print('b=',b)
print('c=',c)
kwargs={'a':1,'b':2} #only the value for c is missing
If I run
function(3,**kwargs) it interprets that a=3 but I want it to interpret that c=3
Alternatively I could find out the name of the missing variable but I cannot manage to feed it correctly to the function. I tried to do:
variable='c'
function(variable=3,**kwargs)
But it returns the error function() got an unexpected keyword argument 'variable'
If you can't modify the definition of function, you can wrap it with functools.partial.
from functools import partial
def function(a,b,c): #multiparameter function
print('a=',a)
print('b=',b)
print('c=',c)
function_caller = partial(function, 1, 2, 3)
Now, any keyword arguments to function_caller will override the values specified in the call to partial:
function_caller() # function(1,2,3)
function_caller(a=5, b=4) # function(5, 4, 3)
kwargs = {'a': 5, 'b': 4}
function_caller(**kwargs) # Same as previous call
If you have variable that contains the name of a parameter, you can easily add that to kwargs, though you can't use it directly as a keyword argument.
variable = 'c'
kwargs = {'a': 5, 'b': 4, variable: 3} # {'a': 5, 'b': 4, 'c': 3}
function_caller, though, does not accept any positional arguments.
First, you should probably read about *args and **kwargs
You can force you parameters to be keyword parameters with * and each of your keyword param can have a default sentinel value to help you spot it.
def function(*,a = None,b = None,c = None)
print('a=',a)
print('b=',b)
print('c=',c)
If you want to input something on the fly you could do:
def function(input, *,a = None,b = None,c = None)
print('a=',a or input)
print('b=',b or input)
print('c=',c or input)
If you can take the parameter as a variable, then this request:
kwargs={'a':1,'b':2} #only the value for c is missing
variable='c'
function(variable=3,**kwargs)
Can work if you just add the variable to the kwargs dictionary:
kwargs[variable] = 3
function(**kwargs)
I am looking for the best way to combine a function with a dictionary that contains more items than the function's inputs
basic **kwarg unpacking fails in this case:
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
foo(**d)
--> TypeError: foo() got an unexpected keyword argument 'c'
After some research I came up with the following approach:
import inspect
# utilities
def get_input_names(function):
'''get arguments names from function'''
return inspect.getargspec(function)[0]
def filter_dict(dict_,keys):
return {k:dict_[k] for k in keys}
def combine(function,dict_):
'''combine a function with a dictionary that may contain more items than the function's inputs '''
filtered_dict = filter_dict(dict_,get_input_names(function))
return function(**filtered_dict)
# examples
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
print combine(foo,d)
--> 3
My question is: is this a good way of dealing with this problem, or is there a better practice or is there a mechanism in the language that I'm missing perhaps?
How about making a decorator that would filter allowed keyword arguments only:
import inspect
def get_input_names(function):
'''get arguments names from function'''
return inspect.getargspec(function)[0]
def filter_dict(dict_,keys):
return {k:dict_[k] for k in keys}
def filter_kwargs(func):
def func_wrapper(**kwargs):
return func(**filter_dict(kwargs, get_input_names(func)))
return func_wrapper
#filter_kwargs
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
print(foo(**d))
What is nice about this decorator is that it is generic and reusable. And you would not need to change the way you call and use your target functions.
All of these answers are wrong.
It is not possible to do what you are asking, because the function might be declared like this:
def foo(**kwargs):
a = kwargs.pop('a')
b = kwargs.pop('b')
if kwargs:
raise TypeError('Unexpected arguments: %r' % kwargs)
Now, why on earth would anyone write that?
Because they don't know all of the arguments ahead of time. Here's a more realistic case:
def __init__(self, **kwargs):
for name in self.known_arguments():
value = kwargs.pop(name, default)
self.do_something(name, value)
super().__init__(**kwargs) # The superclass does not take any arguments
And here is some real-world code which actually does this.
You might ask why we need the last line. Why pass arguments to a superclass that doesn't take any? Cooperative multiple inheritance. If my class gets an argument it does not recognize, it should not swallow that argument, nor should it error out. It should pass the argument up the chain so that another class I might not know about can handle it. And if nobody handles it, then object.__init__() will provide an appropriate error message. Unfortunately, the other answers will not handle that gracefully. They will see **kwargs and either pass no arguments or pass all of them, which are both incorrect.
The bottom line: There is no general way to discover whether a function call is legal without actually making that function call. inspect is a crude approximation, and entirely falls apart in the face of variadic functions. Variadic does not mean "pass whatever you like"; it means "the rules are too complex to express in a signature." As a result, while it may be possible in many cases to do what you're trying to do, there will always be situations where there is no correct answer.
Your problem lies with the way you defined your function, it should be defined like this -
def foo(**kwargs):
And then inside the function you can iterate over the number of arguments sent to the function like so -
if kwargs is not None:
for key, value in kwargs.iteritems():
do something
You can find more info about using **kwargs in this post -
http://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/
You can also use a decorator function to filter out those keyword arguments that are not allowed in you function. Of you use the signature function new in 3.3 to return your function Signature
from inspect import signature
from functools import wraps
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
sig = signature(func)
result = func(*[kwargs[param] for param in sig.parameters])
return result
return wrapper
From Python 3.0 you can use getargspec which is deprecated since version 3.0
import inspect
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
argspec = inspect.getargspec(func).args
result = func(*[kwargs[param] for param in argspec])
return result
return wrapper
To apply your decorate an existing function you need to pass your function as argument to your decorator:
Demo:
>>> def foo(a, b):
... return a + b
...
>>> foo = decorator(foo)
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> foo(**d)
3
To apply your decorator to a newly function simply use #
>>> #decorator
... def foo(a, b):
... return a + b
...
>>> foo(**d)
3
You can also define your function using arbitrary keywords arguments **kwargs.
>>> def foo(**kwargs):
... if 'a' in kwargs and 'b' in kwargs:
... return kwargs['a'] + kwargs['b']
...
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> foo(**d)
3
I would do something like this:
def combine(function, dictionary):
return function(**{key:value for key, value in dictionary.items()
if key in inspect.getargspec(function)[0]}
)
Use:
>>> def this(a, b, c=5):
... print(a, b, c)
...
>>> combine(this, {'a': 4, 'b': 6, 'c': 6, 'd': 8})
4 6 6
>>> combine(this, {'a': 6, 'b': 5, 'd': 8})
6 5 5
This is still modifying the original function, but you can create a kwargs bitbucket at the end of the argument list:
def foo(a, b, **kwargs):
return a + b
foo(**{
'a': 5,
'b': 8,
'c': '🐘'
}) # 13
Suppose I have a function with 10 args:
def foo(arg1,arg2,arg3,arg4.....):
Sometimes, I need to call it with only arg1 and another time arg1, arg4, or arg4 , arg7.
My program doesn't specify the type of the function call. Does python have a way to help me?
One way to do it is to make the parameters optional:
def foo(arg1=None,arg2=None,arg3=None...)
which can be called like this:
foo(arg1=1,arg3=2)
or like this:
a = {'arg1':1, 'arg3':2}
foo(**a)
If this list of parameters is spinning out of control you could simply use **kwargs to let your function take an optional number of (named) keyword arguments:
def foo(**kwargs):
print kwargs
params = {'arg1':1, 'arg2':2}
foo(**params) # Version 1
foo(arg1=3,arg2=4) # Version 2
Output:
{'arg1': 1, 'arg2': 2}
{'arg1': 3, 'arg2': 4}
Note: You can use one asterisk (*) for an arbitrary number of arguments that will be wrapped up in a tuple.
Give your args a default value like None:
def foo(arg1=None, arg2=None, ..., arg10=None):
Now, when calling the function pass a dictionary of keyword arguments:
kwargs = {
'arg1': 'test',
'arg7': 'test2',
}
foo(**kwargs)
It's equivalent to:
foo('test', None, None, None, ..., 'test2', None, None, None)
Or, to be more specific:
foo(arg1='test', arg7='test2')
You could use keyword arguments
I understand that how * and ** operators generally work. In the following code [ taken from django's source ]
def curry(_curried_func, *args, **kwargs):
def _curried(*moreargs, **morekwargs):
return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))
return _curried
I get how the args + moreargs part work - the [non - keyword ] arguments passed initially to the function curry and the arguments passed to the curried function returned by curry are combined. What I don't get is how the **dict(kwargs, **morekwargs) works. Could someone please explain that bit?
dict(kwargs, **morekwargs) is a trick (that Guido dislikes) for combining 2 dictionaries into 1.
>>> d = {'foo':'bar'}
>>> kwargs = {'bar':'baz'}
>>> dict(d,**kwargs)
{'foo': 'bar', 'bar': 'baz'}
So, the curried function takes all kwargs passed to curry and adds them to the additional kwargs passed to the _curried function to create a super dictionary. that super dictionary gets unpacked and sent on to the _curried_func.
It effectively builds a union of two dicts, kwargs and morekwargs. Example:
>>> kwargs = {"ham": 1, "spam": 2}
>>> morekwargs = {"spam": 3, "eggs": 2}
>>> dict(kwargs, **morekwargs)
{'eggs': 2, 'ham': 1, 'spam': 3}
This works because the dict constructor takes keyword arguments, so it's the same as
>>> dict(kwargs, spam=3, eggs=2)
{'eggs': 2, 'ham': 1, 'spam': 3}
Finally, the resulting dict is fed as keyword arguments to _curried_func by means of **.
It seems that many aspects of python are just duplicates of functionality. Is there some difference beyond the redundancy I am seeing in kwargs and dict within Python?
There is a difference in argument unpacking (where many people use kwargs) and passing dict as one of the arguments:
Using argument unpacking:
# Prepare function
def test(**kwargs):
return kwargs
# Invoke function
>>> test(a=10, b=20)
{'a':10,'b':20}
Passing a dict as an argument:
# Prepare function
def test(my_dict):
return my_dict
# Invoke function
>>> test(dict(a=10, b=20))
{'a':10,'b':20}
The differences are mostly:
readability (you can simply pass keyword arguments even if they weren't explicitly defined),
flexibility (you can support some keyword arguments explicitly and the rest using **kwargs),
argument unpacking helps you avoid unexpected changes to the object "containing" the arguments (which is less important, as Python in general assumes developers know what they are doing, which is a different topic),
It is right that in most cases you can just interchange dicts and **kwargs.
For example:
my_dict = {'a': 5, 'b': 6}
def printer1(adict):
return adict
def printer2(**kwargs):
return kwargs
#evaluate:
>>> printer1(my_dict)
{'a': 5, 'b': 6}
>>> printer2(**my_dict)
{'a': 5, 'b': 6}
However with kwargs you have more flexibility if you combine it with other arguments:
def printer3(a, b=0, **kwargs):
return a,b,kwargs
#evaluate
>>> printer3(**my_dict)
(5, 6, {})