I would like to use functools.partial to reduce the number of arguments in one of my functions. Here's the catch: one or more kwargs may be functions themselves. Here's what I mean:
from functools import partial
def B(alpha, x, y):
return alpha(x)*y
def alpha(x):
return x+1
g = partial(B, alpha=alpha, y=2)
print(g(5))
This throws an error:
TypeError: B() got multiple values for argument 'alpha'
Can partial handle functions as provided arguments? If not is there a workaround or something more generic than partial?
partial itself doesn't know that a given positional argument should be assigned to x just because you specified a keyword argument for alpha. If you want alpha to be particular function, pass that function as a positional argument to partial.
>>> g = partial(B, alpha, y=2)
>>> g(5)
12
g is equivalent to
def g(x):
return alpha(x) * 2 # == (x + 1) * 2
Alternately, you can use your original definition of g, but be sure to pass 5 as a keyword argument as well, avoiding any additional positional arguments.
>>> g = partial(B, alpha=alpha, y=2)
>>> g(x=5)
12
This works because between g and partial, you have provided keyword arguments for all required parameters, eliminating the need for any positional arguments.
Related
suppose I define a function of an unpackable array (e.g. list)
def f(params):
a,b,c = params
return a+b**2+c**2
is there a way to integrate over only the first parameter? I tried
import scipy.integrate.quad as quad
def integrate(b,c):
return quad(f,0,1,args=(b,c))
but it returns an error because the function f only takes one positional argument. I guess I have to unpack it in the integration step, but unsure how.
Thanks!
If these are the only two use cases for using this function you can do something like the following:
def f(*args):
if len(args) == 1:
a,b,c = args[0]
else:
a, b,c = args
return a+b**2+c**2
This assumes that if you pass one argument to the function in can be unpacked and if you pass more than one you pass a, b, c directly.
In a more generic setting, you can define different behavior of a function depending on the input type using functools.singledispatch. In this case this would look as follows:
from functools import singledispatch
#singledispatch
def f(arg1, arg2=None, arg3=None):
a, b, c = arg1
return a+b**2+c**2
#f.register(int)
#f.register(float)
def _(arg1, arg2, arg3):
return arg1+arg2**2+arg3**2
The advantage of this would be that you can extend it for different input types if that is needed.
As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'
What's the best way to specify the positions of the arguments that one wishes to evaluate?
EDIT
Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)
END EDIT
For example, consider:
def f(x,y,z):
return x + 2*y + 3*z
Then, using
from functools import partial
both
partial(f,4)(5,6)
and
partial(f,4,5)(6)
give 32.
But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?
Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:
partial_dict(f,{0:4,2:6})(5)
No, partial is not designed to freeze positional arguments at non-sequential positions.
To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:
def partial_positionals(func, positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(positionals[i] if i in positionals else next(arg)
for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, {0: 4, 2: 6})(5))
outputs:
32
Simply use keyword arguments. Using your definition of f above,
>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40
Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:
>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'
But continuing to use keyword arguments is:
>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40
When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling
h = partial(1, z=10)
f(4)
is roughly equivalent to writing
args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)
As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation
For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:
SKIP = object()
def partial_positionals(func, *positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
**{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, 4, SKIP, 6)(5))
outputs:
32
I have the following function signature:
# my signature
def myfunction(x, s, c=None, d=None):
# some irrelevant function body
I need the number of positional arguments. How can I return the number (2) of positional arguments (x & s). The number of keyword arguments is not relevant.
You can get the number of all arguments (using f.__code__.co_argcount), the number of keyword arguments (using f.__defaults__), and subtract the latter from the first:
def myfunction(x, s, c=None, d=None):
pass
all_args = myfunction.__code__.co_argcount
if myfunction.__defaults__ is not None: # in case there are no kwargs
kwargs = len(myfunction.__defaults__)
else:
kwargs = 0
print(all_args - kwargs)
Output:
2
From the Docs:
__defaults__: A tuple containing default argument values for those arguments that have defaults, or None if no arguments have a default value.
and:
co_argcount is the number of positional arguments (including arguments with default values);
You could do that by using the inspect module like,
>>> import inspect
>>> def foo(a, b, *, c=None):
... print(a, b, c)
...
>>> sig = inspect.signature(foo)
>>> sum(1 for param in sig.parameters.values() if param.kind == param.POSITIONAL_OR_KEYWORD)
2
The caveat is,
Python has no explicit syntax for defining positional-only parameters,
but many built-in and extension module functions (especially those
that accept only one or two parameters) accept them.
Suppose I have the following function
def f(x,y,**kwargs):
if 'z' in kwargs:
z = kwargs['z']
else:
z = 0
print(x + y + z)
which takes two arguments and an optional keyword argument. I now want to get a function g that works just as f but for which the value of z is predetermined. Hence, I could do the following
def g(x,y):
z = 3
f(x,y, z = 3)
But what can I do if I do not know the number of non-keyword arguments that f takes. I can get the list of these arguments by
args = inspect.getargspec(f)[0]
But, if I now define g as
g(args):
z = 3
f(args, z=z)
this of course does not work as only one mandatory argument is passed to f. How do I get around this? That is, if I have a function that takes keyword arguments, how do I define a second function exactly the same expect that the keyword arguments take predeterminde values?
You have a few options here:
Define g with varargs:
def g(*args):
return f(*args, z=3)
Or, if you need keyword arguments as well:
def g(*args, **kwargs):
kwargs['z'] = 3
return f(*args, **kwargs)
Use functools.partial:
import functools
g = functools.partial(f, z=3)
See also this related question: Python Argument Binders.
You can use functools.partial to achieve this
import functools
f = functools.partial(f, z=2)
# the following example is the usage of partial function f
x = f(1, 2)
y = f(1, 2, k=3)
z = f(1, 2, z=4)
I am using the partial method from the functools module to map a function over a range of values:
def basic_rule(p,b,vx=1,**kwargs):
return (p / b) if vx != 0 else 0
def rule5(func,**kwargs):
vals = map(functools.partial(func,**kwargs), range(1,kwargs['b']+1))
return [x for i,x in enumerate(vals[:-1]) if x >= vals[i+1]] == []
rule5(basic_rule,p=100,b=10000)
Here is the error I get on line 5:
----> return map(functools.partial(func,**kwargs), range(1,kwargs['b']+1))
TypeError: basic_rule() got multiple values for keyword argument 'p'
It looks like functools.partial is trying to assign the range to the argument p, even though I have already assigned a value to it. I'm trying to assign the range to the value of vx. Any idea how I can make that happen?
EDIT: Added a little bit of extra context to the code. Essentially what I'd like test 5 to do is ensure that the result of the function given to it increases as vt goes up, so that `func(vt=1) < func(vt=2)... < func(vt=n).
functools.partial generates a partial that stores the arguments receiveids in 2 properties:
arguments stores positional arguments
keywords stores all keyword-based arguments
So the partial can call original function exactly as was intended. In other words, when you call the resulting partial with one argument (let's say, 1) it would be the same as:
original_func(1, **kwargs)
As your kwargs contains the first argument - and you're passing the "1" as a positional argument - you get the error.
I'm not sure if it's gonna work in this particular case, but one solution could be use inspect.getargspec to extract arguments from kwargs that can be passed as positional arguments. In this case, the rule5 function would be similar to:
def rule5(func, **kwargs):
# let's save 'b' argument because we'll need it in the range call
b = kwargs['b']
original_args = inspect.getargspec(func).args
# extract from kwargs all arguments that can be passed as positional
new_args = [kwargs.pop(key) for key in original_args if key in kwargs]
# construct the partial passing as much positional arguments as possible
fn = functools.partial(func, *new_args, **kwargs)
# now map will pass the range result as the next positional argument
vals = map(fn, range(1, b+1))
return [x for i,x in enumerate(vals[:-1]) if x >= vals[i+1]] == []