As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'
What's the best way to specify the positions of the arguments that one wishes to evaluate?
EDIT
Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)
END EDIT
For example, consider:
def f(x,y,z):
return x + 2*y + 3*z
Then, using
from functools import partial
both
partial(f,4)(5,6)
and
partial(f,4,5)(6)
give 32.
But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?
Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:
partial_dict(f,{0:4,2:6})(5)
No, partial is not designed to freeze positional arguments at non-sequential positions.
To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:
def partial_positionals(func, positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(positionals[i] if i in positionals else next(arg)
for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, {0: 4, 2: 6})(5))
outputs:
32
Simply use keyword arguments. Using your definition of f above,
>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40
Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:
>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'
But continuing to use keyword arguments is:
>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40
When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling
h = partial(1, z=10)
f(4)
is roughly equivalent to writing
args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)
As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation
For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:
SKIP = object()
def partial_positionals(func, *positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
**{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, 4, SKIP, 6)(5))
outputs:
32
Related
I would like to use functools.partial to reduce the number of arguments in one of my functions. Here's the catch: one or more kwargs may be functions themselves. Here's what I mean:
from functools import partial
def B(alpha, x, y):
return alpha(x)*y
def alpha(x):
return x+1
g = partial(B, alpha=alpha, y=2)
print(g(5))
This throws an error:
TypeError: B() got multiple values for argument 'alpha'
Can partial handle functions as provided arguments? If not is there a workaround or something more generic than partial?
partial itself doesn't know that a given positional argument should be assigned to x just because you specified a keyword argument for alpha. If you want alpha to be particular function, pass that function as a positional argument to partial.
>>> g = partial(B, alpha, y=2)
>>> g(5)
12
g is equivalent to
def g(x):
return alpha(x) * 2 # == (x + 1) * 2
Alternately, you can use your original definition of g, but be sure to pass 5 as a keyword argument as well, avoiding any additional positional arguments.
>>> g = partial(B, alpha=alpha, y=2)
>>> g(x=5)
12
This works because between g and partial, you have provided keyword arguments for all required parameters, eliminating the need for any positional arguments.
Suppose I have the following function
def f(x,y,**kwargs):
if 'z' in kwargs:
z = kwargs['z']
else:
z = 0
print(x + y + z)
which takes two arguments and an optional keyword argument. I now want to get a function g that works just as f but for which the value of z is predetermined. Hence, I could do the following
def g(x,y):
z = 3
f(x,y, z = 3)
But what can I do if I do not know the number of non-keyword arguments that f takes. I can get the list of these arguments by
args = inspect.getargspec(f)[0]
But, if I now define g as
g(args):
z = 3
f(args, z=z)
this of course does not work as only one mandatory argument is passed to f. How do I get around this? That is, if I have a function that takes keyword arguments, how do I define a second function exactly the same expect that the keyword arguments take predeterminde values?
You have a few options here:
Define g with varargs:
def g(*args):
return f(*args, z=3)
Or, if you need keyword arguments as well:
def g(*args, **kwargs):
kwargs['z'] = 3
return f(*args, **kwargs)
Use functools.partial:
import functools
g = functools.partial(f, z=3)
See also this related question: Python Argument Binders.
You can use functools.partial to achieve this
import functools
f = functools.partial(f, z=2)
# the following example is the usage of partial function f
x = f(1, 2)
y = f(1, 2, k=3)
z = f(1, 2, z=4)
Is there an equivalent to R's do.call in python?
do.call(what = 'sum', args = list(1:10)) #[1] 55
do.call(what = 'mean', args = list(1:10)) #[1] 5.5
?do.call
# Description
# do.call constructs and executes a function call from a name or a function and a list of arguments to be passed to it.
There is no built-in for this, but it is easy enough to construct an equivalent.
You can look up any object from the built-ins namespace using the __builtin__ (Python 2) or builtins (Python 3) modules then apply arbitrary arguments to that with *args and **kwargs syntax:
try:
# Python 2
import __builtin__ as builtins
except ImportError:
# Python 3
import builtins
def do_call(what, *args, **kwargs):
return getattr(builtins, what)(*args, **kwargs)
do_call('sum', range(1, 11))
Generally speaking, we don't do this in Python. If you must translate strings into function objects, it is generally preferred to build a custom dictionary:
functions = {
'sum': sum,
'mean': lambda v: sum(v) / len(v),
}
then look up functions from that dictionary instead:
functions['sum'](range(1, 11))
This lets you strictly control what names are available to dynamic code, preventing a user from making a nuisance of themselves by calling built-ins for their destructive or disruptive effects.
do.call is pretty much the equivalent of the splat operator in Python:
def mysum(a, b, c):
return sum([a, b, c])
# normal call:
mysum(1, 2, 3)
# with a list of arguments:
mysum(*[1, 2, 3])
Note that I’ve had to define my own sum function since Python’s sum already expects a list as an argument, so your original code would just be
sum(range(1, 11))
R has another peculiarity: do.call internally performs a function lookup of its first argument. This means that it finds the function even if it’s a character string rather than an actual function. The Python equivalent above doesn’t do this — see Martijn’s answer for a solution to this.
Goes similar to previous answer, but why so complicated?
def do_call(what, args=[], kwargs = {}):
return what(*args, **kwargs)
(Which is more elegant than my previously posted definition:)
def do_call(which, args=None, kwargs = None):
if args is None and kwargs is not None:
return which(**kwargs)
elif args is not None and kwargs is None:
return which(*args)
else:
return which(*args, **kwargs)
Python's sum is different than R's sum (1 argument a list expected vs.
arbitraily many arguments expected in R). So we define our own sum (mysum)
which behaves similarly to R's sum. In a similar way we define mymean.
def mysum(*args):
return sum(args)
def mymean(*args):
return sum(args)/len(args)
Now we can recreate your example in Python - as a reasonable 1:1 translation of the R function call.
do_call(what = mymean, args=[1, 2, 3])
## 2.0
do_call(what = mysum, args=[1, 2, 3])
## 6
For functions with argument names, we use a dict for kwargs, where the parameter
names are keys of the dictionary (as strings) and their values the values.
def myfunc(a, b, c):
return a + b + c
do_call(what = myfunc, kwargs={"a": 1, "b": 2, "c": 3})
## 6
# we can even mix named and unnamed parts
do_call(what = myfunc, args = [1, 2], kwargs={"c": 3})
## 6
I am using the partial method from the functools module to map a function over a range of values:
def basic_rule(p,b,vx=1,**kwargs):
return (p / b) if vx != 0 else 0
def rule5(func,**kwargs):
vals = map(functools.partial(func,**kwargs), range(1,kwargs['b']+1))
return [x for i,x in enumerate(vals[:-1]) if x >= vals[i+1]] == []
rule5(basic_rule,p=100,b=10000)
Here is the error I get on line 5:
----> return map(functools.partial(func,**kwargs), range(1,kwargs['b']+1))
TypeError: basic_rule() got multiple values for keyword argument 'p'
It looks like functools.partial is trying to assign the range to the argument p, even though I have already assigned a value to it. I'm trying to assign the range to the value of vx. Any idea how I can make that happen?
EDIT: Added a little bit of extra context to the code. Essentially what I'd like test 5 to do is ensure that the result of the function given to it increases as vt goes up, so that `func(vt=1) < func(vt=2)... < func(vt=n).
functools.partial generates a partial that stores the arguments receiveids in 2 properties:
arguments stores positional arguments
keywords stores all keyword-based arguments
So the partial can call original function exactly as was intended. In other words, when you call the resulting partial with one argument (let's say, 1) it would be the same as:
original_func(1, **kwargs)
As your kwargs contains the first argument - and you're passing the "1" as a positional argument - you get the error.
I'm not sure if it's gonna work in this particular case, but one solution could be use inspect.getargspec to extract arguments from kwargs that can be passed as positional arguments. In this case, the rule5 function would be similar to:
def rule5(func, **kwargs):
# let's save 'b' argument because we'll need it in the range call
b = kwargs['b']
original_args = inspect.getargspec(func).args
# extract from kwargs all arguments that can be passed as positional
new_args = [kwargs.pop(key) for key in original_args if key in kwargs]
# construct the partial passing as much positional arguments as possible
fn = functools.partial(func, *new_args, **kwargs)
# now map will pass the range result as the next positional argument
vals = map(fn, range(1, b+1))
return [x for i,x in enumerate(vals[:-1]) if x >= vals[i+1]] == []
Is it possible to assign a function to a variable with modified default arguments?
To make it more concrete, I'll give an example.
The following obviously doesn't work in the current form and is only meant to show what I need:
def power(a, pow=2):
ret = 1
for _ in range(pow):
ret *= a
return ret
cube = power(pow=3)
And the result of cube(5) should be 125.
functools.partial to the rescue:
Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords.
from functools import partial
cube = partial(power, pow=3)
Demo:
>>> from functools import partial
>>>
>>> def power(a, pow=2):
... ret = 1
... for _ in range(pow):
... ret *= a
... return ret
...
>>> cube = partial(power, pow=3)
>>>
>>> cube(5)
125
The answer using partial is good, using the standard library, but I think it's worth mentioning that the following approach is equivalent:
def cube(a):
return power(a, pow=3)
Even though this doesn't seem like assignment because there isn't a =, it is doing much the same thing (binding a name to a function object). I think this is often more legible.
In specific there's a special function for exponents:
>>> 2**3
8
But I also solved it with a lambda function, which is a nicer version of a function pointer.
# cube = power(pow=3) # original
cube = lambda x: power(x,3)