Suppose I have the following function
def f(x,y,**kwargs):
if 'z' in kwargs:
z = kwargs['z']
else:
z = 0
print(x + y + z)
which takes two arguments and an optional keyword argument. I now want to get a function g that works just as f but for which the value of z is predetermined. Hence, I could do the following
def g(x,y):
z = 3
f(x,y, z = 3)
But what can I do if I do not know the number of non-keyword arguments that f takes. I can get the list of these arguments by
args = inspect.getargspec(f)[0]
But, if I now define g as
g(args):
z = 3
f(args, z=z)
this of course does not work as only one mandatory argument is passed to f. How do I get around this? That is, if I have a function that takes keyword arguments, how do I define a second function exactly the same expect that the keyword arguments take predeterminde values?
You have a few options here:
Define g with varargs:
def g(*args):
return f(*args, z=3)
Or, if you need keyword arguments as well:
def g(*args, **kwargs):
kwargs['z'] = 3
return f(*args, **kwargs)
Use functools.partial:
import functools
g = functools.partial(f, z=3)
See also this related question: Python Argument Binders.
You can use functools.partial to achieve this
import functools
f = functools.partial(f, z=2)
# the following example is the usage of partial function f
x = f(1, 2)
y = f(1, 2, k=3)
z = f(1, 2, z=4)
Related
I would like to use functools.partial to reduce the number of arguments in one of my functions. Here's the catch: one or more kwargs may be functions themselves. Here's what I mean:
from functools import partial
def B(alpha, x, y):
return alpha(x)*y
def alpha(x):
return x+1
g = partial(B, alpha=alpha, y=2)
print(g(5))
This throws an error:
TypeError: B() got multiple values for argument 'alpha'
Can partial handle functions as provided arguments? If not is there a workaround or something more generic than partial?
partial itself doesn't know that a given positional argument should be assigned to x just because you specified a keyword argument for alpha. If you want alpha to be particular function, pass that function as a positional argument to partial.
>>> g = partial(B, alpha, y=2)
>>> g(5)
12
g is equivalent to
def g(x):
return alpha(x) * 2 # == (x + 1) * 2
Alternately, you can use your original definition of g, but be sure to pass 5 as a keyword argument as well, avoiding any additional positional arguments.
>>> g = partial(B, alpha=alpha, y=2)
>>> g(x=5)
12
This works because between g and partial, you have provided keyword arguments for all required parameters, eliminating the need for any positional arguments.
As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'
What's the best way to specify the positions of the arguments that one wishes to evaluate?
EDIT
Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)
END EDIT
For example, consider:
def f(x,y,z):
return x + 2*y + 3*z
Then, using
from functools import partial
both
partial(f,4)(5,6)
and
partial(f,4,5)(6)
give 32.
But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?
Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:
partial_dict(f,{0:4,2:6})(5)
No, partial is not designed to freeze positional arguments at non-sequential positions.
To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:
def partial_positionals(func, positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(positionals[i] if i in positionals else next(arg)
for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, {0: 4, 2: 6})(5))
outputs:
32
Simply use keyword arguments. Using your definition of f above,
>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40
Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:
>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'
But continuing to use keyword arguments is:
>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40
When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling
h = partial(1, z=10)
f(4)
is roughly equivalent to writing
args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)
As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation
For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:
SKIP = object()
def partial_positionals(func, *positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
**{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, 4, SKIP, 6)(5))
outputs:
32
Additional explanation:
What I want to achieve is
call(function,(*args,*toomanyargs)) == (function(*args),*toomanyargs)
call(function_with_varargs,(*args))) == (function_with_varargs(*args))
what's the pythonic way to achieve this
You can find out how many positional arguments a function accepts by accessing the .__code__.co_argcount attribute:
>>> function = lambda a, b, c: a+b+c
>>> function.__code__.co_argcount
3
However, that doesn't respect varargs:
>>> function = lambda *a: a
>>> function.__code__.co_argcount
0
So the more robust solution is to use inspect.signature:
import inspect
def call(function, args):
# count the positional arguments
params = inspect.signature(function).parameters.values()
if any(param.kind == inspect.Parameter.VAR_POSITIONAL for param in params):
arg_count = len(args)
else:
POSITIONAL_KINDS = {inspect.Parameter.POSITIONAL_ONLY,
inspect.Parameter.POSITIONAL_OR_KEYWORD}
arg_count = sum(1 for param in params if param.kind in POSITIONAL_KINDS)
# take as many arguments as the function accepts
remainder = args[arg_count:]
args = args[:arg_count]
return (function(*args),) + tuple(remainder)
Demo:
>>> function = lambda a, b, c: a+b+c
>>> args = range(5)
>>> call(function, args))
(3, 3, 4)
>>>
>>> function = lambda a, b, c, *d: a+b+c
>>> args = range(5)
>>> call(function, args))
(3,)
One way would be to use locals() (Check the number of parameters passed in Python function; https://docs.python.org/3/library/functions.html#locals), and do some math in the body of each function to figure out how many args are left over (unused). You can then return a result which includes a tuple of the unused arguments.
I have a function that takes 3 keyword parameters. It has default values for x and y and I would like to call the function for different values of z using map. When I run the code below I get the following error:
foo() got multiple values for keyword argument 'x'
def foo(x =1, y = 2, z = 3):
print 'x:%d, y:%d, z:%d'%(x, y, z)
if __name__ == '__main__':
f1 = functools.partial(foo, x= 0, y = -6)
zz = range(10)
res = map(f1, zz)
Is there a Pythonic way to solve this problem?
map(f1, zz) tries to call the function f1 on every element in zz, but it doesn't know with which arguments to do it. partial redefined foo with x=0 but map will try to reassign x because it uses positional arguments.
To counter this you can either use a simple list comprehension as in #mic4ael's answer, or define a lambda inside the map:
res = map(lambda z: f1(z=z), zz)
Another solution would be to change the order of the arguments in the function's signature:
def foo(z=3, x=1, y=2):
You may obtain the equivalent result you expect without using partial:
def foo(x =1, y = 2, z = 3):
print 'x:%d, y:%d, z:%d'%(x, y, z)
if __name__ == '__main__':
f1 = lambda z: foo(x=0, y=-6, z=z)
zz = range(10)
res = map(f1, zz)
Please see this link to have a better understanding: functools.partial wants to use a positional argument as a keyword argument
When trying to call res = map(f1, zz) internally it actually looks like [f1(i) for i in range(10). As you can see you call f1 function with only one positional argument (in this case f1(x=i)). A possible solution would be to do something like this
res = [f1(z=i) for i in range(10)]
Is it possible to make a function's keyword argument default defined to a variable name? I want to do this because I'm finding myself creating functions with a lot of keyword arguments, but I'd like these keyword arguments default to a variable name I define before using the function. I'm also open to solutions that allows me to fix the repetitiveness problem in other ways. this was just one idea i was musing about.
for instance, i'd like to do something like this
def do_something(x=x, y=y, z=z, t=t):
return x, y
I know the above returns an error because I haven't defined x yet.
i'm too lazy to constantly write:
x = 100
y= 50
z= 10
t= 1
do_something(x=x, y=y, z=z, t=t)
I'd rather just write with the defaults already assumed as the variable names:
x = 100
y= 50
z= 10
t= 1
do something()
Python lets you pass in a dictionary and tell the function to unpack it
args = dict(x=100, y=50, z=5, t=1)
def do_something(args) # star tells the function to unpack as dict the variables you made
do_something(**args) # call it like this
You can even pass it in as a list with one star:
args = [100, 50, 5, 1]
def do_something(x,y,z,t) # star tells the function to unpack the variables you made
do_something(*args) # call it like this
Python also allows default parameters:
do_something(x=100, y=50, z=5, t=1) #if you don't pass arguments, it will use these defaults
Have you checked the functools module?
from functools import partial
def do_something(x,y,z,t):
print(x,y,z,t)
do_something = partial(do_something, x=100, y=50, z=10, t=1)
do_something()
do_something(x=30)
The easiest way is to use a dictionary instead of variables:
def do_something(x, y, z, t):
...
data = dict(
x = 100,
y = 50,
z = 10,
t = 1
)
do_something(**data)
Here's a really hacky way to do it:
import inspect
def autofill(fn):
def _autofill(*args, **kwargs):
kwargs.update({arg:globals()[arg] for arg in inspect.getargspec(fn).args if arg not in kwargs})
return fn(*args, **kwargs)
return _autofill
>>> #autofill
... def f(x,y):
... return x,y
...
>>> x = 1; y = 20
>>> f()
(1, 20)
>>> f(y=2)
(1, 2)