I run a decorator demo below.
def logger(func):
def inner(*args, **kwargs):
print(args)
print(kwargs)
return func(*args, **kwargs)
return inner
#logger
def foo1(a, b, c, x=2, y=1):
print(x * y)
foo1(6,7,8)
output is:
(6, 7, 8)
{}
2
Why is the dict empty? I think it should be {'x':2, 'y':1}
That's because of no kwargs provided in a function call. And decorator logger know nothing about that and what function will use. It is kind a "proxy" between kwargs provided there and real call.
See examples below:
# kwargs are not provided (not redefined), function `foo1` will use default.
>>> foo1(6, 7, 8)
(6, 7, 8)
{}
2
# new kwargs are provided and passed to decorator too
>>> foo1(6, 7, 8, x=9, y=10)
(6, 7, 8)
{'x': 9, 'y': 10}
90
This is something similar to:
def foo1(a, b, c, x=2, y=1):
print(x * y)
def logger(func):
def inner(*args, **kwargs):
print(args)
print(kwargs)
return func(*args, **kwargs)
return inner
wrapped_foo1 = logger(foo1)
wrapped_foo1(6,7,8)
Or even simplified to the following, when you can clearly see the problem:
def foo1_decorated(*args, **kwargs):
print(args) # <-- here it has no chance to know that `x=2, y=1`
print(kwargs)
return foo1(*args, **kwargs)
foo1_decorated(6, 7, 8)
The problem is that the default values for arguments are filled in by the wrapped function object when you call it, because only the wrapped function knows them (they are stored in __defaults__ and __kwdefaults__).
If you want your decorator to know about them too, you have to mimic what the wrapped function object would do.
For this task you can use the inspect module:
from inspect import signature
def logger(func):
sig = signature(func)
def inner(*args, **kwargs):
arguments = sig.bind(*args, **kwargs) # these 2 steps are normally handled by func
arguments.apply_defaults()
print(func, "was called with", arguments)
return func(*args, **kwargs)
return inner
#logger
def foo1(a, b, c, x=2, y=1):
print(x * y)
foo1(6,7,8)
Output:
<function foo1 at 0x7f5811a18048> was called with <BoundArguments (a=6, b=7, c=8, x=2, y=1)>
2
If you want to access the arguments, read more about it in the docs.
That dictionary is empty because you have not passed any kwargs in foo1.
To get x and y instead of empty dictionary you can use
foo1(6,7,8, x=2, y=3) # x and y are printed while printing kwargs
instead of
foo1(6,7,8) # no x and y values are passed so empty dict is print while printing kwargs
Note that you should only use variable x and y. Any other variables will cause error.
The process that is exactly happening is this:
1. foo1 function is tried to called
2. Due to presence of #logger, logger function is called first
3. foo1 function is passed to logger function.
4. inner function takes both type of arguments of foo1 function.
4. *args accepts arguments that are comma separated and should not
contain key = value type of argument
5. **kwargs accepts arguments that are only key = value type
6. Since you have passed 6,7,8, they are all treated as *args
7. To pass as **kwargs, you have to pass key = value in foo1 parameters.
8. *args and ** kwargs values are printed
9. foo1 function is called
10. It executes code inside foo1
Related
As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'
What's the best way to specify the positions of the arguments that one wishes to evaluate?
EDIT
Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)
END EDIT
For example, consider:
def f(x,y,z):
return x + 2*y + 3*z
Then, using
from functools import partial
both
partial(f,4)(5,6)
and
partial(f,4,5)(6)
give 32.
But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?
Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:
partial_dict(f,{0:4,2:6})(5)
No, partial is not designed to freeze positional arguments at non-sequential positions.
To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:
def partial_positionals(func, positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(positionals[i] if i in positionals else next(arg)
for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, {0: 4, 2: 6})(5))
outputs:
32
Simply use keyword arguments. Using your definition of f above,
>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40
Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:
>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'
But continuing to use keyword arguments is:
>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40
When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling
h = partial(1, z=10)
f(4)
is roughly equivalent to writing
args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)
As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation
For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:
SKIP = object()
def partial_positionals(func, *positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
**{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, 4, SKIP, 6)(5))
outputs:
32
I'm learning decorators and I have a task asking me to create a decorator to prevent a function from being called twice in a row. If the same function is called again, it should return None
I can't seem to understand how it really works, so far I've achieved this:
def dont_run_twice(f):
global counter
def wrapper(*args, **kwargs):
counter += 1
if counter == 2:
return None
else:
result = f(*args, **kwargs)
return result
counter = 0
(I know that it's a really bad attempt, but I just don't know a way to keep track of a function called with specific arguments in order to check if it was already before)
the output should be something like:
#dont_run_twice
def myPrint(*args):
print(*args)
myPrint("Hello")
myPrint("Hello") #won't do anything (only return None)
myPrint("Hello") #still does nothing.
myPrint("Goodbye") #will work
myPrint("Hello") #will work
Seems this can help you:
import functools
def do_not_run_twice(func):
prev_call = None
#functools.wraps(func) # It is good practice to use this decorator for decorators
def wrapper(*args, **kwargs):
nonlocal prev_call
if (args, kwargs) == prev_call:
return None
prev_call = args, kwargs
return func(*args, **kwargs)
return wrapper
Try this:
my_print("Hello")
my_print("Hello") # won't do anything (only return None)
my_print("Hello") # still does nothing.
my_print("Goodbye") # will work
my_print("Hello") # will work.
Here's a solution which is very similar to Andrey Berenda's solution, but which works by assigning an attribute to the function object, rather than using a non-local variable. The practical difference is that the function's previous arguments become available externally, which might help for debugging purposes.
from functools import wraps
def dont_run_twice(func):
#wraps(func)
def wrapper(*args, **kwargs):
if (args, kwargs) == wrapper._prev_args:
return None
wrapper._prev_args = args, kwargs
return func(*args, **kwargs)
wrapper._prev_args = None
return wrapper
Example:
>>> #dont_run_twice
... def f(x, y):
... return x + y
...
>>> f(1, 2)
3
>>> f(3, 4)
7
>>> f(3, 4) # returns None
>>> f(1, 2)
3
>>> f._prev_args
((1, 2), {})
Note that both solutions have a slight flaw: you can call with the same argument values if you provide them as positional arguments then keyword arguments (or vice-versa):
>>> f(5, 6)
11
>>> f(x=5, y=6)
11
As a workaround, you can declare the wrapped function with positional-only (or keyword-only) arguments:
# positional-only, requires Python 3.8+
#dont_run_twice
def f(x, y, /):
return x + y
# keyword-only
#dont_run_twice
def g(*, x, y):
return x + y
Note also that if the previous args are mutable, then strange things can happen:
>>> a = [1, 2]
>>> b = [3, 4]
>>> f(a, b)
[1, 2, 3, 4]
>>> a[:] = [5, 6]
>>> b[:] = [7, 8]
>>> f([5, 6], [7, 8]) # returns None
The second function call here returns None despite the new arguments not being equal to the original arguments by either value or identity; they are equal to the original arguments' current values which were changed after they were used as arguments. This could lead to rather subtle bugs, but unfortunately there's no easy way to fix it.
With a function f calling another well-known function in Python (e.g. a matplotlib function), what is the most pythonic/efficient/elegant way to define some default values while still giving the possibility to the user of f to fully customize the called function (typically with **kwargs), including to overwrite the default keyword arguments defined in f?
import numpy as np
import matplotlib.pyplot as plt
v = np.linspace(-10.,10.,100)
x,y = np.meshgrid(v, v)
z = -np.hypot(x, y)
def f(ax, n=12, **kwargs):
ax.contourf(x, y, z, n, cmap=plt.cm.autumn, **kwargs)
fig, ((ax0, ax1), (ax2, ax3)) = plt.subplots(2, 2)
f(ax0) # OK
f(ax1, n=100) # OK
f(ax2, n=100, **{'vmax': -2, 'alpha': 0.2}) # OK
# f(ax3, n=100, **{'cmap': plt.cm.cool}) # ERROR
plt.show()
Here, the last call to f throws:
TypeError: contourf() got multiple values for keyword argument 'cmap'
In your wrapper, you could simply adjust kwargs before passing it to wrapped function:
def f(ax, n=12, **kwargs):
kwargs.setdefault('cmap', plt.cm.autumn)
ax.contourf(x, y, z, n, **kwargs)
setdefault will avoid changing the argument if it was passed to your wrapper, but you could just as easily always clobber it if you wanted.
The following minimal example illustrates the options you have when coding this from scratch. Here you can also define a default argument when defining the inner function but NOT specify it when calling it in the wrapper. Note that default variable a is removed from kwargs within the function, if it was already defined as a default argument. Only in run_test_working3 (setting the default value merely with setdefault in the wrapper) default variable a is not removed from kwargs. This might be important if you want to pass kwargs to inner functions within the test function.
kwargs = {"a": 1,
"b": 2,
"c": 3}
kwargs2 = {"b": 2,
"c": 3}
def test(required_arg, a="default", **kwargs):
print(required_arg)
print(a)
print(kwargs)
def test2(required_arg, **kwargs):
print(required_arg)
print(kwargs)
#Set default for a in definition of test but not when calling it in wrapper
# a is removed from kwargs
def run_test_working1(required_arg, **kwargs):
test(required_arg, **kwargs)
#Set default for a different from definition of test in wrapper
# a is removed from kwargs
def run_test_working2(required_arg, **kwargs):
kwargs.setdefault("a", "default2")
test(required_arg, **kwargs)
#Set default value only via setdefault in wrapper
#a is not removed from kwargs
def run_test_working3(required_arg, **kwargs):
kwargs.setdefault("a", "default2")
test2(required_arg, **kwargs)
#Provoke TypeError: test() got multiple values for keyword argument 'a'
def run_test_not_working(required_arg, **kwargs):
test(required_arg, a="default", **kwargs)
print("Demo run_test_working1\n")
run_test_working1("required_arg",**kwargs)
print("\n")
run_test_working1("required_arg",**kwargs2)
print("\n")
print("Demo run_test_working2\n")
run_test_working2("required_arg",**kwargs)
print("\n")
run_test_working2("required_arg",**kwargs2)
print("\n")
print("Demo run_test_working3\n")
run_test_working3("required_arg",**kwargs)
print("\n")
run_test_working3("required_arg",**kwargs2)
print("\n")
print("Demo run_test_not_working\n")
run_test_not_working("required_arg",**kwargs)
print("\n")
test("required_arg")
Consider example:
def decorator(func):
def wrapper(*args, **kwargs):
print(args, kwargs)
func(*args, **kwargs)
return wrapper
#decorator
def foo(x, y, z=0):
pass
foo(5, 5)
Output:
(5, 5) {}
Why not (5, 5) {'z': 0}? How to pass all default values of the function foo to *args or **kwargs using only decorator (for functions) or metaclass (for class methods, e.g. __init__)?
The wrapper is just a normal function. It does not have "access" to the internals of the wrapped function.
You would have to use introspection to get them. See a related question:
How to find out the default values of a particular function's argument in another function in Python?
I'm trying to make a function designed to call another function multiple times:
def iterator(iterations, function, *args):
#called as:
iterator(5, my_function, arg1, arg2, arg3)
Note that the number of arguments here is variable: could 1, could be 2, could be 10.
fill them in based on the function that is being called.
def iterator(iterations, function, *args):
for i in range(iteration):
temp = function(args)
return temp
The problem here is:
TypeError: my_function() takes exactly 4 arguments (1 given)
And this is because (arg1, arg2, arg3, arg4) are being treated as a single argument.
How do I get around this?
By using the same syntax when applying the args sequence:
temp = function(*args)
The *args syntax here is closely related to the *args function parameter syntax; instead of capturing an arbitrary number of arguments, using *args in a call expands the sequence to separate arguments.
You may be interested to know that there is a **kwargs syntax too, to capture and apply keyword arguments:
def iterator(iterations, function, *args, **kwargs):
for i in range(iteration):
temp = function(*args, **kwargs)
return temp
Try this, unpacking the argument list (a.k.a. splatting it):
function(*args)
From the example in the documentation, you'll see that this is what you need:
range(3, 6) # ok
range([3, 6]) # won't work
range(*[3, 6]) # it works!