I'm learning decorators and I have a task asking me to create a decorator to prevent a function from being called twice in a row. If the same function is called again, it should return None
I can't seem to understand how it really works, so far I've achieved this:
def dont_run_twice(f):
global counter
def wrapper(*args, **kwargs):
counter += 1
if counter == 2:
return None
else:
result = f(*args, **kwargs)
return result
counter = 0
(I know that it's a really bad attempt, but I just don't know a way to keep track of a function called with specific arguments in order to check if it was already before)
the output should be something like:
#dont_run_twice
def myPrint(*args):
print(*args)
myPrint("Hello")
myPrint("Hello") #won't do anything (only return None)
myPrint("Hello") #still does nothing.
myPrint("Goodbye") #will work
myPrint("Hello") #will work
Seems this can help you:
import functools
def do_not_run_twice(func):
prev_call = None
#functools.wraps(func) # It is good practice to use this decorator for decorators
def wrapper(*args, **kwargs):
nonlocal prev_call
if (args, kwargs) == prev_call:
return None
prev_call = args, kwargs
return func(*args, **kwargs)
return wrapper
Try this:
my_print("Hello")
my_print("Hello") # won't do anything (only return None)
my_print("Hello") # still does nothing.
my_print("Goodbye") # will work
my_print("Hello") # will work.
Here's a solution which is very similar to Andrey Berenda's solution, but which works by assigning an attribute to the function object, rather than using a non-local variable. The practical difference is that the function's previous arguments become available externally, which might help for debugging purposes.
from functools import wraps
def dont_run_twice(func):
#wraps(func)
def wrapper(*args, **kwargs):
if (args, kwargs) == wrapper._prev_args:
return None
wrapper._prev_args = args, kwargs
return func(*args, **kwargs)
wrapper._prev_args = None
return wrapper
Example:
>>> #dont_run_twice
... def f(x, y):
... return x + y
...
>>> f(1, 2)
3
>>> f(3, 4)
7
>>> f(3, 4) # returns None
>>> f(1, 2)
3
>>> f._prev_args
((1, 2), {})
Note that both solutions have a slight flaw: you can call with the same argument values if you provide them as positional arguments then keyword arguments (or vice-versa):
>>> f(5, 6)
11
>>> f(x=5, y=6)
11
As a workaround, you can declare the wrapped function with positional-only (or keyword-only) arguments:
# positional-only, requires Python 3.8+
#dont_run_twice
def f(x, y, /):
return x + y
# keyword-only
#dont_run_twice
def g(*, x, y):
return x + y
Note also that if the previous args are mutable, then strange things can happen:
>>> a = [1, 2]
>>> b = [3, 4]
>>> f(a, b)
[1, 2, 3, 4]
>>> a[:] = [5, 6]
>>> b[:] = [7, 8]
>>> f([5, 6], [7, 8]) # returns None
The second function call here returns None despite the new arguments not being equal to the original arguments by either value or identity; they are equal to the original arguments' current values which were changed after they were used as arguments. This could lead to rather subtle bugs, but unfortunately there's no easy way to fix it.
Related
How can I bind arguments to a Python function so that I can call it later without arguments (or with fewer additional arguments)?
For example:
def add(x, y):
return x + y
add_5 = magic_function(add, 5)
assert add_5(3) == 8
What is the magic_function I need here?
It often happens with frameworks and libraries that people accidentally call a function immediately when trying to give arguments to a callback: for example on_event(action(foo)). The solution is to bind foo as an argument to action, using one of the techniques described here. See for example How to pass arguments to a Button command in Tkinter? and Using a dictionary as a switch statement in Python.
Some APIs, however, allow you to pass the to-be-bound arguments separately, and will do the binding for you. Notably, the threading API in the standard library works this way. See thread starts running before calling Thread.start. If you are trying to set up your own API like this, see How can I write a simple callback function?.
Explicitly binding arguments is also a way to avoid problems caused by late binding when using closures. This is the problem where, for example, a lambda inside a for loop or list comprehension produces separate functions that compute the same result. See What do lambda function closures capture? and Creating functions (or lambdas) in a loop (or comprehension).
functools.partial returns a callable wrapping a function with some or all of the arguments frozen.
import sys
import functools
print_hello = functools.partial(sys.stdout.write, "Hello world\n")
print_hello()
Hello world
The above usage is equivalent to the following lambda.
print_hello = lambda *a, **kw: sys.stdout.write("Hello world\n", *a, **kw)
Using functools.partial:
>>> from functools import partial
>>> def f(a, b):
... return a+b
...
>>> p = partial(f, 1, 2)
>>> p()
3
>>> p2 = partial(f, 1)
>>> p2(7)
8
If functools.partial is not available then it can be easily emulated:
>>> make_printer = lambda s: lambda: sys.stdout.write("%s\n" % s)
>>> import sys
>>> print_hello = make_printer("hello")
>>> print_hello()
hello
Or
def partial(func, *args, **kwargs):
def f(*args_rest, **kwargs_rest):
kw = kwargs.copy()
kw.update(kwargs_rest)
return func(*(args + args_rest), **kw)
return f
def f(a, b):
return a + b
p = partial(f, 1, 2)
print p() # -> 3
p2 = partial(f, 1)
print p2(7) # -> 8
d = dict(a=2, b=3)
p3 = partial(f, **d)
print p3(), p3(a=3), p3() # -> 5 6 5
lambdas allow you to create a new unnamed function with fewer arguments and call the function:
>>> def foobar(x, y, z):
... print(f'{x}, {y}, {z}')
...
>>> foobar(1, 2, 3) # call normal function
1, 2, 3
>>> bind = lambda x: foobar(x, 10, 20) # bind 10 and 20 to foobar
>>> bind(1)
1, 10, 20
>>> bind = lambda: foobar(1, 2, 3) # bind all elements
>>> bind()
1, 2, 3
You can also use functools.partial. If you are planning to use named argument binding in the function call this is also applicable:
>>> from functools import partial
>>> barfoo = partial(foobar, x=10)
>>> barfoo(y=5, z=6)
10, 5, 6
Note that if you bind arguments from the left you need to call the arguments by name. If you bind from the right it works as expected.
>>> barfoo(5, 6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foobar() got multiple values for argument 'x'
>>> f = partial(foobar, z=20)
>>> f(1, 1)
1, 1, 20
This would work, too:
def curry(func, *args):
def curried(*innerargs):
return func(*(args+innerargs))
curried.__name__ = "%s(%s, ...)" % (func.__name__, ", ".join(map(str, args)))
return curried
>>> w=curry(sys.stdout.write, "Hey there")
>>> w()
Hey there
Functors can be defined this way in Python. They're callable objects. The "binding" merely sets argument values.
class SomeFunctor( object ):
def __init__( self, arg1, arg2=None ):
self.arg1= arg1
self.arg2= arg2
def __call___( self, arg1=None, arg2=None ):
a1= arg1 or self.arg1
a2= arg2 or self.arg2
# do something
return
You can do things like
x= SomeFunctor( 3.456 )
x( arg2=123 )
y= SomeFunctor( 3.456, 123 )
y()
The question asks generally about binding arguments, but all answers are about functions. In case you are wondering, partial also works with class constructors (i.e. using a class instead of a function as a first argument), which can be useful for factory classes. You can do it as follows:
from functools import partial
class Animal(object):
def __init__(self, weight, num_legs):
self.weight = weight
self.num_legs = num_legs
animal_class = partial(Animal, weight=12)
snake = animal_class(num_legs = 0)
print(snake.weight) # prints 12
I run a decorator demo below.
def logger(func):
def inner(*args, **kwargs):
print(args)
print(kwargs)
return func(*args, **kwargs)
return inner
#logger
def foo1(a, b, c, x=2, y=1):
print(x * y)
foo1(6,7,8)
output is:
(6, 7, 8)
{}
2
Why is the dict empty? I think it should be {'x':2, 'y':1}
That's because of no kwargs provided in a function call. And decorator logger know nothing about that and what function will use. It is kind a "proxy" between kwargs provided there and real call.
See examples below:
# kwargs are not provided (not redefined), function `foo1` will use default.
>>> foo1(6, 7, 8)
(6, 7, 8)
{}
2
# new kwargs are provided and passed to decorator too
>>> foo1(6, 7, 8, x=9, y=10)
(6, 7, 8)
{'x': 9, 'y': 10}
90
This is something similar to:
def foo1(a, b, c, x=2, y=1):
print(x * y)
def logger(func):
def inner(*args, **kwargs):
print(args)
print(kwargs)
return func(*args, **kwargs)
return inner
wrapped_foo1 = logger(foo1)
wrapped_foo1(6,7,8)
Or even simplified to the following, when you can clearly see the problem:
def foo1_decorated(*args, **kwargs):
print(args) # <-- here it has no chance to know that `x=2, y=1`
print(kwargs)
return foo1(*args, **kwargs)
foo1_decorated(6, 7, 8)
The problem is that the default values for arguments are filled in by the wrapped function object when you call it, because only the wrapped function knows them (they are stored in __defaults__ and __kwdefaults__).
If you want your decorator to know about them too, you have to mimic what the wrapped function object would do.
For this task you can use the inspect module:
from inspect import signature
def logger(func):
sig = signature(func)
def inner(*args, **kwargs):
arguments = sig.bind(*args, **kwargs) # these 2 steps are normally handled by func
arguments.apply_defaults()
print(func, "was called with", arguments)
return func(*args, **kwargs)
return inner
#logger
def foo1(a, b, c, x=2, y=1):
print(x * y)
foo1(6,7,8)
Output:
<function foo1 at 0x7f5811a18048> was called with <BoundArguments (a=6, b=7, c=8, x=2, y=1)>
2
If you want to access the arguments, read more about it in the docs.
That dictionary is empty because you have not passed any kwargs in foo1.
To get x and y instead of empty dictionary you can use
foo1(6,7,8, x=2, y=3) # x and y are printed while printing kwargs
instead of
foo1(6,7,8) # no x and y values are passed so empty dict is print while printing kwargs
Note that you should only use variable x and y. Any other variables will cause error.
The process that is exactly happening is this:
1. foo1 function is tried to called
2. Due to presence of #logger, logger function is called first
3. foo1 function is passed to logger function.
4. inner function takes both type of arguments of foo1 function.
4. *args accepts arguments that are comma separated and should not
contain key = value type of argument
5. **kwargs accepts arguments that are only key = value type
6. Since you have passed 6,7,8, they are all treated as *args
7. To pass as **kwargs, you have to pass key = value in foo1 parameters.
8. *args and ** kwargs values are printed
9. foo1 function is called
10. It executes code inside foo1
I'm trying to create functions inside of a loop:
functions = []
for i in range(3):
def f():
return i
# alternatively: f = lambda: i
functions.append(f)
The problem is that all functions end up being the same. Instead of returning 0, 1, and 2, all three functions return 2:
print([f() for f in functions])
# expected output: [0, 1, 2]
# actual output: [2, 2, 2]
Why is this happening, and what should I do to get 3 different functions that output 0, 1, and 2 respectively?
You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).
Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:
def f(i=i):
return i
Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.
If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":
def make_f(i):
def f():
return i
return f
and in your loop use f = make_f(i) instead of the def statement.
The Explanation
The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.
If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:
global_var = 'foo'
def my_function():
print(global_var)
global_var = 'bar'
my_function()
When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.
The Solution
There are actually many ways to solve this problem. Here are a few options:
Force early binding of i by using it as a default argument
Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:
for i in range(3):
def f(i=i): # <- right here is the important bit
return i
functions.append(f)
To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.
>>> i = 0
>>> def f(i=i):
... pass
>>> f.__defaults__ # this is where the current value of i is stored
(0,)
>>> # assigning a new value to i has no effect on the function's default arguments
>>> i = 5
>>> f.__defaults__
(0,)
Use a function factory to capture the current value of i in a closure
The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:
def f_factory(i):
def f():
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
functions.append(f)
Use functools.partial to bind the current value of i to f
functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.
import functools
def f(i):
return i
for i in range(3):
f_with_i = functools.partial(f, i) # important: use a different variable than "f"
functions.append(f_with_i)
Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:
>>> i = [] # instead of an int, i is now a *mutable* object
>>> def f(i=i):
... print('i =', i)
...
>>> i.append(5) # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]
Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:
def f(i=i.copy()):
f = f_factory(i.copy())
f_with_i = functools.partial(f, i.copy())
To add onto #Aran-Fey's excellent answer, in the second solution you might also wish to modify the variable inside your function which can be accomplished with the keyword nonlocal:
def f_factory(i):
def f(offset):
nonlocal i
i += offset
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
print(f(10))
You have to save the each of the i value in a separate space in memory e.g.:
class StaticValue:
val = None
def __init__(self, value: int):
StaticValue.val = value
#staticmethod
def get_lambda():
return lambda x: x*StaticValue.val
class NotStaticValue:
def __init__(self, value: int):
self.val = value
def get_lambda(self):
return lambda x: x*self.val
if __name__ == '__main__':
def foo():
return [lambda x: x*i for i in range(4)]
def bar():
return [StaticValue(i).get_lambda() for i in range(4)]
def foo_repaired():
return [NotStaticValue(i).get_lambda() for i in range(4)]
print([x(2) for x in foo()])
print([x(2) for x in bar()])
print([x(2) for x in foo_repaired()])
Result:
[6, 6, 6, 6]
[6, 6, 6, 6]
[0, 2, 4, 6]
You can try like this:
l=[]
for t in range(10):
def up(y):
print(y)
l.append(up)
l[5]('printing in 5th function')
just modify the last line for
functions.append(f())
Edit: This is because f is a function - python treats functions as first-class citizens and you can pass them around in variables to be called later on. So what your original code is doing is appending the function itself to the list, while what you want to do is append the results of the function to the list, which is what the line above achieves by calling the function.
I work in Python. Recently, I discovered a wonderful little package called fn. I've been using it for function composition.
For example, instead of:
baz(bar(foo(x))))
with fn, you can write:
(F() >> foo >> bar >> baz)(x) .
When I saw this, I immediately thought of Clojure:
(-> x foo bar baz) .
But notice how, in Clojure, the input is on the left. I wonder if this possible in python/fn.
You can't replicate the exact syntax, but you can make something similar:
def f(*args):
result = args[0]
for func in args[1:]:
result = func(result)
return result
Seems to work:
>>> f('a test', reversed, sorted, ''.join)
' aestt'
You can't get that exact syntax, although you can get something like F(x)(foo, bar, baz). Here's a simple example:
class F(object):
def __init__(self, arg):
self.arg = arg
def __call__(self, *funcs):
arg = self.arg
for f in funcs:
arg = f(arg)
return arg
def a(x):
return x+2
def b(x):
return x**2
def c(x):
return 3*x
>>> F(2)(a, b, c)
48
>>> F(2)(c, b, a)
38
This is a bit different from Blender's answer since it stores the argument, which can later be re-used with different functions.
This is sort of like the opposite of normal function application: instead of specifying the function up front and leaving some arguments to be specified later, you specify the argument and leave the function(s) to be specified later. It's an interesting toy but it's hard to think why you'd really want this.
If you want to use fn, with a little hack you can get a bit closer to Clojure syntax:
>>> def r(x): return lambda: x
>>> (F() >> r(x) >> foo >> bar >> baz)()
See how I added another function at the beginning of the composition chain that will just return x when called. The problem with this is that you still have to call your composed function, just without any arguments.
I think #Blender's answer is your best bet trying to emulate Clojure's thread function in Python.
I came up with this
def _composition(arg, *funcs_and_args):
"""
_composition(
[1,2,3],
(filter, lambda x: x % 2 == 1),
(map, lambda x: x+3)
)
#=> [4, 6]
"""
for func_and_args in funcs_and_args:
func, *b = func_and_args
arg = func(*b, arg)
return(arg)
This seems to work for simple input. Not sure it is worth the effort for complex input, e.g., ((42, 'spam'), {'spam': 42}).
def compose(function, *functions):
return function if not functions else \
lambda *args, **kwargs: function(compose(*functions)(*args, **kwargs))
def rcompose(*functions):
return compose(*reversed(functions))
def postfix(arg, *functions):
return rcompose(*functions)(arg)
Example:
>>> postfix(1, str, len, hex)
'0x1'
>>> postfix(1, hex, len)
3
My compose function that returns a function
def compose(*args):
length = len(args)
def _composeInner(lastResult, index):
if ((length - 1) < index):
return lastResult
return _composeInner(args[index](lastResult), index + 1)
return (lambda x: _composeInner(x, 0))
Usage:
fn = compose(
lambda x: x * 2,
lambda x: x + 2,
lambda x: x + 1,
lambda x: x / 3
)
result = fn(6) # -> 5
I understand what you mean. It doesn't make sense. In my opinion this python library
does it better.
>>> from compositions.compositions import Compose
>>> foo = Compose(lambda x:x)
>>> foo = Compose(lambda x:x**2)
>>> foo = Compose(lambda x:sin(x))
>>> (baz*bar*foo)(x)
I've got some old code where I stored lists of functions in Python as class attributes. These lists are used as a sort of event hook.
To call each function in the list with appropriate arguments, I've used one-liners, mixing map with lambda expressions. I'm now concerned that there is unnecessary overhead in using lambda expressions like this.. I guess the recommended way would be to drop both map and lambda and just use a standard for loop, for readability.
Is there a better (read faster) one-liner to do this, though?
For example:
class Foo:
"""Dummy class demonstrating event hook usage."""
pre = [] # list of functions to call before entering loop.
mid = [] # list of functions to call inside loop, with value
post = [] # list of functions to call after loop.
def __init__(self, verbose=False, send=True):
"""Attach functions when initialising class."""
self._results = []
if verbose:
self.mid.append( self._print )
self.mid.append( self._store )
if send:
self.post.append( self._send )
def __call__(self, values):
# call each function in self.pre (no functions there)
map( lambda fn: fn(), self.pre )
for val in values:
# call each function in self.mid, with one passed argument
map( lambda fn: fn(val), self.mid )
# call each fn in self.post, with no arguments
map( lambda fn: fn(), self.post )
def _print(self, value):
"""Print argument, when verbose=True."""
print value
def _store(self, value):
"""Store results"""
self._results.append(value)
def _send(self):
"""Send results somewhere"""
# create instance of Foo
foo = Foo(verbose=True)
# equivalent to: foo.__call__( ... )
foo( [1, 2, 3, 4] )
Is there a better way to write those one-liner map calls?
The recommended way is definitely to use for loops, however, if you insist on using map, then operator.methodcaller might be just what you need:
>>> def foo(*args):
... print 'foo',args
...
>>> def bar(*args):
... print 'bar',args
...
>>> from operator import methodcaller
>>>
>>> map(methodcaller('__call__',1,2,3),[foo,bar])
foo (1, 2, 3)
bar (1, 2, 3)
[None, None]
A word of caution about using map for this -- It won't work if you port your code to python 3 since map became lazy.
You could also use list comprehensions pretty trivially (and that works on python3 also):
[fn() for fn in self.pre]
[fn(val) for fn in self.mid]
etc.
First of all "I'm concerned that there is unnecessary overhead" is no way to optimise your code. Use a profiler to find the hotspots.
Secondly, your code could do with comments to let the reader know what is going on.
Finally, until proven otherwise, the following is a fine way to accomplish the task:
for func in self.pre: func()
#apply every function in self.mid to every value in values
for func,val in itertools.product(self.mid, values):
func(val)
If you wanted to capture the values, you could use a list comprehension; if you wanted to delay evaluation, you could use a generator expression.
>>> def chain(*fn):
>>> return lambda *args, **kwargs: [_(*args, **kwargs) for _ in fn]
>>>
>>> def add(x, y):
>>> return(x + y)
>>>
>>> def multiply(x, y):
>>> return(x * y)
>>>
>>> chained = chain(add, multiply)
>>> chained(2, 6)
[8, 12]