Related
This question already has answers here:
How to create a decorator that can be used either with or without parameters?
(16 answers)
Closed 3 years ago.
Suppose I have the following decorator. (To repeat a function n times)
def repeat(num_times=4):
def decorator_repeat(func):
#functools.wraps(func)
def wrapper_repeat(*args, **kwargs):
for _ in range(num_times):
value = func(*args, **kwargs)
return value
return wrapper_repeat
return decorator_repeat
Now, it does have a default value of 4, however, even if I want to call it with default value, I still have to call it as follows
#repeat()
def my_function():
print("hello")
instead of
#repeat
def my_function():
print("hello")
Now, I can change the definition of my decorator to
def repeat(_func=None, *, num_times=2):
def decorator_repeat(func):
#functools.wraps(func)
def wrapper_repeat(*args, **kwargs):
for _ in range(num_times):
value = func(*args, **kwargs)
return value
return wrapper_repeat
if _func is None:
return decorator_repeat
else:
return decorator_repeat(_func)
And enable the functionality to call it without arguments if I want to.
However, can this be achieved without changing the code of the decorator, but by defining another decorator?
i.e. I want to define a decorator enable_direct so that I can just add #enable_direct to my decorator definition and have the same effect. (i.e. as follows)
#enable_direct
def repeat(num_times=4):
def decorator_repeat(func):
#functools.wraps(func)
def wrapper_repeat(*args, **kwargs):
for _ in range(num_times):
value = func(*args, **kwargs)
return value
return wrapper_repeat
return decorator_repeat
Note:
I am aware of the solution mentioned in How to create a Python decorator that can be used either with or without parameters?
The definitions in that question have a different signature, and if one is starting afresh, one can follow that pattern. However, say I have 20-30 such decorator definitions (3 level nested). I want all of these to be enabled to be called without parentheses.
The def repeat statement does not have a function argument. The functions in that question have a 2 level nesting, while mine has 3 level. I wanted to ask if it is possible with such decorator definitions (which are meant to be called with parentheses) without changing the function definition. The accepted answer there has a different signature, and thus does not meat requirement in this question.
Note 2: I did not ask this question before trying out the definition of double wrap given there. Calling it without parentheses returns another function (if the signature of function is as described).
Here you are:
import functools
def enable_direct(decorator):
#functools.wraps(decorator)
def wrapper(*args, **kwargs):
f = args[0]
if callable(f):
return decorator()(f) # pass the function to be decorated
else:
return decorator(*args, **kwargs) # pass the specified params
return wrapper
#enable_direct
def repeat(num_times=4):
def decorator_repeat(func):
#functools.wraps(func)
def wrapper_repeat(*args, **kwargs):
for _ in range(num_times):
value = func(*args, **kwargs)
return value
return wrapper_repeat
return decorator_repeat
#repeat
def my_func(name):
print(name)
#repeat(2)
def my_func2(name):
print(name)
print(my_func)
print(my_func2)
my_func("Gino")
my_func2("Mario")
Which produces
<function my_func at 0x7f629f091b70>
<function my_func2 at 0x7f629f091bf8>
Gino
Gino
Gino
Gino
Mario
Mario
I am attempting to copy the functionality of the built-in property class / decorator; a very basic example of what I want to is this:
# If a condition is met, run the first function, else, the second.
#godspeed()
def test():
print(1, 2, 3, 4)
#test.else_()
def test():
print(5, 6, 7, 8)
Here's what I have so far:
import inspect
class godspeed_class():
def __init__(
self,
func,
args,
kwargs,
value,
):
self.func = func
self.args = args
self.kwargs = kwargs
self.value = value
def __call__(self):
if self.value:
self.func(*self.args, **self.kwargs)
else:
self.else_func(*self.else_args, **self.else_kwargs)
def else_(self, *args, **kwargs):
def wrapper(func):
self.else_func = func
self.else_args = args
self.else_kwargs = kwargs
return wrapper
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
_ = godspeed_class(func, args, kwargs, value)
inspect.stack(1)[1][0].f_globals[func.__name__] = _
return wrapper
I already know how to implement the condition parsing, but I am having trouble with storing the function under the else_ decorator in the class, so that I can call it if the condition isn't met.
In addition, despite injecting the new class directly into the global namespace, when I run print(test), it tells me it's a NoneType object.
Note: Code has been updated; however, it still gives me the "NoneType object" error.
You need to change both of your wrapper functions to return a callable object, probably the instance of your class. Otherwise you're going to have None as the value for the method, since the decorator syntax will assign the return value to the name of the decorated function, which means that even if your inspect hack works, it will get overwritten.
I'd suggest:
class godspeed_class():
... # __init__ and __call__ can remain the same
def else_(self, *args, **kwargs):
def wrapper(func):
self.else_func = func
self.else_args = args
self.else_kwargs = kwargs
return self # add return here
return wrapper
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
return godspeed_class(func, args, kwargs, value) # and here (rather than inspect stuff)
return wrapper
This will do the job for your example with a top-level test function. If you want to be able to decorate methods, you'll also need to add a __get__ method to the class to add binding behavior (otherwise you'll not get the self argument passed in to the wrapped method).
It's a bit misleading to use wrapper as the name there, as the inner functions are the actual decorators being used here (the top level godspeed function and the else_ method are decorator factories). Normally you use wrapper as a name of a function returned by a decorator (but you're using your class for that instead).
I'd also note that it's a bit strange that you're passing the arguments for the functions to the decorator factories, rather than having __call__ accept arguments that it passes along to the relevant function. It's a bit unusual for a decorator that leaves behind a callable (rather than something like property that works differently) to dramatically change a function's calling convention, as it may end up hard for a caller to know what arguments they're expected to pass in, if the function signature isn't representative any more.
A decorator is nothing magical. Basically, the #decorator syntax is just syntactic sugar, so this:
#mydecorator
def func():
pass
is just a convenient shortcut for
def func():
pass
func = mydecorator(func)
IOW, a "decorator" is a callable object that takes a callable as input and returns a callable (well, it's supposed to return a callable at least - you can actually return whatever, but then you'll break everyone's expectations).
Most often, the decorator is written as a simple function returning a closure over the decorated function:
def trace(func):
def wrapper(*args, **kw):
result = func(*args, **kw)
print("{}({}, {}) => {}". format(func, args, kw, result))
return result
return wrapper
#trace
def foo(x):
return 42 * x
But (since closures are the poor man's classes and classes the poor man's closures) you can also implement it as a callable class, in which case the initializer will receive the decorated func, which in turn will be replaced by the instance:
class trace(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kw):
result = self.func(*args, **kw)
print("{}({}, {}) => {}". format(self.func, args, kw, result))
return result
#trace
def foo(x):
return 42 * x
Then you have "parameterized" decorators - the one that can take arguments. In this case you need two level of indirection, the top-level one (the one used as decorator) returning the actual decorator (the one that receives the function), ie:
def trace(out):
def really_trace(func):
def wrapper(*args, **kw):
result = func(*args, **kw)
out.write("{}({}, {}) => {}\n". format(func, args, kw, result))
return result
return wrapper
return really_trace
#trace(sys.stderr)
def foo(x):
return 42 * x
I leave the class-based implementation as an exercise to the reader ;-)
Now in your case, the fact that test ends up being None is quite simply due to the fact that your wrapper func forgets to return the godspeed_class instance as it should (instead messing with the function's f_globals, which, as you noticed, doesn't work as expected).
Since you didn't clearly explained what you're trying to achieve here ("something similar to property" isn't a proper spec), it's hard to provide a working solution, but as a starting point you may want to fix your godspeed func to behave as expected:
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
return godspeed_class(func, args, kwargs, value)
return wrapper
This question already has answers here:
How can I decorate an instance method with a decorator class?
(2 answers)
Closed 4 years ago.
While there are plenty of resources about using classes as decorators, I haven't been able to find any that deal with the problem of decorating methods. The goal of this question is to fix that. I will post my own solution, but of course everyone else is invited to post theirs as well.
Why the "standard" implementation doesn't work
The problem with the standard decorator class implementation is that python will not create a bound method of the decorated function:
class Deco:
def __init__(self, func):
self.func= func
def __call__(self, *args):
self.func(*args)
class Class:
#Deco
def hello(self):
print('hello world')
Class().hello() # throws TypeError: hello() missing 1 required positional argument: 'self'
A method decorator needs to overcome this hurdle.
Requirements
Taking the classes from the previous example, the following things are expected to work:
>>> i= Class()
>>> i.hello()
hello world
>>> i.hello
<__main__.Deco object at 0x7f4ae8b518d0>
>>> Class.hello is Class().hello
False
>>> Class().hello is Class().hello
False
>>> i.hello is i.hello
True
Ideally, the function's __doc__ and signature and similar attributes are preserved as well.
Usually when a method is accessed as some_instance.some_method(), python's descriptor protocol kicks in and calls some_method.__get__(), which returns a bound method. However, because the method has been replaced with an instance of the Deco class, that does not happen - because Deco is not a descriptor. In order to make Deco work as expected, it must implement a __get__ method that returns a bound copy of itself.
Implementation
Here's basic "do nothing" decorator class:
import inspect
import functools
from copy import copy
class Deco(object):
def __init__(self, func):
self.__self__ = None # "__self__" is also used by bound methods
self.__wrapped__ = func
functools.update_wrapper(self, func)
def __call__(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy
bound = copy(self)
bound.__self__ = instance
# update __doc__ and similar attributes
functools.update_wrapper(bound, self.__wrapped__)
# add the bound instance to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
To make the decorator do something, add your code in the __call__ method.
Here's one that takes parameters:
class DecoWithArgs(object):
#== change the constructor's parameters to fit your needs ==
def __init__(self, *args):
self.args = args
self.__wrapped__ = None
self.__self__ = None
def __call__(self, *args, **kwargs):
if self.__wrapped__ is None:
return self.__wrap(*args, **kwargs)
else:
return self.__call_wrapped_function(*args, **kwargs)
def __wrap(self, func):
# update __doc__ and similar attributes
functools.update_wrapper(self, func)
return self
def __call_wrapped_function(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy of this object
bound = copy(self)
bound.__self__ = instance
bound.__wrap(self.__wrapped__)
# add the bound decorator to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
An implementation like this lets us use the decorator on methods as well as functions, so I think it should be considered good practice.
Assuming the following structure:
class SetupTestParam(object):
def setup_method(self, method):
self.foo = bar()
#pytest.fixture
def some_fixture():
self.baz = 'foobar'
I use SetupTestParam as a parent class for test classes.
class TestSomething(SetupTestParam):
def test_a_lot(self, some_fixture):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
... # not repetative code here
assert spam == 'something cool'
Now, writing tests gets repetitive (with statement usage) and I would like to write a decorator to reduce the number of code lines. But there is a problem with pytest and the function signature.
I found out library which should be helpful but I can't manage to get it to work.
I made a classmethod in my SetupTestParam class.
#classmethod
#decorator.decorator
def this_is_decorator(cls, f):
def wrapper(self, *args, **kw):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return f(self, *args)
return wrapper
After I decorate the test_a_lot method, I receive the error TypeError: transaction_decorator() takes exactly 1 argument (2 given)
Can someone explain me please what am I doing wrong? (I assume there is a problem with self from the test method?)
Chaining decorators is not the simplest thing to do. One solution might be to separate the two decorators. Keep the classmethod but move decorator.decorator to the end:
#classmethod
def this_is_decorator(cls, f):
def wrapper(self, *args, **kw):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return f(self, *args)
return decorator.decorator(wrapper, f)
Maybe this works for you.
After some tweaking and realizing that I need to pass a parameter to decorator I choosed to write it as a class:
class ThisIsDecorator(object):
def __init__(self, param):
self.param = param # Parameter may vary with the function being decorated
def __call__(self, fn):
wraps(fn) # [1]
def wrapper(fn, fn_self, *args): # [2] fn_self refers to original self param from function fn (test_a_lot) [2]
with fn_self.baz as fn_self.magic: # I pass magic to fn_self to make magic accesible in function fn (test_a_lot)
with fn_self.magic.fooz as more_magic:
blah = self.param.much_more_magic() # repetative bleh
return fn(fn_self, *args)
return decorator.decorator(wrapper, fn)
[1] I use wraps to have original fn __name__, __module__ and __doc__.
[2] Params passed to wrapper were self = <function test_a_lot at 0x24820c8> args = (<TestSomething object at 0x29c77d0>, None, None, None, None), kw = {} so I took out the args[0] as a fn_self.
Original version (without passing a parameter):
#classmethod
def this_is_decorator(cls, fn):
#wraps(fn)
def wrapper(fn, fn_self, *args):
with fn_self.baz as fn_self.magic:
with fn_self.magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return fn(fn_self, *args)
return decorator.decorator(wrapper,fn)
Thanks go to Mike Muller for pointing out the right direction.
Here's what happens in time order as this method is defined.
this_is_decorator is created (not called).
decorator.decorator(this_is_decorator) is called. This returns a new function which becomes this_is_decorator and has the same usage.
classmethod(this_is_decorator) is called, and the result of that is a classmethod that accepts (cls, f) and returns wrapper.
Later at runtime, a call to this_is_decorator will return wrapper.
But considering that this_is_decorator is a class method, it's not clear to me that this is what you want. I'm guessing that you may want something more like this:
from decorator import decorator
#decorator
def mydecorator(f):
def wrapper(cls, *args, **kw):
# ... logging, reporting, caching, whatever
return f(*args, **kw)
return wrapper
class MyClass(object):
#classmethod
#mydecorator
def myclsmethod(a, b, c):
# no cls or self value accepted here; this is a function not a method
# ...
Here your decorator is defined outside your class, because it's changing an ordinary function into a classmethod (and because you may want to use it in other places). The order of execution here is:
mydecorator is defined, not called.
decorator(mydecorator) is called, and the result becomes mydecorator.
Creation of MyClass starts.
myclsmethod is created. It is an ordinary function, not a method. There is a difference within the VM, so that you do not have to explicitly supply cls or self arguments to methods.
myclsmethod is passed to mydecorator (which has itself been decorated before) and the result (wrapper) is still a function not a method.
The result of mydecorator is passed to classmethod which returns an actual class method that is bound to MyClass.myclsmethod.
Definition of MyClass finishes.
Later when MyClass.myclsmethod(a, b, c) is called, wrapper executes, which then calls the original myclsmethod(a, b, c) function (which it knows as f) without supplying the cls argument.
Since you have an additional need to preserve the argument list exactly, so that even the names of the arguments are preserved in the decorated function, except with an extra initial argument cls, then you could implement mydecorator this way:
from decorator import decorator
from inspect import getargspec
#decorator
def mydecorator(func):
result = [None] # necessary so exec can "return" objects
namespace = {'f': func, 'result': result}
source = []
add = lambda indent, line: source.append(' ' * indent + line) # shorthand
arglist = ', '.join(getargspec(func).args) # this does not cover keyword or default args
add(0, 'def wrapper(cls, %s):' % (arglist,))
add(2, 'return f(%s)' % (arglist,))
add(0, 'result[0] = wrapper') # this is how to "return" something from exec
exec '\n'.join(source) in namespace
return result[0] # this is wrapper
It's kinda ugly, but this is the only way I know to dynamically set the argument list of a function based on data. If returning a lambda is OK, you could use eval instead of exec, which eliminates the need for an array that gets written into but is otherwise about the same.
I have a decorator like below.
def myDecorator(test_func):
return callSomeWrapper(test_func)
def callSomeWrapper(test_func):
return test_func
#myDecorator
def someFunc():
print 'hello'
I want to enhance this decorator to accept another argument like below
def myDecorator(test_func,logIt):
if logIt:
print "Calling Function: " + test_func.__name__
return callSomeWrapper(test_func)
#myDecorator(False)
def someFunc():
print 'Hello'
But this code gives the error,
TypeError: myDecorator() takes exactly 2 arguments (1 given)
Why is the function not automatically passed? How do I explicitly pass the function to the decorator function?
Since you are calling the decorator like a function, it needs to return another function which is the actual decorator:
def my_decorator(param):
def actual_decorator(func):
print("Decorating function {}, with parameter {}".format(func.__name__, param))
return function_wrapper(func) # assume we defined a wrapper somewhere
return actual_decorator
The outer function will be given any arguments you pass explicitly, and should return the inner function. The inner function will be passed the function to decorate, and return the modified function.
Usually you want the decorator to change the function behavior by wrapping it in a wrapper function. Here's an example that optionally adds logging when the function is called:
def log_decorator(log_enabled):
def actual_decorator(func):
#functools.wraps(func)
def wrapper(*args, **kwargs):
if log_enabled:
print("Calling Function: " + func.__name__)
return func(*args, **kwargs)
return wrapper
return actual_decorator
The functools.wraps call copies things like the name and docstring to the wrapper function, to make it more similar to the original function.
Example usage:
>>> #log_decorator(True)
... def f(x):
... return x+1
...
>>> f(4)
Calling Function: f
5
Just to provide a different viewpoint: the syntax
#expr
def func(...): #stuff
is equivalent to
def func(...): #stuff
func = expr(func)
In particular, expr can be anything you like, as long as it evaluates to a callable. In particular particular, expr can be a decorator factory: you give it some parameters and it gives you a decorator. So maybe a better way to understand your situation is as
dec = decorator_factory(*args)
#dec
def func(...):
which can then be shortened to
#decorator_factory(*args)
def func(...):
Of course, since it looks like decorator_factory is a decorator, people tend to name it to reflect that. Which can be confusing when you try to follow the levels of indirection.
Just want to add some usefull trick that will allow to make decorator arguments optional. It will also alows to reuse decorator and decrease nesting
import functools
def myDecorator(test_func=None,logIt=None):
if test_func is None:
return functools.partial(myDecorator, logIt=logIt)
#functools.wraps(test_func)
def f(*args, **kwargs):
if logIt==1:
print 'Logging level 1 for {}'.format(test_func.__name__)
if logIt==2:
print 'Logging level 2 for {}'.format(test_func.__name__)
return test_func(*args, **kwargs)
return f
#new decorator
myDecorator_2 = myDecorator(logIt=2)
#myDecorator(logIt=2)
def pow2(i):
return i**2
#myDecorator
def pow3(i):
return i**3
#myDecorator_2
def pow4(i):
return i**4
print pow2(2)
print pow3(2)
print pow4(2)
Just another way of doing decorators.
I find this way the easiest to wrap my head around.
class NiceDecorator:
def __init__(self, param_foo='a', param_bar='b'):
self.param_foo = param_foo
self.param_bar = param_bar
def __call__(self, func):
def my_logic(*args, **kwargs):
# whatever logic your decorator is supposed to implement goes in here
print('pre action baz')
print(self.param_bar)
# including the call to the decorated function (if you want to do that)
result = func(*args, **kwargs)
print('post action beep')
return result
return my_logic
# usage example from here on
#NiceDecorator(param_bar='baaar')
def example():
print('example yay')
example()
Now if you want to call a function function1 with a decorator decorator_with_arg and in this case both the function and the decorator take arguments,
def function1(a, b):
print (a, b)
decorator_with_arg(10)(function1)(1, 2)