Develop Reference

How to wrap a decorator around another classes method?

I have created a decorator which I am using to manage logging. I want logging to occur before and after the decorated function runs. The function works fine when interacting with very basic functions, however, when interacting with methods that are a part of other classes, things break. I suspect the issue is a result of there being 2 self arguments. Do you have any idea how to resolve it?
Simplified Decorator Class
class Logger:
def __init__(self, logging_type:str = 'debug'):
self.logging_type = logging_type
def __call__(self, decorated_function:callable):
self.func = decorated_function
return getattr(self, self.logging_type)
def debug(self, *args, **kwargs):
print("starting function")
output = self.func(*args, **kwargs)
print("Completing Function")
return output
We see that the decorator works on basic functions:
#Logger(logging_type="debug")
def simple_function(x):
return x**2
In [2]: simple_function(3)
starting function
Completing Function
Out[2]: 9
However, fails when work with other classes:
class BigClass:
def __init__(self, stuff = 10):
self.stuff = stuff
#Logger(logging_type="debug")
def cool_function(self, input1: int):
return self.stuff + input1
In [16]: test = BigClass()
...: test.cool_function(3)
starting function
It then hits a type error on the output line:
TypeError: cool_function() missing 1 required positional argument: 'input1'
Ideas?

By all means read juanpa.arrivillaga's informative answer. But here is a simpler approach. In writing a class decorator of this type, __call__ should return an ordinary function instead of a member function, like this:
class Logger:
def __init__(self, logging_type:str = 'debug'):
self.logging_function = getattr(self, logging_type)
def __call__(self, decorated_function: callable):
def f(*args, **kwargs):
return self.logging_function(decorated_function, *args, **kwargs)
return f
def debug(self, decorated_function, *args, **kwargs):
print("starting function")
output = decorated_function(*args, **kwargs)
print("Completing Function")
return output
#Logger(logging_type="debug")
def simple_function(x):
return x**2
class BigClass:
def __init__(self, stuff = 10):
self.stuff = stuff
#Logger(logging_type="debug")
def cool_function(self, input1: int):
return self.stuff + input1
print(simple_function(12))
test = BigClass()
print(test.cool_function(3))
OUTPUT:
starting function
Completing Function
144
starting function
Completing Function
13

The problem is that you are decorating your function with a bound-method type, look at type(BigClass.cool_function), you'll see something like: <bound method Logger.debug of <__main__.Logger object at 0x11081f7c0>. Since bound-method objects aren't functions, they don't implement the descriptor protocol to bind the instance as the first argument, hence, the instance is never passed implicitly as the first argument.
The best solution is to avoid class-based decorators to begin with. Here's how you could implement what you are doing using function-based decorators, using the closures to maintain internal state:
from functools import wraps
def logger(*, logging_type): # I prefer keyword-only arugments for decorators, but that is your call...
def decorator(func):
#wraps(func)
def debug(*args, **kwargs):
print("starting function")
result = func(*args, **kwargs)
print("ending function")
return result
#wraps(func)
def another_option(*args, **kwargs):
print("another option")
return func(*args, **kwargs)
options = {"debug": debug, "another_option": another_option}
return options[logging_type]
return decorator
class BigClass:
def __init__(self, stuff = 10):
self.stuff = stuff
#logger(logging_type="debug")
def cool_function(self, input1: int):
return self.stuff + input1
#logger(logging_type="another_option")
def another_function(self):
return self.stuff*100

Access variable value from decorator after decorated function executes

def memoize(fn):
memory=dict()
def inner(inpt):
if not inpt in memory:
memory[inpt] = fn(inpt)
return memory[inpt]
return inner
#memoize
def expected_rounds(picks_to_go):
#algorithm....
ans += expected_rounds(new_picks_to_go) #some recursion
#algorithm....
return ans
How to access (and even print) variable memory in memoize decorator after function will successfully end execution and return answer?

my idea that I created after posting question it uses class but I would like to stay with function decorator:
#class callable that can be used as decorator
class Memoize:
def __init__(self):
self.memory = {}
# self.fn = None
def set_fn(self, fn):
self.fn = fn
return self # this line is important
def __call__(self, *args):
if args not in self.memo:
self.memory[args] = self.fn(*args)
return self.memory[args]
memo1 = Memoize()
#memo1.set_fn
def expected_rounds(picks_to_go):
#something
return ans
expected_rounds(some_input)
memo1.memory # <- access to memory possible

How to get the name of decorator for function at run time?

In the following code the methods(print_a and print_b) of class Test, has been decorated by the two different decorators.
How can I determine that the given method(let say print_a), is decorated with some specific decorator (decorator1) at the runtime?
The easy solution is to change the name of the wrapper function, i.e. changing the wrapper to wrapper1 and wrapper2 in the decorator method, but the problem with that is I don't have control over that part of the code.
Does python have any reflection API like Java, and could that help me here ?
def my_decorator1(func):
def wrapper(*args, **kwargs):
print('I am decorated:1')
func(*args, **kwargs)
return wrapper
def my_decorator2(func):
def wrapper(*args, **kwargs):
print('I am decorated:2')
func(*args, **kwargs)
return wrapper
class Test():
def __init__(self, a=None, b=None):
self.a = a
self.b = b
#my_decorator1
def print_a(self):
print('Value of a is {}'.format(self.a))
#my_decorator2
def print_b(self):
print('Value of b is {}'.format(self.b))
if __name__ == '__main__':
d = Test.__dict__
f1 = d.get('print_a')
f2 = d.get('print_b')

Since the decorator function is executed at decoration time, it is difficult to get the name. You can solve your problem with an additional decorator. The decorator annotator writes into a property of the function and you can read this name inside of the decorated function:
def annotator(decorator):
def wrapper(f):
g = decorator(f)
g.decorator = decorator.__name__
return g
return wrapper
#annotator(my_decorator1)
def x():
print x.decorator

Python decorators are more about transforming functions than about acting as simple metadata labels, as one often does with Java annotations. So to do this, I'd have the decorator set a property on the function it is wrapping (or perhaps wrap it in a callable of a particular type):
def my_decorator1(func):
def wrapper(*args, **kwargs):
print('I am decorated:1')
func(*args, **kwargs)
wrapper.decorator_name = 'my_decorator1'
return wrapper
Then:
print(Test.print_a.decorator_name)
Alternatively, in Python 3.3+ you can use PEP 3155's __qualname__ to identify the decorator.

Setting a get/set property in a python memoization decorator class

I have created a decorator memoization class that I am actively using for cache my calls. There are already many excellent suggestions on how to implement python memoization.
The class that I have created currently uses get and set method calls to set the cacheTimeOut. They are called getCacheTimeOut() and setCacheTimeOut(). While this is an adequate solution. I was hoping to use the #property and #cacheTimeOut.setter decorators to enable the functions to be called directly as for example cacheTimeOut=120
The problem is in the details. I do not know how to make these properties accessible in the __get__ method. The __get__ method assigns the different function calls defined within the class to functions.partial.
Here is my script example designed for Python 2.7
import time
from functools import partial
import cPickle
class memoize(object):
def __init__(self, func):
self.func = func
self._cache = {}
self._timestamps = {}
self._cacheTimeOut = 120
self.objtype = None
def __new__(cls, *args, **kwargs):
return object.__new__(cls,*args, **kwargs)
def __get__(self, obj, objtype=None):
"""Used for object methods where decorator has been placed before methods."""
self.objtype = objtype
fn = partial(self, obj)
fn.resetCache = self.resetCache
fn.getTimeStamps = self.getTimeStamps
fn.getCache = self.getCache
fn._timestamps = self._timestamps
fn.setCacheTimeOut = self.setCacheTimeOut
fn.getCacheTimeOut = self.getCacheTimeOut
return fn
def __argsToKey(self, *args, **kwargs):
args = list(args)
for x, arg in enumerate(args): # remove instance from
if self.objtype:
if isinstance(arg, self.objtype):
args.remove(arg)
str = cPickle.dumps(args, 1)+cPickle.dumps(kwargs, 1)
return str
def __call__(self, *args, **kwargs):
"""Main calling function of decorator."""
key = self.__argsToKey(*args, **kwargs)
now = time.time() # get current time to query for key
if self._timestamps.get(key, now) > now:
return self._cache[key]
else:
value = self.func(*args, **kwargs)
self._cache[key] = value
self._timestamps[key] = now + self._cacheTimeOut
return value
def __repr__(self):
'''Return the function's docstring.'''
return self.func.__doc__
def resetCache(self):
"""Resets the cache. Currently called manually upon request."""
self._cache = {}
self._timestamps = {}
def getCacheTimeOut(self):
"""Get the cache time out used to track stale data."""
return self._cacheTimeOut
def setCacheTimeOut(self, timeOut):
"""Set the cache timeout to some other value besides 120. Requires an integer value. If you set timeOut to zero you are ignoring the cache"""
self._cacheTimeOut = timeOut
def getCache(self):
"""Returns the cache dictionary."""
return self._cache
def getTimeStamps(self):
"""Returns the encapsulated timestamp dictionary."""
return self._timestamps
#property
def cacheTimeOut(self):
"""Get cacheTimeOut."""
return self._cacheTimeOut
#cacheTimeOut.setter
def cacheTimeOut(self, timeOut):
"""Set cacheTimeOut."""
self._cacheTimeOut = timeOut
memoize
def increment(x):
increment.count+=1
print("increment.count:%d, x:%d"%(increment.count, x))
x+=1
return x
increment.count = 0 # Define the count to track whether calls to increment vs cache
class basic(object):
def __init__(self):
self.count = 0
#memoize
def increment(self, x):
self.count+=1
print("increment.count:%d, x:%d"%(increment.count, x))
x+=1
return x
def main():
print increment(3)
print increment(3)
# What I am actually doing
print increment.getCacheTimeOut() # print out default of 120
increment.setCacheTimeOut(20) # set to 20
print increment.getCacheTimeOut() # verify that is has been set to 120
# What I would like to do and currently does not work
print increment.cacheTimeOut
# Assign to property
increment.cacheTimeOut = 20
myObject = basic()
print myObject.increment(3)
print myObject.count
print myObject.increment(3)
print myObject.count
print myObject.increment(4)
print myObject.count
####### Unittest code.
import sys
import time
import unittest
from memoize import memoize
class testSampleUsages(unittest.TestCase):
# """This series of unit tests is to show the user how to apply memoize calls."""
def testSimpleUsageMemoize(self):
#memoize
def increment(var=0):
var += 1
return var
increment(3)
increment(3)
def testMethodBasedUsage(self):
"""Add the #memoize before method call."""
class myClass(object):
#memoize
def increment(self,var=0):
var += 1
return var
#memoize
def decrement(self, var=0):
var -=1
return var
myObj = myClass()
myObj.increment(3)
myObj.increment(3)
myObj.decrement(6)
myObj.decrement(6)
def testMultipleInstances(self):
#memoize
class myClass(object):
def __init__(self):
self.incrementCountCalls = 0
self.decrementCountCalls = 0
self.powCountCall = 0
# #memoize
def increment(self,var=0):
var += 1
self.incrementCountCalls+=1
return var
# #memoize
def decrement(self, var=0):
self.decrementCountCalls+=1
var -=1
return var
def pow(self, var=0):
self.powCountCall+=1
return var*var
obj1 = myClass() # Memoizing class above does not seem to work.
obj2 = myClass()
obj3 = myClass()
obj1.increment(3)
obj1.increment(3)
#obj2.increment(3)
#obj2.increment(3)
#obj3.increment(3)
#obj3.increment(3)
obj1.pow(4)
obj2.pow(4)
obj3.pow(4)

There's no way to attach a property to a single instance. Being descriptors, propertys must be part of a class definition in order to function. That means you can't easily add them to the partial object you create in __get__.
Now, you could create a class of your own to reimplement the behavior of partial with your added property. However, I suspect the limitation is actually to your benefit. If memo is applied to a method, its state is shared by all instances of the class (and perhaps even instances of subclasses). If you allow the caching details to be adjusted through instances, you might confuse users with cases like:
obj1 = basic()
print obj1.increment.getCacheTimeout() # prints the initial value, e.g. 120
obj2 = basic()
obj2.increment.setCacheTimeOut(20) # change the timeout value via another instance
print obj1.increment.getCacheTimeout() # the value via the first instance now prints 20
I suggest that you make the memoization-related interfaces of decorated methods accessible only through the class, not through instances. To make that work, you need to update your __get__ method to work if obj is None. It can simply return self:
def __get__(self, obj, objtype=None):
if obj is None:
return self
self.objtype = objtype
return partial(self, obj) # no need to attach our methods to the partial anymore
With this change, using a property on the memo via the class works:
basic.increment.cacheTimeOut = 20 # set property of the "unbound" method basic.increment

There is actually a way to accomplish this - by rebinding the decorator as instance-object with a call-method
class Helper(object):
def __init__(self, d, obj):
self.d = d
self.obj = obj
self.timeout = 0
def __call__(self, *args, **kwargs):
print self, self.timeout
return self.d.func(self.obj, *args, **kwargs)
class decorator(object):
def __init__(self, func):
self.func = func
self.name = func.__name__
def __get__(self, obj, clazz):
if object is not None:
obj.__dict__[self.name] = Helper(self, obj)
return obj.__dict__[self.name]
class Foo(object):
#decorator
def bar(self, args):
return args * 2
f = Foo()
g = Foo()
f.bar.timeout = 10
g.bar.timeout = 20
print f.bar(10)
print g.bar(20)
HTH

Python decorators in classes

Can one write something like:
class Test(object):
def _decorator(self, foo):
foo()
#self._decorator
def bar(self):
pass
This fails: self in #self is unknown
I also tried:
#Test._decorator(self)
which also fails: Test unknown
I would like to temporarily change some instance variables
in the decorator and then run the decorated method, before
changing them back.

Would something like this do what you need?
class Test(object):
def _decorator(foo):
def magic( self ) :
print "start magic"
foo( self )
print "end magic"
return magic
#_decorator
def bar( self ) :
print "normal call"
test = Test()
test.bar()
This avoids the call to self to access the decorator and leaves it hidden in the class namespace as a regular method.
>>> import stackoverflow
>>> test = stackoverflow.Test()
>>> test.bar()
start magic
normal call
end magic
>>>
edited to answer question in comments:
How to use the hidden decorator in another class
class Test(object):
def _decorator(foo):
def magic( self ) :
print "start magic"
foo( self )
print "end magic"
return magic
#_decorator
def bar( self ) :
print "normal call"
_decorator = staticmethod( _decorator )
class TestB( Test ):
#Test._decorator
def bar( self ):
print "override bar in"
super( TestB, self ).bar()
print "override bar out"
print "Normal:"
test = Test()
test.bar()
print
print "Inherited:"
b = TestB()
b.bar()
print
Output:
Normal:
start magic
normal call
end magic
Inherited:
start magic
override bar in
start magic
normal call
end magic
override bar out
end magic

What you're wanting to do isn't possible. Take, for instance, whether or not the code below looks valid:
class Test(object):
def _decorator(self, foo):
foo()
def bar(self):
pass
bar = self._decorator(bar)
It, of course, isn't valid since self isn't defined at that point. The same goes for Test as it won't be defined until the class itself is defined (which its in the process of). I'm showing you this code snippet because this is what your decorator snippet transforms into.
So, as you can see, accessing the instance in a decorator like that isn't really possible since decorators are applied during the definition of whatever function/method they are attached to and not during instantiation.
If you need class-level access, try this:
class Test(object):
#classmethod
def _decorator(cls, foo):
foo()
def bar(self):
pass
Test.bar = Test._decorator(Test.bar)

import functools
class Example:
def wrapper(func):
#functools.wraps(func)
def wrap(self, *args, **kwargs):
print("inside wrap")
return func(self, *args, **kwargs)
return wrap
#wrapper
def method(self):
print("METHOD")
wrapper = staticmethod(wrapper)
e = Example()
e.method()

This is one way to access(and have used) self from inside a decorator defined inside the same class:
class Thing(object):
def __init__(self, name):
self.name = name
def debug_name(function):
def debug_wrapper(*args):
self = args[0]
print 'self.name = ' + self.name
print 'running function {}()'.format(function.__name__)
function(*args)
print 'self.name = ' + self.name
return debug_wrapper
#debug_name
def set_name(self, new_name):
self.name = new_name
Output (tested on Python 2.7.10):
>>> a = Thing('A')
>>> a.name
'A'
>>> a.set_name('B')
self.name = A
running function set_name()
self.name = B
>>> a.name
'B'
The example above is silly, but it works.

Here's an expansion on Michael Speer's answer to take it a few steps further:
An instance method decorator which takes arguments and acts on a function with arguments and a return value.
class Test(object):
"Prints if x == y. Throws an error otherwise."
def __init__(self, x):
self.x = x
def _outer_decorator(y):
def _decorator(foo):
def magic(self, *args, **kwargs) :
print("start magic")
if self.x == y:
return foo(self, *args, **kwargs)
else:
raise ValueError("x ({}) != y ({})".format(self.x, y))
print("end magic")
return magic
return _decorator
#_outer_decorator(y=3)
def bar(self, *args, **kwargs) :
print("normal call")
print("args: {}".format(args))
print("kwargs: {}".format(kwargs))
return 27
And then
In [2]:
test = Test(3)
test.bar(
13,
'Test',
q=9,
lollipop=[1,2,3]
)
​
start magic
normal call
args: (13, 'Test')
kwargs: {'q': 9, 'lollipop': [1, 2, 3]}
Out[2]:
27
In [3]:
test = Test(4)
test.bar(
13,
'Test',
q=9,
lollipop=[1,2,3]
)
​
start magic
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-576146b3d37e> in <module>()
4 'Test',
5 q=9,
----> 6 lollipop=[1,2,3]
7 )
<ipython-input-1-428f22ac6c9b> in magic(self, *args, **kwargs)
11 return foo(self, *args, **kwargs)
12 else:
---> 13 raise ValueError("x ({}) != y ({})".format(self.x, y))
14 print("end magic")
15 return magic
ValueError: x (4) != y (3)

I found this question while researching a very similar problem. My solution is to split the problem into two parts. First, you need to capture the data that you want to associate with the class methods. In this case, handler_for will associate a Unix command with handler for that command's output.
class OutputAnalysis(object):
"analyze the output of diagnostic commands"
def handler_for(name):
"decorator to associate a function with a command"
def wrapper(func):
func.handler_for = name
return func
return wrapper
# associate mount_p with 'mount_-p.txt'
#handler_for('mount -p')
def mount_p(self, slurped):
pass
Now that we've associated some data with each class method, we need to gather that data and store it in a class attribute.
OutputAnalysis.cmd_handler = {}
for value in OutputAnalysis.__dict__.itervalues():
try:
OutputAnalysis.cmd_handler[value.handler_for] = value
except AttributeError:
pass

I use this type of decorator in some debugging situations, it allows overriding class properties by decorating, without having to find the calling function.
class myclass(object):
def __init__(self):
self.property = "HELLO"
#adecorator(property="GOODBYE")
def method(self):
print self.property
Here is the decorator code
class adecorator (object):
def __init__ (self, *args, **kwargs):
# store arguments passed to the decorator
self.args = args
self.kwargs = kwargs
def __call__(self, func):
def newf(*args, **kwargs):
#the 'self' for a method function is passed as args[0]
slf = args[0]
# replace and store the attributes
saved = {}
for k,v in self.kwargs.items():
if hasattr(slf, k):
saved[k] = getattr(slf,k)
setattr(slf, k, v)
# call the method
ret = func(*args, **kwargs)
#put things back
for k,v in saved.items():
setattr(slf, k, v)
return ret
newf.__doc__ = func.__doc__
return newf
Note: because I've used a class decorator you'll need to use #adecorator() with the brackets on to decorate functions, even if you don't pass any arguments to the decorator class constructor.

The simple way to do it.
All you need is to put the decorator method outside the class.
You can still use it inside.
def my_decorator(func):
#this is the key line. There's the aditional self parameter
def wrap(self, *args, **kwargs):
# you can use self here as if you were inside the class
return func(self, *args, **kwargs)
return wrap
class Test(object):
#my_decorator
def bar(self):
pass

Declare in inner class.
This solution is pretty solid and recommended.
class Test(object):
class Decorators(object):
#staticmethod
def decorator(foo):
def magic(self, *args, **kwargs) :
print("start magic")
foo(self, *args, **kwargs)
print("end magic")
return magic
#Decorators.decorator
def bar( self ) :
print("normal call")
test = Test()
test.bar()
The result:
>>> test = Test()
>>> test.bar()
start magic
normal call
end magic
>>>

Decorators seem better suited to modify the functionality of an entire object (including function objects) versus the functionality of an object method which in general will depend on instance attributes. For example:
def mod_bar(cls):
# returns modified class
def decorate(fcn):
# returns decorated function
def new_fcn(self):
print self.start_str
print fcn(self)
print self.end_str
return new_fcn
cls.bar = decorate(cls.bar)
return cls
#mod_bar
class Test(object):
def __init__(self):
self.start_str = "starting dec"
self.end_str = "ending dec"
def bar(self):
return "bar"
The output is:
>>> import Test
>>> a = Test()
>>> a.bar()
starting dec
bar
ending dec

I have a Implementation of Decorators that Might Help
import functools
import datetime
class Decorator(object):
def __init__(self):
pass
def execution_time(func):
#functools.wraps(func)
def wrap(self, *args, **kwargs):
""" Wrapper Function """
start = datetime.datetime.now()
Tem = func(self, *args, **kwargs)
end = datetime.datetime.now()
print("Exection Time:{}".format(end-start))
return Tem
return wrap
class Test(Decorator):
def __init__(self):
self._MethodName = Test.funca.__name__
#Decorator.execution_time
def funca(self):
print("Running Function : {}".format(self._MethodName))
return True
if __name__ == "__main__":
obj = Test()
data = obj.funca()
print(data)

You can decorate the decorator:
import decorator
class Test(object):
#decorator.decorator
def _decorator(foo, self):
foo(self)
#_decorator
def bar(self):
pass