Related
I have to decorate a inherited method, but it decorates all inherited methods. Basically I have to create a decorator that will decorate just one method from the class.
The test looks like this
#my_decorator
class TestClass(Subclass):
pass
t = TestClass()
t.say_hi
Let's say my SubClass looks like this
class SubClass():
def __init__(self):
pass
def say_hi():
print("Hi")
def say_wow():
print("wow")
Now I have to make my_decorator, that has to decorate inherited function say_hi() to print("*****") before it prints "Hi"
I tried doing it like this, but than the decorator applies to all methods from SubClass
def my_decorator(cls)
def say_hi():
print("*********")
cls.say_hi()
return say_hi()
Naturally It applies to every function of the subclass, but how do I make it to apply to only a say_hi() function? -It also returns an TypeError "NoneType" object is not callable
First let us fix SubClass, because instance methods require an explicit instance parameter at definition time:
class SubClass():
def __init__(self):
pass
def say_hi(self):
print("Hi")
def say_wow(self):
print("wow")
Now you want the decorator to replace the say_hi method with a method that prints '****' before calling the original method. Le us write a decorator that just does that(*):
def my_decorator(cls):
orig = cls.say_hi # save the original method
def say_hi(self): # define a new one
print('****')
return orig(self) # ... calling the original method
cls.say_hi = say_hi # replace the method in the class
return cls
You can then use:
#my_decorator
class TestClass(SubClass):
pass
t = TestClass()
t.say_hi()
and get as expected:
****
Hi
(*) this is a very simple decorator that can only replace a say_hi(self) method: neither a different name, nor additional parameters, but decorators can be much smarter...
If you want to decorate a method, then decorate the method, not the class that contains it. If you want a new class, then the decorator applied to the class needs to return a class.
def print_banner(f):
def _(*args, **kwargs):
print("****")
f(*args, **kwargs)
return _
class SubClass():
def __init__(self):
pass
#print_banner
def say_hi(self, ):
print("Hi")
def say_wow(self):
print("wow")
Say I have a decorator like this:
def repeat(repeat_count):
def decorator(func):
def wrapped(self):
for X in range(repeat_count):
func() # Do Function
return wrapped
return decorator
and a class like this
class SomeClass(object):
def __init__(self, do_count):
self.some_method = repeat(do_count)(self.some_method)
def some_method(self):
print("I'm Doing Something")
Because a decorator just returns a method, it's clear that this works. However it unbinds the some_method function from the class instance, so I can no longer do something like:
>>> sc = SomeClass(10)
>>> sc.some_method()
# TypeError: wrapped() missing 1 required positional argument: 'self'
I get an exception because self is no longer automatically passed. To make this work I can simply do this:
sc.some_method(sc)
but I'd much rather not. Is there any way to rebind the method to the instance, preferably without any extra imports (as TypeMethod) would be able to accomplish.
I get an exception because self is no longer automatically passed.
Actually, it is still automatically passed. You got this exception because the way you defined the decorator required it to be passed twice.
From within the runtime of wrapped, func is already bound (i.e. it has already self passed). By defining wrapped to accept one positional argument, you've required to pass in self again, which is why sc.some_method(sc) worked correctly. The self is passed twice, as required - once implicitly, and once explicitly.
The smallest fix to your code is to remove self from the signature of wrapped, because that is already passed implicitly as per the descriptor protocol in the binding of self.some_method.
def repeat(repeat_count):
def decorator(func):
def wrapped():
for X in range(repeat_count):
func() # Do Function
return wrapped
return decorator
However, this is not really the best solution. You'll want to accept *args and **kwargs so your decorator can be applied regardless of the signature of the decorated function:
def repeat(repeat_count): # <-- the "decorator maker"
def decorator(func): # <-- the decorator
def wrapped(*args, **kwargs): # <-- this will replace "func"
for X in range(repeat_count):
func(*args, **kwargs) # <-- note: pass along the arguments!
return wrapped # <-- note: no call here!
return decorator
For a rather simple case, where you don't need to access self from the decorator itself, you can simply use the following (which is pretty much what you already did, minus the wrapper function invocation, and the passing of self).
The method passed to the decorator is already bounded to self when you assign
self.some_method = repeat(do_count)(self.some_method)
The full code:
def repeat(repeat_count):
def decorator(func):
def wrapped():
for X in range(repeat_count):
func()
return wrapped
return decorator
class SomeClass(object):
def __init__(self, do_count):
self.a = 3
self.some_method = repeat(do_count)(self.some_method)
def some_method(self): print("Accesing my a from inside: %d" % self.a)
sc = SomeClass(5)
sc.some_method()
output:
Accesing my a from inside: 3
Accesing my a from inside: 3
Accesing my a from inside: 3
Accesing my a from inside: 3
Accesing my a from inside: 3
I want a decorator that would add the decorated function to list, like this :
class My_Class(object):
def __init__(self):
self.list=[]
#decorator
def my_function(self)
print 'Hi'
I expect my_function to be added to self.list, but I just can't write this decorator. If I try to write it inside My_Class, then I would have to use #self.decorator, and self does not exist since we're outside any function. And if I try to write it out of My_Class, then I can't retrieve self from my_function.
I know quite similar questions exist, but they are overly complicated, and I'm just learning python and decorators.
You can't access self from the decorator, because the decorator is run at the time the function is defined, and at that time there are no instances of My_Class yet.
It's better to put the function list as a class attribute instead of an instance attribute. Then you can pass this list as a parameter to the decorator:
def addToList(funcList):
'''Decorator that adds the function to a given list'''
def actual_decorator(f):
funcList.append(f)
return f
return actual_decorator
class MyClass(object):
funcList = []
#addToList(funcList)
def some_function(self, name):
print 'Hello,', name
Now you can access MyClass.funcList to get the list of decorated functions.
There's nothing really special about writing decorators for bound functions (instance methods). For example, this simple example works fine:
def decorator(fn):
print "I'm decorating!"
return fn
class MyClass(object):
def __init__(self):
self.list = []
#decorator
def my_function(self):
print "Hi"
If you want to use self in your decorator, you'll treat your decorator the same as you'd treat any decorator that uses the function's args:
def decorator(fn):
def _decorator(self):
print "I'm decorating, and here's my list: %s!" % self.list
return fn(self)
return _decorator
Your list attribute should be a class attribute (and it should be renamed, because list is a builtin type). Then you can do something like this:
my_methods = []
def my_method(method):
my_methods.append(method)
return method
class MyClass(object):
my_methods = my_methods
#my_method
def my_function(self):
print 'Hi'
I'd like to create a Python class decorator (*) that would be able to seamlessly wrap all method types the class might have: instance, class and static.
This is the code I have for now, with the parts that break it commented:
def wrapItUp(method):
def wrapped(*args, **kwargs):
print "This method call was wrapped!"
return method(*args, **kwargs)
return wrapped
dundersICareAbout = ["__init__", "__str__", "__repr__"]#, "__new__"]
def doICareAboutThisOne(cls, methodName):
return (callable(getattr(cls, methodName))
and (not (methodName.startswith("__") and methodName.endswith("__"))
or methodName in dundersICareAbout))
def classDeco(cls):
myCallables = ((aname, getattr(cls, aname)) for aname in dir(cls) if doICareAboutThisOne(cls, aname))
for name, call in myCallables:
print "*** Decorating: %s.%s(...)" % (cls.__name__, name)
setattr(cls, name, wrapItUp(call))
return cls
#classDeco
class SomeClass(object):
def instanceMethod(self, p):
print "instanceMethod: p =", p
#classmethod
def classMethod(cls, p):
print "classMethod: p =", p
#staticmethod
def staticMethod(p):
print "staticMethod: p =", p
instance = SomeClass()
instance.instanceMethod(1)
#SomeClass.classMethod(2)
#instance.classMethod(2)
#SomeClass.staticMethod(3)
#instance.staticMethod(3)
I'm having two issues trying to make this work:
When iterating over all callables, how do I find out if it is of an instance, class or static type?
How to I overwrite the method with a proper wrapped version of it that is invoked correctly for each of those cases?
Currently, this code generates different TypeErrors depending on what commented snippet is uncommented, like:
TypeError: unbound method wrapped() must be called with SomeClass instance as first argument (got int instance instead)
TypeError: classMethod() takes exactly 2 arguments (3 given)
(*): The same problem is much simpler if you're decorating the methods directly.
Because methods are wrappers for functions, to apply a decorator to a method on a class after the class has been constructed, you have to:
Extract the underlying function from the method using its im_func attribute.
Decorate the function.
Re-apply the wrapper.
Overwrite the attribute with the wrapped, decorated function.
It is difficult to distinguish a classmethod from a regular method once the #classmethod decorator has been applied; both kinds of methods are of type instancemethod. However, you can check the im_self attribute and see whether it is None. If so, it's a regular instance method; otherwise it's a classmethod.
Static methods are simple functions (the #staticmethod decorator merely prevents the usual method wrapper from being applied). So you don't have to do anything special for these, it looks like.
So basically your algorithm looks like this:
Get the attribute.
Is it callable? If not, proceed to the next attribute.
Is its type types.MethodType? If so, it is either a class method or an instance method.
If its im_self is None, it is an instance method. Extract the underlying function via the im_func attribute, decorate that, and re-apply the instance method: meth = types.MethodType(func, None, cls)
If its im_self is not None, it is a class method. Exctract the underlying function via im_func and decorate that. Now you have to reapply the classmethod decorator but you can't because classmethod() doesn't take a class, so there's no way to specify what class it will be attached to. Instead you have to use the instance method decorator: meth = types.MethodType(func, cls, type). Note that the type here is the actual built-in, type.
If its type is not types.MethodType then it is a static method or other non-bound callable, so just decorate it.
Set the new attribute back onto the class.
These change somewhat in Python 3 -- unbound methods are functions there, IIRC. In any case this will probably need to be completely rethought there.
There is an undocumented function, inspect.classify_class_attrs, which can tell you which attributes are classmethods or staticmethods. Under the hood, it uses isinstance(obj, staticmethod) and isinstance(obj, classmethod) to classify static and class methods. Following that pattern, this works in both Python2 and Python3:
def wrapItUp(method,kind='method'):
if kind=='static method':
#staticmethod
def wrapped(*args, **kwargs):
return _wrapped(*args,**kwargs)
elif kind=='class method':
#classmethod
def wrapped(cls,*args, **kwargs):
return _wrapped(*args,**kwargs)
else:
def wrapped(self,*args, **kwargs):
return _wrapped(self,*args,**kwargs)
def _wrapped(*args, **kwargs):
print("This method call was wrapped!")
return method(*args, **kwargs)
return wrapped
def classDeco(cls):
for name in (name
for name in dir(cls)
if (callable(getattr(cls,name))
and (not (name.startswith('__') and name.endswith('__'))
or name in '__init__ __str__ __repr__'.split()))
):
method = getattr(cls, name)
obj = cls.__dict__[name] if name in cls.__dict__ else method
if isinstance(obj, staticmethod):
kind = "static method"
elif isinstance(obj, classmethod):
kind = "class method"
else:
kind = "method"
print("*** Decorating: {t} {c}.{n}".format(
t=kind,c=cls.__name__,n=name))
setattr(cls, name, wrapItUp(method,kind))
return cls
#classDeco
class SomeClass(object):
def instanceMethod(self, p):
print("instanceMethod: p = {}".format(p))
#classmethod
def classMethod(cls, p):
print("classMethod: p = {}".format(p))
#staticmethod
def staticMethod(p):
print("staticMethod: p = {}".format(p))
instance = SomeClass()
instance.instanceMethod(1)
SomeClass.classMethod(2)
instance.classMethod(2)
SomeClass.staticMethod(3)
instance.staticMethod(3)
I am trying to write a decorator to do logging:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f():
pass
C().f()
I would like this to print:
Entering C.f
but instead I get this error message:
AttributeError: 'function' object has no attribute 'im_class'
Presumably this is something to do with the scope of 'myFunc' inside 'logger', but I've no idea what.
Claudiu's answer is correct, but you can also cheat by getting the class name off of the self argument. This will give misleading log statements in cases of inheritance, but will tell you the class of the object whose method is being called. For example:
from functools import wraps # use this to preserve function signatures and docstrings
def logger(func):
#wraps(func)
def with_logging(*args, **kwargs):
print "Entering %s.%s" % (args[0].__class__.__name__, func.__name__)
return func(*args, **kwargs)
return with_logging
class C(object):
#logger
def f(self):
pass
C().f()
As I said, this won't work properly in cases where you've inherited a function from a parent class; in this case you might say
class B(C):
pass
b = B()
b.f()
and get the message Entering B.f where you actually want to get the message Entering C.f since that's the correct class. On the other hand, this might be acceptable, in which case I'd recommend this approach over Claudiu's suggestion.
Functions only become methods at runtime. That is, when you get C.f you get a bound function (and C.f.im_class is C). At the time your function is defined it is just a plain function, it is not bound to any class. This unbound and disassociated function is what is decorated by logger.
self.__class__.__name__ will give you the name of the class, but you can also use descriptors to accomplish this in a somewhat more general way. This pattern is described in a blog post on Decorators and Descriptors, and an implementation of your logger decorator in particular would look like:
class logger(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, type=None):
return self.__class__(self.func.__get__(obj, type))
def __call__(self, *args, **kw):
print 'Entering %s' % self.func
return self.func(*args, **kw)
class C(object):
#logger
def f(self, x, y):
return x+y
C().f(1, 2)
# => Entering <bound method C.f of <__main__.C object at 0x...>>
Obviously the output can be improved (by using, for example, getattr(self.func, 'im_class', None)), but this general pattern will work for both methods and functions. However it will not work for old-style classes (but just don't use those ;)
Ideas proposed here are excellent, but have some disadvantages:
inspect.getouterframes and args[0].__class__.__name__ are not suitable for plain functions and static-methods.
__get__ must be in a class, that is rejected by #wraps.
#wraps itself should be hiding traces better.
So, I've combined some ideas from this page, links, docs and my own head,
and finally found a solution, that lacks all three disadvantages above.
As a result, method_decorator:
Knows the class the decorated method is bound to.
Hides decorator traces by answering to system attributes more correctly than functools.wraps() does.
Is covered with unit-tests for bound an unbound instance-methods, class-methods, static-methods, and plain functions.
Usage:
pip install method_decorator
from method_decorator import method_decorator
class my_decorator(method_decorator):
# ...
See full unit-tests for usage details.
And here is just the code of the method_decorator class:
class method_decorator(object):
def __init__(self, func, obj=None, cls=None, method_type='function'):
# These defaults are OK for plain functions
# and will be changed by __get__() for methods once a method is dot-referenced.
self.func, self.obj, self.cls, self.method_type = func, obj, cls, method_type
def __get__(self, obj=None, cls=None):
# It is executed when decorated func is referenced as a method: cls.func or obj.func.
if self.obj == obj and self.cls == cls:
return self # Use the same instance that is already processed by previous call to this __get__().
method_type = (
'staticmethod' if isinstance(self.func, staticmethod) else
'classmethod' if isinstance(self.func, classmethod) else
'instancemethod'
# No branch for plain function - correct method_type for it is already set in __init__() defaults.
)
return object.__getattribute__(self, '__class__')( # Use specialized method_decorator (or descendant) instance, don't change current instance attributes - it leads to conflicts.
self.func.__get__(obj, cls), obj, cls, method_type) # Use bound or unbound method with this underlying func.
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)
def __getattribute__(self, attr_name): # Hiding traces of decoration.
if attr_name in ('__init__', '__get__', '__call__', '__getattribute__', 'func', 'obj', 'cls', 'method_type'): # Our known names. '__class__' is not included because is used only with explicit object.__getattribute__().
return object.__getattribute__(self, attr_name) # Stopping recursion.
# All other attr_names, including auto-defined by system in self, are searched in decorated self.func, e.g.: __module__, __class__, __name__, __doc__, im_*, func_*, etc.
return getattr(self.func, attr_name) # Raises correct AttributeError if name is not found in decorated self.func.
def __repr__(self): # Special case: __repr__ ignores __getattribute__.
return self.func.__repr__()
It seems that while the class is being created, Python creates regular function objects. They only get turned into unbound method objects afterwards. Knowing that, this is the only way I could find to do what you want:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
def f(self):
pass
C.f = logger(C.f)
C().f()
This outputs the desired result.
If you want to wrap all the methods in a class, then you probably want to create a wrapClass function, which you could then use like this:
C = wrapClass(C)
Class functions should always take self as their first argument, so you can use that instead of im_class.
def logger(myFunc):
def new(self, *args, **keyargs):
print 'Entering %s.%s' % (self.__class__.__name__, myFunc.__name__)
return myFunc(self, *args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
at first I wanted to use self.__name__ but that doesn't work because the instance has no name. you must use self.__class__.__name__ to get the name of the class.
I found another solution to a very similar problem using the inspect library. When the decorator is called, even though the function is not yet bound to the class, you can inspect the stack and discover which class is calling the decorator. You can at least get the string name of the class, if that is all you need (probably can't reference it yet since it is being created). Then you do not need to call anything after the class has been created.
import inspect
def logger(myFunc):
classname = inspect.getouterframes(inspect.currentframe())[1][3]
def new(*args, **keyargs):
print 'Entering %s.%s' % (classname, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
While this is not necessarily better than the others, it is the only way I can figure out to discover the class name of the future method during the call to the decorator. Make note of not keeping references to frames around in the inspect library documentation.
As shown in Asa Ayers' answer, you don't need to access the class object. It may be worth to know that since Python 3.3, you can also use __qualname__, which gives you the fully qualified name:
>>> def logger(myFunc):
... def new(*args, **keyargs):
... print('Entering %s' % myFunc.__qualname__)
... return myFunc(*args, **keyargs)
...
... return new
...
>>> class C(object):
... #logger
... def f(self):
... pass
...
>>> C().f()
Entering C.f
This has the added advantage of working also in the case of nested classes, as shown in this example taken from PEP 3155:
>>> class C:
... def f(): pass
... class D:
... def g(): pass
...
>>> C.__qualname__
'C'
>>> C.f.__qualname__
'C.f'
>>> C.D.__qualname__
'C.D'
>>> C.D.g.__qualname__
'C.D.g'
Notice also that in Python 3 the im_class attribute is gone, therefore if you really wish to access the class in a decorator, you need an other method. The approach I currently use involves object.__set_name__ and is detailed in my answer to "Can a Python decorator of an instance method access the class?"
You can also use new.instancemethod() to create an instance method (either bound or unbound) from a function.
Instead of injecting decorating code at definition time, when function doesn't know it's class, delay running this code until function is accessed/called. Descriptor object facilitates injecting own code late, at access/call time:
class decorated(object):
def __init__(self, func, type_=None):
self.func = func
self.type = type_
def __get__(self, obj, type_=None):
return self.__class__(self.func.__get__(obj, type_), type_)
def __call__(self, *args, **kwargs):
name = '%s.%s' % (self.type.__name__, self.func.__name__)
print('called %s with args=%s kwargs=%s' % (name, args, kwargs))
return self.func(*args, **kwargs)
class Foo(object):
#decorated
def foo(self, a, b):
pass
Now we can inspect class both at access time (__get__) and at call time (__call__). This mechanism works for plain methods as well as static|class methods:
>>> Foo().foo(1, b=2)
called Foo.foo with args=(1,) kwargs={'b': 2}
Full example at: https://github.com/aurzenligl/study/blob/master/python-robotwrap/Example4.py