Related
I don't really need to do this, but was just wondering, is there a way to bind a decorator to all functions within a class generically, rather than explicitly stating it for every function.
I suppose it then becomes a kind of aspect, rather than a decorator and it does feel a bit odd, but was thinking for something like timing or auth it'd be pretty neat.
The cleanest way to do this, or to do other modifications to a class definition, is to define a metaclass.
Alternatively, just apply your decorator at the end of the class definition using inspect:
import inspect
class Something:
def foo(self):
pass
for name, fn in inspect.getmembers(Something, inspect.isfunction):
setattr(Something, name, decorator(fn))
In practice of course you'll want to apply your decorator more selectively. As soon as you want to decorate all but one method you'll discover that it is easier and more flexible just to use the decorator syntax in the traditional way.
Everytime you think of changing class definition, you can either use the class decorator or metaclass. e.g. using metaclass
import types
class DecoMeta(type):
def __new__(cls, name, bases, attrs):
for attr_name, attr_value in attrs.iteritems():
if isinstance(attr_value, types.FunctionType):
attrs[attr_name] = cls.deco(attr_value)
return super(DecoMeta, cls).__new__(cls, name, bases, attrs)
#classmethod
def deco(cls, func):
def wrapper(*args, **kwargs):
print "before",func.func_name
result = func(*args, **kwargs)
print "after",func.func_name
return result
return wrapper
class MyKlass(object):
__metaclass__ = DecoMeta
def func1(self):
pass
MyKlass().func1()
Output:
before func1
after func1
Note: it will not decorate staticmethod and classmethod
Following code works for python2.x and 3.x
import inspect
def decorator_for_func(orig_func):
def decorator(*args, **kwargs):
print("Decorating wrapper called for method %s" % orig_func.__name__)
result = orig_func(*args, **kwargs)
return result
return decorator
def decorator_for_class(cls):
for name, method in inspect.getmembers(cls):
if (not inspect.ismethod(method) and not inspect.isfunction(method)) or inspect.isbuiltin(method):
continue
print("Decorating function %s" % name)
setattr(cls, name, decorator_for_func(method))
return cls
#decorator_for_class
class decorated_class:
def method1(self, arg, **kwargs):
print("Method 1 called with arg %s" % arg)
def method2(self, arg):
print("Method 2 called with arg %s" % arg)
d=decorated_class()
d.method1(1, a=10)
d.method2(2)
Update for Python 3:
import types
class DecoMeta(type):
def __new__(cls, name, bases, attrs):
for attr_name, attr_value in attrs.items():
if isinstance(attr_value, types.FunctionType):
attrs[attr_name] = cls.deco(attr_value)
return super().__new__(cls, name, bases, attrs)
#classmethod
def deco(cls, func):
def wrapper(*args, **kwargs):
print("before",func.__name__)
result = func(*args, **kwargs)
print("after",func.__name__)
return result
return wrapper
(and thanks to Duncan for this)
Of course that the metaclasses are the most pythonic way to go when you want to modify the way that python creates the objects. Which can be done by overriding the __new__ method of your class. But there are some points around this problem (specially for python 3.X) that I'd like to mention:
types.FunctionType doesn't protect the special methods from being decorated, as they are function types. As a more general way you can just decorate the objects which their names are not started with double underscore (__). One other benefit of this method is that it also covers those objects that exist in namespace and starts with __ but are not function like __qualname__, __module__ , etc.
The namespace argument in __new__'s header doesn't contain class attributes within the __init__. The reason is that the __new__ executes before the __init__ (initializing).
It's not necessary to use a classmethod as the decorator, as in most of the times you import your decorator from another module.
If your class is contain a global item (out side of the __init__) for refusing of being decorated alongside checking if the name is not started with __ you can check the type with types.FunctionType to be sure that you're not decorating a non-function object.
Here is a sample metacalss that you can use:
class TheMeta(type):
def __new__(cls, name, bases, namespace, **kwds):
# if your decorator is a class method of the metaclass use
# `my_decorator = cls.my_decorator` in order to invoke the decorator.
namespace = {k: v if k.startswith('__') else my_decorator(v) for k, v in namespace.items()}
return type.__new__(cls, name, bases, namespace)
Demo:
def my_decorator(func):
def wrapper(self, arg):
# You can also use *args instead of (self, arg) and pass the *args
# to the function in following call.
return "the value {} gets modified!!".format(func(self, arg))
return wrapper
class TheMeta(type):
def __new__(cls, name, bases, namespace, **kwds):
# my_decorator = cls.my_decorator (if the decorator is a classmethod)
namespace = {k: v if k.startswith('__') else my_decorator(v) for k, v in namespace.items()}
return type.__new__(cls, name, bases, namespace)
class MyClass(metaclass=TheMeta):
# a = 10
def __init__(self, *args, **kwargs):
self.item = args[0]
self.value = kwargs['value']
def __getattr__(self, attr):
return "This class hasn't provide the attribute {}.".format(attr)
def myfunction_1(self, arg):
return arg ** 2
def myfunction_2(self, arg):
return arg ** 3
myinstance = MyClass(1, 2, value=100)
print(myinstance.myfunction_1(5))
print(myinstance.myfunction_2(2))
print(myinstance.item)
print(myinstance.p)
Output:
the value 25 gets modified!!
the value 8 gets modified!!
1
This class hasn't provide the attribute p. # special method is not decorated.
For checking the 3rd item from the aforementioned notes you can uncomment the line a = 10 and do print(myinstance.a) and see the result then change the dictionary comprehension in __new__ as follows then see the result again:
namespace = {k: v if k.startswith('__') and not isinstance(v, types.FunctionType)\
else my_decorator(v) for k, v in namespace.items()}
I will repeat my answer here, for a similar issue
It can be done many different ways. I will show how to make it through meta-class, class decorator and inheritance.
by changing meta class
import functools
class Logger(type):
#staticmethod
def _decorator(fun):
#functools.wraps(fun)
def wrapper(*args, **kwargs):
print(fun.__name__, args, kwargs)
return fun(*args, **kwargs)
return wrapper
def __new__(mcs, name, bases, attrs):
for key in attrs.keys():
if callable(attrs[key]):
# if attrs[key] is callable, then we can easily wrap it with decorator
# and substitute in the future attrs
# only for extra clarity (though it is wider type than function)
fun = attrs[key]
attrs[key] = Logger._decorator(fun)
# and then invoke __new__ in type metaclass
return super().__new__(mcs, name, bases, attrs)
class A(metaclass=Logger):
def __init__(self):
self.some_val = "some_val"
def method_first(self, a, b):
print(a, self.some_val)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
# __init__ (<__main__.A object at 0x7f852a52a2b0>,) {}
b.method_first(5, b="Here should be 5")
# method_first (<__main__.A object at 0x7f852a52a2b0>, 5) {'b': 'Here should be 5'}
# 5 some_val
b.method_first(6, b="Here should be 6")
# method_first (<__main__.A object at 0x7f852a52a2b0>, 6) {'b': 'Here should be 6'}
# 6 some_val
b.another_method(7)
# another_method (<__main__.A object at 0x7f852a52a2b0>, 7) {}
# 7
b.static_method(7)
# 7
Also, will show two approaches how to make it without changing meta information of class (through class decorator and class inheritance). The first approach through class decorator put_decorator_on_all_methods accepts decorator to wrap all member callable objects of class.
def logger(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
def put_decorator_on_all_methods(decorator, cls=None):
if cls is None:
return lambda cls: put_decorator_on_all_methods(decorator, cls)
class Decoratable(cls):
def __init__(self, *args, **kargs):
super().__init__(*args, **kargs)
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
return decorator(value)
return value
return Decoratable
#put_decorator_on_all_methods(logger)
class A:
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(8)
# >>> static_method (8,) {}
# >>> 8
And, recently, I've come across on the same problem, but I couldn't put decorator on class or change it in any other way, except I was allowed to add such behavior through inheritance only (I am not sure that this is the best choice if you can change codebase as you wish though).
Here class Logger forces all callable members of subclasses to write information about their invocations, see code below.
class Logger:
def _decorator(self, f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
decorator = object.__getattribute__(self, '_decorator')
return decorator(value)
return value
class A(Logger):
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(7)
# >>> static_method (7,) {}
# >>> 7
Or more abstractly, you can instantiate base class based on some decorator.
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
class Decoratable:
def __init__(self, dec):
self._decorator = dec
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
decorator = object.__getattribute__(self, '_decorator')
return decorator(value)
return value
class A(Decoratable):
def __init__(self, dec):
super().__init__(dec)
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A(decorator)
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(7)
# >>> static_method (7,) {}
# >>> 7
There's another slightly similar thing you might want to do in some cases. Sometimes you want to trigger the attachment for something like debugging and not on all the classes but for every method of an object you might want a record of what it's doing.
def start_debugging():
import functools
import datetime
filename = "debug-{date:%Y-%m-%d_%H_%M_%S}.txt".format(date=datetime.datetime.now())
debug_file = open(filename, "a")
debug_file.write("\nDebug.\n")
def debug(func):
#functools.wraps(func)
def wrapper_debug(*args, **kwargs):
args_repr = [repr(a) for a in args] # 1
kwargs_repr = [f"{k}={v!r}" for k, v in kwargs.items()] # 2
signature = ", ".join(args_repr + kwargs_repr) # 3
debug_file.write(f"Calling {func.__name__}({signature})\n")
value = func(*args, **kwargs)
debug_file.write(f"{func.__name__!r} returned {value!r}\n") # 4
debug_file.flush()
return value
return wrapper_debug
for obj in (self):
for attr in dir(obj):
if attr.startswith('_'):
continue
fn = getattr(obj, attr)
if not isinstance(fn, types.FunctionType) and \
not isinstance(fn, types.MethodType):
continue
setattr(obj, attr, debug(fn))
This function will go through some objects (only self currently) and replace all functions and methods that do not start with _ with a debugging decorator.
The method used for this of just iterating the dir(self) is not addressed above but totally works. And can be called externally from the object and much more arbitrarily.
In Python 3 you could also write a simple function that overwrites/applies a decorator to certain methods like so:
from functools import wraps
from types import MethodType
def logged(func):
#wraps(func)
def wrapper(*args, **kwargs):
res = func(*args, **kwargs)
print("logging:", func.__name__, res)
return res
return wrapper
class Test:
def foo(self):
return 42
...
def aspectize(cls, decorator):
for name, func in cls.__dict__.items():
if not name.startswith("__"):
setattr(cls, name, MethodType(decorator(func), cls)) # MethodType is key
aspectize(Test, logged)
t = Test()
t.foo() # printing "logging: foo 42"; returning 42
I came to this question from:
How to decorate all functions of a class without typing it over and over for each method?
And I want add a one note:
Answers with class decorators or repalcing class methods like this one:
https://stackoverflow.com/a/6307868/11277611
Will not work with staticmethod.
You will get TypeError, unexpected argument because your method will get self/cls as first argument.
Probably:
Decorated class doesn't know about decorators of self methods and can't be distincted even with inspect.ismethod.
I come to such quickfix:
I'm not checked it closely but it passes my (no so comprehensive) tests.
Using dynamically decorators is already a bad approach, so, it must be okay as temporary solution.
TLD:TD Add try/exception to use with staticmethod
def log_sent_data(function):
#functools_wraps(function)
def decorator(*args, **kwargs):
# Quickfix
self, *args = args
try: # If method has self/cls/descriptor
result = function(self, *args, **kwargs)
except TypeError:
if args: # If method is static but has positional args
result = function(*args, **kwargs)
else: # If method is static and doesn't has positional args
result = function(**kwargs)
# End of quickfix
return result
return decorator
Combining information from various answers, here's a DecorateMethods metaclass:
class DecorateMethods(type):
""" Decorate all methods of the class with the decorator provided """
def __new__(cls, name, bases, attrs, **kwargs):
try:
decorator = kwargs['decorator']
except KeyError:
raise ValueError('Please provide the "decorator" argument, eg. '
'MyClass(..., metaclass=DecorateMethods, decorator=my_decorator)')
exclude = kwargs.get('exclude', [])
for attr_name, attr_value in attrs.items():
if isinstance(attr_value, types.FunctionType) and \
attr_name not in exclude and \
not attr_name.startswith('__'):
attrs[attr_name] = decorator(attr_value)
return super(DecorateMethods, cls).__new__(cls, name, bases, attrs)
Used as:
class MyClass(metaclass=DecorateMethods, decorator=my_decorator, exclude=["METHOD_TO_BE_EXCLUDED"]):
...
It works nicely with unittests, as opposed to function-based solutions.
Credit to answers in 1, 2 and other answers in this question.
You could override the __getattr__ method. It's not actually attaching a decorator, but it lets you return a decorated method. You'd probably want to do something like this:
class Eggs(object):
def __getattr__(self, attr):
return decorate(getattr(self, `_` + attr))
There's some ugly recursion hiding in there that you'll want to protect against, but that's a start.
I would like to know if there is an easy way to do some identical edits on several methods of a class. An example :
class Dog():
def __init__(self):
self.name = 'abc'
self.age = 1
def setName(self, newValue):
self.name = newValue
def setAge(self, newValue):
self.age = newValue
class TalkingDog(Dog):
def __init__(self):
super().__init__()
# The end is in pseudo code :
for method in TalkingDog.allMethods :
method = method + 'print('I have been edited !')'
I know that I can also overwrite each method but in a situation with tens of methods, that will be a little boring...
So I tried this :
class TalkingDog(Dog):
def __init__(self):
super().__init__()
for method in self.__dir__():
if method.startswith('set'):
oldMethod = getattr(self, method)
def _newMethod(newValue):
oldMethod(newValue)
print('I have been edited !')
setattr(self, method, _newMethod)
a = TalkingDog()
print(a.setName) >>> <function TalkingDog.__init__.<locals>._newMethod at 0x0000000002C350D0>
That almost works but setName is not anymore a method. It's an attribute which contains a function. I completely understand why but I'm trying to get a cleaner result. With that result, I risk of having problems later. For example I can't use the library pickle with that object (got the error _pickle.PicklingError: Can't pickle <function TalkingDog.__init__.<locals>._newMethod at 0x00000000003DCBF8>: attribute lookup _newMethod on __main__ failed).
The Pythonic way to do this is probably to use the descriptor protocol, which is also what properties use:
class VocalAttribute:
def __init__(self, name, feedback):
"""Called when you first create the descriptor."""
self.name = name # the name of the attribute 'behind' the property
self.feedback = feedback # the feedback to show when the value changes
def __get__(self, obj):
"""Called when you get the descriptor value."""
return getattr(obj, self.name)
def __set__(self, obj, value):
"""Called when you set the descriptor value."""
prev = getattr(obj, self.name, None)
if value != prev:
setattr(obj, self.name, value)
print(self.feedback)
def __delete__(self, obj):
"""Called when you delete the descriptor value."""
delattr(obj, self.name)
class Foo:
bar = VocalAttribute('_bar', 'I have been edited!')
foo = Foo()
print('1.')
foo.bar = 'hello'
print('2.')
foo.bar = 'hello'
print('3.')
foo.bar = 'world'
Output:
1.
I have been edited!
2.
3.
I have been edited!
Note that this only gives feedback when the new value is different to the old one - you can tweak the behaviour as needed in __set__. It also means you can directly read from and assign to foo.bar, rather than needing to call getters and setters (what is this, Java?)
since decorator could explicit called here a way to use it:
def updater(obj, call_back, call_back_args=(), call_back_kw=None, replace=False):
# ability to be called on the fly with different args and kw for the callback
# now it returns the updated obj (instance or class)
# but could a be factory returning a new obj in this case make a copy of obj, update this coy and return it
def update_function(fn, *args, **kw):
def wrapper(*args, **kw):
if replace:
# call only the callback
res = call_back(*call_back_args, **call_back_kw)
else:
res = fn(*args, **kw)
call_back(*call_back_args, **call_back_kw)
return res
return wrapper
# get all methods of the obj
# and apply update_function (a decorator) to all methods
for name, m in inspect.getmembers(
obj, predicate=lambda x: inspect.isfunction(x) or inspect.ismethod(x)):
# make the selection here
# could be made on the name for instance
if not name.startswith('_'):
new_m = update_function(m)
setattr(obj, name, new_m)
return obj
# declare a callback
def call_back(*args, **kw):
# simple callback
print("I have been edited and called with %r args and %r kw " % (args, kw))
a = Dog()
# could be called on instance or class
# apply the callback on all "public" methods
updater(
a,
call_back,
call_back_args=(2, 3, 4),
call_back_kw={"kw1": "v_1"}
)
When defining a decorator using a class, how do I automatically transfer over__name__, __module__ and __doc__? Normally, I would use the #wraps decorator from functools. Here's what I did instead for a class (this is not entirely my code):
class memoized:
"""Decorator that caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned, and
not re-evaluated.
"""
def __init__(self, func):
super().__init__()
self.func = func
self.cache = {}
def __call__(self, *args):
try:
return self.cache[args]
except KeyError:
value = self.func(*args)
self.cache[args] = value
return value
except TypeError:
# uncacheable -- for instance, passing a list as an argument.
# Better to not cache than to blow up entirely.
return self.func(*args)
def __repr__(self):
return self.func.__repr__()
def __get__(self, obj, objtype):
return functools.partial(self.__call__, obj)
__doc__ = property(lambda self:self.func.__doc__)
__module__ = property(lambda self:self.func.__module__)
__name__ = property(lambda self:self.func.__name__)
Is there a standard decorator to automate the creation of name module and doc? Also, to automate the get method (I assume that's for creating bound methods?) Are there any missing methods?
Everyone seems to have missed the obvious solution. Using functools.update_wrapper:
>>> import functools
>>> class memoized(object):
"""Decorator that caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned, and
not re-evaluated.
"""
def __init__(self, func):
self.func = func
self.cache = {}
functools.update_wrapper(self, func) ## TA-DA! ##
def __call__(self, *args):
pass # Not needed for this demo.
>>> #memoized
def fibonacci(n):
"""fibonacci docstring"""
pass # Not needed for this demo.
>>> fibonacci
<__main__.memoized object at 0x0156DE30>
>>> fibonacci.__name__
'fibonacci'
>>> fibonacci.__doc__
'fibonacci docstring'
I'm not aware of such things in stdlib, but we can create our own if we need to.
Something like this can work :
from functools import WRAPPER_ASSIGNMENTS
def class_wraps(cls):
"""Update a wrapper class `cls` to look like the wrapped."""
class Wrapper(cls):
"""New wrapper that will extend the wrapper `cls` to make it look like `wrapped`.
wrapped: Original function or class that is beign decorated.
assigned: A list of attribute to assign to the the wrapper, by default they are:
['__doc__', '__name__', '__module__', '__annotations__'].
"""
def __init__(self, wrapped, assigned=WRAPPER_ASSIGNMENTS):
self.__wrapped = wrapped
for attr in assigned:
setattr(self, attr, getattr(wrapped, attr))
super().__init__(wrapped)
def __repr__(self):
return repr(self.__wrapped)
return Wrapper
Usage:
#class_wraps
class memoized:
"""Decorator that caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned, and
not re-evaluated.
"""
def __init__(self, func):
super().__init__()
self.func = func
self.cache = {}
def __call__(self, *args):
try:
return self.cache[args]
except KeyError:
value = self.func(*args)
self.cache[args] = value
return value
except TypeError:
# uncacheable -- for instance, passing a list as an argument.
# Better to not cache than to blow up entirely.
return self.func(*args)
def __get__(self, obj, objtype):
return functools.partial(self.__call__, obj)
#memoized
def fibonacci(n):
"""fibonacci docstring"""
if n in (0, 1):
return n
return fibonacci(n-1) + fibonacci(n-2)
print(fibonacci)
print("__doc__: ", fibonacci.__doc__)
print("__name__: ", fibonacci.__name__)
Output:
<function fibonacci at 0x14627c0>
__doc__: fibonacci docstring
__name__: fibonacci
EDIT:
And if you are wondering why this wasn't included in the stdlib is because you can
wrap your class decorator in a function decorator and use functools.wraps like this:
def wrapper(f):
memoize = memoized(f)
#functools.wraps(f)
def helper(*args, **kws):
return memoize(*args, **kws)
return helper
#wrapper
def fibonacci(n):
"""fibonacci docstring"""
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
Turns out there's a straightforward solution using functools.wraps itself:
import functools
def dec(cls):
#functools.wraps(cls, updated=())
class D(cls):
decorated = 1
return D
#dec
class C:
"""doc"""
print(f'{C.__name__=} {C.__doc__=} {C.__wrapped__=}')
$ python3 t.py
C.__name__='C' C.__doc__='doc' C.__wrapped__=<class '__main__.C'>
Note that updated=() is needed to prevent an attempt to update the class's __dict__ (this output is without updated=()):
$ python t.py
Traceback (most recent call last):
File "t.py", line 26, in <module>
class C:
File "t.py", line 20, in dec
class D(cls):
File "/usr/lib/python3.8/functools.py", line 57, in update_wrapper
getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
AttributeError: 'mappingproxy' object has no attribute 'update'
I needed something that would wrap both classes and functions and wrote this:
def wrap_is_timeout(base):
'''Adds `.is_timeout=True` attribute to objects returned by `base()`.
When `base` is class, it returns a subclass with same name and adds read-only property.
Otherwise, it returns a function that sets `.is_timeout` attribute on result of `base()` call.
Wrappers make best effort to be transparent.
'''
if inspect.isclass(base):
class wrapped(base):
is_timeout = property(lambda _: True)
for k in functools.WRAPPER_ASSIGNMENTS:
v = getattr(base, k, _MISSING)
if v is not _MISSING:
try:
setattr(wrapped, k, v)
except AttributeError:
pass
return wrapped
#functools.wraps(base)
def fun(*args, **kwargs):
ex = base(*args, **kwargs)
ex.is_timeout = True
return ex
return fun
All we really need to do is modify the behavior of the decorator so that it is "hygienic", i.e. it is attribute-preserving.
#!/usr/bin/python3
def hygienic(decorator):
def new_decorator(original):
wrapped = decorator(original)
wrapped.__name__ = original.__name__
wrapped.__doc__ = original.__doc__
wrapped.__module__ = original.__module__
return wrapped
return new_decorator
This is ALL you need. In general. It doesn't preserve the signature, but if you really want that you can use a library to do that. I also went ahead and rewrote the memoization code so that it works on keyword arguments as well. Also there was a bug where failure to convert it to a hashable tuple would make it not work in 100% of cases.
Demo of rewritten memoized decorator with #hygienic modifying its behavior. memoized is now a function that wraps the original class, though you can (like the other answer) write a wrapping class instead, or even better, something which detects if it's a class and if so wraps the __init__ method.
#hygienic
class memoized:
def __init__(self, func):
self.func = func
self.cache = {}
def __call__(self, *args, **kw):
try:
key = (tuple(args), frozenset(kw.items()))
if not key in self.cache:
self.cache[key] = self.func(*args,**kw)
return self.cache[key]
except TypeError:
# uncacheable -- for instance, passing a list as an argument.
# Better to not cache than to blow up entirely.
return self.func(*args,**kw)
In action:
#memoized
def f(a, b=5, *args, keyword=10):
"""Intact docstring!"""
print('f was called!')
return {'a':a, 'b':b, 'args':args, 'keyword':10}
x=f(0)
#OUTPUT: f was called!
print(x)
#OUTPUT: {'a': 0, 'b': 5, 'keyword': 10, 'args': ()}
y=f(0)
#NO OUTPUT - MEANS MEMOIZATION IS WORKING
print(y)
#OUTPUT: {'a': 0, 'b': 5, 'keyword': 10, 'args': ()}
print(f.__name__)
#OUTPUT: 'f'
print(f.__doc__)
#OUTPUT: 'Intact docstring!'
Another solution using inheritance:
import functools
import types
class CallableClassDecorator:
"""Base class that extracts attributes and assigns them to self.
By default the extracted attributes are:
['__doc__', '__name__', '__module__'].
"""
def __init__(self, wrapped, assigned=functools.WRAPPER_ASSIGNMENTS):
for attr in assigned:
setattr(self, attr, getattr(wrapped, attr))
super().__init__()
def __get__(self, obj, objtype):
return types.MethodType(self.__call__, obj)
And, usage:
class memoized(CallableClassDecorator):
"""Decorator that caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned, and
not re-evaluated.
"""
def __init__(self, function):
super().__init__(function)
self.function = function
self.cache = {}
def __call__(self, *args):
try:
return self.cache[args]
except KeyError:
value = self.function(*args)
self.cache[args] = value
return value
except TypeError:
# uncacheable -- for instance, passing a list as an argument.
# Better to not cache than to blow up entirely.
return self.function(*args)
I don't really need to do this, but was just wondering, is there a way to bind a decorator to all functions within a class generically, rather than explicitly stating it for every function.
I suppose it then becomes a kind of aspect, rather than a decorator and it does feel a bit odd, but was thinking for something like timing or auth it'd be pretty neat.
The cleanest way to do this, or to do other modifications to a class definition, is to define a metaclass.
Alternatively, just apply your decorator at the end of the class definition using inspect:
import inspect
class Something:
def foo(self):
pass
for name, fn in inspect.getmembers(Something, inspect.isfunction):
setattr(Something, name, decorator(fn))
In practice of course you'll want to apply your decorator more selectively. As soon as you want to decorate all but one method you'll discover that it is easier and more flexible just to use the decorator syntax in the traditional way.
Everytime you think of changing class definition, you can either use the class decorator or metaclass. e.g. using metaclass
import types
class DecoMeta(type):
def __new__(cls, name, bases, attrs):
for attr_name, attr_value in attrs.iteritems():
if isinstance(attr_value, types.FunctionType):
attrs[attr_name] = cls.deco(attr_value)
return super(DecoMeta, cls).__new__(cls, name, bases, attrs)
#classmethod
def deco(cls, func):
def wrapper(*args, **kwargs):
print "before",func.func_name
result = func(*args, **kwargs)
print "after",func.func_name
return result
return wrapper
class MyKlass(object):
__metaclass__ = DecoMeta
def func1(self):
pass
MyKlass().func1()
Output:
before func1
after func1
Note: it will not decorate staticmethod and classmethod
Following code works for python2.x and 3.x
import inspect
def decorator_for_func(orig_func):
def decorator(*args, **kwargs):
print("Decorating wrapper called for method %s" % orig_func.__name__)
result = orig_func(*args, **kwargs)
return result
return decorator
def decorator_for_class(cls):
for name, method in inspect.getmembers(cls):
if (not inspect.ismethod(method) and not inspect.isfunction(method)) or inspect.isbuiltin(method):
continue
print("Decorating function %s" % name)
setattr(cls, name, decorator_for_func(method))
return cls
#decorator_for_class
class decorated_class:
def method1(self, arg, **kwargs):
print("Method 1 called with arg %s" % arg)
def method2(self, arg):
print("Method 2 called with arg %s" % arg)
d=decorated_class()
d.method1(1, a=10)
d.method2(2)
Update for Python 3:
import types
class DecoMeta(type):
def __new__(cls, name, bases, attrs):
for attr_name, attr_value in attrs.items():
if isinstance(attr_value, types.FunctionType):
attrs[attr_name] = cls.deco(attr_value)
return super().__new__(cls, name, bases, attrs)
#classmethod
def deco(cls, func):
def wrapper(*args, **kwargs):
print("before",func.__name__)
result = func(*args, **kwargs)
print("after",func.__name__)
return result
return wrapper
(and thanks to Duncan for this)
Of course that the metaclasses are the most pythonic way to go when you want to modify the way that python creates the objects. Which can be done by overriding the __new__ method of your class. But there are some points around this problem (specially for python 3.X) that I'd like to mention:
types.FunctionType doesn't protect the special methods from being decorated, as they are function types. As a more general way you can just decorate the objects which their names are not started with double underscore (__). One other benefit of this method is that it also covers those objects that exist in namespace and starts with __ but are not function like __qualname__, __module__ , etc.
The namespace argument in __new__'s header doesn't contain class attributes within the __init__. The reason is that the __new__ executes before the __init__ (initializing).
It's not necessary to use a classmethod as the decorator, as in most of the times you import your decorator from another module.
If your class is contain a global item (out side of the __init__) for refusing of being decorated alongside checking if the name is not started with __ you can check the type with types.FunctionType to be sure that you're not decorating a non-function object.
Here is a sample metacalss that you can use:
class TheMeta(type):
def __new__(cls, name, bases, namespace, **kwds):
# if your decorator is a class method of the metaclass use
# `my_decorator = cls.my_decorator` in order to invoke the decorator.
namespace = {k: v if k.startswith('__') else my_decorator(v) for k, v in namespace.items()}
return type.__new__(cls, name, bases, namespace)
Demo:
def my_decorator(func):
def wrapper(self, arg):
# You can also use *args instead of (self, arg) and pass the *args
# to the function in following call.
return "the value {} gets modified!!".format(func(self, arg))
return wrapper
class TheMeta(type):
def __new__(cls, name, bases, namespace, **kwds):
# my_decorator = cls.my_decorator (if the decorator is a classmethod)
namespace = {k: v if k.startswith('__') else my_decorator(v) for k, v in namespace.items()}
return type.__new__(cls, name, bases, namespace)
class MyClass(metaclass=TheMeta):
# a = 10
def __init__(self, *args, **kwargs):
self.item = args[0]
self.value = kwargs['value']
def __getattr__(self, attr):
return "This class hasn't provide the attribute {}.".format(attr)
def myfunction_1(self, arg):
return arg ** 2
def myfunction_2(self, arg):
return arg ** 3
myinstance = MyClass(1, 2, value=100)
print(myinstance.myfunction_1(5))
print(myinstance.myfunction_2(2))
print(myinstance.item)
print(myinstance.p)
Output:
the value 25 gets modified!!
the value 8 gets modified!!
1
This class hasn't provide the attribute p. # special method is not decorated.
For checking the 3rd item from the aforementioned notes you can uncomment the line a = 10 and do print(myinstance.a) and see the result then change the dictionary comprehension in __new__ as follows then see the result again:
namespace = {k: v if k.startswith('__') and not isinstance(v, types.FunctionType)\
else my_decorator(v) for k, v in namespace.items()}
I will repeat my answer here, for a similar issue
It can be done many different ways. I will show how to make it through meta-class, class decorator and inheritance.
by changing meta class
import functools
class Logger(type):
#staticmethod
def _decorator(fun):
#functools.wraps(fun)
def wrapper(*args, **kwargs):
print(fun.__name__, args, kwargs)
return fun(*args, **kwargs)
return wrapper
def __new__(mcs, name, bases, attrs):
for key in attrs.keys():
if callable(attrs[key]):
# if attrs[key] is callable, then we can easily wrap it with decorator
# and substitute in the future attrs
# only for extra clarity (though it is wider type than function)
fun = attrs[key]
attrs[key] = Logger._decorator(fun)
# and then invoke __new__ in type metaclass
return super().__new__(mcs, name, bases, attrs)
class A(metaclass=Logger):
def __init__(self):
self.some_val = "some_val"
def method_first(self, a, b):
print(a, self.some_val)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
# __init__ (<__main__.A object at 0x7f852a52a2b0>,) {}
b.method_first(5, b="Here should be 5")
# method_first (<__main__.A object at 0x7f852a52a2b0>, 5) {'b': 'Here should be 5'}
# 5 some_val
b.method_first(6, b="Here should be 6")
# method_first (<__main__.A object at 0x7f852a52a2b0>, 6) {'b': 'Here should be 6'}
# 6 some_val
b.another_method(7)
# another_method (<__main__.A object at 0x7f852a52a2b0>, 7) {}
# 7
b.static_method(7)
# 7
Also, will show two approaches how to make it without changing meta information of class (through class decorator and class inheritance). The first approach through class decorator put_decorator_on_all_methods accepts decorator to wrap all member callable objects of class.
def logger(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
def put_decorator_on_all_methods(decorator, cls=None):
if cls is None:
return lambda cls: put_decorator_on_all_methods(decorator, cls)
class Decoratable(cls):
def __init__(self, *args, **kargs):
super().__init__(*args, **kargs)
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
return decorator(value)
return value
return Decoratable
#put_decorator_on_all_methods(logger)
class A:
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(8)
# >>> static_method (8,) {}
# >>> 8
And, recently, I've come across on the same problem, but I couldn't put decorator on class or change it in any other way, except I was allowed to add such behavior through inheritance only (I am not sure that this is the best choice if you can change codebase as you wish though).
Here class Logger forces all callable members of subclasses to write information about their invocations, see code below.
class Logger:
def _decorator(self, f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
decorator = object.__getattribute__(self, '_decorator')
return decorator(value)
return value
class A(Logger):
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A()
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(7)
# >>> static_method (7,) {}
# >>> 7
Or more abstractly, you can instantiate base class based on some decorator.
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
print(f.__name__, args, kwargs)
return f(*args, **kwargs)
return wrapper
class Decoratable:
def __init__(self, dec):
self._decorator = dec
def __getattribute__(self, item):
value = object.__getattribute__(self, item)
if callable(value):
decorator = object.__getattribute__(self, '_decorator')
return decorator(value)
return value
class A(Decoratable):
def __init__(self, dec):
super().__init__(dec)
def method(self, a, b):
print(a)
def another_method(self, c):
print(c)
#staticmethod
def static_method(d):
print(d)
b = A(decorator)
b.method(5, b="Here should be 5")
# >>> method (5,) {'b': 'Here should be 5'}
# >>> 5
b.method(6, b="Here should be 6")
# >>> method (6,) {'b': 'Here should be 6'}
# >>> 6
b.another_method(7)
# >>> another_method (7,) {}
# >>> 7
b.static_method(7)
# >>> static_method (7,) {}
# >>> 7
There's another slightly similar thing you might want to do in some cases. Sometimes you want to trigger the attachment for something like debugging and not on all the classes but for every method of an object you might want a record of what it's doing.
def start_debugging():
import functools
import datetime
filename = "debug-{date:%Y-%m-%d_%H_%M_%S}.txt".format(date=datetime.datetime.now())
debug_file = open(filename, "a")
debug_file.write("\nDebug.\n")
def debug(func):
#functools.wraps(func)
def wrapper_debug(*args, **kwargs):
args_repr = [repr(a) for a in args] # 1
kwargs_repr = [f"{k}={v!r}" for k, v in kwargs.items()] # 2
signature = ", ".join(args_repr + kwargs_repr) # 3
debug_file.write(f"Calling {func.__name__}({signature})\n")
value = func(*args, **kwargs)
debug_file.write(f"{func.__name__!r} returned {value!r}\n") # 4
debug_file.flush()
return value
return wrapper_debug
for obj in (self):
for attr in dir(obj):
if attr.startswith('_'):
continue
fn = getattr(obj, attr)
if not isinstance(fn, types.FunctionType) and \
not isinstance(fn, types.MethodType):
continue
setattr(obj, attr, debug(fn))
This function will go through some objects (only self currently) and replace all functions and methods that do not start with _ with a debugging decorator.
The method used for this of just iterating the dir(self) is not addressed above but totally works. And can be called externally from the object and much more arbitrarily.
In Python 3 you could also write a simple function that overwrites/applies a decorator to certain methods like so:
from functools import wraps
from types import MethodType
def logged(func):
#wraps(func)
def wrapper(*args, **kwargs):
res = func(*args, **kwargs)
print("logging:", func.__name__, res)
return res
return wrapper
class Test:
def foo(self):
return 42
...
def aspectize(cls, decorator):
for name, func in cls.__dict__.items():
if not name.startswith("__"):
setattr(cls, name, MethodType(decorator(func), cls)) # MethodType is key
aspectize(Test, logged)
t = Test()
t.foo() # printing "logging: foo 42"; returning 42
I came to this question from:
How to decorate all functions of a class without typing it over and over for each method?
And I want add a one note:
Answers with class decorators or repalcing class methods like this one:
https://stackoverflow.com/a/6307868/11277611
Will not work with staticmethod.
You will get TypeError, unexpected argument because your method will get self/cls as first argument.
Probably:
Decorated class doesn't know about decorators of self methods and can't be distincted even with inspect.ismethod.
I come to such quickfix:
I'm not checked it closely but it passes my (no so comprehensive) tests.
Using dynamically decorators is already a bad approach, so, it must be okay as temporary solution.
TLD:TD Add try/exception to use with staticmethod
def log_sent_data(function):
#functools_wraps(function)
def decorator(*args, **kwargs):
# Quickfix
self, *args = args
try: # If method has self/cls/descriptor
result = function(self, *args, **kwargs)
except TypeError:
if args: # If method is static but has positional args
result = function(*args, **kwargs)
else: # If method is static and doesn't has positional args
result = function(**kwargs)
# End of quickfix
return result
return decorator
Combining information from various answers, here's a DecorateMethods metaclass:
class DecorateMethods(type):
""" Decorate all methods of the class with the decorator provided """
def __new__(cls, name, bases, attrs, **kwargs):
try:
decorator = kwargs['decorator']
except KeyError:
raise ValueError('Please provide the "decorator" argument, eg. '
'MyClass(..., metaclass=DecorateMethods, decorator=my_decorator)')
exclude = kwargs.get('exclude', [])
for attr_name, attr_value in attrs.items():
if isinstance(attr_value, types.FunctionType) and \
attr_name not in exclude and \
not attr_name.startswith('__'):
attrs[attr_name] = decorator(attr_value)
return super(DecorateMethods, cls).__new__(cls, name, bases, attrs)
Used as:
class MyClass(metaclass=DecorateMethods, decorator=my_decorator, exclude=["METHOD_TO_BE_EXCLUDED"]):
...
It works nicely with unittests, as opposed to function-based solutions.
Credit to answers in 1, 2 and other answers in this question.
You could override the __getattr__ method. It's not actually attaching a decorator, but it lets you return a decorated method. You'd probably want to do something like this:
class Eggs(object):
def __getattr__(self, attr):
return decorate(getattr(self, `_` + attr))
There's some ugly recursion hiding in there that you'll want to protect against, but that's a start.
I want to be able to ask a class's __init__ method what it's parameters are. The straightforward approach is the following:
cls.__init__.__func__.__code__.co_varnames[:code.co_argcount]
However, that won't work if the class has any decorators. It will give the parameter list for the function returned by the decorator. I want to get down to the original __init__ method and get those original parameters. In the case of a decorator, the decorator function is going to be found in the closure of the function returned by the decorator:
cls.__init__.__func__.__closure__[0]
However, it is more complicated if there are other things in the closure, which decorators may do from time to time:
def Something(test):
def decorator(func):
def newfunc(self):
stuff = test
return func(self)
return newfunc
return decorator
def test():
class Test(object):
#Something(4)
def something(self):
print Test
return Test
test().something.__func__.__closure__
(<cell at 0xb7ce7584: int object at 0x81b208c>, <cell at 0xb7ce7614: function object at 0xb7ce6994>)
And then I have to decide if I want to the parameters from decorator or the parameters from the original function. The function returned by the decorator could have *args and **kwargs for its parameters. What if there are multiple decorators and I have to decide which is the one I care about?
So what is the best way to find a function's parameters even when the function may be decorated? Also, what is the best way to go down a chain of decorators back to the decorated function?
Update:
Here is effectively how I am doing this right now (names have been changed to protect the identity of the accused):
import abc
import collections
IGNORED_PARAMS = ("self",)
DEFAULT_PARAM_MAPPING = {}
DEFAULT_DEFAULT_PARAMS = {}
class DICT_MAPPING_Placeholder(object):
def __get__(self, obj, type):
DICT_MAPPING = {}
for key in type.PARAMS:
DICT_MAPPING[key] = None
for cls in type.mro():
if "__init__" in cls.__dict__:
cls.DICT_MAPPING = DICT_MAPPING
break
return DICT_MAPPING
class PARAM_MAPPING_Placeholder(object):
def __get__(self, obj, type):
for cls in type.mro():
if "__init__" in cls.__dict__:
cls.PARAM_MAPPING = DEFAULT_PARAM_MAPPING
break
return DEFAULT_PARAM_MAPPING
class DEFAULT_PARAMS_Placeholder(object):
def __get__(self, obj, type):
for cls in type.mro():
if "__init__" in cls.__dict__:
cls.DEFAULT_PARAMS = DEFAULT_DEFAULT_PARAMS
break
return DEFAULT_DEFAULT_PARAMS
class PARAMS_Placeholder(object):
def __get__(self, obj, type):
func = type.__init__.__func__
# unwrap decorators here
code = func.__code__
keys = list(code.co_varnames[:code.co_argcount])
for name in IGNORED_PARAMS:
try: keys.remove(name)
except ValueError: pass
for cls in type.mro():
if "__init__" in cls.__dict__:
cls.PARAMS = tuple(keys)
break
return tuple(keys)
class BaseMeta(abc.ABCMeta):
def __init__(self, name, bases, dict):
super(BaseMeta, self).__init__(name, bases, dict)
if "__init__" not in dict:
return
if "PARAMS" not in dict:
self.PARAMS = PARAMS_Placeholder()
if "DEFAULT_PARAMS" not in dict:
self.DEFAULT_PARAMS = DEFAULT_PARAMS_Placeholder()
if "PARAM_MAPPING" not in dict:
self.PARAM_MAPPING = PARAM_MAPPING_Placeholder()
if "DICT_MAPPING" not in dict:
self.DICT_MAPPING = DICT_MAPPING_Placeholder()
class Base(collections.Mapping):
__metaclass__ = BaseMeta
"""
Dict-like class that uses its __init__ params for default keys.
Override PARAMS, DEFAULT_PARAMS, PARAM_MAPPING, and DICT_MAPPING
in the subclass definition to give non-default behavior.
"""
def __init__(self):
pass
def __nonzero__(self):
"""Handle bool casting instead of __len__."""
return True
def __getitem__(self, key):
action = self.DICT_MAPPING[key]
if action is None:
return getattr(self, key)
try:
return action(self)
except AttributeError:
return getattr(self, action)
def __iter__(self):
return iter(self.DICT_MAPPING)
def __len__(self):
return len(self.DICT_MAPPING)
print Base.PARAMS
# ()
print dict(Base())
# {}
At this point Base reports uninteresting values for the four contants and the dict version of instances is empty. However, if you subclass you can override any of the four, or you can include other parameters to the __init__:
class Sub1(Base):
def __init__(self, one, two):
super(Sub1, self).__init__()
self.one = one
self.two = two
Sub1.PARAMS
# ("one", "two")
dict(Sub1(1,2))
# {"one": 1, "two": 2}
class Sub2(Base):
PARAMS = ("first", "second")
def __init__(self, one, two):
super(Sub2, self).__init__()
self.first = one
self.second = two
Sub2.PARAMS
# ("first", "second")
dict(Sub2(1,2))
# {"first": 1, "second": 2}
Consider this decorator:
def rickroll(old_function):
return lambda junk, junk1, junk2: "Never Going To Give You Up"
class Foo(object):
#rickroll
def bar(self, p1, p2):
return p1 * p2
print Foo().bar(1, 2)
In it, the rickroll decorator takes the bar method, discards it, replaces it with a new function that ignores its differently-named (and possibly numbered!) parameters and instead returns a line from a classic song.
There are no further references to the original function, and the garbage collector can come and remove it any time it likes.
In such a case, I cannot see how you could find the parameter names p1 and p2. In my understanding, even the Python interpreter itself has no idea what they used to be called.