Suppose I have defined:
def to_class(cls):
""" returns a decorator
aimed to force the result to be of class cls. """
def decorating_func(func):
def wrapper(*args, **kwargs):
return cls(func(*args, **kwargs))
return wrapper
return decorator(decorating_func)
I wish to use it to create decorators which turn function results to objects of a given class. However, this will not work:
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#staticmethod
#to_test_class
def test_func(value):
return value
to_test_class = to_class(TestClass)
as test_func will look for to_test_class and will not find it. On the other hand, putting the assignment to to_test_class before the class definition will fail as well, as TestClass will not be defined yet.
Trying to put #to_class(TestClass) above the definition of test_func will also fail, as the method is constructed before the class (if I am not wrong).
The only workaround I have found is to define to_test_class manually as a decorator, and not as one returned from the general "to_class" def.
It might be important to mention that this is only a basic example, but I wish to use to_class for many applications, such as modifying the returned value before 'plugging' it into the class' constructor; and I wish to use it as a decorator for other class' methods as well.
I am sure some think a "to_class" decorator is pointless; manipulations can be done within the decorated method, instead. Though, I find it convenient, and it helps me with readability.
Finally I wish to add that this interests me 20% for practical reasons and 80% for studying reasons, as I find this is something I do not fully understand about decorators in Python in general.
Indeed, at class construction time, the class object itself has not yet been constructed, thus you cannot use it as the basis of a decorator.
One work-around I can think of, is to not use the staticmethod decorator. Instead, internally in your own decorator, re-use the classmethod decorator. That way you ensure that Python at the very least passes in the associated class for you:
def to_class(func):
""" returns a decorator
aimed to force the result to be of class cls. """
def wrapper(cls, *args, **kwargs):
return cls(func(*args, **kwargs))
return classmethod(wrapper)
Then use it like this:
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#to_class
def test_func(value):
return value
Demonstration:
>>> def to_class(func):
... """ returns a decorator
... aimed to force the result to be of class cls. """
... def wrapper(cls, *args, **kwargs):
... return cls(func(*args, **kwargs))
... return classmethod(wrapper)
...
>>> class TestClass(object):
... def __init__(self, value):
... self._value = (value, value)
... def __str__(self):
... return str(self._value)
... #to_class
... def test_func(value):
... return value
...
>>> TestClass.test_func('foo')
<__main__.TestClass object at 0x102a77210>
>>> print TestClass.test_func('foo')
('foo', 'foo')
A generic version of your decorator is not easy; the only other workaround to your conundrum is to use a metaclass hack; see another answer of mine where I describe the method in more detail.
You basically need to reach into the class-under-construction namespace, set a temporary metaclass, and then rely on there being at least one instance of the class before your decorator will work; the temporary metaclass approach hooks into the class creation mechanisms to retrieve the constructed class at a later time.
Seeing as you are using this decorator as an alternative class factory however, that is probably not going to be ideal; if someone used your decorated functions to create class instances exclusively the metaclass would be called too late.
Well, you forgot that class is the first parameter passed to method decorated with classmethod, so you can write it like this:
def to_this_class(func):
def wrapped(cls, value):
res = func(cls, value)
return cls(res)
return wrapped
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#classmethod
#to_this_class
def test_func(cls, value):
return value
x = TestClass('a')
print x.test_func('b')
The problem is that a decorator gets evaluated upon defining the thing it decorates, so when defining the method test_func(), the decorator to_test_class gets called, and even if it already exists, the thing it shall work on (the class TestClass) does not exist yet (as this is created after all methods are created).
Maybe you can use a placeholder at the point where the class is used and later (after the class is created) fill in that value (the class) at the point of the placeholder.
Example:
lazyClasses = {}
def to_lazy_class(className):
""" returns a decorator
aimed to force the result to be of class cls. """
def decorating_func(func):
def wrapper(*args, **kwargs):
return lazyClasses[className](func(*args, **kwargs))
return wrapper
return decorating_func
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#staticmethod
#to_lazy_class('TestClass')
def test_func(value):
return value
lazyClasses['TestClass'] = TestClass
>>> TestClass.test_func('hallo')
<__main__.TestClass object at 0x7f76d8cba190>
Related
In my undertstanding, decorator class should contain __call__ or __new__ method. But cached_property in cpython repo doesn't follow the rules. Can anyone explain it for me?
class cached_property:
def __init__(self, func):
xxx
def __set_name__(self, owner, name):
xxx
def __get__(self, instance, owner=None):
xxx
__class_getitem__ = classmethod(GenericAlias)
Do all decorator classes need __call__?
decorator class should contain __call__ or __new__ method
Not all decorator classes need to implement __call__.
It's only required when we want to call the decorated object with ().
A decorator class that takes a callable to produce a callable has to implement __call__.
In this example, __call__ is implemented because we want to do data.calculate().
# Decorator to call and cache the function immediately
class PreCompute:
def __init__(self, func):
self.value = func()
def __call__(self, *args, **kwds):
return self.value
class Data:
#PreCompute
def calculate():
print("Data.calculate called")
return 42
data = Data()
# This actually calls PreCompute's __call__
print(data.calculate())
The definition of class Data here is roughly desugared to something like this,
so when calling data.calculate() we're actually calling the __call__ function from class PreCompute.
class Data:
def calculate():
print("Data.calculate called")
return 42
calculate = PreCompute(calculate)
A decorator class that takes a callable but does not produce a callable does not have to implement __call__.
For example, we can modify the class Precompute decorator to the following code, which allows us to access data.calculate as if it's an attribute.
For more information about what __get__ does, see Descriptor HowTo Guide from Python docs.
class PreCompute:
def __init__(self, func):
self.value = func()
def __get__(self, instance, owner):
return self.value
class Data:
#PreCompute
def calculate():
print("Data.calculate called")
return 42
data = Data()
# Access .calculate like an attribute
print(data.calculate)
What about __new__?
I'm not sure how OP got the impression that decorator classes must define either __call__ or __new__. I've seen __new__ being defined for use cases like #singleton decorator for classes, but as discussed in the previous section about __call__, this is also not strictly required. The only function we must define is an __init__ that receives the object to be decorated.
How does #functools.cached_property work, then?
Now going back to the question, notice from the documentation of #functools.cached_property that
it "transform a method of a class into a property", which is to be accessed without the parentheses ().
Therefore, class cached_property implements __get__ but not __call__, which is similar to the second example above.
I’m trying to create a decorator that is called within a class, which would pull attributes from that class, and use those class attributes to edit the function’s docstring.
My problem is that I have found examples of decorators that edit the docstring of the function (setting the function's __doc__ attribute equal to a new string), and I have also found examples of decorators that pull attributes from the parent class (by passing self into the decorator), but I haven’t been able to find an example of a decorator that is able to do both.
I have tried to combine these two examples, but it isn't working:
def my_decorator(func):
def wrapper(self, *args, **kwargs):
name = func.__name__ # pull function name
cls = self.__class__.__name__ # pull class name
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), name, cls) # set them to docstring
return func(self, *args, **kwargs)
return wrapper
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
With this, I would hope that the following would return "Docstring is now new for the function: example":
Test().example.__doc__
Instead it returns None.
Edit: Note that I am not interested in how to access the name of the class specifically, so much as how to access the class attributes in general (where here self.__class__.__name__ is used as an example).
example is replaced with wrapper; the decoration is equivalent to
def example(self, examplearg=1):
"""Docstring"""
pass
example = my_decorator(example)
so you need to set wrapper.__doc__, not func.__doc__.
def my_decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for the function {}".format(
str(func.__doc__),
func.__name__)
return wrapper
Note that at the time you call my_decorator, you don't have any information about what class the decorated function/method belongs to. You would have to pass its name explicitly:
def my_decorator(cls_name):
def _decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for function {} in class {}".format(
func.__doc__,
func.__name__,
cls_name)
return wrapper
return _decorator
class Test():
#my_decorator("Test")
def example(self, examplearg=1):
"""Docstring"""
# or
# def example(self, examplearg=1):
# """Docstring"""
#
# example = my_decorator("Test")(example)
You can simply modify the __doc__ attribute when the decorator is called instead, and use the first token of the dot-delimited __qualname__ attribute of the function to obtain the class name:
def my_decorator(func):
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), func.__name__, func.__qualname__.split('.')[0])
return func
so that:
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
print(Test().example.__doc__)
would output:
Docstring is new for the function example in class Test
Turns out that accessing class attributes from within a class is impossible, as the class has yet to be executed when the decorator is called. So the original goal - using a decorator within a class to access class attributes - does not seem to be possible.
However, thanks to jdehesa for pointing me to a workaround that allows access to the class attributes using a class decorator, here: Can a Python decorator of an instance method access the class?.
I was able to use the class decorator to alter the specific method's docstring using class attributes like so:
def class_decorator(cls):
for name, method in cls.__dict__.items():
if name == 'example':
# do something with the method
method.__doc__ = "{} is new for function {} in class {}".format(method.__doc__, name, cls.__name__)
# Note that other class attributes such as cls.__base__
# can also be accessed in this way
return cls
#class_decorator
class Test():
def example(self, examplearg=1):
"""Docstring"""
print(Test().example.__doc__)
# Returns "Docstring is new for function example in class Test"
I have one django model which needs to do some processing referring the custom user model.
I can't work with the class of this model at class loading time because the loading order of the classes is unknown.
So I need to add some class attributes at runtime, at the moment I'm adding them in the __init__ or __new__ like:
def __new__(cls, *args, **kwargs):
# hack to avoid INSTALLED_APPS initialization conflicts.
# get_user_model() can't be called from this module at class loading time,
# so some class attributes must be added later.
# Metaclasses could me more appropiate but I don't want to override
# dango's metaclasses.
if not hasattr(cls, '_reverse_field_name_to_user'):
cls._find_reverse_field_name_to_user()
return Group.__new__(cls, *args, **kwargs)
It works but looks horrible so I've thought about using something like #lazyclassproperty for these attributes.
I've found several #classproperty and #lazyproperty decorators but not one for both and I have no idea how to write one myself.
Question: How could I code such decorator? or suggest another cleaner alternative to my current silly implementation.
Pyramid framework has a very nice decorator called reify, but it only works at instance level, and you want class level, so let's modify it a bit
class class_reify(object):
def __init__(self, wrapped):
self.wrapped = wrapped
try:
self.__doc__ = wrapped.__doc__
except: # pragma: no cover
pass
# original sets the attributes on the instance
# def __get__(self, inst, objtype=None):
# if inst is None:
# return self
# val = self.wrapped(inst)
# setattr(inst, self.wrapped.__name__, val)
# return val
# ignore the instance, and just set them on the class
# if called on a class, inst is None and objtype is the class
# if called on an instance, inst is the instance, and objtype
# the class
def __get__(self, inst, objtype=None):
# ask the value from the wrapped object, giving it
# our class
val = self.wrapped(objtype)
# and set the attribute directly to the class, thereby
# avoiding the descriptor to be called multiple times
setattr(objtype, self.wrapped.__name__, val)
# and return the calculated value
return val
class Test(object):
#class_reify
def foo(cls):
print("foo called for class", cls)
return 42
print(Test.foo)
print(Test.foo)
Run the program and it prints
foo called for class <class '__main__.Test'>
42
42
I'm writing a decorator for methods that must inspect the parent methods (the methods of the same name in the parents of the class in which I'm decorating).
Example (from the fourth example of PEP 318):
def returns(rtype):
def check_returns(f):
def new_f(*args, **kwds):
result = f(*args, **kwds)
assert isinstance(result, rtype), \
"return value %r does not match %s" % (result,rtype)
return result
new_f.func_name = f.func_name
# here I want to reach the class owning the decorated method f,
# it should give me the class A
return new_f
return check_returns
class A(object):
#returns(int)
def compute(self, value):
return value * 3
So I'm looking for the code to type in place of # here I want...
Thanks.
As bobince said it, you can't access the surrounding class, because at the time the decorator is invoked, the class does not exist yet. If you need access to the full dictionary of the class and the bases, you should consider a metaclass:
__metaclass__
This variable can be any callable accepting arguments for name, bases, and dict. Upon class creation, the callable is used instead of the built-in type().
Basically, we convert the returns decorator into something that just tells the metaclass to do some magic on class construction:
class CheckedReturnType(object):
def __init__(self, meth, rtype):
self.meth = meth
self.rtype = rtype
def returns(rtype):
def _inner(f):
return CheckedReturnType(f, rtype)
return _inner
class BaseInspector(type):
def __new__(mcs, name, bases, dct):
for obj_name, obj in dct.iteritems():
if isinstance(obj, CheckedReturnType):
# do your wrapping & checking here, base classes are in bases
# reassign to dct
return type.__new__(mcs, name, bases, dct)
class A(object):
__metaclass__ = BaseInspector
#returns(int)
def compute(self, value):
return value * 3
Mind that I have not tested this code, please leave comments if I should update this.
There are some articles on metaclasses by the highly recommendable David Mertz, which you might find interesting in this context.
here I want to reach the class owning the decorated method f
You can't because at the point of decoration, no class owns the method f.
class A(object):
#returns(int)
def compute(self, value):
return value * 3
Is the same as saying:
class A(object):
pass
#returns(int)
def compute(self, value):
return value*3
A.compute= compute
Clearly, the returns() decorator is built before the function is assigned to an owner class.
Now when you write a function to a class (either inline, or explicitly like this) it becomes an unbound method object. Now it has a reference to its owner class, which you can get by saying:
>>> A.compute.im_class
<class '__main__.A'>
So you can read f.im_class inside ‘new_f’, which is executed after the assignment, but not in the decorator itself.
(And even then it's a bit ugly relying on a CPython implementation detail if you don't need to. I'm not quite sure what you're trying to do, but things involving “get the owner class” are often doable using metaclasses.)
I am trying to write a decorator to do logging:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f():
pass
C().f()
I would like this to print:
Entering C.f
but instead I get this error message:
AttributeError: 'function' object has no attribute 'im_class'
Presumably this is something to do with the scope of 'myFunc' inside 'logger', but I've no idea what.
Claudiu's answer is correct, but you can also cheat by getting the class name off of the self argument. This will give misleading log statements in cases of inheritance, but will tell you the class of the object whose method is being called. For example:
from functools import wraps # use this to preserve function signatures and docstrings
def logger(func):
#wraps(func)
def with_logging(*args, **kwargs):
print "Entering %s.%s" % (args[0].__class__.__name__, func.__name__)
return func(*args, **kwargs)
return with_logging
class C(object):
#logger
def f(self):
pass
C().f()
As I said, this won't work properly in cases where you've inherited a function from a parent class; in this case you might say
class B(C):
pass
b = B()
b.f()
and get the message Entering B.f where you actually want to get the message Entering C.f since that's the correct class. On the other hand, this might be acceptable, in which case I'd recommend this approach over Claudiu's suggestion.
Functions only become methods at runtime. That is, when you get C.f you get a bound function (and C.f.im_class is C). At the time your function is defined it is just a plain function, it is not bound to any class. This unbound and disassociated function is what is decorated by logger.
self.__class__.__name__ will give you the name of the class, but you can also use descriptors to accomplish this in a somewhat more general way. This pattern is described in a blog post on Decorators and Descriptors, and an implementation of your logger decorator in particular would look like:
class logger(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, type=None):
return self.__class__(self.func.__get__(obj, type))
def __call__(self, *args, **kw):
print 'Entering %s' % self.func
return self.func(*args, **kw)
class C(object):
#logger
def f(self, x, y):
return x+y
C().f(1, 2)
# => Entering <bound method C.f of <__main__.C object at 0x...>>
Obviously the output can be improved (by using, for example, getattr(self.func, 'im_class', None)), but this general pattern will work for both methods and functions. However it will not work for old-style classes (but just don't use those ;)
Ideas proposed here are excellent, but have some disadvantages:
inspect.getouterframes and args[0].__class__.__name__ are not suitable for plain functions and static-methods.
__get__ must be in a class, that is rejected by #wraps.
#wraps itself should be hiding traces better.
So, I've combined some ideas from this page, links, docs and my own head,
and finally found a solution, that lacks all three disadvantages above.
As a result, method_decorator:
Knows the class the decorated method is bound to.
Hides decorator traces by answering to system attributes more correctly than functools.wraps() does.
Is covered with unit-tests for bound an unbound instance-methods, class-methods, static-methods, and plain functions.
Usage:
pip install method_decorator
from method_decorator import method_decorator
class my_decorator(method_decorator):
# ...
See full unit-tests for usage details.
And here is just the code of the method_decorator class:
class method_decorator(object):
def __init__(self, func, obj=None, cls=None, method_type='function'):
# These defaults are OK for plain functions
# and will be changed by __get__() for methods once a method is dot-referenced.
self.func, self.obj, self.cls, self.method_type = func, obj, cls, method_type
def __get__(self, obj=None, cls=None):
# It is executed when decorated func is referenced as a method: cls.func or obj.func.
if self.obj == obj and self.cls == cls:
return self # Use the same instance that is already processed by previous call to this __get__().
method_type = (
'staticmethod' if isinstance(self.func, staticmethod) else
'classmethod' if isinstance(self.func, classmethod) else
'instancemethod'
# No branch for plain function - correct method_type for it is already set in __init__() defaults.
)
return object.__getattribute__(self, '__class__')( # Use specialized method_decorator (or descendant) instance, don't change current instance attributes - it leads to conflicts.
self.func.__get__(obj, cls), obj, cls, method_type) # Use bound or unbound method with this underlying func.
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)
def __getattribute__(self, attr_name): # Hiding traces of decoration.
if attr_name in ('__init__', '__get__', '__call__', '__getattribute__', 'func', 'obj', 'cls', 'method_type'): # Our known names. '__class__' is not included because is used only with explicit object.__getattribute__().
return object.__getattribute__(self, attr_name) # Stopping recursion.
# All other attr_names, including auto-defined by system in self, are searched in decorated self.func, e.g.: __module__, __class__, __name__, __doc__, im_*, func_*, etc.
return getattr(self.func, attr_name) # Raises correct AttributeError if name is not found in decorated self.func.
def __repr__(self): # Special case: __repr__ ignores __getattribute__.
return self.func.__repr__()
It seems that while the class is being created, Python creates regular function objects. They only get turned into unbound method objects afterwards. Knowing that, this is the only way I could find to do what you want:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
def f(self):
pass
C.f = logger(C.f)
C().f()
This outputs the desired result.
If you want to wrap all the methods in a class, then you probably want to create a wrapClass function, which you could then use like this:
C = wrapClass(C)
Class functions should always take self as their first argument, so you can use that instead of im_class.
def logger(myFunc):
def new(self, *args, **keyargs):
print 'Entering %s.%s' % (self.__class__.__name__, myFunc.__name__)
return myFunc(self, *args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
at first I wanted to use self.__name__ but that doesn't work because the instance has no name. you must use self.__class__.__name__ to get the name of the class.
I found another solution to a very similar problem using the inspect library. When the decorator is called, even though the function is not yet bound to the class, you can inspect the stack and discover which class is calling the decorator. You can at least get the string name of the class, if that is all you need (probably can't reference it yet since it is being created). Then you do not need to call anything after the class has been created.
import inspect
def logger(myFunc):
classname = inspect.getouterframes(inspect.currentframe())[1][3]
def new(*args, **keyargs):
print 'Entering %s.%s' % (classname, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
While this is not necessarily better than the others, it is the only way I can figure out to discover the class name of the future method during the call to the decorator. Make note of not keeping references to frames around in the inspect library documentation.
As shown in Asa Ayers' answer, you don't need to access the class object. It may be worth to know that since Python 3.3, you can also use __qualname__, which gives you the fully qualified name:
>>> def logger(myFunc):
... def new(*args, **keyargs):
... print('Entering %s' % myFunc.__qualname__)
... return myFunc(*args, **keyargs)
...
... return new
...
>>> class C(object):
... #logger
... def f(self):
... pass
...
>>> C().f()
Entering C.f
This has the added advantage of working also in the case of nested classes, as shown in this example taken from PEP 3155:
>>> class C:
... def f(): pass
... class D:
... def g(): pass
...
>>> C.__qualname__
'C'
>>> C.f.__qualname__
'C.f'
>>> C.D.__qualname__
'C.D'
>>> C.D.g.__qualname__
'C.D.g'
Notice also that in Python 3 the im_class attribute is gone, therefore if you really wish to access the class in a decorator, you need an other method. The approach I currently use involves object.__set_name__ and is detailed in my answer to "Can a Python decorator of an instance method access the class?"
You can also use new.instancemethod() to create an instance method (either bound or unbound) from a function.
Instead of injecting decorating code at definition time, when function doesn't know it's class, delay running this code until function is accessed/called. Descriptor object facilitates injecting own code late, at access/call time:
class decorated(object):
def __init__(self, func, type_=None):
self.func = func
self.type = type_
def __get__(self, obj, type_=None):
return self.__class__(self.func.__get__(obj, type_), type_)
def __call__(self, *args, **kwargs):
name = '%s.%s' % (self.type.__name__, self.func.__name__)
print('called %s with args=%s kwargs=%s' % (name, args, kwargs))
return self.func(*args, **kwargs)
class Foo(object):
#decorated
def foo(self, a, b):
pass
Now we can inspect class both at access time (__get__) and at call time (__call__). This mechanism works for plain methods as well as static|class methods:
>>> Foo().foo(1, b=2)
called Foo.foo with args=(1,) kwargs={'b': 2}
Full example at: https://github.com/aurzenligl/study/blob/master/python-robotwrap/Example4.py