I would like to provide a decorator that allows for an optional configuration when applied to a function.
A simple implementation follows:
import functools
class Deco(object):
config = {'message': 'hello'}
def __init__(self, func):
self.func = func
functools.wraps(func)(self)
def __call__(self, *args, **kwargs):
print self.config['message']
return self.func(*args, **kwargs)
#classmethod
def customize(cls, **kwargs):
"""Return a customized instance of this class. """
return type(cls.__name__, (Deco, ), {'config': kwargs})
#Deco
def add(a, b):
return a + b
#Deco.customize(message='bye')
def sub(a, b):
return a - b
>>> add(1, 2)
'hello'
>>> sub(2, 1)
'bye'
I would like to use it to provide user-friendly decorators for Django views.
This approach works without errors, but is there something bad about allowing a class to have a static factory method instantiating customized instances of it self, as a decorator?
You could work without creating an extra sub-class for each time the decorator is used there, but your code is fine. The way without extra subclass could be something along:
class Deco(object):
config = {'message': 'hello'}
def __init__(self, func=None, **kwargs):
if kwargs:
self.config = kwargs
if func is not None:
self._decorate(func)
def _decorate(self, func):
self.func = func
functools.wraps(func)(self)
def __call__(self, *args, **kwargs):
if not hasattr(self, "func"):
self._decorate(func)
return self
print self.config['message']
return self.func(*args, **kwargs)
So, while performance wise there would be no difference to your code (unless you would be decorating at least hundreds of thousands of functions - your code create an extra object - a class - for each time the decorator is used, besides the instance of that class) - there is an impact on people would review your code (either to use your modules, or to maintain it after you are done). I mean "a decorator that dynamically generates subclasses of itself" may sound too advanced and scare people away. Although it is as simple as my suggestion above once one understands the mechanisms of class generation in Python as you had.
Related
I am trying to add a decorator to required class methods and I have come up with the following code for it. I need this to work with all the similar classes.
import allure
def class_method_dec(cls):
"""
Decorator function to decorate required class methods.
"""
if cls.method1:
cls.method1= allure.step(cls.method1)
if cls.method2:
cls.method2= allure.step(cls.method2)
if cls.method3:
cls.method3= allure.step(cls.method3)
return cls
#class_method_dec
class TestClass:
def __init__(self, a, b):
self.a = a
self.b = b
def method1(self):
"""
method docstring
"""
pass
def method2(self):
"""
method docstring
"""
pass
def method3(self):
"""
method docstring
"""
pass
Is this the right way to do it? I am looking for the best way to do this.
Also, I understand that we can use functools.wraps to preserve the docstring when decorating functions. Is there a need of something like it when we are decorating classes?
From Satwik Kansal’s brilliant Metaprogramming in Python IBM tutorial , I discovered this gem:
Satwik first defined a decorator:
from functools import wraps
import random
import time
def wait_random(min_wait=1, max_wait=30):
def inner_function(func):
#wraps(func)
def wrapper(args, **kwargs):
time.sleep(random.randint(min_wait, max_wait))
return func(args, **kwargs)
return wrapper
return inner_function
And then he created a class wrapper that will apply this decorator to a class:
def classwrapper(cls):
for name, val in vars(cls).items():
#callable return True if the argument is callable
#i.e. implements the __call
if callable(val):
#instead of val, wrap it with our decorator.
setattr(cls, name, wait_random()(val))
return cls
Application:
# decorate a function
#wait_random(10, 15)
def function_to_scrape():
#some scraping stuff
# decorate a class
#classwrapper
class Scraper:
# some scraping stuff
To make use of it in your case, substitute wait_random decorator with your own. Turn your function to a decorator.
E.g
from functools import wraps
import allure
def apply_allure():
def inner_function(func):
#wraps(func)
def wrapper(args, **kwargs):
func = allure.step(func)
return func(args, **kwargs)
return wrapper
return inner_function
In the classwrapper replace wait_random with apply_allure:
Do read the tutorial for more information and explanations
I’m trying to create a decorator that is called within a class, which would pull attributes from that class, and use those class attributes to edit the function’s docstring.
My problem is that I have found examples of decorators that edit the docstring of the function (setting the function's __doc__ attribute equal to a new string), and I have also found examples of decorators that pull attributes from the parent class (by passing self into the decorator), but I haven’t been able to find an example of a decorator that is able to do both.
I have tried to combine these two examples, but it isn't working:
def my_decorator(func):
def wrapper(self, *args, **kwargs):
name = func.__name__ # pull function name
cls = self.__class__.__name__ # pull class name
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), name, cls) # set them to docstring
return func(self, *args, **kwargs)
return wrapper
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
With this, I would hope that the following would return "Docstring is now new for the function: example":
Test().example.__doc__
Instead it returns None.
Edit: Note that I am not interested in how to access the name of the class specifically, so much as how to access the class attributes in general (where here self.__class__.__name__ is used as an example).
example is replaced with wrapper; the decoration is equivalent to
def example(self, examplearg=1):
"""Docstring"""
pass
example = my_decorator(example)
so you need to set wrapper.__doc__, not func.__doc__.
def my_decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for the function {}".format(
str(func.__doc__),
func.__name__)
return wrapper
Note that at the time you call my_decorator, you don't have any information about what class the decorated function/method belongs to. You would have to pass its name explicitly:
def my_decorator(cls_name):
def _decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for function {} in class {}".format(
func.__doc__,
func.__name__,
cls_name)
return wrapper
return _decorator
class Test():
#my_decorator("Test")
def example(self, examplearg=1):
"""Docstring"""
# or
# def example(self, examplearg=1):
# """Docstring"""
#
# example = my_decorator("Test")(example)
You can simply modify the __doc__ attribute when the decorator is called instead, and use the first token of the dot-delimited __qualname__ attribute of the function to obtain the class name:
def my_decorator(func):
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), func.__name__, func.__qualname__.split('.')[0])
return func
so that:
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
print(Test().example.__doc__)
would output:
Docstring is new for the function example in class Test
Turns out that accessing class attributes from within a class is impossible, as the class has yet to be executed when the decorator is called. So the original goal - using a decorator within a class to access class attributes - does not seem to be possible.
However, thanks to jdehesa for pointing me to a workaround that allows access to the class attributes using a class decorator, here: Can a Python decorator of an instance method access the class?.
I was able to use the class decorator to alter the specific method's docstring using class attributes like so:
def class_decorator(cls):
for name, method in cls.__dict__.items():
if name == 'example':
# do something with the method
method.__doc__ = "{} is new for function {} in class {}".format(method.__doc__, name, cls.__name__)
# Note that other class attributes such as cls.__base__
# can also be accessed in this way
return cls
#class_decorator
class Test():
def example(self, examplearg=1):
"""Docstring"""
print(Test().example.__doc__)
# Returns "Docstring is new for function example in class Test"
After spending several hours on the topic of decorators in python, I still have two issues.
First; if you have decorator without argument, sytntax is like this:
#decorator
def bye():
return "bye"
which is just a syntactic sugar and is same as this
bye = decorator(bye)
but if I have a decorator with argument:
#decorator(*args)
def bye():
return "bye"
How does "no-sugar" version looks like? Is the function passed inside as one of the arguments?
bye = decorator("argument", bye)
Second issue(which is related to the first, yet more practical example);
def permission_required(permission):
def wrap(function):
#functools.wraps(function)
def wrapped_func(*args, **kwargs):
if not current_user.can(permission):
abort(403)
return function(*args, **kwargs)
return wrapped_function
return wrap
def admin_required(f):
return permission_required(Permission.ADMINISTER)(f)
Here permission_required decorator is passed to a return statement of newly created decorator named admin_required. I have no idea how this works. Mainly the return statement where we returning original decorator + the function(in strange syntax). Can someone elaborate on this? - details are extremely welcome
When arguments are given in decorator notation,
#decorator(a, b, c)
def function(): pass
it is syntactic sugar for writing
def function(): pass
function = decorator(a, b, c)(function)
That is, decorator is called with arguments a, b, c, and then the object it returns is called with sole argument function.
It is easiest to understand how that makes sense when the decorator is a class. I'm going to use your permission_required decorator for a running example. It could have been written thus:
class permission_required:
def __init__(self, permission):
self.permission = permission
def __call__(self, function):
#functools.wraps(function)
def wrapped_func(*args, **kwargs):
if not current_user.can(permission):
abort(403)
return function(*args, **kwargs)
return wrapped_func
admin_required = permission_required(Permission.ADMINISTER)
When you use the decorator, e.g.
#permission_required(Permission.DESTRUCTIVE)
def erase_the_database():
raise NotImplemented # TBD: should we even have this?
you instantiate the class first, passing Permission.DESTRUCTIVE to __init__, and then you call the instance as a function with erase_the_database as an argument, which invokes the __call__ method, which constructs the wrapped function and returns it.
Thinking about it this way, admin_required should be easier to understand: it's an instance of the permission_required class, that hasn't yet been called. It's basically for shorthand:
#admin_required
def add_user(...): ...
instead of typing out
#permission_required(Permission.ADMINISTER)
def add_user(...): ...
Now, the way you had it...
def permission_required(permission):
def wrap(function):
#functools.wraps(function)
def wrapped_func(*args, **kwargs):
if not current_user.can(permission):
abort(403)
return function(*args, **kwargs)
return wrapped_func
return wrap
is really just another way of writing the same thing. Returning wrap from permission_required implicitly creates a closure object. It can be called like a function, and when you do it calls wrap. It remembers the value of permission passed to permission_required so that wrap can use it. That's exactly what the class I showed above does. (In fact, compiled languages like C++ and Rust often implement closures by desugaring them into class definitions like the one I showed.)
Notice that wrap itself does the same thing! We could expand it out even further...
class permission_check_wrapper:
def __init__(self, function, permission):
self.function = function
self.permission = permission
functools.update_wrapper(self, function)
def __call__(self, *args, **kwargs):
if not current_user.can(permission):
abort(403)
return function(*args, **kwargs)
class permission_required:
def __init__(self, permission):
self.permission = permission
def __call__(self, function):
return permission_check_wrapper(self.permission, function)
Or we could do the entire job with functools.partial:
def permission_check_wrapper(*args, function, permission, **kwargs):
if not current_user.can(permission):
abort(403)
return function(*args, **kwargs)
def wrap_fn_with_permission_check(function, *, permission):
return functools.update_wrapper(
functools.partial(permission_check_wrapper,
function=function,
permission=permission),
wrapped=function)
def permission_required(permission):
return functools.partial(wrap_fn_with_permission_check,
permission=permission)
The beauty of defining #decorator(a,b,c) def foo to desugar to foo = decorator(a,b,c)(foo) is that the language doesn't care which of these several implementation techniques you pick.
A decorator with an argument is simply called (with that argument), to produce another decorator. That decorator is then called with the decorated function as its argument, as usual. So the translation of:
#decorator(*args)
def bye():
return "bye"
would be:
bye = decorator(*args)(bye)
Or maybe you'd find that clearer as:
temp = decorator(*args)
bye = temp(bye)
(except that no temp variable is actually created, of course.)
In your second issue, #admin_required is being defined as a shortcut for #permission_required(Permission.ADMINISTER).
I've been trying to pickle an object which contains references to static class methods.
Pickle fails (for example on module.MyClass.foo) stating it cannot be pickled, as module.foo does not exist.
I have come up with the following solution, using a wrapper object to locate the function upon invocation, saving the container class and function name:
class PicklableStaticMethod(object):
"""Picklable version of a static method.
Typical usage:
class MyClass:
#staticmethod
def doit():
print "done"
# This cannot be pickled:
non_picklable = MyClass.doit
# This can be pickled:
picklable = PicklableStaticMethod(MyClass.doit, MyClass)
"""
def __init__(self, func, parent_class):
self.func_name = func.func_name
self.parent_class = parent_class
def __call__(self, *args, **kwargs):
func = getattr(self.parent_class, self.func_name)
return func(*args, **kwargs)
I am wondering though, is there a better - more standard way - to pickle such an object?
I do not want to make changes to the global pickle process (using copy_reg for example), but the following pattern would be great:
class MyClass(object):
#picklable_staticmethod
def foo():
print "done."
My attempts at this were unsuccessful, specifically because I could not extract the owner class from the foo function. I was even willing to settle for explicit specification (such as #picklable_staticmethod(MyClass)) but I don't know of any way to refer to the MyClass class right where it's being defined.
Any ideas would be great!
Yonatan
This seems to work.
class PickleableStaticMethod(object):
def __init__(self, fn, cls=None):
self.cls = cls
self.fn = fn
def __call__(self, *args, **kwargs):
return self.fn(*args, **kwargs)
def __get__(self, obj, cls):
return PickleableStaticMethod(self.fn, cls)
def __getstate__(self):
return (self.cls, self.fn.__name__)
def __setstate__(self, state):
self.cls, name = state
self.fn = getattr(self.cls, name).fn
The trick is to snag the class when the static method is gotten from it.
Alternatives: You could use metaclassing to give all your static methods a .__parentclass__ attribute. Then you could subclass Pickler and give each subclass instance its own .dispatch table which you can then modify without affecting the global dispatch table (Pickler.dispatch). Pickling, unpickling, and calling the method might then be a little faster.
EDIT: modified after Jason comment.
I think python is correct in not letting pickling a staticmethod object - as it is impossible to pickle instance or class methods! Such an object would make very little sense outside of its context:
Check this: Descriptor Tutorial
import pickle
def dosomething(a, b):
print a, b
class MyClass(object):
dosomething = staticmethod(dosomething)
o = MyClass()
pickled = pickle.dumps(dosomething)
This works, and that's what should be done - define a function, pickle it, and use such function as a staticmethod in a certain class.
If you've got an use case for your need, please write it down and I'll be glad to discuss it.
I am trying to write a decorator to do logging:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f():
pass
C().f()
I would like this to print:
Entering C.f
but instead I get this error message:
AttributeError: 'function' object has no attribute 'im_class'
Presumably this is something to do with the scope of 'myFunc' inside 'logger', but I've no idea what.
Claudiu's answer is correct, but you can also cheat by getting the class name off of the self argument. This will give misleading log statements in cases of inheritance, but will tell you the class of the object whose method is being called. For example:
from functools import wraps # use this to preserve function signatures and docstrings
def logger(func):
#wraps(func)
def with_logging(*args, **kwargs):
print "Entering %s.%s" % (args[0].__class__.__name__, func.__name__)
return func(*args, **kwargs)
return with_logging
class C(object):
#logger
def f(self):
pass
C().f()
As I said, this won't work properly in cases where you've inherited a function from a parent class; in this case you might say
class B(C):
pass
b = B()
b.f()
and get the message Entering B.f where you actually want to get the message Entering C.f since that's the correct class. On the other hand, this might be acceptable, in which case I'd recommend this approach over Claudiu's suggestion.
Functions only become methods at runtime. That is, when you get C.f you get a bound function (and C.f.im_class is C). At the time your function is defined it is just a plain function, it is not bound to any class. This unbound and disassociated function is what is decorated by logger.
self.__class__.__name__ will give you the name of the class, but you can also use descriptors to accomplish this in a somewhat more general way. This pattern is described in a blog post on Decorators and Descriptors, and an implementation of your logger decorator in particular would look like:
class logger(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, type=None):
return self.__class__(self.func.__get__(obj, type))
def __call__(self, *args, **kw):
print 'Entering %s' % self.func
return self.func(*args, **kw)
class C(object):
#logger
def f(self, x, y):
return x+y
C().f(1, 2)
# => Entering <bound method C.f of <__main__.C object at 0x...>>
Obviously the output can be improved (by using, for example, getattr(self.func, 'im_class', None)), but this general pattern will work for both methods and functions. However it will not work for old-style classes (but just don't use those ;)
Ideas proposed here are excellent, but have some disadvantages:
inspect.getouterframes and args[0].__class__.__name__ are not suitable for plain functions and static-methods.
__get__ must be in a class, that is rejected by #wraps.
#wraps itself should be hiding traces better.
So, I've combined some ideas from this page, links, docs and my own head,
and finally found a solution, that lacks all three disadvantages above.
As a result, method_decorator:
Knows the class the decorated method is bound to.
Hides decorator traces by answering to system attributes more correctly than functools.wraps() does.
Is covered with unit-tests for bound an unbound instance-methods, class-methods, static-methods, and plain functions.
Usage:
pip install method_decorator
from method_decorator import method_decorator
class my_decorator(method_decorator):
# ...
See full unit-tests for usage details.
And here is just the code of the method_decorator class:
class method_decorator(object):
def __init__(self, func, obj=None, cls=None, method_type='function'):
# These defaults are OK for plain functions
# and will be changed by __get__() for methods once a method is dot-referenced.
self.func, self.obj, self.cls, self.method_type = func, obj, cls, method_type
def __get__(self, obj=None, cls=None):
# It is executed when decorated func is referenced as a method: cls.func or obj.func.
if self.obj == obj and self.cls == cls:
return self # Use the same instance that is already processed by previous call to this __get__().
method_type = (
'staticmethod' if isinstance(self.func, staticmethod) else
'classmethod' if isinstance(self.func, classmethod) else
'instancemethod'
# No branch for plain function - correct method_type for it is already set in __init__() defaults.
)
return object.__getattribute__(self, '__class__')( # Use specialized method_decorator (or descendant) instance, don't change current instance attributes - it leads to conflicts.
self.func.__get__(obj, cls), obj, cls, method_type) # Use bound or unbound method with this underlying func.
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)
def __getattribute__(self, attr_name): # Hiding traces of decoration.
if attr_name in ('__init__', '__get__', '__call__', '__getattribute__', 'func', 'obj', 'cls', 'method_type'): # Our known names. '__class__' is not included because is used only with explicit object.__getattribute__().
return object.__getattribute__(self, attr_name) # Stopping recursion.
# All other attr_names, including auto-defined by system in self, are searched in decorated self.func, e.g.: __module__, __class__, __name__, __doc__, im_*, func_*, etc.
return getattr(self.func, attr_name) # Raises correct AttributeError if name is not found in decorated self.func.
def __repr__(self): # Special case: __repr__ ignores __getattribute__.
return self.func.__repr__()
It seems that while the class is being created, Python creates regular function objects. They only get turned into unbound method objects afterwards. Knowing that, this is the only way I could find to do what you want:
def logger(myFunc):
def new(*args, **keyargs):
print 'Entering %s.%s' % (myFunc.im_class.__name__, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
def f(self):
pass
C.f = logger(C.f)
C().f()
This outputs the desired result.
If you want to wrap all the methods in a class, then you probably want to create a wrapClass function, which you could then use like this:
C = wrapClass(C)
Class functions should always take self as their first argument, so you can use that instead of im_class.
def logger(myFunc):
def new(self, *args, **keyargs):
print 'Entering %s.%s' % (self.__class__.__name__, myFunc.__name__)
return myFunc(self, *args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
at first I wanted to use self.__name__ but that doesn't work because the instance has no name. you must use self.__class__.__name__ to get the name of the class.
I found another solution to a very similar problem using the inspect library. When the decorator is called, even though the function is not yet bound to the class, you can inspect the stack and discover which class is calling the decorator. You can at least get the string name of the class, if that is all you need (probably can't reference it yet since it is being created). Then you do not need to call anything after the class has been created.
import inspect
def logger(myFunc):
classname = inspect.getouterframes(inspect.currentframe())[1][3]
def new(*args, **keyargs):
print 'Entering %s.%s' % (classname, myFunc.__name__)
return myFunc(*args, **keyargs)
return new
class C(object):
#logger
def f(self):
pass
C().f()
While this is not necessarily better than the others, it is the only way I can figure out to discover the class name of the future method during the call to the decorator. Make note of not keeping references to frames around in the inspect library documentation.
As shown in Asa Ayers' answer, you don't need to access the class object. It may be worth to know that since Python 3.3, you can also use __qualname__, which gives you the fully qualified name:
>>> def logger(myFunc):
... def new(*args, **keyargs):
... print('Entering %s' % myFunc.__qualname__)
... return myFunc(*args, **keyargs)
...
... return new
...
>>> class C(object):
... #logger
... def f(self):
... pass
...
>>> C().f()
Entering C.f
This has the added advantage of working also in the case of nested classes, as shown in this example taken from PEP 3155:
>>> class C:
... def f(): pass
... class D:
... def g(): pass
...
>>> C.__qualname__
'C'
>>> C.f.__qualname__
'C.f'
>>> C.D.__qualname__
'C.D'
>>> C.D.g.__qualname__
'C.D.g'
Notice also that in Python 3 the im_class attribute is gone, therefore if you really wish to access the class in a decorator, you need an other method. The approach I currently use involves object.__set_name__ and is detailed in my answer to "Can a Python decorator of an instance method access the class?"
You can also use new.instancemethod() to create an instance method (either bound or unbound) from a function.
Instead of injecting decorating code at definition time, when function doesn't know it's class, delay running this code until function is accessed/called. Descriptor object facilitates injecting own code late, at access/call time:
class decorated(object):
def __init__(self, func, type_=None):
self.func = func
self.type = type_
def __get__(self, obj, type_=None):
return self.__class__(self.func.__get__(obj, type_), type_)
def __call__(self, *args, **kwargs):
name = '%s.%s' % (self.type.__name__, self.func.__name__)
print('called %s with args=%s kwargs=%s' % (name, args, kwargs))
return self.func(*args, **kwargs)
class Foo(object):
#decorated
def foo(self, a, b):
pass
Now we can inspect class both at access time (__get__) and at call time (__call__). This mechanism works for plain methods as well as static|class methods:
>>> Foo().foo(1, b=2)
called Foo.foo with args=(1,) kwargs={'b': 2}
Full example at: https://github.com/aurzenligl/study/blob/master/python-robotwrap/Example4.py