I have a decorator class validatekeys() and a Node3D() class.
The intention is for Node3D to hold coordinate values of x, y, and z which are retrieved using a #property decorator, and can be set using either a #coords.setter decorator (which calls set_coords()) or directly using set_coords() which is itself decorated with validatekeys(). I am using decorators to accomplish this so that I can add other classes later, like Node2D(), for example.
Code:
class validatekeys(object):
def __init__(self,*keysIterable):
self.validkeys = []
for k in keysIterable:
self.validkeys.append(k)
def __call__(self,f):
def wrapped_f(*args,**kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
self.__dict__.update(kwargs)
return f(self,**kwargs)
return wrapped_f
class Node3D(object):
#property
def coords(self):
return self.__dict__
#coords.setter
def coords(self,Coords):
self.set_coords(**Coords)
#validatekeys('x','y','z')
def set_coords(self,**Coords):
pass
However, part of the output is not as expected:
n = Node2D()
n.coords #{} <--expected
n.set_coords(x=1,y=2)
n.coords #{} <--not expected
n.set_coords(a=1,b=2) #Exception <--expected
It looks like the self.__dict__ is not being updated correctly. However, I've been unable to figure out how to fix this. Any suggestions?
Note that although I am certainly interested in alternative formulations/approaches on solving this problem (validating keys input to a setter), this is mostly a learning exercise to understand how decorators, classes, etc etc work.
Your decorator is updating the wrong __dict__; self in your decorator __call__ is the decorator object itself.
You need to extract the bound self argument from the called wrapper:
def wrapped_f(*args, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance = args[0]
instance.__dict__.update(kwargs)
return f(*args, **kwargs)
You can give your wrapped_f() an explicit first argument too:
def wrapped_f(instance, *args, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance.__dict__.update(kwargs)
return f(instance, *args, **kwargs)
Here instance is bound to the Node3D instance. Note that there is no hard requirement to name this variable self; that is just a convention.
The self in your __call__ refers to the validator, not the Node3D object, so the validator is updating its own __dict__. Try this instead:
class validatekeys(object):
def __init__(self,*keysIterable):
self.validkeys = []
for k in keysIterable:
self.validkeys.append(k)
def __call__(validator_self,f):
def wrapped_f(self, *args,**kwargs):
for a in kwargs:
if not a in validator_self.validkeys:
raise Exception()
self.__dict__.update(kwargs)
return f(self, *args, **kwargs)
return wrapped_f
Here I've renamed the self in the __call__ to validator_self to make it clear that that self refers to the validator. I added a self to the wrapper function; this self will refer to the "real" self of the Node3D object where the validated method is.
Related
I have used the following Mixin class to inherit decorators along sub-classes. The Problem is that when a method has more than one decorator than they are not recognized (just the last one). For example, if I have the class:
class Example(InheritDecoratorsMixin):
#decorator1
#decorator2
def method():
pass
Then any subclass would just inherit decorator1 but not decorator2 which I would like to have. Here is the Mixin class:
class InheritDecoratorsMixin:
"""Mixin Class that allows to inherit decorators.
Each subclass of this class will have a '_decorator_registry'
attribute which contains all decorators to be applied.
Each decorator output must contain the attribute 'inherit_decorator'
with itself as the value.
"""
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
decorator_registry = getattr(cls, '_decorator_registry', {}).copy()
cls._decorator_registry = decorator_registry
# Check for decorated objects in the mixin itself:
for name, obj in __class__.__dict__.items():
if (getattr(obj, 'inherit_decorator', False)
and name not in decorator_registry):
decorator_registry[name] = obj.inherit_decorator
# annotate newly decorated methods in the current subclass:
for name, obj in cls.__dict__.items():
if (getattr(obj, 'inherit_decorator', False)
and name not in decorator_registry):
decorator_registry[name] = obj.inherit_decorator
# finally, decorate all methods annotated in the registry:
for name, decorator in decorator_registry.items():
if (name in cls.__dict__ and getattr(
getattr(cls, name), 'inherit_decorator', None) != decorator):
setattr(cls, name, decorator(cls.__dict__[name]))
Thanks for any help.
For any decorator I would do:
def decorator(func):
def wrapper(*args, **kwargs):
...
return result
wrapper.inherit_decorator = decorator
return wrapper
What you are trying to do is cumbersome, and if you are sure of this approach, just modify what you are doing accordingly:
your "decorator_registry" as is in the code specifically ties one decorator per method name. Change your code so that each entry in the registry is a list, instead of a single object, and maintain that list, instead of just replacing decorators when finding new ones. Also, your decorators, instead of just marking the decorated function with the topmost decorator itself in the "inherit_decorator" method, should maintain a list of all underlying decorators in this attribute.
I have not tested this, as there should be plenty of corner cases,
but the general idea is:
from copy import copy
from functools import wraps
class InheritDecoratorsMixin:
"""Mixin Class that allows to inherit decorators.
Each subclass of this class will have a '_decorator_registry'
attribute which contains all decorators to be applied.
Each decorator output must contain the attribute 'inherit_decorator'
with itself as the value.
"""
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
decorator_registry = getattr(cls, '_decorator_registry', {}).copy()
cls._decorator_registry = decorator_registry
for class_obj in (__class__, cls):
for name, obj in class_obj.__dict__.items():
if getattr(obj, 'inherit_decorator', False):
new_decorators = [deco for deco in obj.inherit_decorator if deco.__name__ not in decorator_registry.get(name)]
decorator_registry.setdefault(name, []).extend(new_decorators)
# finally, decorate all methods annotated in the registry.
# note that this code might apply some decorators more than once.
for name, decorators in decorator_registry.items():
if name not in cls.__dict__:
continue
method = getattr(cls, name)
new_method = None
for deco in decorators:
if deco in getattr(method, inherit_decorators, []):
continue
new_method = deco(method)
if new_method:
setattr(cls, name, new_method)
def decorator(func):
#wraps(func) # this call preserves name and other properties from the decorated function
def wrapper(*args, **kwargs):
...
return result
decorators = copy(getattr(func, "inherit_decorator", []))
decorators.append(decorator)
wrapper.inherit_decorator = decorators
return wrapper
I am attempting to copy the functionality of the built-in property class / decorator; a very basic example of what I want to is this:
# If a condition is met, run the first function, else, the second.
#godspeed()
def test():
print(1, 2, 3, 4)
#test.else_()
def test():
print(5, 6, 7, 8)
Here's what I have so far:
import inspect
class godspeed_class():
def __init__(
self,
func,
args,
kwargs,
value,
):
self.func = func
self.args = args
self.kwargs = kwargs
self.value = value
def __call__(self):
if self.value:
self.func(*self.args, **self.kwargs)
else:
self.else_func(*self.else_args, **self.else_kwargs)
def else_(self, *args, **kwargs):
def wrapper(func):
self.else_func = func
self.else_args = args
self.else_kwargs = kwargs
return wrapper
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
_ = godspeed_class(func, args, kwargs, value)
inspect.stack(1)[1][0].f_globals[func.__name__] = _
return wrapper
I already know how to implement the condition parsing, but I am having trouble with storing the function under the else_ decorator in the class, so that I can call it if the condition isn't met.
In addition, despite injecting the new class directly into the global namespace, when I run print(test), it tells me it's a NoneType object.
Note: Code has been updated; however, it still gives me the "NoneType object" error.
You need to change both of your wrapper functions to return a callable object, probably the instance of your class. Otherwise you're going to have None as the value for the method, since the decorator syntax will assign the return value to the name of the decorated function, which means that even if your inspect hack works, it will get overwritten.
I'd suggest:
class godspeed_class():
... # __init__ and __call__ can remain the same
def else_(self, *args, **kwargs):
def wrapper(func):
self.else_func = func
self.else_args = args
self.else_kwargs = kwargs
return self # add return here
return wrapper
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
return godspeed_class(func, args, kwargs, value) # and here (rather than inspect stuff)
return wrapper
This will do the job for your example with a top-level test function. If you want to be able to decorate methods, you'll also need to add a __get__ method to the class to add binding behavior (otherwise you'll not get the self argument passed in to the wrapped method).
It's a bit misleading to use wrapper as the name there, as the inner functions are the actual decorators being used here (the top level godspeed function and the else_ method are decorator factories). Normally you use wrapper as a name of a function returned by a decorator (but you're using your class for that instead).
I'd also note that it's a bit strange that you're passing the arguments for the functions to the decorator factories, rather than having __call__ accept arguments that it passes along to the relevant function. It's a bit unusual for a decorator that leaves behind a callable (rather than something like property that works differently) to dramatically change a function's calling convention, as it may end up hard for a caller to know what arguments they're expected to pass in, if the function signature isn't representative any more.
A decorator is nothing magical. Basically, the #decorator syntax is just syntactic sugar, so this:
#mydecorator
def func():
pass
is just a convenient shortcut for
def func():
pass
func = mydecorator(func)
IOW, a "decorator" is a callable object that takes a callable as input and returns a callable (well, it's supposed to return a callable at least - you can actually return whatever, but then you'll break everyone's expectations).
Most often, the decorator is written as a simple function returning a closure over the decorated function:
def trace(func):
def wrapper(*args, **kw):
result = func(*args, **kw)
print("{}({}, {}) => {}". format(func, args, kw, result))
return result
return wrapper
#trace
def foo(x):
return 42 * x
But (since closures are the poor man's classes and classes the poor man's closures) you can also implement it as a callable class, in which case the initializer will receive the decorated func, which in turn will be replaced by the instance:
class trace(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kw):
result = self.func(*args, **kw)
print("{}({}, {}) => {}". format(self.func, args, kw, result))
return result
#trace
def foo(x):
return 42 * x
Then you have "parameterized" decorators - the one that can take arguments. In this case you need two level of indirection, the top-level one (the one used as decorator) returning the actual decorator (the one that receives the function), ie:
def trace(out):
def really_trace(func):
def wrapper(*args, **kw):
result = func(*args, **kw)
out.write("{}({}, {}) => {}\n". format(func, args, kw, result))
return result
return wrapper
return really_trace
#trace(sys.stderr)
def foo(x):
return 42 * x
I leave the class-based implementation as an exercise to the reader ;-)
Now in your case, the fact that test ends up being None is quite simply due to the fact that your wrapper func forgets to return the godspeed_class instance as it should (instead messing with the function's f_globals, which, as you noticed, doesn't work as expected).
Since you didn't clearly explained what you're trying to achieve here ("something similar to property" isn't a proper spec), it's hard to provide a working solution, but as a starting point you may want to fix your godspeed func to behave as expected:
def godspeed(*args, value = 0, **kwargs):
def wrapper(func):
return godspeed_class(func, args, kwargs, value)
return wrapper
This question already has answers here:
How can I decorate an instance method with a decorator class?
(2 answers)
Closed 4 years ago.
While there are plenty of resources about using classes as decorators, I haven't been able to find any that deal with the problem of decorating methods. The goal of this question is to fix that. I will post my own solution, but of course everyone else is invited to post theirs as well.
Why the "standard" implementation doesn't work
The problem with the standard decorator class implementation is that python will not create a bound method of the decorated function:
class Deco:
def __init__(self, func):
self.func= func
def __call__(self, *args):
self.func(*args)
class Class:
#Deco
def hello(self):
print('hello world')
Class().hello() # throws TypeError: hello() missing 1 required positional argument: 'self'
A method decorator needs to overcome this hurdle.
Requirements
Taking the classes from the previous example, the following things are expected to work:
>>> i= Class()
>>> i.hello()
hello world
>>> i.hello
<__main__.Deco object at 0x7f4ae8b518d0>
>>> Class.hello is Class().hello
False
>>> Class().hello is Class().hello
False
>>> i.hello is i.hello
True
Ideally, the function's __doc__ and signature and similar attributes are preserved as well.
Usually when a method is accessed as some_instance.some_method(), python's descriptor protocol kicks in and calls some_method.__get__(), which returns a bound method. However, because the method has been replaced with an instance of the Deco class, that does not happen - because Deco is not a descriptor. In order to make Deco work as expected, it must implement a __get__ method that returns a bound copy of itself.
Implementation
Here's basic "do nothing" decorator class:
import inspect
import functools
from copy import copy
class Deco(object):
def __init__(self, func):
self.__self__ = None # "__self__" is also used by bound methods
self.__wrapped__ = func
functools.update_wrapper(self, func)
def __call__(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy
bound = copy(self)
bound.__self__ = instance
# update __doc__ and similar attributes
functools.update_wrapper(bound, self.__wrapped__)
# add the bound instance to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
To make the decorator do something, add your code in the __call__ method.
Here's one that takes parameters:
class DecoWithArgs(object):
#== change the constructor's parameters to fit your needs ==
def __init__(self, *args):
self.args = args
self.__wrapped__ = None
self.__self__ = None
def __call__(self, *args, **kwargs):
if self.__wrapped__ is None:
return self.__wrap(*args, **kwargs)
else:
return self.__call_wrapped_function(*args, **kwargs)
def __wrap(self, func):
# update __doc__ and similar attributes
functools.update_wrapper(self, func)
return self
def __call_wrapped_function(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy of this object
bound = copy(self)
bound.__self__ = instance
bound.__wrap(self.__wrapped__)
# add the bound decorator to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
An implementation like this lets us use the decorator on methods as well as functions, so I think it should be considered good practice.
I have a decorated function (simplified version):
class Memoize:
def __init__(self, function):
self.function = function
self.memoized = {}
def __call__(self, *args, **kwds):
hash = args
try:
return self.memoized[hash]
except KeyError:
self.memoized[hash] = self.function(*args)
return self.memoized[hash]
#Memoize
def _DrawPlot(self, options):
do something...
now I want to add this method to a pre-esisting class.
ROOT.TChain.DrawPlot = _DrawPlot
when I call this method:
chain = TChain()
chain.DrawPlot(opts)
I got:
self.memoized[hash] = self.function(*args)
TypeError: _DrawPlot() takes exactly 2 arguments (1 given)
why doesn't it propagate self?
The problem is that you have defined your own callable class then tried to use it as a method. When you use a function as an attribute, accessing the function as an attribute calls it its __get__ method to return something other than the function itself—the bound method. When you have your own class without defining __get__, it just returns your instance without implicitly passing self.
Descriptors are explained some on http://docs.python.org/reference/datamodel.html#descriptors if you are not familiar with them. The __get__, __set__, and __delete__ methods change how interacting with your object as an attribute works.
You could implement memoize as a function and use the built-in __get__ magic that functions already have
import functools
def memoize(f):
#functools.wraps(f)
def memoized(*args, _cache={}):
# This abuses the normally-unwanted behaviour of mutable default arguments.
if args not in _cache:
_cache[args] = f(*args)
return _cache[args]
return memoized
or by modifying your class along the lines of
import functools
class Memoize(object): #inherit object
def __init__(self, function):
self.function = function
self.memoized = {}
def __call__(self, *args): #don't accept kwargs you don't want.
# I removed "hash = args" because it shadowed a builtin function and
# because it was untrue--it wasn't a hash, it was something you intended for
# Python to hash for you.
try:
return self.memoized[args]
except KeyError:
self.memoized[args] = self.function(*args)
return self.memoized[args]
def __get__(self, obj, type):
if obj is None: #We looked up on the class
return self
return functools.partial(self, obj)
Note that both of these choke if any of the arguments you pass in are mutable (well, unhashable technically). This might be suitable for your case, but you may also want to deal with the case where args is unhashable.
I am instantiating a class A (which I am importing from somebody
else, so I can't modify it) into my class X.
Is there a way I can intercept or wrap calls to methods in A?
I.e., in the code below can I call
x.a.p1()
and get the output
X.pre
A.p1
X.post
Many TIA!
class A:
# in my real application, this is an imported class
# that I cannot modify
def p1(self): print 'A.p1'
class X:
def __init__(self):
self.a=A()
def pre(self): print 'X.pre'
def post(self): print 'X.post'
x=X()
x.a.p1()
Here is the solution I and my colleagues came up with:
from types import MethodType
class PrePostCaller:
def __init__(self, other):
self.other = other
def pre(self): print 'pre'
def post(self): print 'post'
def __getattr__(self, name):
if hasattr(self.other, name):
func = getattr(self.other, name)
return lambda *args, **kwargs: self._wrap(func, args, kwargs)
raise AttributeError(name)
def _wrap(self, func, args, kwargs):
self.pre()
if type(func) == MethodType:
result = func( *args, **kwargs)
else:
result = func(self.other, *args, **kwargs)
self.post()
return result
#Examples of use
class Foo:
def stuff(self):
print 'stuff'
a = PrePostCaller(Foo())
a.stuff()
a = PrePostCaller([1,2,3])
print a.count()
Gives:
pre
stuff
post
pre
post
0
So when creating an instance of your object, wrap it with the PrePostCaller object. After that you continue using the object as if it was an instance of the wrapped object. With this solution you can do the wrapping on a per instance basis.
You could just modify the A instance and replace the p1 function with a wrapper function:
def wrapped(pre, post, f):
def wrapper(*args, **kwargs):
pre()
retval = f(*args, **kwargs)
post()
return retval
return wrapper
class Y:
def __init__(self):
self.a=A()
self.a.p1 = wrapped(self.pre, self.post, self.a.p1)
def pre(self): print 'X.pre'
def post(self): print 'X.post'
The no-whistles-or-bells solution would be to write a wrapper class for class A that does just that.
As others have mentioned, the wrapper/decorator solution is probably be the easiest one. I don't recommend modifyng the wrapped class itself, for the same reasons that you point out.
If you have many external classes you can write a code generator to generate the wrapper classes for you. Since you are doing this in Python you can probably even implement the generator as a part of the program, generating the wrappers at startup, or something.
I've just recently read about decorators in python, I'm not understanding them yet but it seems to me that they can be a solution to your problem. see Bruce Eckel intro to decorators at:
http://www.artima.com/weblogs/viewpost.jsp?thread=240808
He has a few more posts on that topic there.
Edit: Three days later I stumble upon this article, which shows how to do a similar task without decorators, what's the problems with it and then introduces decorators and develop a quite full solution:
http://wordaligned.org/articles/echo
Here's what I've received from Steven D'Aprano on comp.lang.python.
# Define two decorator factories.
def precall(pre):
def decorator(f):
def newf(*args, **kwargs):
pre()
return f(*args, **kwargs)
return newf
return decorator
def postcall(post):
def decorator(f):
def newf(*args, **kwargs):
x = f(*args, **kwargs)
post()
return x
return newf
return decorator
Now you can monkey patch class A if you want. It's probably not a great
idea to do this in production code, as it will effect class A everywhere.
[this is ok for my application, as it is basically a protocol converter and there's exactly one instance of each class being processed.]
class A:
# in my real application, this is an imported class
# that I cannot modify
def p1(self): print 'A.p1'
class X:
def __init__(self):
self.a=A()
A.p1 = precall(self.pre)(postcall(self.post)(A.p1))
def pre(self): print 'X.pre'
def post(self): print 'X.post'
x=X()
x.a.p1()
Gives the desired result.
X.pre
A.p1
X.post