I need to decorate a object's method. It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program (arguments supplied with argv), so a same object could be decorated 3 times, 2 times, or not be decorated at all.
Here is some context, the program is a puzzle solver, the main behavior is to find a solution for the puzzle automatically, by automatically I mean without user intervention. And here is where the decoration gets to play, one of the things I want to is draw a graph of what happened during the execution, but I want to do so only when the flag --draw-graph is used.
Here is what I've tried:
class GraphDecorator(object):
def __init__(self, wrappee):
self.wrappee = wrappee
def method(self):
# do my stuff here
self.wrappee.method()
# do more of stuff here
def __getattr__(self,attr):
return getattr(self.wrappee,attr)
And why it did NOT work:
It did not work because of the way I built the application, when a method that did not exist in my Decorator class was called it felt back to the implementation of the decorated class, the problem is that the application always started invoking the method run that did not need to be decorated, so the undecorated fall back was used and from inside the undecorated form it always called undecorated methods, what I needed was to replace the method from the object, not to proxy it:
# method responsible to replace the undecorated form by the decorated one
def graphDecorator(obj):
old_method = obj.method
def method(self):
# do my stuff here
old_method()
# do more of my stuff
setattr(obj,'method',method) # replace with the decorated form
And here is my problem, the decorated form does not receive self when it is called resulting on a TypeError because of the wrong number of arguments.
The problem was that I couldn't use func(self) as a method. The reason is that setattr() method does not bound the function, and the function acts like it a static method - not a class method -, thanks to the introspective nature of python I've able to come up with this solution:
def decorator(obj):
old_func = obj.func # can't call 'by name' because of recursion
def decorated_func(self):
# do my stuff here
old_func() # does not need pass obj
# do some othere stuff here
# here is the magic, this get the type of a 'normal method' of a class
method = type(obj.func)
# this bounds the method to the object, so self is passed by default
obj.func = method(decorated_func, obj)
I think this is the best way to decorate a object's method at runtime, though it would be nice to find a way to call method() directly, without the line method = type(obj.func)
You might want to use __getattribute__ instead of __getattr__ (the latter being only called if "standard" lookup fails):
class GraphDecorator(object):
def __init__(self, wrappee):
self.__wrappee = wrappee
def method(self):
# do my stuff here
self.wrappe.method()
# do more of stuff here
def __getattribute__(self, name):
try:
wrappee = object.__getattribute__(self, "_GraphDecorator__wrappee")
return getattr(wrappee, name)
except AttributeError:
return object.__getattribute__(self, name)
I need to decorate a object's method. It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program (arguments supplied with argv), so a same object could be decorated 3 times, 2 times, or not be decorated at all.
The above is unfortunately incorrect, and what you are trying to do is unnecessary.
You can do this at runtime like so. Example:
import sys
args = sys.argv[1:]
class MyClass(object):
pass
if args[0]=='--decorateWithFoo':
MyClass = decoratorFoo(MyClass)
if args[1]=='--decorateWithBar'
MyClass = decoratorBar(MyClass)
The syntax:
#deco
define something
Is the same thing as:
define something
something = deco(something)
You could also make a decorator factory #makeDecorator(command_line_arguments)
"It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program"
The don't use decorators. Decorators are only syntactical support for wrappers, you can just as well use normal function/method calls instead.
Related
I am trying to create a final method in my class, where I want that it cannot be overridden by any sub-class, just like when creating a final class using final decorator which cannot be inherited.
from final_class import final
class Dummy:
def show(self):
print("show id running from dummy")
#final
def display(self):
print("display from dummy")
class Demo(Dummy):
def show(self):
print("show from demo")
def display(self):
print("display from demo")
d = Demo()
d.display()
I think we should get an error when accessing the display method from Demo, but when I run the program it gives "display from demo".
So what am I missing? I have checked final annotation and decorators in python3.8 but it talks about typechecking in typing packages while I was trying it from the final_class package.
As seem in the comments, the 3rd party library final_class.final is one thing: a class decorator that will prevent, at runtime, that a class is further inherited, anf typing.final which ships with Python, and is intended to decorate both classes and methods, but which has no enforcing behavior during program execution - it will, instead, make any compliant static analysis tool to raise an error in the type-checking stage.
It is, due to Python flexibility and dynamism, possible to create a final decorator for methods that will be enforced at runtime: i.e. whenever a subclass is created overriding a method marked as final in the inheritance chain, a RuntimeError, or other custom error can be raised.
The idea is that whenever a new class is created, both methods on the metaclass and the __init_subclass__ method of the bases is called - so, if one wants to create a custom metaclass or custom base-class to be used along with such a #final decorator, it should be something more or less straightforward.
What would be less straightforward would be such a decorator that would work regardless of an specific base class or custom-metaclass - and this also can be done: by injecting in the class being constructed an __init_subclass__ method which will perform a check of violation of the final clause.
The complicated part is to co-exist with eventual pre-existing __init_subclass__ methods which also need to be called, either on the same class or in any superclass, as well as emulate the working of super(), since we are creating a method outside the class body. The decorator code can inspect the context from which its called and inject a __init_subclass__ there, taking some care:
import sys
def final(method):
f = sys._getframe().f_back
_init_subclass_meth = "super"
def __init_subclass__(cls, *args, **kw):
# all of these are readonly, so nonlocal is optional:
# nonlocal final_methods, _init_subclass_meth, _original_init_subclass, __init_subclass__
# In a normal __init_subclass__, one can know about the class in which
# a method is declared, and call super(), via the `__class__`
# magic variable. But that won't work for a method defined
# outside the class and inkected in it.
# the line bellow should retrieve the equivalent to __class__
current_class = next(supercls for supercls in cls.__mro__ if getattr(supercls.__dict__.get("__init_subclass__", None), "__func__", None) is __init_subclass__)
for meth_name in cls.__dict__:
if meth_name in final_methods:
raise RuntimeError(f"Final method {meth_name} is redeclared in subclass {cls.__name__} from {current_class.__name__}")
if _init_subclass_meth == "wrap":
return _original_init_subclass(cls, *args, **kwd)
return super(current_class, None).__init_subclass__(*args, **kw)
__init_subclass__._final_mark = True
if "__init_subclass__" in f.f_locals and not getattr(f.f_locals["__init_subclass__"], "_final_mark", False):
_init_subclass_meth = "wrap"
_original_init_subclass = f.f_locals["__init_subclass__"]
# locals assignment: will work in this case because the caller context
# is a class body, inside which `f_locals` refers usually to a
# plain dict (unless a custom metaclass changed it).
# This normally would not work (= no effect) in an ordinary frame,
# represnting a plain function or method in execution:
f.f_locals["__init_subclass__"] = __init_subclass__
final_methods = f.f_locals.setdefault("_final_methods", set())
final_methods.add(method.__name__)
return method
class A:
#final
def b(self):
print("final b")
And this will raise an error:
class B(A):
def b(self):
# RuntimeError expected
...
I'm trying to add a __set_name__ hook to the descriptor produced by functools.wraps inside a decorator, but it is never called and I don't see any error messages:
import functools
def wrap(fn):
"""Decorator."""
#functools.wraps(fn)
def w(*args, **kwargs):
return fn(*args, **kwargs)
# This never gets called.
def set_name(self, obj, name):
print(f"inside __set_name__: {self=}, {obj=}, {name=}")
w.__set_name__ = set_name.__get__(w)
return w
class Foo:
#wrap
def foo(self):
pass
From what I understand, wrap() is called and its return value bound to the foo variable in the class's execution frame before the Foo class is created, so the __set_name__ hook should be in place by the time Python looks for it. So why isn't it being called?
Whenever Python looks for magic methods, it looks on the type of the object, not the instance. What you've done is take a function object (the return value of functools.wrap, in this case) and assign something on its __dict__. But for efficiency (and correctness, in some cases), special methods like __set_name__ bypass __dict__ and look on the type object directly. See Special method lookup for details and a rationale.
To make your code work, you need to create a custom callable class (i.e. a class which defines a function called __call__), define __set_name__ on that class, and then make w an instance of that class.
I am writing a class for a neural network and I want to give it some form of customization, so that you can choose different cost functions and regularizations. For this I want to set them as default parameters in the __init__() method.
But when I pass MyClass.static_method in my example, the Interpreter then tells me that MyClass is not (yet) defined. Why is this and is there a nicer workaround than mine?
You can of course just set the static method as a default parameter, but then other problems arise. For example, if I want to access the functions name (which I actually want), I cannot use __name__ rightaway. I know how to do it another way, by accessing static_method.__func__.__name__. But this seems clumsy and as you get a staticmethod object, seems like its not intended to be used this way.
class MyClass:
#staticmethod
def static_method():
do_something()
def __init__(self, func=MyClass.static_method, func2=static_method):
self.name = func.__name__ #Does not work
self.name2 = func2.__func__.__name__ #Should work
I did expect for the MyClass.static_method to work, but the class does not seem to exist then. So, one last time, why?
The reason you're having problems with your static method usage as a default argument is due to a combination of two issues.
The first issue is that the default argument needs to be well defined when the def statement is run, not only when the function is called. That's because the default argument gets built into the function object, rather than being recalculated each time the function runs (this is the same reason why a mutable default argument like an empty list is often an error). Anyway, this is why you can't use MyClass.static_method as the default argument, since MyClass isn't defined yet when the function is being defined (the class object is only made after all its contents have been created).
The next issue is that a staticmethod object doesn't have all the same attributes and methods as a regular function. Normally this doesn't matter, as when you access it through a class object (e.g. MyClass.static_method once MyClass exists) or through an instance (e.g. self.static_method), it will be callable and have a __name__. But that's because you get the underlying function in those situations, rather than the staticmethod object itself. The staticmethod object itself is a descriptor, but not a callable.
So neither of these functions will work correctly:
class MyClass:
#staticmethod
def static_method():
pass
def foo(self, func=MyClass.static_method): # won't work because MyClass doesn't exist yet
pass
def bar(self, func=static_method): # this declaration will work (if you comment out foo)
name = func.__name__ # but this doesn't work when the bar() is called
func() # nor this, as func is the staticmethod object
What does work would be to use the actual function underlying the staticmethod object as the default:
def baz(self, func=static_method.__func__): # this works!
name = func.__name__
func()
This also works when you pass in some other function (or bound method), unlike the version of your code that used name = func.__func__.__name__.
DEFAULT = object()
class MyClass:
#staticmethod
def static_method():
do_something()
def __init__(self, func=DEFAULT, func2=DEFAULT):
self.name = self.static_method.__name__ if func is DEFAULT else func.__name__
self.name2 = self.static_method.__func__.__name__ if func2 is DEFAULT else func2.__func__.__name__
I guess??
There are some interesting ways to run a method before every method in a class in questions such as Python: Do something for any method of a class?
However that solution doesn't let us pass arguments.
There's a decorator solution on Catch "before/after function call" events for all functions in class but I don't want to have to go back and decorate all my classes.
Is there a way to run a pre/post operation that's dependent on the arguments passed for every invocation of an object's method?
Example:
class Stuff(object):
def do_stuff(self, stuff):
print(stuff)
a = Stuff()
a.do_stuff('foobar')
"Pre operation for foobar"
"foobar"
"Post operation for foobar"
So I figured it out after a lot of experimentation.
Basically in the metaclass' __new__ you can iterate through every method in the class' namespace and swap out every method in the class being created with a new version that runs the before logic, the function itself, and the after logic. Here's a sample:
class TestMeta(type):
def __new__(mcl, name, bases, nmspc):
def replaced_fnc(fn):
def new_test(*args, **kwargs):
# do whatever for before function run
result = fn(*args, **kwargs)
# do whatever for after function run
return result
return new_test
for i in nmspc:
if callable(nmspc[i]):
nmspc[i] = replaced_fnc(nmspc[i])
return (super(TestMeta, mcl).__new__(mcl, name, bases, nmspc))
Note that if you use this code as is it will run the pre/post operation for init and other builtin functions as well.
When you decorate a method, it is not bound yet to the class, and therefor doesn't have the im_class attribute yet. I looking for a way to get the information about the class inside the decorator. I tried this:
import types
def decorator(method):
def set_signal(self, name, value):
print name
if name == 'im_class':
print "I got the class"
method.__setattr__ = types.MethodType(set_signal, method)
return method
class Test(object):
#decorator
def bar(self, foo):
print foo
But it doesn't print anything.
I can imagine doing this:
class Test(object):
#decorator(klass=Test)
def bar(self, foo):
print foo
But if I can avoid it, it would make my day.
__setattr__ is only called on explicit object.attribute = assignments; building a class does not use attribute assignment but builds a dictionary (Test.__dict__) instead.
To access the class you have a few different options though:
Use a class decorator instead; it'll be passed the completed class after building it, you could decorate individual methods on that class by replacing them (decorated) in the class. You could use a combination of a function decorator and a class decorator to mark which methods are to be decorated:
def methoddecoratormarker(func):
func._decorate_me = True
return func
def realmethoddecorator(func):
# do something with func.
# Note: it is still an unbound function here, not a method!
return func
def classdecorator(klass):
for name, item in klass.__dict__.iteritems():
if getattr(item, '_decorate_me', False):
klass.__dict__[name] = realmethoddecorator(item)
You could use a metaclass instead of a class decorator to achieve the same, of course.
Cheat, and use sys._getframe() to retrieve the class from the calling frame:
import sys
def methoddecorator(func):
callingframe = sys._getframe(1)
classname = callingframe.f_code.co_name
Note that all you can retrieve is the name of the class; the class itself is still being built at this time. You can add items to callingframe.f_locals (a mapping) and they'll be made part of the new class object.
Access self whenever the method is called. self is a reference to the instance after all, and self.__class__ is going to be, at the very least, a sub-class of the original class the function was defined in.
My strict answer would be: It's not possible, because the class does not yet exist when the decorator is executed.
The longer answer would depend on your very exact requirements. As I wrote, you cannot access the class if it does not yet exists. One solution would be, to mark the decorated method to be "transformed" later. Then use a metaclass or class decorator to apply your modifications after the class has been created.
Another option involves some magic. Look for the implementation of the implements method in zope.interfaces. It has some access to the information about the class which is just been parsed. Don't know if it will be enough for your use case.
You might want to take a look at descriptors. They let you implement a __get__ that is used when an attribute is accessed, and can return different things depending on the object and its type.
Use method decorators to add some marker attributes to the interesting methods, and use a metaclass which iterates over the methods, finds the marker attributes, and does the logic. The metaclass code is run when the class is created, so it has a reference to the newly created class.
class MyMeta(object):
def __new__(...):
...
cls = ...
... iterate over dir(cls), find methods having .is_decorated, act on them
return cls
def decorator(f):
f.is_decorated = True
return f
class MyBase(object):
__metaclass__ = MyMeta
class MyClass(MyBase):
#decorator
def bar(self, foo):
print foo
If you worry about that the programmer of MyClass forgets to use MyBase, you can forcibly set the metaclass in decorator, by exampining the globals dicitionary of the caller stack frame (sys._getframe()).