I would like to add some small functionality to a method of another class' instance.
What I tried is listed below. With the same approach it's possible to completely change the method. But I would like to keep a slightly altered version of the original method. Is that possible?
BTW: The classes can not inherit from one another, because I don't necessarily know what class B is.
class A:
def alter_fn(self, obj):
def wrapper():
func = getattr(obj, obj.fn_name)
out = func()
print('out = ', out)
return out
setattr(obj, obj.fn_name, wrapper)
return
class B:
def __init__(self) -> None:
self.fn_name = 'fn'
def fn(self):
return 123
a=A()
b=B()
a.alter_fn(b)
b.fn()
For obvious reasons this raises a recursion depth error but I do not know how to avoid that. I also tried func = copy(getattr(obj, obj.fn_name))
After altering the method b.fn I would like it to return the value, the unaltered function would return (here 123) but I would also like it to do something else with this output, in this example print it.
Move the assignment of func above the definition of wrapper.
...
def alter_fn(self, obj):
func = getattr(obj, obj.fn_name)
def wrapper():
...
This way getattr is called only once and wrapper can access func as a closure variable.
You just need to take the func = getattr(obj, obj.fn_name) out of the wrapperfunction.
class A:
def alter_fn(self, obj):
func = getattr(obj, obj.fn_name)
def wrapper():
out = func()
print('out = ', out)
return out
setattr(obj, obj.fn_name, wrapper)
return
You need to get the wrapped function before re-assigning it:
def alter_fn(self, obj):
func = getattr(obj, obj.fn_name)
def wrapper():
out = func()
print('out = ', out)
return out
setattr(obj, obj.fn_name, wrapper)
return
This way func is assigned only once, when the original function is called. Thus, you can safely call it without calling wrapper() again by accident.
Also consider using functools.wraps() to keep the docstring and name information of the string the same.
Related
I am not new to python but I am far from being an expert (or intermediate). Right now, I play around with objects and their behavior (like setattr, monkey-patch, etc.). During this, I stumbled upon a problem where I do not have any idea on how this might work.
Imagine following code:
class MyCalculator():
def __init__(self):
pass
def addition(self, a, b):
return a + b
def substraction(self, a, b):
return a - b
import inspect
class Changing():
def __init__(self):
pass
def listUserMethods(self, myObject):
object_names = [object_name for object_name in inspect.getmembers(myObject) if (inspect.ismethod(object_name[1]))]
return object_names
def setMethodAttribute(self, myMethod):
pass
if __name__=="__main__":
myCalc = MyCalculator()
change = Changing()
Now, I would like that setMethodAttribute() will change the code of the method I provide itself. Like, inserting a print() statement before the rest of the original method is executed. E.g. printing the input parameter before executing the addition, etc.
In my case, this does not need to be done during runtime (even if this is very interesting to know). I could imagine, that using inheritance or something similar could be a way. Perhaps somebody has a great idea?
Thanks for the help!
The answer really depends what you are after.
Wrapping a method of a class (before runtime)
This is very typical use case of decorators (the #something above a function definition).
def with_printing(func):
def wrapper(*args, **kwargs):
print("Before calling method")
ret = func(*args, **kwargs)
print("After calling method")
return ret
return wrapper
class MyCalculator:
#with_printing
def addition(self, a, b):
print("calling addition")
return a + b
If you want to keep the docstring of the original method, you would use the functools.wraps().
Example output
mycalc = MyCalculator()
print(mycalc.addition(2, 3))
would print
Before calling method
calling addition
After calling method
5
Wrapping a method of an object instance (runtime)
Here is one implementation which changes the method of an object. Note that this changes the method of an instance and not every instance of that class.
class MyCalculator:
def addition(self, a, b):
print("calling addition")
return a + b
class Changing:
def set_method_attribute(self, obj, method_name):
method = getattr(obj, method_name)
def wrapper(*args, **kwargs):
print("Before calling method")
ret = method(*args, **kwargs)
print("After calling method")
return ret
setattr(obj, method_name, wrapper)
Example usage
# Create two calculator objects for testing
mycalc = MyCalculator()
mycalc2 = MyCalculator()
change = Changing()
# Before change
print(mycalc.addition(2, 3))
print("----")
# After change
change.set_method_attribute(mycalc, "addition")
print(mycalc.addition(2, 3))
print("----")
# The another calculator object remains unchanged
print(mycalc2.addition(2, 3))
will print
calling addition
5
----
Before calling method
calling addition
After calling method
5
----
calling addition
5
I recreated some of the code in a new way. Would you mind taking a look on it and telling me if this is a "good" way?
import inspect
class InBetween():
def __init__(self):
object_names = [object_name for object_name in inspect.getmembers(self) if (inspect.ismethod(object_name[1]) and (object_name[0] != 'with_print'))]
for name in object_names:
method = getattr(self, name[0])
wrapper = self.with_print(method)
setattr(self, name[0], wrapper)
def with_print(self, method):
def wrapper(*args, **kwargs):
print("before")
ret = method(*args, **kwargs)
print("after")
return ret
return wrapper
class MyCalculator(InBetween):
def __init__(self):
super().__init__()
def addition(self, a, b):
return a + b
def substraction(self, a, b):
return a - b
def multiply(self, a, b):
return a * b
if __name__=="__main__":
myCalc = MyCalculator()
print(myCalc.addition(2,5))
print(myCalc.multiply(2,5))
The basic idea is, that every class wich will inherit "InBetween" can add via super() the wrapper to each method. Without doing this manually in the script. My next idea is, to replace the print statements by logging etc. At the end, if I call the parent "init" infromation will be easily logged, and if not, "nothing" happens.
Love to hear other opinions on that!
Thank you all!
I would like to create a python context manager, which would allow the following (with reverse_decorator applying the decorated function with first argument reversed if it is string):
print('hi')
with MyFunctionDecorator('print', reverse_decorator):
print('hello')
print('bye')
to result in:
hi
olleh
bye
The point is not the print function itself, but writing this kind of context manager, that could decorate any function - local, global, builtin, from whatever module. Is this even possible in python? How should I do it?
EDIT: To clarify a bit, the point was not to have to change the code inside the with context.
This is my approach:
from contextlib import contextmanager
from importlib import import_module
#contextmanager
def MyFunctionDecorator(func, decorator):
if hasattr(func, '__self__'):
owner = func.__self__
elif hasattr(func, '__objclass__'):
owner = func.__objclass__
else:
owner = import_module(func.__module__)
qname = func.__qualname__
while '.' in qname:
parent, qname = qname.split('.', 1)
owner = getattr(owner, parent)
setattr(owner, func.__name__, decorator(func))
yield
setattr(owner, func.__name__, func)
# Example decorator, reverse all str arguments
def reverse_decorator(f):
def wrapper(*args, **kwargs):
newargs = []
for arg in args:
newargs.append(arg[::-1] if isinstance(arg, str) else arg)
newkwargs = {}
for karg, varg in kwargs.values():
newkwargs[karg] = varg[::-1] if isinstance(varg, str) else varg
return f(*newargs, **newkwargs)
return wrapper
# Free functions
print('hi')
with MyFunctionDecorator(print, reverse_decorator):
print('hello')
print('bye')
# Class for testing methods (does not work with builtins)
class MyClass(object):
def __init__(self, objId):
self.objId = objId
def print(self, arg):
print('Printing from object', self.objId, arg)
# Class level (only affects instances created within managed context)
# Note for decorator: first argument of decorated function is self here
with MyFunctionDecorator(MyClass.print, reverse_decorator):
myObj = MyClass(1)
myObj.print('hello')
# Instance level (only affects one instance)
myObj = MyClass(2)
myObj.print('hi')
with MyFunctionDecorator(myObj.print, reverse_decorator):
myObj.print('hello')
myObj.print('bye')
Output:
hi
olleh
bye
Printing from object 1 olleh
Printing from object 2 hi
Printing from object 2 olleh
Printing from object 2 bye
This should work across functions and other modules and so on, since it modifies the attributes of the module or class. Class methods are complicated, because once you create an instance of a class its attributes point to the functions defined in the class at the time the object was created, so you have to choose between modifying the behavior of a particular instance or modifying the behavior of new instances within the managed context, as in the example. Also, trying to decorate methods of builtin classes like list or dict does not work.
It is possible if you modify it add a bit:
print('hi')
with MyFunctionDecorator(print, reverse_decorator) as print:
print('hello')
print('bye')
Here is a definition that works for this example*:
def reverse_decorator(func):
def wrapper(*args, **kwargs):
if len(args) == 1 and not kwargs and isinstance(args[0], str):
return func(args[0][::-1])
return func(*args, **kwargs)
return wrapper
class MyFunctionDecorator:
def __init__(self, func, decorator):
self.func = func
self.decorator = decorator
def __enter__(self):
"""Return the decorated function"""
return self.decorator(self.func)
def __exit__(self, *args):
"""Reset the function in the global namespace"""
globals()[self.func.__name__] = self.func
But its probably easier to just do it explicitly, following the Python Zen:
print('hi')
print('hello'[::-1])
print('bye')
*This code does not work under many circumstances, as #AranFey noted in the comments:
Inside functions
If the function you want to decorate is imported with import x from y as z
If you care that afterwards you have a print function defined in the globals(), instead of directly being a built-in
Since this is more a proof-of-concept, that yes, one can write a decorator that works in this example, I will not try to fix these shortcomings. Just use the way I gave above, or use only the decorator:
print('hi')
reverse_decorator(print)('hello')
print('bye')
I'm writing some code and I have a dictionary where the key is any string, and the value is a function. I then loop through each key in the dictionary and call the functions, like so:
class SomeClass:
dictionary = {}
# Not sure how to use this decorator function
def decorator(key):
def wrapper(funct):
self.dictionary[key] = funct
return funct
return wrapper
#decorator("some_val1")
def function_1(self):
...
#decorator("some_val2")
def function_2(self):
...
#decorator("some_val3")
def function_3(self):
...
def execute_all_functions(self):
for key, _ in self.dictionary.items():
self.dictionary[key]()
if __name__ == "__main__":
sclass = SomeClass()
sclass.execute_all_functions()
So this should populate dictionary with:
{
"some_val1": function_1(),
"some_val2": function_2(),
"some_val3": function_3()
}
I'm getting this error though
self.dictionary[key] = funct
NameError: name 'self' is not defined
How would I be able to do this. Help appreciated.
I don't think it's possible.
First you should read this: https://docs.python.org/3.3/howto/descriptor.html , to know the difference of function vs method.
In your code, the key equals self of a method.
def decorator(key):
def wrapper(funct):
self.dictionary[key] = funct
return funct
return wrapper
If you want to use a class's property, you should refer by cls. Correct code might be:
#classmethod
def decorator(cls, key):
def wrapper(funct):
self.dictionary[key] = funct
return funct
return wrapper
So let's say if you want to update a class property, you must have cls reference. I have tried the code below, make the decorator_maker a classmethod.
class SomeClass:
dictionary = {}
#classmethod
def decorator_maker(cls, key):
print(cls, key)
def decorator(funct):
cls.dictionary[key] = funct
return funct
return decorator
#decorator_maker("some_val1")
def function_1(self):
...
#decorator_maker("some_val2")
def function_2(self):
...
#decorator_maker("some_val3")
def function_3(self):
...
def execute_all_functions(self):
for key, _ in self.dictionary.items():
self.dictionary[key]()
And you will get an error like TypeError: 'classmethod' object is not callable. Same as this question: 'classmethod' object is not callable. AKA, you can't call a classmethod until the class is defined.
So you may want to make the decorator outside the class. But for the same reason, you can't get a reference of cls until the classmethod. method is also an attribute of class, you can't dynamic change an attribute while define another. see Python class method run when another method is invoked.
It would be easier if you move dictionary outside the class.
functools.wraps might be useful. In the bellow trivial example, without wraps the decorator will not function correctly.
from functools import wraps
class SomeClass:
var = 1
#wraps
def decorator(self, fn):
return fn
#decorator
def return_value(self):
print(self.var)
return self.var
if __name__ == "__main__":
sclass = SomeClass()
sclass.return_value()
Assuming the following structure:
class SetupTestParam(object):
def setup_method(self, method):
self.foo = bar()
#pytest.fixture
def some_fixture():
self.baz = 'foobar'
I use SetupTestParam as a parent class for test classes.
class TestSomething(SetupTestParam):
def test_a_lot(self, some_fixture):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
... # not repetative code here
assert spam == 'something cool'
Now, writing tests gets repetitive (with statement usage) and I would like to write a decorator to reduce the number of code lines. But there is a problem with pytest and the function signature.
I found out library which should be helpful but I can't manage to get it to work.
I made a classmethod in my SetupTestParam class.
#classmethod
#decorator.decorator
def this_is_decorator(cls, f):
def wrapper(self, *args, **kw):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return f(self, *args)
return wrapper
After I decorate the test_a_lot method, I receive the error TypeError: transaction_decorator() takes exactly 1 argument (2 given)
Can someone explain me please what am I doing wrong? (I assume there is a problem with self from the test method?)
Chaining decorators is not the simplest thing to do. One solution might be to separate the two decorators. Keep the classmethod but move decorator.decorator to the end:
#classmethod
def this_is_decorator(cls, f):
def wrapper(self, *args, **kw):
with self.baz as magic:
with magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return f(self, *args)
return decorator.decorator(wrapper, f)
Maybe this works for you.
After some tweaking and realizing that I need to pass a parameter to decorator I choosed to write it as a class:
class ThisIsDecorator(object):
def __init__(self, param):
self.param = param # Parameter may vary with the function being decorated
def __call__(self, fn):
wraps(fn) # [1]
def wrapper(fn, fn_self, *args): # [2] fn_self refers to original self param from function fn (test_a_lot) [2]
with fn_self.baz as fn_self.magic: # I pass magic to fn_self to make magic accesible in function fn (test_a_lot)
with fn_self.magic.fooz as more_magic:
blah = self.param.much_more_magic() # repetative bleh
return fn(fn_self, *args)
return decorator.decorator(wrapper, fn)
[1] I use wraps to have original fn __name__, __module__ and __doc__.
[2] Params passed to wrapper were self = <function test_a_lot at 0x24820c8> args = (<TestSomething object at 0x29c77d0>, None, None, None, None), kw = {} so I took out the args[0] as a fn_self.
Original version (without passing a parameter):
#classmethod
def this_is_decorator(cls, fn):
#wraps(fn)
def wrapper(fn, fn_self, *args):
with fn_self.baz as fn_self.magic:
with fn_self.magic.fooz as more_magic:
blah = more_magic.much_more_magic() # repetative bleh
return fn(fn_self, *args)
return decorator.decorator(wrapper,fn)
Thanks go to Mike Muller for pointing out the right direction.
Here's what happens in time order as this method is defined.
this_is_decorator is created (not called).
decorator.decorator(this_is_decorator) is called. This returns a new function which becomes this_is_decorator and has the same usage.
classmethod(this_is_decorator) is called, and the result of that is a classmethod that accepts (cls, f) and returns wrapper.
Later at runtime, a call to this_is_decorator will return wrapper.
But considering that this_is_decorator is a class method, it's not clear to me that this is what you want. I'm guessing that you may want something more like this:
from decorator import decorator
#decorator
def mydecorator(f):
def wrapper(cls, *args, **kw):
# ... logging, reporting, caching, whatever
return f(*args, **kw)
return wrapper
class MyClass(object):
#classmethod
#mydecorator
def myclsmethod(a, b, c):
# no cls or self value accepted here; this is a function not a method
# ...
Here your decorator is defined outside your class, because it's changing an ordinary function into a classmethod (and because you may want to use it in other places). The order of execution here is:
mydecorator is defined, not called.
decorator(mydecorator) is called, and the result becomes mydecorator.
Creation of MyClass starts.
myclsmethod is created. It is an ordinary function, not a method. There is a difference within the VM, so that you do not have to explicitly supply cls or self arguments to methods.
myclsmethod is passed to mydecorator (which has itself been decorated before) and the result (wrapper) is still a function not a method.
The result of mydecorator is passed to classmethod which returns an actual class method that is bound to MyClass.myclsmethod.
Definition of MyClass finishes.
Later when MyClass.myclsmethod(a, b, c) is called, wrapper executes, which then calls the original myclsmethod(a, b, c) function (which it knows as f) without supplying the cls argument.
Since you have an additional need to preserve the argument list exactly, so that even the names of the arguments are preserved in the decorated function, except with an extra initial argument cls, then you could implement mydecorator this way:
from decorator import decorator
from inspect import getargspec
#decorator
def mydecorator(func):
result = [None] # necessary so exec can "return" objects
namespace = {'f': func, 'result': result}
source = []
add = lambda indent, line: source.append(' ' * indent + line) # shorthand
arglist = ', '.join(getargspec(func).args) # this does not cover keyword or default args
add(0, 'def wrapper(cls, %s):' % (arglist,))
add(2, 'return f(%s)' % (arglist,))
add(0, 'result[0] = wrapper') # this is how to "return" something from exec
exec '\n'.join(source) in namespace
return result[0] # this is wrapper
It's kinda ugly, but this is the only way I know to dynamically set the argument list of a function based on data. If returning a lambda is OK, you could use eval instead of exec, which eliminates the need for an array that gets written into but is otherwise about the same.
If I have a function :
#aDecorator
def myfunc1():
# do something here
if __name__ = "__main__":
# this will call the function and will use the decorator #aDecorator
myfunc1()
# now I want the #aDecorator to be replaced with the decorator #otherDecorator
# so that when this code executes, the function no longer goes through
# #aDecorator, but instead through #otherDecorator. How can I do this?
myfunc1()
Is it possible to replace a decorator at runtime?
As Miya mentioned, you can replace the decorator with another function any point before the interpreter gets to that function declaration. However, once the decorator is applied to the function, I don't think there is a way to dynamically replace the decorator with a different one. So for example:
#aDecorator
def myfunc1():
pass
# Oops! I didn't want that decorator after all!
myfunc1 = bDecorator(myfunc1)
Won't work, because myfunc1 is no longer the function you originally defined; it has already been wrapped. The best approach here is to manually apply the decorators, oldskool-style, i.e:
def myfunc1():
pass
myfunc2 = aDecorator(myfunc1)
myfunc3 = bDecorator(myfunc1)
Edit: Or, to be a little clearer,
def _tempFunc():
pass
myfunc1 = aDecorator(_tempFunc)
myfunc1()
myfunc1 = bDecorator(_tempFunc)
myfunc1()
I don't know if there's a way to "replace" a decorator once it has been applied, but I guess that probably there's not, because the function has already been changed.
You might, anyway, apply a decorator at runtime based on some condition:
#!/usr/bin/env python
class PrintCallInfo:
def __init__(self,f):
self.f = f
def __call__(self,*args,**kwargs):
print "-->",self.f.__name__,args,kwargs
r = self.f(*args,**kwargs)
print "<--",self.f.__name__,"returned: ",r
return r
# the condition to modify the function...
some_condition=True
def my_decorator(f):
if (some_condition): # modify the function
return PrintCallInfo(f)
else: # leave it as it is
return f
#my_decorator
def foo():
print "foo"
#my_decorator
def bar(s):
print "hello",s
return s
#my_decorator
def foobar(x=1,y=2):
print x,y
return x + y
foo()
bar("world")
foobar(y=5)
Here's a terrific recipe to get you started. Basically, the idea is to pass a class instance into the decorator. You can then set attributes on the class instance (make it a Borg if you like) and use that to control the behavior of the decorator itself.
Here's an example:
class Foo:
def __init__(self, do_apply):
self.do_apply = do_apply
def dec(foo):
def wrap(f):
def func(*args, **kwargs):
if foo.do_apply:
# Do something!
pass
return f(*args, **kwargs)
return func
return wrap
foo = Foo(False)
#dec(foo)
def bar(x):
return x
bar('bar')
foo.do_apply = True
# Decorator now active!
bar('baz')
Naturally, you can also incorporate the "decorator decorator" to preserve signatures, etc.
If you want to explicitely change the decorator, you might as well choose a more explicit approach instead of creating a decorated function:
deco1(myfunc1, arg1, arg2)
deco2(myfunc1, arg2, arg3)
deco1() and deco2() would apply the functionality your decorators provide and call myfunc1() with the arguments.
Sure - you can get the function object and do whatever you want with it:
# Bypass a decorator
import types
class decorator_test(object):
def __init__(self, f):
self.f = f
def __call__(self):
print "In decorator ... entering: ", self.f.__name__
self.f()
print "In decorator ... exiting: ", self.f.__name__
#decorator_test
def func1():
print "inside func1()"
print "\nCalling func1 with decorator..."
func1()
print "\nBypassing decorator..."
for value in func1.__dict__.values():
if isinstance(value, types.FunctionType) and value.func_name == "func1":
value.__call__()
I know it's an old thread, but I had fun doing this
def change_deco(name, deco, placeholder=' #'):
with open(name + '.py', 'r') as file:
lines = file.readlines()
for idx, string in enumerate(lines):
if placeholder in string and repr(placeholder) not in string:
lines[idx] = f' #{deco}\r\n'
exec(''.join(lines))
return locals()[name]
If the decorator is a function, just replace it.
aDecorator = otherDecorator