I'm attempting to create some decorators which will allow class members to be decorated on instantiation, because I'd like to have some instances which are decorated and others which aren't.
In the example below, the intended outcome of applying the once decorator to an instance of SomeClass is that when some_func has been called, calling other_func prints a message rather than calling the original function.
#!/usr/bin/env python
import functools
def some_once(func):
#functools.wraps(func)
def wrapped(self, *args, **kwargs):
if not self._new_attr:
print("don't have new attr yet")
func(self, *args, **kwargs)
self._new_attr = True
return wrapped
def other_once(func):
#functools.wraps(func)
def wrapped(self, *args, **kwargs):
if self._new_attr:
print("We have a new attr")
else:
func(self, *args, **kwargs)
return wrapped
def once(cls):
setattr(cls, "_new_attr", False)
setattr(cls, "some_func", some_once(cls.some_func))
setattr(cls, "other_func", other_once(cls.other_func))
return cls
class SomeClass:
def some_func(self, parameter):
return "The parameter is " + str(parameter)
def other_func(self, parameter):
return "The other parameter is " + str(parameter)
if __name__ == '__main__':
a = SomeClass()
print(dir(a))
print(a.some_func("p1"))
print(a.other_func("p2"))
b = once(SomeClass())
print(dir(b))
print(b.some_func("p3"))
print(b.other_func("p4"))
The problem that results is that rather than looking at self._new_attr, the decorated functions instead look at string._new_attr, where the string is the parameter to the functions. I'm confused about what I'm doing wrong here.
Don't decorate an instance. Make a new, decorated class.
Changing this line:
b = once(SomeClass())
into:
b = once(SomeClass)()
Should do the trick. once(SomeClass) gives you a new, decorated class. In the next step make an instance of it.
Related
In the following code the methods(print_a and print_b) of class Test, has been decorated by the two different decorators.
How can I determine that the given method(let say print_a), is decorated with some specific decorator (decorator1) at the runtime?
The easy solution is to change the name of the wrapper function, i.e. changing the wrapper to wrapper1 and wrapper2 in the decorator method, but the problem with that is I don't have control over that part of the code.
Does python have any reflection API like Java, and could that help me here ?
def my_decorator1(func):
def wrapper(*args, **kwargs):
print('I am decorated:1')
func(*args, **kwargs)
return wrapper
def my_decorator2(func):
def wrapper(*args, **kwargs):
print('I am decorated:2')
func(*args, **kwargs)
return wrapper
class Test():
def __init__(self, a=None, b=None):
self.a = a
self.b = b
#my_decorator1
def print_a(self):
print('Value of a is {}'.format(self.a))
#my_decorator2
def print_b(self):
print('Value of b is {}'.format(self.b))
if __name__ == '__main__':
d = Test.__dict__
f1 = d.get('print_a')
f2 = d.get('print_b')
Since the decorator function is executed at decoration time, it is difficult to get the name. You can solve your problem with an additional decorator. The decorator annotator writes into a property of the function and you can read this name inside of the decorated function:
def annotator(decorator):
def wrapper(f):
g = decorator(f)
g.decorator = decorator.__name__
return g
return wrapper
#annotator(my_decorator1)
def x():
print x.decorator
Python decorators are more about transforming functions than about acting as simple metadata labels, as one often does with Java annotations. So to do this, I'd have the decorator set a property on the function it is wrapping (or perhaps wrap it in a callable of a particular type):
def my_decorator1(func):
def wrapper(*args, **kwargs):
print('I am decorated:1')
func(*args, **kwargs)
wrapper.decorator_name = 'my_decorator1'
return wrapper
Then:
print(Test.print_a.decorator_name)
Alternatively, in Python 3.3+ you can use PEP 3155's __qualname__ to identify the decorator.
This is a question similar to How to call a method implicitly after every method call? but for python
Say I have a crawler class with some attributes (e.g. self.db) with a crawl_1(self, *args, **kwargs) and another one save_to_db(self, *args, **kwargs) which saves the crawling results to a database (self.db).
I want somehow to have save_to_db run after every crawl_1, crawl_2, etc. call. I've tried making this as a "global" util decorator but I don't like the result since it involves passing around self as an argument.
If you want to implicitly run a method after all of your crawl_* methods, the simplest solution may be to set up a metaclass that will programatically wrap the methods for you. Start with this, a simple wrapper function:
import functools
def wrapit(func):
#functools.wraps(func)
def _(self, *args, **kwargs):
func(self, *args, **kwargs)
self.save_to_db()
return _
That's a basic decorator that wraps func, calling
self.save_to_db() after calling func. Now, we set up a metaclass
that will programatically apply this to specific methods:
class Wrapper (type):
def __new__(mcls, name, bases, nmspc):
for attrname, attrval in nmspc.items():
if callable(attrval) and attrname.startswith('crawl_'):
nmspc[attrname] = wrapit(attrval)
return super(Wrapper, mcls).__new__(mcls, name, bases, nmspc)
This will iterate over the methods in the wrapped class, looking for
method names that start with crawl_ and wrapping them with our
decorator function.
Finally, the wrapped class itself, which declares Wrapper as a
metaclass:
class Wrapped (object):
__metaclass__ = Wrapper
def crawl_1(self):
print 'this is crawl 1'
def crawl_2(self):
print 'this is crawl 2'
def this_is_not_wrapped(self):
print 'this is not wrapped'
def save_to_db(self):
print 'saving to database'
Given the above, we get the following behavior:
>>> W = Wrapped()
>>> W.crawl_1()
this is crawl 1
saving to database
>>> W.crawl_2()
this is crawl 2
saving to database
>>> W.this_is_not_wrapped()
this is not wrapped
>>>
You can see the our save_to_database method is being called after
each of crawl_1 and crawl_2 (but not after this_is_not_wrapped).
The above works in Python 2. In Python 3, replase this:
class Wrapped (object):
__metaclass__ = Wrapper
With:
class Wrapped (object, metaclass=Wrapper):
Something like this:
from functools import wraps
def my_decorator(f):
#wraps(f)
def wrapper(*args, **kwargs):
print 'Calling decorated function'
res = f(*args, **kwargs)
obj = args[0] if len(args) > 0 else None
if obj and hasattr(obj, "bar"):
obj.bar()
return wrapper
class MyClass(object):
#my_decorator
def foo(self, *args, **kwargs):
print "Calling foo"
def bar(self, *args, **kwargs):
print "Calling bar"
#my_decorator
def example():
print 'Called example function'
example()
obj = MyClass()
obj.foo()
It will give you the following output:
Calling decorated function
Called example function
Calling decorated function
Calling foo
Calling bar
A decorator in Python looks like this, it's a method taking a single method as argument and returning another wrapper method that shall be called instead of the decorated one. Usually the wrapper "wraps" the decorated method, i.e. calls it before/after performing some other actions.
Example:
# define a decorator method:
def save_db_decorator(fn):
# The wrapper method which will get called instead of the decorated method:
def wrapper(self, *args, **kwargs):
fn(self, *args, **kwargs) # call the decorated method
MyTest.save_to_db(self, *args, **kwargs) # call the additional method
return wrapper # return the wrapper method
Now learn how to use it:
class MyTest:
# The additional method called by the decorator:
def save_to_db(self, *args, **kwargs):
print("Saver")
# The decorated methods:
#save_db_decorator
def crawl_1(self, *args, **kwargs):
print("Crawler 1")
#save_db_decorator
def crawl_2(self, *args, **kwargs):
print("Crawler 2")
# Calling the decorated methods:
my_test = MyTest()
print("Starting Crawler 1")
my_test.crawl_1()
print("Starting Crawler 1")
my_test.crawl_2()
This would output the following:
Starting Crawler 1
Crawler 1
Saver
Starting Crawler 1
Crawler 2
Saver
See this code running on ideone.com
Suppose I have some python code, e.g. some class defined somewhere, which cannot be modified
class MyClass(object):
def __init__(self, arg1, arg2):
do_something...
def foo(self):
do_something
Now I want to add a trace capability, e.g. some mechanism from outside that traces each and every method call for the above class. I want to be able to print out when e.g, __init__ has been called, or foo or even the __del__ method of MyClass.
Is this possible to do, and how is this done best?
Create a proxy class that wraps the original class and then delegates the work after printing a trace:
class MyClassProxy(object):
def __init__(*args, **kwds):
print 'Initializing'
self.o = MyClass(*args, **kwds)
def foo(self):
print 'Fooing'
return self.o.foo()
You can create a trace decorator and attach it to all the methods of the class instance or class definition as shown in decorate_methods function.
import functools
import inspect
import types
class TestClass(object):
def func1(self):
pass
def func2(self, a, b):
pass
def trace(func):
#functools.wraps(func)
def decorator(*args, **kwargs):
print "TRACE:", func.__name__, args, kwargs
return func(*args, **kwargs)
return decorator
def decorate_methods(obj, decorator):
for name, func in inspect.getmembers(obj):
if isinstance(func, types.MethodType):
setattr(obj, name, decorator(func))
# Apply the decorator to a class instance
test1 = TestClass()
decorate_methods(test1, trace)
test1.func1()
test1.func2('bar1', b='bar2')
# Apply the decorator to the class definition
decorate_methods(TestClass, trace)
test2 = TestClass()
test2.func1()
test2.func2('bar1', b='bar2')
The output of the script will be:
TRACE: func1 () {}
TRACE: func2 ('bar1',) {'b': 'bar2'}
TRACE: func1 (<__main__.TestClass object at 0x7f5a8d888150>,) {}
TRACE: func2 (<__main__.TestClass object at 0x7f5a8d888150>, 'bar1') {'b': 'bar2'}
Use decorator as shown below:
def call_trace(orig_func):
def decorated_func(*args, **kwargs):
print "========>In function: " + orig_func.__name__ + "<========"
orig_func(*args, **kwargs)
return decorated_func
Apply this decorator to trace the function. It prints function name before entering the function.
Ex:
#call_trace
def foo(self):
do_something
Hope it helps.
[Update]: You can use metaclass, only thing you got to change is to add "metaclass" parameter to your class as shown below. As you can see, below code applies "call_trace" decorator to every function in the class "ExBase".
I tried this out yesterday, it worked fine. I am also new to python.:)
def call_trace(orig_func):
def inner_func(*args, **kwargs):
print ("function name:" + str(orig_func.__name__))
orig_func(*args, **kwargs)
return inner_func
class ExMeta(type):
def __new__(cls, name, bases, attrs):
for attr in attrs:
if hasattr(attrs[attr], '__call__'):
attrs[attr] = call_trace(attrs[attr])
return type.__new__(cls, name, bases, attrs)
class ExBase(metaclass=ExMeta):
x = "x"
y = "y"
def __init__(self):
self.__name = "name"
def getname(self):
return self.__name
b = ExBase()
b.getname()
Get the code for OnlinePythonTutor from github.com/pgbovine/OnlinePythonTutor/tree/master/v3.
You don't need to bother with all the JS stuff. Extract the files into some directory. You can run your scripts using python /path/to/my/OnlinePythonTutor-master/v3/generate_json_trace my_script.py
This will give you basically everything your program is doing in a step by step manner. It will probably be overkill so if you want look into the source code and the underlying source in bdb http://docs.python.org/2/library/bdb.html. The docs for bdb are horrible so I'm having trouble figuring out what exactly is going on but I think this is a pretty cool problem, good luck.
decorator 1:
def dec(f):
def wrap(obj, *args, **kwargs):
f(obj, *args,**kwargs)
return wrap
decorator 2:
class dec:
def __init__(self, f):
self.f = f
def __call__(self, obj, *args, **kwargs):
self.f(obj, *args, **kwargs)
A sample class,
class Test:
#dec
def disp(self, *args, **kwargs):
print(*args,**kwargs)
The follwing code works with decorator 1 but not with decorator 2.
a = Test()
a.disp("Message")
I dont understand why decorator 2 is not working here. Can someone help me with this?
When you decorate with the dec class, your disp method is no more an instance method, but an instance of class dec. So a.disp is just a plain member of Test, which happens to be callable because it has a __call__ method, and in the self passed as the first argument of its f instance is "Message" (it is by no way bound to the "test" instance).
with the decorator function:
a = Test()
print a.disp
# disp <bound method Test.wrap of <__main__.Test instance at 0xb739df0c>>
with the decorator class:
a = Test()
print a.disp
# disp <__main__.dec instance at 0xb739deec>
edit
That should answer your question far better and clearer than I did:
http://irrepupavel.com/documents/python/instancemethod/
Here's an example of what I mean:
class MyDecorator(object):
def __call__(self, func):
# At which point would I be able to access the decorated method's parent class's instance?
# In the below example, I would want to access from here: myinstance
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
class SomeClass(object):
##self.name = 'John' #error here
name="John"
#MyDecorator()
def nameprinter(self):
print(self.name)
myinstance = SomeClass()
myinstance.nameprinter()
Do I need to decorate the actual class?
class MyDecorator(object):
def __call__(self, func):
def wrapper(that, *args, **kwargs):
## you can access the "self" of func here through the "that" parameter
## and hence do whatever you want
return func(that, *args, **kwargs)
return wrapper
Please notice in this context that the use of "self" is just a convention, a method just uses the first argument as a reference to the instance object:
class Example:
def __init__(foo, a):
foo.a = a
def method(bar, b):
print bar.a, b
e = Example('hello')
e.method('world')
The self argument is passed as the first argument. Also your MyDecorator is a class emulating a function. Easier to make it an actual function.
def MyDecorator(method):
def wrapper(self, *args, **kwargs):
print 'Self is', self
return method(self, *args, **kwargs)
return wrapper
class SomeClass(object):
#MyDecorator
def f(self):
return 42
print SomeClass().f()