Suppose I have some python code, e.g. some class defined somewhere, which cannot be modified
class MyClass(object):
def __init__(self, arg1, arg2):
do_something...
def foo(self):
do_something
Now I want to add a trace capability, e.g. some mechanism from outside that traces each and every method call for the above class. I want to be able to print out when e.g, __init__ has been called, or foo or even the __del__ method of MyClass.
Is this possible to do, and how is this done best?
Create a proxy class that wraps the original class and then delegates the work after printing a trace:
class MyClassProxy(object):
def __init__(*args, **kwds):
print 'Initializing'
self.o = MyClass(*args, **kwds)
def foo(self):
print 'Fooing'
return self.o.foo()
You can create a trace decorator and attach it to all the methods of the class instance or class definition as shown in decorate_methods function.
import functools
import inspect
import types
class TestClass(object):
def func1(self):
pass
def func2(self, a, b):
pass
def trace(func):
#functools.wraps(func)
def decorator(*args, **kwargs):
print "TRACE:", func.__name__, args, kwargs
return func(*args, **kwargs)
return decorator
def decorate_methods(obj, decorator):
for name, func in inspect.getmembers(obj):
if isinstance(func, types.MethodType):
setattr(obj, name, decorator(func))
# Apply the decorator to a class instance
test1 = TestClass()
decorate_methods(test1, trace)
test1.func1()
test1.func2('bar1', b='bar2')
# Apply the decorator to the class definition
decorate_methods(TestClass, trace)
test2 = TestClass()
test2.func1()
test2.func2('bar1', b='bar2')
The output of the script will be:
TRACE: func1 () {}
TRACE: func2 ('bar1',) {'b': 'bar2'}
TRACE: func1 (<__main__.TestClass object at 0x7f5a8d888150>,) {}
TRACE: func2 (<__main__.TestClass object at 0x7f5a8d888150>, 'bar1') {'b': 'bar2'}
Use decorator as shown below:
def call_trace(orig_func):
def decorated_func(*args, **kwargs):
print "========>In function: " + orig_func.__name__ + "<========"
orig_func(*args, **kwargs)
return decorated_func
Apply this decorator to trace the function. It prints function name before entering the function.
Ex:
#call_trace
def foo(self):
do_something
Hope it helps.
[Update]: You can use metaclass, only thing you got to change is to add "metaclass" parameter to your class as shown below. As you can see, below code applies "call_trace" decorator to every function in the class "ExBase".
I tried this out yesterday, it worked fine. I am also new to python.:)
def call_trace(orig_func):
def inner_func(*args, **kwargs):
print ("function name:" + str(orig_func.__name__))
orig_func(*args, **kwargs)
return inner_func
class ExMeta(type):
def __new__(cls, name, bases, attrs):
for attr in attrs:
if hasattr(attrs[attr], '__call__'):
attrs[attr] = call_trace(attrs[attr])
return type.__new__(cls, name, bases, attrs)
class ExBase(metaclass=ExMeta):
x = "x"
y = "y"
def __init__(self):
self.__name = "name"
def getname(self):
return self.__name
b = ExBase()
b.getname()
Get the code for OnlinePythonTutor from github.com/pgbovine/OnlinePythonTutor/tree/master/v3.
You don't need to bother with all the JS stuff. Extract the files into some directory. You can run your scripts using python /path/to/my/OnlinePythonTutor-master/v3/generate_json_trace my_script.py
This will give you basically everything your program is doing in a step by step manner. It will probably be overkill so if you want look into the source code and the underlying source in bdb http://docs.python.org/2/library/bdb.html. The docs for bdb are horrible so I'm having trouble figuring out what exactly is going on but I think this is a pretty cool problem, good luck.
Related
This is a question similar to How to call a method implicitly after every method call? but for python
Say I have a crawler class with some attributes (e.g. self.db) with a crawl_1(self, *args, **kwargs) and another one save_to_db(self, *args, **kwargs) which saves the crawling results to a database (self.db).
I want somehow to have save_to_db run after every crawl_1, crawl_2, etc. call. I've tried making this as a "global" util decorator but I don't like the result since it involves passing around self as an argument.
If you want to implicitly run a method after all of your crawl_* methods, the simplest solution may be to set up a metaclass that will programatically wrap the methods for you. Start with this, a simple wrapper function:
import functools
def wrapit(func):
#functools.wraps(func)
def _(self, *args, **kwargs):
func(self, *args, **kwargs)
self.save_to_db()
return _
That's a basic decorator that wraps func, calling
self.save_to_db() after calling func. Now, we set up a metaclass
that will programatically apply this to specific methods:
class Wrapper (type):
def __new__(mcls, name, bases, nmspc):
for attrname, attrval in nmspc.items():
if callable(attrval) and attrname.startswith('crawl_'):
nmspc[attrname] = wrapit(attrval)
return super(Wrapper, mcls).__new__(mcls, name, bases, nmspc)
This will iterate over the methods in the wrapped class, looking for
method names that start with crawl_ and wrapping them with our
decorator function.
Finally, the wrapped class itself, which declares Wrapper as a
metaclass:
class Wrapped (object):
__metaclass__ = Wrapper
def crawl_1(self):
print 'this is crawl 1'
def crawl_2(self):
print 'this is crawl 2'
def this_is_not_wrapped(self):
print 'this is not wrapped'
def save_to_db(self):
print 'saving to database'
Given the above, we get the following behavior:
>>> W = Wrapped()
>>> W.crawl_1()
this is crawl 1
saving to database
>>> W.crawl_2()
this is crawl 2
saving to database
>>> W.this_is_not_wrapped()
this is not wrapped
>>>
You can see the our save_to_database method is being called after
each of crawl_1 and crawl_2 (but not after this_is_not_wrapped).
The above works in Python 2. In Python 3, replase this:
class Wrapped (object):
__metaclass__ = Wrapper
With:
class Wrapped (object, metaclass=Wrapper):
Something like this:
from functools import wraps
def my_decorator(f):
#wraps(f)
def wrapper(*args, **kwargs):
print 'Calling decorated function'
res = f(*args, **kwargs)
obj = args[0] if len(args) > 0 else None
if obj and hasattr(obj, "bar"):
obj.bar()
return wrapper
class MyClass(object):
#my_decorator
def foo(self, *args, **kwargs):
print "Calling foo"
def bar(self, *args, **kwargs):
print "Calling bar"
#my_decorator
def example():
print 'Called example function'
example()
obj = MyClass()
obj.foo()
It will give you the following output:
Calling decorated function
Called example function
Calling decorated function
Calling foo
Calling bar
A decorator in Python looks like this, it's a method taking a single method as argument and returning another wrapper method that shall be called instead of the decorated one. Usually the wrapper "wraps" the decorated method, i.e. calls it before/after performing some other actions.
Example:
# define a decorator method:
def save_db_decorator(fn):
# The wrapper method which will get called instead of the decorated method:
def wrapper(self, *args, **kwargs):
fn(self, *args, **kwargs) # call the decorated method
MyTest.save_to_db(self, *args, **kwargs) # call the additional method
return wrapper # return the wrapper method
Now learn how to use it:
class MyTest:
# The additional method called by the decorator:
def save_to_db(self, *args, **kwargs):
print("Saver")
# The decorated methods:
#save_db_decorator
def crawl_1(self, *args, **kwargs):
print("Crawler 1")
#save_db_decorator
def crawl_2(self, *args, **kwargs):
print("Crawler 2")
# Calling the decorated methods:
my_test = MyTest()
print("Starting Crawler 1")
my_test.crawl_1()
print("Starting Crawler 1")
my_test.crawl_2()
This would output the following:
Starting Crawler 1
Crawler 1
Saver
Starting Crawler 1
Crawler 2
Saver
See this code running on ideone.com
I'm attempting to create some decorators which will allow class members to be decorated on instantiation, because I'd like to have some instances which are decorated and others which aren't.
In the example below, the intended outcome of applying the once decorator to an instance of SomeClass is that when some_func has been called, calling other_func prints a message rather than calling the original function.
#!/usr/bin/env python
import functools
def some_once(func):
#functools.wraps(func)
def wrapped(self, *args, **kwargs):
if not self._new_attr:
print("don't have new attr yet")
func(self, *args, **kwargs)
self._new_attr = True
return wrapped
def other_once(func):
#functools.wraps(func)
def wrapped(self, *args, **kwargs):
if self._new_attr:
print("We have a new attr")
else:
func(self, *args, **kwargs)
return wrapped
def once(cls):
setattr(cls, "_new_attr", False)
setattr(cls, "some_func", some_once(cls.some_func))
setattr(cls, "other_func", other_once(cls.other_func))
return cls
class SomeClass:
def some_func(self, parameter):
return "The parameter is " + str(parameter)
def other_func(self, parameter):
return "The other parameter is " + str(parameter)
if __name__ == '__main__':
a = SomeClass()
print(dir(a))
print(a.some_func("p1"))
print(a.other_func("p2"))
b = once(SomeClass())
print(dir(b))
print(b.some_func("p3"))
print(b.other_func("p4"))
The problem that results is that rather than looking at self._new_attr, the decorated functions instead look at string._new_attr, where the string is the parameter to the functions. I'm confused about what I'm doing wrong here.
Don't decorate an instance. Make a new, decorated class.
Changing this line:
b = once(SomeClass())
into:
b = once(SomeClass)()
Should do the trick. once(SomeClass) gives you a new, decorated class. In the next step make an instance of it.
I have a decorator method to check the argument types passed to a function.
def accepts(*types):
def check_accepts(f):
assert len(types) == f.func_code.co_argcount
def new_f(*args, **kwds):
for (a, t) in zip(args, types):
assert isinstance(a, t), \
"arg %r does not match %s" % (a,t)
return f(*args, **kwds)
new_f.func_name = f.func_name
return new_f
return check_accepts
Now, in a class (in classA.py), I want a method to only accept arguments of the same class:
class ClassA:
#accepts(WHATTOPUTHERE)
def doSomething(otherObject):
# Do something
In other classes I can just put ClassA in place of WHATTOPUTHERE, but inside classA.py, ClassA is not known. How can I pass the current class to the #accepts() function?
Use the function based version of the decorator and apply it after the class definition:
class ClassA:
def doSomething(otherObject):
# Do something
ClassA.doSomething = accepts(ClassA)(ClassA.doSomething)
Another way would be write a Metaclass that would automatically apply this after class creation:
class Meta(type):
def __new__(cls, clsname, bases, dct):
fields = ('doSomething', ) #Fields on which you want to apply the decorator
for name, val in dct.items():
if name in fields:
dct[name] = accepts(cls)(val)
return type.__new__(cls, clsname, bases, dct)
class ClassA(object):
__metaclass__ = Meta
def doSomething(otherObject):
pass
Instead of manually doing things like new_f.func_name = f.func_name, use functools.wraps. This would also preserve things like docstring, argument list etc.
from functools import wraps
def accepts(*types):
def check_accepts(f):
print "inside"
assert len(types) == f.func_code.co_argcount
#wraps(f)
def new_f(*args, **kwds):
wouldn't adding a self variable to the function doSomething and then referencing args[0] inside check_accepts (provided you add (*args) or extra argument to your definition) solve your problem? If the function doSomething is supposed to be a class method, still you can outsource this self. How?
add a self to some dummy method inside the class
make a decorator which populates a variable which accepts can somehow reach (like metadata)
make sure to call this additional method before the doSomething()
you've got the class instance! Enjoy!
NOTE: This is not the only way to store metadata like this and use it later, you can do it as you wish.
I have come across this singleton implementation here: http://blog.amir.rachum.com/post/21850841339/implementing-the-singleton-pattern-in-python in the first reply.
def singleton(cls):
return cls()
#singleton
class Foo(object):
def bar(self):
pass
if __name__ == '__main__':
print id(Foo)
print id(Foo)
But I don't understand the inner workings, the decorator returns a class instance, but why the same instance every time ?
You can rewrite that code to
class Foo(object):
pass
Foo = singleton(Foo)
# which is
Foo = Foo()
So here the name of the class is replaced by an instantiation of it. A bit cheesy in my opinion, especially since you can still create new objects of the same class by using Foo.__class__ and you are messing with the naming schema.
The singleton does that by holding internal state. This state here would probably be an instance of the class. The decorator can be something arbitrary.
Have a look at this:
http://hairysun.com/downloads/DecoratorHandout.pdf
class Decorator(object):
# in __init__ set up state
def __call__(self, function):
#functools.wraps(function)
def wrapper(*args, **kw): # 1.
print "before func"
result = function(*args, **kw) # 2.
print "after func"
return result
return wrapper # 3.
>>> decorator2 = Decorator()
>>> #decorator2
... def nothing(): pass
The decorator is essentially a function that
Defines a function
That calls the function that you passed in
Returns the newly 'wrapped' function to be called later
The surrounding class (here: the decorator) could do something like this:
class Singleton(object):
def __init__(self):
self.instance = None
def __call__(self, function):
#functools.wraps(function)
def wrapper(*args, **kw):
if self.instance is None:
self.instance = function(*args, **kw)
return self.instance
return wrapper
I did not run the code, but I assume this is in general how it works. If there is no instance available create one. If one is available, don't create a new one - return the single old one instead. One might probably want to check for other properties of the callable before using this in production.
Can you create a decorator inside a class that will see the classes methods and variables?
The decorator here doesnt see: self.longcondition()
class Foo:
def __init__(self, name):
self.name = name
# decorator that will see the self.longcondition ???
class canRun(object):
def __init__(self, f):
self.f = f
def __call__(self, *args):
if self.longcondition(): # <-------- ???
self.f(*args)
# this is supposed to be a very long condition :)
def longcondition(self):
return isinstance(self.name, str)
#canRun # <------
def run(self, times):
for i in xrange(times):
print "%s. run... %s" % (i, self.name)
There's no real need to implement this decorator as a class, and there's no need to implement it inside the definition of the Foo class. The following will suffice:
def canRun(meth):
def decorated_meth(self, *args, **kwargs):
if self.longcondition():
print 'Can run'
return meth(self, *args, **kwargs)
else:
print 'Cannot run'
return None
return decorated_meth
Using that decorator seems to work:
>>> Foo('hello').run(5)
Can run
0. run... hello
1. run... hello
2. run... hello
3. run... hello
4. run... hello
>>> Foo(123).run(5)
Cannot run
You can have it be a class but you need to use the descriptor protocol
import types
class canRun(object):
def __init__(self, f):
self.f = f
self.o = object # <-- What the hell is this about?
def __call__(self, *args):
if self.longcondition():
self.f(*args)
def __get__(self, instance, owner):
return types.MethodType(self, instance)
You always need to use a descriptor when you want to decorate class methods with a class instance using the __call__ method. The reason for this is that there will only be one self passed in which refers to the instance of the decorating class and not the instance of the decorated method.
My previous answer was made in haste. If you're wanting to write a decorator you should really use wraps from the functools module. It takes care of the hard stuff for you.
A proper way to define the canRun decorator is:
from functools import wraps
def canRun(f):
#wraps(f)
def wrapper(instance, *args):
if instance.longcondition():
return f(instance, *args)
return wrapper
The canRun function should be defined outside of the class.