I would like to proxy "all" methods of a class using following code:
import paramiko
class SFTPProxy():
def __init__(self, sftp):
self.sftp = TRANSPORT.open_sftp_client()
for x, y in type(self.sftp).__dict__.items():
if re.search(r'^__', x):
continue
def fn(self, *args, **kwargs):
return y(self.sftp, *args, **kwargs)
setattr(SFTPProxy, x, fn)
When I call the method like this:
fooproxy.sftp.listdir()
It works.
When I call the method like this:
fooproxy.listdir() # this is the method belongs to the proxied class
The program just hangs, is there any shallow problems in the code?
One issue I can see with your approach is that not all values in type(self.sftp).__dict__ are functions. Hence, y(...) will fail. Isn't it simpler and cleaner to override __getattr__:
class SFTPProxy(object):
def __init__(self, sftp):
self.sftp = TRANSPORT.open_sftp_client()
def __getattr__(self, item):
if hasattr(self.sftp, item):
return getattr(self.sftp, item)
raise AttributeError(item)
This will handle all kinds of attributes rather swiftly: instance/class fields, instance/class/static methods.
Related
I'd like to implement something like this
def after(f):
def inner(*args, **kwargs):
super().f(*args, **kwargs)
f(*args, **kwargs)
return inner
class A:
def f(self):
print ('hello')
class B(A):
#after
def f(self):
print ('world')
b = B()
b.f()
that is I would like to get rid of explicit super in some of my classes and replace it with #before / #after decorator (maybe with parameters).
That is, in this example, I would like hello world to be printed.
the idea is to increase the readability of the code, as in some classes I often use multiple inheritance, so I often override methods and often have to use super().
I think I could use inspect to determine the class instance that calls the decorator (although not sure about performance if I have many class instances).
is there a way to do this without sacrificing performance?
You can make your decorator work, you just need it to make it a descriptor class, rather than a function. You need to implement the __set_name__ method to get a reference to the class you've been added to. With the class reference, you can make a two-argument super call:
import functools
class after:
def __init__(self, method):
self.method = method
def __set_name__(self, owner, name):
self.owner = owner
self.name = name # using self.method.__name__ might be better?
def __get__(self, instance, owner):
if instance is None:
return self
return functools.partial(self, instance)
def __call__(self, instance, *args, **kwargs):
assert(self.owner is not None and self.name is not None)
getattr(super(self.owner, instance), self.name)(*args, **kwargs)
return self.method(instance, *args, **kwargs)
You could do a before too, which would be nearly the same, just with the last two lines in the reverse order (and some fiddling to handle the return value).
I'd note that this decorator is quite a bit less generally useful than calling super the normal way since you can't usefully interact with the value returned by the overridden method, or change the arguments being passed in to it. There's no before or after decorated method that can replicate these classes:
class Foo:
def foo(self, x, y):
return x + y
class Bar(Foo):
def foo(self, x, y, z):
return super().foo(x//2, y+1) * z
I am attempting to wrap a class from a third-party package in such a way that my new class looks exactly like a subclass of the third-party class. The third-party class does not support inheritance, and it has nontrivial features, such as functions that have a __getitem__ method. I can wrap almost every attribute and method using a solution based on Wrapping a class whose methods return instances of that class and How can I intercept calls to python's "magic" methods in new style classes?. However, I still need to override the __init__ method of the third-party class. How can I do that? Note: I am using new-style classes.
Code so far:
import copy
class WrapperMetaclass(type):
"""
Works with the `Wrapper` class to create proxies for the wrapped object's magic methods.
"""
def __init__(cls, name, bases, dct):
def make_proxy(name):
def proxy(self, *args):
return getattr(self._obj, name)
return proxy
type.__init__(cls, name, bases, dct)
if cls.__wraps__:
ignore = set("__%s__" % n for n in cls.__ignore__.split())
for name in dir(cls.__wraps__):
if name.startswith("__"):
if name not in ignore and name not in dct:
setattr(cls, name, property(make_proxy(name)))
class Wrapper(object):
"""
Used to provide a (nearly) seamless inheritance-like interface for classes that do not support direct inheritance.
"""
__metaclass__ = WrapperMetaclass
__wraps__ = None
# note that the __init__ method will be ignored by WrapperMetaclass
__ignore__ = "class mro new init setattr getattr getattribute dict"
def __init__(self, obj):
if self.__wraps__ is None:
raise TypeError("base class Wrapper may not be instantiated")
elif isinstance(obj, self.__wraps__):
self._obj = obj
else:
raise ValueError("wrapped object must be of %s" % self.__wraps__)
def __getattr__(self, name):
if name is '_obj':
zot = 1
orig_attr = self._obj.__getattribute__(name)
if callable(orig_attr) and not hasattr(orig_attr, '__getitem__'):
def hooked(*args, **kwargs):
result = orig_attr(*args, **kwargs)
if result is self._obj:
return self
elif isinstance(result, self.__wraps__):
return self.__class__(result)
else:
return result
return hooked
else:
return orig_attr
def __setattr__(self, attr, val):
object.__setattr__(self, attr, val)
if getattr(self._obj, attr, self._obj) is not self._obj: # update _obj's member if it exists
setattr(self._obj, attr, getattr(self, attr))
class ClassToWrap(object):
def __init__(self, data):
self.data = data
def theirfun(self):
new_obj = copy.deepcopy(self)
new_obj.data += 1
return new_obj
def __str__(self):
return str(self.data)
class Wrapped(Wrapper):
__wraps__ = ClassToWrap
def myfun(self):
new_obj = copy.deepcopy(self)
new_obj.data += 1
return new_obj
# can't instantiate Wrapped directly! This is the problem!
obj = ClassToWrap(0)
wr0 = Wrapped(obj)
print wr0
>> 0
print wr0.theirfun()
>> 1
This works, but for truly seamless inheritance-like behavior, I need to instantiate Wrapped directly, e.g.
wr0 = Wrapped(0)
which currently throws
ValueError: wrapped object must be of <class '__main__.ClassToWrap'>
I attempted to override by defining a new proxy for __init__ in WrapperMetaclass, but rapidly ran into infinite recursions.
My codebase is complex with users at different skill levels, so I can't afford to use monkey-patching or solutions that modify the definition of the example classes ClassToWrap or Wrapped. I am really hoping for an extension to the code above that overrides Wrapped.__init__.
Please note that this question is not simply a duplicate of e.g. Can I exactly mimic inheritance behavior with delegation by composition in Python?. That post does not have any answer that is nearly as detailed as what I'm already providing here.
It sounds like you just want Wrapper.__init__ method to work differently that it currently does. Rather than taking an already existing instance of the __wraps__ class, it should take the arguments that the other class expects in its constructor and built the instance for you. Try something like this:
def __init__(self, *args, **kwargs):
if self.__wraps__ is None:
raise TypeError("base class Wrapper may not be instantiated")
else:
self._obj = self.__wraps__(*args, **kwargs)
If you want Wrapper to remain the same for some reason, you could put the logic in a new Wrapped.__init__ method instead:
def __init__(self, data): # I'm explicitly naming the argument here, but you could use *args
super(self, Wrapped).__init__(self.__wraps__(data)) # and **kwargs to make it extensible
I have problem solving this question, I have the following class:
class test:
#auth
def method1(self, x):
return x
#auth
def method2(self, x, y):
return x+y
def method3(self, z):
return z
I applied the decorator in both methods, follow:
class auth:
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
self.f(*args, **kwargs)
So far no problem, however I need (NEED) to use the following code:
def run():
klass = globals()["test"]()
method1 = getattr(klass, "method1")
print(method1.__code__.co_varnames)
# should print (self, x)
method2 = getattr(klass, "method2")
print(method2.__code__.co_varnames)
# should print (self, x, y)
method3 = getattr(klass, "method3")
print(method3.__code__.co_varnames)
# i get (self, z) < without decorator
But I get now:
AttributeError: 'auth' object has no attribute '__code__'
What makes sense if we think that the signature of method "method1 and method2" is now "auth".
So how do I get the arguments with or without decorators.
I started reading about the "inspect" but there are many reports about being slow.
The "original" method is stored in the f attribute of the auth object. Instead of method1.__code__.co_varnames use method1.f.__code__.co_varnames
Annotations just contain an object and are not the object itsself, it is an object of class auth and not function. To access the function itsself, you can write methodN.f.__code__.co_varnames or assign a copy of the __dict__ object of the function to the auth-object in __init__ itsself.
class auth:
def __init__(self, f):
self.__dict__.update(f.__dict__)
# now the initialisations
self.f = f
def __call__(self, *args, **kwargs):
self.f(*args, **kwargs)
Edit:
You should initialize the members/call super after updating the dict, because f could be overriden by the update, eg. you define another decorator-class and it has also a member f.
I have the following classes.
Validator is a decorator that receives a class which defines validation criteria for a decorated function. ValidateKeys is the validation criteria for this example. Node2D is a class using validation criteria.
class Validator(object):
def __init__(self, TheValidator, *args, **kwargs):
self.validator = TheValidator(*args,**kwargs)
def __call__(self,f):
def wrapped_f(instance, *args,**kwargs):
self.TheValidator(instance, *args, **kwargs)
return f(instance,*args,**kwargs)
return wrapped_f
class ValidateKeys(object):
def __init__(self,*keysIterable):
self.validkeys = keysIterable
def __call__(self, instance, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance.__dict__.update(kwargs)
class Node2D(object):
#property
def coords(self):
return self.__dict__
#coords.setter
def coords(self,Coords):
self.set_coords(**Coords)
#Validator(ValidateKeys, 'x','y')
def set_coords(self,**Coords):
pass
From what I understand, as things are written here, every instance of Node2D will produce a duplicate Validator (as will any other class decorated with Validator) and ValidateKeys.
EDIT: THIS IS WRONG! See answer below.
Note that this is primarily a learning exercise for me and although I would be interested in hearing criticisms/suggestions for improving my over all approach, my primary goal is to learn more about how to use decorators effectively.
Also note that I normally would not use capitalization for a decorator class but am using it here since it makes it easier to read on SO.
My assumption was incorrect.
As things are written, only one instance of Validator and ValidateKeys is created per class. I did not realize that the line #Validator(ValidateKeys, 'x','y') only runs once (at the time of class definition) and not at instance creation.
I should have realized this, since decorator expressions appear at the same level of hierarchy as class attributes, e.g.:
class MyClass():
class_attribute = None #only one class_attribute is created
#decorator #only one decorator (i.e., decorated method) is created
def method():
pass
Test:
class Validator(object):
def __init__(self, TheValidator, *args, **kwargs):
print("New Validator Object")
self.TheValidator = TheValidator(*args,**kwargs)
def __call__(self,f):
def wrapped_f(instance, *args,**kwargs):
self.TheValidator(instance, *args, **kwargs)
return f(instance,*args,**kwargs)
return wrapped_f
class ValidateKeys(object):
def __init__(self,*keysIterable):
print("New ValidateKeys Object")
self.validkeys = keysIterable
def __call__(self, instance, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance.__dict__.update(kwargs)
class Node2D(object):
#property
def coords(self):
return self.__dict__
#coords.setter
def coords(self,Coords):
self.set_coords(**Coords)
#Validator(ValidateKeys, 'x','y')
def set_coords(self,**Coords):
pass
n1 = Node2D()
n2 = Node2D()
n1.setcoords(x=1,y=2)
n1.coords
Output:
'New Validator Object' #<-- Seen only once when module is loaded (class defined)
'New ValidateKeys Object' #<-- Seen only once when module is loaded (class defined)
'{'x': 1, 'y': 2}'
I do not have the problem I thought I had. Thanks to all for the help.
Is it possible to write an exception handler to catch the run-time errors generated by ALL the methods in class? I can do it by surrounding each one with try/except:
class MyError(Exception):
def __init__(self, obj, method):
print 'Debug info:', repr(obj.data), method.__name__
raise
class MyClass:
def __init__(self, data):
self.data = data
def f1(self):
try:
all method code here, maybe failing at run time
except:
raise MyError(self, self.f1)
I wonder if is there a more general way to achieve the same - for any error raising anywhere in the class. I would like to be able to access the class data to print some debug info.
Also, how do I get the failing method name (f1 in the example)?
Update: thanks to all for the insighful answers, the decorator idea looks like the way to go.
About the risk of trapping ALL the exceptions: a raise statement in the except branch should re-raise the exception without losing any information, doesn't it? That's why I put it in MyError also...
Warning: if you want something like this, it's likely you don't... but if you really want to...
Something like:
import functools
def catch_exception(f):
#functools.wraps(f)
def func(*args, **kwargs):
try:
return f(*args, **kwargs)
except Exception as e:
print 'Caught an exception in', f.__name__
return func
class Test(object):
def __init__(self, val):
self.val = val
#catch_exception
def calc():
return self.val / 0
t = Test(3)
t.calc()
shows how to decorate individual functions. You can then create a class decorator to apply this decorator to each method (be careful of classmethod's/staticmethod's/properties etc...)
Assuming you've got a decorator catch_exception as in #Jon Clement's answer...
class ErrorCatcher(type):
def __new__(cls, name, bases, dct):
for m in dct:
if hasattr(dct[m], '__call__'):
dct[m] = catch_exception(dct[m])
return type.__new__(cls, name, bases, dct)
class Test(object):
__metaclass__ = ErrorCatcher
def __init__(self, val):
self.val = val
def calc(self):
return self.val / 0
The metaclass applies catch_exception to everything that appears to be a method while it is defining Test.
In response to a comment regarding custom messages for each method, one could attach such a message (or even a callback function to generate a message) as an attribute:
class Test(object):
__metaclass__ = ErrorCatcher
def __init__(self, val):
self.val = val
def calc(self):
return self.val / 0
calc.msg = "Dividing by 0 is ill-advised"
The catch_exception decorator would look for a msg attribute on its argument and use it, if found, in handling the exception.
This approach could be extended; instead of a string, msg could be a mapping of exception types to strings. In either case, the string could be replaced (with support from catch_exception, of course) with an arbitrary callback function that takes, say, the raised exception as an argument.
def calc_handler(exc):
# ...
calc.callback = calc_handler
A decorator would be a good solution here.
Here's an example of how you could do it:
import inspect
def catch_exception_decorator(function):
def decorated_function:
try:
function()
except:
raise MyError(self.__class__, inspect.stack()[1][3])
return decorated_function
class MyClass(object):
def __init__(self):
...
#catch_exception_decorator
def f1(self):
...
#catch_exception_decorator on top of the function is a shortcut for f1 = catch_exception_decorator(f1).
Instead of doing self.class, you could also access class data from the instance, as long as you're not shadowing variables. inspect.stack()[1][3] is the function name of the current function. You can use these to create the exception attributes.