Related
Is there a way to check if an object has an attribute, that doesn't rely on __getattr__ or object implementation specifics?
Consider the code below. I want Proxy to delegate to Wrapped what it can't handle. The code works, but I'd like to avoid the test attr in self.__dict__ (I'd prefer a stable interface to do this, without using implementation quirks). The function hasattr doesn't help here, because it gets routed to the wrapped object via __getattr__.
class Wrapped:
def __init__(self):
self.a = 0
self.b = 1
class Proxy:
def __init__(self, wrapped):
object.__setattr__(self, 'wrapped', wrapped)
object.__setattr__(self, 'a', 2)
def __getattr__(self, attr):
return getattr(self.wrapped, attr)
def __setattr__(self, attr, value):
if attr in self.__dict__: # <-- Don't like this, hasattr doesn't work
object.__setattr__(self, attr, value)
elif hasattr(self.wrapped, attr):
setattr(self.wrapped, attr, value)
else: object.__setattr__(self, attr, value)
Testdrive:
>>> w = Wrapped()
>>> p = Proxy(w)
>>> p.a
2
>>> p.b
1
>>> p.a = 3
>>> p.a
3
>>> w.a
0
>>> p.b = 4
>>> p.b
4
>>> w.b
4
>>> p.c = 5
>>> p.c
5
>>> w.c
AttributeError: 'Wrapped' object has no attribute 'c'
The built-in hasattr() function
is implemented by calling getattr(object, name) and seeing whether it raises an AttributeError or not.
The inspect module has the getattr_static() method, which can be used to
Retrieve attributes without triggering dynamic lookup via the descriptor protocol, __getattr__() or __getattribute__()".
Using this method, you could define a hasattr_static() method similarly to hasattr(), but calling inspect.getattr_static(object, name) instead of getattr(object, name):
import inspect
def hasattr_static(object, name):
try:
inspect.getattr_static(object, name)
return True
except AttributeError:
return False
And then use this in the __setattr__() method of your Proxy class as a check:
def __setattr__(self, attr, value):
if hasattr_static(self, attr):
object.__setattr__(self, attr, value)
elif hasattr(self.wrapped, attr):
setattr(self.wrapped, attr, value)
else:
object.__setattr__(self, attr, value)
Alternatively, you coud use the dir() function instead of __dict__ to retrieve a list of attributes of your object, or use inspect.getmembers().
I would like to define metaclass that will enable me to create properties (i.e. setter, getter) in new class on the base of the class's attributes.
For example, I would like to define the class:
class Person(metaclass=MetaReadOnly):
name = "Ketty"
age = 22
def __str__(self):
return ("Name: " + str(self.name) + "; age: "
+ str(self.age))
But I would like to get something like this:
class Person():
__name = "Ketty"
__age = 22
#property
def name(self):
return self.__name;
#name.setter
def name(self, value):
raise RuntimeError("Read only")
#property
def age(self):
return self.__age
#age.setter
def age(self, value):
raise RuntimeError("Read only")
def __str__(self):
return ("Name: " + str(self.name) + "; age: "
+ str(self.age))
Here is the metaclass I have written:
class MetaReadOnly(type):
def __new__(cls, clsname, bases, dct):
result_dct = {}
for key, value in dct.items():
if not key.startswith("__"):
result_dct["__" + key] = value
fget = lambda self: getattr(self, "__%s" % key)
fset = lambda self, value: setattr(self, "__"
+ key, value)
result_dct[key] = property(fget, fset)
else:
result_dct[key] = value
inst = super(MetaReadOnly, cls).__new__(cls, clsname,
bases, result_dct)
return inst
def raiseerror(self, attribute):
raise RuntimeError("%s is read only." % attribute)
However it dosen't work properly.
client = Person()
print(client)
Sometimes I get:
Name: Ketty; age: Ketty
sometimes:
Name: 22; age: 22
or even an error:
Traceback (most recent call last):
File "F:\Projects\TestP\src\main.py", line 38, in <module>
print(client)
File "F:\Projects\TestP\src\main.py", line 34, in __str__
return ("Name: " + str(self.name) + "; age: " + str(self.age))
File "F:\Projects\TestP\src\main.py", line 13, in <lambda>
fget = lambda self: getattr(self, "__%s" % key)
AttributeError: 'Person' object has no attribute '____qualname__'
I have found example how it can be done other way Python classes: Dynamic properties, but I would like to do it with metaclass. Do you have any idea how this can be done or is it possible at all ?
Bind the current value of key to the parameter in your fget and fset definitions:
fget = lambda self, k=key: getattr(self, "__%s" % k)
fset = lambda self, value, k=key: setattr(self, "__" + k, value)
This is a classic Python pitfall. When you define
fget = lambda self: getattr(self, "__%s" % key)
the value of key is determined when fget is called, not when fget is defined. Since it is a nonlocal variable, its value is found in the enclosing scope of the __new__ function. By the time fget gets called, the for-loop has ended, so the last value of key is the value found. Python3's dict.item method returns items in a unpredictable order, so sometimes the last key is, say, __qualname__, which is why a suprising error is sometimes raised, and sometimes the same wrong value is returned for all attributes with no error.
When you define a function with a parameter with a default value, the default value is bound to the parameter at the time the function is defined.
Thus, the current values of key get corrected bound to fget and fset when you bind default values to k.
Unlike before, k is now a local variable. The default value is stored in fget.__defaults__ and fset.__defaults__.
Another option is to use a closure. You can define this outside of the metaclass:
def make_fget(key):
def fget(self):
return getattr(self, "__%s" % key)
return fget
def make_fset(key):
def fset(self, value):
setattr(self, "__" + key, value)
return fset
and use it inside the metaclass like this:
result_dct[key] = property(make_fget(key), make_fset(key))
Now when fget or fset is called, the proper value of key is found in the enclosing scope of make_fget or make_fset.
#unutbu's answer is perfectly fine and addressed the main caveats very well.
However, there is one more thing to note (I cannot comment yet so here I am posing an entire answer), note that classproperty's and instanceproperty's have different keys.
What you are doing here is entirely based on class properties. If one wishes to make a setter for an "instance" property the key must be f"_{clsname}__{key}". I lack the technical understanding of Python data model to explain why or if what I'm saying is entirely correct or has other caveats. Just posting this here as a side note.
This is especially important if you wish to allow the user to override the methods with custom ones like the example I provide here (I'm not sure even if it has no syntax error but still I post it here):
class MyMeta(type):
def __new__(cls, clsname, bases, info, **kwargs):
property_key = "__my_private" # should have been f"_{clsname}__my_private"
info[property_key] = "default_value"
if "my_getter" not in info:
info["my_getter"] = lambda self, key=property_key: getattr(self, key)
return super().__new__(cls, clsname, bases, info, **kwargs)
class MyClass(metaclass=MyMeta):
def my_getter(self):
return f"will fail to get {self.__my_private}"
MyClass().my_getter() # => Raises: AttributeError "_MyClass__my_private" not found.
# This would work so fine if no `self` keyword was being used (treated as classproperty)
I have the following code:
#-*-coding:utf-8-*-
class A(object):
def __init__(self, x):
self.x = x
def __getattr__(self, name): # `__getattr__` will be called undefined attribute
print "get: ", name
return self.__dict__.get(name)
def __setattr__(self, name, value):
print "set:", name, value
self.__dict__[name] = value
def __getattribute__(self, name): # `__getattribute__` will be called all attributes
print "attribute:", name
return object.__getattribute__(self, name)
if __name__ == "__main__":
a = A(10)
print '---------------'
a.x
print '---------------'
a.y = 20
print '---------------'
a.z
And the result is :
set: x 10
attribute: __dict__
---------------
attribute: x
---------------
set: y 20
attribute: __dict__
---------------
attribute: z
get: z
attribute: __dict__
When I called a=A(10), why __getattribute__ is called ? This is my thought: there is self.x = x in __init__ , and __setattr__ catch __init__, self.__dict__[name] = value catch __getattrbute__. So, __getattribute__ is called. Does my thought right ? What's wrong ?
The arrow is pointing to where __setattr__ invokes __getattribute__:
def __setattr__(self, name, value):
print "set:", name, value
self.__dict__[name] = value
# ^ attribute access!
__getattribute__ handles all explicit attribute lookup, including __dict__. I believe this is the conclusion you already came to; I couldn't quite understand what you were trying to say.
I'm writing a python class that uses __setattr__ and __getattr__ to provide custom attribute access.
However, some attributes can't be handled in a generic way, so I was hoping to use descriptors for those.
A problem arises in that for a descriptor, the descriptor's __get__ will be invoked in favour of the instances __getattr__, but when assigning to an attribute, __setattr__ will be invoked in favour of the descriptors __set__.
An example:
class MyDesc(object):
def __init__(self):
self.val = None
def __get__(self, instance, owner):
print "MyDesc.__get__"
return self.val
def __set__(self, instance, value):
print "MyDesc.__set__"
self.val = value
class MyObj(object):
foo = MyDesc()
def __init__(self, bar):
object.__setattr__(self, 'names', dict(
bar=bar,
))
object.__setattr__(self, 'new_names', dict())
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
self.new_names[name] = value
def __getattr__(self, name):
print "MyObj.__getattr__ for %s" % name
if name in self.new_names:
return self.new_names[name]
if name in self.names:
return self.names[name]
raise AttributeError(name)
if __name__ == "__main__":
o = MyObj('bar-init')
o.bar = 'baz'
print o.bar
o.foo = 'quux'
print o.foo
prints:
MyObj.__setattr__ for bar
MyObj.__getattr__ for bar
baz
MyObj.__setattr__ for foo
MyDesc.__get__
None
The descriptor's __set__ is never called.
Since the __setattr__ definition isn't just overriding behaviour for a limited set of names, there's no clear place that it can defer to object.__setattr__
Is there a recommended way to have assigning to attributes use the descriptor, if available, and __setattr__ otherwise?
I think I'd approach this by having a mechanism to automatically mark which are the
descriptors in each class, and wrap the __setattr__ in a way that it'd call
object's normal behavior for those names.
This can be easily achieved with a metaclass (and a decorator for __setattr__
def setattr_deco(setattr_func):
def setattr_wrapper(self, attr, value):
if attr in self._descriptors:
return object.__setattr__(self, attr, value)
return setattr_func(self, attr, value)
return setattr_wrapper
class MiscSetattr(type):
def __new__(metacls, name, bases, dct):
descriptors = set()
for key, obj in dct.items():
if key == "__setattr__":
dct[key] = setattr_deco(obj)
elif hasattr(obj, "__get__"):
descriptors.add(key)
dct["_descriptors"] = descriptors
return type.__new__(metacls, name, bases, dct)
# and use MiscSetattr as metaclass for your classes
One of possible ways:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
for cls in self.__class__.__mro__ + (self, ):
if name in cls.__dict__:
return object.__setattr__(self, name, value)
print 'New name', name, value
self.new_names[name] = value
It checks if name already defined in class, base classes or instance and then it calls object.__setattr__ which will execute descriptor __set__.
Another way:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
try:
object.__getattribute__(self, name)
except AttributeError:
print 'New name', name, value
self.new_names[name] = value
else:
object.__setattr__(self, name, value)
But it will call descriptor's __get__.
P.S.
I'm not sure about need to check all __mro__ members since MyObj will contain inherited class members in __dict__.
Maybe for cls in (self.__class__, self):... will be enough.
I'm solving the python koans.
I haven't got any real problem until the 34th.
this is the problem:
Project: Create a Proxy Class
In this assignment, create a proxy class (one is started for you
below). You should be able to initialize the proxy object with any
object. Any attributes called on the proxy object should be forwarded
to the target object. As each attribute call is sent, the proxy
should record the name of the attribute sent.
The proxy class is started for you. You will need to add a method
missing handler and any other supporting methods. The specification
of the Proxy class is given in the AboutProxyObjectProject koan.
Note: This is a bit trickier that it's Ruby Koans counterpart, but you
can do it!
and this is my solution until now:
class Proxy(object):
def __init__(self, target_object):
self._count = {}
#initialize '_obj' attribute last. Trust me on this!
self._obj = target_object
def __setattr__(self, name, value):pass
def __getattr__(self, attr):
if attr in self._count:
self._count[attr]+=1
else:
self._count[attr]=1
return getattr(self._obj, attr)
def messages(self):
return self._count.keys()
def was_called(self, attr):
if attr in self._count:
return True
else: False
def number_of_times_called(self, attr):
if attr in self._count:
return self._count[attr]
else: return False
It works until this test:
def test_proxy_records_messages_sent_to_tv(self):
tv = Proxy(Television())
tv.power()
tv.channel = 10
self.assertEqual(['power', 'channel='], tv.messages())
where tv.messages() is ['power'] because tv.channel=10 is taken by the proxy object and not the television object.
I've tried to manipulate the __setattr__ method, but I always end in a unlimited loop.
edit 1:
I'm trying this:
def __setattr__(self, name, value):
if hasattr(self, name):
object.__setattr__(self,name,value)
else:
object.__setattr__(self._obj, name, value)
But then I get this error in a loop on the last entry:
RuntimeError: maximum recursion depth exceeded while calling a Python object
File "/home/kurojishi/programmi/python_koans/python 2/koans/about_proxy_object_project.py", line 60, in test_proxy_method_returns_wrapped_object
tv = Proxy(Television())
File "/home/kurojishi/programmi/python_koans/python 2/koans/about_proxy_object_project.py", line 25, in __init__
self._count = {}
File "/home/kurojishi/programmi/python_koans/python 2/koans/about_proxy_object_project.py", line 33, in __setattr__
object.__setattr__(self._obj, name, value)
File "/home/kurojishi/programmi/python_koans/python 2/koans/about_proxy_object_project.py", line 36, in __getattr__
if attr in self._count:
The loop is in __getattr__.
You are using hasattr in __setattr__ to decide whether you should write to the local or proxied object. This works well for all but one case.
In your __init__ you have the following line:
self._count = {}
This calls __setattr__ with '_count' which does not exist at that point and therefore (hence hasattr returns False) is forwarded to the proxied object.
If you want to use your approach you have to write your __init__ like this:
def __init__(self, target_object):
object.__setattr__(self, '_count', {})
#initialize '_obj' attribute last. Trust me on this!
object.__setattr__(self, '_obj', target_object)
As I understand maybe your problem is related with the recursive call when you set and attribute value. From docs:
If __setattr__() wants to assign to an instance attribute, it should not simply execute "self.name = value" -- this would cause a recursive call to itself. Instead, it should insert the value in the dictionary of instance attributes, e.g., "self.__dict__[name] = value". For new-style classes, rather than accessing the instance dictionary, it should call the base class method with the same name, for example, "object.__setattr__(self, name, value)".
setattr is called on all assignments. It's more like getattribute than getattr. This also affects code in the __init__ method.
This means that the first branch of this code will almost always fail, only attributes inherited from object will pass the test:
def __setattr__(self, name, value):
if hasattr(self, name):
object.__setattr__(self,name,value)
else:
object.__setattr__(self._obj, name, value)
Instead we
can assume that assignments are meant for the Proxy unless it has an _obj attribute. Hence the comment in __init__. We set up our proxy's attributes, then add the target object and all future assignments get sent to it.
def __setattr__(self, name, value):
if hasattr(self, '_obj'):
object.__setattr__(self._obj, name, value)
else:
object.__setattr__(self, name, value)
But by using hasattr we would also need to alter __getattr__ to check for _obj to prevent recursion:
def __getattr__(self, name):
if '_obj' == name:
raise AttributeError
if attr in self._count:
self._count[attr]+=1
else:
self._count[attr]=1
return getattr(self._obj, attr)
An alternative would be to inspect the proxy's __dict__ attribute directly in the __setattr__ method:
def __setattr__(self, name, value):
if '_obj' in self.__dict__:
...
from the test, it is a requirement for proxy to log all the attribute calls via proxy. And the proxy has only few built-in methods which are exceptionally used for logging, so my answer was:
class Proxy(object):
def __init__(self, target_object):
self.logs=[]
self._obj = target_object
def __getattribute__(self, attrname):
if attrname in ['_obj','logs','messages','was_called','number_of_times_called'] :
return object.__getattribute__(self, attrname)
else:
self.logs.append(attrname)
return object.__getattribute__((object.__getattribute__(self, '_obj')), attrname)
def __setattr__(self, name, value):
if hasattr(self, '_obj'):
self.logs.append(name)
object.__setattr__(object.__getattribute__(self,'_obj'), name, value)
else :
object.__setattr__(self, name, value)
After this it is quite easy to implement other methods ('messages', 'was_called', ... )
Sorry for necro'ing old question.
and I found out that getattribute can be changed : just check whether the attribute is in the target object.
def __getattribute__(self, attrname):
if attrname not in dir(object.__getattribute__(self, '_obj')):
return object.__getattribute__(self, attrname)
else:
self.logs.append(attrname)
return object.__getattribute__((object.__getattribute__(self, '_obj')), attrname)
class Proxy(object):
"""Proxy class wraps any other class, and adds functionality to remember and report all messages called.
Limitations include that proxy blocks all direct subclass calls to:
messages, number_of_times_called, was_called, _obj, and _message_counts.
These calls must be made directly like my_proxy_instance._obj.messages.
"""
def __init__(self, target_object):
print 'initializing a proxy for ' + target_object.__class__.__name__
# WRITE CODE HERE
self._message_counts = Counter();
#initialize '_obj' attribute last. Trust me on this!
self._obj = target_object
# WRITE CODE HERE
def __getattr__(self, attr_name):
print 'getting an attribute: "' + attr_name + '" from "' + self._obj.__class__.__name__ + '"'
self._message_counts[attr_name] += 1
print self._message_counts
return object.__getattribute__(self._obj, attr_name)
#def __getattribute__(self, attr_name):
# print "intercepted!~ " + attr_name
# object.__getattribute__(self, attr_name)
def __setattr__(self, attr_name, value):
if((attr_name == '_obj') | (attr_name == '_message_counts')): # special proxy attributes.
print 'setting the PROXY attribute: "' + attr_name + '"'
object.__setattr__(self, attr_name, value)
else:
print 'setting the REAL attribute: "' + attr_name + '"'
self._message_counts[attr_name+"="] += 1
object.__setattr__(self._obj, attr_name, value)
def messages(self):
return self._message_counts.keys()
def number_of_times_called(self, attr_name):
return self._message_counts[attr_name]
def was_called(self, attr_name):
return attr_name in self._message_counts
What I did was take all the calls to attributes in the proxy and call them via object.__getattribute__ to avoid recursion.
That did not work for methods so I wrapped the method calls in a try..except AttributeError to try them first in the proxy. and then if they raise an error try them in the child object.
If anyone has a more elegant solution would love to see it.
from runner.koan import *
from collections import Counter
class Proxy(object):
def __init__(self, target_object):
self._messages=[]
self._obj = target_object
def messages(self):
return self._messages
def was_called(self, message):
return message in self._messages
def number_of_times_called(self, message):
_count = Counter(self._messages).get(message)
if _count:
return _count
else: # catch None
return 0
def __getattribute__(self, attr_name):
try: # call on self
retval = object.__getattribute__(self, attr_name)
except AttributeError: # call on child object
retval = self._obj.__getattribute__(attr_name)
object.__getattribute__(self, '_messages').append(attr_name)
return retval
def __setattr__(self, attr_name, attr_value):
if hasattr(self, '_obj'): # call child object and log message
self._obj.__setattr__(attr_name, attr_value)
attr_name += "="
object.__getattribute__(self, '_messages').append(attr_name)
else: # use this before_obj is set in __init__
object.__setattr__(self, attr_name, attr_value)
def messages(self):
return self._messages
why not use method_missing?
my answer:
class Proxy
def initialize(target_object)
#object = target_object
# ADD MORE CODE HERE
#messages = []
end
# WRITE CODE HERE
def method_missing(method_name, *args, &block)
#messages.push method_name unless method_name == :messages
#object.send method_name, *args, &block
end
def messages
#messages
end
def called? target
#messages.include? target
end
def number_of_times_called target
result = 0
#messages.each do |t|
result += 1 if t == target
end
result
end
end