Develop Reference

Cheking if an object has an attribute, without relying on '__getattr__'

Is there a way to check if an object has an attribute, that doesn't rely on __getattr__ or object implementation specifics?
Consider the code below. I want Proxy to delegate to Wrapped what it can't handle. The code works, but I'd like to avoid the test attr in self.__dict__ (I'd prefer a stable interface to do this, without using implementation quirks). The function hasattr doesn't help here, because it gets routed to the wrapped object via __getattr__.
class Wrapped:
def __init__(self):
self.a = 0
self.b = 1
class Proxy:
def __init__(self, wrapped):
object.__setattr__(self, 'wrapped', wrapped)
object.__setattr__(self, 'a', 2)
def __getattr__(self, attr):
return getattr(self.wrapped, attr)
def __setattr__(self, attr, value):
if attr in self.__dict__: # <-- Don't like this, hasattr doesn't work
object.__setattr__(self, attr, value)
elif hasattr(self.wrapped, attr):
setattr(self.wrapped, attr, value)
else: object.__setattr__(self, attr, value)
Testdrive:
>>> w = Wrapped()
>>> p = Proxy(w)
>>> p.a
2
>>> p.b
1
>>> p.a = 3
>>> p.a
3
>>> w.a
0
>>> p.b = 4
>>> p.b
4
>>> w.b
4
>>> p.c = 5
>>> p.c
5
>>> w.c
AttributeError: 'Wrapped' object has no attribute 'c'

The built-in hasattr() function
is implemented by calling getattr(object, name) and seeing whether it raises an AttributeError or not.
The inspect module has the getattr_static() method, which can be used to
Retrieve attributes without triggering dynamic lookup via the descriptor protocol, __getattr__() or __getattribute__()".
Using this method, you could define a hasattr_static() method similarly to hasattr(), but calling inspect.getattr_static(object, name) instead of getattr(object, name):
import inspect
def hasattr_static(object, name):
try:
inspect.getattr_static(object, name)
return True
except AttributeError:
return False
And then use this in the __setattr__() method of your Proxy class as a check:
def __setattr__(self, attr, value):
if hasattr_static(self, attr):
object.__setattr__(self, attr, value)
elif hasattr(self.wrapped, attr):
setattr(self.wrapped, attr, value)
else:
object.__setattr__(self, attr, value)
Alternatively, you coud use the dir() function instead of __dict__ to retrieve a list of attributes of your object, or use inspect.getmembers().

Explain the use of __setattr__

Can you please explain the use of the __setattr__ method in the below code :
class Human(object):
def __setattr__(self,name,value):
if name == 'gender':
if value in ('male', 'female'):
self.gender = value
else :
raise AttributeError('Gender can only be Male or Female')
h = Human()
h.name = 'Mary'
h.gender = 'female'
print h.gender

In your code, __setattr__() function is making a validation that while assigning the value of self.gender, it's value should be only between male and female. In case of any other value, it will raise AttributeError exception.
Note: In your __setattr__() function, there is no call to super of __setattr__, and hence this function is not actually updating the properties of class. You should add below line in your __setattr__() in order to make it work:
super(Human, self).__setattr__(name, value)
Overview about .__setattr__(self, name, value):
Whenever you assign any value to the property of the class, __setattr__(self, name, value) function is called where name is the property of class and value is the new value to be assigned. Below is the sample code to demonstrate that:
>>> class MyClass(object):
... def __init__(self, x, y):
... self.x = x
... self.y = y
... def __setattr__(self, name, value):
... print 'In Set Attr for: ', name, ', ', value
... super(MyClass, self).__setattr__(name, value)
...
...
>>> my_object = MyClass(1, 2)
In Set Attr for: x, 1
In Set Attr for: y, 2
>>> my_object.x = 10 # <-- Called while assigning value x
In Set Attr for: x, 10

How to write metaclass which would prevent creating new attributes after __init__()?

Currently I override the class' __setattr__() towards the end of the class' __init__() method to prevent new attribute creation -
class Point(object):
def __init__(self):
self.x = 0
self.y = 0
Point.__setattr__ = self._setattr
def _setattr(self, name, value):
if not hasattr(self, name):
raise AttributeError("'" + name + "' not an attribute of Point object.")
else:
super(Point, self).__setattr__(name, value)
Is there a way to avoid manually overriding __setattr__() and do this automatically with the help of metaclasses?
The closest I came was -
class attr_block_meta(type):
def __new__(meta, cname, bases, dctry):
def _setattr(self, name, value):
if not hasattr(self, name):
raise AttributeError("'" + name + "' not an attribute of " + cname + " object.")
object.__setattr__(self, name, value)
dctry.update({'x': 0, 'y': 0})
cls = type.__new__(meta, cname, bases, dctry)
cls.__setattr__ = _setattr
return cls
class ImPoint(object):
__metaclass__ = attr_block_meta
Is there a more generic way of doing this such that apriori knowledge of the subclass attributes is not required?
Basically, how to avoid the line dctry.update({'x': 0, 'y': 0}) and make this work irrespective of what the names of class attributes are?
P.S. - FWIW I have already evaluated the __slots__ and namedtuple options and found them lacking for my needs. Please don't narrow your focus to the pared down Points() example that I have used to illustrate the question; the actual use case involves a far more complex class.

Don't reinvent the wheel.
Two simple ways to achieve that (without directly using a metaclass) are using:
namedtuples
__slots__
For example, using namedtuple (based on the example in the docs):
Point = namedtuple('Point', ['x', 'y'])
p = Point(11, 22)
p.z = 33 # ERROR
For example, using __slots__:
class Point(object):
__slots__ = ['x', 'y']
def __init__(self, x=0, y=0):
self.x = x
self.y = y
p = Point(11,22)
p.z = 33 # ERROR

Would this make sense for your case?
from functools import wraps
class attr_block_meta(type):
def __new__(meta, cname, bases, dctry):
def _setattr(self, name, value):
if not hasattr(self, name):
raise AttributeError("'" + name + "' not an attibute of " + cname + " object.")
object.__setattr__(self, name, value)
def override_setattr_after(fn):
#wraps(fn)
def _wrapper(*args, **kwargs):
cls.__setattr__ = object.__setattr__
fn(*args, **kwargs)
cls.__setattr__ = _setattr
return _wrapper
cls = type.__new__(meta, cname, bases, dctry)
cls.__init__ = override_setattr_after(cls.__init__)
return cls
class ImPoint(object):
__metaclass__ = attr_block_meta
def __init__(self, q, z):
self.q = q
self.z = z
point = ImPoint(1, 2)
print point.q, point.z
point.w = 3 # Raises AttributeError
See this for more details on 'wraps'.
You probably need to fiddle a little bit more with it to get it more elegant, but the general idea is to override __setattr__ only after init is called.
Having said that, a common approach to this is just to use object.__setattr__(self, field, value) internally to bypass the AttributeError:
class attr_block_meta(type):
def __new__(meta, cname, bases, dctry):
def _setattr(self, name, value):
if not hasattr(self, name):
raise AttributeError("'" + name + "' not an attibute of " + cname + " object.")
object.__setattr__(self, name, value)
cls = type.__new__(meta, cname, bases, dctry)
cls.__setattr__ = _setattr
return cls
class ImPoint(object):
__metaclass__ = attr_block_meta
def __init__(self, q, z):
object.__setattr__(self, 'q', q)
object.__setattr__(self, 'z', z)
point = ImPoint(1, 2)
print point.q, point.z
point.w = 3 # Raises AttributeError

You don't need metaclasses to solve this kind of problem.
If you want to create the data once up front and then have it be immutable, I would definitely use a namedtuple as shx2 suggests.
Otherwise, just define a collection of allowed fields on the class, and have __setattr__ check to see if the name that you're attempting to set is in the allowed fields collection. You don't need to change the implementation of __setattr__ part way through __init__ -- it will work during __init__ the just the same as it will work later. Use a tuple or a frozenset as the data structure for the allowed fields, if you want to discourage mutating/changing them on a given class.
class Point(object):
_allowed_attrs = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
def __setattr__(self, name, value):
if name not in self._allowed_attrs:
raise AttributeError(
"Cannot set attribute {!r} on type {}".format(
name, self.__class__.__name__))
super(Point, self).__setattr__(name, value)
p = Point(5, 10)
p.x = 9
p.y = "some string"
p.z = 11 # raises AttributeError
This can easily be factored out into a base-class for re-use:
class RestrictedAttributesObject(object):
_allowed_attrs = ()
def __setattr__(self, name, value):
if name not in self._allowed_attrs:
raise AttributeError(
"Cannot set attribute {!r} on type {}".format(
name, self.__class__.__name__))
super(RestrictedAttributesObject, self).__setattr__(name, value)
class Point(RestrictedAttributesObject):
_allowed_attrs = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
I don't think it would be considered pythonic to lock down the allowed attributes of an object in this way, and it will cause some complication for subclasses that need additional attributes (a subclass will have to ensure that the _allowed_attrs field has contents appropriate for it).

I have this same need (for a development quick-hack API). I don't use metaclasses for this, just inheritance:
class LockedObject(object):
def __setattr__(self, name, value):
if name == "_locked":
object.__setattr__(self, name, value)
return
if hasattr(self, "_locked"):
if not self._locked or hasattr(self, name):
object.__setattr__(self, name, value)
else:
raise NameError("Not allowed to create new attribute {} in locked object".format(name))
else: # never called _lock(), so go on
object.__setattr__(self, name, value)
def _lock(self):
self._locked = True
def _unlock(self):
self._locked = False
Then:
class Base(LockedObject):
def __init__(self):
self.a = 0
self.b = 1
self._lock()
If I need to subclass Base and add extra attributes I use unlock:
class Child(Base):
def __init__(self):
Base.__init__(self)
self._unlock()
self.c = 2
self._lock()
If Base is abstract you can skip its locking and just lock the childs.
I have then some unittests that check that every public class is locked after init to catch me if I forget the locking.

Python: how to access an attribute from a __getattribute__ method

I have the following class:
class StrLogger(str):
def __init__(self, *args):
self._log_ = []
str.__init__(self, *args)
def __getattribute__(self, attr):
self._log_.append((self.__name__, attr))
return str.__getattribute__(self, attr)
I can initialize a StrLogger with slog = StrLogger('foo') and I can access all of its inherited methods from str and it runs with no problem. The problem is, when I try to retreive the log with either slog._log_ or slog.__dict__['_log_'], the __getattribute__ method gets stuck in an infinite recursion. I understand why this is happening but my question is, how can I access the log?

I can think of one way. Use object.__getattribute__ (or whatever your superclass is) whenever you need to bypass your customized attribute access.
class C(object):
def __init__(self):
self._log = []
def __getattribute__(self, attr):
_log = object.__getattribute__(self, '_log')
_log.append(attr)
return object.__getattribute__(self, attr)
>>> a = C()
>>> a.x = 1
>>> a.x
1
>>> a._log
['x', '_log']

The following slightly modified class works:
class StrLogger(str):
def __init__(self, *args):
self._log_ = []
str.__init__(self, *args)
def __getattribute__(self, attr):
log = str.__getattribute__(self, '_log_')
cls = str.__getattribute__(self, '__class__')
name = cls.__name__
log.append((name, attr))
return str.__getattribute__(self, attr)
s = StrLogger('abc')
print(s.title())
print(s.lower())
print(s.upper())
print(s.__dict__)
Running it results in
Abc
abc
ABC
{'_log_': [('StrLogger', 'title'), ('StrLogger', 'lower'), ('StrLogger', 'upper'), ('StrLogger', '__dict__')]}

Your __getattribute__ should exclude __dict__ and maybe as well _log_ from logging. Alternatively, you could do something like
slog = StrLogger('foo')
thelog = slog._log_
do_stuff_with(slog)
print thelog
(untested!)

Using both __setattr__ and descriptors for a python class

I'm writing a python class that uses __setattr__ and __getattr__ to provide custom attribute access.
However, some attributes can't be handled in a generic way, so I was hoping to use descriptors for those.
A problem arises in that for a descriptor, the descriptor's __get__ will be invoked in favour of the instances __getattr__, but when assigning to an attribute, __setattr__ will be invoked in favour of the descriptors __set__.
An example:
class MyDesc(object):
def __init__(self):
self.val = None
def __get__(self, instance, owner):
print "MyDesc.__get__"
return self.val
def __set__(self, instance, value):
print "MyDesc.__set__"
self.val = value
class MyObj(object):
foo = MyDesc()
def __init__(self, bar):
object.__setattr__(self, 'names', dict(
bar=bar,
))
object.__setattr__(self, 'new_names', dict())
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
self.new_names[name] = value
def __getattr__(self, name):
print "MyObj.__getattr__ for %s" % name
if name in self.new_names:
return self.new_names[name]
if name in self.names:
return self.names[name]
raise AttributeError(name)
if __name__ == "__main__":
o = MyObj('bar-init')
o.bar = 'baz'
print o.bar
o.foo = 'quux'
print o.foo
prints:
MyObj.__setattr__ for bar
MyObj.__getattr__ for bar
baz
MyObj.__setattr__ for foo
MyDesc.__get__
None
The descriptor's __set__ is never called.
Since the __setattr__ definition isn't just overriding behaviour for a limited set of names, there's no clear place that it can defer to object.__setattr__
Is there a recommended way to have assigning to attributes use the descriptor, if available, and __setattr__ otherwise?

I think I'd approach this by having a mechanism to automatically mark which are the
descriptors in each class, and wrap the __setattr__ in a way that it'd call
object's normal behavior for those names.
This can be easily achieved with a metaclass (and a decorator for __setattr__
def setattr_deco(setattr_func):
def setattr_wrapper(self, attr, value):
if attr in self._descriptors:
return object.__setattr__(self, attr, value)
return setattr_func(self, attr, value)
return setattr_wrapper
class MiscSetattr(type):
def __new__(metacls, name, bases, dct):
descriptors = set()
for key, obj in dct.items():
if key == "__setattr__":
dct[key] = setattr_deco(obj)
elif hasattr(obj, "__get__"):
descriptors.add(key)
dct["_descriptors"] = descriptors
return type.__new__(metacls, name, bases, dct)
# and use MiscSetattr as metaclass for your classes

One of possible ways:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
for cls in self.__class__.__mro__ + (self, ):
if name in cls.__dict__:
return object.__setattr__(self, name, value)
print 'New name', name, value
self.new_names[name] = value
It checks if name already defined in class, base classes or instance and then it calls object.__setattr__ which will execute descriptor __set__.
Another way:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
try:
object.__getattribute__(self, name)
except AttributeError:
print 'New name', name, value
self.new_names[name] = value
else:
object.__setattr__(self, name, value)
But it will call descriptor's __get__.
P.S.
I'm not sure about need to check all __mro__ members since MyObj will contain inherited class members in __dict__.
Maybe for cls in (self.__class__, self):... will be enough.