I have the following code, it is from Learning Python published by O'Reilly Media. Why line 3 (self._name = name) won't trigger __getattribute__? Is it because __setattr__ overrides it?
class Person: # Portable: 2.X or 3.X
def __init__(self, name): # On [Person()]
self._name = name # Triggers __setattr__!
def __getattribute__(self, attr): # On [obj.any]
print('get: ' + attr)
if attr == 'name': # Intercept all names
attr = '_name' # Map to internal name
return object.__getattribute__(self, attr) # Avoid looping here
def __setattr__(self, attr, value): # On [obj.any = value]
print('set: ' + attr)
if attr == 'name':
attr = '_name' # Set internal name
self.__dict__[attr] = value # Avoid looping here
You are setting an attribute. Assignment to an attribute always uses __setattr__.
__getattr__ and __getattribute__ are only consulted when looking up the value of a specific attribute; when setting you are not retrieving a value.
This is not an override; even if __setattr__ was not defined, the __getattribute__ method would not be consulted.
Related
I'm trying to create a proxy (wrapper) object so I can modify behaviour of an already instantiated object. Attributes of a wrapper class are set to a newly generated class (using type) along with attributes of an underlying object, it's done this way because Python __magic __ methods work correctly only if they are members of a class. So cls is a wrapper class, client is the underlying object:
def __new__(cls, client, *args, **kwargs):
ns = {}
for i, attr in inspect.getmembers(client):
if i in ('__init__', '__new__', '__getattribute__', '__dict__'):
continue
ns[i] = attr
for i in cls.__dict__:
if i in ('__new__',):
continue
elif i == '__init__':
ns['_init_'] = getattr(cls, i)
continue
attr = getattr(cls, i)
ns[i] = attr
P = type(cls.__name__ + "." + client.__class__.__name__,
(Proxy2.BaseProxy,), ns)
P._client_ = client
return P(*args, **kwargs)
The problem comes from #staticmethod/#classmethod in the wrapper class. I cannot call static method from an instance because self is being passed to it. I've tried to use __get__ but without success. Here's a minimal example which fails:
class SuperA:
#staticmethod
def static():
return 'static?'
class A:
def __getattribute__(self, attr):
v = object.__getattribute__(self, attr)
if hasattr(v, '__get__'):
v2 = v.__get__(None, self)
return v2
return v
A.static = getattr(SuperA, "static")
print(A.static()) # success
print(A().static()) # fail
Instead of getattr which invokes descriptors mechanism inspect.getattr_static can be used like this:
A.static = inspect.getattr_static(SuperA, "static")
My question is how to create a class like slice?
slice (built-in type) doesn't have a __dict__ attribute
even that the metaclass of this slice is type.
And it is not using __slots__, and all it's attribute are readonly and it's not overriding
__setattr__ (this i'm not sure about it but look at my code and see if I'm right).
Check this code:
# how slice is removing the __dict__ from the class object
# and the metaclass is type!!
class sliceS(object):
pass
class sliceS0(object):
def __setattr__(self, name, value):
pass
# this means that both have the same
# metaclass type.
print type(slice) == type(sliceS) # prints True
# from what i understand the metaclass is the one
# that is responsible for making the class object
sliceS2 = type('sliceS2', (object,), {})
# witch is the same
# sliceS2 = type.__new__(type, 'sliceS2', (object,), {})
print type(sliceS2) # prints type
# but when i check the list of attribute using dir
print '__dict__' in dir(slice) # prints False
print '__dict__' in dir(sliceS) # prints True
# now when i try to set an attribute on slice
obj_slice = slice(10)
# there is no __dict__ here
print '__dict__' in dir(obj_slice) # prints False
obj_sliceS = sliceS()
try:
obj_slice.x = 1
except AttributeError as e:
# you get AttributeError
# mean you cannot add new properties
print "'slice' object has no attribute 'x'"
obj_sliceS.x = 1 # Ok: x is added to __dict__ of obj_sliceS
print 'x' in obj_sliceS.__dict__ # prints True
# and slice is not using __slots__ because as you see it's not here
print '__slots__' in dir(slice) # print False
# and this why i'm saying it's not overriding the __settattr__
print id(obj_slice.__setattr__) == id(obj_sliceS.__setattr__) # True: it's the same object
obj_sliceS0 = sliceS0()
print id(obj_slice.__setattr__) == id(obj_sliceS0.__setattr__) # False: it's the same object
# so slice have only start, stop, step and are all readonly attribute and it's not overriding the __setattr__
# what technique it's using?!!!!
How to make this kind of first-class object all of it's attributes are readonly and you cannot
add new attributes.
The thing is that Python's built-in slice class is programmed in C. And when you code using the C-Python API you can code the equivalent of attributes accessible with the __slots__ without using any mechanisms visible from the Python side. (You can even have 'real' private attributes, which are virtually impossible with Python only code).
The mechanism used for Python code to be able to prevent a __dict__ for a class' instances and subsequent "any attribute can be set" is the __slots__ exactly the attribute.
However, unlike magic dunder methods that have to be present when the class is actually used, the information on __slots__ is used when the class is created, and only then. So, if what concerns you is to have a visible __slots__ in your final class, you can just remove it from the class before exposing it:
In [8]: class A:
...: __slots__ = "b"
...:
In [9]: del A.__slots__
In [10]: a = A()
In [11]: a.b = 5
In [12]: a.c = 5
------------------------
AttributeError
...
In [13]: A.__slots__
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-13-68a69c802e74> in <module>()
----> 1 A.__slots__
AttributeError: type object 'A' has no attribute '__slots__'
If you won't like a del MyClass.__slots__ line to be visible wherever you declare a class, it is a one-line class decorator:
def slotless(cls):
del cls.__slots__
return cls
#slotless
class MyClass:
__slots__ = "x y".split()
Or, you could use a metaclass to auto-create, and auto-destroy the Python visible __slots__, so that you could declare your descriptors and attributes in the class body, and have the class protected against extra attributes:
class AttrOnly(type):
def __new__(metacls, name, bases, namespace, **kw):
namespace["__slots__"] = list(namespace.keys()) # not sure if "list(" is needed
cls = super().__new__(metacls, name, bases, namespace, **kw)
del cls.__slots__
return cls
class MyClass(metaclass=AttrOnly):
x = int
y = int
If you want pure Python readonly attributes which does not have a visible counterpart in the instance itself (like a ._x which is used by a property descriptor to keep the value of a x attribute), the straightforward way is to customize __setattr__ . Another approach is to have your metaclass to auto-add a read-only property for each attribute on the class creation stage. The metaclass bellow does that and uses the __slots__ class attribute to create the desired descriptors:
class ReadOnlyAttrs(type):
def __new__(metacls, name, bases, namespace, **kw):
def get_setter(attr):
def setter(self, value):
if getattr(self, "_initialized", False):
raise ValueError("Can't set " + attr)
setattr(self, "_" + attr, value)
return setter
slots = namespace.get("__slots__", [])
slots.append("initialized")
def __new__(cls, *args, **kw):
self = object.__new__(cls) # for production code that could have an arbitrary hierarchy, this needs to be done more carefully
for attr, value in kw.items():
setattr(self, attr, value)
self.initialized = True
return self
namespace["__new__"] = __new__
real_slots = []
for attr in slots:
real_slots.append("_" + attr)
namespace[attr] = property(
(lambda attr: lambda self: getattr(self, "_" + attr))(attr), # Getter. Extra lambda needed to create an extra closure containing each attr
get_setter(attr)
)
namespace["__slots__"] = real_slots
cls = super().__new__(metacls, name, bases, namespace, **kw)
del cls.__slots__
return cls
Have in mind you can also customize the class' __dir__ method so that _x shadow attributes would not be seen, if you want to.
In my object's init, I would like to create object properties from an iterable. For example:
class MyClass(object):
def __init__(self, parameters):
attributes = ['name',
'memory',
'regressors',
'use_const']
for attr_name in attributes():
try:
attr_val = parameters[attr_name]
except KeyError:
raise Error("parameters must contain {}".format(attr_name))
setattr(self, attr_name, attr_val)
This lets me get the attributes that I want. However, what I lose compared to defining
#property
def name(self):
"""str: This class' name"""
return self._name
is that I don't get the docstrings for the properties now.
I'd like to have the docstrings for each property (for my auto-generated documentation), but I'd also like to use an iterable instead of having to define each property separately. For example, can I turn attributes into a dict with the docstring as a value, and set the attribute's docstring dynamically?
Can I have my cake and eat it too?
You can only set property objects on the class. You can do this in a loop, but this has to be done when building the class, not instances.
Simply produce property objects:
def set_property(cls, name, attr, docstring):
def getter(self):
return getattr(self, attr)
prop = property(getter, None, None, docstring)
setattr(cls, name, prop)
for name in attributes:
attr = '_' + name
docstring = "str: This class' {}".format(name)
set_property(SomeClass, name, attr, docstring)
I had to write a class of some sort that overrides __getattribute__.
basically my class is a container, which saves every user-added property to self._meta which is a dictionary.
class Container(object):
def __init__(self, **kwargs):
super(Container, self).__setattr__('_meta', OrderedDict())
#self._meta = OrderedDict()
super(Container, self).__setattr__('_hasattr', lambda key : key in self._meta)
for attr, value in kwargs.iteritems():
self._meta[attr] = value
def __getattribute__(self, key):
try:
return super(Container, self).__getattribute__(key)
except:
if key in self._meta : return self._meta[key]
else:
raise AttributeError, key
def __setattr__(self, key, value):
self._meta[key] = value
#usage:
>>> a = Container()
>>> a
<__main__.Container object at 0x0000000002B2DA58>
>>> a.abc = 1 #set an attribute
>>> a._meta
OrderedDict([('abc', 1)]) #attribute is in ._meta dictionary
I have some classes which inherit Container base class and some of their methods have #property decorator.
class Response(Container):
#property
def rawtext(self):
if self._hasattr("value") and self.value is not None:
_raw = self.__repr__()
_raw += "|%s" %(self.value.encode("utf-8"))
return _raw
problem is that .rawtext isn't accessible. (I get attributeerror.) every key in ._meta is accessible, every attributes added by __setattr__ of object base class is accessible, but method-to-properties by #property decorator isn't. I think it has to do with my way of overriding __getattribute__ in Container base class. What should I do to make properties from #property accessible?
I think you should probably think about looking at __getattr__ instead of __getattribute__ here. The difference is this: __getattribute__ is called inconditionally if it exists -- __getattr__ is only called if python can't find the attribute via other means.
I completely agree with mgilson. If you want a sample code which should be equivalent to your code but work well with properties you can try:
class Container(object):
def __init__(self, **kwargs):
self._meta = OrderedDict()
#self._hasattr = lambda key: key in self._meta #???
for attr, value in kwargs.iteritems():
self._meta[attr] = value
def __getattr__(self, key):
try:
return self._meta[key]
except KeyError:
raise AttributeError(key)
def __setattr__(self, key, value):
if key in ('_meta', '_hasattr'):
super(Container, self).__setattr__(key, value)
else:
self._meta[key] = value
I really do not understand your _hasattr attribute. You put it as an attribute but it's actually a function that has access to self... shouldn't it be a method?
Actually I think you should simple use the built-in function hasattr:
class Response(Container):
#property
def rawtext(self):
if hasattr(self, 'value') and self.value is not None:
_raw = self.__repr__()
_raw += "|%s" %(self.value.encode("utf-8"))
return _raw
Note that hasattr(container, attr) will return True also for _meta.
An other thing that puzzles me is why you use an OrderedDict. I mean, you iterate over kwargs, and the iteration has random order since it's a normal dict, and add the items in the OrderedDict. Now you have _meta which contains the values in random order.
If you aren't sure whether you need to have a specific order or not, simply use dict and eventually swap to OrderedDict later.
By the way: never ever use an try: ... except: without specifying the exception to catch. In your code you actually wanted to catch only AttributeErrors so you should have done:
try:
return super(Container, self).__getattribute__(key)
except AttributeError:
#stuff
I would like to have a special obj that does the following:
obj.newly_created_attribute = some_value
Obviously, all objects will allow this. But I would like the previous code to automatically call a method when newly_created_attribute is not yet a attribute of obj. In my particular case, I wish to set up a custom get and set method for obj.newly_created_attribute (a property now).
Is there any way to do this? Some way to specify a callback that will be run whenever a new attribute is added to a object?
You can accomplish this by overriding __setattr__:
class SomeClass(object):
def __setattr__(self, name, value):
if not hasattr(self, name):
print "new attribute", name
# do stuff here
return object.__setattr__(self, name, value)
__setattr__ will help you there:
Called when an attribute assignment is attempted. This is called instead of the normal mechanism (i.e. store the value in the instance dictionary). name is the attribute name, value is the value to be assigned to it.
#!/usr/bin/env python
class Klass(object):
def __setattr__(self, name, value):
if not hasattr(self, name):
self.on_first_setattr()
return object.__setattr__(self, name, value)
def on_first_setattr(self):
print "I am just a callback and my story's seldom told."
obj = Klass()
obj.some_attr = 1 # will call callback
obj.some_attr = 2 # no output
Overload __setattr__. Example:
class Foo(object):
def __setattr__(self, attr, val):
print "setattr"
if attr not in self.__dict__:
print "new attr:", attr
self.__dict__[attr] = val
else:
print "extant attr:", attr
self.__dict__[attr] = val