Recently I have been learning about managed attributes in Python and a common theme with properties and descriptors is, that they have to be assigned as class attributes. But nowhere can I find an explanation of why and especially why they cannot be assigned as instance attributes. So my question has actually two parts:
why do properties / descriptor instances have to be class attributes?
why can properties / descriptor instances not be instance attributes?
It is because of the way Python tries to resolve attributes:
First it checks if it is defined at the class level
If yes, it checks if it is a property or a data descriptor
If yes, it follows this "path"
If no, it checks if it is a simple class variable (up to its parent classes if any)
If yes, it checks the instance overrides this class attribute value, if yes, it returns the overriden value, if no it returns the class attribute value
If no, it checks if the instance declares this attribute
If yes, it returns the instance attribute value
If no, it throws AttributeError
Voila ;-)
EDIT
I just found this link which explains it better than me.
Another nice illustration.
why do properties/descriptor instances have to be class attributes?
They don't have to be, they just are. This was a design decision that probably has many more reasons to back it up than I can think (simplifying implementation, separating classes from objects).
why can properties/descriptor instances not be instance attributes?
They could be, you can always override __getattribute__ in order to invoke any descriptors accessed on an instance or forbid them altogether if you desire.
Keep in mind that the fact that Python won't stop you from doing this doesn't mean it's a good idea.
Related
I've been working on multiprocessing and C++ extensions and I don't quite get the __getstate_manages_dict__ function (I know how to use it, but I'm not really sure how it works). The boost/Python documentation for pickle support says this:
The author of a Boost.Python extension class might provide a __getstate__ method without considering the possibilities that: * his class is used in Python as a base class. Most likely the dict of instances of the derived class needs to be pickled in order to restore the instances correctly. * the user adds items to the instance's __dict__ directly. Again, the __dict__ of the instance then needs to be pickled.
To alert the user to this highly unobvious problem, a safety guard is
provided. If __getstate__ is defined and the instance's __dict__ is
not empty, Boost.Python tests if the class has an attribute
__getstate_manages_dict__. An exception is raised if this attribute is not defined:
I've seen some examples where the object's __dict__ is returned in __getstate__ and then updated in __setstate__. What is this __dict__ refering to? Is it the __dict__ attribute of the derived class object? Also, why does this dict needs to be handled explicitly if pickle calls __init__ to create a new object and then sets the attribute?
Thanks
I know how to use it, but I'm not really sure how it works
It's a boolean value that is False by default. The point is to signal to Boost that the __getstate__/__setstate__ implementation handles the class' __dict__ attribute, so that the information won't be lost in the pickling process.
The idea is that Boost::Python can't actually determine whether the code is written properly, so instead you are made to jump through this extra hurdle so that, if you are unaware of the problem, you see an error message - as they say, to alert the user to this highly unobvious problem.
It's not doing anything magical. It's just there to confirm that you "read the warranty", so to speak.
The boost/Python documentation for pickle support says this:
This is just explaining the reasons why it's important to consider the __dict__ contents - even if you don't want to pickle all the attributes that you set explicitly in the __init__ (for example, because the class holds a reference to a large resource that you intend to load in some other way, or a results cache, or...). In short, instances of your class might contain information that you didn't expect them to contain, and that your __getstate__ implementation won't know how to handle, unless it takes the instance's __dict__ into account.
Hence the "practical advice" offered: "If __getstate__ is required, include the instance's __dict__ in the Python object that is returned."
What is this __dict__ referring to? Is it the __dict__ attribute of the derived class object?
It's the __dict__ attribute of the instance that __getstate__ was called upon. That could be an instance of a derived class, if that class doesn't override the methods. Or it could be an instance of the base class, which may or may not have had extra attributes added outside the class implementation.
Also, why does this dict needs to be handled explicitly if pickle calls init to create a new object and then sets the attribute?
See above. When you get the attributes (so that the pickle file can be written), you need to make sure that you actually get all the necessary attributes, or else they'll be missing upon restoration. Hard-coded logic can miss some.
Python allows for methods to be add to instances of a class rather than the whole class as demonstrated her Adding a Method to an Existing Object Instance. Most of the time this seems like a bad idea for consistent behavior of classes. When might this be necessary? Why does python allow this at all?
Python doesn't specifically allow this, it's just a consequence of the way the Python object model works. Methods are just object attributes like any other; and generally you can add any attribute to an existing object.
In source examples and SO answers that have some degree of Python object introspection, a common pattern is:
getattr(some_object, attribute_name_string)
Is there a reason why this pattern is preferred to:
some_object.__dict__[attribute_name_string]
which seems to be more directly showing what's going on? Is it because the latter is too close to a particular implementation in CPython that may be subject to change?
NB Original question incorrectly identified the popular idiom as:
some_object.__getattr__(attribute_name_string)
some_object.__getattr__ is not a common pattern. getattr(some_object, attribute_name_string) is the proper way of accessing attributes dynamically.
Not all instances have a __dict__ attribute; a class that uses __slots__ for example won't have that attribute. Next, attributes are not necessarily found on the instance, but are class attributes instead. getattr() will find those, looking at __dict__ will not find them. Attributes on the class may also depend on the descriptor protocol.
There may be uses for direct access to the __getattr__ hook or the __dict__ attribute, but those are specialised uses only. __getattr__ is only used if the attribute was not first found elsewhere, for example (so for attributes not present on the class or in instance.__dict__).
Some answers from this question bring very silly ways to cripple the ability to access methods and attributes for instances of objects overriding __dir__ and __getattribute__.
The attributes and methods are still there, but are they really inaccessible?
For example, type(x) still returns the correct answer even if x.__class__ raises AttributeError.
Is there any way to access the hidden methods and attributes?
For instances of a new-style class you could do something like this:
object.__getattribute__(instance, '__dict__')
I got the idea while reading a section titled More attribute access for new-style classes in the documentation, where they suggest doing something like that to avoid infinite recursion in its implementation.
I'm having a tricky problem in the game I'm working on. I'm using Pygame to develop it. I happen to be one of those developers who never uses the default__dict__ object variable; I always define __slots__ to clarify the variables an object can have (I have a classmethod that reads the slots to determine the variables needed from a config file).
Anyway, I just realized that this effort isn't working in some of my classes; they still have a __dict__ variable and can have arbitrary attributes assigned to, even though they explicitly define their __slots__. I think this is because they are inheriting from pygame.sprite.Sprite, which has a __dict__. If this is the case, how do I suppress creation of this dict? (I though explicitly defining __slots__ was supposed to) Or could I be mistaken about the cause? Thanks for any insight; it's hard to find information about this particular problem via searches.
The only way to suppress arbitrary attributes and the __dict__ container of them, is to use __slots__ as you are and inherit from a class that does the same. A subclass of a class that has a __dict__ will always have a __dict__. The only way around it is to not inherit from this class (but, for example, use composition instead.)