I'm having a tricky problem in the game I'm working on. I'm using Pygame to develop it. I happen to be one of those developers who never uses the default__dict__ object variable; I always define __slots__ to clarify the variables an object can have (I have a classmethod that reads the slots to determine the variables needed from a config file).
Anyway, I just realized that this effort isn't working in some of my classes; they still have a __dict__ variable and can have arbitrary attributes assigned to, even though they explicitly define their __slots__. I think this is because they are inheriting from pygame.sprite.Sprite, which has a __dict__. If this is the case, how do I suppress creation of this dict? (I though explicitly defining __slots__ was supposed to) Or could I be mistaken about the cause? Thanks for any insight; it's hard to find information about this particular problem via searches.
The only way to suppress arbitrary attributes and the __dict__ container of them, is to use __slots__ as you are and inherit from a class that does the same. A subclass of a class that has a __dict__ will always have a __dict__. The only way around it is to not inherit from this class (but, for example, use composition instead.)
Related
I've been working on multiprocessing and C++ extensions and I don't quite get the __getstate_manages_dict__ function (I know how to use it, but I'm not really sure how it works). The boost/Python documentation for pickle support says this:
The author of a Boost.Python extension class might provide a __getstate__ method without considering the possibilities that: * his class is used in Python as a base class. Most likely the dict of instances of the derived class needs to be pickled in order to restore the instances correctly. * the user adds items to the instance's __dict__ directly. Again, the __dict__ of the instance then needs to be pickled.
To alert the user to this highly unobvious problem, a safety guard is
provided. If __getstate__ is defined and the instance's __dict__ is
not empty, Boost.Python tests if the class has an attribute
__getstate_manages_dict__. An exception is raised if this attribute is not defined:
I've seen some examples where the object's __dict__ is returned in __getstate__ and then updated in __setstate__. What is this __dict__ refering to? Is it the __dict__ attribute of the derived class object? Also, why does this dict needs to be handled explicitly if pickle calls __init__ to create a new object and then sets the attribute?
Thanks
I know how to use it, but I'm not really sure how it works
It's a boolean value that is False by default. The point is to signal to Boost that the __getstate__/__setstate__ implementation handles the class' __dict__ attribute, so that the information won't be lost in the pickling process.
The idea is that Boost::Python can't actually determine whether the code is written properly, so instead you are made to jump through this extra hurdle so that, if you are unaware of the problem, you see an error message - as they say, to alert the user to this highly unobvious problem.
It's not doing anything magical. It's just there to confirm that you "read the warranty", so to speak.
The boost/Python documentation for pickle support says this:
This is just explaining the reasons why it's important to consider the __dict__ contents - even if you don't want to pickle all the attributes that you set explicitly in the __init__ (for example, because the class holds a reference to a large resource that you intend to load in some other way, or a results cache, or...). In short, instances of your class might contain information that you didn't expect them to contain, and that your __getstate__ implementation won't know how to handle, unless it takes the instance's __dict__ into account.
Hence the "practical advice" offered: "If __getstate__ is required, include the instance's __dict__ in the Python object that is returned."
What is this __dict__ referring to? Is it the __dict__ attribute of the derived class object?
It's the __dict__ attribute of the instance that __getstate__ was called upon. That could be an instance of a derived class, if that class doesn't override the methods. Or it could be an instance of the base class, which may or may not have had extra attributes added outside the class implementation.
Also, why does this dict needs to be handled explicitly if pickle calls init to create a new object and then sets the attribute?
See above. When you get the attributes (so that the pickle file can be written), you need to make sure that you actually get all the necessary attributes, or else they'll be missing upon restoration. Hard-coded logic can miss some.
Functions in a python class can be either instance methods, class methods or static methods.
The former is characterised by the self as its first (implicit) argument, acts directly on the instance of the class, and does not require any decorators to be treated as such.
The other two, however, need decorators #classmethod and #staticmethod before the name of the method - this is why I refer to the instance method as the "default" one, i.e. the one for which a wrapper is not needed.
My question is: suppose I am in a class, and I am breaking up my calculation into several functions for readibility. Only one of these methods will need access to the self.something variables that I share instance-wise, but most of the others do not need to know about the class they belong to - they are just there for "housekeeping".
Should make these functions (the ones that do not need any self.something knowledge) all #staticmethod? Doing so would require a decorator and hence an extra step. It would be easier (not requiring the extra step of using a decotrator) for every method to just be an instance method, thus inheritig a lot of potential but also waisting it since it is not needed for the scope of the functions in question.
Why is the instance method the "default"? Why not have every method a static method by default, and give it the extra functionality associated with being a instance method with a wrapper?
The reason to default to instance methods is because that's usually what you want when you're doing object oriented programming. I can't think of a single language that claims to support OOP and has methods default to anything but instance methods. Classes are templates for "data with behaviors", so the default is to make methods that provide behaviors to each instantiation of the class. If you just want a collection of functions, you can just define them at the top level of a module and save the unnecessary class after all.
In general, #staticmethod is used to mean "I know this isn't a behavior of the class or its instances, but it helps implement the real behaviors and isn't very useful outside the class, so I'll namespace it inside it." If the features are useful outside the class, you'd just make it a plain top-level function rather putting it inside the class at all. It is advantageous to use #staticmethod where appropriate; it's a little faster to call than an instance method, so if you don't need the instance, #staticmethod will speed up your code a bit (note: This may not be true in 3.7+, where they added an optimization to avoid the creation of bound methods, which may speed up instance/class methods).
#classmethod basically has two use cases:
(Primary) Defining alternate constructors in a subclass friendly way (the cls it receives is the actual subclass, if applicable, not just the class it was defined in)
(Mostly unnecessary) As an alternative to #staticmethod when the method needs to call other static methods and you'd rather not have to refer to the class by name over and over
Point is, #staticmethod is mostly for when you're opting out of OOP, and #classmethods are for niche use cases; instance methods are just more useful, so they're the default. Beyond that, as a historical note, static and class methods were introduced later, so making them the default would have broken all existing Python code, for no real benefit.
The main reason to use #staticmethod over instance methods with an ignored self (when self isn't needed) is that it will continue to work when called on the class itself, not just on instances of the class; if you tried to call MyClass.notreallystatic(), it would die for lack of a self, while MyClass.actuallystatic() would work.
Using WTForms form definition classes as an example:
class RegistrationForm(Form):
username = StringField('Username', [validators.Length(min=4, max=25)])
email = StringField('Email Address', [validators.Length(min=6, max=35)])
accept_rules = BooleanField('I accept the site rules', [validators.InputRequired()])
and looking at the source of the library it seems WTForms allows a user to define a very simple class of a customised form structure (as above), which in turn then gets used to construct a new field class which is not instantiated when the class is generated.
I've read a number of tutorials about class factories and metaclasses and the general consensus is to avoid metaclasses and use things like class decorators instead. The problem is the tutorials either start importing extra libraries eg: import six, mix explanations of different Python versions together, use overly complex examples or advise not to use metaclasses at all.
Please can somebody provide a very simple explanation (for Python 3) of how to use a simple class definition (like the WTForms example above) along with metaclasses to customise a brand new construction of a class without actually instantiating the class when it's constructed.
Edit: Apologies for finding it difficult to explain what my end goal is but as I have gone through tutorials it has been unclear as to whether class decorators, metaclasses, magic methods (call, new, init) or a combination of such are what I needed to achieve what I visualised, or if what I was visualising was the wrong way of doing things. Unfortunately it seems impossible to judge if my goal was wrong without being able to understand the mechanisms needed to achieve it. I've realised metaclasses are the way to go and just need pointing in the right direction for a very simple metaclass example done the Python 3.x way.
You can create classes dynamically - with no custom metaclasses and no decorators with what looks to the programmer as simple function call.
Just make a call to Python's builtin type with three parameters:
the name of the class, a tuple with its bases, and a mapping object with its namespace (i.e. a dictionary containing the attributes and methods you would ordinarily define on the class body).
def __init__(self):
...
namespace = {
'__init__': init,
'name': 'default name'
}
MyClass = type("MyClass", (object,), namespace)
You loose some features that are only possible due to the compiler doing a couple special things during building functions declared within a class body - mostly the ability to use paramterless super and name mangling of attributes starting with __, but that is it.
That said it should be noted this is not with "no metaclasses". "type" is itself a metaclass - the default Python metaclass for all objects - and is calling a metaclass that create a class. There is no other way to create a class. A "class decorator" is just a method that can makes changes to a class object after it is created.
Any function or method that yields a new, dynamic class, will have inside it to, at some point, call type or other metaclass. In the same mood, a "metaclass" does not create dynamic classes by itself - it needs do be either used in a class body declaration, or called with (at least) the same parameters used for calling type.
As for the recommendations for "class decorators" instead of metaclasses, I am not sure is that true (beyond the fact there is no way a "class decorator" can create classes dynamically by itself): their main drawback is that there is no ordinary way for subclasses of decorated classes to have the parent's class decorators applied to themselves automatically, while metaclasses are inherited.
In Python 3.6 you have the __init_subclass__ protocol which, yes, can avoid a lot of the traditional uses for a metaclass (but still, it won't "create classes dynamically" - calling type does that).
Recently I have been learning about managed attributes in Python and a common theme with properties and descriptors is, that they have to be assigned as class attributes. But nowhere can I find an explanation of why and especially why they cannot be assigned as instance attributes. So my question has actually two parts:
why do properties / descriptor instances have to be class attributes?
why can properties / descriptor instances not be instance attributes?
It is because of the way Python tries to resolve attributes:
First it checks if it is defined at the class level
If yes, it checks if it is a property or a data descriptor
If yes, it follows this "path"
If no, it checks if it is a simple class variable (up to its parent classes if any)
If yes, it checks the instance overrides this class attribute value, if yes, it returns the overriden value, if no it returns the class attribute value
If no, it checks if the instance declares this attribute
If yes, it returns the instance attribute value
If no, it throws AttributeError
Voila ;-)
EDIT
I just found this link which explains it better than me.
Another nice illustration.
why do properties/descriptor instances have to be class attributes?
They don't have to be, they just are. This was a design decision that probably has many more reasons to back it up than I can think (simplifying implementation, separating classes from objects).
why can properties/descriptor instances not be instance attributes?
They could be, you can always override __getattribute__ in order to invoke any descriptors accessed on an instance or forbid them altogether if you desire.
Keep in mind that the fact that Python won't stop you from doing this doesn't mean it's a good idea.
I'm hoping someone may be able to help me out with a design issue I'm dealing with. It's specifically in the game development domain, but I think it's really a broader issue that has probably been solved in an accepted way. I'm working in Python.
I have a GameObject class that holds the position of the object (and other general state attributes) and a reference to my Engine object, which holds information about the game world at large. GameObjects can be a categorized further: they can be VisibleGameObjects, PhysicalGameObjects (collidable), or both, in concrete form. For example, I could have an invisible boundary, which is physical, but does not have a visible representation.
VisibleGameObjects implement a draw() method that handles drawing functionality, delegating this through its parent's Engine reference. PhysicalGameObjects have bounding boxes, and define logic to handle collisions, also requiring access to GameObject attributes (acceleration, velocity, etc.)
The problem is, what happens when I'd like to define a concrete object that needs to inherit the behavior of both a VisibleGameObject, and a PhysicalGameObject (which both share a parent GameObject)? It's my understanding that this type of circular inheritance is a big-bad idea.
How can I refactor this to essentially bolt on the specific behaviors to a concrete child class (drawable, collidable) that depend on the state of the parent abstract class?
EDIT: My one thought was to assign them to concrete instances of GameObjects as components, favoring a has-a relationship over an is-a relationship. Even that doesn't seem so clean however; trying to check to see if an object is collidable by searching a "components" list for a collidable component doesn't seem great either.
It seems like you're looking for a trait
Unfortunately, python doesn't support traits natively, although there are multiple modules that try to implement the model.
My suggestion (unless you want to depend on the mentioned modules) would be to write abstract classes to expose the behaviour you want, but that don't inherit the main class - leaving that to a third class, which inherits both the main, and the behaviour-class.
It's probably less confusing with an example:
create a Visible abstract class that does not inherit from GameObject, and exposes all the intended behaviour/functions (as if it inherited from GameObject). Then, have VisibleGameObject inherit from both GameObject and Visible.
Obviously, you can only manage to write Visible on a dynamic language like python - otherwise the compiler would complain that it couldn't access inexistent fields.