Using WTForms form definition classes as an example:
class RegistrationForm(Form):
username = StringField('Username', [validators.Length(min=4, max=25)])
email = StringField('Email Address', [validators.Length(min=6, max=35)])
accept_rules = BooleanField('I accept the site rules', [validators.InputRequired()])
and looking at the source of the library it seems WTForms allows a user to define a very simple class of a customised form structure (as above), which in turn then gets used to construct a new field class which is not instantiated when the class is generated.
I've read a number of tutorials about class factories and metaclasses and the general consensus is to avoid metaclasses and use things like class decorators instead. The problem is the tutorials either start importing extra libraries eg: import six, mix explanations of different Python versions together, use overly complex examples or advise not to use metaclasses at all.
Please can somebody provide a very simple explanation (for Python 3) of how to use a simple class definition (like the WTForms example above) along with metaclasses to customise a brand new construction of a class without actually instantiating the class when it's constructed.
Edit: Apologies for finding it difficult to explain what my end goal is but as I have gone through tutorials it has been unclear as to whether class decorators, metaclasses, magic methods (call, new, init) or a combination of such are what I needed to achieve what I visualised, or if what I was visualising was the wrong way of doing things. Unfortunately it seems impossible to judge if my goal was wrong without being able to understand the mechanisms needed to achieve it. I've realised metaclasses are the way to go and just need pointing in the right direction for a very simple metaclass example done the Python 3.x way.
You can create classes dynamically - with no custom metaclasses and no decorators with what looks to the programmer as simple function call.
Just make a call to Python's builtin type with three parameters:
the name of the class, a tuple with its bases, and a mapping object with its namespace (i.e. a dictionary containing the attributes and methods you would ordinarily define on the class body).
def __init__(self):
...
namespace = {
'__init__': init,
'name': 'default name'
}
MyClass = type("MyClass", (object,), namespace)
You loose some features that are only possible due to the compiler doing a couple special things during building functions declared within a class body - mostly the ability to use paramterless super and name mangling of attributes starting with __, but that is it.
That said it should be noted this is not with "no metaclasses". "type" is itself a metaclass - the default Python metaclass for all objects - and is calling a metaclass that create a class. There is no other way to create a class. A "class decorator" is just a method that can makes changes to a class object after it is created.
Any function or method that yields a new, dynamic class, will have inside it to, at some point, call type or other metaclass. In the same mood, a "metaclass" does not create dynamic classes by itself - it needs do be either used in a class body declaration, or called with (at least) the same parameters used for calling type.
As for the recommendations for "class decorators" instead of metaclasses, I am not sure is that true (beyond the fact there is no way a "class decorator" can create classes dynamically by itself): their main drawback is that there is no ordinary way for subclasses of decorated classes to have the parent's class decorators applied to themselves automatically, while metaclasses are inherited.
In Python 3.6 you have the __init_subclass__ protocol which, yes, can avoid a lot of the traditional uses for a metaclass (but still, it won't "create classes dynamically" - calling type does that).
Related
Basically, I know that abstract base classes are used as skeleton classes just like regular classes, and there main advantage would be to enforce their implementation on the child classes.
But I was wondering if I have the next case:
I have a class which is having only static methods / no init -> it would make sense to make it abstract? It would be pythonic?
I was thinking the advantage would be that some one reading the code would know that that class should not be instantiated...
It seems that you're trying to emulate namespaces. It's better to use modules. The mechanism is built into Python, and functions as a namespace:
https://docs.python.org/3/tutorial/modules.html
An abstract class with only static-methods can work as a namespace, but it's confusing to people reading the source code
I have a class which is having only static methods / no init -> it
would make sense to make it abstract? It would be pythonic?
PEP 3119 gives following rationale for Abstract Base Class
This PEP proposes a particular strategy for organizing these tests
known as Abstract Base Classes, or ABC. ABCs are simply Python classes
that are added into an object’s inheritance tree to signal certain
features of that object to an external inspector. Tests are done using
isinstance(), and the presence of a particular ABC means that the test
has passed.
Taking this in account I would find it confusing to find Abstract Base Class which is then not inherited at all. If all methods are static why do not simply make all of them just functions?
I know the inheritance in class. Every methods which a superclass has is inherited to its subclass. So unless needed particularly, the subclass doesn't have to implement the inherited methods again.
But I want to make sure a subclass to re-implement all the methods which a superclass has. The point is that all classes in the same group(such as classes inherit the same superclass) have to implement the same methods individually. The classes need some structure that designate what methods have to be implemented.
What am I supposed to do?
In object oriented programming, this is achieved through the use of an interface. You'd have to import the python interface module, then have your class inherit from this module. Here is a guide with some practical examples: https://www.geeksforgeeks.org/python-interface-module/#:~:text=In%20python%2C%20interface%20is%20defined%20using%20python%20class,interfaces%20are%20implemented%20using%20implementer%20decorator%20on%20class.
In C++ you would be looking to make pure virtual methods on your base class, prompting a compile-time error on a subclass not implementing the method, this is typically the way interfaces are defined.
Python provides the abstractmethod decorator (see here:
https://docs.python.org/3/library/abc.html#abc.abstractmethod) for similar ends. Unlike C++ you will be able to run your program with subclasses which haven't implemented the base class abstractmethod, but the subclass will throw a TypeError exception at instantiation.
There's an example here: https://stackoverflow.com/a/26458670/5932855
Use abstract base classes in Python. Create a class definition that extends (subclasses) from ABC. This is an abstract class, similar to interfaces in other languages such as Java for example, which additionally can implement concrete methods if so desired. To create a stub method, mark it as #abstractmethod (or a similar decorator for class and static methods) and add a comment block; no method definition is necessary. To create a concrete method, simply add logic under the method as you normally would to any class. Then you can have your concrete subclasses extend from this abstract class, and the interpreter will by default require you to implement all declared abstract methods.
Functions in a python class can be either instance methods, class methods or static methods.
The former is characterised by the self as its first (implicit) argument, acts directly on the instance of the class, and does not require any decorators to be treated as such.
The other two, however, need decorators #classmethod and #staticmethod before the name of the method - this is why I refer to the instance method as the "default" one, i.e. the one for which a wrapper is not needed.
My question is: suppose I am in a class, and I am breaking up my calculation into several functions for readibility. Only one of these methods will need access to the self.something variables that I share instance-wise, but most of the others do not need to know about the class they belong to - they are just there for "housekeeping".
Should make these functions (the ones that do not need any self.something knowledge) all #staticmethod? Doing so would require a decorator and hence an extra step. It would be easier (not requiring the extra step of using a decotrator) for every method to just be an instance method, thus inheritig a lot of potential but also waisting it since it is not needed for the scope of the functions in question.
Why is the instance method the "default"? Why not have every method a static method by default, and give it the extra functionality associated with being a instance method with a wrapper?
The reason to default to instance methods is because that's usually what you want when you're doing object oriented programming. I can't think of a single language that claims to support OOP and has methods default to anything but instance methods. Classes are templates for "data with behaviors", so the default is to make methods that provide behaviors to each instantiation of the class. If you just want a collection of functions, you can just define them at the top level of a module and save the unnecessary class after all.
In general, #staticmethod is used to mean "I know this isn't a behavior of the class or its instances, but it helps implement the real behaviors and isn't very useful outside the class, so I'll namespace it inside it." If the features are useful outside the class, you'd just make it a plain top-level function rather putting it inside the class at all. It is advantageous to use #staticmethod where appropriate; it's a little faster to call than an instance method, so if you don't need the instance, #staticmethod will speed up your code a bit (note: This may not be true in 3.7+, where they added an optimization to avoid the creation of bound methods, which may speed up instance/class methods).
#classmethod basically has two use cases:
(Primary) Defining alternate constructors in a subclass friendly way (the cls it receives is the actual subclass, if applicable, not just the class it was defined in)
(Mostly unnecessary) As an alternative to #staticmethod when the method needs to call other static methods and you'd rather not have to refer to the class by name over and over
Point is, #staticmethod is mostly for when you're opting out of OOP, and #classmethods are for niche use cases; instance methods are just more useful, so they're the default. Beyond that, as a historical note, static and class methods were introduced later, so making them the default would have broken all existing Python code, for no real benefit.
The main reason to use #staticmethod over instance methods with an ignored self (when self isn't needed) is that it will continue to work when called on the class itself, not just on instances of the class; if you tried to call MyClass.notreallystatic(), it would die for lack of a self, while MyClass.actuallystatic() would work.
My IDE keeps suggesting I convert my instance methods to static methods. I guess because I haven't referenced any self within these methods.
An example is :
class NotificationViewSet(NSViewSet):
def pre_create_processing(self, request, obj):
log.debug(" creating messages ")
# Ensure data is consistent and belongs to the sending bot.
obj['user_id'] = request.auth.owner.id
obj['bot_id'] = request.auth.id
So my question would be: do I lose anything by just ignoring the IDE suggestions, or is there more to it?
This is a matter of workflow, intentions with your design, and also a somewhat subjective decision.
First of all, you are right, your IDE suggests converting the method to a static method because the method does not use the instance. It is most likely a good idea to follow this suggestion, but you might have a few reasons to ignore it.
Possible reasons to ignore it:
The code is soon to be changed to use the instance (on the other hand, the idea of soon is subjective, so be careful)
The code is legacy and not entirely understood/known
The interface is used in a polymorphic/duck typed way (e.g. you have a collection of objects with this method and you want to call them in a uniform way, but the implementation in this class happens to not need to use the instance - which is a bit of a code smell)
The interface is specified externally and cannot be changed (this is analog to the previous reason)
The AST of the code is read/manipulated either by itself or something that uses it and expects this method to be an instance method (this again is an external dependency on the interface)
I'm sure there can be more, but failing these types of reasons I would follow the suggestion. However, if the method does not belong to the class (e.g. factory method or something similar), I would refactor it to not be part of the class.
I think that you might be mixing up some terminology - the example is not a class method. Class methods receive the class as the first argument, they do not receive the instance. In this case you have a normal instance method that is not using its instance.
If the method does not belong in the class, you can move it out of the class and make it a standard function. Otherwise, if it should be bundled as part of the class, e.g. it's a factory function, then you should probably make it a static method as this (at a minimum) serves as useful documentation to users of your class that the method is coupled to the class, but not dependent on it's state.
Making the method static also has the advantage this it can be overridden in subclasses of the class. If the method was moved outside of the class as a regular function then subclassing is not possible.
I'm learning about metaclasses in Python. I think it is a very powerful technique, and I'm looking for good uses for them. I'd like some feedback of good useful real-world examples of using metaclasses. I'm not looking for example code on how to write a metaclass (there are plenty examples of useless metaclasses out there), but real examples where you have applied the technique and it was really the appropriate solution. The rule is: no theoretical possibilities, but metaclasses at work in a real application.
I'll start with the one example I know:
Django models, for declarative programming, where the base class Model uses a metaclass to fill the model objects of useful ORM functionality from the attribute definitions.
Looking forward to your contributions.
In Python 2.6 and 3.1, the Python standard library provides an abc.ABCMeta, a meta-class for Abstract Base Classes ("ABCs"). Classes that use the meta-class can use #abstractmethod and #abstractproperty to define abstract methods and properties. The meta-class will ensure that derived classes override the abstract methods and properties.
Also, classes that implement the ABC without actually inheriting from it can register as implementing the interface, so that issubclass and isinstance will work.
For example, the collections module defines the Sequence ABC. It also calls Sequence.register(tuple) to register the built-in tuple type as a Sequence, even though tuple does not actually inherit from Sequence.
The Python implementation of Protocol Buffers uses metaclasses to generate the Python bindings that represent your data format. From the tutorial:
The important line in each class is __metaclass__ = reflection.GeneratedProtocolMessageType. While the details of how Python metaclasses work is beyond the scope of this tutorial, you can think of them as like a template for creating classes. At load time, the GeneratedProtocolMessageType metaclass uses the specified descriptors to create all the Python methods you need to work with each message type and adds them to the relevant classes. You can then use the fully-populated classes in your code.
FormEncode validators and Turbogears / Tosca widgets.
You might also be interested in class decorators: they can be written with the latest releases, and cover many use cases that were previously handled with metaclasses.
SQLalchemy also uses them for declarative database models.
Sorry my answer isn't very different from your example, but if you're looking for example code, I found declarative to be pretty readable.
The only time I used a metaclass so far was to write a deprecation warning mechanism. It was something along the following lines - syntax may be very approximative, but code will illustrate my point more easily than a complicated sentence :
class New(object):
pass
class Old(object):
def __new__(self):
deprecation_warning("Old class is no more supported, use New class instead")
return New()