I want to implement a singleton pattern in python, and I liked the pattern described in the http://www.python-course.eu/python3_metaclasses.php.
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class SingletonClass(metaclass=Singleton):
pass
class RegularClass():
pass
x = SingletonClass()
y = SingletonClass()
print(x == y)
x = RegularClass()
y = RegularClass()
print(x == y)
And the code works perfect. But, the __call__() does not have the self, and it also does not have #classmethod or #staticmethod declaration.
But, in the Python data model https://docs.python.org/3/reference/datamodel.html#object.__call__ the __call__() method has a self in the arguments.
The code does not work if I pass self, or declare as #staticmethod or #classmethod.
Can someone please explain the logic of the syntax behind the __call__() method.
Naming the first argument of a method cls or self are just a convention. The __call__ method does have a self argument, only it is named cls here. That's because for a metaclass, the method is bound to a class object, and the name reflects this.
The same convention is applied to #classmethod methods; the first argument is a class, always, due to the nature of how a classmethod object is bound, so it makes sense to name that first argument cls.
But you are free to name that first argument anything else. It is not the name that makes a classmethod or a regular method or a method on a metatype work. All that using self or cls does is document what type of object this is, making it easier for other developers to mentally track what is going on.
So no, this is not an implicit class method. That first argument is not bound to the Singleton metaclass object, it is bound to the class that was called. That makes sense, because that class object is an instance of the Singleton metatype.
If you want to dive into how binding works (the process that causes that first argument to be passed in, whatever the name), you can read up on the Descriptor HOWTO. TLDR: functions, property, classmethod and staticmethod objects are all descriptors, and whenever you access them as an attribute on a supporting object such as an instance or a class, they are bound, often causing a different object to be returned as a result, which when called passes in the bound object to the actual function.
Martin's answer says it all. I am adding this so maybe a different wording can throw in more light for different people:
Python call() is this an implicit classmethod?
No. But all "metaclass" methods are implicit "class methods" for the classes that use that metaclass.
That is implicit when we take in account the fact that classes are simply instances of the metaclass. From the language point of view, a class behave almost exactly like any other instance - and any interactions with a class that would trigger dunder (__magic__) methods in an object - like using the "+, -, *, /" operators, or index retrieval with [ ], or calling them with ( ) trigger the corresponding methods on its class. That is ordinarily type.
And, as put in the other answer, Python does not care what name you ut on the first argument of a method. For metaclasses it makes sense to use cls there, since the "instances" the methods are dealing with are metaclasses. As it makes sense that the first argument to a metaclass' __new__ method be named metacls (like in the example bellow). Because the new "cls" is the object we get after calling type.__new__ - the only call possible in pure Python that will actually create a class object.
class Meta(type):
def __new__(metacls, name, bases, namespace):
...
cls = super().__new__(metacls, name, bases, namespace)
...
return cls
(Now, still on the topic of the question: __new__ is a special case
of an implicit static method (even for ordinary classes that are not intended to be metaclasses) - to which Python specially add the first argument, using a different mechanism than what is done to regular classmethods. Thats is why the super().__new__ call above needs to include the metacls as the first parameter)
Related
Ok, obviously __new__ in a metaclass runs when an instance of the metaclass i.e. a class object is instantiated, so __new__ in a metaclass provides a hook to intercept events (/code that runs) at class definition time (e.g. validating/enforcing rules for class attributes such as methods etc.).
Many online examples of __new__ in a metaclass return an instance of the type constructor from __new__, which seems a bit problematic since this blocks __init__ (docs: "If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked").
While tinkering with return values of __new__ in a metaclass I came across some somewhat strange cases which I do not fully understand, e.g.:
class Meta(type):
def __new__(self, name, bases, attrs):
print("Meta __new__ running!")
# return type(name, bases, attrs) # 1.
# return super().__new__(self, name, bases, attrs) # 2.
# return super().__new__(name, bases, attrs) # 3.
# return super().__new__(type, name, bases, attrs) # 4.
# return self(name, bases, attrs) # 5.
def __init__(self, *args, **kwargs):
print("Meta __init__ running!")
return super().__init__(*args, **kwargs)
class Cls(metaclass=Meta):
pass
This is often seen in examples and generally works, but blocks __init__
This works and __init__ also fires; but why pass self to a super() call? Shouldn't self/cls get passed automatically with super()?
This throws a somewhat strange error I can't really make sense of: TypeError: type.__new__(X): X is not a type object (str); what is X? shouldn't self be auto-passed?
The error of 3. inspired me to play with the first arg of the super() call, so I tried to pass type directly; this also blocks __init__. What is happening here?
tried just for fun; this leads to a RecursionError
Also especially cases 1. and 2. appear to have quite profound implications for inheriting from classes bound to metaclasses:
class LowerCaseEnforcer(type):
""" Allows only lower case names as class attributes! """
def __new__(self, name, bases, attrs):
for name in attrs:
if name.lower() != name:
raise TypeError(f"Inappropriate method name: {name}")
# return type(name, bases, attrs) # 1.
# return super().__new__(self, name, bases, attrs) # 2.
class Super(metaclass=LowerCaseEnforcer):
pass
class Sub(Super):
def some_method(self):
pass
## this will error in case 2 but not case 1
def Another_method(self):
pass
expected behavior: metaclass is bound to superclass, but not to subclass
binds the superclass /and/ subclasses to the metaclass; ?
I would much appreciate if someone could slowly and kindly explain what exactly is going on in the above examples! :D
It is simpler than what you got too.
As you have noted, the correct thing to do is your 2 above:
return super().__new__(self, name, bases, attrs) # 2.
Here it goes: __new__ is a special method - although in certain documentations, even part of the official documentation, it is described as being a classmethod, it is not quite so: it, as an object, behaves more like a static method - in a sense that Python does not automatically fill the first parameter when one calls MyClass.__new__() - i.e., you'd have to call MyClass.__new__(MyClass) for it to work. (I am a step back here - this info applies to all classes: metaclasses and ordinary classes).
When you call MyClass() to create a new instance, then Python will call MyClass.__new__ and insert the cls parameter as first parameter.
With metaclasses, the call to create a new instance of the metaclass is triggered by the execution of the class statement and its class body. Likewise, Python fills in the first parameter to Metaclass.__new__, passing the metaclass itself.
When you call super().__new__ from within your metaclass' __new__ you are in the same case of one calling __new__ manually: the parameter specifying which class' that __new__ should apply have to be explicitly filled.
Now, what is confusing you is that you are writting the first parameter to __new__ as self - which would be correct if it were an instance of the metaclass (i.e. an ordinary class). As it is, that parameter is a reference to the metaclass itself.
The docs does not inform an official, or recomended name for the first parameter of a metaclass __new__, but usually it is something along mcls, mcs, metaclass, metacls - to make it different from cls which is the usuall name for the first parameter of a non-metaclass __new__ method. In a metaclass, the "class" - cls is what is created by the ultimate call to type.__new__ (either hardcoded, or using super()) the return of it is the new-born class (it can be further modified in the __new__ method after the call to the superclass) - and when returned, the call to __init__ takes place normally.
So, I will just comment further the use of trying to call type(name, bases, namespace) instead of type.__new__(mcls, name, bases, namespace): the first form will just create a plain class, as if the metaclass had not been used at all - (lines in the metaclass __new__ that modify the namespace or bases, of course, have their effect. But the resulting class will have type as its metaclass, and subclasses of it won't call the metaclass at all. (For the record, it works as a "pre-class decorator" - which can act on class parameters before it is created, and it could even be an ordinary function, instead of a class with a __new__ method - the call to type is what will create the new class after all)
A simple way to check if the metaclass is "bound" to your class is to check its type with type(MyClass) or MyClass.__class__ .
I think I finally figured this out (somewhat), my initial confusion can mainly be ascribed to my failure to realize that
there is a difference between object.__new__ and type.__new__
there is a difference between returning type() and returning super().__new__ from a metaclass
Discussion of these two points should clear up my initial example as well as the seemingly enigmatic inheritance behavior.
1. The difference between object.__new__ and type.__new__
First a few words concerning __new__.
The documenation is imo pretty clear on this, but I'd still like to add and/or emphasize some things:
__new__ can be understood as a special cased static method that takes cls as first parameter and passes the remaining parameters (most often *args, **kwargs) to __init__.
_ __new__ and __init__ are invoked successively (actually by the metaclass's __call__!), whereby __init__ is only invoked if __new__ returns an instance of cls.
__new__ takes a single argument (this is about calling __new__, not defining/overloading it) i.e. cls and returns an instance of that class.
An important thing that eluded me at first is that there is a difference between object.__new__ and type.__new__.
I discovered this while I was playing with __new__'s parameters/arguments; take a look at these 'instructive errors':
class ObjNewExample:
def __new__(cls, *args, **kwargs):
# return super().__new__(cls) # correct
return super().__new__(cls, *args, **kwargs) # instructive error
def __init__(self, some_attr):
self._some_attr = some_attr
one = ObjNewExample(42)
class TypeNewExample(type):
def __new__(mcls, name, bases, attrs):
# return super().__new__(mcls, name, bases, attrs) # correct
return super().__new__(mcls) # instructive error
# class Cls(metaclass=TypeNewExample):
# pass
ObjNewexample with return super().__new__(cls, *args, **kwargs) throws something like
TypeError: object.__new__() takes exactly one argument (the type to instantiate),
while TypeNewexample with return super().__new__(mcls) throws
TypeError: type.__new__() takes exactly 3 arguments,
which shows that object.__new__ and type.__new__ are quite different methods!
Also: consider the difference between parameters and arguments with __new__:
object.__new__ takes cls, *args, **kwargs as parameters, but requires only cls as argument (*args, **kwargs get passed to __init__)
type.__new__ takes mcls, name, bases, attrs as parameters and arguments
The difference between returning type() and returning super().__new__ from a metaclass
The main problem with the example I initially posted however is the difference between returning type() and returning super().__new__ from a metaclass's __new__ (which is embarrassingly obvious now..). (See also this discussion)
returning type(name, bases, attrs) from mcls: creates an instance of type
returning super().__new__(mcls, name, bases, attrs) from mcls: creates an instance of the actual metaclass (which is derived from type),
which also explains why __init__ is inhibited in case 1 but not case 2 of the initial example! (Remember: __init__ does not get invoked if __new__ returns anything but an instance if __new__'s first parameter i.e. (m)cls)
This should be instructive:
class Meta(type):
def __new__(mcls, name, bases, attrs):
# this creates an intance of type (the default metaclass)
# This should be eql to super().__new__(type, name, base, attrs)!
obj1 = type(name, bases, attrs)
# this creates an instance of the actual custom metaclass (which is derived from type)
# i.e. it creates an instance of type.__new__'s first arg
obj2 = super().__new__(mcls, name, bases, attrs)
print(isinstance(obj1, mcls))
print(obj1.__class__)
print(isinstance(obj2, mcls))
print(obj2.__class__)
class Fun(metaclass=Meta):
pass
So quickly walking through cases 1-5 from my initial post:
1: returns a new type object, that is an instance of type, not the actual custom metaclass (derived from type), thus __init__ of the custom metaclass is inhibited; this appears to be actually equivalent to case 4!
2: as #jsbueno pointed out, this is the most likely intended ('correct') behavior: this creates an instance of the actual custom metaclass.
3: this barfs itself because type.__new__ expects an object of type type (the object to be instantiated) as first argument
4: see case 1
5: self (probably better named 'cls' or 'mcls') is Meta; calling a class in its own constructor is obviously recursive.
The above also provides an explanation for the seemingly weird inheritance behavior of the second snippet from my initial posts!
So why does Sub's definition of Another_method error in case 2 of LowerCaseEnforcer, but not case 1?
Because in case 1 Lowercaseenforcer returns an instance of type (not of LowerCaseEnforcer!), so Super is of type type (its metaclass is type, not LowerCaseEnforcer)! So while LowerCaseEnforcer.__new__ fires and enforces the lowercase restriction for Super, Super is just a vanilla class of type type and Sub is derived from it (with no special effect).
Whereas in case 2 Super's metaclass is of type LowerCaseEnforcer and so is Sub's, so LowerCaseEnforcer.__new__ is involved in the definition of Sub.
One thing that is still a bit unclear however is the behavior of static methods in super calls (see also this discussion).
E.g. why does super().__new__(cls) work? Shouldn't this be super(cls, cls).__new__(cls) (or something like that)?
But I guess this is another (interesting) topic! :D
Trying to change singleton using metaclass of Python 2 to Python 3, __new__ returns:
[ ERROR ] Error in file Importing test library 'C:\Users\TestTabs.py' failed: __class__ not set defining 'BrowserDriver' as <class 'BrowserDriver.BrowserDriver'>. Was __classcell__ propagated to type.__new__?
CODE:
class Singleton(type):
_instance = None
def __new__(cls, *args, **kwargs):
print('Newtest')
if cls._instance is None:
Singleton._instance = type.__new__(cls, *args, **kwargs)
return Singleton._instance
This one is called:
class BrowserDriver(metaclass=Singleton)
first: you should not be using a metaclass for having a singleton
Second: your "singleton" code is broken, even if it would work:
By luck it crossed the way of a new mechanism used in class creation, which requires type.__new__ to receive the "class cell" when creating a new class, and this was detected.
So, the misterious __class__ cell will exit if any method in your class uses a call to super(). Python will create a rathr magic __class__ variable that will receive a reference to the class that will be created, when the class body execution ends. At that point, the metaclass.__new__ is called. When the call to metaclass.__new__ returns, the Python runtime expects that the __class__ magic variable for that class is now "filled in" with a reference to the class itself.
This is for a working class creation - now we come to the bug in your code:
I don't know where you got this "singleton metaclass code" at all, but it is broken: (if it would work), it creates ONE SINGLE CLASS, for all classes using this metaclass - and not, as probably was desired, allow one single-instance of each class using this metaclass. (as the new class body do not have its __class__ attribute set, you get the error you described under Python 3.8)
In other words: any classes past the first one using this metaclass is simply ignored, and not used by the program at all.
The (overkill) idea of using a metaclass to create singleton-enforcing classes is, yes, to allow a single-instance of a class, but the cache for the single instance should be set in the class itself, not on the metaclass - or in an attribute in the metaclass that holds one instance for each class created, like a dictionary would. A simple class attribute of the metaclass as featured in this code just makes classes past the first be ignored.
So, to fix that using metaclasses, the cache logic should be in the metaclass __call__ method, not in its __new__ method -
This is the expressly not recommended, but working, metaclass to enforce singletons:
class SingletonEnforcingmeta(type):
def __call__(cls, *args, **kw):
# check "__dict__" entry insead of "hasattr" - allows inheritance
# and one instance per subclass
if "_instance" not in cls.__dict__:
cls._instance = super().__call__(*args, **kw)
return cls._instance
But, as I wrote above, it is overkill to have a metaclass if you just once a singleton - the instantiation mechanism in __new__ itself is enough for creating a single-instance cache.
But before doing that - on should think: is a "singleton enforcing class really necessary" ? This is Python - the flexible structure and "consenting adults" mindset of the language can have you simply create an instance of your class in the same namespace you created the class itself - and just use that single instance from that point on.
Actually, if your single-instance have the same name the class have, one can't even create a new instance by accident, as the class itself will be reachable only indirectly. That is:
nice thing to do: if you need a singleton, create a singleton, not a 'singleton-enforcing-class
class BrowserDriver(...):
# normal code for the class here
...
BrowserDriver = BrowserDriver()
That is all there is to it. All you have now is a single-instance of
the BrowserDriver class that can be used from any place in your code.
Now, if you really need a singleton-enforcing class, one that upon
trying to create any instance beyond the first will silently do not
raise this attempt as an error, and just return the first instance ever created,
then the code you need in then __new__ method of the class is like the code
you were trying to use as the metaclass´ __new__. It records the sinvgle instance in the class itself:
if really needed: singleton enforcing-class using __new__:
class SingletonBase:
def __new__(cls, *args, **kw):
if "_instance" not in cls.__dict__:
cls._instance = super().__new__(cls, *args, **kw)
return cls._instance
And then just inherit your "I must be a singleton" classes from this base.
Note however, that __init__ will be called on the single-instance at each instantiation attempt - so, these singletons should use __new__ (and call super() as appropriate, instead of having an __init__ method, or have an idempotent __init__ (i.e. it can be called more than once, but this extra call have no effects)
In Python, I can create a class method using the #classmethod decorator:
>>> class C:
... #classmethod
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> C.f()
f called with cls=<class '__main__.C'>
Alternatively, I can use a normal (instance) method on a metaclass:
>>> class M(type):
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> class C(metaclass=M):
... pass
...
>>> C.f()
f called with cls=<class '__main__.C'>
As shown by the output of C.f(), these two approaches provide similar functionality.
What are the differences between using #classmethod and using a normal method on a metaclass?
As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the abc.ABCMeta.register method.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict).
Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
#classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in #classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a #property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.
When you phrase it like you did in the question, the #classmethod and metaclasses may look similar but they have rather different purposes. The class that is injected in the #classmethod's argument is usually used for constructing an instance (i.e. an alternative constructor). On the other hand, the metaclasses are usually used to modify the class itself (e.g. like what Django does with its models DSL).
That is not to say that you can't modify the class inside a classmethod. But then the question becomes why didn't you define the class in the way you want to modify it in the first place? If not, it might suggest a refactor to use multiple classes.
Let's expand the first example a bit.
class C:
#classmethod
def f(cls):
print(f'f called with cls={cls}')
Borrowing from the Python docs, the above will expand to something like the following:
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
class C:
def f(cls):
print(f'f called with cls={cls}')
f = ClassMethod(f)
Note how __get__ can take either an instance or the class (or both), and thus you can do both C.f and C().f. This is unlike the metaclass example you give which will throw an AttributeError for C().f.
Moreover, in the metaclass example, f does not exist in C.__dict__. When looking up the attribute f with C.f, the interpreter looks at C.__dict__ and then after failing to find, looks at type(C).__dict__ (which is M.__dict__). This may matter if you want the flexibility to override f in C, although I doubt this will ever be of practical use.
In your example, the difference would be in some other classes that will have M set as their metaclass.
class M(type):
def f(cls):
pass
class C(metaclass=M):
pass
class C2(metaclass=M):
pass
C.f()
C2.f()
class M(type):
pass
class C(metaclass=M):
#classmethod
def f(cls):
pass
class C2(metaclass=M):
pass
C.f()
# C2 does not have 'f'
Here is more on metaclasses
What are some (concrete) use-cases for metaclasses?
Both #classmethod and Metaclass are different.
Everything in python is an object. Every thing means every thing.
What is Metaclass ?
As said every thing is an object. Classes are also objects in fact classes are instances of other mysterious objects formally called as meta-classes. Default metaclass in python is "type" if not specified
By default all classes defined are instances of type.
Classes are instances of Meta-Classes
Few important points are to understand metioned behaviour
As classes are instances of meta classes.
Like every instantiated object, like objects(instances) get their attributes from class. Class will get it's attributes from Meta-Class
Consider Following Code
class Meta(type):
def foo(self):
print(f'foo is called self={self}')
print('{} is instance of {}: {}'.format(self, Meta, isinstance(self, Meta)))
class C(metaclass=Meta):
pass
C.foo()
Where,
class C is instance of class Meta
"class C" is class object which is instance of "class Meta"
Like any other object(instance) "class C" has access it's attributes/methods defined in it's class "class Meta"
So, decoding "C.foo()" . "C" is instance of "Meta" and "foo" is method calling through instance of "Meta" which is "C".
First argument of method "foo" is reference to instance not class unlike "classmethod"
We can verify as if "class C" is instance of "Class Meta
isinstance(C, Meta)
What is classmethod?
Python methods are said to be bound. As python imposes the restriction that method has to be invoked with instance only.
Sometimes we might want to invoke methods directly through class without any instance (much like static members in java) with out having to create any instance.By default instance is required to call method. As a workaround python provides built-in function classmethod to bind given method to class instead of instance.
As class methods are bound to class. It takes at least one argument which is reference to class itself instead of instance (self)
if built-in function/decorator classmethod is used. First argument
will be reference to class instead of instance
class ClassMethodDemo:
#classmethod
def foo(cls):
print(f'cls is ClassMethodDemo: {cls is ClassMethodDemo}')
As we have used "classmethod" we call method "foo" without creating any instance as follows
ClassMethodDemo.foo()
Above method call will return True. Since first argument cls is indeed reference to "ClassMethodDemo"
Summary:
Classmethod's receive first argument which is "a reference to class(traditionally referred as cls) itself"
Methods of meta-classes are not classmethods. Methods of Meta-classes receive first argument which is "a reference to instance(traditionally referred as self) not class"
TL;DR -
I have a class that uses a metaclass.
I would like to access the parameters of the object's constructor from the metaclass, just before the initialization process, but I couldn't find a way to access those parameters.
How can I access the constructor's parameters from the metaclass function __new__?
In order to practice the use of metaclasses in python, I would like to create a class that would be used as the supercomputer "Deep Thought" from the book "The Hitchhiker's Guide to the Galaxy".
The purpose of my class would be to store the various queries the supercomputer gets from users.
At the bottom line, it would just get some arguments and store them.
If one of the given arguments is number 42 or the string "The answer to life, the universe, and everything", I don't want to create a new object but rather return a pointer to an existing object.
The idea behind this is that those objects would be the exact same so when using the is operator to compare those two, the result would be true.
In order to be able to use the is operator and get True as an answer, I would need to make sure those variables point to the same object. So, in order to return a pointer to an existing object, I need to intervene in the middle of the initialization process of the object. I cannot check the given arguments at the constructor itself and modify the object's inner-variables accordingly because it would be too late: If I check the given parameters only as part of the __init__ function, those two objects would be allocated on different portions of the memory (they might be equal but won't return True when using the is operator).
I thought of doing something like that:
class SuperComputer(type):
answer = 42
def __new__(meta, name, bases, attributes):
# Check if args contains the number "42"
# or has the string "The answer to life, the universe, and everything"
# If so, just return a pointer to an existing object:
return SuperComputer.answer
# Else, just create the object as it is:
return super(SuperComputer, meta).__new__(meta, name, bases, attributes)
class Query(object):
__metaclass__ = SuperComputer
def __init__(self, *args, **kwargs):
self.args = args
for key, value in kwargs.items():
setattr(self, key, value)
def main():
number = Query(42)
string = Query("The answer to life, the universe, and everything")
other = Query("Sunny", "Sunday", 123)
num2 = Query(45)
print number is string # Should print True
print other is string # Should print False
print number is num2 # Should print False
if __name__ == '__main__':
main()
But I'm stuck on getting the parameters from the constructor.
I saw that the __new__ method gets only four arguments:
The metaclass instance itself, the name of the class, its bases, and its attributes.
How can I send the parameters from the constructor to the metaclass?
What can I do in order to achieve my goal?
You don't need a metaclass for that.
The fact is __init__ is not the "constructor" of an object in Python, rather, it is commonly called an "initializator" . The __new__ is closer to the role of a "constructor" in other languages, and it is not available only for the metaclass - all classes have a __new__ method. If it is not explicitly implemented, the object.__new__ is called directly.
And actually, it is object.__new__ which creates a new object in Python. From pure Python code, there is no other possible way to create an object: it will always go through there. That means that if you implement the __new__ method on your own class, you have the option of not creating a new instance, and instead return another pre-existing instance of the same class (or any other object).
You only have to keep in mind that: if __new__ returns an instance of the same class, then the default behavior is that __init__ is called on the same instance. Otherwise, __init__ is not called.
It is also worth noting that in recent years some recipe for creating "singletons" in Python using metaclasses became popular - it is actually an overkill approach,a s overriding __new__ is also preferable for creating singletons.
In your case, you just need to have a dictionary with the parameters you want to track as your keys, and check if you create a new instance or "recycle" one whenever __new__ runs. The dictionary may be a class attribute, or a global variable at module level - that is your pick:
class Recycler:
_instances = {}
def __new__(cls, parameter1, ...):
if parameter1 in cls._instances:
return cls._instances[parameter1]
self = super().__new__(cls) # don't pass remaining parameters to object.__new__
_instances[parameter1] = self
return self
If you'd have any code in __init__ besides that, move it to __new__ as well.
You can have a baseclass with this behavior and have a class hierarchy without needing to re-implement __new__ for every class.
As for a metaclass, none of its methods are called when actually creating a new instance of the classes created with that metaclass. It would only be of use to automatically insert this behavior, by decorating or creating a fresh __new__ method, on classes created with that metaclass. Since this behavior is easier to track, maintain, and overall to combine with other classes just using ordinary inheritance, no need for a metaclass at all.
I'm reading through http://blog.thedigitalcatonline.com/blog/2014/09/01/python-3-oop-part-5-metaclasses/#.VwPSjDG1XGD. In it, they have:
class Singleton(type):
instance = None
def __call__(cls, *args, **kw):
if not cls.instance:
cls.instance = super(Singleton, cls).__call__(*args, **kw)
return cls.instance
The explanation is:
We are defining a new type, which inherits from type to provide all bells and whistles of Python classes. We override the call method, that is a special method invoked when we call the class, i.e. when we instance it. The new method wraps the original method of type by calling it only when the instance attribute is not set, i.e. the first time the class is instanced, otherwise it just returns the recorded instance. As you can see this is a very basic cache class, the only trick is that it is applied to the creation of instances.
I'm not sure I understand the line:
cls.instance = super(Singleton, cls).__call__(*args, **kw)
Can someone explain what is happening here in another way?
By default, __call__ing on a class object produces an instance of such class (remember that classes are callables like functions, in contrast to other languages like PHP where they are completely separate monsters in their interface). This instance will be stored in cls.instance.
cls.instance = super(Singleton, cls).__call__(*args, **kw)
By wrapping in the previous condition, if the instance is already an... instance, it is returned. This means: stuff like __new__ is not called on the class again, and stuff like __init__ is not called on the instance again, but just returned the old -already existent- instance.
Notes: When you call __call__ in the default implementation, the __new__ method is called for the class (it is always a class method; you never use the #classmethod decorator there) to create the instance. After that, the __init__ message is sent to the instance to initialize it.
Notes on Metaclasses: Remember that Singleton is not an usual class, but a Metaclass. It is not a class you will inherit from, but a Metaclass you will instantiate your class with.
Objects are instances of Classes, and Classes are instances of Metaclasses (which may confuse you since they are also classes). This means that there's a closed loop in which the reference type is an instance of itself:
assert isinstance(type, type)
There, your code will come like this:
class MySingletonClass(object):
__metaclass__ = Singleton
And the magic will begin:
The __call__ method you are overriding with your code will be executed on the class since it is defined on the metaclass (Singleton). This means:
i = MySingletonClass() # Executes what is defined in Singleton as __call__
You should not confuse with this one:
i() # Will crash if MySingletonClass does not define __call__ for its instances.
The call you are making in your code has another equivalent:
super(Singleton, cls).__call__(*a, **kwa)
Is the same as:
(a type isntance)(*a, **kwa)
Or
(a type isntance).__call__(*a, **kwa)
Which is the same behavior every class, not using a custom metaclass like yours, uses to create their instances.