Develop Reference

How a metaclass singleton works

It's not clear to me how the typical metaclass singleton implementation works. I would expect the starred print to execute twice; it only happens once:
class _Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
print('Within __call__', cls, cls._instances, *args, **kwargs)
if cls not in cls._instances:
print('**About to call __call__', super(_Singleton, cls).__call__, flush=True)
print("Is cls the '...of object'?", hex(id(cls)).upper())
cls._instances[cls] = super(_Singleton, cls).__call__(*args, **kwargs)
print(cls._instances[cls])
return cls._instances[cls]
class MySingleton(metaclass=_Singleton):
pass
if __name__ == '__main__':
print('Making mysingleton')
mysingleton = MySingleton()
print("Is mysingleton the 'cls'?", mysingleton)
print('Making another')
another = MySingleton()
# Verify singletude
assert another == mysingleton
This prints
Making mysingleton
Within __call__ <class '__main__.MySingleton'> {}
**About to call __call__ <method-wrapper '__call__' of _Singleton object at 0x000001C950C28780>
Is cls the '...of object'? 0X1C950C28780
<__main__.MySingleton object at 0x000001C9513FCA30>
Is mysingleton the 'cls'? <__main__.MySingleton object at 0x000001C9513FCA30>
Making another
Within __call__ <class '__main__.MySingleton'> {<class '__main__.MySingleton'>: <__main__.MySingleton object at 0x000001C9513FCA30>}
As is my experience with the Python docs, they're terribly circular and confusing. The docs say that __call__() is called when an instance is "called". Fair enough; MySingleton.__call__ is run because the "()" on mysingleton = MySingleton() indicates a function call. MySingleton is of the _Singleton type, so it has an _instances dict. The dict is naturally empty on first call. The conditional fails and Super(_Singleton, cls).__call__ executes.
What does super() do here? It's barely intelligible from the docs. It returns a "proxy object", explained elsewhere as "an object which 'refers' to a shared object", that delegates method calls to a parent or sibling class of 'type'. Okay, fine; it will be used to call a method of some related _Singleton type.
The two argument form of super(), which is used here, "specifies the arguments exactly and makes the appropriate references". What references are those? The type is _Singleton and object is cls, which isn't mysingleton. It's whatever object 0x000001C950C28780 is. Anyway, the super() search order is that of getattr() or super(). I think that means references are looked up according to _Singleton.__mro__ since __call__ isn't an attribute (or is it?). That is, the super() call looks up according to super(), which I assume is _Singleton. Clear as mud. The __mro__ yields (<class '__main__._Singleton'>, <class 'type'>, <class 'object'>). So, super(_Singleton, cls) will look for the "related _Singleton type" and call its __call__ method; I assume that's cls.__call__().
Since cls is a _Singleton, I would expect to see the second print. Actually, I would expect some kind of recursion. Neither happen. What's going on in there?

The super builtin is not the most simple thing in Python syntax. It is used when a method has been overriden in a hierarchy of classes and allows to specify indirectly which version (the method defined in which ancestor class) will actually be called.
Here, _Singleton is a subclass of type. Fair enough. But as the __call__ method of _Singleton has been overriden, it will have to call the same method in its parent class to actually build an object. That is the purpose of super(_Singleton, cls).__call__(*args, **kwargs): it will forward the call to the parent of _Singleton. So it is the same as:
type.__call__(cls, *args, **kwargs)
That is: it call the __call__ method of type but still uses cls as a self object allowing the creation a MySingleton object and bypassing a recursive call to _Singleton.__call__. Alternatives would be to use S.__new__(S, *args, **kwargs) or directly object.__new__(S), but that last one would bypass any possible object initialization.
In fact, super is the Pythonic way here, because if you later build a more complex hierarchy of metaclasses (_Singleton <- _GenericMeta <- type), super(_Singleton, cls) will ensure to use the class immediately preceding _Singleton in the hierachy.

Behavior of __new__ in a metaclass (also in context of inheritance)

Ok, obviously __new__ in a metaclass runs when an instance of the metaclass i.e. a class object is instantiated, so __new__ in a metaclass provides a hook to intercept events (/code that runs) at class definition time (e.g. validating/enforcing rules for class attributes such as methods etc.).
Many online examples of __new__ in a metaclass return an instance of the type constructor from __new__, which seems a bit problematic since this blocks __init__ (docs: "If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked").
While tinkering with return values of __new__ in a metaclass I came across some somewhat strange cases which I do not fully understand, e.g.:
class Meta(type):
def __new__(self, name, bases, attrs):
print("Meta __new__ running!")
# return type(name, bases, attrs) # 1.
# return super().__new__(self, name, bases, attrs) # 2.
# return super().__new__(name, bases, attrs) # 3.
# return super().__new__(type, name, bases, attrs) # 4.
# return self(name, bases, attrs) # 5.
def __init__(self, *args, **kwargs):
print("Meta __init__ running!")
return super().__init__(*args, **kwargs)
class Cls(metaclass=Meta):
pass
This is often seen in examples and generally works, but blocks __init__
This works and __init__ also fires; but why pass self to a super() call? Shouldn't self/cls get passed automatically with super()?
This throws a somewhat strange error I can't really make sense of: TypeError: type.__new__(X): X is not a type object (str); what is X? shouldn't self be auto-passed?
The error of 3. inspired me to play with the first arg of the super() call, so I tried to pass type directly; this also blocks __init__. What is happening here?
tried just for fun; this leads to a RecursionError
Also especially cases 1. and 2. appear to have quite profound implications for inheriting from classes bound to metaclasses:
class LowerCaseEnforcer(type):
""" Allows only lower case names as class attributes! """
def __new__(self, name, bases, attrs):
for name in attrs:
if name.lower() != name:
raise TypeError(f"Inappropriate method name: {name}")
# return type(name, bases, attrs) # 1.
# return super().__new__(self, name, bases, attrs) # 2.
class Super(metaclass=LowerCaseEnforcer):
pass
class Sub(Super):
def some_method(self):
pass
## this will error in case 2 but not case 1
def Another_method(self):
pass
expected behavior: metaclass is bound to superclass, but not to subclass
binds the superclass /and/ subclasses to the metaclass; ?
I would much appreciate if someone could slowly and kindly explain what exactly is going on in the above examples! :D

It is simpler than what you got too.
As you have noted, the correct thing to do is your 2 above:
return super().__new__(self, name, bases, attrs) # 2.
Here it goes: __new__ is a special method - although in certain documentations, even part of the official documentation, it is described as being a classmethod, it is not quite so: it, as an object, behaves more like a static method - in a sense that Python does not automatically fill the first parameter when one calls MyClass.__new__() - i.e., you'd have to call MyClass.__new__(MyClass) for it to work. (I am a step back here - this info applies to all classes: metaclasses and ordinary classes).
When you call MyClass() to create a new instance, then Python will call MyClass.__new__ and insert the cls parameter as first parameter.
With metaclasses, the call to create a new instance of the metaclass is triggered by the execution of the class statement and its class body. Likewise, Python fills in the first parameter to Metaclass.__new__, passing the metaclass itself.
When you call super().__new__ from within your metaclass' __new__ you are in the same case of one calling __new__ manually: the parameter specifying which class' that __new__ should apply have to be explicitly filled.
Now, what is confusing you is that you are writting the first parameter to __new__ as self - which would be correct if it were an instance of the metaclass (i.e. an ordinary class). As it is, that parameter is a reference to the metaclass itself.
The docs does not inform an official, or recomended name for the first parameter of a metaclass __new__, but usually it is something along mcls, mcs, metaclass, metacls - to make it different from cls which is the usuall name for the first parameter of a non-metaclass __new__ method. In a metaclass, the "class" - cls is what is created by the ultimate call to type.__new__ (either hardcoded, or using super()) the return of it is the new-born class (it can be further modified in the __new__ method after the call to the superclass) - and when returned, the call to __init__ takes place normally.
So, I will just comment further the use of trying to call type(name, bases, namespace) instead of type.__new__(mcls, name, bases, namespace): the first form will just create a plain class, as if the metaclass had not been used at all - (lines in the metaclass __new__ that modify the namespace or bases, of course, have their effect. But the resulting class will have type as its metaclass, and subclasses of it won't call the metaclass at all. (For the record, it works as a "pre-class decorator" - which can act on class parameters before it is created, and it could even be an ordinary function, instead of a class with a __new__ method - the call to type is what will create the new class after all)
A simple way to check if the metaclass is "bound" to your class is to check its type with type(MyClass) or MyClass.__class__ .

I think I finally figured this out (somewhat), my initial confusion can mainly be ascribed to my failure to realize that
there is a difference between object.__new__ and type.__new__
there is a difference between returning type() and returning super().__new__ from a metaclass
Discussion of these two points should clear up my initial example as well as the seemingly enigmatic inheritance behavior.
1. The difference between object.__new__ and type.__new__
First a few words concerning __new__.
The documenation is imo pretty clear on this, but I'd still like to add and/or emphasize some things:
__new__ can be understood as a special cased static method that takes cls as first parameter and passes the remaining parameters (most often *args, **kwargs) to __init__.
_ __new__ and __init__ are invoked successively (actually by the metaclass's __call__!), whereby __init__ is only invoked if __new__ returns an instance of cls.
__new__ takes a single argument (this is about calling __new__, not defining/overloading it) i.e. cls and returns an instance of that class.
An important thing that eluded me at first is that there is a difference between object.__new__ and type.__new__.
I discovered this while I was playing with __new__'s parameters/arguments; take a look at these 'instructive errors':
class ObjNewExample:
def __new__(cls, *args, **kwargs):
# return super().__new__(cls) # correct
return super().__new__(cls, *args, **kwargs) # instructive error
def __init__(self, some_attr):
self._some_attr = some_attr
one = ObjNewExample(42)
class TypeNewExample(type):
def __new__(mcls, name, bases, attrs):
# return super().__new__(mcls, name, bases, attrs) # correct
return super().__new__(mcls) # instructive error
# class Cls(metaclass=TypeNewExample):
# pass
ObjNewexample with return super().__new__(cls, *args, **kwargs) throws something like
TypeError: object.__new__() takes exactly one argument (the type to instantiate),
while TypeNewexample with return super().__new__(mcls) throws
TypeError: type.__new__() takes exactly 3 arguments,
which shows that object.__new__ and type.__new__ are quite different methods!
Also: consider the difference between parameters and arguments with __new__:
object.__new__ takes cls, *args, **kwargs as parameters, but requires only cls as argument (*args, **kwargs get passed to __init__)
type.__new__ takes mcls, name, bases, attrs as parameters and arguments
The difference between returning type() and returning super().__new__ from a metaclass
The main problem with the example I initially posted however is the difference between returning type() and returning super().__new__ from a metaclass's __new__ (which is embarrassingly obvious now..). (See also this discussion)
returning type(name, bases, attrs) from mcls: creates an instance of type
returning super().__new__(mcls, name, bases, attrs) from mcls: creates an instance of the actual metaclass (which is derived from type),
which also explains why __init__ is inhibited in case 1 but not case 2 of the initial example! (Remember: __init__ does not get invoked if __new__ returns anything but an instance if __new__'s first parameter i.e. (m)cls)
This should be instructive:
class Meta(type):
def __new__(mcls, name, bases, attrs):
# this creates an intance of type (the default metaclass)
# This should be eql to super().__new__(type, name, base, attrs)!
obj1 = type(name, bases, attrs)
# this creates an instance of the actual custom metaclass (which is derived from type)
# i.e. it creates an instance of type.__new__'s first arg
obj2 = super().__new__(mcls, name, bases, attrs)
print(isinstance(obj1, mcls))
print(obj1.__class__)
print(isinstance(obj2, mcls))
print(obj2.__class__)
class Fun(metaclass=Meta):
pass
So quickly walking through cases 1-5 from my initial post:
1: returns a new type object, that is an instance of type, not the actual custom metaclass (derived from type), thus __init__ of the custom metaclass is inhibited; this appears to be actually equivalent to case 4!
2: as #jsbueno pointed out, this is the most likely intended ('correct') behavior: this creates an instance of the actual custom metaclass.
3: this barfs itself because type.__new__ expects an object of type type (the object to be instantiated) as first argument
4: see case 1
5: self (probably better named 'cls' or 'mcls') is Meta; calling a class in its own constructor is obviously recursive.
The above also provides an explanation for the seemingly weird inheritance behavior of the second snippet from my initial posts!
So why does Sub's definition of Another_method error in case 2 of LowerCaseEnforcer, but not case 1?
Because in case 1 Lowercaseenforcer returns an instance of type (not of LowerCaseEnforcer!), so Super is of type type (its metaclass is type, not LowerCaseEnforcer)! So while LowerCaseEnforcer.__new__ fires and enforces the lowercase restriction for Super, Super is just a vanilla class of type type and Sub is derived from it (with no special effect).
Whereas in case 2 Super's metaclass is of type LowerCaseEnforcer and so is Sub's, so LowerCaseEnforcer.__new__ is involved in the definition of Sub.
One thing that is still a bit unclear however is the behavior of static methods in super calls (see also this discussion).
E.g. why does super().__new__(cls) work? Shouldn't this be super(cls, cls).__new__(cls) (or something like that)?
But I guess this is another (interesting) topic! :D

super().__new__() for object vs type in Python 3.7

Calling super().__new__() when overriding __new__ in a metaclass contains the following function signature:
class MyMeta(type):
def __new__(cls, name, bases, classdict):
clsobj = super().__new__(cls, name, bases, classdict)
However, when overriding __new__ in a normal class, we have the following:
class A:
def __new__(cls, *a, **kw):
clsobj = super().__new__(cls)
Passing any other arguments to super() in A.__new__ will lead to the following error:
TypeError: object.__new__() takes exactly one argument (the type to instantiate)
I understand that in the second case, we are dealing with object.__new__ while in the first case we are dealing with type.__new__.
My question is, why are these function signatures different? Why does object.__new__ only accept cls?

object and type have an interesting relationship. object is an instance of type, but type is a subclass of object.
__new__, however, focuses on the creation of an instance of a class, and so object.__new__ is the lowest common denominator: it does virtually nothing on its own, because there is virtually nothing that instances of every possible class have in common. As such, object.__new__ needs no more information than the type its return value will have.
type.__new__ does quite a bit more than object.__new__: it has to create an object that itself is capable of creating new objects. This requires quite a bit more information, so type.__new__ defines several additional parameters when it overrides object.__new__.
Note, however, that type itself does not use super; if you define two classes
class A:
def __new__(cls, *args, **kwargs):
print("In A.__new__")
return super().__new__(cls)
class B(type, A):
def __new__(cls, name, bases, dct, *args, **kwargs):
print("In B.__new__")
return super().__new__(cls, name, bases, dct)
you'll see that B("B", (), {}) outputs In B.__new__ but not In A.__new__. type.__new__ is the end of the line, so to speak, for creating metaclasses.
Typically, though, you wouldn't mix classes like this. Just like you rarely include object in the list of base classes for a class (I'm almost willing to say you never would), you don't often inherit from type and another (meta)class, and I can't think of any reason why you would try to inherit from both type and a non-type subclass of object.

Arguments of __new__ and __init__ for metaclasses

I am a bit surprised by the method call order and the different arguments when overriding new and init in a metaclass. Consider the following:
class AT(type):
def __new__(mcs, name, bases, dct):
print(f"Name as received in new: {name}")
return super().__new__(mcs, name + 'HELLO', bases + (list,), dct)
def __init__(cls, name, bases, dct):
print(f"Name as received in init: {name}")
pass
class A(metaclass=AT):
pass
A.__name__
The output is:
Name as received in new: A
Name as received in init: A
'AHELLO'
In short I would have expected init to receive AHELLO with the argument name.
I imagined that __init__ was called by super().__new__: if the call is not done in the overridden __new__ then my __init__ is not called.
Could someone clarify how __init__ is called in this case?
For information my use case for this is that I wanted to make creation of classes, in a special case, easier at runtime by providing only a single "base" class (and not a tuple), I then added this code in __new__:
if not isinstance(bases, tuple):
bases = (bases, )
however, I found out that I also need to add it in __init__.

Your __init__ method is obviously called and the reason for that is because your __new__ method is returning an instance of your class.
From https://docs.python.org/3/reference/datamodel.html#object.new:
If __new__() returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__().
As you can see the arguments passed to __init__ are those passed to __new__ method's caller not when you call it using super. It's a little bit vague but that's what it means if you read it closely.
And regarding the rest it works just as expected:
In [10]: A.__bases__
Out[10]: (list,)
In [11]: a = A()
In [12]: a.__class__.__bases__
Out[12]: (list,)

The fact is that what orchestrates the call of __new__ and __init__ of an ordinary class is the __call__ method on its metaclass. The code in the __call__ method of type, the default metatype, is in C, but the equivalent of it in Python would be:
class type:
...
def __call__(cls, *args, **kw):
instance = cls.__new__(cls, *args, **kw) # __new__ is actually a static method - cls has to be passed explicitly
if isinstance(instance, cls):
instance.__init__(*args, **kw)
return instance
That takes place for most object instantiation in Python, including when instantiating classes themselves - the metaclass is implicitly called as part of a class statement. In this case, the __new__ and __init__ called from type.__call__ are the methods on the metaclass itself. And in this case, type is acting as the "metametaclass" - a concept seldom needed, but it is what creates the behavior you are exploring.
When creating classes, type.__new__ will be responsible for calling the class (not the metaclass) __init_subclass__, and its descriptors' __set_name__ methods - so, the "metametaclass" __call__ method can't control that.
So, if you want the args passed to the metaclass __init__ to be programmatically modified, the "normal" way will be to have a "metametaclass", inheriting from type and distinct from your metaclass itself, and override its __call__ method:
class MM(type):
def __call__(metacls, name, bases, namespace, **kw):
name = modify(name)
cls = metacls.__new__(metacls, name, bases, namespace, **kw)
metacls.__init__(cls, name, bases, namespace, **kw)
return cls
# or you could delegate to type.__call__, replacing the above with just
# return super().__call__(modify(name), bases, namespace, **kw)
Of course that is a way to get to go closer to "turtles all way to the bottom" than anyone would ever like in production code.
An alternative is to keep the modified name as an attribute on the metaclass, so that its __init__ method can take the needed information from there, and ignore the name passed in from its own metaclass' __call__ invocation. The information channel can be an ordinary attribute on the metaclass instance. Well - it happens that the "metaclass instance" is the class being created itself - and oh, see - that the name passed to type.__new__ already gets recorded in it - on the __name__ atribute.
In other words, all you have to do to use a class name modified in a metaclass __new__ method in its own __init__ method, is to ignore the passed in name argument, and use cls.__name__ instead:
class Meta(type):
def __new__(mcls, name, bases, namespace, **kw):
name = modified(name)
return super().__new__(mcls, name, bases, namespace, **kw)
def __init__(cls, name, bases, namespace, **kw):
name = cls.__name__ # noQA (otherwise linting tools would warn on the overriden parameter name)
...

Prevent __init__ from being called after __new__? [duplicate]

I'm just trying to streamline one of my classes and have introduced some functionality in the same style as the flyweight design pattern.
However, I'm a bit confused as to why __init__ is always called after __new__. I wasn't expecting this. Can anyone tell me why this is happening and how I can implement this functionality otherwise? (Apart from putting the implementation into the __new__ which feels quite hacky.)
Here's an example:
class A(object):
_dict = dict()
def __new__(cls):
if 'key' in A._dict:
print "EXISTS"
return A._dict['key']
else:
print "NEW"
return super(A, cls).__new__(cls)
def __init__(self):
print "INIT"
A._dict['key'] = self
print ""
a1 = A()
a2 = A()
a3 = A()
Outputs:
NEW
INIT
EXISTS
INIT
EXISTS
INIT
Why?

Use __new__ when you need to control
the creation of a new instance.
Use
__init__ when you need to control initialization of a new instance.
__new__ is the first step of instance creation. It's called first, and is
responsible for returning a new
instance of your class.
In contrast,
__init__ doesn't return anything; it's only responsible for initializing the
instance after it's been created.
In general, you shouldn't need to
override __new__ unless you're
subclassing an immutable type like
str, int, unicode or tuple.
From April 2008 post: When to use __new__ vs. __init__? on mail.python.org.
You should consider that what you are trying to do is usually done with a Factory and that's the best way to do it. Using __new__ is not a good clean solution so please consider the usage of a factory. Here's a good example: ActiveState Fᴀᴄᴛᴏʀʏ ᴘᴀᴛᴛᴇʀɴ Recipe.

__new__ is static class method, while __init__ is instance method.
__new__ has to create the instance first, so __init__ can initialize it. Note that __init__ takes self as parameter. Until you create instance there is no self.
Now, I gather, that you're trying to implement singleton pattern in Python. There are a few ways to do that.
Also, as of Python 2.6, you can use class decorators.
def singleton(cls):
instances = {}
def getinstance():
if cls not in instances:
instances[cls] = cls()
return instances[cls]
return getinstance
#singleton
class MyClass:
...

In most well-known OO languages, an expression like SomeClass(arg1, arg2) will allocate a new instance, initialise the instance's attributes, and then return it.
In most well-known OO languages, the "initialise the instance's attributes" part can be customised for each class by defining a constructor, which is basically just a block of code that operates on the new instance (using the arguments provided to the constructor expression) to set up whatever initial conditions are desired. In Python, this corresponds to the class' __init__ method.
Python's __new__ is nothing more and nothing less than similar per-class customisation of the "allocate a new instance" part. This of course allows you to do unusual things such as returning an existing instance rather than allocating a new one. So in Python, we shouldn't really think of this part as necessarily involving allocation; all that we require is that __new__ comes up with a suitable instance from somewhere.
But it's still only half of the job, and there's no way for the Python system to know that sometimes you want to run the other half of the job (__init__) afterwards and sometimes you don't. If you want that behavior, you have to say so explicitly.
Often, you can refactor so you only need __new__, or so you don't need __new__, or so that __init__ behaves differently on an already-initialised object. But if you really want to, Python does actually allow you to redefine "the job", so that SomeClass(arg1, arg2) doesn't necessarily call __new__ followed by __init__. To do this, you need to create a metaclass, and define its __call__ method.
A metaclass is just the class of a class. And a class' __call__ method controls what happens when you call instances of the class. So a metaclass' __call__ method controls what happens when you call a class; i.e. it allows you to redefine the instance-creation mechanism from start to finish. This is the level at which you can most elegantly implement a completely non-standard instance creation process such as the singleton pattern. In fact, with less than 10 lines of code you can implement a Singleton metaclass that then doesn't even require you to futz with __new__ at all, and can turn any otherwise-normal class into a singleton by simply adding __metaclass__ = Singleton!
class Singleton(type):
def __init__(self, *args, **kwargs):
super(Singleton, self).__init__(*args, **kwargs)
self.__instance = None
def __call__(self, *args, **kwargs):
if self.__instance is None:
self.__instance = super(Singleton, self).__call__(*args, **kwargs)
return self.__instance
However this is probably deeper magic than is really warranted for this situation!

To quote the documentation:
Typical implementations create a new instance of the class by invoking
the superclass's __new__() method using "super(currentclass,
cls).__new__(cls[, ...])"with appropriate arguments and then
modifying the newly-created instance as necessary before returning it.
...
If __new__() does not return an instance of cls, then the new
instance's __init__() method will not be invoked.
__new__() is intended mainly to allow subclasses of immutable
types (like int, str, or tuple) to customize instance creation.

I realize that this question is quite old but I had a similar issue.
The following did what I wanted:
class Agent(object):
_agents = dict()
def __new__(cls, *p):
number = p[0]
if not number in cls._agents:
cls._agents[number] = object.__new__(cls)
return cls._agents[number]
def __init__(self, number):
self.number = number
def __eq__(self, rhs):
return self.number == rhs.number
Agent("a") is Agent("a") == True
I used this page as a resource http://infohost.nmt.edu/tcc/help/pubs/python/web/new-new-method.html

When __new__ returns instance of the same class, __init__ is run afterwards on returned object. I.e. you can NOT use __new__ to prevent __init__ from being run. Even if you return previously created object from __new__, it will be double (triple, etc...) initialized by __init__ again and again.
Here is the generic approach to Singleton pattern which extends vartec answer above and fixes it:
def SingletonClass(cls):
class Single(cls):
__doc__ = cls.__doc__
_initialized = False
_instance = None
def __new__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(Single, cls).__new__(cls, *args, **kwargs)
return cls._instance
def __init__(self, *args, **kwargs):
if self._initialized:
return
super(Single, self).__init__(*args, **kwargs)
self.__class__._initialized = True # Its crucial to set this variable on the class!
return Single
Full story is here.
Another approach, which in fact involves __new__ is to use classmethods:
class Singleton(object):
__initialized = False
def __new__(cls, *args, **kwargs):
if not cls.__initialized:
cls.__init__(*args, **kwargs)
cls.__initialized = True
return cls
class MyClass(Singleton):
#classmethod
def __init__(cls, x, y):
print "init is here"
#classmethod
def do(cls):
print "doing stuff"
Please pay attention, that with this approach you need to decorate ALL of your methods with #classmethod, because you'll never use any real instance of MyClass.

I think the simple answer to this question is that, if __new__ returns a value that is the same type as the class, the __init__ function executes, otherwise it won't. In this case your code returns A._dict('key') which is the same class as cls, so __init__ will be executed.

class M(type):
_dict = {}
def __call__(cls, key):
if key in cls._dict:
print 'EXISTS'
return cls._dict[key]
else:
print 'NEW'
instance = super(M, cls).__call__(key)
cls._dict[key] = instance
return instance
class A(object):
__metaclass__ = M
def __init__(self, key):
print 'INIT'
self.key = key
print
a1 = A('aaa')
a2 = A('bbb')
a3 = A('aaa')
outputs:
NEW
INIT
NEW
INIT
EXISTS
NB As a side effect M._dict property automatically becomes accessible from A as A._dict so take care not to overwrite it incidentally.

An update to #AntonyHatchkins answer, you probably want a separate dictionary of instances for each class of the metatype, meaning that you should have an __init__ method in the metaclass to initialize your class object with that dictionary instead of making it global across all the classes.
class MetaQuasiSingleton(type):
def __init__(cls, name, bases, attibutes):
cls._dict = {}
def __call__(cls, key):
if key in cls._dict:
print('EXISTS')
instance = cls._dict[key]
else:
print('NEW')
instance = super().__call__(key)
cls._dict[key] = instance
return instance
class A(metaclass=MetaQuasiSingleton):
def __init__(self, key):
print 'INIT'
self.key = key
print()
I have gone ahead and updated the original code with an __init__ method and changed the syntax to Python 3 notation (no-arg call to super and metaclass in the class arguments instead of as an attribute).
Either way, the important point here is that your class initializer (__call__ method) will not execute either __new__ or __init__ if the key is found. This is much cleaner than using __new__, which requires you to mark the object if you want to skip the default __init__ step.

__new__ should return a new, blank instance of a class. __init__ is then called to initialise that instance. You're not calling __init__ in the "NEW" case of __new__, so it's being called for you. The code that is calling __new__ doesn't keep track of whether __init__ has been called on a particular instance or not nor should it, because you're doing something very unusual here.
You could add an attribute to the object in the __init__ function to indicate that it's been initialised. Check for the existence of that attribute as the first thing in __init__ and don't proceed any further if it has been.

Digging little deeper into that!
The type of a generic class in CPython is type and its base class is Object (Unless you explicitly define another base class like a metaclass). The sequence of low level calls can be found here. The first method called is the type_call which then calls tp_new and then tp_init.
The interesting part here is that tp_new will call the Object's (base class) new method object_new which does a tp_alloc (PyType_GenericAlloc) which allocates the memory for the object :)
At that point the object is created in memory and then the __init__ method gets called. If __init__ is not implemented in your class then the object_init gets called and it does nothing :)
Then type_call just returns the object which binds to your variable.

One should look at __init__ as a simple constructor in traditional OO languages. For example, if you are familiar with Java or C++, the constructor is passed a pointer to its own instance implicitly. In the case of Java, it is the this variable. If one were to inspect the byte code generated for Java, one would notice two calls. The first call is to an "new" method, and then next call is to the init method (which is the actual call to the user defined constructor). This two step process enables creation of the actual instance before calling the constructor method of the class which is just another method of that instance.
Now, in the case of Python, __new__ is a added facility that is accessible to the user. Java does not provide that flexibility, due to its typed nature. If a language provided that facility, then the implementor of __new__ could do many things in that method before returning the instance, including creating a totally new instance of a unrelated object in some cases. And, this approach also works out well for especially for immutable types in the case of Python.

However, I'm a bit confused as to why __init__ is always called after __new__.
I think the C++ analogy would be useful here:
__new__ simply allocates memory for the object. The instance variables of an object needs memory to hold it, and this is what the step __new__ would do.
__init__ initialize the internal variables of the object to specific values (could be default).

Referring to this doc:
When subclassing immutable built-in types like numbers and strings,
and occasionally in other situations, the static method __new__ comes
in handy. __new__ is the first step in instance construction, invoked
before __init__.
The __new__ method is called with the class as its
first argument; its responsibility is to return a new instance of that
class.
Compare this to __init__: __init__ is called with an instance
as its first argument, and it doesn't return anything; its
responsibility is to initialize the instance.
There are situations
where a new instance is created without calling __init__ (for example
when the instance is loaded from a pickle). There is no way to create
a new instance without calling __new__ (although in some cases you can
get away with calling a base class's __new__).
Regarding what you wish to achieve, there also in same doc info about Singleton pattern
class Singleton(object):
def __new__(cls, *args, **kwds):
it = cls.__dict__.get("__it__")
if it is not None:
return it
cls.__it__ = it = object.__new__(cls)
it.init(*args, **kwds)
return it
def init(self, *args, **kwds):
pass
you may also use this implementation from PEP 318, using a decorator
def singleton(cls):
instances = {}
def getinstance():
if cls not in instances:
instances[cls] = cls()
return instances[cls]
return getinstance
#singleton
class MyClass:
...

Now I've got the same problem, and for some reasons I decided to avoid decorators, factories and metaclasses. I did it like this:
Main file
def _alt(func):
import functools
#functools.wraps(func)
def init(self, *p, **k):
if hasattr(self, "parent_initialized"):
return
else:
self.parent_initialized = True
func(self, *p, **k)
return init
class Parent:
# Empty dictionary, shouldn't ever be filled with anything else
parent_cache = {}
def __new__(cls, n, *args, **kwargs):
# Checks if object with this ID (n) has been created
if n in cls.parent_cache:
# It was, return it
return cls.parent_cache[n]
else:
# Check if it was modified by this function
if not hasattr(cls, "parent_modified"):
# Add the attribute
cls.parent_modified = True
cls.parent_cache = {}
# Apply it
cls.__init__ = _alt(cls.__init__)
# Get the instance
obj = super().__new__(cls)
# Push it to cache
cls.parent_cache[n] = obj
# Return it
return obj
Example classes
class A(Parent):
def __init__(self, n):
print("A.__init__", n)
class B(Parent):
def __init__(self, n):
print("B.__init__", n)
In use
>>> A(1)
A.__init__ 1 # First A(1) initialized
<__main__.A object at 0x000001A73A4A2E48>
>>> A(1) # Returned previous A(1)
<__main__.A object at 0x000001A73A4A2E48>
>>> A(2)
A.__init__ 2 # First A(2) initialized
<__main__.A object at 0x000001A7395D9C88>
>>> B(2)
B.__init__ 2 # B class doesn't collide with A, thanks to separate cache
<__main__.B object at 0x000001A73951B080>
Warning: You shouldn't initialize Parent, it will collide with other classes - unless you defined separate cache in each of the children, that's not what we want.
Warning: It seems a class with Parent as grandparent behaves weird. [Unverified]
Try it online!

The __init__ is called after __new__ so that when you override it in a subclass, your added code will still get called.
If you are trying to subclass a class that already has a __new__, someone unaware of this might start by adapting the __init__ and forwarding the call down to the subclass __init__. This convention of calling __init__ after __new__ helps that work as expected.
The __init__ still needs to allow for any parameters the superclass __new__ needed, but failing to do so will usually create a clear runtime error. And the __new__ should probably explicitly allow for *args and '**kw', to make it clear that extension is OK.
It is generally bad form to have both __new__ and __init__ in the same class at the same level of inheritance, because of the behavior the original poster described.

However, I'm a bit confused as to why __init__ is always called after __new__.
Not much of a reason other than that it just is done that way. __new__ doesn't have the responsibility of initializing the class, some other method does (__call__, possibly-- I don't know for sure).
I wasn't expecting this. Can anyone tell me why this is happening and how I implement this functionality otherwise? (apart from putting the implementation into the __new__ which feels quite hacky).
You could have __init__ do nothing if it's already been initialized, or you could write a new metaclass with a new __call__ that only calls __init__ on new instances, and otherwise just returns __new__(...).

The simple reason is that the new is used for creating an instance, while init is used for initializing the instance. Before initializing, the instance should be created first. That's why new should be called before init.

When instantiating a class, first, __new__() is called to create the instance of a class, then __init__() is called to initialize the instance.
__new__():
Called to create a new instance of class cls. ...
If __new__() is invoked during object construction and it returns an
instance of cls, then the new instance’s __init__() method will be
invoked like __init__(self[, ...]), ...
__init__():
Called after the instance has been created (by __new__()), ...
Because __new__() and __init__() work together in constructing objects
(__new__() to create it, and __init__() to customize it), ...
For example, when instantiating Teacher class, first, __new__() is called to create the instance of Teacher class, then __init__() is called to initialize the instance as shown below:
class Teacher:
def __init__(self, name):
self.name = name
class Student:
def __init__(self, name):
self.name = name
obj = Teacher("John") # Instantiation
print(obj.name)
This is the output:
<class '__main__.Teacher'>
John
And, using __new__() of the instance of Teacher class, we can create the instance of Student class as shown below:
# ...
obj = Teacher("John")
print(type(obj))
print(obj.name)
obj = obj.__new__(Student) # Creates the instance of "Student" class
print(type(obj))
Now, the instance of Student class is created as shown below:
<class '__main__.Teacher'>
<__main__.Teacher object at 0x7f4e3950bf10>
<class '__main__.Student'> # Here
Next, if we try to get the value of name variable from **the instance of Student class as shown below:
obj = Teacher("John")
print(type(obj))
print(obj.name)
obj = obj.__new__(Student)
print(type(obj))
print(obj.name) # Tries to get the value of "name" variable
The error below occurs because the instance of Student class has not been initialized by __init__() yet:
AttributeError: 'Student' object has no attribute 'name'
So, we initialize the instance of Student class as shown below:
obj = Teacher("John")
print(type(obj))
print(obj.name)
obj = obj.__new__(Student)
print(type(obj))
obj.__init__("Tom") # Initializes the instance of "Student" class
print(obj.name)
Then, we can get the value of name variable from the instance of Student class as shown below:
<class '__main__.Teacher'>
John
<class '__main__.Student'>
Tom # Here

People have already detailed the question and answer both use some examples like singleton etc. See the code below:
__instance = None
def __new__(cls):
if cls.__instance is None:
cls.__instance = object.__new__(cls)
return cls.__instance
I got the above code from this link, it has detailed overview of new vs init. Worth reading!