How come a metaclass does not show up in the MRO?
For instance:
>>> class Foo(type):
... foo = 21
...
>>> class Bar(metaclass=Foo):
... pass
...
>>> Bar.mro()
[<class '__main__.Bar'>, <class 'object'>]
>>> Bar.foo
21
Also, I saw elsewhere that Python uses C3 linearization to compute the MRO, but this linearization does not handle metaclasses. So what algorithm does Python use in that case?
Because none of your classes derive from the metaclass. A metaclass is not a base class. The metaclass is the factory that produced the Bar class object, just like a class object produces instances.
There is no linearisation required here. Bar is an object of type Foo, just like other classes are of type type. Any subclasses of Bar will have the same type (the metaclass); they have that relationship directly. A class can have only one metaclass at a time.
A metaclass comes last when looking up attributes; so Bar.spam will first be looked for in the MRO, and only then on type(Bar).
Of course, metaclasses use an inheritance hierarchy too; Foo was derived from type in your example. That relationship uses an MRO too.
Related
I wrote a class in python which inherits from type . I thought that this was the only requirement for a class so as to be called as a metaclass but had not defined a __new__ method for it. But on instantiating with this new class as the metaclass I got an error stating the below :
TypeError: type.__new__() takes exactly 3 arguments (0 given)
The following is my code :
class a(type) :
pass
c= a()
Now when the class statement is being processed , that the __new__ method of type is being called is my assumption. This is because the default metaclass of all classes in python is type .
Now when I am instantiating the class a , which I have assumed to be a metaclass under the assumption that any class inheriting from (type) is a metaclass , isn't it the same as creating a class ? Why should this not result in type.__new__ being called with correct arguments ?
This does not work:
class a(type) :
pass
c = a()
...for the same reason for which this does not work:
c = type()
In the end, both do the same.
To use it as a metaclass, do this:
>>> class Class(metaclass=a):
... pass
...
>>> Class
<class '__main__.Class'>
>>> type(Class)
<class '__main__.a'>
You could also instantiate the class directly, as you tried, but you have to provide the correct arguments:
AnotherClass = type('AnotherClass', (), {})
YetAnotherClass = a('YetAnotherClass', (), {})
This error is due to you not respecting type's signature.
Inheriting from type is indeed enough for a class to be used as a metaclass, but the thing is you actually have to use it as a metaclass.
type itself has "two working modes: if called with 3 positional arguments, it creates a new class. And then type is the metaclass of that class. If called with 1 positional argument, it creates no new class or object at all - instead, it just returns that object's class.
But it makes no sense calling type with no arguments at all. And the arguments in the modes above are not optional. So, you will get a TypeError if your try to call type with no arguments at all - and that is not a "TypeError because something went wrong with the type class" - it is a "TypeError because your call did not match the callable signature".
When you inherit from type and change nothing, you class will behave exactly the same as the original type: you can call it with either one or three positional arguments, and the code responsible for working in either mode lies in type.__new__.
Now, if you want to use your class as a metaclass, you can indeed call it, but in the three argument form: you ass it the new class name, its bases and its attributes - which can actually be all empty, but you have to pass a string, a tuple and a dictionary as these three arguments:
class A(type): pass
myclass = A("", (), {})
And now, A is working as the metaclass for myclass:
In [16]: type(myclass)
Out[16]: __main__.A
However, whenever one defines a metaclass it is more usual to use it with the metaclass= named argument when declaring a class body:
In [17]: class MyOtherClass(metaclass=A):
...: pass
...:
In [18]: type(MyOtherClass)
Out[18]: __main__.A
Python's runtime will then compile this class body, and when the bytecod for it is executed, it will make the call to your metaclass' __new__ (and then __init__, and before that its __prepare__) method, so that it works as a metaclass.
So, just in case it is not clear: when you derive a class from type intending to use it as a metaclass, there is no need to further instantiate it to say that "it is now a metaclass". A subclass of type already can be a metaclass, and its instances will be classes, that will have it as a metaclass.
This is pertaining to python 2.x
In the following class, if we subclass "object", I understand the methods are inherited in the derived class Foo which includes __hash__ (can see this by printing dir(Foo() )
Hence calling hash(Foo()) calls the magic method __hash__ and gives us a hash value.
However, if we don't subclass "object", resulting in dir(Foo()) not listing out the __hash__ method, so then why do we still get a hash value in python2?
I believe in python3 this problem has been addressed since the methods from the "object*" class are inherited by default.
#class Foo(object) Works since __hash__ is available in the base class
class Foo: #Why does this work?
def __init__(self):
self.x = None
a = Foo()
print dir(a) # No __hash__ magic method
print hash(a)
# Expecting an error like non-hashable or __hash__ not implemented
# or something similar
Old-style classes are weird. Officially, instances of old-style classes aren't wholly instances of their class, they're all instances of type instance. The instance type defines __hash__ (tp_hash is the C level slot that's equivalent to __hash__ for C defined types), so even though it's not defined on your instance directly, nor on the class that created it, it finds __hash__ on the instance type itself through weird and terrible magic (actually, the magic is in how it manages to use your class's features at all, given that its type is instance).
You can see this in the interactive interpreter:
>>> class Foo: pass
>>> Foo().__hash__ # Same basic error for Foo.__hash__ too
AttributeError Traceback (most recent call last)
...
----> 1 Foo().__hash__
AttributeError: Foo instance has no attribute '__hash__'
>>> type(Foo())
<type 'instance'>
>>> type(Foo()).__hash__
<slot wrapper '__hash__' of 'instance' objects>
This works even though the instance itself can't see __hash__ because "special methods" (those documented special methods that begin and end with double underscores) are looked up on the type, not the instance, so __hash__ is found on instance itself. At the C level, hash(x) is doing the equivalent of type(x).__hash__(x) (it's a little more complicated because it won't use the default __hash__ implementation if __eq__ has a custom definition, but that's the general idea).
In Python 2, with a trick it is possible to create a class with several bases, although the bases have metaclasses that are not subclass of each other.
The trick is that these metaclasses have themselves a metaclass (name it a "metametaclass"), and this metametaclass provides the metaclasses with a call method that dynamically creates a common sub-metaclass of the base metaclasses, if necessary. Eventually, a class results whose metaclass is the new sub-metaclass. Here is the code:
>>> class MetaMeta(type):
... def __call__(mcls, name, bases, methods):
... metabases = set(type(X) for X in bases)
... metabases.add(mcls)
... if len(metabases) > 1:
... mcls = type(''.join([X.__name__ for X in metabases]), tuple(metabases), {})
... return mcls.__new__(mcls, name, bases, methods)
...
>>> class Meta1(type):
... __metaclass__ = MetaMeta
...
>>> class Meta2(type):
... __metaclass__ = MetaMeta
...
>>> class C1:
... __metaclass__ = Meta1
...
>>> class C2:
... __metaclass__ = Meta2
...
>>> type(C1)
<class '__main__.Meta1'>
>>> type(C2)
<class '__main__.Meta2'>
>>> class C3(C1,C2): pass
...
>>> type(C3)
<class '__main__.Meta1Meta2'>
This example (of course changing the syntax to class C1(metaclass=Meta1) etc) doesn't work in Python 3.
Question 1: Do I understand correctly that in Python 2, first C3 is constructed, using the metaclass of the first base, and an error would only result if type(C3) were not a common subclass of type(C1) and type(C2), whereas in Python 3 the error is raised earlier?
Question 2: (How) Is it possible to make the above example work in Python 3? I did try to use a subclass of abc.ABCMeta as metametaclass, but even though using a custom __subclasscheck__ makes issubclass(Meta1, Meta2) return True, the creation of C3 would still result in an error.
Note: Of course I could make Python 3 happy by statically defining Meta1Meta2 and explicitly using it as a metaclass for C3. However, that's not what I want. I want that the common sub-metaclass is created dynamically.
In Python 3 at the time the metaclass is used it have to be ready, and it can't know about the bases of the final (non-meta) class in order to dynamically create a metaclass at that point.
But instead of complicating things (I confess I could not wrap my head around your need for a meta-meta-class) - you can simply use normal class hierarchy with collaborative use of super for your metaclasses.
You can even build the final metaclass dynamically with a simple
call to type:
class A(type):
def __new__(metacls, name, bases,attrs):
attrs['A'] = "Metaclass A processed"
return super().__new__(metacls, name, bases,attrs)
class B(type):
def __new__(metacls, name, bases,attrs):
attrs['B'] = "Metaclass A processed"
return super().__new__(metacls, name, bases,attrs)
C = type("C", (A, B), {})
class Example(metaclass=C): pass
And:
In[47] :Example.A
Out[47]: 'Metaclass A processed'
In[48]: Example.B
Out[48]: 'Metaclass A processed'
If your metaclasses are not designed to be collaborative in the first place, it will be very tricky to create any automatic method to combine them - and it would possibly involve monkey-patching the call to type.__new__ in some of the metaclasses constructors.
As for not needing to explictly build C, you can use a normal function as the metaclass parameter, that will inspect the bases and build a dynamic derived metaclass:
def Auto(name, bases, attrs):
basemetaclasses = []
for base in bases:
metacls = type(base)
if isinstance(metacls, type) and metacls is not type and not metacls in basemetaclasses:
basemetaclasses.append(metacls)
dynamic = type(''.join(b.__name__ for b in basemetaclasses), tuple(basemetaclasses), {})
return dynamic(name, bases, attrs)
(This code is very similar to yours - but I used a three-line explicit for instead of a set in order to preserve the metaclass order - which might matter)
You have them to pass Auto as a metaclass for derived classes, but otherwise it works as in your example:
In [61]: class AA(metaclass=A):pass
In [62]: class BB(metaclass=B):pass
In [63]: class CC(AA,BB): pass
---------------------------------------------------------------------------
...
TypeError: metaclass conflict
...
In [66]: class CC(AA,BB, metaclass=Auto): pass
In [67]: type(CC)
Out[67]: __main__.AB
In [68]: CC.A
Out[68]: 'Metaclass A processed'
Here's an example that shows some options that you have in python3.x. Specifically, C3 has a metaclass that is created dynamically, but in a lot of ways, explicitly. C4 has a metaclass that is created dynamically within it's metaclass function. C5 is just to demonstrate that it too has the same metaclass properies that C4 has. (We didn't lose anything through inheritance which can happen if you use a function as a metaclass instead of a type ...)
class Meta1(type):
def foo(cls):
print(cls)
class Meta2(type):
def bar(cls):
print(cls)
class C1(object, metaclass=Meta1):
"""C1"""
class C2(object, metaclass=Meta2):
"""C2"""
class C3(C1, C2, metaclass=type('Meta3', (Meta1, Meta2), {})):
"""C3"""
def meta_mixer(name, bases, dct):
cls = type('MixedMeta', tuple(type(b) for b in bases), dct)
return cls(name, bases, dct)
class C4(C1, C2, metaclass=meta_mixer):
"""C4"""
C1.foo()
C2.bar()
C3.foo()
C3.bar()
C4.foo()
C4.bar()
class C5(C4):
"""C5"""
C5.foo()
C5.bar()
It should be noted that we're playing with fire here (the same way that you're playing with fire in your original example). There is no guarantee that the metaclasses will play nicely in cooperative multiple inheritance. If they weren't designed for it, chances are that you'll run into bugs using this at some point. If they were designed for it, there'd be no reason to be doing this hacky runtime mixing :-).
In 95% of cases, it should be possible to use the machinery introduced in Python 3.6 due to PEP 447 to do most of what metaclasses can do using special new hooks. In that case, you will not need to combine metaclasses since your hooks can call super and their behavior is combined due to inheritance.
As for your general case, I believe that mgilson is right and that you are probably making things too complicated. I have yet to see a case for combining metaclasses that is not covered by PEP 447.
I've little bit test to fully understand metaclass in python.
class Test(object):
pass
print Test.__class__
print Test.__class__.__class__
print Test.__class__.__class__.__class__
All of result is same type. but each of their address is not same
I can't really understand why metaclass has a metaclass recursively.
Explain me please?
Actually, addresses are the same:
>>> id(Test.__class__)
6384576
>>> id(Test.__class__.__class__)
6384576
>>> id(Test.__class__.__class__.__class__)
6384576
Everything is an object in Python, and each object must have a class (it should belong to some type). You can access that class/type reference by __class__ attribute, e.g.:
>>> (1).__class__
<type 'int'>
Everything includes classes itself, which are of class/type called type:
>>> (1).__class__.__class__
<type 'type'>
In the same time type 'type'> is also an object and should reference to some class/type. But since this is kind of special object, its __class__ attribute refers to itself:
>>> (1).__class__.__class__.__class__ is (1).__class__.__class__
True
When you do Test.__class__, you get back the type of Test, which is type (because Test is a class identifier).
type itself is again a class identifier, so you can call __class__ (which is inherited from object) on it and you get back that its type is, again, type because it is a class identifier.
Because you will always get back type which is a class itself, you can do this infinitely many times and will always get back that the current object's type is type.
All classes are classes which means they are derived from a class called class...
All the python's class object is build by the built-in function type(). You could also try this.
>>> T.__class__ == type
True
>>> type(type)
<type 'type'>
The T.class is equal to the build-in function type which is also an object implemented the call function. It's a attribute will be interpret as class(T). As your T class have no base class so type() is used which will return the type object.
You could check the python doc about customizing class creation to get detail about class creation.
To determining the appropriate metaclass
if no bases and no explicit metaclass are given, then type() is used
if an explicit metaclass is given and it is not an instance of type(), then it is used directly as the metaclass
if an instance of type() is given as the explicit metaclass, or bases are defined, then the most derived metaclass is used
I can't understand why the following code behaves a particular way, which is described below:
from abc import ABCMeta
class PackageClass(object):
__metaclass__ = ABCMeta
class MyClass1(PackageClass):
pass
MyClass2 = type('MyClass2', (PackageClass, ), {})
print MyClass1
print MyClass2
>>> <class '__main__.MyClass1'>
>>> <class 'abc.MyClass2'>
Why does repr(MyClass2) says abc.MyClass2 (which is by the way not true)?
Thank you!
The problem stems from the fact that ABCMeta overrides __new__ and calls its superclass constructor (type()) there. type() derives the __module__ for the new class from its calling context1; in this case, the type call appears to come from the abc module. Hence, the new class has __module__ set to abc (since type() has no way of knowing that the actual class construction took place in __main__).
The easy way around is to just set __module__ yourself after creating the type:
MyClass2 = type('MyClass2', (PackageClass, ), {})
MyClass2.__module__ = __name__
I would also recommend filing a bug report.
Related: Base metaclass overriding __new__ generates classes with a wrong __module__, Weird inheritance with metaclasses
1: type is a type object defined in C. Its new method uses the current global __name__ as the __module__, unless it calls a metaclass constructor.