I have a class object, cls. I want to know its metaclass. How do I do this?
(If I wanted to know its parent classes, I would do cls.__mro__. Is there something like this to get the metaclass?)
Ok - so, a class's metaclass is just its own "type", and can be given by
type(cls) and other means such as cls.__class__.
In Python 3.x there are no further ambiguities - as the syntax for creating a metaclass just passes it as a named parameter on the class declaration statement anyway.
However, the syntax used for creating a metaclass in Python 2.x generates a side-effect that is worth noting.
Upon doing
class A(object):
__metaclass__ = MyMeta
The __metaclass__ attribute is set to that value in the actual class, even if the actual metaclass is another one.
Consider:
def class_pre_decorator(name, bases, namespace):
# do something with namespace
return type(name, bases, namespace)
This is a callable that can be used in the metaclass declaration of both Python 2 and 3 - and it is valid. After resolving, the actual metaclass in both cases will simply be type. However, in Python 2.x, cls.__metaclass__ will point to the callable class_pre_decorator, even tough type(cls) returns type, which is the correct metaclass.(Note that using callables in this way, they will not be used agian when the class is further subclassed)
There is no way in Python 3 to guess the callable actually used to instantiate a class if it gives no other hint (like setting an attribute on the class) that it was used:
# python 2
class A(object):
__metaclass__ = class_pre_decorator
On the console:
In [8]: type(A)
Out[8]: type
In [9]: A.__metaclass__
Out[9]: <unbound method A.class_pre_decorator>
and
# Python 3
class A(metaclass=class_pre_decorator):
pass
And trying to read A.__metaclass__ will simply raise an AttributeError.
Related
In Python, I can create a class method using the #classmethod decorator:
>>> class C:
... #classmethod
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> C.f()
f called with cls=<class '__main__.C'>
Alternatively, I can use a normal (instance) method on a metaclass:
>>> class M(type):
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> class C(metaclass=M):
... pass
...
>>> C.f()
f called with cls=<class '__main__.C'>
As shown by the output of C.f(), these two approaches provide similar functionality.
What are the differences between using #classmethod and using a normal method on a metaclass?
As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the abc.ABCMeta.register method.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict).
Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
#classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in #classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a #property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.
When you phrase it like you did in the question, the #classmethod and metaclasses may look similar but they have rather different purposes. The class that is injected in the #classmethod's argument is usually used for constructing an instance (i.e. an alternative constructor). On the other hand, the metaclasses are usually used to modify the class itself (e.g. like what Django does with its models DSL).
That is not to say that you can't modify the class inside a classmethod. But then the question becomes why didn't you define the class in the way you want to modify it in the first place? If not, it might suggest a refactor to use multiple classes.
Let's expand the first example a bit.
class C:
#classmethod
def f(cls):
print(f'f called with cls={cls}')
Borrowing from the Python docs, the above will expand to something like the following:
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
class C:
def f(cls):
print(f'f called with cls={cls}')
f = ClassMethod(f)
Note how __get__ can take either an instance or the class (or both), and thus you can do both C.f and C().f. This is unlike the metaclass example you give which will throw an AttributeError for C().f.
Moreover, in the metaclass example, f does not exist in C.__dict__. When looking up the attribute f with C.f, the interpreter looks at C.__dict__ and then after failing to find, looks at type(C).__dict__ (which is M.__dict__). This may matter if you want the flexibility to override f in C, although I doubt this will ever be of practical use.
In your example, the difference would be in some other classes that will have M set as their metaclass.
class M(type):
def f(cls):
pass
class C(metaclass=M):
pass
class C2(metaclass=M):
pass
C.f()
C2.f()
class M(type):
pass
class C(metaclass=M):
#classmethod
def f(cls):
pass
class C2(metaclass=M):
pass
C.f()
# C2 does not have 'f'
Here is more on metaclasses
What are some (concrete) use-cases for metaclasses?
Both #classmethod and Metaclass are different.
Everything in python is an object. Every thing means every thing.
What is Metaclass ?
As said every thing is an object. Classes are also objects in fact classes are instances of other mysterious objects formally called as meta-classes. Default metaclass in python is "type" if not specified
By default all classes defined are instances of type.
Classes are instances of Meta-Classes
Few important points are to understand metioned behaviour
As classes are instances of meta classes.
Like every instantiated object, like objects(instances) get their attributes from class. Class will get it's attributes from Meta-Class
Consider Following Code
class Meta(type):
def foo(self):
print(f'foo is called self={self}')
print('{} is instance of {}: {}'.format(self, Meta, isinstance(self, Meta)))
class C(metaclass=Meta):
pass
C.foo()
Where,
class C is instance of class Meta
"class C" is class object which is instance of "class Meta"
Like any other object(instance) "class C" has access it's attributes/methods defined in it's class "class Meta"
So, decoding "C.foo()" . "C" is instance of "Meta" and "foo" is method calling through instance of "Meta" which is "C".
First argument of method "foo" is reference to instance not class unlike "classmethod"
We can verify as if "class C" is instance of "Class Meta
isinstance(C, Meta)
What is classmethod?
Python methods are said to be bound. As python imposes the restriction that method has to be invoked with instance only.
Sometimes we might want to invoke methods directly through class without any instance (much like static members in java) with out having to create any instance.By default instance is required to call method. As a workaround python provides built-in function classmethod to bind given method to class instead of instance.
As class methods are bound to class. It takes at least one argument which is reference to class itself instead of instance (self)
if built-in function/decorator classmethod is used. First argument
will be reference to class instead of instance
class ClassMethodDemo:
#classmethod
def foo(cls):
print(f'cls is ClassMethodDemo: {cls is ClassMethodDemo}')
As we have used "classmethod" we call method "foo" without creating any instance as follows
ClassMethodDemo.foo()
Above method call will return True. Since first argument cls is indeed reference to "ClassMethodDemo"
Summary:
Classmethod's receive first argument which is "a reference to class(traditionally referred as cls) itself"
Methods of meta-classes are not classmethods. Methods of Meta-classes receive first argument which is "a reference to instance(traditionally referred as self) not class"
Class objects have a __bases__ (and a __base__) attribute:
>>> class Foo(object):
... pass
...
>>> Foo.__bases__
(<class 'object'>,)
Sadly, these attributes aren't accessible in the class body, which would be very convenient for accessing parent class attributes without having to hard-code the name:
class Foo:
cls_attr = 3
class Bar(Foo):
cls_attr = __base__.cls_attr + 2
# throws NameError: name '__base__' is not defined
Is there a reason why __bases__ and __base__ can't be accessed in the class body?
(To be clear, I'm asking if this is a conscious design decision. I'm not asking about the implementation; I know that __bases__ is a descriptor in type and that this descriptor can't be accessed until a class object has been created. I want to know why python doesn't create __bases__ as a local variable in the class body.)
I want to know why python doesn't create __bases__ as a local variable in the class body
As you know, class is mostly a shortcut for type.__new__() - when the runtime hits a class statements, it executes all statements at the top-level of the class body, collects all resulting bindings in a dedicated namespace dict, calls type() with the concrete metaclass, the class name, the base classes and the namespace dict, and binds the resulting class object to the class name in the enclosing scope (usually but not necessarily the module's top-level namespace).
The important point here is that it's the metaclass responsabilty to build the class object, and to allow for class object creation customisations, the metaclass must be free to do whatever it wants with its arguments. Most often a custom metaclass will mainly work on the attrs dict, but it must also be able to mess with the bases argument. Now since the metaclass is only invoked AFTER the class body statements have been executed, there's no way the runtime can reliably expose the bases in the class body scope since those bases could be modified afterward by the metaclass.
There are also some more philosophical considerations here, notably wrt/ explicit vs implicit, and as shx2 mentions, Python designers try to avoid magic variables popping out of the blue. There are indeed a couple implementation variables (__module__ and, in py3, __qualname__) that are "automagically" defined in the class body namespace, but those are just names, mostly intended as additional debugging / inspection informations for developers) and have absolutely no impact on the class object creation nor on its properties, behaviour and whatnots.
As always with Python, you have to consider the whole context (the execution model, the object model, how the different parts work together etc) to really understand the design choices. Whether you agree with the whole design and philosophy is another debate (and one that doesn't belong here), but you can be sure that yes, those choices are "conscious design decisions".
I am not answering as to why it was decided to be implemented the way it was, I'm answering why it wasn't implemented as a "local variable in the class body":
Simply because nothing in python is a local variable magically defined in the class body. Python doesn't like names magically appearing out of nowhere.
It's because it's simply is not yet created.
Consider the following:
>>> class baselessmeta(type):
... def __new__(metaclass, class_name, bases, classdict):
... return type.__new__(
... metaclass,
... class_name,
... (), # I can just ignore all the base
... {}
... )
...
>>> class Baseless(int, metaclass=baselessmeta):
... # imaginary print(__bases__, __base__)
... ...
...
>>> Baseless.__bases__
(<class 'object'>,)
>>> Baseless.__base__
<class 'object'>
>>>
What should the imaginary print result in?
Every python class is created via the type metaclass one way or another.
You have the int argument for the type() in bases argument, yet you do not know what is the return value is going to be. You may use that directly as a base in your metaclass, or you may return another base with your LOC.
Just realized your to be clear part and now my answer is useless haha. Oh welp.
How can I override the metaclass of a Python class, with a unittest.mock.MagicMock instance instead?
I have a function whose job involves working with the metaclass of an argument:
# lorem.py
class Foo(object):
pass
def quux(existing_class):
…
metaclass = type(existing_class)
new_class = metaclass(…)
The unit tests for this function will need to assert that the calls to
the metaclass go as expected, without actually calling a real class
object.
Note: The test case does not care about the metaclass's behaviour; it cares that quux retrieves that metaclass (using type(existing_class)) and calls the metaclass with the correct arguments.
So to write a unit test for this function, I want to pass a class object whose metaclass is a mock object instead. This will allow, for example, making assertions about how the metaclass was called, and ensuring no unwanted side effects.
# test_lorem.py
import unittest
import unittest.mock
import lorem
class stub_metaclass(type):
def __new__(metaclass, name, bases, namespace):
return super().__new__(metaclass, name, bases, namespace)
class quux_TestCase(unittest.TestCase):
#unittest.mock.patch.object(
lorem.Foo, '__class__', side_effect=stub_metaclass)
def test_calls_expected_metaclass_with_class_name(
self,
mock_foo_metaclass,
):
expected_name = 'Foo'
expected_bases = …
expected_namespace = …
lorem.quux(lorem.Foo)
mock_foo_metaclass.assert_called_with(
expected_name, expected_bases, expected_namespace)
When I try to mock the __class__ attribute of an existing class, though, I get this error:
File "/usr/lib/python3/dist-packages/mock/mock.py", line 1500, in start
result = self.__enter__()
File "/usr/lib/python3/dist-packages/mock/mock.py", line 1460, in __enter__
setattr(self.target, self.attribute, new_attr)
TypeError: __class__ must be set to a class, not 'MagicMock' object
This is telling me that unittest.mock.patch is attempting to set the __class__ attribute temporarily to a MagicMock instance, as I want; but Python is refusing that with a TypeError.
But placing a mock object as the metaclass is exactly what I'm trying to do: put a unittest.mock.MagicMock instance in the __class__ attribute in order that the mock object will do all that it does: record calls, pretend valid behaviour, etc.
How can I set a mock object in place of the Foo class's __class__ attribute, in order to instrument Foo and test that my code uses Foo's metaclass correctly?
You can't do exactly what you want. As you can see an object's __class__ attribute is very special in Python, and even for ordinary instances there are checks in runtime to verify it is assigned to a proper type.
When you get down to a class's __class__, that is even more strict.
Possible approach:
One thing to do in there is not pass a class to your test - but an object that is an instance from a crafted ordinary class, which will have an artificial __class__ attribute. Even them, you will have to change your code from calling type(existing_class) to do existing_class.__class__ directly. For an instance object to "falsify" its __class__ anyway, you have to implement __class__ as a property on its class (or override __getattribute__; (the class itself will report its true metaclass, but an instance can return whatever is coded on the __class__ property.
class Foo:
#property
def __class__(self):
return stub_metaclass
Actual suggestion:
But then, since you are at it, maybe the simplest thing is to mock type instead on the target module where quux is defined.
class MockType:
def __init__(self):
self.mock = mock.Mock()
def __call__(self, *args):
return self.mock
...
class ...:
...
def test_calls_expected_metaclass_with_class_name(
self,
):
try:
new_type = MockType()
# This creates "type" on the module "lorem" namespace
# as a global variable. It will then override the built-in "type"
lorem.type = new_type
lorem.quux(lorem.Foo)
finally:
del lorem.type # un-shadows the built-in type on the module
new_type.mock.assert_called_with(
'Foo', unittest.mock.ANY, unittest.mock.ANY)
Still another approach
Another thing that can be done is to craft a full "MockMetaclass" in the "old fashion": without unittest.magicmock at all, instead, with intrumented __new__ and other relevant methods that will record the called parameters, and function as a true metaclass for a class you pass in as parameter.
Considerations on what is being done
People reaching here, please note that one should not test the class creation (and metaclass) mechanisms themselves. One can just assume the Python runtime have these working and tested already.
I am trying to learn inheritance in python. I write class "Course" as super class of class "AdvacedCourse" as shown below.
class Course(object):
def __init__(self, crsName ="python", duration=45):
self.crsName = crsName
self.duration = 25
And the sub class is:
import Course
class AdvancedCourse (Course):
def __init__(self, crsName ="python", duration=45):
self.crsName = "java"
self.duration = 25
But I got stuck on an error:
class AdvancedCourse (Course):
TypeError: module.__init__() takes at most 2 arguments (3 given)
Any suggestions?
This is a problem with importing, not inheritance. Course is the module: you need to inherit from Course.Course. (In Python we usually name modules in lower case, though).
I assume that class Course is in another module Course.py.
Then you should import it with from Course import Course. And #Daniel is right - you should have module in file course.py (lowercase) and import statement will be from course import Course.
Note: I've only made this an answer because ElmoVanKielmo suggested it. It definitely shouldn't be the accepted answer, as it will only be confusing to novices… but maybe it will be interesting to others.
As Daniel Roseman's answer explains, import Course means that Course is a module, and Course.Course is the class you want.
So, what happens when you try to inherit from a module?
In Python, classes are objects, just like anything else. A class's type (which you can see by printing out type(AdvancedCourse)) is usually type, but you can specify a different type by setting a metaclass. When you inherit from a superclass, if you don't specify a metaclass, you get your superclass's metaclass. So, when you do this:
import Course
class AdvancedCourse(Course):
… you're saying that your metaclass is type(Course)—that is, module.*
Just as creating an instance means a call to the class's __init__, creating a class means a call to the metaclass's __init__. The arguments, besides self (which is a class here, not an instance, of course, and therefore usually named cls instead of self) are the class name, the list of base classes, and the dictionary of methods and other attributes. So, this definition:
class AdvancedCourse(Course):
pass
… tries to initialize a module object by calling module.__init__(cls, 'AdvancedCourse', (Course,), {}).**
Of course modules are also just objects, so they have an __init__ too, but their arguments are just self, name and docstring. So, you're passing one too many arguments.
You're actually just getting lucky here; if module and type happened to take the same number of arguments in their constructor, you'd end up with something very weird that acted sort of like a class in some ways, but not others, causing all kinds of subtle bugs.
If you want to play with this in the interactive interpreter, try this:
>>> class meta(type):
... def __init__(self, name, bases, d):
... print('{}\n{}\n{}\n{}\n'.format(self, name, bases, d))
... super(meta, self).__init__(name, bases, d)
>>> class Silly(metaclass=meta):
... __metaclass__=meta
<class '__main__.Silly'>
Silly
()
{'__module__': '__main__', '__qualname__': 'Silly', '__metaclass__': <class '__main__.meta'>}
>>> class Sillier(Silly):
... pass
<class '__main__.Sillier'>
Sillier
(<class '__main__.Silly'>,)
{'__module__': '__main__', '__qualname__': 'Sillier'}
In Python 2.x, you don't want the metaclass=meta in the class header; you can just put object there. In 3.x, you don't want the __metaclass__=meta in the body; you can just put pass there.
* module is one of those "hidden" types not accessible by name in builtins, but you can get at it as types.ModuleType, or just by importing something and using type(Course).
** Actually, even an empty class has a few members in its dictionary. For example, in Python 3.3, there's always at least a __module__ attribute and a __qualname__ attribute.
I can't understand why the following code behaves a particular way, which is described below:
from abc import ABCMeta
class PackageClass(object):
__metaclass__ = ABCMeta
class MyClass1(PackageClass):
pass
MyClass2 = type('MyClass2', (PackageClass, ), {})
print MyClass1
print MyClass2
>>> <class '__main__.MyClass1'>
>>> <class 'abc.MyClass2'>
Why does repr(MyClass2) says abc.MyClass2 (which is by the way not true)?
Thank you!
The problem stems from the fact that ABCMeta overrides __new__ and calls its superclass constructor (type()) there. type() derives the __module__ for the new class from its calling context1; in this case, the type call appears to come from the abc module. Hence, the new class has __module__ set to abc (since type() has no way of knowing that the actual class construction took place in __main__).
The easy way around is to just set __module__ yourself after creating the type:
MyClass2 = type('MyClass2', (PackageClass, ), {})
MyClass2.__module__ = __name__
I would also recommend filing a bug report.
Related: Base metaclass overriding __new__ generates classes with a wrong __module__, Weird inheritance with metaclasses
1: type is a type object defined in C. Its new method uses the current global __name__ as the __module__, unless it calls a metaclass constructor.