I am trying to acquire a basic understanding of python introspection and in doing so I found the in that regard useful mro() method mentioned here.
When I (as an introspection exercise) tried to use said method and the builtin dir() function in an attempt to find out where mro() might live, I however was unable to succeed. Why?
Here is my approach:
the mro() method obviously is available without any imports (unlike inspect.getmro() which is part of the inspect module; e.g. str.mro() returns [<class 'str'>, <class 'object'>]
Since str.mro() returns [<class 'str'>, <class 'object'>], the mro() method should live somewhere in str and/or object.
Yet neither dir(str) nor dir(object) appear to contain the mro() method. Also help() and help(str.mro) do not enlighten the puzzled student of introspection.
If you look with dir(type) you'll find the magic method attribute __mro__. Quoted from here:
class.__mro__
This attribute is a tuple of classes that are considered when looking for base classes during method resolution.
class.mro()
This method can be overridden by a metaclass to customize the method resolution order for its instances. It is called at class instantiation, and its result is stored in mro
Look at this answer for more about type which is the usual metaclass used in Python. Quoted from there:
type is the usual metaclass in Python. type is itself a class, and it is its own type. You won't be able to recreate something like type purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass type.
Related
I have the following code. From the output the MROs are different but still issubclass returns true. Can someone explain how Python finds they are equal?
My understanding is that MRO show the inheritance tree of classes. And classes with different inheritance tree (MRO) should not satisfy sub-class validation. I am using Python-3.9.5 on Windows-10
Code
from collections.abc import Iterator
it_type = type(iter([]))
print(it_type.__mro__)
print(Iterator.__mro__)
print(issubclass(it_type, Iterator))
Output
(<class 'list_iterator'>, <class 'object'>)
(<class 'collections.abc.Iterator'>, <class 'collections.abc.Iterable'>, <class 'object'>)
True
Quoting the documentation for the issubclass function (emphasis mine):
Return True if class is a subclass (direct, indirect, or virtual) of classinfo.
Following the link to the definition of a virtual subclass:
ABCs introduce virtual subclasses, which are classes that don’t inherit from a class but are still recognized by isinstance() and issubclass(); see the abc module documentation.
Following that link to the abc module documentation:
An ABC can be subclassed directly, and then acts as a mix-in class. You can also register unrelated concrete classes (even built-in classes) and unrelated ABCs as “virtual subclasses” – these and their descendants will be considered subclasses of the registering ABC by the built-in issubclass() function, but the registering ABC won’t show up in their MRO (Method Resolution Order)
This is a question about how python's (dynamic) type system works. I have read articles online saying that to define a class to be "an iterable", we need to define a __iter__ function for it. We don't in fact have to explicitly state that that class "is an iterable". I would have guessed based on experience with other languages that I'd have to write something like
class Foo extends Iterable:
def __iter__(self):
return self
When I test the type of Foo and one of its instances I get:
print(type(Foo))
print(type(Foo()))
print(isinstance(Foo(), collections.abc.Iterable))
Output:
<class 'type'>
<class '__main__.Foo'>
True
My question is: What is the status of a concept like "Iterable" in python's (dynamic) type system? Should I think of it as having anything to do with types at all?
A type is what python calls any object that's been defined via a class statement. Using Java as a reference point, type is akin to java.lang.Class - the class that represents classes.
"Iterable" is more akin to an interface than a class - whereas a class/type defines both an internal state and certain methods, an interface only defines those methods. Python doesn't formalize this like other languages, but the principle is used in most of python's "hidden" methods (the ones that have two underscores on both sides). If a particular hidden method is defined for a class (for example, __iter__()), then that class is considered to be iterable*.
In your example, you use isinstance() to prove your point. The python documentation actually has a page on collections.abc, which goes into detail about their behavior:
This module provides abstract base classes that can be used to test whether a class provides a particular interface; for example, whether it is hashable or whether it is a mapping.
(bold added for emphasis).
And it even mentions Iterators specifically:
class collections.abc.Iterable
ABC for classes that provide the __iter__() method.
Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an __iter__() method, but it does not detect classes that iterate with the __getitem__() method. The only reliable way to determine whether an object is iterable is to call iter(obj).
Python allows for ABC classes essentially hijack the builtin isinstance() call via defining the __instancecheck__() metaclass method, which is why isinstance(Foo(), Iterable) is able to return True despite Foo not inheriting from Iterable.
*while the inputs and outputs for these functions aren't as strictly defined as in static-typed languages like Java, the built-in methods that call them have very specific expectations that, in effect, result in the same thing. For example, I once ran into a problem trying to override __len__() on an object to return a float instead of an int, because the built-in len() threw an error when I tried to use it on that object.
In Python, we are always told that dir lists any attribute of its questioned object. But why isn't there __dict__ in dir("an imported module")?
You cannot rely on dir([object]) returning accurate results. As stated here:
If the object has a method named __dir__(), this method will be
called and must return the list of attributes. This allows objects
that implement a custom __getattr__() or __getattribute__()
function to customize the way dir() reports their attributes.
If the object does not provide __dir__(), the function tries its
best to gather information from the object’s __dict__ attribute, if
defined, and from its type object. The resulting list is not
necessarily complete, and may be inaccurate when the object has a
custom __getattr__().
So try looking into the object you are trying to get the information from and look for the attributes yourself.
The answer is in python documentation for the dir function:
Note: Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class.
__dict__ is a special attribute that is part of that exception.
dir() doesn't just look up an object's __dict__ (which sometimes doesn't even exist), it will use the object's heritage (its class or type, and any superclasses, or parents, of that class or type) to give you a complete picture of all available attributes.
An instance __dict__ is just the 'local' set of attributes on that instance, and does not contain every attribute available on the instance. Instead, you need to look at the class and the class's inheritance tree too.
For more information, check out the main differences between dir() and __dict__.
Here's a piece of code that I cannot understand:
class COWMeta(type):
pass
class COWDictMeta(COWMeta):
....
I know how to create a new class in python:
class MyClass(BaseClass):
...
But as the manual states, 'type' is function.
type(...)
Function of __builtin__ module
type(object) -> the object’s type type(name, bases, dict) -> a new type
How can a class inherit from a function? And what does that piece of code mean?
type is the basic object type in python. Like many object types in python, it acts as a constructor for creating new types, but in it's simplest form it'll return the type of existing objects. It then looks like a function. Compare this to int() and list(), for example.
In python, you can create new types, also called metaclasses, allowing you to do all sorts of powerful and interesting tricks in Python. Basing a class definition on type means you are creating a new metaclass.
See What is a metaclass in Python? for an in-depth answer on what metaclasses are.
type is not a function in the same way that, eg:
def foo():
pass
is a function. It is callable like a function (and like many other objects in Python), but it is actually coded as a class. type itself can show you this difference:
>>> type(type)
<class 'type'>
>>> type(foo)
<class 'function'>
The docs call it a 'function' not because of how it is implemented, but because of how it is commonly used. This is broadly similar to, for example, itertools.permutations, which while not explicitly called a function by the docs is implied to be one:
Return successive r length permutations of elements in the iterable.
But itertools.permutations is implemented as a class:
>>> type(itertools.permutations)
<class 'type'>
I am new to python. I think non-class objects do not have bases attribute whereas class objects do have it. But I am not sure. How does python\cpython checks if an object is non-class or class and passes the correct arguments to the object's descriptor attribute accordingly during the attribute access?
============================================
updated:
I was learning how __getattribute__ and descriptor cooperate together to make bounded methods. I was wondering how class object & non-class object invokes the descriptor's __get__ differently. I thought those 2 types of objects shared the same __getattribute__ CPython function and that same function would have to know if the invoking object was a class or non-class. But I was wrong. This article explains it well:
http://docs.python.org/dev/howto/descriptor.html#functions-and-methods
So class object use type.__getattribute__ whereas non-class object use object.__getattribute__. They are different CPython functions. And super has a third __getattribute__ CPython implementation as well.
However, about the super one, the above article states that:
quote and quote
The object returned by super() also has a custom _getattribute_() method for invoking descriptors. The call super(B, obj).m() searches obj._class_._mro_ for the base class A immediately following B and then returns A._dict_['m']._get_(obj, A). If not a descriptor, m is returned unchanged. If not in the dictionary, m reverts to a search using object._getattribute_().
The statement above didn't seem to match my experiment with Python3.1. What I saw is, which is reasonable to me:
super(B, obj).m ---> A.__dict__['m'].__get__(obj, type(obj))
objclass = type(obj)
super(B, objclass).m ---> A.__dict__['m'].__get__(None, objclass)
A was never passed to __get__
It is reasonable to me because I believe objclass (rather than A) 's mro chain is the one needed within m especially for the second case.
Was I doing something wrong? Or I didn't understand it correctly?
As the commenters asked: Why do you care? Usually that's a sign of not using Python the way it was meant to be used.
A very powerful concept of Python is duck typing. You don't care about the type or class of an object as long as it exposes the attributes you need.
how about inspect.isclass(objectname)?
more info here: http://docs.python.org/library/inspect.html