Develop Reference

Python generate a list of classes from folders and subfolders only of a certain type [duplicate]

I need a working approach of getting all classes that are inherited from a base class in Python.

New-style classes (i.e. subclassed from object, which is the default in Python 3) have a __subclasses__ method which returns the subclasses:
class Foo(object): pass
class Bar(Foo): pass
class Baz(Foo): pass
class Bing(Bar): pass
Here are the names of the subclasses:
print([cls.__name__ for cls in Foo.__subclasses__()])
# ['Bar', 'Baz']
Here are the subclasses themselves:
print(Foo.__subclasses__())
# [<class '__main__.Bar'>, <class '__main__.Baz'>]
Confirmation that the subclasses do indeed list Foo as their base:
for cls in Foo.__subclasses__():
print(cls.__base__)
# <class '__main__.Foo'>
# <class '__main__.Foo'>
Note if you want subsubclasses, you'll have to recurse:
def all_subclasses(cls):
return set(cls.__subclasses__()).union(
[s for c in cls.__subclasses__() for s in all_subclasses(c)])
print(all_subclasses(Foo))
# {<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>}
Note that if the class definition of a subclass hasn't been executed yet - for example, if the subclass's module hasn't been imported yet - then that subclass doesn't exist yet, and __subclasses__ won't find it.
You mentioned "given its name". Since Python classes are first-class objects, you don't need to use a string with the class's name in place of the class or anything like that. You can just use the class directly, and you probably should.
If you do have a string representing the name of a class and you want to find that class's subclasses, then there are two steps: find the class given its name, and then find the subclasses with __subclasses__ as above.
How to find the class from the name depends on where you're expecting to find it. If you're expecting to find it in the same module as the code that's trying to locate the class, then
cls = globals()[name]
would do the job, or in the unlikely case that you're expecting to find it in locals,
cls = locals()[name]
If the class could be in any module, then your name string should contain the fully-qualified name - something like 'pkg.module.Foo' instead of just 'Foo'. Use importlib to load the class's module, then retrieve the corresponding attribute:
import importlib
modname, _, clsname = name.rpartition('.')
mod = importlib.import_module(modname)
cls = getattr(mod, clsname)
However you find the class, cls.__subclasses__() would then return a list of its subclasses.

If you just want direct subclasses then .__subclasses__() works fine. If you want all subclasses, subclasses of subclasses, and so on, you'll need a function to do that for you.
Here's a simple, readable function that recursively finds all subclasses of a given class:
def get_all_subclasses(cls):
all_subclasses = []
for subclass in cls.__subclasses__():
all_subclasses.append(subclass)
all_subclasses.extend(get_all_subclasses(subclass))
return all_subclasses

The simplest solution in general form:
def get_subclasses(cls):
for subclass in cls.__subclasses__():
yield from get_subclasses(subclass)
yield subclass
And a classmethod in case you have a single class where you inherit from:
#classmethod
def get_subclasses(cls):
for subclass in cls.__subclasses__():
yield from subclass.get_subclasses()
yield subclass

Python 3.6 - __init_subclass__
As other answer mentioned you can check the __subclasses__ attribute to get the list of subclasses, since python 3.6 you can modify this attribute creation by overriding the __init_subclass__ method.
class PluginBase:
subclasses = []
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.subclasses.append(cls)
class Plugin1(PluginBase):
pass
class Plugin2(PluginBase):
pass
This way, if you know what you're doing, you can override the behavior of of __subclasses__ and omit/add subclasses from this list.

Note: I see that someone (not #unutbu) changed the referenced answer so that it no longer uses vars()['Foo'] — so the primary point of my post no longer applies.
FWIW, here's what I meant about #unutbu's answer only working with locally defined classes — and that using eval() instead of vars() would make it work with any accessible class, not only those defined in the current scope.
For those who dislike using eval(), a way is also shown to avoid it.
First here's a concrete example demonstrating the potential problem with using vars():
class Foo(object): pass
class Bar(Foo): pass
class Baz(Foo): pass
class Bing(Bar): pass
# unutbu's approach
def all_subclasses(cls):
return cls.__subclasses__() + [g for s in cls.__subclasses__()
for g in all_subclasses(s)]
print(all_subclasses(vars()['Foo'])) # Fine because Foo is in scope
# -> [<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>]
def func(): # won't work because Foo class is not locally defined
print(all_subclasses(vars()['Foo']))
try:
func() # not OK because Foo is not local to func()
except Exception as e:
print('calling func() raised exception: {!r}'.format(e))
# -> calling func() raised exception: KeyError('Foo',)
print(all_subclasses(eval('Foo'))) # OK
# -> [<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>]
# using eval('xxx') instead of vars()['xxx']
def func2():
print(all_subclasses(eval('Foo')))
func2() # Works
# -> [<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>]
This could be improved by moving the eval('ClassName') down into the function defined, which makes using it easier without loss of the additional generality gained by using eval() which unlike vars() is not context-sensitive:
# easier to use version
def all_subclasses2(classname):
direct_subclasses = eval(classname).__subclasses__()
return direct_subclasses + [g for s in direct_subclasses
for g in all_subclasses2(s.__name__)]
# pass 'xxx' instead of eval('xxx')
def func_ez():
print(all_subclasses2('Foo')) # simpler
func_ez()
# -> [<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>]
Lastly, it's possible, and perhaps even important in some cases, to avoid using eval() for security reasons, so here's a version without it:
def get_all_subclasses(cls):
""" Generator of all a class's subclasses. """
try:
for subclass in cls.__subclasses__():
yield subclass
for subclass in get_all_subclasses(subclass):
yield subclass
except TypeError:
return
def all_subclasses3(classname):
for cls in get_all_subclasses(object): # object is base of all new-style classes.
if cls.__name__.split('.')[-1] == classname:
break
else:
raise ValueError('class %s not found' % classname)
direct_subclasses = cls.__subclasses__()
return direct_subclasses + [g for s in direct_subclasses
for g in all_subclasses3(s.__name__)]
# no eval('xxx')
def func3():
print(all_subclasses3('Foo'))
func3() # Also works
# -> [<class '__main__.Bar'>, <class '__main__.Baz'>, <class '__main__.Bing'>]

Here is a simple but efficient version of code:
def get_all_subclasses(cls):
subclass_list = []
def recurse(klass):
for subclass in klass.__subclasses__():
subclass_list.append(subclass)
recurse(subclass)
recurse(cls)
return set(subclass_list)
Its time complexity is O(n) where n is the number of all subclasses if there's no multiple inheritance.
It's more efficient than the functions that recursively create lists or yield classes with generators, whose complexity could be (1) O(nlogn) when the class hierarchy is a balanced tree or (2) O(n^2) when the class hierarchy is a biased tree.

A much shorter version for getting a list of all subclasses:
from itertools import chain
def subclasses(cls):
return list(
chain.from_iterable(
[list(chain.from_iterable([[x], subclasses(x)])) for x in cls.__subclasses__()]
)
)

Here's a version without recursion:
def get_subclasses_gen(cls):
def _subclasses(classes, seen):
while True:
subclasses = sum((x.__subclasses__() for x in classes), [])
yield from classes
yield from seen
found = []
if not subclasses:
return
classes = subclasses
seen = found
return _subclasses([cls], [])
This differs from other implementations in that it returns the original class.
This is because it makes the code simpler and:
class Ham(object):
pass
assert(issubclass(Ham, Ham)) # True
If get_subclasses_gen looks a bit weird that's because it was created by converting a tail-recursive implementation into a looping generator:
def get_subclasses(cls):
def _subclasses(classes, seen):
subclasses = sum(*(frozenset(x.__subclasses__()) for x in classes))
found = classes + seen
if not subclasses:
return found
return _subclasses(subclasses, found)
return _subclasses([cls], [])

How can I find all subclasses of a class given its name?
We can certainly easily do this given access to the object itself, yes.
Simply given its name is a poor idea, as there can be multiple classes of the same name, even defined in the same module.
I created an implementation for another answer, and since it answers this question and it's a little more elegant than the other solutions here, here it is:
def get_subclasses(cls):
"""returns all subclasses of argument, cls"""
if issubclass(cls, type):
subclasses = cls.__subclasses__(cls)
else:
subclasses = cls.__subclasses__()
for subclass in subclasses:
subclasses.extend(get_subclasses(subclass))
return subclasses
Usage:
>>> import pprint
>>> list_of_classes = get_subclasses(int)
>>> pprint.pprint(list_of_classes)
[<class 'bool'>,
<enum 'IntEnum'>,
<enum 'IntFlag'>,
<class 'sre_constants._NamedIntConstant'>,
<class 'subprocess.Handle'>,
<enum '_ParameterKind'>,
<enum 'Signals'>,
<enum 'Handlers'>,
<enum 'RegexFlag'>]

This isn't as good an answer as using the special built-in __subclasses__() class method which #unutbu mentions, so I present it merely as an exercise. The subclasses() function defined returns a dictionary which maps all the subclass names to the subclasses themselves.
def traced_subclass(baseclass):
class _SubclassTracer(type):
def __new__(cls, classname, bases, classdict):
obj = type(classname, bases, classdict)
if baseclass in bases: # sanity check
attrname = '_%s__derived' % baseclass.__name__
derived = getattr(baseclass, attrname, {})
derived.update( {classname:obj} )
setattr(baseclass, attrname, derived)
return obj
return _SubclassTracer
def subclasses(baseclass):
attrname = '_%s__derived' % baseclass.__name__
return getattr(baseclass, attrname, None)
class BaseClass(object):
pass
class SubclassA(BaseClass):
__metaclass__ = traced_subclass(BaseClass)
class SubclassB(BaseClass):
__metaclass__ = traced_subclass(BaseClass)
print subclasses(BaseClass)
Output:
{'SubclassB': <class '__main__.SubclassB'>,
'SubclassA': <class '__main__.SubclassA'>}

While I'm very partial to the __init_subclass__ approach, this will preserve definition order, and avoid combinatorial order of growth if you have a very dense hierarchy with multiple inheritance everywhere:
def descendents(cls):
'''Does not return the class itself'''
R = {}
def visit(cls):
for subCls in cls.__subclasses__():
R[subCls] = True
visit(subCls)
visit(cls)
return list(R.keys())
This works because dictionaries remember the insertion order of their keys. A list approach would also work.

Python: Class instances are classes

Is it possible to create a class whose instances are classes?
At first it seemed possible, but I wondered if this would actually be possible without any external interference.
Eg:
NewClass = ClassClass(someArgs)
instance = NewClass(someOtherArgs)
Is this possible? If so, how would it structurally work? Is this even useful?

If you check the type of 1 you get type(1) == int. If you check the type of int you get type(int) == type.
A class whose instances are classes is called a metaclass and in Python you create metaclasses by subclassing type.
class ClassClass(type):
pass
Technically collections.namedtuple is a metaclass, but it's actually cheating.
There's a great PyCon talk by David Beazly about metaprogramming in Python. It's long, but very interesting. He starts talking about metaclasses around the 27 minute mark.

You're looking for metaclasses:
class Foo(type):
# Foo is a subclass of type and just like instances of type are
# classes, instances of Foo are classes.
pass
class Bar(object):
# The Bar class is an instance of Foo
__metaclass__ = Foo
# You can also create instances of Foo dynamically.
Bar = Foo("Bar", (object,), {})
More on metaclasses.

From the Python help:
type(name, bases, dict) -> a new type
So lets create a simple class:
Dog = type('Dog', (object,), {})
And now you can create a dog:
fido = Dog()
If you literally want a class that creates classes, you can do it, by extending type... Honestly I have no idea what you want it for, but here it is:
class ClassMaker(type):
def __new__(self, name):
return type(name, (object,), {})
NewClass = ClassMaker('NewClass')
print NewClass

In (modern) python, classes are first order objects. This is easy to verify by accessing a class' __class__ property:
class A:
pass
print(A.__class__)
-> <class 'type'>
Note that type's class is also type:
print(type.__class__)
-> <class 'type'>
It is possible to actually derive from type:
class A:
def do(self):
print("DOING")
class mytype(type):
def __new__(cls, *args, **kwargs):
return type.__new__(cls, 'mytype', cls.__bases__, {})
def __init__(self):
return type.__init__(self, 'mytype', self.__class__.__bases__, {})
def __call__(self, *args, **kwargs):
return A(*args, **kwargs)
aClass = mytype()
-> <class '__main__.mytype'>
aObj = aClass()
print(aObj)
-> <__main__.A object at 0xdeadbeaf>
aObj.do()
-> DOING
However, with proper metaclasses I do not see a good reason jumping through such hoops. Even metaclasses are only needed in special cases. Most often, a factory method that returns a class object is sufficient:
def ClassClass():
return A
aClass = ClassClass()
aObj = aClass()
Syntax is the same and if the only operation on the factory is instantiation, there is no difference.

In Python, class from which a given instance is inheriting a particular method

I know that in Python, given a class ClassA, with
inspect.getmembers(ClassA, predicate=inspect.ismethod)
I can iterate over the different methods present in ClassA. Inherited methods are also gathered, which is convenient in my case. But what I would need is, given a particular method method1 of ClassA, to get the class from which ClassA inherited method1. It might be ClassA itself, or any of its parents/grandparents. I thought I could recursively traverse the __bases__ attribute, looking for the method1 attribute at each step. But maybe this functionality is already implemented somewhere. Is there another way?

Look through the MRO (Method Resolution Order), using inspect.getmro() (which works on both old and new-style classes):
def class_for_method(cls, method):
return next((c for c in inspect.getmro(cls)
if method.__func__ in vars(c).values()), None)
There is currently no stdlib method to do this search for you, no.
Demo:
>>> import inspect
>>> def class_for_method(cls, method):
... return next((c for c in inspect.getmro(cls)
... if method.__func__ in vars(c).values()), None)
...
>>> class Base1(object):
... def foo(self): pass
...
>>> class Base2(object):
... pass
...
>>> class ClassA(Base1, Base2):
... pass
...
>>> class_for_method(ClassA, ClassA.foo)
<class '__main__.Base1'>
If no base class is found, the above expression returns None:
>>> class Bar:
... def spam(): pass
...
>>> class_for_method(ClassA, Bar.spam) is None
True

Get parent class name? [duplicate]

This question already has answers here:
Get class that defined method
(8 answers)
Closed 6 years ago.
The community reviewed whether to reopen this question 6 months ago and left it closed:
Original close reason(s) were not resolved
class A(object):
def get_class(self):
return self.__class__
class B(A):
def __init__(self):
A.__init__(self)
b = B()
print b.get_class()
This code will print <class '__main__.B'>.
How can I get the class name where the method has been defined (namely A)?

From the documentation: https://docs.python.org/2/reference/datamodel.html#the-standard-type-hierarchy
Class objects have a __name__ attribute. It might cleaner to introspect the base class(es) through the __bases__ attr of the derived class (if the code is to live in the derived class for example).
>>> class Base(object):
... pass
...
>>> class Derived(Base):
... def print_base(self):
... for base in self.__class__.__bases__:
... print base.__name__
...
>>> foo = Derived()
>>> foo.print_base()
Base

inspect.getmro(cls)
Return a tuple of class cls’s base classes,
including cls, in method resolution order. No class appears more than
once in this tuple. Note that the method resolution order depends on
cls’s type. Unless a very peculiar user-defined metatype is in use,
cls will be the first element of the tuple.
import inspect
inspect.getmro(B)
result will be:
(<class '__main__.B'>, <class '__main__.A'>, <type 'object'>)
First element is the class itself, second element is always first of the parents. After that things can get bit more complicated.

You could change
return self.__class__
return A().__class__
Since there is no other instance of A() available...

Python Reflection and callable objects

I have a two part question.
>>> class One(object):
... pass
...
>>> class Two(object):
... pass
...
>>> def digest(constr):
... c = apply(constr)
... print c.__class__.__name__
... print constr.__class__.__name__
...
>>> digest(Two)
Two
type
How would one create object 'Two'? Neither constr() or c() work; and it seems that apply turns it into a type.
What happens when you pass a class rather and an instance into a method?

Classes are high level objects, so you can simply pass them like this:
def createMyClass ( myClass ):
obj = myClass()
return obj
class A ( object ):
pass
>>> x = createMyClass( A )
>>> type( x )
<class '__main__.A'>

How would one create object 'Two'?
Neither constr() or c() work; and it
seems that apply turns it into a
type.
The above comment was made in regards to this code:
>>> def digest(constr):
... c = apply(constr)
... print c.__class__.__name__
... print constr.__class__.__name__
apply (deprecated: see #pyfunc's answer) certainly does not turn the class Two into a type: It already is one.
>>> class Two(object): pass
...
>>> type(Two)
<type 'type'>
Classes are first class objects: they're instances of type. This makes sense if you look at the next example.
>>> two = Two()
>>> type(two)
<class '__main__.Two'>
You can see that a class very clearly functions as a type because it can be returned from type. Here's another example.
>>> Three = type('Three', (Two, ), {'foo': 'bar'})
>>> type(Three)
<type 'type'>
>>> three = Three()
>>> type(three)
<class '__main__.Three'>
You can see that type is a class that can be instantiated. Its constructor takes three arguments: the name of the class, a tuple of base classes and a dictionary containing the class attributes. It returns a new type aka class.
As to your final question,
What happens when you pass a class
rather and an instance into a method?
You're going to have to be more specific. Classes are just instances of type and so are first class objects. Asking what happens if I pass a class into a method is like asking what happens if I pass an integer into a method: It depends entirely on what the method is expecting.

Just another one example:
def InstanceFactory(classname):
cls = globals()[classname]
return cls()
class A(object):
def start(self):
print "a.start"
class B(object):
def start(self):
print "b.start"
InstanceFactory("A").start()
InstanceFactory("B").start()
If the class belongs to another module:
def InstanceFactory(modulename, classname):
if '.' in modulename:
raise ValueError, "can't handle dotted modules yet"
mod = __import__(modulename)
cls = getattr(mod, classname]
return cls()

I am confused though. Wasn't apply() deprecated since 2.3
http://www.python.org/dev/peps/pep-0290/
We don't need this any more.
apply(f, args, kwds) --> f(*args, **kwds)
Others have been moved / considered deprecated in modern usage:
buffer()
coerce()
and intern()
Simply use : Classname() to create an object.