Develop Reference

How to dynamically change base class of instances at runtime?

This article has a snippet showing usage of __bases__ to dynamically change the inheritance hierarchy of some Python code, by adding a class to an existing classes collection of classes from which it inherits. Ok, that's hard to read, code is probably clearer:
class Friendly:
def hello(self):
print 'Hello'
class Person: pass
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
That is, Person doesn't inherit from Friendly at the source level, but rather this inheritance relation is added dynamically at runtime by modification of the __bases__attribute of the Person class. However, if you change Friendly and Person to be new style classes (by inheriting from object), you get the following error:
TypeError: __bases__ assignment: 'Friendly' deallocator differs from 'object'
A bit of Googling on this seems to indicate some incompatibilities between new-style and old style classes in regards to changing the inheritance hierarchy at runtime. Specifically: "New-style class objects don't support assignment to their bases attribute".
My question, is it possible to make the above Friendly/Person example work using new-style classes in Python 2.7+, possibly by use of the __mro__ attribute?
Disclaimer: I fully realise that this is obscure code. I fully realize that in real production code tricks like this tend to border on unreadable, this is purely a thought experiment, and for funzies to learn something about how Python deals with issues related to multiple inheritance.

Ok, again, this is not something you should normally do, this is for informational purposes only.
Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:
setattr(Person, '__mro__', (Person, Friendly, object))
The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.
A possible workaround is to simply redefine the class:
def modify_Person_to_be_friendly():
# so that we're modifying the global identifier 'Person'
global Person
# now just redefine the class using type(), specifying that the new
# class should inherit from Friendly and have all attributes from
# our old Person class
Person = type('Person', (Friendly,), dict(Person.__dict__))
def main():
modify_Person_to_be_friendly()
p = Person()
p.hello() # works!
What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):
def main():
oldperson = Person()
ModifyPersonToBeFriendly()
p = Person()
p.hello()
# works! But:
oldperson.hello()
# does not
If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.

I've been struggling with this too, and was intrigued by your solution, but Python 3 takes it away from us:
AttributeError: attribute '__dict__' of 'type' objects is not writable
I actually have a legitimate need for a decorator that replaces the (single) superclass of the decorated class. It would require too lengthy a description to include here (I tried, but couldn't get it to a reasonably length and limited complexity -- it came up in the context of the use by many Python applications of an Python-based enterprise server where different applications needed slightly different variations of some of the code.)
The discussion on this page and others like it provided hints that the problem of assigning to __bases__ only occurs for classes with no superclass defined (i.e., whose only superclass is object). I was able to solve this problem (for both Python 2.7 and 3.2) by defining the classes whose superclass I needed to replace as being subclasses of a trivial class:
## T is used so that the other classes are not direct subclasses of object,
## since classes whose base is object don't allow assignment to their __bases__ attribute.
class T: pass
class A(T):
def __init__(self):
print('Creating instance of {}'.format(self.__class__.__name__))
## ordinary inheritance
class B(A): pass
## dynamically specified inheritance
class C(T): pass
A() # -> Creating instance of A
B() # -> Creating instance of B
C.__bases__ = (A,)
C() # -> Creating instance of C
## attempt at dynamically specified inheritance starting with a direct subclass
## of object doesn't work
class D: pass
D.__bases__ = (A,)
D()
## Result is:
## TypeError: __bases__ assignment: 'A' deallocator differs from 'object'

I can not vouch for the consequences, but that this code does what you want at py2.7.2.
class Friendly(object):
def hello(self):
print 'Hello'
class Person(object): pass
# we can't change the original classes, so we replace them
class newFriendly: pass
newFriendly.__dict__ = dict(Friendly.__dict__)
Friendly = newFriendly
class newPerson: pass
newPerson.__dict__ = dict(Person.__dict__)
Person = newPerson
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
We know that this is possible. Cool. But we'll never use it!

Right of the bat, all the caveats of messing with class hierarchy dynamically are in effect.
But if it has to be done then, apparently, there is a hack that get's around the "deallocator differs from 'object" issue when modifying the __bases__ attribute for the new style classes.
You can define a class object
class Object(object): pass
Which derives a class from the built-in metaclass type.
That's it, now your new style classes can modify the __bases__ without any problem.
In my tests this actually worked very well as all existing (before changing the inheritance) instances of it and its derived classes felt the effect of the change including their mro getting updated.

I needed a solution for this which:
Works with both Python 2 (>= 2.7) and Python 3 (>= 3.2).
Lets the class bases be changed after dynamically importing a dependency.
Lets the class bases be changed from unit test code.
Works with types that have a custom metaclass.
Still allows unittest.mock.patch to function as expected.
Here's what I came up with:
def ensure_class_bases_begin_with(namespace, class_name, base_class):
""" Ensure the named class's bases start with the base class.
:param namespace: The namespace containing the class name.
:param class_name: The name of the class to alter.
:param base_class: The type to be the first base class for the
newly created type.
:return: ``None``.
Call this function after ensuring `base_class` is
available, before using the class named by `class_name`.
"""
existing_class = namespace[class_name]
assert isinstance(existing_class, type)
bases = list(existing_class.__bases__)
if base_class is bases[0]:
# Already bound to a type with the right bases.
return
bases.insert(0, base_class)
new_class_namespace = existing_class.__dict__.copy()
# Type creation will assign the correct ‘__dict__’ attribute.
del new_class_namespace['__dict__']
metaclass = existing_class.__metaclass__
new_class = metaclass(class_name, tuple(bases), new_class_namespace)
namespace[class_name] = new_class
Used like this within the application:
# foo.py
# Type `Bar` is not available at first, so can't inherit from it yet.
class Foo(object):
__metaclass__ = type
def __init__(self):
self.frob = "spam"
def __unicode__(self): return "Foo"
# … later …
import bar
ensure_class_bases_begin_with(
namespace=globals(),
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
Use like this from within unit test code:
# test_foo.py
""" Unit test for `foo` module. """
import unittest
import mock
import foo
import bar
ensure_class_bases_begin_with(
namespace=foo.__dict__,
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
class Foo_TestCase(unittest.TestCase):
""" Test cases for `Foo` class. """
def setUp(self):
patcher_unicode = mock.patch.object(
foo.Foo, '__unicode__')
patcher_unicode.start()
self.addCleanup(patcher_unicode.stop)
self.test_instance = foo.Foo()
patcher_frob = mock.patch.object(
self.test_instance, 'frob')
patcher_frob.start()
self.addCleanup(patcher_frob.stop)
def test_instantiate(self):
""" Should create an instance of `Foo`. """
instance = foo.Foo()

The above answers are good if you need to change an existing class at runtime. However, if you are just looking to create a new class that inherits by some other class, there is a much cleaner solution. I got this idea from https://stackoverflow.com/a/21060094/3533440, but I think the example below better illustrates a legitimate use case.
def make_default(Map, default_default=None):
"""Returns a class which behaves identically to the given
Map class, except it gives a default value for unknown keys."""
class DefaultMap(Map):
def __init__(self, default=default_default, **kwargs):
self._default = default
super().__init__(**kwargs)
def __missing__(self, key):
return self._default
return DefaultMap
DefaultDict = make_default(dict, default_default='wug')
d = DefaultDict(a=1, b=2)
assert d['a'] is 1
assert d['b'] is 2
assert d['c'] is 'wug'
Correct me if I'm wrong, but this strategy seems very readable to me, and I would use it in production code. This is very similar to functors in OCaml.

This method isn't technically inheriting during runtime, since __mro__ can't be changed. But what I'm doing here is using __getattr__ to be able to access any attributes or methods from a certain class. (Read comments in order of numbers placed before the comments, it makes more sense)
class Sub:
def __init__(self, f, cls):
self.f = f
self.cls = cls
# 6) this method will pass the self parameter
# (which is the original class object we passed)
# and then it will fill in the rest of the arguments
# using *args and **kwargs
def __call__(self, *args, **kwargs):
# 7) the multiple try / except statements
# are for making sure if an attribute was
# accessed instead of a function, the __call__
# method will just return the attribute
try:
return self.f(self.cls, *args, **kwargs)
except TypeError:
try:
return self.f(*args, **kwargs)
except TypeError:
return self.f
# 1) our base class
class S:
def __init__(self, func):
self.cls = func
def __getattr__(self, item):
# 5) we are wrapping the attribute we get in the Sub class
# so we can implement the __call__ method there
# to be able to pass the parameters in the correct order
return Sub(getattr(self.cls, item), self.cls)
# 2) class we want to inherit from
class L:
def run(self, s):
print("run" + s)
# 3) we create an instance of our base class
# and then pass an instance (or just the class object)
# as a parameter to this instance
s = S(L) # 4) in this case, I'm using the class object
s.run("1")
So this sort of substitution and redirection will simulate the inheritance of the class we wanted to inherit from. And it even works with attributes or methods that don't take any parameters.

Python: thinking of a module and its variables as a singleton — Clean approach?

I'd like to implement some sort of singleton pattern in my Python program. I was thinking of doing it without using classes; that is, I'd like to put all the singleton-related functions and variables within a module and consider it an actual singleton.
For example, say this is to be in the file 'singleton_module.py':
# singleton_module.py
# Singleton-related variables
foo = 'blah'
bar = 'stuff'
# Functions that process the above variables
def work(some_parameter):
global foo, bar
if some_parameter:
bar = ...
else:
foo = ...
Then, the rest of the program (i.e., other modules) would use this singleton like so:
# another_module.py
import singleton_module
# process the singleton variables,
# which changes them across the entire program
singleton_module.work(...)
# freely access the singleton variables
# (at least for reading)
print singleton_module.foo
This seemed to be a pretty good idea to me, because it looks pretty clean in the modules that use the singleton.
However, all these tedious 'global' statements in the singleton module are ugly. They occur in every function that processes the singleton-related variables. That's not much in this particular example, but when you have 10+ variables to manage across several functions, it's not pretty.
Also, this is pretty error-prone if you happen to forget the global statements: variables local to the function will be created, and the module's variables won't be changed, which is not what you want!
So, would this be considered to be clean? Is there an approach similar to mine that manages to do away with the 'global' mess?
Or is this simply not the way to go?

A common alternative to using a module as a singleton is Alex Martelli's Borg pattern:
class Borg:
__shared_state = {}
def __init__(self):
self.__dict__ = self.__shared_state
# and whatever else you want in your class -- that's all!
There can be multiple instances of this class, but they all share the same state.

Maybe you can put all the variables in a global dict, and you can directly use the dict in your functions without "global".
# Singleton-related variables
my_globals = {'foo': 'blah', 'bar':'stuff'}
# Functions that process the above variables
def work(some_parameter):
if some_parameter:
my_globals['bar'] = ...
else:
my_globals['foo'] = ...
why you can do it like this is Python Scopes and Namespaces.

One approach to implementing a singleton pattern with Python can also be:
have singleton __init()__ method raise an exception if an instance of the class already exists. More precisely, class has a member _single. If this member is different from None, exception is raised.
class Singleton:
__single = None
def __init__( self ):
if Singleton.__single:
raise Singleton.__single
Singleton.__single = self
It could be argued that handling the singleton instance creation with exceptions is not very clean also. We may hide implementation details with a method handle() as in
def Handle( x = Singleton ):
try:
single = x()
except Singleton, s:
single = s
return single
this Handle() method is very similar to what would be a C++ implementation of the Singleton pattern. We could have in Singleton class the handle()
Singleton& Singleton::Handle() {
if( !psingle ) {
psingle = new Singleton;
}
return *psingle;
}
returning either a new Singleton instance or a reference to the existing unique instance of class Singleton.
Handling the whole hierarchy
If Single1 and Single2 classes derive from Singleton, a single instance of Singleton through one of the derived class exists. This can be verify with this:
>>> child = S2( 'singlething' )
>>> junior = Handle( S1)
>>> junior.name()
'singlething'

Similar to Sven's "Borg pattern" suggestion, you could just keep all your state data in a class, without creating any instances of the class. This method utilizes new-style classes, I believe.
This method could even be adapted into the Borg pattern, with the caveat that modifying the state members from the instances of the class would require accessing the __class__ attribute of the instance (instance.__class__.foo = 'z' rather than instance.foo = 'z', though you could also just do stateclass.foo = 'z').
class State: # in some versions of Python, may need to be "class State():" or "class State(object):"
__slots__ = [] # prevents additional attributes from being added to instances and same-named attributes from shadowing the class's attributes
foo = 'x'
bar = 'y'
#classmethod
def work(cls, spam):
print(cls.foo, spam, cls.bar)
Note that modifications to the class's attributes will be reflected in instances of the class even after instantiation. This includes adding new attributes and removing existing ones, which could have some interesting, possibly useful effects (though I can also see how that might actually cause problems in some cases). Try it out yourself.

Building off of WillYang's answer and taking it a step further for cleanliness: define a simple class to hold your global dictionary to make it easier to reference:
class struct(dict):
def __init__(self, **kwargs):
dict.__init__(self, kwargs)
self.__dict__ = self
g = struct(var1=None, var2=None)
def func():
g.var1 = dict()
g.var3 = 10
g["var4"] = [1, 2]
print(g["var3"])
print(g.var4)
Just like before you put anything you want in g but now it's super clean. :)

For a legitimate Singleton:
class SingletonMeta(type):
__classes = {} # protect against defining class with the same name
def __new__(cls, cls_name, cls_ancestors, cls_dict):
if cls_name in cls.__classes:
return cls.__classes[cls_name]
type_instance = super(SingletonMeta, cls).__new__(cls, cls_name, cls_ancestors, cls_dict) # pass 'type' instead of 'cls' if you dont want SingletonMeta's attributes reflected in the class
return type_instance() # call __init__
class Singleton:
__metaclass__ = SingletonMeta
# define __init__ however you want
__call__(self, *args, *kwargs):
print 'hi!'
To see that it truly is a singleton, try to instantiate this class, or any class that inherits from it.
singleton = Singleton() # prints "hi!"

What is the purpose of python's inner classes?

Python's inner/nested classes confuse me. Is there something that can't be accomplished without them? If so, what is that thing?

Quoted from http://www.geekinterview.com/question_details/64739:
Advantages of inner class:
Logical grouping of classes: If a class is useful to only one other class then it is logical to embed it in that class and keep the two together. Nesting such "helper classes" makes their package more streamlined.
Increased encapsulation: Consider two top-level classes A and B where B needs access to members of A that would otherwise be declared private. By hiding class B within class A A's members can be declared private and B can access them. In addition B itself can be hidden from the outside world.
More readable, maintainable code: Nesting small classes within top-level classes places the code closer to where it is used.
The main advantage is organization. Anything that can be accomplished with inner classes can be accomplished without them.

Is there something that can't be accomplished without them?
No. They are absolutely equivalent to defining the class normally at top level, and then copying a reference to it into the outer class.
I don't think there's any special reason nested classes are ‘allowed’, other than it makes no particular sense to explicitly ‘disallow’ them either.
If you're looking for a class that exists within the lifecycle of the outer/owner object, and always has a reference to an instance of the outer class — inner classes as Java does it – then Python's nested classes are not that thing. But you can hack up something like that thing:
import weakref, new
class innerclass(object):
"""Descriptor for making inner classes.
Adds a property 'owner' to the inner class, pointing to the outer
owner instance.
"""
# Use a weakref dict to memoise previous results so that
# instance.Inner() always returns the same inner classobj.
#
def __init__(self, inner):
self.inner= inner
self.instances= weakref.WeakKeyDictionary()
# Not thread-safe - consider adding a lock.
#
def __get__(self, instance, _):
if instance is None:
return self.inner
if instance not in self.instances:
self.instances[instance]= new.classobj(
self.inner.__name__, (self.inner,), {'owner': instance}
)
return self.instances[instance]
# Using an inner class
#
class Outer(object):
#innerclass
class Inner(object):
def __repr__(self):
return '<%s.%s inner object of %r>' % (
self.owner.__class__.__name__,
self.__class__.__name__,
self.owner
)
>>> o1= Outer()
>>> o2= Outer()
>>> i1= o1.Inner()
>>> i1
<Outer.Inner inner object of <__main__.Outer object at 0x7fb2cd62de90>>
>>> isinstance(i1, Outer.Inner)
True
>>> isinstance(i1, o1.Inner)
True
>>> isinstance(i1, o2.Inner)
False
(This uses class decorators, which are new in Python 2.6 and 3.0. Otherwise you'd have to say “Inner= innerclass(Inner)” after the class definition.)

There's something you need to wrap your head around to be able to understand this. In most languages, class definitions are directives to the compiler. That is, the class is created before the program is ever run. In python, all statements are executable. That means that this statement:
class foo(object):
pass
is a statement that is executed at runtime just like this one:
x = y + z
This means that not only can you create classes within other classes, you can create classes anywhere you want to. Consider this code:
def foo():
class bar(object):
...
z = bar()
Thus, the idea of an "inner class" isn't really a language construct; it's a programmer construct. Guido has a very good summary of how this came about here. But essentially, the basic idea is this simplifies the language's grammar.

Nesting classes within classes:
Nested classes bloat the class definition making it harder to see whats going on.
Nested classes can create coupling that would make testing more difficult.
In Python you can put more than one class in a file/module, unlike Java, so the class still remains close to top level class and could even have the class name prefixed with an "_" to help signify that others shouldn't be using it.
The place where nested classes can prove useful is within functions
def some_func(a, b, c):
class SomeClass(a):
def some_method(self):
return b
SomeClass.__doc__ = c
return SomeClass
The class captures the values from the function allowing you to dynamically create a class like template metaprogramming in C++

I understand the arguments against nested classes, but there is a case for using them in some occasions. Imagine I'm creating a doubly-linked list class, and I need to create a node class for maintaing the nodes. I have two choices, create Node class inside the DoublyLinkedList class, or create the Node class outside the DoublyLinkedList class. I prefer the first choice in this case, because the Node class is only meaningful inside the DoublyLinkedList class. While there's no hiding/encapsulation benefit, there is a grouping benefit of being able to say the Node class is part of the DoublyLinkedList class.

Is there something that can't be accomplished without them? If so,
what is that thing?
There is something that cannot be easily done without: inheritance of related classes.
Here is a minimalist example with the related classes A and B:
class A(object):
class B(object):
def __init__(self, parent):
self.parent = parent
def make_B(self):
return self.B(self)
class AA(A): # Inheritance
class B(A.B): # Inheritance, same class name
pass
This code leads to a quite reasonable and predictable behaviour:
>>> type(A().make_B())
<class '__main__.A.B'>
>>> type(A().make_B().parent)
<class '__main__.A'>
>>> type(AA().make_B())
<class '__main__.AA.B'>
>>> type(AA().make_B().parent)
<class '__main__.AA'>
If B were a top-level class, you could not write self.B() in the method make_B but would simply write B(), and thus lose the dynamic binding to the adequate classes.
Note that in this construction, you should never refer to class A in the body of class B. This is the motivation for introducing the parent attribute in class B.
Of course, this dynamic binding can be recreated without inner class at the cost of a tedious and error-prone instrumentation of the classes.

1. Two functionally equivalent ways
The two ways shown before are functionally identical. However, there are some subtle differences, and there are situations when you would like to choose one over another.
Way 1: Nested class definition (="Nested class")
class MyOuter1:
class Inner:
def show(self, msg):
print(msg)
Way 2: With module level Inner class attached to Outer class(="Referenced inner class")
class _InnerClass:
def show(self, msg):
print(msg)
class MyOuter2:
Inner = _InnerClass
Underscore is used to follow PEP8 "internal interfaces (packages, modules, classes, functions, attributes or other names) should -- be prefixed with a single leading underscore."
2. Similarities
Below code snippet demonstrates the functional similarities of the "Nested class" vs "Referenced inner class"; They would behave the same way in code checking for the type of an inner class instance. Needless to say, the m.inner.anymethod() would behave similarly with m1 and m2
m1 = MyOuter1()
m2 = MyOuter2()
innercls1 = getattr(m1, 'Inner', None)
innercls2 = getattr(m2, 'Inner', None)
isinstance(innercls1(), MyOuter1.Inner)
# True
isinstance(innercls2(), MyOuter2.Inner)
# True
type(innercls1()) == mypackage.outer1.MyOuter1.Inner
# True (when part of mypackage)
type(innercls2()) == mypackage.outer2.MyOuter2.Inner
# True (when part of mypackage)
3. Differences
The differences of "Nested class" and "Referenced inner class" are listed below. They are not big, but sometimes you would like to choose one or the other based on these.
3.1 Code Encapsulation
With "Nested classes" it is possible to encapsulate code better than with "Referenced inner class". A class in the module namespace is a global variable. The purpose of nested classes is to reduce clutter in the module and put the inner class inside the outer class.
While no-one* is using from packagename import *, low amount of module level variables can be nice for example when using an IDE with code completion / intellisense.
*Right?
3.2 Readability of code
Django documentation instructs to use inner class Meta for model metadata. It is a bit more clearer* to instruct the framework users to write a class Foo(models.Model) with inner class Meta;
class Ox(models.Model):
horn_length = models.IntegerField()
class Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
instead of "write a class _Meta, then write a class Foo(models.Model) with Meta = _Meta";
class _Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
class Ox(models.Model):
Meta = _Meta
horn_length = models.IntegerField()
With the "Nested class" approach the code can be read a nested bullet point list, but with the "Referenced inner class" method one has to scroll back up to see the definition of _Meta to see its "child items" (attributes).
The "Referenced inner class" method can be more readable if your code nesting level grows or the rows are long for some other reason.
* Of course, a matter of taste
3.3 Slightly different error messages
This is not a big deal, but just for completeness: When accessing non-existent attribute for the inner class, we see slighly different exceptions. Continuing the example given in Section 2:
innercls1.foo()
# AttributeError: type object 'Inner' has no attribute 'foo'
innercls2.foo()
# AttributeError: type object '_InnerClass' has no attribute 'foo'
This is because the types of the inner classes are
type(innercls1())
#mypackage.outer1.MyOuter1.Inner
type(innercls2())
#mypackage.outer2._InnerClass

The main use case I use this for is the prevent proliferation of small modules and to prevent namespace pollution when separate modules are not needed. If I am extending an existing class, but that existing class must reference another subclass that should always be coupled to it. For example, I may have a utils.py module that has many helper classes in it, that aren't necessarily coupled together, but I want to reinforce coupling for some of those helper classes. For example, when I implement https://stackoverflow.com/a/8274307/2718295
:utils.py:
import json, decimal
class Helper1(object):
pass
class Helper2(object):
pass
# Here is the notorious JSONEncoder extension to serialize Decimals to JSON floats
class DecimalJSONEncoder(json.JSONEncoder):
class _repr_decimal(float): # Because float.__repr__ cannot be monkey patched
def __init__(self, obj):
self._obj = obj
def __repr__(self):
return '{:f}'.format(self._obj)
def default(self, obj): # override JSONEncoder.default
if isinstance(obj, decimal.Decimal):
return self._repr_decimal(obj)
# else
super(self.__class__, self).default(obj)
# could also have inherited from object and used return json.JSONEncoder.default(self, obj)
Then we can:
>>> from utils import DecimalJSONEncoder
>>> import json, decimal
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234'),
... 'key2':'strKey2Value'}, cls=DecimalJSONEncoder)
{"key2": "key2_value", "key_1": 1.12345678901234}
Of course, we could have eschewed inheriting json.JSONEnocder altogether and just override default():
:
import decimal, json
class Helper1(object):
pass
def json_encoder_decimal(obj):
class _repr_decimal(float):
...
if isinstance(obj, decimal.Decimal):
return _repr_decimal(obj)
return json.JSONEncoder(obj)
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234')}, default=json_decimal_encoder)
'{"key1": 1.12345678901234}'
But sometimes just for convention, you want utils to be composed of classes for extensibility.
Here's another use-case: I want a factory for mutables in my OuterClass without having to invoke copy:
class OuterClass(object):
class DTemplate(dict):
def __init__(self):
self.update({'key1': [1,2,3],
'key2': {'subkey': [4,5,6]})
def __init__(self):
self.outerclass_dict = {
'outerkey1': self.DTemplate(),
'outerkey2': self.DTemplate()}
obj = OuterClass()
obj.outerclass_dict['outerkey1']['key2']['subkey'].append(4)
assert obj.outerclass_dict['outerkey2']['key2']['subkey'] == [4,5,6]
I prefer this pattern over the #staticmethod decorator you would otherwise use for a factory function.

I have used Python's inner classes to create deliberately buggy subclasses within unittest functions (i.e. inside def test_something():) in order to get closer to 100% test coverage (e.g. testing very rarely triggered logging statements by overriding some methods).
In retrospect it's similar to Ed's answer https://stackoverflow.com/a/722036/1101109
Such inner classes should go out of scope and be ready for garbage collection once all references to them have been removed. For instance, take the following inner.py file:
class A(object):
pass
def scope():
class Buggy(A):
"""Do tests or something"""
assert isinstance(Buggy(), A)
I get the following curious results under OSX Python 2.7.6:
>>> from inner import A, scope
>>> A.__subclasses__()
[]
>>> scope()
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A, scope
>>> from inner import A
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A
>>> import gc
>>> gc.collect()
0
>>> gc.collect() # Yes I needed to call the gc twice, seems reproducible
3
>>> from inner import A
>>> A.__subclasses__()
[]
Hint - Don't go on and try doing this with Django models, which seemed to keep other (cached?) references to my buggy classes.
So in general, I wouldn't recommend using inner classes for this kind of purpose unless you really do value that 100% test coverage and can't use other methods. Though I think it's nice to be aware that if you use the __subclasses__(), that it can sometimes get polluted by inner classes. Either way if you followed this far, I think we're pretty deep into Python at this point, private dunderscores and all.

Printing all instances of a class

With a class in Python, how do I define a function to print every single instance of the class in a format defined in the function?

I see two options in this case:
Garbage collector
import gc
for obj in gc.get_objects():
if isinstance(obj, some_class):
dome_something(obj)
This has the disadvantage of being very slow when you have a lot of objects, but works with types over which you have no control.
Use a mixin and weakrefs
from collections import defaultdict
import weakref
class KeepRefs(object):
__refs__ = defaultdict(list)
def __init__(self):
self.__refs__[self.__class__].append(weakref.ref(self))
#classmethod
def get_instances(cls):
for inst_ref in cls.__refs__[cls]:
inst = inst_ref()
if inst is not None:
yield inst
class X(KeepRefs):
def __init__(self, name):
super(X, self).__init__()
self.name = name
x = X("x")
y = X("y")
for r in X.get_instances():
print r.name
del y
for r in X.get_instances():
print r.name
In this case, all the references get stored as a weak reference in a list. If you create and delete a lot of instances frequently, you should clean up the list of weakrefs after iteration, otherwise there's going to be a lot of cruft.
Another problem in this case is that you have to make sure to call the base class constructor. You could also override __new__, but only the __new__ method of the first base class is used on instantiation. This also works only on types that are under your control.
Edit: The method for printing all instances according to a specific format is left as an exercise, but it's basically just a variation on the for-loops.

You'll want to create a static list on your class, and add a weakref to each instance so the garbage collector can clean up your instances when they're no longer needed.
import weakref
class A:
instances = []
def __init__(self, name=None):
self.__class__.instances.append(weakref.proxy(self))
self.name = name
a1 = A('a1')
a2 = A('a2')
a3 = A('a3')
a4 = A('a4')
for instance in A.instances:
print(instance.name)

You don't need to import ANYTHING! Just use "self". Here's how you do this
class A:
instances = []
def __init__(self):
self.__class__.instances.append(self)
print('\n'.join(A.instances)) #this line was suggested by #anvelascos
It's this simple. No modules or libraries imported

Very nice and useful code, but it has a big problem: list is always bigger and it is never cleaned-up, to test it just add print(len(cls.__refs__[cls])) at the end of the get_instances method.
Here a fix for the get_instances method:
__refs__ = defaultdict(list)
#classmethod
def get_instances(cls):
refs = []
for ref in cls.__refs__[cls]:
instance = ref()
if instance is not None:
refs.append(ref)
yield instance
# print(len(refs))
cls.__refs__[cls] = refs
or alternatively it could be done using WeakSet:
from weakref import WeakSet
__refs__ = defaultdict(WeakSet)
#classmethod
def get_instances(cls):
return cls.__refs__[cls]

Same as almost all other OO languages, keep all instances of the class in a collection of some kind.
You can try this kind of thing.
class MyClassFactory( object ):
theWholeList= []
def __call__( self, *args, **kw ):
x= MyClass( *args, **kw )
self.theWholeList.append( x )
return x
Now you can do this.
object= MyClassFactory( args, ... )
print MyClassFactory.theWholeList

Python doesn't have an equivalent to Smallktalk's #allInstances as the architecture doesn't have this type of central object table (although modern smalltalks don't really work like that either).
As the other poster says, you have to explicitly manage a collection. His suggestion of a factory method that maintains a registry is a perfectly reasonable way to do it. You may wish to do something with weak references so you don't have to explicitly keep track of object disposal.

It's not clear if you need to print all class instances at once or when they're initialized, nor if you're talking about a class you have control over vs a class in a 3rd party library.
In any case, I would solve this by writing a class factory using Python metaclass support. If you don't have control over the class, manually update the __metaclass__ for the class or module you're tracking.
See http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html for more information.

In my project, I faced a similar problem and found a simple solution that may also work for you in listing and printing your class instances. The solution worked smoothly in Python version 3.7; gave partial errors in Python version 3.5.
I will copy-paste the relevant code blocks from my recent project.
```
instances = []
class WorkCalendar:
def __init__(self, day, patient, worker):
self.day = day
self.patient = patient
self.worker= worker
def __str__(self):
return f'{self.day} : {self.patient} : {self.worker}'
In Python the __str__ method in the end, determines how the object will be interpreted in its string form. I added the : in between the curly brackets, they are completely my preference for a "Pandas DataFrame" kind of reading. If you apply this small __str__ function, you will not be seeing some machine-readable object type descriptions- which makes no sense for human eyes. After adding this __str__ function you can append your objects to your list and print them as you wish.
appointment= WorkCalendar("01.10.2020", "Jane", "John")
instances.append(appointment)
For printing, your format in __str__ will work as default. But it is also possible to call all attributes separately:
for instance in instances:
print(instance)
print(instance.worker)
print(instance.patient)
For detailed reading, you may look at the source: https://dbader.org/blog/python-repr-vs-str

Is there a benefit to defining a class inside another class in Python?

What I'm talking about here are nested classes. Essentially, I have two classes that I'm modeling. A DownloadManager class and a DownloadThread class. The obvious OOP concept here is composition. However, composition doesn't necessarily mean nesting, right?
I have code that looks something like this:
class DownloadThread:
def foo(self):
pass
class DownloadManager():
def __init__(self):
dwld_threads = []
def create_new_thread():
dwld_threads.append(DownloadThread())
But now I'm wondering if there's a situation where nesting would be better. Something like:
class DownloadManager():
class DownloadThread:
def foo(self):
pass
def __init__(self):
dwld_threads = []
def create_new_thread():
dwld_threads.append(DownloadManager.DownloadThread())

You might want to do this when the "inner" class is a one-off, which will never be used outside the definition of the outer class. For example to use a metaclass, it's sometimes handy to do
class Foo(object):
class __metaclass__(type):
....
instead of defining a metaclass separately, if you're only using it once.
The only other time I've used nested classes like that, I used the outer class only as a namespace to group a bunch of closely related classes together:
class Group(object):
class cls1(object):
...
class cls2(object):
...
Then from another module, you can import Group and refer to these as Group.cls1, Group.cls2 etc. However one might argue that you can accomplish exactly the same (perhaps in a less confusing way) by using a module.

I don't know Python, but your question seems very general. Ignore me if it's specific to Python.
Class nesting is all about scope. If you think that one class will only make sense in the context of another one, then the former is probably a good candidate to become a nested class.
It is a common pattern make helper classes as private, nested classes.

There is another usage for nested class, when one wants to construct inherited classes whose enhanced functionalities are encapsulated in a specific nested class.
See this example:
class foo:
class bar:
... # functionalities of a specific sub-feature of foo
def __init__(self):
self.a = self.bar()
...
... # other features of foo
class foo2(foo):
class bar(foo.bar):
... # enhanced functionalities for this specific feature
def __init__(self):
foo.__init__(self)
Note that in the constructor of foo, the line self.a = self.bar() will construct a foo.bar when the object being constructed is actually a foo object, and a foo2.bar object when the object being constructed is actually a foo2 object.
If the class bar was defined outside of class foo instead, as well as its inherited version (which would be called bar2 for example), then defining the new class foo2 would be much more painful, because the constuctor of foo2 would need to have its first line replaced by self.a = bar2(), which implies re-writing the whole constructor.

You could be using a class as class generator. Like (in some off the cuff code :)
class gen(object):
class base_1(object): pass
...
class base_n(object): pass
def __init__(self, ...):
...
def mk_cls(self, ..., type):
'''makes a class based on the type passed in, the current state of
the class, and the other inputs to the method'''
I feel like when you need this functionality it will be very clear to you. If you don't need to be doing something similar than it probably isn't a good use case.

There is really no benefit to doing this, except if you are dealing with metaclasses.
the class: suite really isn't what you think it is. It is a weird scope, and it does strange things. It really doesn't even make a class! It is just a way of collecting some variables - the name of the class, the bases, a little dictionary of attributes, and a metaclass.
The name, the dictionary and the bases are all passed to the function that is the metaclass, and then it is assigned to the variable 'name' in the scope where the class: suite was.
What you can gain by messing with metaclasses, and indeed by nesting classes within your stock standard classes, is harder to read code, harder to understand code, and odd errors that are terribly difficult to understand without being intimately familiar with why the 'class' scope is entirely different to any other python scope.

A good use case for this feature is Error/Exception handling, e.g.:
class DownloadManager(object):
class DowndloadException(Exception):
pass
def download(self):
...
Now the one who is reading the code knows all the possible exceptions related to this class.

Either way, defined inside or outside of a class, would work. Here is an employee pay schedule program where the helper class EmpInit is embedded inside the class Employee:
class Employee:
def level(self, j):
return j * 5E3
def __init__(self, name, deg, yrs):
self.name = name
self.deg = deg
self.yrs = yrs
self.empInit = Employee.EmpInit(self.deg, self.level)
self.base = Employee.EmpInit(self.deg, self.level).pay
def pay(self):
if self.deg in self.base:
return self.base[self.deg]() + self.level(self.yrs)
print(f"Degree {self.deg} is not in the database {self.base.keys()}")
return 0
class EmpInit:
def __init__(self, deg, level):
self.level = level
self.j = deg
self.pay = {1: self.t1, 2: self.t2, 3: self.t3}
def t1(self): return self.level(1*self.j)
def t2(self): return self.level(2*self.j)
def t3(self): return self.level(3*self.j)
if __name__ == '__main__':
for loop in range(10):
lst = [item for item in input(f"Enter name, degree and years : ").split(' ')]
e1 = Employee(lst[0], int(lst[1]), int(lst[2]))
print(f'Employee {e1.name} with degree {e1.deg} and years {e1.yrs} is making {e1.pay()} dollars')
print("EmpInit deg {0}\nlevel {1}\npay[deg]: {2}".format(e1.empInit.j, e1.empInit.level, e1.base[e1.empInit.j]))
To define it outside, just un-indent EmpInit and change Employee.EmpInit() to simply EmpInit() as a regular "has-a" composition. However, since Employee is the controller of EmpInit and users don't instantiate or interface with it directly, it makes sense to define it inside as it is not a standalone class. Also note that the instance method level() is designed to be called in both classes here. Hence it can also be conveniently defined as a static method in Employee so that we don't need to pass it into EmpInit, instead just invoke it with Employee.level().