With a class in Python, how do I define a function to print every single instance of the class in a format defined in the function?
I see two options in this case:
Garbage collector
import gc
for obj in gc.get_objects():
if isinstance(obj, some_class):
dome_something(obj)
This has the disadvantage of being very slow when you have a lot of objects, but works with types over which you have no control.
Use a mixin and weakrefs
from collections import defaultdict
import weakref
class KeepRefs(object):
__refs__ = defaultdict(list)
def __init__(self):
self.__refs__[self.__class__].append(weakref.ref(self))
#classmethod
def get_instances(cls):
for inst_ref in cls.__refs__[cls]:
inst = inst_ref()
if inst is not None:
yield inst
class X(KeepRefs):
def __init__(self, name):
super(X, self).__init__()
self.name = name
x = X("x")
y = X("y")
for r in X.get_instances():
print r.name
del y
for r in X.get_instances():
print r.name
In this case, all the references get stored as a weak reference in a list. If you create and delete a lot of instances frequently, you should clean up the list of weakrefs after iteration, otherwise there's going to be a lot of cruft.
Another problem in this case is that you have to make sure to call the base class constructor. You could also override __new__, but only the __new__ method of the first base class is used on instantiation. This also works only on types that are under your control.
Edit: The method for printing all instances according to a specific format is left as an exercise, but it's basically just a variation on the for-loops.
You'll want to create a static list on your class, and add a weakref to each instance so the garbage collector can clean up your instances when they're no longer needed.
import weakref
class A:
instances = []
def __init__(self, name=None):
self.__class__.instances.append(weakref.proxy(self))
self.name = name
a1 = A('a1')
a2 = A('a2')
a3 = A('a3')
a4 = A('a4')
for instance in A.instances:
print(instance.name)
You don't need to import ANYTHING! Just use "self". Here's how you do this
class A:
instances = []
def __init__(self):
self.__class__.instances.append(self)
print('\n'.join(A.instances)) #this line was suggested by #anvelascos
It's this simple. No modules or libraries imported
Very nice and useful code, but it has a big problem: list is always bigger and it is never cleaned-up, to test it just add print(len(cls.__refs__[cls])) at the end of the get_instances method.
Here a fix for the get_instances method:
__refs__ = defaultdict(list)
#classmethod
def get_instances(cls):
refs = []
for ref in cls.__refs__[cls]:
instance = ref()
if instance is not None:
refs.append(ref)
yield instance
# print(len(refs))
cls.__refs__[cls] = refs
or alternatively it could be done using WeakSet:
from weakref import WeakSet
__refs__ = defaultdict(WeakSet)
#classmethod
def get_instances(cls):
return cls.__refs__[cls]
Same as almost all other OO languages, keep all instances of the class in a collection of some kind.
You can try this kind of thing.
class MyClassFactory( object ):
theWholeList= []
def __call__( self, *args, **kw ):
x= MyClass( *args, **kw )
self.theWholeList.append( x )
return x
Now you can do this.
object= MyClassFactory( args, ... )
print MyClassFactory.theWholeList
Python doesn't have an equivalent to Smallktalk's #allInstances as the architecture doesn't have this type of central object table (although modern smalltalks don't really work like that either).
As the other poster says, you have to explicitly manage a collection. His suggestion of a factory method that maintains a registry is a perfectly reasonable way to do it. You may wish to do something with weak references so you don't have to explicitly keep track of object disposal.
It's not clear if you need to print all class instances at once or when they're initialized, nor if you're talking about a class you have control over vs a class in a 3rd party library.
In any case, I would solve this by writing a class factory using Python metaclass support. If you don't have control over the class, manually update the __metaclass__ for the class or module you're tracking.
See http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html for more information.
In my project, I faced a similar problem and found a simple solution that may also work for you in listing and printing your class instances. The solution worked smoothly in Python version 3.7; gave partial errors in Python version 3.5.
I will copy-paste the relevant code blocks from my recent project.
```
instances = []
class WorkCalendar:
def __init__(self, day, patient, worker):
self.day = day
self.patient = patient
self.worker= worker
def __str__(self):
return f'{self.day} : {self.patient} : {self.worker}'
In Python the __str__ method in the end, determines how the object will be interpreted in its string form. I added the : in between the curly brackets, they are completely my preference for a "Pandas DataFrame" kind of reading. If you apply this small __str__ function, you will not be seeing some machine-readable object type descriptions- which makes no sense for human eyes. After adding this __str__ function you can append your objects to your list and print them as you wish.
appointment= WorkCalendar("01.10.2020", "Jane", "John")
instances.append(appointment)
For printing, your format in __str__ will work as default. But it is also possible to call all attributes separately:
for instance in instances:
print(instance)
print(instance.worker)
print(instance.patient)
For detailed reading, you may look at the source: https://dbader.org/blog/python-repr-vs-str
Related
For example—say I want to add a helloWorld() method to Python's dict type. Can I do this?
JavaScript has a prototype object that behaves this way. Maybe it's bad design and I should subclass the dict object, but then it only works on the subclasses and I want it to work on any and all future dictionaries.
Here's how it would go down in JavaScript:
String.prototype.hello = function() {
alert("Hello, " + this + "!");
}
"Jed".hello() //alerts "Hello, Jed!"
Here's a useful link with more examples— http://www.javascriptkit.com/javatutors/proto3.shtml
You can't directly add the method to the original type. However, you can subclass the type then substitute it in the built-in/global namespace, which achieves most of the effect desired. Unfortunately, objects created by literal syntax will continue to be of the vanilla type and won't have your new methods/attributes.
Here's what it looks like
# Built-in namespace
import __builtin__
# Extended subclass
class mystr(str):
def first_last(self):
if self:
return self[0] + self[-1]
else:
return ''
# Substitute the original str with the subclass on the built-in namespace
__builtin__.str = mystr
print str(1234).first_last()
print str(0).first_last()
print str('').first_last()
print '0'.first_last()
output = """
14
00
Traceback (most recent call last):
File "strp.py", line 16, in <module>
print '0'.first_last()
AttributeError: 'str' object has no attribute 'first_last'
"""
Just tried the forbbidenfruit!
here is the code, very simple!
from forbiddenfruit import curse
def list_size(self):
return len(self)
def string_hello(self):
print("Hello, {}".format(self))
if __name__ == "__main__":
curse(list, "size", list_size)
a = [1, 2, 3]
print(a.size())
curse(str, "hello", string_hello)
"Jesse".hello()
NOTE: this QA is marked as duplicate to this one, but IMO it asks for something different. I cannot answer there, so I am answering here.
Specifically, I wanted to inherit from str and add custom attributes. Existing answers (especially the ones saying you can't) didn't quite solve it, but this worked for me:
class TaggedString(str):
"""
A ``str`` with a ``.tags`` set and ``.kwtags`` dict of tags.
Usage example::
ts = TaggedString("hello world!", "greeting", "cliche",
what_am_i="h4cker")
(ts.upper(), ts.tags, ts.kwtags)
"""
def __new__(cls, *args, **kwargs):
return super().__new__(cls, args[0])
def __init__(self, s, *tags, **kwtags):
super().__init__()
self.tags = set(tags)
self.kwtags = kwtags
Hopefully this helps someone! Cheers,
Andres
Yes indeed, but you have to define a new class of the same type and it should inherit from that type.
For example:
class list(list):
def __init__(self, *args):
super().__init__(args)
def map(self, function):
return [function(i) for i in self]
a = list(1, 2, 3, 4, 5)
def double(i):
return i * 2
print(a.map(double))
Yes, by subclassing those types. See unifying types and classes in Python.
No, this doesn't mean that actual dicts will have this type, because that would be confusing. Subclassing a builtin type is the preferred way to add functionality.
class MyString:
def __init__(self, string):
self.string = string
def bigger_string(self):
print(' '.join(self.string))
mystring = MyString("this is the string")
mystring.bigger_string()
output
t h i s i s t h e s t r i n g
Dataclass in Python 3.7
from dataclasses import dataclass
#dataclass
class St:
text : str
def bigger(self) -> None:
self.text = list(self.text)
print(" ".join(self.text))
mys = St("Hello")
mys.bigger()
output
H e l l o
Yes, we can add custom methods and attributes to built-in python types. For example, let us say, you wanna define a new method inside the list class.
Let us think of defining a 'list' class and writing your own function like as follows :
class list:
def custom_method (self):
return("Hey, I'm a custom method of list class")
#lets create an object here
obj = list([1,2,3])
print(obj.custom_method())
#The above runs fine, but a list has append() method also right?? let's try it
print(obj.append(1))
"""Now you will get Attribute error : list object has no attribute append()"""
Because, when you define class having 'list' as class name, you will no longer be able to access the 'in-built list' class methods as 'list' is treated as a user-defined class rather than a inbuilt class.
So, in order to get rid of this error, you can inherit the properties/members of 'list' class and you can define own methods or attributes. So, in this way, you can call user-defined / in-built class methods using the same class name.
Here's how it looks :
#Extending in-built list class
class list(list):
def custom_method (self):
return("Hey, I'm a custom method of list class")
obj = list([1,2,3])
print(obj.custom_method())
obj.append(1)
print(obj)
It runs fine, and outputs modified list as [1,2,3,1].
NOTE : But when you do like this, it may create some ambiguity issues in long run like naming conflicts
For example, if you had a method having same signature that of an inbuilt function in user-defined class(say 'list' here), then it will be overridden without your knowledge or notice, thus you may not be able to use its original functionality in future. Considering the above code, if you ever define a method like append(self, value), the original functionality of append() will be lost.
So, it is better to use a different class name for your class name rather than same name as inbuilt class name
For example, you can declare a class like here as follows which does not raise any errors or you will not face any naming conflicts.
class custom_list(list):
def custom_method (self):
return("Hey, I'm a custom method of list class")
obj = custom_list([1,2,3])
print(obj.custom_method())
obj.append(1)
print(obj)
Subclassing is the way to go in Python. Polyglot programmers learn to use the right tool for the right situation - within reason. Something as artfully constructed as Rails (a DSL using Ruby) is painfully difficult to implement in a language with more rigid syntax like Python. People often compare the two saying how similar they are. The comparison is somewhat unfair. Python shines in its own ways. totochto.
For example—say I want to add a helloWorld() method to Python's dict type. Can I do this?
JavaScript has a prototype object that behaves this way. Maybe it's bad design and I should subclass the dict object, but then it only works on the subclasses and I want it to work on any and all future dictionaries.
Here's how it would go down in JavaScript:
String.prototype.hello = function() {
alert("Hello, " + this + "!");
}
"Jed".hello() //alerts "Hello, Jed!"
Here's a useful link with more examples— http://www.javascriptkit.com/javatutors/proto3.shtml
You can't directly add the method to the original type. However, you can subclass the type then substitute it in the built-in/global namespace, which achieves most of the effect desired. Unfortunately, objects created by literal syntax will continue to be of the vanilla type and won't have your new methods/attributes.
Here's what it looks like
# Built-in namespace
import __builtin__
# Extended subclass
class mystr(str):
def first_last(self):
if self:
return self[0] + self[-1]
else:
return ''
# Substitute the original str with the subclass on the built-in namespace
__builtin__.str = mystr
print str(1234).first_last()
print str(0).first_last()
print str('').first_last()
print '0'.first_last()
output = """
14
00
Traceback (most recent call last):
File "strp.py", line 16, in <module>
print '0'.first_last()
AttributeError: 'str' object has no attribute 'first_last'
"""
Just tried the forbbidenfruit!
here is the code, very simple!
from forbiddenfruit import curse
def list_size(self):
return len(self)
def string_hello(self):
print("Hello, {}".format(self))
if __name__ == "__main__":
curse(list, "size", list_size)
a = [1, 2, 3]
print(a.size())
curse(str, "hello", string_hello)
"Jesse".hello()
NOTE: this QA is marked as duplicate to this one, but IMO it asks for something different. I cannot answer there, so I am answering here.
Specifically, I wanted to inherit from str and add custom attributes. Existing answers (especially the ones saying you can't) didn't quite solve it, but this worked for me:
class TaggedString(str):
"""
A ``str`` with a ``.tags`` set and ``.kwtags`` dict of tags.
Usage example::
ts = TaggedString("hello world!", "greeting", "cliche",
what_am_i="h4cker")
(ts.upper(), ts.tags, ts.kwtags)
"""
def __new__(cls, *args, **kwargs):
return super().__new__(cls, args[0])
def __init__(self, s, *tags, **kwtags):
super().__init__()
self.tags = set(tags)
self.kwtags = kwtags
Hopefully this helps someone! Cheers,
Andres
Yes indeed, but you have to define a new class of the same type and it should inherit from that type.
For example:
class list(list):
def __init__(self, *args):
super().__init__(args)
def map(self, function):
return [function(i) for i in self]
a = list(1, 2, 3, 4, 5)
def double(i):
return i * 2
print(a.map(double))
Yes, by subclassing those types. See unifying types and classes in Python.
No, this doesn't mean that actual dicts will have this type, because that would be confusing. Subclassing a builtin type is the preferred way to add functionality.
class MyString:
def __init__(self, string):
self.string = string
def bigger_string(self):
print(' '.join(self.string))
mystring = MyString("this is the string")
mystring.bigger_string()
output
t h i s i s t h e s t r i n g
Dataclass in Python 3.7
from dataclasses import dataclass
#dataclass
class St:
text : str
def bigger(self) -> None:
self.text = list(self.text)
print(" ".join(self.text))
mys = St("Hello")
mys.bigger()
output
H e l l o
Yes, we can add custom methods and attributes to built-in python types. For example, let us say, you wanna define a new method inside the list class.
Let us think of defining a 'list' class and writing your own function like as follows :
class list:
def custom_method (self):
return("Hey, I'm a custom method of list class")
#lets create an object here
obj = list([1,2,3])
print(obj.custom_method())
#The above runs fine, but a list has append() method also right?? let's try it
print(obj.append(1))
"""Now you will get Attribute error : list object has no attribute append()"""
Because, when you define class having 'list' as class name, you will no longer be able to access the 'in-built list' class methods as 'list' is treated as a user-defined class rather than a inbuilt class.
So, in order to get rid of this error, you can inherit the properties/members of 'list' class and you can define own methods or attributes. So, in this way, you can call user-defined / in-built class methods using the same class name.
Here's how it looks :
#Extending in-built list class
class list(list):
def custom_method (self):
return("Hey, I'm a custom method of list class")
obj = list([1,2,3])
print(obj.custom_method())
obj.append(1)
print(obj)
It runs fine, and outputs modified list as [1,2,3,1].
NOTE : But when you do like this, it may create some ambiguity issues in long run like naming conflicts
For example, if you had a method having same signature that of an inbuilt function in user-defined class(say 'list' here), then it will be overridden without your knowledge or notice, thus you may not be able to use its original functionality in future. Considering the above code, if you ever define a method like append(self, value), the original functionality of append() will be lost.
So, it is better to use a different class name for your class name rather than same name as inbuilt class name
For example, you can declare a class like here as follows which does not raise any errors or you will not face any naming conflicts.
class custom_list(list):
def custom_method (self):
return("Hey, I'm a custom method of list class")
obj = custom_list([1,2,3])
print(obj.custom_method())
obj.append(1)
print(obj)
Subclassing is the way to go in Python. Polyglot programmers learn to use the right tool for the right situation - within reason. Something as artfully constructed as Rails (a DSL using Ruby) is painfully difficult to implement in a language with more rigid syntax like Python. People often compare the two saying how similar they are. The comparison is somewhat unfair. Python shines in its own ways. totochto.
This article has a snippet showing usage of __bases__ to dynamically change the inheritance hierarchy of some Python code, by adding a class to an existing classes collection of classes from which it inherits. Ok, that's hard to read, code is probably clearer:
class Friendly:
def hello(self):
print 'Hello'
class Person: pass
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
That is, Person doesn't inherit from Friendly at the source level, but rather this inheritance relation is added dynamically at runtime by modification of the __bases__attribute of the Person class. However, if you change Friendly and Person to be new style classes (by inheriting from object), you get the following error:
TypeError: __bases__ assignment: 'Friendly' deallocator differs from 'object'
A bit of Googling on this seems to indicate some incompatibilities between new-style and old style classes in regards to changing the inheritance hierarchy at runtime. Specifically: "New-style class objects don't support assignment to their bases attribute".
My question, is it possible to make the above Friendly/Person example work using new-style classes in Python 2.7+, possibly by use of the __mro__ attribute?
Disclaimer: I fully realise that this is obscure code. I fully realize that in real production code tricks like this tend to border on unreadable, this is purely a thought experiment, and for funzies to learn something about how Python deals with issues related to multiple inheritance.
Ok, again, this is not something you should normally do, this is for informational purposes only.
Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:
setattr(Person, '__mro__', (Person, Friendly, object))
The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.
A possible workaround is to simply redefine the class:
def modify_Person_to_be_friendly():
# so that we're modifying the global identifier 'Person'
global Person
# now just redefine the class using type(), specifying that the new
# class should inherit from Friendly and have all attributes from
# our old Person class
Person = type('Person', (Friendly,), dict(Person.__dict__))
def main():
modify_Person_to_be_friendly()
p = Person()
p.hello() # works!
What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):
def main():
oldperson = Person()
ModifyPersonToBeFriendly()
p = Person()
p.hello()
# works! But:
oldperson.hello()
# does not
If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.
I've been struggling with this too, and was intrigued by your solution, but Python 3 takes it away from us:
AttributeError: attribute '__dict__' of 'type' objects is not writable
I actually have a legitimate need for a decorator that replaces the (single) superclass of the decorated class. It would require too lengthy a description to include here (I tried, but couldn't get it to a reasonably length and limited complexity -- it came up in the context of the use by many Python applications of an Python-based enterprise server where different applications needed slightly different variations of some of the code.)
The discussion on this page and others like it provided hints that the problem of assigning to __bases__ only occurs for classes with no superclass defined (i.e., whose only superclass is object). I was able to solve this problem (for both Python 2.7 and 3.2) by defining the classes whose superclass I needed to replace as being subclasses of a trivial class:
## T is used so that the other classes are not direct subclasses of object,
## since classes whose base is object don't allow assignment to their __bases__ attribute.
class T: pass
class A(T):
def __init__(self):
print('Creating instance of {}'.format(self.__class__.__name__))
## ordinary inheritance
class B(A): pass
## dynamically specified inheritance
class C(T): pass
A() # -> Creating instance of A
B() # -> Creating instance of B
C.__bases__ = (A,)
C() # -> Creating instance of C
## attempt at dynamically specified inheritance starting with a direct subclass
## of object doesn't work
class D: pass
D.__bases__ = (A,)
D()
## Result is:
## TypeError: __bases__ assignment: 'A' deallocator differs from 'object'
I can not vouch for the consequences, but that this code does what you want at py2.7.2.
class Friendly(object):
def hello(self):
print 'Hello'
class Person(object): pass
# we can't change the original classes, so we replace them
class newFriendly: pass
newFriendly.__dict__ = dict(Friendly.__dict__)
Friendly = newFriendly
class newPerson: pass
newPerson.__dict__ = dict(Person.__dict__)
Person = newPerson
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
We know that this is possible. Cool. But we'll never use it!
Right of the bat, all the caveats of messing with class hierarchy dynamically are in effect.
But if it has to be done then, apparently, there is a hack that get's around the "deallocator differs from 'object" issue when modifying the __bases__ attribute for the new style classes.
You can define a class object
class Object(object): pass
Which derives a class from the built-in metaclass type.
That's it, now your new style classes can modify the __bases__ without any problem.
In my tests this actually worked very well as all existing (before changing the inheritance) instances of it and its derived classes felt the effect of the change including their mro getting updated.
I needed a solution for this which:
Works with both Python 2 (>= 2.7) and Python 3 (>= 3.2).
Lets the class bases be changed after dynamically importing a dependency.
Lets the class bases be changed from unit test code.
Works with types that have a custom metaclass.
Still allows unittest.mock.patch to function as expected.
Here's what I came up with:
def ensure_class_bases_begin_with(namespace, class_name, base_class):
""" Ensure the named class's bases start with the base class.
:param namespace: The namespace containing the class name.
:param class_name: The name of the class to alter.
:param base_class: The type to be the first base class for the
newly created type.
:return: ``None``.
Call this function after ensuring `base_class` is
available, before using the class named by `class_name`.
"""
existing_class = namespace[class_name]
assert isinstance(existing_class, type)
bases = list(existing_class.__bases__)
if base_class is bases[0]:
# Already bound to a type with the right bases.
return
bases.insert(0, base_class)
new_class_namespace = existing_class.__dict__.copy()
# Type creation will assign the correct ‘__dict__’ attribute.
del new_class_namespace['__dict__']
metaclass = existing_class.__metaclass__
new_class = metaclass(class_name, tuple(bases), new_class_namespace)
namespace[class_name] = new_class
Used like this within the application:
# foo.py
# Type `Bar` is not available at first, so can't inherit from it yet.
class Foo(object):
__metaclass__ = type
def __init__(self):
self.frob = "spam"
def __unicode__(self): return "Foo"
# … later …
import bar
ensure_class_bases_begin_with(
namespace=globals(),
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
Use like this from within unit test code:
# test_foo.py
""" Unit test for `foo` module. """
import unittest
import mock
import foo
import bar
ensure_class_bases_begin_with(
namespace=foo.__dict__,
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
class Foo_TestCase(unittest.TestCase):
""" Test cases for `Foo` class. """
def setUp(self):
patcher_unicode = mock.patch.object(
foo.Foo, '__unicode__')
patcher_unicode.start()
self.addCleanup(patcher_unicode.stop)
self.test_instance = foo.Foo()
patcher_frob = mock.patch.object(
self.test_instance, 'frob')
patcher_frob.start()
self.addCleanup(patcher_frob.stop)
def test_instantiate(self):
""" Should create an instance of `Foo`. """
instance = foo.Foo()
The above answers are good if you need to change an existing class at runtime. However, if you are just looking to create a new class that inherits by some other class, there is a much cleaner solution. I got this idea from https://stackoverflow.com/a/21060094/3533440, but I think the example below better illustrates a legitimate use case.
def make_default(Map, default_default=None):
"""Returns a class which behaves identically to the given
Map class, except it gives a default value for unknown keys."""
class DefaultMap(Map):
def __init__(self, default=default_default, **kwargs):
self._default = default
super().__init__(**kwargs)
def __missing__(self, key):
return self._default
return DefaultMap
DefaultDict = make_default(dict, default_default='wug')
d = DefaultDict(a=1, b=2)
assert d['a'] is 1
assert d['b'] is 2
assert d['c'] is 'wug'
Correct me if I'm wrong, but this strategy seems very readable to me, and I would use it in production code. This is very similar to functors in OCaml.
This method isn't technically inheriting during runtime, since __mro__ can't be changed. But what I'm doing here is using __getattr__ to be able to access any attributes or methods from a certain class. (Read comments in order of numbers placed before the comments, it makes more sense)
class Sub:
def __init__(self, f, cls):
self.f = f
self.cls = cls
# 6) this method will pass the self parameter
# (which is the original class object we passed)
# and then it will fill in the rest of the arguments
# using *args and **kwargs
def __call__(self, *args, **kwargs):
# 7) the multiple try / except statements
# are for making sure if an attribute was
# accessed instead of a function, the __call__
# method will just return the attribute
try:
return self.f(self.cls, *args, **kwargs)
except TypeError:
try:
return self.f(*args, **kwargs)
except TypeError:
return self.f
# 1) our base class
class S:
def __init__(self, func):
self.cls = func
def __getattr__(self, item):
# 5) we are wrapping the attribute we get in the Sub class
# so we can implement the __call__ method there
# to be able to pass the parameters in the correct order
return Sub(getattr(self.cls, item), self.cls)
# 2) class we want to inherit from
class L:
def run(self, s):
print("run" + s)
# 3) we create an instance of our base class
# and then pass an instance (or just the class object)
# as a parameter to this instance
s = S(L) # 4) in this case, I'm using the class object
s.run("1")
So this sort of substitution and redirection will simulate the inheritance of the class we wanted to inherit from. And it even works with attributes or methods that don't take any parameters.
I've seen a few "solutions" to this, but the solution every time seems to be "Don't use nested classes, define the classes outside and then use them normally". I don't like that answer, because it ignores the primary reason I chose nested classes, which is, to have a pool of constants (associated with the base class) accessible to all sub-class instances which are created.
Here is example code:
class ParentClass:
constant_pool = []
children = []
def __init__(self, stream):
self.constant_pool = ConstantPool(stream)
child_count = stream.read_ui16()
for i in range(0, child_count):
children.append(ChildClass(stream))
class ChildClass:
name = None
def __init__(self, stream):
idx = stream.read_ui16()
self.name = constant_pool[idx]
All classes are passed a single param, which is a custom bitstream class. My intention is to have a solution that does not require me to read the idx value for ChildClass while still in the ParentClass. All child-class stream reading should be done in the child class.
This example is over simplified. The constant pool is not the only variable i need available to all subclasses. The idx variable is not the only thing read from the stream reader.
Is this even possible in python? Is there no way to access the parent's information?
Despite my "bit patronizing" comment (fair play to call it that!), there are actually ways to achieve what you want: a different avenue of inheritance. A couple:
Write a decorator that introspects a class just after it's declared, finds inner classes, and copies attributes from the outer class into them.
Do the same thing with a metaclass.
Here's the decorator approach, since it's the most straightforward:
def matryoshka(cls):
# get types of classes
class classtypes:
pass
classtypes = (type, type(classtypes))
# get names of all public names in outer class
directory = [n for n in dir(cls) if not n.startswith("_")]
# get names of all non-callable attributes of outer class
attributes = [n for n in directory if not callable(getattr(cls, n))]
# get names of all inner classes
innerclasses = [n for n in directory if isinstance(getattr(cls, n), classtypes)]
# copy attributes from outer to inner classes (don't overwrite)
for c in innerclasses:
c = getattr(cls, c)
for a in attributes:
if not hasattr(c, a):
setattr(c, a, getattr(cls, a))
return cls
Here is a simple example of its use:
#matryoshka
class outer(object):
answer = 42
class inner(object):
def __call__(self):
print self.answer
outer.inner()() # 42
However, I can't help but think some of the ideas suggested in other answers would serve you better.
You don't need two classes here. Here's your example code written in a more concise fashion.
class ChildClass:
def __init__(self, stream):
idx = stream.read_ui16()
self.name = self.constant_pool[idx]
def makeChildren(stream):
ChildClass.constant_pool = ConstantPool(stream)
return [ChildClass(stream) for i in range(stream.read_ui16())]
Welcome to Python. Classes are mutable at runtime. Enjoy.
You can access the parent class through its name:
class ChildClass:
name = None
def __init__(self, stream):
idx = stream.read_ui16()
self.name = ParentClass.constant_pool[idx]
Then again, I'm not sure I understand what you are trying to achieve.
Another alternative design to consider:
When you find yourself trying to use classes as namespaces, it might make more sense to put the inner classes into a module of their own and make what were the attributes of the outer class global variables. In other words, if you never intend to instantiate your ParentClass, then it's just serving as a glorified module.
Global variables get a bad rap in most programming languages, but they are not truly global in Python, and are nicely encapsulated to the module.
Well, the following works (further simplified from your example). Note that you don't have to "declare" member variables at class level like C++/C#/Java etc, just set them on self within __init__:
class ParentClass:
def __init__(self):
self.constant_pool = ["test"]
self.ChildClass.constant_pool = self.constant_pool
self.children = [self.ChildClass()]
class ChildClass:
def __init__(self):
self.name = self.constant_pool[0]
print "child name is", self.name
p = ParentClass() # Prints "child name is test"
Note that you could still do the same sort of thing without the child classes being nested.
Your question uses the word subclass, so I'm keying from that to interpret your question. As with the others who have answered, I am not certain I understand what you are looking for.
class ParentClass(object):
constant_pool = [c1, c2, c3]
def __init__(self):
# anything not included in your question
class ChildClass(ParentClass):
def __init__(self, stream):
ParentClass.__init__(self)
self.name = ParentClass.constant_pool[stream.read_ui16()]
stream = get_new_stream()
children = []
for count in range(stream.read_ui16()):
children.append(ChildClass(stream))
This code uses inheritance to derive ChildClass from ParentClass (and all methods, etc). The constant_pool is an attribute of ParentClass itself, though it is OK to treat as an attribute of any instance of ParentClass or ChildClass (saying self.constant_pool within ChildClass.__init__ would be equivalent to the above but, in my view, misleading).
Nesting the class definitions is not necessary. Nesting the definition of ChildClass within ParentClass just means that ChildClass is an attribute of ParentClass, nothing more. It does not make instances of ChildClass inherit anything from ParentClass.
Python's inner/nested classes confuse me. Is there something that can't be accomplished without them? If so, what is that thing?
Quoted from http://www.geekinterview.com/question_details/64739:
Advantages of inner class:
Logical grouping of classes: If a class is useful to only one other class then it is logical to embed it in that class and keep the two together. Nesting such "helper classes" makes their package more streamlined.
Increased encapsulation: Consider two top-level classes A and B where B needs access to members of A that would otherwise be declared private. By hiding class B within class A A's members can be declared private and B can access them. In addition B itself can be hidden from the outside world.
More readable, maintainable code: Nesting small classes within top-level classes places the code closer to where it is used.
The main advantage is organization. Anything that can be accomplished with inner classes can be accomplished without them.
Is there something that can't be accomplished without them?
No. They are absolutely equivalent to defining the class normally at top level, and then copying a reference to it into the outer class.
I don't think there's any special reason nested classes are ‘allowed’, other than it makes no particular sense to explicitly ‘disallow’ them either.
If you're looking for a class that exists within the lifecycle of the outer/owner object, and always has a reference to an instance of the outer class — inner classes as Java does it – then Python's nested classes are not that thing. But you can hack up something like that thing:
import weakref, new
class innerclass(object):
"""Descriptor for making inner classes.
Adds a property 'owner' to the inner class, pointing to the outer
owner instance.
"""
# Use a weakref dict to memoise previous results so that
# instance.Inner() always returns the same inner classobj.
#
def __init__(self, inner):
self.inner= inner
self.instances= weakref.WeakKeyDictionary()
# Not thread-safe - consider adding a lock.
#
def __get__(self, instance, _):
if instance is None:
return self.inner
if instance not in self.instances:
self.instances[instance]= new.classobj(
self.inner.__name__, (self.inner,), {'owner': instance}
)
return self.instances[instance]
# Using an inner class
#
class Outer(object):
#innerclass
class Inner(object):
def __repr__(self):
return '<%s.%s inner object of %r>' % (
self.owner.__class__.__name__,
self.__class__.__name__,
self.owner
)
>>> o1= Outer()
>>> o2= Outer()
>>> i1= o1.Inner()
>>> i1
<Outer.Inner inner object of <__main__.Outer object at 0x7fb2cd62de90>>
>>> isinstance(i1, Outer.Inner)
True
>>> isinstance(i1, o1.Inner)
True
>>> isinstance(i1, o2.Inner)
False
(This uses class decorators, which are new in Python 2.6 and 3.0. Otherwise you'd have to say “Inner= innerclass(Inner)” after the class definition.)
There's something you need to wrap your head around to be able to understand this. In most languages, class definitions are directives to the compiler. That is, the class is created before the program is ever run. In python, all statements are executable. That means that this statement:
class foo(object):
pass
is a statement that is executed at runtime just like this one:
x = y + z
This means that not only can you create classes within other classes, you can create classes anywhere you want to. Consider this code:
def foo():
class bar(object):
...
z = bar()
Thus, the idea of an "inner class" isn't really a language construct; it's a programmer construct. Guido has a very good summary of how this came about here. But essentially, the basic idea is this simplifies the language's grammar.
Nesting classes within classes:
Nested classes bloat the class definition making it harder to see whats going on.
Nested classes can create coupling that would make testing more difficult.
In Python you can put more than one class in a file/module, unlike Java, so the class still remains close to top level class and could even have the class name prefixed with an "_" to help signify that others shouldn't be using it.
The place where nested classes can prove useful is within functions
def some_func(a, b, c):
class SomeClass(a):
def some_method(self):
return b
SomeClass.__doc__ = c
return SomeClass
The class captures the values from the function allowing you to dynamically create a class like template metaprogramming in C++
I understand the arguments against nested classes, but there is a case for using them in some occasions. Imagine I'm creating a doubly-linked list class, and I need to create a node class for maintaing the nodes. I have two choices, create Node class inside the DoublyLinkedList class, or create the Node class outside the DoublyLinkedList class. I prefer the first choice in this case, because the Node class is only meaningful inside the DoublyLinkedList class. While there's no hiding/encapsulation benefit, there is a grouping benefit of being able to say the Node class is part of the DoublyLinkedList class.
Is there something that can't be accomplished without them? If so,
what is that thing?
There is something that cannot be easily done without: inheritance of related classes.
Here is a minimalist example with the related classes A and B:
class A(object):
class B(object):
def __init__(self, parent):
self.parent = parent
def make_B(self):
return self.B(self)
class AA(A): # Inheritance
class B(A.B): # Inheritance, same class name
pass
This code leads to a quite reasonable and predictable behaviour:
>>> type(A().make_B())
<class '__main__.A.B'>
>>> type(A().make_B().parent)
<class '__main__.A'>
>>> type(AA().make_B())
<class '__main__.AA.B'>
>>> type(AA().make_B().parent)
<class '__main__.AA'>
If B were a top-level class, you could not write self.B() in the method make_B but would simply write B(), and thus lose the dynamic binding to the adequate classes.
Note that in this construction, you should never refer to class A in the body of class B. This is the motivation for introducing the parent attribute in class B.
Of course, this dynamic binding can be recreated without inner class at the cost of a tedious and error-prone instrumentation of the classes.
1. Two functionally equivalent ways
The two ways shown before are functionally identical. However, there are some subtle differences, and there are situations when you would like to choose one over another.
Way 1: Nested class definition (="Nested class")
class MyOuter1:
class Inner:
def show(self, msg):
print(msg)
Way 2: With module level Inner class attached to Outer class(="Referenced inner class")
class _InnerClass:
def show(self, msg):
print(msg)
class MyOuter2:
Inner = _InnerClass
Underscore is used to follow PEP8 "internal interfaces (packages, modules, classes, functions, attributes or other names) should -- be prefixed with a single leading underscore."
2. Similarities
Below code snippet demonstrates the functional similarities of the "Nested class" vs "Referenced inner class"; They would behave the same way in code checking for the type of an inner class instance. Needless to say, the m.inner.anymethod() would behave similarly with m1 and m2
m1 = MyOuter1()
m2 = MyOuter2()
innercls1 = getattr(m1, 'Inner', None)
innercls2 = getattr(m2, 'Inner', None)
isinstance(innercls1(), MyOuter1.Inner)
# True
isinstance(innercls2(), MyOuter2.Inner)
# True
type(innercls1()) == mypackage.outer1.MyOuter1.Inner
# True (when part of mypackage)
type(innercls2()) == mypackage.outer2.MyOuter2.Inner
# True (when part of mypackage)
3. Differences
The differences of "Nested class" and "Referenced inner class" are listed below. They are not big, but sometimes you would like to choose one or the other based on these.
3.1 Code Encapsulation
With "Nested classes" it is possible to encapsulate code better than with "Referenced inner class". A class in the module namespace is a global variable. The purpose of nested classes is to reduce clutter in the module and put the inner class inside the outer class.
While no-one* is using from packagename import *, low amount of module level variables can be nice for example when using an IDE with code completion / intellisense.
*Right?
3.2 Readability of code
Django documentation instructs to use inner class Meta for model metadata. It is a bit more clearer* to instruct the framework users to write a class Foo(models.Model) with inner class Meta;
class Ox(models.Model):
horn_length = models.IntegerField()
class Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
instead of "write a class _Meta, then write a class Foo(models.Model) with Meta = _Meta";
class _Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
class Ox(models.Model):
Meta = _Meta
horn_length = models.IntegerField()
With the "Nested class" approach the code can be read a nested bullet point list, but with the "Referenced inner class" method one has to scroll back up to see the definition of _Meta to see its "child items" (attributes).
The "Referenced inner class" method can be more readable if your code nesting level grows or the rows are long for some other reason.
* Of course, a matter of taste
3.3 Slightly different error messages
This is not a big deal, but just for completeness: When accessing non-existent attribute for the inner class, we see slighly different exceptions. Continuing the example given in Section 2:
innercls1.foo()
# AttributeError: type object 'Inner' has no attribute 'foo'
innercls2.foo()
# AttributeError: type object '_InnerClass' has no attribute 'foo'
This is because the types of the inner classes are
type(innercls1())
#mypackage.outer1.MyOuter1.Inner
type(innercls2())
#mypackage.outer2._InnerClass
The main use case I use this for is the prevent proliferation of small modules and to prevent namespace pollution when separate modules are not needed. If I am extending an existing class, but that existing class must reference another subclass that should always be coupled to it. For example, I may have a utils.py module that has many helper classes in it, that aren't necessarily coupled together, but I want to reinforce coupling for some of those helper classes. For example, when I implement https://stackoverflow.com/a/8274307/2718295
:utils.py:
import json, decimal
class Helper1(object):
pass
class Helper2(object):
pass
# Here is the notorious JSONEncoder extension to serialize Decimals to JSON floats
class DecimalJSONEncoder(json.JSONEncoder):
class _repr_decimal(float): # Because float.__repr__ cannot be monkey patched
def __init__(self, obj):
self._obj = obj
def __repr__(self):
return '{:f}'.format(self._obj)
def default(self, obj): # override JSONEncoder.default
if isinstance(obj, decimal.Decimal):
return self._repr_decimal(obj)
# else
super(self.__class__, self).default(obj)
# could also have inherited from object and used return json.JSONEncoder.default(self, obj)
Then we can:
>>> from utils import DecimalJSONEncoder
>>> import json, decimal
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234'),
... 'key2':'strKey2Value'}, cls=DecimalJSONEncoder)
{"key2": "key2_value", "key_1": 1.12345678901234}
Of course, we could have eschewed inheriting json.JSONEnocder altogether and just override default():
:
import decimal, json
class Helper1(object):
pass
def json_encoder_decimal(obj):
class _repr_decimal(float):
...
if isinstance(obj, decimal.Decimal):
return _repr_decimal(obj)
return json.JSONEncoder(obj)
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234')}, default=json_decimal_encoder)
'{"key1": 1.12345678901234}'
But sometimes just for convention, you want utils to be composed of classes for extensibility.
Here's another use-case: I want a factory for mutables in my OuterClass without having to invoke copy:
class OuterClass(object):
class DTemplate(dict):
def __init__(self):
self.update({'key1': [1,2,3],
'key2': {'subkey': [4,5,6]})
def __init__(self):
self.outerclass_dict = {
'outerkey1': self.DTemplate(),
'outerkey2': self.DTemplate()}
obj = OuterClass()
obj.outerclass_dict['outerkey1']['key2']['subkey'].append(4)
assert obj.outerclass_dict['outerkey2']['key2']['subkey'] == [4,5,6]
I prefer this pattern over the #staticmethod decorator you would otherwise use for a factory function.
I have used Python's inner classes to create deliberately buggy subclasses within unittest functions (i.e. inside def test_something():) in order to get closer to 100% test coverage (e.g. testing very rarely triggered logging statements by overriding some methods).
In retrospect it's similar to Ed's answer https://stackoverflow.com/a/722036/1101109
Such inner classes should go out of scope and be ready for garbage collection once all references to them have been removed. For instance, take the following inner.py file:
class A(object):
pass
def scope():
class Buggy(A):
"""Do tests or something"""
assert isinstance(Buggy(), A)
I get the following curious results under OSX Python 2.7.6:
>>> from inner import A, scope
>>> A.__subclasses__()
[]
>>> scope()
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A, scope
>>> from inner import A
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A
>>> import gc
>>> gc.collect()
0
>>> gc.collect() # Yes I needed to call the gc twice, seems reproducible
3
>>> from inner import A
>>> A.__subclasses__()
[]
Hint - Don't go on and try doing this with Django models, which seemed to keep other (cached?) references to my buggy classes.
So in general, I wouldn't recommend using inner classes for this kind of purpose unless you really do value that 100% test coverage and can't use other methods. Though I think it's nice to be aware that if you use the __subclasses__(), that it can sometimes get polluted by inner classes. Either way if you followed this far, I think we're pretty deep into Python at this point, private dunderscores and all.