Like the question posted here, I want to create a class that inherits from another class passed as an argument.
class A():
def __init__(self, args):
stuff
class B():
def __init__(self, args):
stuff
class C():
def __init__(self, cls, args):
self.inherit(cls, args)
args = #arguments to create instances of A and B
class_from_A = C(A, args) #instance of C inherited from A
class_from_B = C(B, args) #instance of C inherited from B
I want to do this so that I can keep track of calls I make to different web api's. The thought is that I am just adding my own functionality to any api-type object. The problem with the solution to the linked question is that I don't want to have to go through the additional "layer" to use the api-type object. I want to say obj.get_data() instead of obj.api.get_data().
I've tried looking into how super() works but haven't came across anything that would help (although I could've easily missed something). Any help would be nice, and I'm open to any other approaches for what I'm trying to do, however, just out of curiosity I'd like to know if this is possible.
I don't think it's possible because __init__ is called after __new__ which is where you would specify base classes, but I think you can achieve your goal of tracking api calls using a metaclass. Since you didn't give any examples of what tracking the calls means, I'll leave you with an example metaclass which counts method calls. You can adapt it to your needs.
Another alternative would be to subclass A and B with methods that track whatever you need, and just return super().whatever(). I think I'd prefer that method unless A and B contain too many methods worth managing like that.
Here's an implementation from python-course.eu, by Bernd Klein. Click the link for more detail.
class FuncCallCounter(type):
""" A Metaclass which decorates all the methods of the
subclass using call_counter as the decorator
"""
#staticmethod
def call_counter(func):
""" Decorator for counting the number of function
or method calls to the function or method func
"""
def helper(*args, **kwargs):
helper.calls += 1
return func(*args, **kwargs)
helper.calls = 0
helper.__name__= func.__name__
return helper
def __new__(cls, clsname, superclasses, attributedict):
""" Every method gets decorated with the decorator call_counter,
which will do the actual call counting
"""
for attr in attributedict:
if callable(attributedict[attr]) and not attr.startswith("__"):
attributedict[attr] = cls.call_counter(attributedict[attr])
return type.__new__(cls, clsname, superclasses, attributedict)
I work on a project with a huge class. Initially they were implemented as functions that just get imported, like in this answer:
def plot(self, x, y):
print(self.field)
def clear(self):
# body
#classmethod
def from_file(cls, path):
# body
class Fitter(object):
def __init__(self, whatever):
self.field = whatever
# Imported methods
from .some_file import plot, clear, from_file
...
But I think it's not the best solution, IDE is mad on the code because it cannot find field in the external methods and considers classmethod on some functions as an error. I hope mixins can help with it.
But I see the similar problem in this approach: mixin classes don't have a base class with all the common methods and fields specified (an example in Django), so IDE and linters cannot find definitions and understand the code correctly too... I tried to use some common base class, as the following:
class FitterBase(object):
def __init__(self, whatever):
self.field = whatever
class FiterMixin(FitterBase):
def plot(self, x, y):
print(self.field)
def clear(self):
# body
#classmethod
def from_file(cls, path):
# body
class Fitter(FitterBase, FiterMixin):
pass
But the interpreter raises an error:
TypeError: Cannot create a consistent method resolution
Is there any solution to this problem? It's really important because the class contains dozens of big methods, and correct inheritance would help a lot.
Python is trying to construct an MRO for Fitter in which FitterBase both precedes and follows FitterMixin; the former because FitterBase is listed first in the seance of base classes, the latter because it is the parent of FitterMixin.
To resolve that issue, simply swap the order of the two base classes:
class Fitter(FitterMixin, FitterBase):
pass
There's no reason to list both, though, because FitterMixin already inherits from FitterBase:
class Fitter(FitterMixin):
pass
As this makes more obvious, FitterMixin isn't really a mix-in class, because you aren't mixing it with anything. Or, don't make FitterMixin subclass FitterBase:
class FitterBase:
def __init__(self, whatever):
self.field = whatever
class FitterMixin:
def plot(self, x, y):
print(self.field)
def clear(self):
pass
#classmethod
def from_file(cls, path):
pass
class Fitter(FitterBase, FitterMixin):
pass
I have a python class Foo that registers observers to an observable.
I have several classes that inherits from Foo and populate the list of observers, each with different observers.
For reasons related to the current design, the list of observers is initialized explicitly in every derived class __init__ and cannot be moved to an overridable method (that would be called from the base class __init__).
As a result I can't call observable.RegisterObservers in the constructor (because the list of observers is not populated yet).
To make sure that the observers are registered, I created the observable as cached property. This way I can guarantee that no one will access the field directly without registering the observers first.
The first call to my_observable should have happened after init is done.
The problem is that there is an inspect.getmembers call at the end of the base class init.
inspect.getmembers() uses getattr, and by that evaluates the cached_property.
I tried to throw AttributeError if some condition is not met (i.e. we are still in the init), but I don't like this solution, and it would fail if someone will implement the derived class in a different way.
Is there a way to tell inspect to skip cached properties?
Also ok to have a trusted way to tell if init (of derived classes) is done.
Pseudo code:
class Foo(object):
def __init__(self):
self._observers = []
for _, member in inspect.getmembers(self):
# do something with member
continue
#cached_property
def my_observable:
observable = Observable()
observable.RegisterObservers(self._observers)
return observable
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self._observers = [MyObserver()]
Found a solution for my problem.
It doesn't solve the inspect.getmembers evaluating my cached property, but it actually solves the design issue in a much better way.
I will define a meta class that will override the call and will call a specific method on the class.
overriding call in a metaclass lets us inject that step just before the new instance is returned.
So it will look something like this:
class MyMetaClass(type):
def __call__(cls, *args, **kwargs):
"""Called when the object is initialized."""
obj = type.__call__(cls, *args, **kwargs)
obj.FinalizeInit()
return obj
class Foo(object):
__metaclass__ = MyMetaClass
def __init__(self):
self._observers = []
for _, member in inspect.getmembers(self):
# do something with member
continue
self._observable = None
def FinalizeInit(self):
self._observable = Observable()
self._observable.RegisterObservers(self._observers)
I'm trying to wrap my head around how to utilize inheritance in some code I'm writing for an API. I have the following parent class which holds a bunch of common variables that I'd like to instantiate once, and inherit with other classes to make my code look cleaner:
class ApiCommon(object):
def __init__(self, _apikey, _serviceid=None, _vclversion=None,
_aclname=None, _aclid=None):
self.BaseApiUrl = "https://api.fastly.com"
self.APIKey = _apikey
self.headers = {'Fastly-Key': self.APIKey}
self.ServiceID = _serviceid
self.VCLVersion = _vclversion
self.ACLName = _aclname
self.ACLid = _aclid
self.Data = None
self.IP = None
self.CIDR = None
self.fullurl = None
self.r = None
self.jsonresp = None
self.ACLcomment = None
self.ACLentryid = None
And I am inheriting it in another class below, like so in a lib file called lib/security.py:
from apicommon import ApiCommon
class EdgeAclControl(ApiCommon):
def __init__(self):
super(EdgeAclControl, self).__init__()
...
def somemethodhere(self):
return 'stuff'
When I instantiate an object for ApiCommon(object), I can't access the methods in EdgeAclControl(ApiCommon). Example of what I'm trying which isn't working:
from lib import security
gza = security.ApiCommon(_aclname='pytest', _apikey='mykey',
_serviceid='stuffhere', _vclversion=5)
gza.somemethodhere()
How would I instantiate ApiCommon and have access to the methods in EdgeAclControl?
Your current code appears to be trying to use inheritance backwards. When you create an instance of ApiCommon, it will only get the methods defined in that base class. If you want to get methods from a subclass, you need to create an instance of the subclass instead.
So the first fix you need to make is to change gza = security.ApiCommon(...) to gza = EdgeAclControl(...) (though depending on how you're doing your imports, you might need to prefix the class name with a module).
The second issue is that your EdgeAclControl class doesn't take the arguments that its base class needs. Your current code doesn't pass any arguments to super(...).__init__, which doesn't work since the _apikey parameter is required. You could repeat all the arguments again in the subclass, but a lot of the time it's easier to use variable-argument syntax instead.
I suggest that you change EdgeAclControl.__init__ to accept *args and/or **kwargs and pass on those variable arguments when it calls its parent's __init__ method using super. That would look like this:
def __init__(self, *args, **kwargs):
super(EdgeAclControl, self).__init__(*args, **kwargs)
Note that if, as in this example, you're not doing anything other than calling the parent __init__ method in the derived __init__ method, you could get the same effect by just deleting the derived version entirely!
It's likely that your real code does something in EdgeAclControl.__init__, so you may need to keep it in some form. Note that it can take arguments normally in addition to the *args and **kwargs. Just remember to pass on the extra arguments, if necessary, when calling the base class.
May I ask why you have to instantiate an ApiCommon object? I don't see any point of doing so.
If you insist doing that, you have to add methods in superclass and then subclass may override theses methods. But you still couldn't access methods of EdgeAclControl from ApiCommon object
Why did the Python designers decide that subclasses' __init__() methods don't automatically call the __init__() methods of their superclasses, as in some other languages? Is the Pythonic and recommended idiom really like the following?
class Superclass(object):
def __init__(self):
print 'Do something'
class Subclass(Superclass):
def __init__(self):
super(Subclass, self).__init__()
print 'Do something else'
The crucial distinction between Python's __init__ and those other languages constructors is that __init__ is not a constructor: it's an initializer (the actual constructor (if any, but, see later;-) is __new__ and works completely differently again). While constructing all superclasses (and, no doubt, doing so "before" you continue constructing downwards) is obviously part of saying you're constructing a subclass's instance, that is clearly not the case for initializing, since there are many use cases in which superclasses' initialization needs to be skipped, altered, controlled -- happening, if at all, "in the middle" of the subclass initialization, and so forth.
Basically, super-class delegation of the initializer is not automatic in Python for exactly the same reasons such delegation is also not automatic for any other methods -- and note that those "other languages" don't do automatic super-class delegation for any other method either... just for the constructor (and if applicable, destructor), which, as I mentioned, is not what Python's __init__ is. (Behavior of __new__ is also quite peculiar, though really not directly related to your question, since __new__ is such a peculiar constructor that it doesn't actually necessarily need to construct anything -- could perfectly well return an existing instance, or even a non-instance... clearly Python offers you a lot more control of the mechanics than the "other languages" you have in mind, which also includes having no automatic delegation in __new__ itself!-).
I'm somewhat embarrassed when people parrot the "Zen of Python", as if it's a justification for anything. It's a design philosophy; particular design decisions can always be explained in more specific terms--and they must be, or else the "Zen of Python" becomes an excuse for doing anything.
The reason is simple: you don't necessarily construct a derived class in a way similar at all to how you construct the base class. You may have more parameters, fewer, they may be in a different order or not related at all.
class myFile(object):
def __init__(self, filename, mode):
self.f = open(filename, mode)
class readFile(myFile):
def __init__(self, filename):
super(readFile, self).__init__(filename, "r")
class tempFile(myFile):
def __init__(self, mode):
super(tempFile, self).__init__("/tmp/file", mode)
class wordsFile(myFile):
def __init__(self, language):
super(wordsFile, self).__init__("/usr/share/dict/%s" % language, "r")
This applies to all derived methods, not just __init__.
Java and C++ require that a base class constructor is called because of memory layout.
If you have a class BaseClass with a member field1, and you create a new class SubClass that adds a member field2, then an instance of SubClass contains space for field1 and field2. You need a constructor of BaseClass to fill in field1, unless you require all inheriting classes to repeat BaseClass's initialization in their own constructors. And if field1 is private, then inheriting classes can't initialise field1.
Python is not Java or C++. All instances of all user-defined classes have the same 'shape'. They're basically just dictionaries in which attributes can be inserted. Before any initialisation has been done, all instances of all user-defined classes are almost exactly the same; they're just places to store attributes that aren't storing any yet.
So it makes perfect sense for a Python subclass not to call its base class constructor. It could just add the attributes itself if it wanted to. There's no space reserved for a given number of fields for each class in the hierarchy, and there's no difference between an attribute added by code from a BaseClass method and an attribute added by code from a SubClass method.
If, as is common, SubClass actually does want to have all of BaseClass's invariants set up before it goes on to do its own customisation, then yes you can just call BaseClass.__init__() (or use super, but that's complicated and has its own problems sometimes). But you don't have to. And you can do it before, or after, or with different arguments. Hell, if you wanted you could call the BaseClass.__init__ from another method entirely than __init__; maybe you have some bizarre lazy initialization thing going.
Python achieves this flexibility by keeping things simple. You initialise objects by writing an __init__ method that sets attributes on self. That's it. It behaves exactly like a method, because it is exactly a method. There are no other strange and unintuitive rules about things having to be done first, or things that will automatically happen if you don't do other things. The only purpose it needs to serve is to be a hook to execute during object initialisation to set initial attribute values, and it does just that. If you want it to do something else, you explicitly write that in your code.
To avoid confusion it is useful to know that you can invoke the base_class __init__() method if the child_class does not have an __init__() class.
Example:
class parent:
def __init__(self, a=1, b=0):
self.a = a
self.b = b
class child(parent):
def me(self):
pass
p = child(5, 4)
q = child(7)
z= child()
print p.a # prints 5
print q.b # prints 0
print z.a # prints 1
In fact the MRO in python will look for __init__() in the parent class when can not find it in the children class. You need to invoke the parent class constructor directly if you have already an __init__() method in the children class.
For example the following code will return an error:
class parent:
def init(self, a=1, b=0):
self.a = a
self.b = b
class child(parent):
def __init__(self):
pass
def me(self):
pass
p = child(5, 4) # Error: constructor gets one argument 3 is provided.
q = child(7) # Error: constructor gets one argument 2 is provided.
z= child()
print z.a # Error: No attribute named as a can be found.
"Explicit is better than implicit." It's the same reasoning that indicates we should explicitly write 'self'.
I think in in the end it is a benefit-- can you recite all of the rules Java has regarding calling superclasses' constructors?
Right now, we have a rather long page describing the method resolution order in case of multiple inheritance: http://www.python.org/download/releases/2.3/mro/
If constructors were called automatically, you'd need another page of at least the same length explaining the order of that happening. That would be hell...
Often the subclass has extra parameters which can't be passed to the superclass.
Maybe __init__ is the method that the subclass needs to override. Sometimes subclasses need the parent's function to run before they add class-specific code, and other times they need to set up instance variables before calling the parent's function. Since there's no way Python could possibly know when it would be most appropriate to call those functions, it shouldn't guess.
If those don't sway you, consider that __init__ is Just Another Function. If the function in question were dostuff instead, would you still want Python to automatically call the corresponding function in the parent class?
i believe the one very important consideration here is that with an automatic call to super.__init__(), you proscribe, by design, when that initialization method is called, and with what arguments. eschewing automatically calling it, and requiring the programmer to explicitly do that call, entails a lot of flexibility.
after all, just because class B is derived from class A does not mean A.__init__() can or should be called with the same arguments as B.__init__(). making the call explicit means a programmer can have e.g. define B.__init__() with completely different parameters, do some computation with that data, call A.__init__() with arguments as appropriate for that method, and then do some postprocessing. this kind of flexibility would be awkward to attain if A.__init__() would be called from B.__init__() implicitly, either before B.__init__() executes or right after it.
As Sergey Orshanskiy pointed out in the comments, it is also convenient to write a decorator to inherit the __init__ method.
You can write a decorator to inherit the __init__ method, and even perhaps automatically search for subclasses and decorate them. – Sergey Orshanskiy Jun 9 '15 at 23:17
Part 1/3: The implementation
Note: actually this is only useful if you want to call both the base and the derived class's __init__ since __init__ is inherited automatically. See the previous answers for this question.
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
super(type(self), self).__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
child(42)
Outputs:
Base: 42
Part 2/3: A warning
Warning: this doesn't work if base itself called super(type(self), self).
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
'''Warning: recursive calls.'''
super(type(self), self).__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
class child2(child):
#default_init
def __init__(self, n: int) -> None:
pass
child2(42)
RecursionError: maximum recursion depth exceeded while calling a Python object.
Part 3/3: Why not just use plain super()?
But why not just use the safe plain super()? Because it doesn't work since the new rebinded __init__ is from outside the class, and super(type(self), self) is required.
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
child(42)
Errors:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-9-6f580b3839cd> in <module>
13 pass
14
---> 15 child(42)
<ipython-input-9-6f580b3839cd> in wrapper(self, *args, **kwargs)
1 def default_init(func):
2 def wrapper(self, *args, **kwargs) -> None:
----> 3 super().__init__(*args, **kwargs)
4 return wrapper
5
RuntimeError: super(): __class__ cell not found
Background - We CAN AUTO init a parent AND child class!
A lot of answers here and say "This is not the python way, use super().__init__() from the subclass". The question is not asking for the pythonic way, it's comparing to the expected behavior from other languages to python's obviously different one.
The MRO document is pretty and colorful but it's really a TLDR situation and still doesn't quite answer the question, as is often the case in these types of comparisons - "Do it the Python way, because.".
Inherited objects can be overloaded by later declarations in subclasses, a pattern building on #keyvanrm's (https://stackoverflow.com/a/46943772/1112676) answer solves the case where I want to AUTOMATICALLY init a parent class as part of calling a class without explicitly calling super().__init__() in every child class.
In my case where a new team member might be asked to use a boilerplate module template (for making extensions to our application without touching the core application source) which we want to make as bare and easy to adopt without them needing to know or understand the underlying machinery - to only need to know of and use what is provided by the application's base interface which is well documented.
For those who will say "Explicit is better than implicit." I generally agree, however, when coming from many other popular languages inherited automatic initialization is the expected behavior and it is very useful if it can be leveraged for projects where some work on a core application and others work on extending it.
This technique can even pass args/keyword args for init which means pretty much any object can be pushed to the parent and used by the parent class or its relatives.
Example:
class Parent:
def __init__(self, *args, **kwargs):
self.somevar = "test"
self.anothervar = "anothertest"
#important part, call the init surrogate pass through args:
self._init(*args, **kwargs)
#important part, a placeholder init surrogate:
def _init(self, *args, **kwargs):
print("Parent class _init; ", self, args, kwargs)
def some_base_method(self):
print("some base method in Parent")
self.a_new_dict={}
class Child1(Parent):
# when omitted, the parent class's __init__() is run
#def __init__(self):
# pass
#overloading the parent class's _init() surrogate
def _init(self, *args, **kwargs):
print(f"Child1 class _init() overload; ",self, args, kwargs)
self.a_var_set_from_child = "This is a new var!"
class Child2(Parent):
def __init__(self, onevar, twovar, akeyword):
print(f"Child2 class __init__() overload; ", self)
#call some_base_method from parent
self.some_base_method()
#the parent's base method set a_new_dict
print(self.a_new_dict)
class Child3(Parent):
pass
print("\nRunning Parent()")
Parent()
Parent("a string", "something else", akeyword="a kwarg")
print("\nRunning Child1(), keep Parent.__init__(), overload surrogate Parent._init()")
Child1()
Child1("a string", "something else", akeyword="a kwarg")
print("\nRunning Child2(), overload Parent.__init__()")
#Child2() # __init__() requires arguments
Child2("a string", "something else", akeyword="a kwarg")
print("\nRunning Child3(), empty class, inherits everything")
Child3().some_base_method()
Output:
Running Parent()
Parent class _init; <__main__.Parent object at 0x7f84a721fdc0> () {}
Parent class _init; <__main__.Parent object at 0x7f84a721fdc0> ('a string', 'something else') {'akeyword': 'a kwarg'}
Running Child1(), keep Parent.__init__(), overload surrogate Parent._init()
Child1 class _init() overload; <__main__.Child1 object at 0x7f84a721fdc0> () {}
Child1 class _init() overload; <__main__.Child1 object at 0x7f84a721fdc0> ('a string', 'something else') {'akeyword': 'a kwarg'}
Running Child2(), overload Parent.__init__()
Child2 class __init__() overload; <__main__.Child2 object at 0x7f84a721fdc0>
some base method in Parent
{}
Running Child3(), empty class, inherits everything, access things set by other children
Parent class _init; <__main__.Child3 object at 0x7f84a721fdc0> () {}
some base method in Parent
As one can see, the overloaded definition(s) take the place of those declared in Parent class but can still be called BY the Parent class thereby allowing one to emulate the classical implicit inheritance initialization behavior Parent and Child classes both initialize without needing to explicitly invoke the Parent's init() from the Child class.
Personally, I call the surrogate _init() method main() because it makes sense to me when switching between C++ and Python for example since it is a function that will be automatically run for any subclass of Parent (the last declared definition of main(), that is).