Develop Reference

Inheriting from classes with and without **kwargs

I am building a plotting class in Python, and am hoping to do the following. I want a graphics window using PyQt5 that also inherits from some custom classes I have made (such as a curve fitting class). In order for the curve fitting class to manipulate data that persists in the plotting class, it must have a reference to the data that is contained in the plotting class. Because of this, I have chosen the plotting class to inherit from the CurveFitting class.
The problem seems to arise in inheriting both from PyQt5's GraphicsWindow class and my custom class, which accept different numbers of arguments. I have read that Python does not play nice with classes that inherit different numbers of arguments using the "super" functionality, so I decided to make my custom CurveFitting class accept **kwargs, which would then give it a reference to the parent. However, I then encountered a different error which I do not understand. Below is a tidied up example of what I'm trying to do
import numpy as np
from pyqtgraph import GraphicsWindow
class ClassA():
def __init__(self, **kwargs):
super().__init__()
self.kwargs = kwargs
self.parent = self.kwargs['parent']
self.xdata = self.parent.xdata
def print_data(self):
print(self.parent.xdata)
print(self.parent.ydata)
class classC(GraphicsWindow, ClassA):
def __init__(self):
kwargs = {}
kwargs['parent'] = self
kargs = kwargs
self.xdata = np.linspace(0, 100, 101)
self.ydata = np.linspace(0, 200, 101)
super().__init__(**kwargs)
# ClassA.__init__(self, **kwargs)
# GraphicsWindow.__init__(self)
instC = classC()
instC.print_data()
When I run the above I get "RuntimeError: super-class init() of type classC was never called" on the "super().__init(**kwargs)" line, which I honestly do not understand at all, and have tried googling for a while but to no avail.
Additionally, I have tried commenting out the line, and uncommenting the next two lines to inherit from each class manually, but this also does not work. What I find pretty weird is that if I comment one of those two lines out, they both work individually, but together they do not work. For example, if I run it with both lines, it gives me an error that kwargs has no key word 'parent', as if it didn't even pass **kwargs.
Is there a way to inherit from two classes that take a different number of initialization parameters like this? Is there a totally different way I could be approaching this problem? Thanks.

The immediate problem with your code is that ClassC inherits from GraphicsWindow as its first base class, and ClassA is the second base class. When you call super, only one gets called (GraphicsWindow) and if it was not designed to work with multiple inheritance (as seems to be the case), it may not call super itself or may not pass on the arguments that ClassA expects.
Just switching the order of the base classes may be enough to make it work. Python guarantees that the base classes will be called in the same relative order that they appear in the class statement in (though other classes may be inserted between them in the MRO if more inheritance happens later). Since ClassA.__init__ does call super, it should work better!
It can be tricky to make __init__ methods work with multiple inheritance though, even if all the classes involved are designed to work with it. This is why positional arguments are often avoided, since their order can become very confusing (since child classes can only add positional arguments ahead of their parent's positional arguments unless they want to repeat all the names). Using keyword arguments is definitely a better approach.
But the code you have is making dealing with keyword arguments a bit more complicated than it should be. You shouldn't need to explicitly create dictionaries to pass on with **kwargs syntax, nor should you need to extract keyword values from a a dict you accepted with a **kwargs argument. Usually each function should name the arguments it takes, and only use **kwargs for unknown arguments (that may be needed by some other class in the MRO). Here's what that looks like:
class Base1:
def __init__(self, *, arg1, arg2, arg3, **kwargs): # the * means the other args are kw-only
super().__init__(**kwargs) # always pass on all unknown arguments
... # use the named args here (not kwargs)
class Base2:
def __init__(self, *, arg4, arg5, arg6, **kwargs):
super().__init__(**kwargs)
...
class Derived(Base1, Base2):
def __init__(self, *, arg2, arg7, **kwargs): # child can accept args used by parents
super().__init__(arg2=arg2+1, arg6=3, **kwargs) # it can then modify or create from
... # scratch args to pass to its parents
obj = Derived(arg1=1, arg2=2, arg3=3, arg4=4, arg5=5, arg7=7) # note, we've skipped arg6
# and Base1 will get 3 for arg2
But I'd also give serious though to whether inheritance makes any sense in your situation. It may make more sense for one of your two base classes to be encapsulated within your child class, rather than being inherited from. That is, you'd inherit from only one of ClassA or GraphicsWindow, and store an instance of the other in each instance of ClassC. (You could even inherit from neither base class, and encapsulate them both.) Encapsulation is often a lot easier to reason about and get right than inheritance.

Porting Subclass of Unicode to Python 3

I'm porting a legacy codebase from Python 2.7 to Python 3.6. In that codebase I have a number of instances of things like:
class EntityName(unicode):
#staticmethod
def __new__(cls, s):
clean = cls.strip_junk(s)
return super(EntityName, cls).__new__(cls, clean)
def __init__(self, s):
self._clean = s
self._normalized = normalized_name(self._clean)
self._simplified = simplified_name(self._clean)
self._is_all_caps = None
self._is_all_lower = None
super(EntityName, self).__init__(self._clean)
It might be called like this:
EntityName("Guy DeFalt")
When porting this to Python 3 the above code fails because unicode is no longer a class you can extend (at least, if there is an equivalent class I cannot find it). Given that str is unicode now, I tried to just swap str in, but the parent init doesn't take a the string value I'm trying to pass:
TypeError: object.__init__() takes no parameters
This makes sense because str does not have an __init__ method - this does not seem to be an idiomatic way of using this class. So my question has two major branches:
Is there a better way to be porting classes that sub-classed the old unicode class?
If subclassing str is appropriate, how should I modify the __init__ function for idiomatic behavior?

The right way to subclass a string or another immutable class in Python 3 is same as in Python 2:
class MyString(str):
def __new__(cls, initial_arguments): # no staticmethod
desired_string_value = get_desired_string_value(initial_arguments)
return super(MyString, cls).__new__(cls, desired_string_value)
# can be shortened to super().__new__(...)
def __init__(self, initial_arguments): # arguments are unused
self.whatever = whatever(self)
# no need to call super().__init__(), but if you must, do not pass arguments
There are several issues with your sample. First, why __new__ is #staticmethod? It's #classmethod, although you don't need to specify this. Second, the code seems to operate under the assumption that when you call __new__ of the superclass, it somehow calls your __init__ as well. I'm deriving this from looking at how self._clean is supposed to be set. This is not the case. When you call MyString(arguments), the following happens:
First Python calls __new__ with the class parameter (usually called cls) and arguments. __new__ must return the class instance. To do this it can create it, as we do, or do something else; e.g. it may return an existing one or, in fact, anything.
Then Python calls __init__ with the instance it received from __new__ (this parameter is usually called self) and the same arguments.
(There's a special case: Python won't call __init__ if __new__ returned something that is not a subclass of the passed class.)
Python uses class hierarchy to see which __new__ and __init__ to call. It's up to you to correctly sort out the arguments and use proper superclass calls in these two methods.

Python __call__() is this an implicit classmethod?

I want to implement a singleton pattern in python, and I liked the pattern described in the http://www.python-course.eu/python3_metaclasses.php.
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class SingletonClass(metaclass=Singleton):
pass
class RegularClass():
pass
x = SingletonClass()
y = SingletonClass()
print(x == y)
x = RegularClass()
y = RegularClass()
print(x == y)
And the code works perfect. But, the __call__() does not have the self, and it also does not have #classmethod or #staticmethod declaration.
But, in the Python data model https://docs.python.org/3/reference/datamodel.html#object.__call__ the __call__() method has a self in the arguments.
The code does not work if I pass self, or declare as #staticmethod or #classmethod.
Can someone please explain the logic of the syntax behind the __call__() method.

Naming the first argument of a method cls or self are just a convention. The __call__ method does have a self argument, only it is named cls here. That's because for a metaclass, the method is bound to a class object, and the name reflects this.
The same convention is applied to #classmethod methods; the first argument is a class, always, due to the nature of how a classmethod object is bound, so it makes sense to name that first argument cls.
But you are free to name that first argument anything else. It is not the name that makes a classmethod or a regular method or a method on a metatype work. All that using self or cls does is document what type of object this is, making it easier for other developers to mentally track what is going on.
So no, this is not an implicit class method. That first argument is not bound to the Singleton metaclass object, it is bound to the class that was called. That makes sense, because that class object is an instance of the Singleton metatype.
If you want to dive into how binding works (the process that causes that first argument to be passed in, whatever the name), you can read up on the Descriptor HOWTO. TLDR: functions, property, classmethod and staticmethod objects are all descriptors, and whenever you access them as an attribute on a supporting object such as an instance or a class, they are bound, often causing a different object to be returned as a result, which when called passes in the bound object to the actual function.

Martin's answer says it all. I am adding this so maybe a different wording can throw in more light for different people:
Python call() is this an implicit classmethod?
No. But all "metaclass" methods are implicit "class methods" for the classes that use that metaclass.
That is implicit when we take in account the fact that classes are simply instances of the metaclass. From the language point of view, a class behave almost exactly like any other instance - and any interactions with a class that would trigger dunder (__magic__) methods in an object - like using the "+, -, *, /" operators, or index retrieval with [ ], or calling them with ( ) trigger the corresponding methods on its class. That is ordinarily type.
And, as put in the other answer, Python does not care what name you ut on the first argument of a method. For metaclasses it makes sense to use cls there, since the "instances" the methods are dealing with are metaclasses. As it makes sense that the first argument to a metaclass' __new__ method be named metacls (like in the example bellow). Because the new "cls" is the object we get after calling type.__new__ - the only call possible in pure Python that will actually create a class object.
class Meta(type):
def __new__(metacls, name, bases, namespace):
...
cls = super().__new__(metacls, name, bases, namespace)
...
return cls
(Now, still on the topic of the question: __new__ is a special case
of an implicit static method (even for ordinary classes that are not intended to be metaclasses) - to which Python specially add the first argument, using a different mechanism than what is done to regular classmethods. Thats is why the super().__new__ call above needs to include the metacls as the first parameter)

self nessary in setting class instance Var from different class instance Var Python

Is the self needed in Parser.self.totalEntries?
class Iter(Parser):
def __init__(self, Parser):
self.totalEntriesI = Parser.self.totalEntries
I have given up looking for it in the docs, perhaps I'm just a silly.

Short answer is : no, you don't need it.
I was trying to make sense of your code here but this part bugged me :
def __init__(self, Parser):
However, the capital P for parser can be a bit ambiguous : it can mean either the Parser class that you inherit from (in which case this parameter is useless), or 'any parser given as parameter' (in which case you might want to lowercase that for clarity).
Assuming you mean a specific given parser, you could rewrite all to have the following:
def __init__(self, myLovelyParser):
self.totalEntriesI = myLovelyParser.totalEntries
Which is a lot clearer.
The only point of self is to give a bound name for the implicit instance parameter.
More philosophically : "From the outside, there is no self"

The self parameter (which does not need, but should be called that) is just the same as the object being called on. Now __init__ is a little bit special, but in general obj.method(args) is the same as cls.method(obj, args) where cls is the class of obj. There's nothing more magical with self than it is automatically inserted when calling the method. With __init__ of course the self argument is the object under construction.
In other word the self (or first) argument is just the object on which the method has been called.
It's a bit unclear what you're trying to perform (since it is somewhat confusing to have an argument named the same as the class of the method), but self is NOT what you would do if you want to access some other object, first of all you have to supply the other object also (for example as an argument) and access it in the same way:
def method(self, other):
self.attr = other.attr
Note that other.self.attr would require that the object other has an attribute called self which it normally doesn't. Consequently it is normally incorrect to do that - and in any case it is quite different from other.attr anyway.

Why aren't superclass __init__ methods automatically invoked?

Why did the Python designers decide that subclasses' __init__() methods don't automatically call the __init__() methods of their superclasses, as in some other languages? Is the Pythonic and recommended idiom really like the following?
class Superclass(object):
def __init__(self):
print 'Do something'
class Subclass(Superclass):
def __init__(self):
super(Subclass, self).__init__()
print 'Do something else'

The crucial distinction between Python's __init__ and those other languages constructors is that __init__ is not a constructor: it's an initializer (the actual constructor (if any, but, see later;-) is __new__ and works completely differently again). While constructing all superclasses (and, no doubt, doing so "before" you continue constructing downwards) is obviously part of saying you're constructing a subclass's instance, that is clearly not the case for initializing, since there are many use cases in which superclasses' initialization needs to be skipped, altered, controlled -- happening, if at all, "in the middle" of the subclass initialization, and so forth.
Basically, super-class delegation of the initializer is not automatic in Python for exactly the same reasons such delegation is also not automatic for any other methods -- and note that those "other languages" don't do automatic super-class delegation for any other method either... just for the constructor (and if applicable, destructor), which, as I mentioned, is not what Python's __init__ is. (Behavior of __new__ is also quite peculiar, though really not directly related to your question, since __new__ is such a peculiar constructor that it doesn't actually necessarily need to construct anything -- could perfectly well return an existing instance, or even a non-instance... clearly Python offers you a lot more control of the mechanics than the "other languages" you have in mind, which also includes having no automatic delegation in __new__ itself!-).

I'm somewhat embarrassed when people parrot the "Zen of Python", as if it's a justification for anything. It's a design philosophy; particular design decisions can always be explained in more specific terms--and they must be, or else the "Zen of Python" becomes an excuse for doing anything.
The reason is simple: you don't necessarily construct a derived class in a way similar at all to how you construct the base class. You may have more parameters, fewer, they may be in a different order or not related at all.
class myFile(object):
def __init__(self, filename, mode):
self.f = open(filename, mode)
class readFile(myFile):
def __init__(self, filename):
super(readFile, self).__init__(filename, "r")
class tempFile(myFile):
def __init__(self, mode):
super(tempFile, self).__init__("/tmp/file", mode)
class wordsFile(myFile):
def __init__(self, language):
super(wordsFile, self).__init__("/usr/share/dict/%s" % language, "r")
This applies to all derived methods, not just __init__.

Java and C++ require that a base class constructor is called because of memory layout.
If you have a class BaseClass with a member field1, and you create a new class SubClass that adds a member field2, then an instance of SubClass contains space for field1 and field2. You need a constructor of BaseClass to fill in field1, unless you require all inheriting classes to repeat BaseClass's initialization in their own constructors. And if field1 is private, then inheriting classes can't initialise field1.
Python is not Java or C++. All instances of all user-defined classes have the same 'shape'. They're basically just dictionaries in which attributes can be inserted. Before any initialisation has been done, all instances of all user-defined classes are almost exactly the same; they're just places to store attributes that aren't storing any yet.
So it makes perfect sense for a Python subclass not to call its base class constructor. It could just add the attributes itself if it wanted to. There's no space reserved for a given number of fields for each class in the hierarchy, and there's no difference between an attribute added by code from a BaseClass method and an attribute added by code from a SubClass method.
If, as is common, SubClass actually does want to have all of BaseClass's invariants set up before it goes on to do its own customisation, then yes you can just call BaseClass.__init__() (or use super, but that's complicated and has its own problems sometimes). But you don't have to. And you can do it before, or after, or with different arguments. Hell, if you wanted you could call the BaseClass.__init__ from another method entirely than __init__; maybe you have some bizarre lazy initialization thing going.
Python achieves this flexibility by keeping things simple. You initialise objects by writing an __init__ method that sets attributes on self. That's it. It behaves exactly like a method, because it is exactly a method. There are no other strange and unintuitive rules about things having to be done first, or things that will automatically happen if you don't do other things. The only purpose it needs to serve is to be a hook to execute during object initialisation to set initial attribute values, and it does just that. If you want it to do something else, you explicitly write that in your code.

To avoid confusion it is useful to know that you can invoke the base_class __init__() method if the child_class does not have an __init__() class.
Example:
class parent:
def __init__(self, a=1, b=0):
self.a = a
self.b = b
class child(parent):
def me(self):
pass
p = child(5, 4)
q = child(7)
z= child()
print p.a # prints 5
print q.b # prints 0
print z.a # prints 1
In fact the MRO in python will look for __init__() in the parent class when can not find it in the children class. You need to invoke the parent class constructor directly if you have already an __init__() method in the children class.
For example the following code will return an error:
class parent:
def init(self, a=1, b=0):
self.a = a
self.b = b
class child(parent):
def __init__(self):
pass
def me(self):
pass
p = child(5, 4) # Error: constructor gets one argument 3 is provided.
q = child(7) # Error: constructor gets one argument 2 is provided.
z= child()
print z.a # Error: No attribute named as a can be found.

"Explicit is better than implicit." It's the same reasoning that indicates we should explicitly write 'self'.
I think in in the end it is a benefit-- can you recite all of the rules Java has regarding calling superclasses' constructors?

Right now, we have a rather long page describing the method resolution order in case of multiple inheritance: http://www.python.org/download/releases/2.3/mro/
If constructors were called automatically, you'd need another page of at least the same length explaining the order of that happening. That would be hell...

Often the subclass has extra parameters which can't be passed to the superclass.

Maybe __init__ is the method that the subclass needs to override. Sometimes subclasses need the parent's function to run before they add class-specific code, and other times they need to set up instance variables before calling the parent's function. Since there's no way Python could possibly know when it would be most appropriate to call those functions, it shouldn't guess.
If those don't sway you, consider that __init__ is Just Another Function. If the function in question were dostuff instead, would you still want Python to automatically call the corresponding function in the parent class?

i believe the one very important consideration here is that with an automatic call to super.__init__(), you proscribe, by design, when that initialization method is called, and with what arguments. eschewing automatically calling it, and requiring the programmer to explicitly do that call, entails a lot of flexibility.
after all, just because class B is derived from class A does not mean A.__init__() can or should be called with the same arguments as B.__init__(). making the call explicit means a programmer can have e.g. define B.__init__() with completely different parameters, do some computation with that data, call A.__init__() with arguments as appropriate for that method, and then do some postprocessing. this kind of flexibility would be awkward to attain if A.__init__() would be called from B.__init__() implicitly, either before B.__init__() executes or right after it.

As Sergey Orshanskiy pointed out in the comments, it is also convenient to write a decorator to inherit the __init__ method.
You can write a decorator to inherit the __init__ method, and even perhaps automatically search for subclasses and decorate them. – Sergey Orshanskiy Jun 9 '15 at 23:17
Part 1/3: The implementation
Note: actually this is only useful if you want to call both the base and the derived class's __init__ since __init__ is inherited automatically. See the previous answers for this question.
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
super(type(self), self).__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
child(42)
Outputs:
Base: 42
Part 2/3: A warning
Warning: this doesn't work if base itself called super(type(self), self).
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
'''Warning: recursive calls.'''
super(type(self), self).__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
class child2(child):
#default_init
def __init__(self, n: int) -> None:
pass
child2(42)
RecursionError: maximum recursion depth exceeded while calling a Python object.
Part 3/3: Why not just use plain super()?
But why not just use the safe plain super()? Because it doesn't work since the new rebinded __init__ is from outside the class, and super(type(self), self) is required.
def default_init(func):
def wrapper(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
return wrapper
class base():
def __init__(self, n: int) -> None:
print(f'Base: {n}')
class child(base):
#default_init
def __init__(self, n: int) -> None:
pass
child(42)
Errors:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-9-6f580b3839cd> in <module>
13 pass
14
---> 15 child(42)
<ipython-input-9-6f580b3839cd> in wrapper(self, *args, **kwargs)
1 def default_init(func):
2 def wrapper(self, *args, **kwargs) -> None:
----> 3 super().__init__(*args, **kwargs)
4 return wrapper
5
RuntimeError: super(): __class__ cell not found

Background - We CAN AUTO init a parent AND child class!
A lot of answers here and say "This is not the python way, use super().__init__() from the subclass". The question is not asking for the pythonic way, it's comparing to the expected behavior from other languages to python's obviously different one.
The MRO document is pretty and colorful but it's really a TLDR situation and still doesn't quite answer the question, as is often the case in these types of comparisons - "Do it the Python way, because.".
Inherited objects can be overloaded by later declarations in subclasses, a pattern building on #keyvanrm's (https://stackoverflow.com/a/46943772/1112676) answer solves the case where I want to AUTOMATICALLY init a parent class as part of calling a class without explicitly calling super().__init__() in every child class.
In my case where a new team member might be asked to use a boilerplate module template (for making extensions to our application without touching the core application source) which we want to make as bare and easy to adopt without them needing to know or understand the underlying machinery - to only need to know of and use what is provided by the application's base interface which is well documented.
For those who will say "Explicit is better than implicit." I generally agree, however, when coming from many other popular languages inherited automatic initialization is the expected behavior and it is very useful if it can be leveraged for projects where some work on a core application and others work on extending it.
This technique can even pass args/keyword args for init which means pretty much any object can be pushed to the parent and used by the parent class or its relatives.
Example:
class Parent:
def __init__(self, *args, **kwargs):
self.somevar = "test"
self.anothervar = "anothertest"
#important part, call the init surrogate pass through args:
self._init(*args, **kwargs)
#important part, a placeholder init surrogate:
def _init(self, *args, **kwargs):
print("Parent class _init; ", self, args, kwargs)
def some_base_method(self):
print("some base method in Parent")
self.a_new_dict={}
class Child1(Parent):
# when omitted, the parent class's __init__() is run
#def __init__(self):
# pass
#overloading the parent class's _init() surrogate
def _init(self, *args, **kwargs):
print(f"Child1 class _init() overload; ",self, args, kwargs)
self.a_var_set_from_child = "This is a new var!"
class Child2(Parent):
def __init__(self, onevar, twovar, akeyword):
print(f"Child2 class __init__() overload; ", self)
#call some_base_method from parent
self.some_base_method()
#the parent's base method set a_new_dict
print(self.a_new_dict)
class Child3(Parent):
pass
print("\nRunning Parent()")
Parent()
Parent("a string", "something else", akeyword="a kwarg")
print("\nRunning Child1(), keep Parent.__init__(), overload surrogate Parent._init()")
Child1()
Child1("a string", "something else", akeyword="a kwarg")
print("\nRunning Child2(), overload Parent.__init__()")
#Child2() # __init__() requires arguments
Child2("a string", "something else", akeyword="a kwarg")
print("\nRunning Child3(), empty class, inherits everything")
Child3().some_base_method()
Output:
Running Parent()
Parent class _init; <__main__.Parent object at 0x7f84a721fdc0> () {}
Parent class _init; <__main__.Parent object at 0x7f84a721fdc0> ('a string', 'something else') {'akeyword': 'a kwarg'}
Running Child1(), keep Parent.__init__(), overload surrogate Parent._init()
Child1 class _init() overload; <__main__.Child1 object at 0x7f84a721fdc0> () {}
Child1 class _init() overload; <__main__.Child1 object at 0x7f84a721fdc0> ('a string', 'something else') {'akeyword': 'a kwarg'}
Running Child2(), overload Parent.__init__()
Child2 class __init__() overload; <__main__.Child2 object at 0x7f84a721fdc0>
some base method in Parent
{}
Running Child3(), empty class, inherits everything, access things set by other children
Parent class _init; <__main__.Child3 object at 0x7f84a721fdc0> () {}
some base method in Parent
As one can see, the overloaded definition(s) take the place of those declared in Parent class but can still be called BY the Parent class thereby allowing one to emulate the classical implicit inheritance initialization behavior Parent and Child classes both initialize without needing to explicitly invoke the Parent's init() from the Child class.
Personally, I call the surrogate _init() method main() because it makes sense to me when switching between C++ and Python for example since it is a function that will be automatically run for any subclass of Parent (the last declared definition of main(), that is).