A use case of the super() builtin in python is to call an overridden method. Here is a simple example of using super() to call Parent class's echo function:
class Parent():
def echo(self):
print("in Parent")
class Child(Parent):
def echo(self):
super().echo()
print("in Child")
I've seen code that passes 2 parameters to super(). In that case, the signature looks somehing like super(subClass, instance) where subClass is the sub class calling super() from, and instance is the instance the call being made from, ie self. So in the above example, the super() line would become:
super(Child, self).echo()
Looking at python3 docs, these 2 use cases are the same when calling from inside of a class.
Is calling super() with 2 parameters completely deprecated as of python3? If this is only deprecated for calling overridden functions, can you show an example why they're needed for other cases?
I'm also interested to know why python needed those 2 arguments? Are they injected/evaluated when making super() calls in python3, or they're just not needed in that case?
If you don't pass the arguments, Python 3 makes an effort to provide them for you. It's a little kludgy, but it usually works. Essentially, it just assumes the first parameter to your method is self (the second argument to super), and when the class definition completes, it provides a virtual closure scope for any function that refers to super or __class__ that defines __class__ as the class you just defined, so no-arg super() can check the stack frame to find __class__ as the first argument.
This usually works, but there are cases where it doesn't:
In staticmethods (since the first argument isn't actually the class), though staticmethods aren't really supposed to participate in the inheritance hierarchy the same way, so that's not exactly unexpected.
In functions that take *args as the first argument (which was the only safe way to implement any method that accepted arbitrary keyword arguments like dict subclasses prior to 3.8, when they introduced positional-only arguments).
If the super call is in a nested scope, which can be implicit, e.g. a generator expression, or comprehension of any kind (list, set, dict), since the nested scope isn't directly attached to the class, so it doesn't get the __class__ defining magic that methods attached to the class itself get.
There are also rare cases where you might want to explicitly bypass a particular class for resolving the parent method, in which case you'd need to explicitly pass a different class from later in the MRO than the one it would pull by default. This is deep and terrible magic, so I don't recommend it, but I think I've had cause to do it once.
All of those are relatively rare cases, so most of the time, for code purely targeting Python 3, you're fine using no-arg super, and only falling back to explicitly passing arguments when you can't use it for whatever reason. No-arg super is cleaner, faster, and during live development can be more correct (if you replace MyClass, instances of the old version of the class will have the old version cached and no-arg super will continue to work, where looking it up manually in your globals would find the new version of the class and explode), so use it whenever you can.
In Python 3, super() with zero arguments is already the shortcut for super(__class__, self). See PEP3135 for complete explanation.
This is not the case for Python 2, so I guess that most code examples you found were actually written for Python 2 (or Python2+3 compatible)
Related
I have a class like this
class A(object):
def __init__(self, name):
self.name = name
def run(self):
pass
if we look at the type of run it is a function. I am now writing a decorator and this decorator should be used with either a stand alone function or a method but has different behavior if the function it is decorating is a method. When registering the method run, the decorator cannot really tell if the function is a method because it has not been bounded to an object yet. I have tried inspect.ismethod and it also does not work. Is there a way that I can detect run is a method in my decorator instead of a standalone function? Thanks!
To add a bit more info:
Basically I am logging something out. If it is decorating an object method, I need the name of the class of that object and the method name, if it is the decorating a function, I just need the function name.
As mentionned by chepner, a function only becomes a method when it's used as one - ie when it's looked up on an instance and resolved on the class. What you are decorating is and will always be a function (well, unless you already decorated it with something that returns another callable type of course, cf the classmethod type).
At this point you have two options: the safe and explicit one, and the unsafe guessing game one.
The safe and explicit solution is, simply, to have two distinct decorators, one for plain functions, and another for "functions to be used as methods".
The unsafe guessing game one is to inspect the function's first arg name (using inspect.getargspecs()) and consider it's a "function to be used as method" if the first argument is named "self".
Obviously the safe and explicit solution is also much simpler ;-)
This question is in relation to posts at What does 'super' do in Python? , How do I initialize the base (super) class? , and Python: How do I make a subclass from a superclass? which describes two ways to initialize a SuperClass from within a SubClass as
class SuperClass:
def __init__(self):
return
def superMethod(self):
return
## One version of Initiation
class SubClass(SuperClass):
def __init__(self):
SuperClass.__init__(self)
def subMethod(self):
return
or
class SuperClass:
def __init__(self):
return
def superMethod(self):
return
## Another version of Initiation
class SubClass(SuperClass):
def __init__(self):
super(SubClass, self).__init__()
def subMethod(self):
return
So I'm a little confused about needing to explicitly pass self as a parameter in
SuperClass.__init__(self)
and
super(SubClass, self).__init__().
(In fact if I call SuperClass.__init__() I get the error
TypeError: __init__() missing 1 required positional argument: 'self'
). But when calling constructors or any other class method (ie :
## Calling class constructor / initiation
c = SuperClass()
k = SubClass()
## Calling class methods
c.superMethod()
k.superMethod()
k.subMethod()
), The self parameter is passed implicitly .
My understanding of the self keyword is it is not unlike the this pointer in C++, whereas it provides a reference to the class instance. Is this correct?
If there would always be a current instance (in this case SubClass), then why does self need to be explicitly included in the call to SuperClass.__init__(self)?
Thanks
This is simply method binding, and has very little to do with super. When you can x.method(*args), Python checks the type of x for a method named method. If it finds one, it "binds" the function to x, so that when you call it, x will be passed as the first parameter, before the rest of the arguments.
When you call a (normal) method via its class, no such binding occurs. If the method expects its first argument to be an instance (e.g. self), you need to pass it in yourself.
The actual implementation of this binding behavior is pretty neat. Python objects are "descriptors" if they have a __get__ method (and/or __set__ or __delete__ methods, but those don't matter for methods). When you look up an attribute like a.b, Python checks the class of a to see if it has a attribute b that is a descriptor. If it does, it translates a.b into type(a).b.__get__(a, type(a)). If b is a function, it will have a __get__ method that implements the binding behavior I described above. Other kinds of descriptors can have different behaviors. For instance, the classmethod decorator replaces a method with a special descriptor that binds the function the class, rather than the instance.
Python's super creates special objects that handle attribute lookups differently than normal objects, but the details don't matter too much for this issue. The binding behavior of methods called through super is just like what I described in the first paragraph, so self gets passed automatically to the bound method when it is called. The only thing special about super is that it may bind a different function than you'd get lookup up the same method name on self (that's the whole point of using it).
The following example might elucidate things:
class Example:
def method(self):
pass
>>> print(Example.method)
<unbound method Example.method>
>>> print(Example().method)
<bound method Example.method of <__main__.Example instance at 0x01EDCDF0>>
When a method is bound, the instance is passed implicitly. When a method is unbound, the instance needs to be passed explicitly.
The other answers will definitely offer some more detail on the binding process, but I think it's worth showing the above snippet.
The answer is non-trivial and would probably warrant a good article. A very good explanation of how super() works is brilliantly given by Raymond Hettinger in a Pycon 2015 talk, available here and a related article.
I will attempt a short answer and if it is not sufficient I (and hopefully the community) will expand on it.
The answer has two key pieces:
Python's super() needs to have an object on which the method being overridden is called, so it is explicitly passed with self. This is not the only possible implementation and in fact, in Python 3, it is no longer required that you pass the self instance.
Python super() is not like Java, or other compiled languages, super. Python's implementation is designed to support the multiple collaborative inheritance paradigm, as explained in Hettinger's talk.
This has an interesting consequence in Python: the method resolution in super() depends not only on the parent class, but on the children classes as well (consequence of multiple inheritance). Note that Hettinger is using Python 3.
The official Python 2.7 documentation on super is also a good source of information (better understood after watching the talk, in my opinion).
Because in SuperClass.__init__(self), you're calling the method on the class, not the instance, so it cannot be passed implicitly. Similarly you cannot just call SubClass.subMethod(), but you can call SubClass.subMethod(k) and it'll be equivalent to k.subMethod(). Similarly if self refers to a SubClass then self.__init__() means SubClass.__init__(self), so if you want to call SuperClass.__init you have to call it directly.
I have somewhat of a strange question here. Let's say I'm making a simple, basic class as follows:
class MyClass(object):
def __init__(self):
super(MyClass, self).__init__()
Is there any purpose in calling super()? My class only has the default object parent class. The reason why I'm asking this is because my IDE automagically gives me this snippet when I create a new class. I usually remove the super() function because I don't see any purpose in leaving it. But maybe I'm missing something ?
You're not obliged to call object.__init__ (via super or otherwise). It does nothing.
However, the purpose of writing that snippet in that way in an __init__ function (or any function that calls the superclass) is to give you some flexibility to change the superclass without modifying that code.
So it doesn't buy you much, but it does buy you the ability to change the superclass of MyClass to a different class whose __init__ likewise accepts no-args, but which perhaps does do something and does need to be called by subclass __init__ functions. All without modifying your MyClass.__init__.
Your call whether that's worth having.
Also in this particular example you can leave out MyClass.__init__ entirely, because yours does nothing too.
Recently I faced a problem in a C-based python extension while trying to instantiate objects without calling its constructor -- which is a requirement of the extension.
The class to be used to create instances is obtained dynamically: at some point, I have an instance x whose class I wish to use to create other instances, so I store x.__class__ for later use -- let this value be klass.
At a later point, I invoke PyInstance_NewRaw(klass, PyDict_New()) and then, the problem arises. It seems that if klass is an old-style class, the result of that call is the desired new instance. However, if it is a new-style class, the result is NULL and the exception raised is:
SystemError: ../Objects/classobject.c:521: bad argument to internal function
For the record, I'm using Python version 2.7.5. Googling around, I observed no more than one other person looking for a solution (and it seemed to me he was doing a workaround, but didn't detailed it).
For the record #2: the instances the extension is creating are proxies for these same x instances -- the x.__class__ and x.__dict__'s are known, so the extension is spawning new instances based on __class__ (using the aforementioned C function) and setting the respective __dict__ to the new instance (those __dict__'s have inter-process shared-memory data). Not only is conceptually problematic to call an instance's __init__ a second time (first: it's state is already know, second: the expected behavior for ctors is that they should be called exactly once for each instance), it is also impractical, since the extension cannot figure out the arguments and their order to call the __init__() for each instance in the system. Also, changing the __init__ of each class in the system whose instances may be proxies and making them aware there is a proxy mechanism they will be subjected to is conceptually problematic (they shouldn't know about it) and impractical.
So, my question is: how to perform the same behavior of PyInstance_NewRaw regardless of the instance's class style?
The type of new-style classes isn't instance, it's the class itself. So, the PyInstance_* methods aren't even meaningful for new-style classes.
In fact, the documentation explicitly explains this:
Note that the class objects described here represent old-style classes, which will go away in Python 3. When creating new types for extension modules, you will want to work with type objects (section Type Objects).
So, you will have to write code that checks whether klass is an old-style or new-style class and does the appropriate thing for each case. An old-style class's type is PyClass_Type, while a new-style class's type is either PyType_Type, or a custom metaclass.
Meanwhile, there is no direct equivalent of PyInstance_NewRaw for new-style classes. Or, rather, the direct equivalent—calling its tp_alloc slot and then adding a dict—will give you a non-functional class. You could try to duplicate all the other appropriate work, but that's going to be tricky. Alternatively, you could use tp_new, but that will do the wrong thing if there's a custom __new__ function in the class (or any of its bases). See the rejected patches from #5180 for some ideas.
But really, what you're trying to do is probably not a good idea in the first place. Maybe if you explained why this is a requirement, and what you're trying to do, there would be a better way to do it.
If the goal is to build objects by creating a new uninitialized instance of the class, then copying over its _dict__ from an initialized prototype, there's a much easier solution that I think will work for you:
__class__ is a writeable attribute. So (showing it in Python; the C API is basically the same, just a lot more verbose, and I'd probably screw up the refcounting somewhere):
class NewStyleDummy(object):
pass
def make_instance(cls, instance_dict):
if isinstance(cls, types.ClassType):
obj = do_old_style_thing(cls)
else:
obj = NewStyleDummy()
obj.__class__ = cls
obj.__dict__ = instance_dict
return obj
The new object will be an instance of cls—in particular, it will have the same class dictionary, including the MRO, metaclass, etc.
This won't work if cls has a metaclass that's required for its construction, or a custom __new__ method, or __slots__… but then your design of copying over the __dict__ doesn't make any sense in those cases anyway. I believe that in any case where anything could possibly work, this simple solution will work.
Calling cls.__new__ seems like a good solution at first, but it actually isn't. Let me explain the background.
When you do this:
foo = Foo(1, 2)
(where Foo is a new-style class), it gets converted into something like this pseudocode:
foo = Foo.__new__(1, 2)
if isinstance(foo, Foo):
foo.__init__(1, 2)
The problem is that, if Foo or one of its bases has defined a __new__ method, it will expect to get the arguments from the constructor call, just like an __init__ method will.
As you explained in your question, you don't know the constructor call arguments—in fact, that's the main reason you can't call the normal __init__ method in the first place. So, you can't call __new__ either.
The base implementation of __new__ accepts and ignores any arguments it's given. So, if none of your classes has a __new__ override or a __metaclass__, you will happen to get away with this, because of a quirk in object.__new__ (a quirk which works differently in Python 3.x, by the way). But those are the exact same cases the previous solution can handle, and that solution works for much more obvious reason.
Put another way: The previous solution depends on nobody defining __new__ because it never calls __new__. This solution depends on nobody defining __new__ because it calls __new__ with the wrong arguments.
I was going through DiveIntoPython and came across this:
Java and Powerbuilder support function overloading by argument list,
i.e. one class can have multiple methods with the same name but a
different number of arguments, or arguments of different types. Other
languages (most notably PL/SQL) even support function overloading by
argument name; i.e. one class can have multiple methods with the same
name and the same number of arguments of the same type but different
argument names. Python supports neither of these; it has no form of
function overloading whatsoever. Methods are defined solely by their
name, and there can be only one method per class with a given name. So
if a descendant class has an __init__ method, it always overrides the
ancestor __init__ method, even if the descendant defines it with a
different argument list. And the same rule applies to any other
method.
Isn't this a major disadvantage that a subclass's __init__ method will always override a superclass's __init__ method? So if I'm initializing some variables and calling some functions in a class class1's __init__, then I derive a subclass class2(class1) of it, I'd have to reinitialize all of class1's variables and call those functions in class2's __init__?
I'm pretty sure I'm misunderstanding all this, so it'd be great if someone clarifies this up.
You're right that defining __init__ in a subclass overrides the superclass's __init__, but you can always use super(CurrentClass, self).__init__ to call the superclass's constructor from the subclass. So, you don't have to "manually" duplicate the superclass's initialization work.
As a side note, even though Python doesn't support method overloading, it supports default arguments (in addition to optional arguments via *args and **kwargs), which means you can easily emulate the behavior of overloaded functions by simply accepting different subsets of arguments in your function/method implementation.
So if I'm initializing some variables and calling some functions in a class class1's __init__, then I derive a subclass class2(class1) of it, I'd have to reinitialize all of class1's variables and call those functions in class2's __init__?
No. You just have to call the superclass's __init__(). Here, and here, you can find how to do it.