Lets define simple class decorator function, which creates subclass and adds 'Dec' to original class name only:
def decorate_class(klass):
new_class = type(klass.__name__ + 'Dec', (klass,), {})
return new_class
Now apply it on a simple subclass definition:
class Base(object):
def __init__(self):
print 'Base init'
#decorate_class
class MyClass(Base):
def __init__(self):
print 'MyClass init'
super(MyClass, self).__init__()
Now, if you try instantiate decorated MyClass, it will end up in an infinite loop:
c = MyClass()
# ...
# File "test.py", line 40, in __init__
# super(MyClass, self).__init__()
# RuntimeError: maximum recursion depth exceeded while calling a Python object
It seems, super can't handle this case and does not skip current class from inheritance chain.
The question, how correctly use class decorator on classes using super ?
Bonus question, how get final class from proxy-object created by super ? Ie. get object class from super(Base, self).__init__ expression, as determined parent class defining called __init__.
If you just want to change the class's .__name__ attribute, make a decorator that does that.
from __future__ import print_function
def decorate_class(klass):
klass.__name__ += 'Dec'
return klass
class Base(object):
def __init__(self):
print('Base init')
#decorate_class
class MyClass(Base):
def __init__(self):
print('MyClass init')
super(MyClass, self).__init__()
c = MyClass()
cls = c.__class__
print(cls, cls.__name__)
Python 2 output
MyClass init
Base init
<class '__main__.MyClassDec'> MyClassDec
Python 3 output
MyClass init
Base init
<class '__main__.MyClass'> MyClassDec
Note the difference in the repr of cls. (I'm not sure why you'd want to change a class's name though, it sounds like a recipe for confusion, but I guess it's ok for this simple example).
As others have said, an #decorator isn't intended to create a subclass. You can do it in Python 3 by using the arg-less form of super (i.e., super().__init__()). And you can make it work in both Python 3 and Python 2 by explicitly supplying the parent class rather than using super.
from __future__ import print_function
def decorate_class(klass):
name = klass.__name__
return type(name + 'Dec', (klass,), {})
class Base(object):
def __init__(self):
print('Base init')
#decorate_class
class MyClass(Base):
def __init__(self):
print('MyClass init')
Base.__init__(self)
c = MyClass()
cls = c.__class__
print(cls, cls.__name__)
Python 2 & 3 output
MyClass init
Base init
<class '__main__.MyClassDec'> MyClassDec
Finally, if we just call decorate_class using normal function syntax rather than as an #decorator we can use super.
from __future__ import print_function
def decorate_class(klass):
name = klass.__name__
return type(name + 'Dec', (klass,), {})
class Base(object):
def __init__(self):
print('Base init')
class MyClass(Base):
def __init__(self):
print('MyClass init')
super(MyClass, self).__init__()
MyClassDec = decorate_class(MyClass)
c = MyClassDec()
cls = c.__class__
print(cls, cls.__name__)
The output is the same as in the last version.
Since your decorator returns an entirely new class with different name, for that class MyClass object doesn't even exist. This is not the case class decorators are intended for. They are intended to add additional functionality to an existing class, not outright replacing it with some other class.
Still if you are using Python3, solution is simple -
#decorate_class
class MyClass(Base):
def __init__(self):
print 'MyClass init'
super().__init__()
Otherwise, I doubt there is any straight-forward solution, you just need to change your implementation. When you are renaming the class, you need to rewrite overwrite __init__ as well with newer name.
The problem is that your decorator creates a subclass of the original one. That means that super(Myclass) now point to... the original class itself!
I cannot even explain how the 0 arg form of super manages to do the job in Python 3, I could not find anything explicit in the reference manual. I assume it must use the class in which it is used at the time of declaration. But I cannot imagine a way to get that result in Python2.
If you want to be able to use super in the decorated class in Python 2, you should not create a derived class, but directly modify the original class in place.
For example, here is a decorator that prints a line before and after calling any method:
def decorate_class(klass):
for name, method in klass.__dict__.iteritems(): # iterate the class attributes
if isinstance(method, types.FunctionType): # identify the methods
def meth(*args, **kwargs): # define a wrapper
print "Before", name
method(*args, **kwargs)
print "After", name
setattr(klass, name, meth) # tell the class to use the wrapper
return klass
With your example it gives as expected:
>>> c = MyClass()
Before __init__
MyClass init
Base init
After __init__
Related
In most Python examples, when super is used to call a parent class's constructors, it appears at the top.
Is it bad form to have it at the bottom of an init method?
In the examples below, super is at the bottom of A's constructor, but at the top of B's constructor.
class A:
def __init__(self):
# Do some stuff
b = result_of_complex_operation()
super(A, self).__init__(b)
class B:
def __init__(self):
super(A, self).__init__(b)
# Do some stuff
This totally depends on the use case. Consider this.
class Foo():
def __init__(self):
print(self.name)
#property
def name(self):
return self.__class__.__name__
class Bar(Foo):
def __init__(self, name):
self.name = name
super().__init__()
#property
def name(self):
return self.__name
#name.setter
def name(self, name):
self.__name = name
If you'd invoke super() before setting self.name within Bar.__init__ you'd get an AttributeError because the required name has not yet been set.
Is it bad form to have it at the bottom of an init method?
You're asking the wrong question. Regardless of whether it's bad from or not, there are valid use cases for moving the superclass initialization to the bottom of a sub-class's constructor. Where to put the call to the superclass's constructor entirely depends on the implementation of the superclass's constructor.
For example, suppose you have a superclass. When constructing the superclass, you want to give an attribute a certain value depending on an attribute of the subclasses:
class Superclass:
def __init__(self):
if self.subclass_attr:
self.attr = 1
else:
self.attr = 2
As you can see from above, we expect the subclasses to have the attribute subclass_attr. So what does this mean? We can't initialize Supperclass until we've given the subclasses the subclass_attr attribute.
Thus, we have to defer calling the superclass's constructor until we initialize subclass_attr. In other words, the call to super will have to be put at the bottom of a subclasses constructor:
class Subclass(Superclass):
def __init__(self):
self.subclass_attr = True
super(Superclass, self).__init__()
In the end, the choice of where to put super should not be based upon some style, but on what's necessary.
I'm trying to define self.data inside a class inheriting from a class
class Object():
def __init__(self):
self.data="1234"
class New_Object(Object):
# Code changing self.data here
But I ran into an issue.
class Object():
def __init__(self):
self.data="1234"
So I have the beginning class here, which is imported from elsewhere, and let's say that the class is a universal one so I can't modify the original at all.
In the original, the instance is referred to as "self" inside the class, and it is defined as self inside the definition __init__.
class New_Object(Object):
# Code changing self.data here
So if I wanted to inherit from the class Object, but define self.data inside New_Object, I thought I would have to define __init__ in New_Object, but this overrides the __init__ from New_Object
Is there any way I could do this without copypasting the __init__ from Object?
You use super to call the original implementation.
class New_Object(Object):
def __init__(self):
super(NewObject, self).__init__()
self.info = 'whatever'
That's what super is for:
class NewObject(Object):
def __init__(self):
super(NewObject, self).__init__()
# self.data exists now, and you can modify it if necessary
You can use super().__init__() to call Object.__init__() from New_Object.__init__().
What you would do:
class Object:
def __init__(self):
print("Object init")
self.data = "1234"
class New_Object(Object):
def __init__(self):
print("calling super")
super().__init__()
print("data is now", self.data)
self.data = self.data.split("3")
o = New_Object()
# calling super
# Object init
# data is now 1234
Note that you do not have to give any arguments to super(), as long as you are using Python 3.
The answer is that you call the superclass's __init__ explicitly during the subclass's __init__. This can be done either of two ways:
Object.__init__(self) # requires you to name the superclass explicitly
or
super(NewObject, self).__init__() # requires you to name the subclass explicitly
The latter also requires you to ensure that you're using "new-style" classes: in Python 3 that's always the case, but in Python 2 you must be sure to inherit from the builtin object class. In Python 3 it can actually be expressed even more simply:
super().__init__()
Personally, in most of my code the "disadvantage" of having to name the superclass explicitly is no disadvantage at all, and Object.__init__() lends transparency since it makes it absolutely clear what is being called. This is because most of my code is single-inheritance only. The super route comes into its own when you have multiple inheritance. See What does 'super' do in Python?
Python 2 example:
class Object(object):
def __init__(self):
self.data = "1234"
class NewObject:
def __init__(self):
# subclass-specific stuff
super(NewObject, self).__init__()
# more subclass-specific stuff
If you have multiple layers of inheritance and know that a particular variable exists, is there a way to trace back to where the variable originated? Without having to navigate backwards by looking through each file and classes. Possibly calling some sort of function that will do it?
Example:
parent.py
class parent(object):
def __init__(self):
findMe = "Here I am!"
child.py
from parent import parent
class child(parent):
pass
grandson.py
from child import child
class grandson(child):
def printVar(self):
print self.findMe
Try to locate where the findMe variable came from with a function call.
If the "variable" is an instance variable - , so , if at any point in chain of __init__ methods you do:
def __init__(self):
self.findMe = "Here I am!"
It is an instance variable from that point on, and cannot, for all effects, be made distinct of any other instance variable. (Unless you put in place a mechanism, like a class with a special __setattr__ method, that will keep track of attributes changing, and introspect back which part of the code set the attribute - see last example on this answer)
Please also note that on your example,
class parent(object):
def __init__(self):
findMe = "Here I am!"
findMe is defined as a local variable to that method and does not even exist after __init__ is finished.
Now, if your variable is set as a class attribute somewhere on the inheritance chain:
class parent(object):
findMe = False
class childone(parent):
...
It is possible to find the class where findMe is defined by introspecting each class' __dict__ in the MRO (method resolution order) chain . Of course, there is no way, and no sense, in doing that without introspecting all classes in the MRO chain - except if one keeps track of attributes as defined, like in the example bellow this - but introspecting the MRO itself is a oneliner in Python:
def __init__(self):
super().__init__()
...
findme_definer = [cls for cls in self.__class__.__mro__ if "findMe" in cls.__dict__][0]
Again - it would be possible to have a metaclass to your inheritance chain which would keep track of all defined attributes in the inheritance tree, and use a dictionary to retrieve where each attribute is defined. The same metaclass could also auto-decorate all __init__ (or all methods), and set a special __setitem__ so that it could track instance attributes as they are created, as listed above.
That can be done, is a bit complicated, would be hard to maintain, and probably is a signal you are taking the wrong approach to your problem.
So, the metaclass to record just class attributes could simply be (python3 syntax - define a __metaclass__ attribute on the class body if you are still using Python 2.7):
class MetaBase(type):
definitions = {}
def __init__(cls, name, bases, dct):
for attr in dct.keys():
cls.__class__.definitions[attr] = cls
class parent(metaclass=MetaBase):
findMe = 5
def __init__(self):
print(self.__class__.definitions["findMe"])
Now, if one wants to find which of the superclasses defined an attribute of the currentclass, just a "live" tracking mechanism, wrapping each method in each class can work - it is a lot trickier.
I've made it - even if you won't need this much, this combines both methods - keeping track of class attributes in the class'class definitions and on an instance _definitions dictionary - since in each created instance an arbitrary method might have been the last to set a particular instance attribute: (This is pure Python3, and maybe not that straighforward porting to Python2 due to the "unbound method" that Python2 uses, and is a simple function in Python3)
from threading import current_thread
from functools import wraps
from types import MethodType
from collections import defaultdict
def method_decorator(func, cls):
#wraps(func)
def wrapper(self, *args, **kw):
self.__class__.__class__.current_running_class[current_thread()].append(cls)
result = MethodType(func, self)(*args, **kw)
self.__class__.__class__.current_running_class[current_thread()].pop()
return result
return wrapper
class MetaBase(type):
definitions = {}
current_running_class = defaultdict(list)
def __init__(cls, name, bases, dct):
for attrname, attr in dct.items():
cls.__class__.definitions[attr] = cls
if callable(attr) and attrname != "__setattr__":
setattr(cls, attrname, method_decorator(attr, cls))
class Base(object, metaclass=MetaBase):
def __setattr__(self, attr, value):
if not hasattr(self, "_definitions"):
super().__setattr__("_definitions", {})
self._definitions[attr] = self.__class__.current_running_class[current_thread()][-1]
return super().__setattr__(attr,value)
Example Classes for the code above:
class Parent(Base):
def __init__(self):
super().__init__()
self.findMe = 10
class Child1(Parent):
def __init__(self):
super().__init__()
self.findMe1 = 20
class Child2(Parent):
def __init__(self):
super().__init__()
self.findMe2 = 30
class GrandChild(Child1, Child2):
def __init__(self):
super().__init__()
def findall(self):
for attr in "findMe findMe1 findMe2".split():
print("Attr '{}' defined in class '{}' ".format(attr, self._definitions[attr].__name__))
And on the console one will get this result:
In [87]: g = GrandChild()
In [88]: g.findall()
Attr 'findMe' defined in class 'Parent'
Attr 'findMe1' defined in class 'Child1'
Attr 'findMe2' defined in class 'Child2'
Python 3 doesn't allow you to reference a class inside its body (except in methods):
class A:
static_attribute = A()
def __init__(self):
...
This raises a NameError in the second line because 'A' is not defined.
Alternatives
I have quickly found one workaround:
class A:
#property
#classmethod
def static_property(cls):
return A()
def __init__(self):
...
Although this isn't exactly the same since it returns a different instance every time (you could prevent this by saving the instance to a static variable the first time).
Are there simpler and/or more elegant alternatives?
EDIT:
I have moved the question about the reasons for this restriction to a separate question
The expression A() can't be run until the class A has been defined. In your first block of code, the definition of A is not complete at the point you are trying to execute A().
Here is a simpler alternative:
class A:
def __init__(self):
...
A.static_attribute = A()
When you define a class, Python immediately executes the code within the definition. Note that's different than defining a function where Python compiles the code, but doesn't execute it.
That's why this will create an error:
class MyClass(object):
a = 1 / 0
But this won't:
def my_func():
a = 1 / 0
In the body of A's class definition, A is not yet defined, so you can't reference it until after it's been defined.
There are several ways you can accomplish what you're asking, but it's not clear to me why this would be useful in the first place, so if you can provide more details about your use case, it'll be easier to recommend which path to go down.
The simplest would be what khelwood posted:
class A(object):
pass
A.static_attribute = A()
Because this is modifying class creation, using a metaclass could be appropriate:
class MetaA(type):
def __new__(mcs, name, bases, attrs):
cls = super(MetaA, mcs).__new__(mcs, name, bases, attrs)
cls.static_attribute = cls()
return cls
class A(object):
__metaclass__ = MetaA
Or you could use descriptors to have the instance lazily created or if you wanted to customize access to it further:
class MyDescriptor(object):
def __get__(self, instance, owner):
owner.static_attribute = owner()
return owner.static_attribute
class A(object):
static_attribute = MyDescriptor()
Using the property decorator is a viable approach, but it would need to be done something like this:
class A:
_static_attribute = None
#property
def static_attribute(self):
if A._static_attribute is None:
A._static_attribute = A()
return A._static_attribute
def __init__(self):
pass
a = A()
print(a.static_attribute) # -> <__main__.A object at 0x004859D0>
b = A()
print(b.static_attribute) # -> <__main__.A object at 0x004859D0>
You can use a class decorator:
def set_static_attribute(cls):
cls.static_attribute = cls()
return cls
#set_static_attribute
class A:
pass
Now:
>>>> A.static_attribute
<__main__.A at 0x10713a0f0>
Applying the decorator on top of the class makes it more explicit than setting static_attribute after a potentially long class definition. The applied decorator "belongs" to the class definition. So if you move the class around in your source code you will more likely move it along than an extra setting of the attribute outside the class.
If I am inheriting from a class and not changing anything in a method, is it required to use super to initialize the method from the parent class?
class A:
def __init__(self):
self.html = requests.get("example.com").text
class B(A):
def __init__(self):
# is this needed?
super(B, self).__init__()
def new_method(self):
print self.html
Because you created a __init__ method in your class B, it overrides the method in class A. If you want it executed, you'll have to use super(), yes.
However, if you are not doing anything else in B.__init__, you may as well just omit it:
class A:
def __init__(self):
self.html = requests.get("example.com").text
class B(A):
def new_method(self):
print self.html
If you want to do anything in addition to what A.__init__() does, then it makes sense to create a B.__init__() method, and from that method, invoke the parent __init__.
It's not needed to define the overriding method at all. Python's default behavior is to call the method on the parent class (the next class in the method resolution order) if the current class doesn't override it.
>>> class Foo(object):
... def __init__(self):
... print("Foo")
...
>>> class Bar(Foo): pass
...
>>> Bar()
Foo
<__main__.Bar object at 0x7f5ac7d1b990>
Notice "Foo" got printed when I initialized a Bar instance.
If you do define the method, you need to call the super class's method (either explicitly or via super) if you want to make sure that it gets called.