Let's say we have a class A, a class B that inherits from A and classes C, D and E that inherit from B.
We want all of those classes to have an attribute _f initialized with a default value, and we want that attribute to be mutable and to have a separate value for each instance of the class, i.e. it should not be a static, constant value of A used by all subclasses.
One way to do this is to define _f in A's __init__ method, and then rely on this method in the subclasses:
class A:
def __init__(self):
self._f = 'default_value'
class B(A):
def __init__(self):
super(B, self).__init__()
class C(B):
def __init__(self):
super(C, self).__init__()
Is there any nice Pythonic way to avoid this, and possibly avoid using metaclasses?
If your goal is to simplify subclass constructors by eliminating the need the call the base class constructor, but still be able to override the default value in subclasses, there's a common paradigm of exploiting the fact that Python will return the class's value for an attribute if it doesn't exist on the instance.
Using a slightly more concrete example, instead of doing...
class Human(object):
def __init__(self):
self._fingers = 10
def __repr__(self):
return 'I am a %s with %d fingers' % (self.__class__.__name__, self._fingers)
class MutatedHuman(Human):
def __init__(self, fingers):
super(MutatedHuman, self).__init__()
self._fingers = fingers
print MutatedHuman(fingers=11)
print Human()
...you can use...
class Human(object):
_fingers = 10
def __repr__(self):
return 'I am a %s with %d fingers' % (self.__class__.__name__, self._fingers)
class MutatedHuman(Human):
def __init__(self, fingers):
self._fingers = fingers
print MutatedHuman(fingers=11)
print Human()
...both of which output...
I am a MutatedHuman with 11 fingers
I am a Human with 10 fingers
The important point being that the line self._fingers = fingers in the second example doesn't overwrite the default value set on class Human, but merely hides it when referenced as self._fingers.
It's slightly hairy when the variable refers to a mutable type, such as a list. You have to be careful not to perform a operation on the default value which will modify it, although it's still safe to do a self.name = value.
What's neat about this approach is it tends to lead to fewer lines of code than other approaches, which is usually a Good Thing (tm).
Related
If I had a parent class attribute that all of the child classes are going to inherit, can I set a default so that when the object is created, it automatically take the default from the parent class and no argument has to be given when creating it?
class F1(object):
def __init__(self, sick="flu"):
self.sick = sick
class F2(F1):
def __init__(self, sick, cure):
super(F2, self).__init__(sick)
self.cure = cure
a = F2("bed rest")
print(a.sick)
print(a.cure)
this is just a sample bit of code to show what I mean. I want every child to inherit the "sick" from the parent so I do not have to send that argument in when creating the object. Is this possible? Is there a different way of doing this same thing? Would it be better to make "sick" a class attribute?
the problem with your code is, that you are declaring F2.__init__ to have two explicit arguments, even though you only want to pass one.
If you want to be able to optionally override the creation argument of F1 you need to handle that yourself (see F3)
class F1(object):
def __init__(self, sick="flu"):
self.sick = sick
class F2(F1):
def __init__(self, cure):
super(F2, self).__init__()
self.cure = cure
class F3(F1):
def __init__(self, cure, sick=None):
if sick is None:
super(F3, self).__init__()
else:
super(F3, self).__init__(sick)
self.cure = cure
a = F2("bed rest")
print("%s -> %s" % (a.sick, a.cure))
b = F3("inhale")
print("%s -> %s" % (b.sick, b.cure))
c = F3(sick="curiosity", cure="none")
print("%s -> %s" % (c.sick, c.cure))
Using super is the standard way of doing this in Python. If you want to override, just override...
class F2(F1):
def __init__(self, sick, cure):
super(F2, self).__init__(sick)
self.cure = cure
self.sick = sick + 1
Adding class attribute could be an option, depending on your need. From your description, I'd say it sounds better, because the default value of sick never change, and that's probably what you need.
Using class attribute does not affect overriding, because when assigning attribute on an instance, class attribute is not touched. An example:
>>> class F:
... a = 1
...
>>> f1, f2 = F(), F()
>>> f2.a = 2
>>> f1.a
1
Let B inherit from A. Suppose that some of B's behavior depends on the class attribute cls_x and we want to set up this dependency during construction of B objects. Since it is not a simple operation, we want to wrap it in a class method, which the constructor will call. Example:
class B(A):
cls_x = 'B'
#classmethod
def cm(cls):
return cls.cls_x
def __init__(self):
self.attr = B.cm()
Problem: cm as well as __init__ will always be doing the same things and their behavior must stay the same in each derived class. Thus, we would like to put them both in the base class and not define it in any of the derived classes. The only difference will be the caller of cm - either A or B (or any of B1, B2, each inheriting from A), whatever is being constructed. So what we'd like to have is something like this:
class A:
cls_x = 'A'
#classmethod
def cm(cls):
return cls.cls_x
def __init__(self):
self.attr = ClassOfWhateverIsInstantiated.cm() #how to do this?
class B(A):
cls_x = 'B'
I feel like it's either something very simple I'm missing about Python's inheritance mechanics or the whole issue should be handled entirely differently.
This is different than this question as I do not want to override the class method, but move its implementation to the base class entirely.
Look at it this way: Your question is essentially "How do I get the class of an instance?". The answer to that question is to use the type function:
ClassOfWhateverIsInstantiated = type(self)
But you don't even need to do that, because classmethods can be called directly through an instance:
def __init__(self):
self.attr = self.cm() # just use `self`
This works because classmethods automatically look up the class of the instance for you. From the docs:
[A classmethod] can be called either on the class (such as C.f()) or on an instance
(such as C().f()). The instance is ignored except for its class.
For ClassOfWhateverIsInstantiated you can just use self:
class A:
cls_x = 'A'
#classmethod
def cm(cls):
return cls.cls_x
def __init__(self):
self.attr = self.cm() # 'self' refers to B, if called from B
class B(A):
cls_x = 'B'
a = A()
print(a.cls_x) # = 'A'
print(A.cls_x) # = 'A'
b = B()
print(b.cls_x) # = 'B'
print(B.cls_x) # = 'B'
To understand this, just remember that class B is inheriting the methods of class A. So when __init__() is called during B's instantiation, it's called in the context of class B, to which self refers.
I have a parent class and two child class. The parent class is an abstract base class that has method combine that gets inherited by the child classes. But each child implements combine differently from a parameter perspective therefore each of their own methods take different number of parameters. In Python, when a child inherits a method and requires re-implementing it, that newly re-implemented method must match parameter by parameter. Is there a way around this? I.e. the inherited method can have dynamic parameter composition?
This code demonstrates that signature of overridden method can easily change.
class Parent(object):
def foo(self, number):
for _ in range(number):
print "Hello from parent"
class Child(Parent):
def foo(self, number, greeting):
for _ in range(number):
print greeting
class GrandChild(Child):
def foo(self):
super(GrandChild,self).foo(1, "hey")
p = Parent()
p.foo(3)
c = Child()
c.foo(2, "Hi")
g = GrandChild()
g.foo()
As the other answer demonstrates for plain classes, the signature of an overridden inherited method can be different in the child than in the parent.
The same is true even if the parent is an abstract base class:
import abc
class Foo:
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def bar(self, x, y):
return x + y
class ChildFoo(Foo):
def bar(self, x):
return super(self.__class__, self).bar(x, 3)
class DumbFoo(Foo):
def bar(self):
return "derp derp derp"
cf = ChildFoo()
print cf.bar(5)
df = DumbFoo()
print df.bar()
Inappropriately complicated detour
It is an interesting exercise in Python metaclasses to try to restrict the ability to override methods, such that their argument signature must match that of the base class. Here is an attempt.
Note: I'm not endorsing this as a good engineering idea, and I did not spend time tying up loose ends so there are likely little caveats about the code below that could make it more efficient or something.
import types
import inspect
def strict(func):
"""Add some info for functions having strict signature.
"""
arg_sig = inspect.getargspec(func)
func.is_strict = True
func.arg_signature = arg_sig
return func
class StrictSignature(type):
def __new__(cls, name, bases, attrs):
func_types = (types.MethodType,) # include types.FunctionType?
# Check each attribute in the class being created.
for attr_name, attr_value in attrs.iteritems():
if isinstance(attr_value, func_types):
# Check every base for #strict functions.
for base in bases:
base_attr = base.__dict__.get(attr_name)
base_attr_is_function = isinstance(base_attr, func_types)
base_attr_is_strict = hasattr(base_attr, "is_strict")
# Assert that inspected signatures match.
if base_attr_is_function and base_attr_is_strict:
assert (inspect.getargspec(attr_value) ==
base_attr.arg_signature)
# If everything passed, create the class.
return super(StrictSignature, cls).__new__(cls, name, bases, attrs)
# Make a base class to try out strictness
class Base:
__metaclass__ = StrictSignature
#strict
def foo(self, a, b, c="blah"):
return a + b + len(c)
def bar(self, x, y, z):
return x
#####
# Now try to make some classes inheriting from Base.
#####
class GoodChild(Base):
# Was declared strict, better match the signature.
def foo(self, a, b, c="blah"):
return c
# Was never declared as strict, so no rules!
def bar(im_a_little, teapot):
return teapot/2
# These below can't even be created. Uncomment and try to run the file
# and see. It's not just that you can't instantiate them, you can't
# even get the *class object* defined at class creation time.
#
#class WrongChild(Base):
# def foo(self, a):
# return super(self.__class__, self).foo(a, 5)
#
#class BadChild(Base):
# def foo(self, a, b, c="halb"):
# return super(self.__class__, self).foo(a, b, c)
Note, like with most "strict" or "private" type ideas in Python, that you are still free to monkey-patch functions onto even a "good class" and those monkey-patched functions don't have to satisfy the signature constraint.
# Instance level
gc = GoodChild()
gc.foo = lambda self=gc: "Haha, I changed the signature!"
# Class level
GoodChild.foo = lambda self: "Haha, I changed the signature!"
and even if you add more complexity to the meta class that checks whenever any method type attributes are updated in the class's __dict__ and keeps making the assert statement when the class is modified, you can still use type.__setattr__ to bypass customized behavior and set an attribute anyway.
In these cases, I imagine Jeff Goldblum as Ian Malcolm from Jurassic Park, looking at you blankly and saying "Consenting adults, uhh, find a way.."
I watched a great video on YouTube about Python metaprogramming. I tried to write the following code (which is almost the same from the video):
class Descriptor:
def __init__(self, name):
self.name = name
def __get__(self, instance, cls):
return instance.__dict__[self.name]
def __set__(self, instance, val):
instance.__dict__[self.name] = val
def __delete__(self, instance):
del instance.__dict__[self.name]
class Type(Descriptor):
ty = object
def __set__(self, instance, val):
if not isinstance(val, self.ty):
raise TypeError("%s should be of type %s" % (self.name, self.ty))
super().__set__(instance, val)
class String(Type):
ty = str
class Integer(Type):
ty = int
class Positive(Descriptor):
def __set__(self, instance, val):
if val <= 0:
raise ValueError("Must be > 0")
super().__set__(instance, val)
class PositiveInteger(Integer, Positive):
pass
class Person(metaclass=StructMeta):
_fields = ['name', 'gender', 'age']
name = String('name')
gender = String('gender')
age = PositiveInteger('age')
So PositiveInteger is inherited from Integer and Positive, and both classes have __get__ method defined to do some validation. I wrote some test code to convince myself that both methods will run:
class A:
def test(self):
self.a = 'OK'
class B:
def test(self):
self.b = 'OK'
class C(A, B):
pass
c = C()
c.test()
print(self.a)
print(self.b)
Only to find that only the first print statement works. The second will raise an AttributeError, which indicates that when there's name conflict, the first base class wins.
So I wonder why both validations work? It's even more weird that when only the Integer check passes (e.g. person.age = -3), it's super().__set__(instance, val) has no effect, leaving person.age untouched.
The validation logic of both Positive and Integer runs because both Type and Positive have this line in __set__:
super().__set__(instance, val)
This doesn't skip to Descriptor.__set__. Instead, it calls the next method in method resolution order. Type.__set__ gets called, and its super().__set__(instance, val) calls Positive.__set__. Positive.__set__ runs its validation and calls Descriptor.__set__, which does the setting. This behavior is one of the reasons we have super.
If you wanted your test methods to behave like that, you would need to do two things. First, you would need to make A and B inherit from a common base class with a test method that doesn't do anything, so the super chains end at a place with a test method instead of going to object:
class Base:
def test():
pass
Then, you would need to add super().test() to both A.test and B.test:
class A(Base):
def test(self):
self.a = 'OK'
super().test()
class B(Base):
def test(self):
self.b = 'OK'
super().test()
For more reading, see Python's super() considered super.
Sorry, my bad.
The video gave perfect explanation just minute after where I paused and asked this question.
So when multiple inheritance happends, there's MRO thing (Method Resolution Order) defined in each class that determines the resolution order of methods in the super() chain.
The order is determined by depth-first search, e.g.
class A:
pass
class B(A):
pass
class C(B):
pass
class D(A):
pass
class E(C, D):
pass
E.__mro__ will be:
(<class '__main__.E'>, <class '__main__.C'>, <class '__main__.B'>, <class '__main__.D'>, <class '__main__.A'>, <class 'object'>)
One thing to notice is that A will appear in the inheritance tree multiple times, and in the MRO list it will only be in the last place where all A's appear.
Here's the trick: the call to super() won't necessarily go to its base. Instead, it'll find in the MRO list what comes next.
So to explain what happens in the code:
The super() call in Integer.__get__ (which is inherited from Type.__get__) won't go to Descriptor.__get__, because Descriptor appears last in the MRO list. It will fall into Positive.__set__, and then its super() will fall into Descriptor, which will eventually set the value of the attribute.
In class B below I wanted the __set__ function in class A to be called whenever you assign a value to B().a . Instead, setting a value to B().a overwrites B().a with the value. Class C assigning to C().a works correctly, but I wanted to have a separate instance of A for each user class, i.e. I don't want changing 'a' in one instance of C() to change 'a' in all other instances. I wrote a couple of tests to help illustrate the problem. Can you help me define a class that will pass both test1 and test2?
class A(object):
def __set__(self, instance, value):
print "__set__ called: ", value
class B(object):
def __init__(self):
self.a = A()
class C(object):
a = A()
def test1( class_in ):
o = class_in()
o.a = "test"
if isinstance(o.a, A):
print "pass"
else:
print "fail"
def test2( class_in ):
o1, o2 = class_in(), class_in()
if o1.a is o2.a:
print "fail"
else:
print "pass"
Accordingly to the documentation:
The following methods only apply when an instance of the class containing
the method (a so-called descriptor
class) appears in the class dictionary
of another new-style class, known as
the owner class. In the examples
below, “the attribute” refers to the
attribute whose name is the key of the
property in the owner class’ __dict__.
Descriptors can only be implemented as
new-style classes themselves.
So you can't have descriptors on instances.
However, since the descriptor gets a ref to the instance being used to access it, just use that as a key to storing state and you can have different behavior depending on the instance.
Here's a class that can pass the original tests, but don't try using it in most situations. it fails the isinstance test on itself!
class E(object):
def __new__(cls, state):
class E(object):
a = A(state)
def __init__(self, state):
self.state = state
return E(state)
#>>> isinstance(E(1), E)
#False
I was bitten by a similar issue in that I wanted to class objects with attributes governed by a descriptor. When I did this, I noticed that the attributes were being overwritten in all of the objects such that they weren't individual.
I raised a SO question and the resultant answer is here: class attribute changing value for no reason
A good document link discussing descriptors is here: http://martyalchin.com/2007/nov/24/python-descriptors-part-2-of-2/
An example descriptor from the aforementioned link is below:
class Numberise(object):
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
if self.name not in instance.__dict__:
raise (AttributeError, self.name)
return '%o'%(instance.__dict__[self.name])
def __set__(self, instance, value):
print ('setting value to: %d'%value)
instance.__dict__[self.name] = value