I have this code, showing a classic diamond pattern:
class A:
def __init__( self, x ):
print( "A:" + x )
class B( A ):
def __init__( self, x ):
print( "B:" + x )
super().__init__( "b" )
class C( A ):
def __init__( self, x ):
print( "C:" + x )
super().__init__( "c" )
class D( B, C ):
def __init__( self ):
super().__init__( "d" )
d = D()
The output is:
B:d
C:b
A:c
B:d makes sense, since D derives from B.
The A:c I almost get, though I could equally see A:b.
However, the C:b bit doesn't make sense: C does not derive from B.
Could someone explain?
Questions such as this unfortunately do not mention the parameters.
Python uses the C3 linearization algorithm to establish the method resolution order, which is the same order that super delegates in.
Basically, the algorithm keeps lists for every class containing that class and every class it inherits from, for all classes that the class in question inherits from. It then constructs an ordering of classes by taking classes that aren't inherited by any unexamined classes one by one, until it reaches the root, object. Below, I use O for object for brevity:
L(O) = [O]
L(A) = [A] + merge(L(O), [O]) = [A, O]
L(B) = [B] + merge(L(A), [A]) = [B] + merge([A, O], [A]) = [B, A] + merge([O])
= [B, A, O]
L(C) = [C] + merge(L(A), [A]) = [C] + merge([A, O], [A]) = [C, A] + merge([O])
= [C, A, O]
L(D) = [D] + merge(L(B), L(C), [B, C]) = [D] + merge([B, A, O], [C, A, O], [B, C])
= [D, B] + merge([A, O], [C, A, O], [C]) = [D, B, C] + merge([A, O], [A, O])
= [D, B, C, A, O]
Classes in Python are dynamically composed - that includes inheritance.
The C:b output does not imply that B magically inherits from C. If you instantiate either B or C, none knows about the other.
>>> B('root')
B:root
A:b
However, D does know about both B and C:
class D(B,C):
...
There is a lot of technicalities available on this. However, there are basically two parts in how this works:
Direct Base Classes are resolved in order they appear.
B comes before C.
Recursive Base Classes are resolved to not duplicate.
A Base Class of both B and C must follow both.
For the class D, that means the base classes resolve as B->C->A! C has sneaked in between B and A - but only for class D, not for class B.
Note that there is actually another class involved: all classes derive from object by default.
>>> D.__mro__
(__main__.D, __main__.B, __main__.C, __main__.A, object)
You have already written A knowing that there is no base to take its parameters. However, neither B nor C can assume this. They both expect to derive from an A object. Subclassing does imply that both B and C are valid A-objects as well, though!
It is valid for both B and C to precede B and C, since the two are subclasses of A. B->C->A->object does not break that B expects its super class to be of type A.
With all other combinations, one ends up with C preceding nothing (invalid) or object preceding something (invalid). That rules out depth-first resolution B->A->object->C and duplicates B->A->object->C->A->object.
This method resolution order is practical to enable mixins: classes that rely on other classes to define how methods are resolved.
There is a nice example of how a logger for dictionary access can accept both dict and OrderedDict.
# basic Logger working on ``dict``
class LoggingDict(dict):
def __setitem__(self, key, value):
logging.info('Settingto %r' % (key, value))
super().__setitem__(key, value)
# mixin of different ``dict`` subclass
class LoggingOD(LoggingDict, collections.OrderedDict):
pass
You can always check the method resolution order that any class should have:
>>> D.mro()
[__main__.D, __main__.B, __main__.C, __main__.A, object]
As you can see, if everybody is doing the right thing (i.e. calling super), the MRO will be 1st parent, 2nd parent, 1st parent's parent and so on...
You can just think of depth first and then left to right to find the order although ever since python 2.3 the algorithm changed but the outcome is usually the same.
In this case B and C have the same parent A and A doesn't call super
Related
I have created a class foo as below:
class foo():
def __new__(cls, a, b, c, add=True):
return cls.sum(a, b, c) if add else cls.subtract(a, b, c)
def sum(a, b, c):
return a + b + c
def subtract(a, b, c):
return c - b - a
print(foo(1, 2, 3, True))
This program returns the required result as 6. However, I needed to get the clarity of a few concepts:
Is using this methodology or design of OOP correct or is there a better way to do it? I want the class to return a value or any other object(not it's own class instance)
Regardless of the structure above, if sum and subtract are instance methods, how can they be called without instantiating an object as in the above example i.e., print(...)?
I have observed many python APIs and frameworks returning an object or a value through class instantiation.
I am trying to understand the core concepts of OOPs in python please help.
The way you have it now, sum and subtract are indeed instance methods.
>>> foo_obj = object.__new__(foo) # This will actually create a foo object to demonstrate
>>> foo_obj.sum
<bound method foo.sum of <__main__.foo object at 0x0000000000000000>>
>>> type(foo_obj.sum)
<class 'method'>
But that's just because when you access them through an instance, Python dynamically creates a method (Basically just binds the first argument to the object, usually self)
But, you can access the wrapped function through the class:
>>> foo_obj.sum.__func__
<function foo.sum at 0x0000000000000001>
>>> foo.sum
<function foo.sum at 0x0000000000000001>
>>> foo_obj.sum.__func__ is foo.sum
True
So in your __new__ function, it won't bind the first argument, and they call the underlying function instead of making them an instance method.
To fix the warnings, you can make them classmethods or staticmethods. But it is generally bad practice to not return an object that is an instance of the class from the __new__. If you really wanted to use OOP, either subclass int or make a wrapper, so you can have:
>>> class Foo:
__slots__ = 'value',
def __init__(self, a, b, c, add=True):
self.value = self.sum(a, b, c) if add else self.subtract(a, b, c)
#staticmethod
def sum(a, b, c):
return a + b + c
#staticmethod
def subtract(a, b, c):
return c - b - a
>>> foo = Foo(1, 2, 3, True)
>>> foo
<__main__.foo object at 0x0000000000000002>
>>> foo.value
6
or
>>> class Foo(int):
__slots__ = ()
def __new__(cls, a, b, c, add=True):
value = cls.sum(a, b, c) if add else cls.subtract(a, b, c)
return super().__new__(cls, value)
#staticmethod
def sum(a, b, c):
return a + b + c
#staticmethod
def subtract(a, b, c):
return c - b - a
>>> foo = Foo(1, 2, 3, True)
>>> foo
6
>>> type(foo)
<class '__main__.Foo'>
I just want to be able to unpack the instance variables of class foo, for example:
x = foo("name", "999", "24", "0.222")
a, b, c, d = *x
a, b, c, d = [*x]
I am not sure as to which is the correct method for doing so when implementing my own __iter__ method, however, the latter is the one that has worked with mixed "success". I say mixed because doing so with the presented code appears to alter the original instance object x, such that it is no longer valid.
class foo:
def __init__(self, a, b, c, d):
self.a = a
self.b = b
self.c = c
self.d = d
def __iter__(self):
return iter([a, b, c, d])
I have read the myriad posts on this site regarding __iter__, __next__, generators etc., and also a python book and docs.python.org and seem unable to figure what I am not understanding. I've gathered that __iter__ needs to return an iterable (which can be just be self, but I am not sure how that works for what I want). I've also tried various ways of playing around with implementing __next__ and iterating over vars(foo).items(), either by casting to a list or as a dictionary, with no success.
I don't believe this is a duplicate post on account that the only similar questions I've seen present a single list sequence object attribute or employ a range of numbers instead of a four non-container variables.
If you want the instance's variables, you should access them with .self:
def __iter__(self):
return iter([self.a, self.b, self.c, self.d])
with this change,
a, b, c, d = list(x)
will get you the variables.
You could go to the more risky method of using vars(x) or x.__dict__, sort it by the variables name (and that's why it is also a limited one, the variables are saved in no-order), and extract the second element of each tuple. But I would say the iterator is definitely better.
You can store the arguments in an attribute (self.e below) or return them on function call:
class foo:
def __init__(self, *args):
self.a, self.b, self.c, self.d = self.e = args
def __call__(self):
return self.e
x = foo("name", "999", "24", "0.222")
a, b, c, d = x.e
# or
a, b, c, d = x()
When I read about Python's C3 method resolution order, I often hear it reduced to "children come before parents, and the order of subclasses is respected". Yet that only seems to hold true if all the subclasses inherit from the same ancestor.
E.g.
class X():
def FOO(self):
return 11
class A(X):
def doIt(self):
return super().FOO()
def FOO(self):
return 42
class B(X):
def doIt(self):
return super().FOO()
def FOO(self):
return 52
class KID(A,B):
pass
Here the MRO of KID is:
KID, A, B, X
However, if I changed B to be instead:
class B(object):
The MRO of KID becomes:
KID, A, X, B
It seems we are searching A's superclass before we have finished searching all KID's parents.
So it seems a bit less intuitive now than "kids first, breadth first" to "kids first, breadth first if common ancestor else depth first".
It would be quite the gotcha that if a class stopped using a common ancestor the MRO changes (even though the overall hierarchy is the same apart from that one link), and you started calling a deeper ancestor method rather than the one in that class.
All classes in Python 3 have a common base class, object. You can omit the class from the class definition, but it is there unless you already indirectly inherit from object. (In Python 2 you have to explicitly inherit from object to even have the use of super() as this is a new-style class feature).
You changed the base class of B from X to object, but X also inherits from object. The MRO changed to take this into account. The same simplification of the C3 rules (children come before parents, and the order of subclasses is respected) is still applicable here. B comes before object, as does X, and A and B are still listed in the same order. However, X should come before B, as both inherit from object and the subclass A(X) comes before B in KID.
Note that nowhere it is said C3 is breadth first. If anything, it is depth first. See The Python 2.3 Method Resolution Order for an in-depth description of the algorithm and how it applies to Python, but the linearisation of any class is the result of merging the linearisations of the base classes plus the base classes themselves:
L[KID] = KID + merge(L[A], L[B], (A, B))
where L[..] is the C3 linearisation of that class (their MRO).
So the linearisation of A comes before B when merging, making C3 look at hierarchies in depth rather than in breadth. Merging starts with the left-most list and takes any element that doesn't appear in the tails of the other lists (so everything but the first element), then takes the next, etc.
In your first example, L[A] and L[B] are almost the same (they both end in (X, object) as their MRO, with only the first element differing), so merging is simple; you merge (A, X, object) and (B, X, object), and merging these gives you only A from the first list, then the whole second list, ending up with (KID, A, B, X, object) after prepending KID:
L[KID] = KID + merge((A, X, object), (B, X, object), (A, B))
# ^ ^^^^^^
# \ & \ both removed as they appear in the next list
= KID + (A,) + (B, X, object)
= (KID, A, B, X, object)
In your second example, L[A] is unchanged, but L[B] is now (B, object) (dropping X), so merging prefers X before B as (A, X, object) comes first when merging and X doesn't appear in the second list. Thus
L[KID] = KID + merge((A, X, object), (B, object), (A, B))
# ^^^^^^
# \removed as it appears in the next list
= KID + (A, X) + (B, object)
= (KID, A, X, B, object)
class A(object):
def a(self, b=1):
print 'Up'
d = {1 : a}
def b( self ):
print self.d[1]
print self.b
print self.d[1].__get__( self, A )()
# print self.d[1]()
class B( object ):
def a( self ):
print 'here??'
return 10000
d = {1 : a}
def b( self ):
print 'hurray'
o = A()
o.b()
b = B()
type( o ).__dict__['b'].__get__( b, type( b ) )()
Hi Folks,
I was going through Python: Bind an Unbound Method? and http://users.rcn.com/python/download/Descriptor.htm and trying to experiment on my learning.
But, I have hit some new doubts now:-
In the last line of my code, I'm able to use __get__ with b object and instance: type(b). This only works if method b is defined in class B. Why is it so?
Even though the last line requires me to provide a method b in class B, still the method b in class A gets called. Why is it so?
To my utter surprise, after the above step, I notice that the method a of class A is not called by the code of method b of class A; instead, it calls the method a of class B. Why is it so?
I'm quite confused after seeing this behaviour. I might also need to learn more on descriptors. But, it would be a great help if you could answer my doubts
In the last line of my code, I'm able to use __get__ with b object and instance: type(b). This only works if method b is defined in class B. Why is it so?
You have to define a method b in class B, because in A.b you have print self.b. Here, self is an instance of the B class, so self.b means "the b method belonging to this B", not "the b method belonging to the class that this method exists in". If you delete print self.b, then the code will work even if B has no b.
Even though the last line requires me to provide a method b in class B, still the method b in class A gets called. Why is it so?
A.b is being called because you are explicitly accessing it with type( o ).__dict__['b']. Whether you bind that method to an A instance or a B instance doesn't matter; it's still A.b.
To my utter surprise, after the above step, I notice that the method a of class A is not called by the code of method b of class A; instead, it calls the method a of class B. Why is it so?
Even though b belongs to the class A, the self you pass to it is still an instance of the B class. Any attributes you access on that self will be B attributes, and any methods you call on it will be B methods.
Class hierarchies and constructors are related. Parameters from a child class need to be passed to their parent.
So, in Python, we end up with something like this:
class Parent(object):
def __init__(self, a, b, c, ka=None, kb=None, kc=None):
# do something with a, b, c, ka, kb, kc
class Child(Parent):
def __init__(self, a, b, c, d, e, f, ka=None, kb=None, kc=None, kd=None, ke=None, kf=None):
super(Child, self).__init__(a, b, c, ka=ka, kb=kb, kc=kc)
# do something with d, e, f, kd, ke, kf
Imagine this with a dozen child classes and lots of parameters. Adding new parameters becomes very tedious.
Of course one can dispense with named parameters completely and use *args and **kwargs, but that makes the method declarations ambiguous.
Is there a pattern for elegantly dealing with this in Python (2.6)?
By "elegantly" I mean I would like to reduce the number of times the parameters appear. a, b, c, ka, kb, kc all appear 3 times: in the Child constructor, in the super() call to Parent, and in the Parent constructor.
Ideally, I'd like to specify the parameters for Parent's init once, and in Child's init only specify the additional parameters.
I'd like to do something like this:
class Parent(object):
def __init__(self, a, b, c, ka=None, kb=None, kc=None):
print 'Parent: ', a, b, c, ka, kb, kc
class Child(Parent):
def __init__(self, d, e, f, kd='d', ke='e', kf='f', *args, **kwargs):
super(Child, self).__init__(*args, **kwargs)
print 'Child: ', d, e, f, kd, ke, kf
x = Child(1, 2, 3, 4, 5, 6, ka='a', kb='b', kc='c', kd='d', ke='e', kf='f')
This unfortunately doesn't work, since 4, 5, 6 end up assigned to kd, ke, kf.
Is there some elegant python pattern for accomplishing the above?
"dozen child classes and lots of parameters" sounds like a problem irrespective of parameter naming.
I suspect that a little refactoring can peel out some Strategy objects that would simplify this hierarchy and make the super-complex constructors go away.
Well, the only solution I could see is using a mixture of listed variables as well as *args and **kwargs, as such:
class Parent(object):
def __init__(self, a, b, c, ka=None, kb=None, kc=None):
pass
class Child(Parent):
def __init__(self, d, e, f, *args, kd=None, ke=None, kf=None, **kwargs):
Parent.__init__(self, *args, **kwargs)
pass
This way, you could see which parameters are required by each of the classes, but without having to re-type them.
One thing to note is that you lose your desired ordering (a, b, c, d, e, f) as it becomes (d, e, f, a, b, c). I'm not sure if there's a way to have the *args before the other non-named parameters.
I try to group the parameters into their own objects, e.g, instead of passing
sourceDirectory, targetDirectory, temporaryDirectory, serverName, serverPort, I'd have a
DirectoryContext and ServerContext objects.
If the context objects start having more
behavior or logic it might lead to the strategy objects mentioned in here.