I have the following setup:
class A:
def __init__(self, **kwargs):
# Some variables initialized
for k, v in kwargs.items():
setattr(self, k, v)
class B(A):
def __init__(self, **kwargs):
A.__init__(self, **kwargs)
self._b = {}
for k, v in kwargs.items():
setattr(self, k, v)
#property
def b(self):
return self._b
#b.setter
def b(self, value):
self._b.update(value)
class C(B):
def __init__(self, **kwargs):
B.__init__(self, **kwargs)
# Some variables initialized
for k, v in kwargs.items():
setattr(self, k, v)
When I now create a new instance of C I get the following error:
AttributeError: 'C' object has no attribute '_b'
Now this makes sense since B._b hasn't been initialized when A.__init__(self, **kwargs) is being called. I can resolve this issue simply by re-ordering the B's initialization like so,
class B(A):
def __init__(self, **kwargs):
self._b = {}
A.__init__(self, **kwargs)
for k, v in kwargs.items():
setattr(self, k, v)
I'd like to understand if there is a recommended/best practice approach when I need to pass kwargs from child to parent classes during initialization? It seems to me like the following things would work,
Re-order the initialization like I have above
Assign kwargs in each child class then pop them and pass the remaining kwargs along to the parent initialization
Something better
Hoping to get some approaches for 3.
The issue you have is with these loops:
for k, v in kwargs.items():
setattr(self, k, v)
You have one in each class, and that means that every one of the classes is setting all the keyword arguments as attributes on self.
When that loop runs in A, it fails because B has a property that needs initializing before it can work.
As you noted in the question, a quick fix would be to make sure that B sets up its dictionary before it runs A.__init__:
class B(A):
def __init__(self, **kwargs):
_b = {} # set this up first
A.__init__(self, **kwargs) # before calling the superclass
for k, v in kwargs.items():
setattr(self, k, v)
But there's probably a better approach that would let you avoid the redundant loops. I'd suggest explicitly naming the keyword arguments you expect in each class. That way b will only be seen by the B class, not by A, nor C (except as part of kwargs).
class A:
def __init__(self, *, a): # a is keyword-only arg, no kwargs accepted here
self.a = a
class B(A):
def __init__(self, *, b, **kwargs):
super().__init__(**kwargs) # doesn't mess with b!
self._b = {}
self.b = b
#property
def b(self):
...
class C(B):
def __init__(self, *, c, **kwargs):
super().__init__(**kwargs)
self.c = c
Now you can call C(a="foo", b={1: 2}, c="bar") and each class will only pay attention to the attribute it cares about.
Related
I am trying to design a class structure that allows the user to define their own class that overloads predefined methods in other classes. In this case the user would create the C class to overload the "function" method in D. The user created C class has common logic for other user created classes A and B so they inherit from C to overload "function" but also inherit from D to use D's other methods. The issue I am having is how to pass "value" from A and B to D and ignore passing it to C. What I currently have written will produce an error as C does not have "value" as an argument.
I know that I can add "value" (or *args) to C's init method and the super call but I don't want to have to know what inputs other classes need in order to add new classes to A and B. Also, if I swap the order of C and D I won't get an error but then I don't use C's overloaded "function". Is there an obvious way around this?
class D(SomethingElse):
def __init__(self, value, **kwargs):
super(D, self).__init__(**kwargs)
self.value = value
def function(self):
return self.value
def other_method(self):
pass
class C(object):
def __init__(self):
super(C, self).__init__()
def function(self):
return self.value*2
class B(C, D):
def __init__(self, value, **kwargs):
super(B, self).__init__(value, **kwargs)
class A(C, D):
def __init__(self, value, **kwargs):
super(A, self).__init__(value, **kwargs)
a = A(3)
print(a.function())
>>> 6
Essentially, there are two things you need to do to make your __init__ methods play nice with multiple inheritance in Python:
Always take a **kwargs parameter, and always call super().__init__(**kwargs), even if you think you are the base class. Just because your superclass is object doesn't mean you are last (before object) in the method resolution order.
Don't pass your parent class's __init__ arguments explicitly; only pass them via **kwargs. Your parent class isn't necessarily the next one after you in the method resolution order, so positional arguments might be passed to the wrong other __init__ method.
This is called "co-operative subclassing". Let's try with your example code:
class D:
def __init__(self, value, **kwargs):
self.value = value
super().__init__(**kwargs)
def function(self):
return self.value
class C:
# add **kwargs parameter
def __init__(self, **kwargs):
# pass kwargs to super().__init__
super().__init__(**kwargs)
def function(self):
return self.value * 2
class B(C, D):
# don't take parent class's value arg explicitly
def __init__(self, **kwargs):
# pass value arg via kwargs
super().__init__(**kwargs)
class A(C, D):
# don't take parent class's value arg explicitly
def __init__(self, **kwargs):
# pass value arg via kwargs
super().__init__(**kwargs)
Demo:
>>> a = A(value=3)
>>> a.value
3
>>> a.function()
6
Note that value must be passed to the A constructor as a keyword argument, not as a positional argument. It's also recommended to set self.value = value before calling super().__init__.
I've also simplified class C(object): to class C:, and super(C, self) to just super() since these are equivalent in Python 3.
So I'm trying to understand the point of A AND B. I'm guessing that maybe you want to mix in the superclass behavior and sometimes have local behavior. So suppose A is just mixing together behaviors, and B has some local behavior and state.
If you don't need your own state, you probably don't need an __init__. So for A and C just omit __init__.
class SomethingElse(object):
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
class D(SomethingElse):
def __init__(self, value, *args, **kwargs):
super(D, self).__init__(*args, **kwargs)
self.value = value
def function(self):
return self.value
def other_method(self):
return self.__dict__
class C(object):
#def __init__(self):
# super(C, self).__init__()
def function(self):
return self.value*2
class B(C, D):
def __init__(self, value, bstate, *args, **kwargs):
super(B, self).__init__(value, *args, **kwargs)
self.bstate = bstate
def __repr__(self):
return (self.__class__.__name__ + ' ' +
self.bstate + ' ' + str(self.other_method()))
class A(C, D):
pass
a = A(3)
b = B(21, 'extra')
a.function()
6
b.function()
42
repr(a)
'<xx.A object at 0x107cf5e10>'
repr(b)
"B extra {'args': (), 'bstate': 'extra', 'value': 21, 'kwargs': {}}"
I've kept python2 syntax assuming you might still be using it, but as another answer points out, python3 simplifies super() syntax, and you really should be using python3 now.
If you swap C and D you are changing the python method resolution order, and that will indeed change the method to which a call to A.function resolves.
I want to add keyword arguments to a derived class, but can't figure out how to go about it. Trying the obvious
class ClassA(some.package.Class):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
class ClassB(ClassA):
def __init__(self, *args, a='A', b='B', c='C', **kwargs):
super().__init__(*args, **kwargs)
self.a=a
self.b=b
self.c=c
fails because I can't list parameters like that for ClassB's __init__. And
class ClassB(ClassA):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.a=a
self.b=b
self.c=c
of course doesn't work because the new keywords aren't specified.
How do I add keyword arguments to the __init__ for a derived class?
Try doing it like this:
class ClassA:
def __init__(self, *args, **kwargs):
pass
class ClassB(ClassA):
def __init__(self, *args, **kwargs):
self.a = kwargs.pop('a', 'A')
self.b = kwargs.pop('b', 'B')
self.c = kwargs.pop('c', 'C')
super().__init__(*args, **kwargs)
Effectively you add the keyword arguments a, b and c to ClassB, while passing on other keyword arguments to ClassA.
All you need do is rearrange the arguments.
def __init__(self, a='A', b='B', c='C', *args, **kwargs):
Effect of def __init__(self, a='A', b='B', c='C', *args, **kwargs):
Modifying the OP's code with the child class's constructor having the above signature:
class BaseClass(object):
def __init__(self, *args, **kwargs):
self.args = args
for k, v in kwargs.items():
setattr(self, k, v)
class ClassA(BaseClass):
def __init__(self, *args, **kwargs):
super(ClassA, self).__init__(*args, **kwargs)
class ClassB(ClassA):
def __init__(self, a='A', b='B', c='C', *args, **kwargs):
self.a = a
self.b = b
self.c = c
super(ClassA, self).__init__(*args, **kwargs)
A = ClassA('hello', 'world', myname='hal',myemail='hal#hal.hal')
B = ClassB('hello', 'world', myname='hal', myemail='hal#hal.hal')
print("CLASS A:", A.__dict__)
print("CLASS B:", B.__dict__)
# yields the following:
CLASS A: {'args': ('hello', 'world'), 'myname': 'hal', 'myemail': 'hal#hal.hal'}
CLASS B: {'args': (), 'myname': 'hal', 'a': 'hello', 'myemail': 'hal#hal.hal', 'c': 'C', 'b': 'world'}
The code will not throw an exception, however, a, b & c are now positionally the first 3 arguments passed to the method (ignoring self) , as seen from the two object's dicts.
Accessing the attributes a, b & c shows this other effect
>>> B.a # expect to see 'A'
'hello'
>>> B.b # similarly
'world'
>>> B.c # but this retails the default value
'C'
AFAIK, you can't add new keyword only arguments to the method signature. Please correct me if I'm wrong.
However, both #aknuds1 and #Oleh Prypin provided solutions that effectively add new keyword arguments to the child class. Oleh's solution is a bit more clever, but I find aknuds1's version easier to understand.
pop elements from the kwargs dict, coalesced to the default value
assign to the instance attribute using setattr
call parent constructor with args, kwargs
Use Python 3.x, which makes keyword-only arguments valid.
Or use this workaround...
class ClassB(ClassA):
def __init__(self, *args, **kwargs):
for arg, default in [('a', 'A'), ('b', 'B'), ('c', 'C')]:
setattr(self, arg, kwargs.pop(arg, default))
super(ClassA, self).__init__(*args, **kwargs)
I am trying to override the __setattr__ method of a Python class, since I want to call another function each time an instance attribute changes its value. However, I don't want this behaviour in the __init__ method, because during this initialization I set some attributes which are going to be used later:
So far I have this solution, without overriding __setattr__ at runtime:
class Foo(object):
def __init__(self, a, host):
object.__setattr__(self, 'a', a)
object.__setattr__(self, 'b', b)
result = self.process(a)
for key, value in result.items():
object.__setattr__(self, key, value)
def __setattr__(self, name, value):
print(self.b) # Call to a function using self.b
object.__setattr__(self, name, value)
However, I would like to avoid these object.__setattr__(...) and override __setattr__ at the end of the __init__ method:
class Foo(object):
def __init__(self, a, b):
self.a = a
self.b = b
result = self.process(a)
for key, value in result.items():
setattr(self, key, value)
# override self.__setattr__ here
def aux(self, name, value):
print(self.b)
object.__setattr__(self, name, value)
I have tried with self.__dict__['__setitem__'] = self.aux and object.__setitem__['__setitem__'] = self.aux, but none of these attemps has effect. I have read this section of the data model reference, but it looks like the assignment of the own __setattr__ is a bit tricky.
How could be possible to override __setattr__ at the end of __init__, or at least have a pythonic solution where __setattr__ is called in the normal way only in the constructor?
Unfortunately, there's no way to "override, after init" python special methods; as a side effect of how that lookup works. The crux of the problem is that python doesn't actually look at the instance; except to get its class; before it starts looking up the special method; so there's no way to get the object's state to affect which method is looked up.
If you don't like the special behavior in __init__, you could refactor your code to put the special knowledge in __setattr__ instead. Something like:
class Foo(object):
__initialized = False
def __init__(self, a, b):
try:
self.a = a
self.b = b
# ...
finally:
self.__initialized = True
def __setattr__(self, attr, value):
if self.__initialzed:
print(self.b)
super(Foo, self).__setattr__(attr, value)
Edit: Actually, there is a way to change which special method is looked up, so long as you change its class after it has been initialized. This approach will send you far into the weeds of metaclasses, so without further explanation, here's how that looks:
class AssignableSetattr(type):
def __new__(mcls, name, bases, attrs):
def __setattr__(self, attr, value):
object.__setattr__(self, attr, value)
init_attrs = dict(attrs)
init_attrs['__setattr__'] = __setattr__
init_cls = super(AssignableSetattr, mcls).__new__(mcls, name, bases, init_attrs)
real_cls = super(AssignableSetattr, mcls).__new__(mcls, name, (init_cls,), attrs)
init_cls.__real_cls = real_cls
return init_cls
def __call__(cls, *args, **kwargs):
self = super(AssignableSetattr, cls).__call__(*args, **kwargs)
print "Created", self
real_cls = cls.__real_cls
self.__class__ = real_cls
return self
class Foo(object):
__metaclass__ = AssignableSetattr
def __init__(self, a, b):
self.a = a
self.b = b
for key, value in process(a).items():
setattr(self, key, value)
def __setattr__(self, attr, value):
frob(self.b)
super(Foo, self).__setattr__(attr, value)
def process(a):
print "processing"
return {'c': 3 * a}
def frob(x):
print "frobbing", x
myfoo = Foo(1, 2)
myfoo.d = myfoo.c + 1
#SingleNegationElimination's answer is great, but it cannot work with inheritence, since the child class's __mro__ store's the original class of super class. Inspired by his answer, with little change,
The idea is simple, switch __setattr__ before __init__, and restore it back after __init__ completed.
class CleanSetAttrMeta(type):
def __call__(cls, *args, **kwargs):
real_setattr = cls.__setattr__
cls.__setattr__ = object.__setattr__
self = super(CleanSetAttrMeta, cls).__call__(*args, **kwargs)
cls.__setattr__ = real_setattr
return self
class Foo(object):
__metaclass__ = CleanSetAttrMeta
def __init__(self):
super(Foo, self).__init__()
self.a = 1
self.b = 2
def __setattr__(self, key, value):
print 'after __init__', self.b
super(Foo, self).__setattr__(key, value)
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self.c = 3
>>> f = Foo()
>>> f.a = 10
after __init__ 2
>>>
>>> b = Bar()
>>> b.c = 30
after __init__ 2
This question is specifically in the context of Python classes here, but could be more generalized. I am creating a class m that will be initialized with four variables, which I would then like to assign to the instantiation of the class. Right now, this looks like this:
class mm(object):
def __init__(self, a, b, c, r):
self.a = a
self.b = b
self.c = c
self.r = r
# etc.
However, I would like to be able to assign the variables to self in one line, something like the pseudocode:
def __init__(self, a, b, c, r):
self.varname = var for var in [a, b, c, r]
It would be even better if this could be generalized to an arbitrary number of variables, so that you could get something like:
def __init__(self, **kwargs):
assign(self, varname, value) for varname, value in kwargs.iteritems()
However, as far as I know, it's not really possible to loop through statements like that. Does anyone know a way to do this?
You can use setattr:
class mm(object):
def __init__(self, **kwargs):
for varname, value in kwargs.iteritems():
setattr(self, varname, value)
You can also use the instance's namespace dictionary for a mass assignment:
class mm(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
Is there a way that I can define __init__ so keywords defined in **kwargs are assigned to the class?
For example, if I were to initialize a ValidationRule class with ValidationRule(other='email'), the value for self.other should be added to the class without having to explicitly name every possible kwarg.
class ValidationRule:
def __init__(self, **kwargs):
# code to assign **kwargs to .self
I think somewhere on the stackoverflow I've seen such solution. Anyway it can look like:
class ValidationRule:
__allowed = ("other", "same", "different")
def __init__(self, **kwargs):
for k, v in kwargs.iteritems():
assert( k in self.__class__.__allowed )
setattr(self, k, v)
This class will only accept arguments with a whitelisted attribute names listed in __allowed.
This may not be the cleanest way, but it works:
class ValidationRule:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
I think I prefer ony's solution because it restricts available properties to keep you out of trouble when your input comes from external sources.
You could do something like this:
class ValidationRule:
def __init__(self, **kwargs):
for (k, v) in kwargs.items():
setattr(self, k, v)
class ValidationRule:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
You can set your kwargs arguments by updating __dict__ attribute of the instance.
class ValidationRule:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
This could be considered nicer than updating __dict__:
class C:
def __init__(self, **kwargs):
vars(self).update(kwargs)
>>> c = C(a='a', b='b')
>>> c.a # returns 'a'
>>> c.b # returns 'b'
I found the above answers helpful and then refined:
class MyObj(object):
def __init__(self, key1=1, key2=2, key3=3):
for (k, v) in locals().iteritems():
if k != 'self':
setattr(self, k, v)
Test:
>>> myobj = MyObj(key1=0)
>>> print myobj.key1
0
And validation is also there:
>>> myobj = MyObj(key4=4)
TypeError: __init__() got an unexpected keyword argument 'key4'