Develop Reference

Dynamically select which subclass to inherit methods from?

Suppose I have different classes providing access to different subsystems but with a common interface. They all provide the same set of methods but each class implements them in a different way (think using foo.write() to write to a file or send data via socket, etc)
Since the interface is the same, I wanted to make a single class that is able to pick the correct class but only based on the constructor/initializer parameters.
On code, it would look like
class Foo(object):
def who_am_i(self):
print "Foo"
class Bar(object):
def who_am_i(self):
print "Bar"
# Class that decides which one to use and adds some methods that are common to both
class SomeClass(Foo, Bar):
def __init__(self, use_foo):
# How inherit methods from Foo -OR- Bar?
How can the SomeClass inherit methods from Foo or Bar given the __init__ and/or __new__ arguments?
The goal should be something like
>>> some_object = SomeClass(use_foo=True)
>>> some_object.who_am_i()
Foo
>>> another_object = SomeClass(use_foo=False)
>>> another_object.who_am_i()
Bar
Is there some clean "pythonic" way to achieve this? I didn't wanted to use a function to dynamically define SomeClass, but I'm not finding another way to do this.
Thanks!

As mentioned in the comments, this can be done with a factory function (a function that pretends to be a class):
def SomeClass(use_foo):
if use_foo:
return Foo()
else:
return Bar()

As far as I can see, you have your inheritance completely backwards; instead of the multiple inheritance you're proposing:
Foo Bar
- foo code - bar code
\ /
SomeClass(Foo, Bar)
- common code
you could use a much simpler single inheritance model:
SomeClass
- common code
/ \
Foo(SomeClass) Bar(SomeClass)
- foo code - bar code
This then makes your problem one of choosing which subclass to instantiate (a decision that only needs to be made once) rather than which superclass method to call (which potentially needs to be made on every method call). This could be solved with as little as:
thing = Foo() if use_foo else Bar()

A class factory can be used here. Note the use of dictionary to make sure that the same subclass instance is used for each base class.
def makeclass(baseclass, classes={}):
if baseclass not in classes:
class Class(baseclass):
pass # define your methods here
classes[baseclass] = Class
return classes[baseclass]
obj1 = makeclass(Foo)(...)
obj2 = makeclass(Bar)(...)
isinstance(obj1, makeclass(Foo)) # True
isinstance(obj1, Foo) # True
issubclass(makeclass(Foo), Foo) # True
issubclass(type(obj1), Foo) # True
You could also make a dict subclass with a __missing__ method to do essentially the same thing; it makes it more explicit that you've got a container that stores classes, but creates them on demand:
class ClassDict(dict):
def __missing__(self, baseclass):
class Class(baseclass):
pass # define your methods here
self[baseclass] = Class
return Class
subclasses = ClassDict()
obj1 = subclasses[Foo]
obj2 = subclasses[Bar]

Judging by the lack of agreement upon the answer, maybe the problem is the question. jonrsharpe's comment gave an interesting insight on the problem: this should not be solved via inheritance.
Consider SomeClass defined as follows:
# Class that uses Foo or Bar depending on the environment
# Notice it doesn't subclasses either Foo or Bar
class SomeClass(object):
def __init__(self, use_foo):
if use_foo:
self.handler = Foo()
else:
self.handler = Bar()
# Makes more sense asking 'Who implements?' instead of 'Who am I?'
def who_implements(self):
return self.handler
# Explicitly expose methods from the handler
def some_handler_method(self, *args, **kwargs):
return self.handler.some_handler_method(*args, **kwargs)
def another_handler_method(self, *args, **kwargs):
return self.handler.another_handler_method(*args, **kwargs)
Should we need to get details on the handler implementation, just get the handler attribute. Other classes that subclass SomeClass won't even see the handler directly, which actually makes sense.

One could use a __new__ method for this purpose:
_foos = {}
_bars = {}
class SomeClass(object):
def __new__(cls,use_foo,*args,**kwargs):
if use_foo:
if cls not in _foos:
class specialized(cls,Foo):pass
_foos[cls] = specialized
else:
specialized = _foos[cls]
else:
if cls not in _bars:
class specialized(cls,Bar):pass
_bars[cls] = specialized
else:
specialized = _bars[cls]
specialized.__name__ = cls.__name__
return object.__new__(specialized,*args,**kwargs)
#common methods to both go here
pass
The advantage of this over a factory function is that isinstance(SomeClass(True),SomeClass) works, and that SomeClass can be subclassed.

Extract a mixin from a common function in Python

Consider two classes that each have an existing shared function, but separate inheritance paths:
class ClazzA(SuperClazzX):
def foo(self):
return goo(super(SuperClazzX, self).foo())
class ClazzB(SuperClazzY):
def foo(self):
return goo(super(SuperClazzY, self).foo())
foo is clearly a common function which can be extracted to a mixin, what is the right way to do so, such that the functionality remains even though a different super foo is to be called?
Edit: I removed the other mixin, it was confusing and irrelevant.

EDIT: simpler code
Mixin has access to other bases of (future) child class, here C::Mixin::foo has access to the other base of C, namely C::Base::foo. Authoritative explanation here.
class Base(object): # must be new-style class in py 2.x
def foo(self):
print "base foo called"
class Mixin(object):
def foo(self):
rv = super(Mixin, self).foo() # call actual implementation
# use rv
class C(Mixin, Base):
pass
C().foo()
base foo called
What this does:
self is instance of C, it's __mro__ is (Mixin, Base)
when Mixin calls super(Mixin, self), the resulting object retains bases (Base,)
when .foo attribute is resolved, this object finds it in Base
thus Base.foo is invoked with original self
If you want custom control over implementation, you have access to your own bases, e.g.:
class Foo(...):
def foo(self):
print self.__class__.__bases__
Your mixin could look something like this, super-manual approach:
class Mixin(object):
def foo(self):
assert self.__class__ is not Mixin # no point calling directly
# find the other base
other = [b for b in self.__class__.__bases__ if b is not Mixin]
# what to do if there's more than 1 real base?
# pick base explicitly
base = other[1]
# call it,
return something(base.foo(self, some_args))

Is there a way apply a decorator to a Python method that needs informations about the class?

When you decorate a method, it is not bound yet to the class, and therefor doesn't have the im_class attribute yet. I looking for a way to get the information about the class inside the decorator. I tried this:
import types
def decorator(method):
def set_signal(self, name, value):
print name
if name == 'im_class':
print "I got the class"
method.__setattr__ = types.MethodType(set_signal, method)
return method
class Test(object):
#decorator
def bar(self, foo):
print foo
But it doesn't print anything.
I can imagine doing this:
class Test(object):
#decorator(klass=Test)
def bar(self, foo):
print foo
But if I can avoid it, it would make my day.

__setattr__ is only called on explicit object.attribute = assignments; building a class does not use attribute assignment but builds a dictionary (Test.__dict__) instead.
To access the class you have a few different options though:
Use a class decorator instead; it'll be passed the completed class after building it, you could decorate individual methods on that class by replacing them (decorated) in the class. You could use a combination of a function decorator and a class decorator to mark which methods are to be decorated:
def methoddecoratormarker(func):
func._decorate_me = True
return func
def realmethoddecorator(func):
# do something with func.
# Note: it is still an unbound function here, not a method!
return func
def classdecorator(klass):
for name, item in klass.__dict__.iteritems():
if getattr(item, '_decorate_me', False):
klass.__dict__[name] = realmethoddecorator(item)
You could use a metaclass instead of a class decorator to achieve the same, of course.
Cheat, and use sys._getframe() to retrieve the class from the calling frame:
import sys
def methoddecorator(func):
callingframe = sys._getframe(1)
classname = callingframe.f_code.co_name
Note that all you can retrieve is the name of the class; the class itself is still being built at this time. You can add items to callingframe.f_locals (a mapping) and they'll be made part of the new class object.
Access self whenever the method is called. self is a reference to the instance after all, and self.__class__ is going to be, at the very least, a sub-class of the original class the function was defined in.

My strict answer would be: It's not possible, because the class does not yet exist when the decorator is executed.
The longer answer would depend on your very exact requirements. As I wrote, you cannot access the class if it does not yet exists. One solution would be, to mark the decorated method to be "transformed" later. Then use a metaclass or class decorator to apply your modifications after the class has been created.
Another option involves some magic. Look for the implementation of the implements method in zope.interfaces. It has some access to the information about the class which is just been parsed. Don't know if it will be enough for your use case.

You might want to take a look at descriptors. They let you implement a __get__ that is used when an attribute is accessed, and can return different things depending on the object and its type.

Use method decorators to add some marker attributes to the interesting methods, and use a metaclass which iterates over the methods, finds the marker attributes, and does the logic. The metaclass code is run when the class is created, so it has a reference to the newly created class.
class MyMeta(object):
def __new__(...):
...
cls = ...
... iterate over dir(cls), find methods having .is_decorated, act on them
return cls
def decorator(f):
f.is_decorated = True
return f
class MyBase(object):
__metaclass__ = MyMeta
class MyClass(MyBase):
#decorator
def bar(self, foo):
print foo
If you worry about that the programmer of MyClass forgets to use MyBase, you can forcibly set the metaclass in decorator, by exampining the globals dicitionary of the caller stack frame (sys._getframe()).

In Python can one implement mixin behavior without using inheritance?

Is there a reasonable way in Python to implement mixin behavior similar to that found in Ruby -- that is, without using inheritance?
class Mixin(object):
def b(self): print "b()"
def c(self): print "c()"
class Foo(object):
# Somehow mix in the behavior of the Mixin class,
# so that all of the methods below will run and
# the issubclass() test will be False.
def a(self): print "a()"
f = Foo()
f.a()
f.b()
f.c()
print issubclass(Foo, Mixin)
I had a vague idea to do this with a class decorator, but my attempts led to confusion. Most of my searches on the topic have led in the direction of using inheritance (or in more complex scenarios, multiple inheritance) to achieve mixin behavior.

def mixer(*args):
"""Decorator for mixing mixins"""
def inner(cls):
for a,k in ((a,k) for a in args for k,v in vars(a).items() if callable(v)):
setattr(cls, k, getattr(a, k).im_func)
return cls
return inner
class Mixin(object):
def b(self): print "b()"
def c(self): print "c()"
class Mixin2(object):
def d(self): print "d()"
def e(self): print "e()"
#mixer(Mixin, Mixin2)
class Foo(object):
# Somehow mix in the behavior of the Mixin class,
# so that all of the methods below will run and
# the issubclass() test will be False.
def a(self): print "a()"
f = Foo()
f.a()
f.b()
f.c()
f.d()
f.e()
print issubclass(Foo, Mixin)
output:
a()
b()
c()
d()
e()
False

You can add the methods as functions:
Foo.b = Mixin.b.im_func
Foo.c = Mixin.c.im_func

I am not that familiar with Python, but from what I know about Python metaprogramming, you could actually do it pretty much the same way it is done in Ruby.
In Ruby, a module basically consists of two things: a pointer to a method dictionary and a pointer to a constant dictionary. A class consists of three things: a pointer to a method dictionary, a pointer to a constant dictionary and a pointer to the superclass.
When you mix in a module M into a class C, the following happens:
an anonymous class α is created (this is called an include class)
α's method dictionary and constant dictionary pointers are set equal to M's
α's superclass pointer is set equal to C's
C's superclass pointer is set to α
In other words: a fake class which shares its behavior with the mixin is injected into the inheritance hierarchy. So, Ruby actually does use inheritance for mixin composition.
I left out a couple of subleties above: first off, the module doesn't actually get inserted as C's superclass, it gets inserted as C's superclasses' (which is C's singleton class) superclass. And secondly, if the mixin itself has mixed in other mixins, then those also get wrapped into fake classes which get inserted directly above α, and this process is applied recursively, in case the mixed in mixins in turn have mixins.
Basically, the whole mixin hierarchy gets flattened into a straight line and spliced into the inheritance chain.
AFAIK, Python actually allows you to change a class's superclass(es) after the fact (something which Ruby does not allow you to do), and it also gives you access to a class's dict (again, something that is impossible in Ruby), so you should be able to implement this yourself.

EDIT: Fixed what could (and probably should) be construed as a bug. Now it builds a new dict and then updates that from the class's dict. This prevents mixins from overwriting methods that are defined directly on the class. The code is still untested but should work. I'm busy ATM so I'll test it later. It worked fine except for a syntax error. In retrospect, I decided that I don't like it (even after my further improvements) and much prefer my other solution even if it is more complicated. The test code for that one applies here as well but I wont duplicate it.
You could use a metaclass factory:
import inspect
def add_mixins(*mixins):
Dummy = type('Dummy', mixins, {})
d = {}
for mixin in reversed(inspect.getmro(Dummy)):
d.update(mixin.__dict__)
class WithMixins(type):
def __new__(meta, classname, bases, classdict):
d.update(classdict)
return super(WithMixins, meta).__new__(meta, classname, bases, d)
return WithMixins
then use it like:
class Foo(object):
__metaclass__ = add_mixins(Mixin1, Mixin2)
# rest of the stuff

This one is based on the way it's done in ruby as explained by Jörg W Mittag. All of the wall of code after if __name__=='__main__' is test/demo code. There's actually only 13 lines of real code to it.
import inspect
def add_mixins(*mixins):
Dummy = type('Dummy', mixins, {})
d = {}
# Now get all the class attributes. Use reversed so that conflicts
# are resolved with the proper priority. This rules out the possibility
# of the mixins calling methods from their base classes that get overridden
# using super but is necessary for the subclass check to fail. If that wasn't a
# requirement, we would just use Dummy above (or use MI directly and
# forget all the metaclass stuff).
for base in reversed(inspect.getmro(Dummy)):
d.update(base.__dict__)
# Create the mixin class. This should be equivalent to creating the
# anonymous class in Ruby.
Mixin = type('Mixin', (object,), d)
class WithMixins(type):
def __new__(meta, classname, bases, classdict):
# The check below prevents an inheritance cycle from forming which
# leads to a TypeError when trying to inherit from the resulting
# class.
if not any(issubclass(base, Mixin) for base in bases):
# This should be the the equivalent of setting the superclass
# pointers in Ruby.
bases = (Mixin,) + bases
return super(WithMixins, meta).__new__(meta, classname, bases,
classdict)
return WithMixins
if __name__ == '__main__':
class Mixin1(object):
def b(self): print "b()"
def c(self): print "c()"
class Mixin2(object):
def d(self): print "d()"
def e(self): print "e()"
class Mixin3Base(object):
def f(self): print "f()"
class Mixin3(Mixin3Base): pass
class Foo(object):
__metaclass__ = add_mixins(Mixin1, Mixin2, Mixin3)
def a(self): print "a()"
class Bar(Foo):
def f(self): print "Bar.f()"
def test_class(cls):
print "Testing {0}".format(cls.__name__)
f = cls()
f.a()
f.b()
f.c()
f.d()
f.e()
f.f()
print (issubclass(cls, Mixin1) or
issubclass(cls, Mixin2) or
issubclass(cls, Mixin3))
test_class(Foo)
test_class(Bar)

You could decorate the classes __getattr__ to check in the mixin. The problem is that all methods of the mixin would always require an object the type of the mixin as their first parameter, so you would have to decorate __init__ as well to create a mixin-object. I believe you could achieve this using a class decorator.

from functools import partial
class Mixin(object):
#staticmethod
def b(self): print "b()"
#staticmethod
def c(self): print "c()"
class Foo(object):
def __init__(self, mixin_cls):
self.delegate_cls = mixin_cls
def __getattr__(self, attr):
if hasattr(self.delegate_cls, attr):
return partial(getattr(self.delegate_cls, attr), self)
def a(self): print "a()"
f = Foo(Mixin)
f.a()
f.b()
f.c()
print issubclass(Foo, Mixin)
This basically uses the Mixin class as a container to hold ad-hoc functions (not methods) that behave like methods by taking an object instance (self) as the first argument. __getattr__ will redirect missing calls to these methods-alike functions.
This passes your simple tests as shown below. But I cannot guarantee it will do all the things you want. Make more thorough test to make sure.
$ python mixin.py
a()
b()
c()
False

Composition? It seems like that would be the simplest way to handle this: either wrap your object in a decorator or just import the methods as an object into your class definition itself. This is what I usually do: put the methods that I want to share between classes in a file and then import the file. If I want to override some behavior I import a modified file with the same method names as the same object name. It's a little sloppy, but it works.
For example, if I want the init_covers behavior from this file (bedg.py)
import cove as cov
def init_covers(n):
n.covers.append(cov.Cover((set([n.id]))))
id_list = []
for a in n.neighbors:
id_list.append(a.id)
n.covers.append(cov.Cover((set(id_list))))
def update_degree(n):
for a in n.covers:
a.degree = 0
for b in n.covers:
if a != b:
a.degree += len(a.node_list.intersection(b.node_list))
In my bar class file I would do: import bedg as foo
and then if I want to change my foo behaviors in another class that inherited bar, I write
import bild as foo
Like I say, it is sloppy.

Python: How to distinguish between inherited methods

Newbie Python question. I have a class that inherits from several classes, and some of the specialization classes override some methods from the base class. In certain cases, I want to call the unspecialized method. Is this possible? If so, what's the syntax?
class Base(object):
def Foo(self):
print "Base.Foo"
def Bar(self):
self.Foo() # Can I force this to call Base.Foo even if Foo has an override?
class Mixin(object):
def Foo(self):
print "Mixin.Foo"
class Composite(Mixin, Base):
pass
x = Composite()
x.Foo() # executes Mixin.Foo, perfect
x.Bar() # indirectly executes Mixin.Foo, but I want Base.Foo

you can specifically make the call you want using the syntax
Base.Foo(self)
in your case:
class Base(object):
# snipped
def Bar(self):
Base.Foo(self) # this will now call Base.Foo regardless of if a subclass
# overrides it
# snipped
x = Composite()
x.Foo() # executes Mixin.Foo, perfect
x.Bar() # prints "Base.Foo"
This works because Python executes calls to bound methods of the form
instance.method(argument)
as if they were a call to an unbound method
Class.method(instance, argument)
so making the call in that form gives you the desired result. Inside the methods, self is just the instance that the method was called on, i.e, the implicit first argument (that's explicit as a parameter)
Note however that if a subclass overrides Bar, then there's nothing (good) that you can effectively do about it AFAIK. But that's just the way things work in python.