I have class Base. I'd like to extend its functionality in a class Derived. I was planning to write:
class Derived(Base):
def __init__(self, base_arg1, base_arg2, derived_arg1, derived_arg2):
super().__init__(base_arg1, base_arg2)
# ...
def derived_method1(self):
# ...
Sometimes I already have a Base instance, and I want to create a Derived instance based on it, i.e., a Derived instance that shares the Base object (doesn't re-create it from scratch). I thought I could write a static method to do that:
b = Base(arg1, arg2) # very large object, expensive to create or copy
d = Derived.from_base(b, derived_arg1, derived_arg2) # reuses existing b object
but it seems impossible. Either I'm missing a way to make this work, or (more likely) I'm missing a very big reason why it can't be allowed to work. Can someone explain which one it is?
[Of course, if I used composition rather than inheritance, this would all be easy to do. But I was hoping to avoid the delegation of all the Base methods to Derived through __getattr__.]
Rely on what your Base class is doing with with base_arg1, base_arg2.
class Base(object):
def __init__(self, base_arg1, base_arg2):
self.base_arg1 = base_arg1
self.base_arg2 = base_arg2
...
class Derived(Base):
def __init__(self, base_arg1, base_arg2, derived_arg1, derived_arg2):
super().__init__(base_arg1, base_arg2)
...
#classmethod
def from_base(cls, b, da1, da2):
return cls(b.base_arg1, b.base_arg2, da1, da2)
The alternative approach to Alexey's answer (my +1) is to pass the base object in the base_arg1 argument and to check, whether it was misused for passing the base object (if it is the instance of the base class). The other agrument can be made technically optional (say None) and checked explicitly when decided inside the code.
The difference is that only the argument type decides what of the two possible ways of creation is to be used. This is neccessary if the creation of the object cannot be explicitly captured in the source code (e.g. some structure contains a mix of argument tuples, some of them with the initial values, some of them with the references to the existing objects. Then you would probably need pass the arguments as the keyword arguments:
d = Derived(b, derived_arg1=derived_arg1, derived_arg2=derived_arg2)
Updated: For the sharing the internal structures with the initial class, it is possible using both approaches. However, you must be aware of the fact, that if one of the objects tries to modify the shared data, the usual funny things can happen.
To be clear here, I'll make an answer with code. pepr talks about this solution, but code is always clearer than English. In this case Base should not be subclassed, but it should be a member of Derived:
class Base(object):
def __init__(self, base_arg1, base_arg2):
self.base_arg1 = base_arg1
self.base_arg2 = base_arg2
class Derived(object):
def __init__(self, base, derived_arg1, derived_arg2):
self.base = base
self.derived_arg1 = derived_arg1
self.derived_arg2 = derived_arg2
def derived_method1(self):
return self.base.base_arg1 * self.derived_arg1
Related
I have a base class with function run. For example:
class A:
#abstractmethod
def run(self, steps):
...
It is possible to define class B with more arguments to the run method.
class B(A):
def run(self, steps, save):
...
Working with typing, I can specify if a function gets either A or B as argument. By specifying the function gets A, I tell that I only need the basic interface of run. While specifying B says I need the extended one.
The purpose of this design is to declare a base interface that all the children share but each one can have an extended API.
This is impossible to be done in other languages. Hence I wonder, is it an anti-pattern? Is it something legit to do?
In Python you can do something like the following.
class A:
def run(self, steps):
print("Using class A's run.")
print(f"steps are {steps}")
class B(A):
def run(self, steps, other_arg=None):
if other_arg:
print("Using class B's override.")
print(f"steps are {steps}")
else:
# Use parent's run logic instead.
super().run(steps)
x = B()
x.run(100)
x.run(30, other_arg="something")
# Using class A's run.
# steps are 100
# Using class B's override.
# steps are 30
Now, should you do this? There is a time and a place. You can get into trouble as well. Imagine you break the interface of the core object you're inheriting from, so the core object loses its abstraction value. You'd have been better off having two objects or rewriting your abstraction to be more robust to the differences in object you wish you represent.
Edit: Note that the original question changed to make the base run method abstract. The solution posted here is mostly invalidated by that.
I have a few classes with almost identical contents, so I tried two methods to copy the classes and their attributes over. The classes copy correctly, but the randint function is only invoked in the main class, so the same number is output every time. Is there any way to recalculate the random number for each class?
class a:
exampleData = random.randint(1,100)
b = type('b', a.__bases__, dict(a.__dict__))
class c(a):
pass
For example if a.exampleData = 50, b.exampleData and c.exampleData would be the same. Is there any way around this?
Edit -- Part of my program displays characters with random stats each time, and the class contains the stats associated with each character. The random numbers pick the stats out of a list, but the same stats are being chosen, instead of being random in each class. I may not be explaining this right, so basically:
data = [stat1,stat2,stat3,ect,,]
data[random.randint(1,3)]
When you write this:
b = type('b', a.__bases__, dict(a.__dict__))
… you're just copying a.__dict__. Since a.__dict__ is just {'exampleData': 50}, the new copy that ends up as b.__dict__ is also going to be {'exampleData': 50}.
There are many ways you could get a new random number. The simplest is to just create a new random number for b explicitly:
bdict = dict(a.__dict__)
b['exampleData'] = random.randint(1,100)
b = type('b', a.__bases__, bdict)
If you want to create a bunch of classes this way, you can wrap that up in a function:
def make_clone(proto, name):
clonedict = dict(proto.__dict__)
clonedict['exampleData'] = random.randint(1,100)
return type(name, proto.__bases__, clonedict)
You can make that factory function more complicated if you want to be (see namedtuple for a pretty extreme example).
You could wrap that behavior up in a decorator:
def randomize(cls):
cls.exampleData = random.randint(1,100)
#randomize
class a:
pass
b = randomize(type('b', a.__bases__, dict(a.__dict__)))
Notice that I had to call the decorator with normal function-call syntax here, because there's no declaration statement to attach an #decorator to.
Or you can wrap it up in a metaclass:
class RandomMeta(type):
def __new__(mcls, name, bases, namespace):
d = dict(namespace)
d['exampleData'] = random.randint(1,100)
return type.__new__(mcls, name, bases, d)
class a(metaclass=RandomMeta):
pass
b = type(a)('b', a.__bases__, dict(a.__dict__))
Notice that we have to call type(a) here, the same way a class definition statement does, not the base metaclass type.
Also notice that I'm not taking **kwds in the __new__ method, and I'm calling type.__new__ directly. This means that if you try to use RandomMeta together with another metaclass (besides type), you should get an immediate TypeError, rather than something that may or may not be correct.
Meanwhile, I have a suspicion that what you're really trying to do here is build a prototype-based inheritance system, a la Self or JavaScript on top of Python's class-based system. While you can do that by using a special Prototype metaclass and a bunch of class objects, it's a whole lot simpler to just have a Prototype class and a bunch of instance objects. The only advantage to the metaclass approach is that you can use class statements (misleadingly, but conveniently) to clone prototypes, and you're explicitly not doing that here.
While my other answer covers the question as asked, I suspect it's all completely unnecessary to the OP's actual problem.
If you just want to create a bunch of separate objects, which each have a separate value for exampleData, you just want a bunch of instances of a single class, not a bunch of separate classes.
A class is a special kind of object that, in addition to doing all the normal object stuff, also works as a factory for other objects, which are instances of that class. You don't need a, b, and c to all be factories for for different kinds of objects, you just need them to be different objects of the same type. So:
class RandomThing:
def __init__(self):
self.exampleData = random.randint(1,100)
a = RandomThing()
b = RandomThing()
… or, if you want to make sure b is the same type of thing as a but don't know what type that is:
b = type(a)()
That's as fancy as you need to get here.
See the official tutorial on Classes (or maybe search for a friendlier tutorial, because there are probably better ones out there).
I am trying to define a number of classes based on an abstract base class. Each of these classes basically defines a cell shape for a visualisation package. The cell is comprised of a number of vertices (points) and each subclass will require a different number of points. Each class can be thought of as a container for a fixed number of point coordinates.
As an example, consider the base class Shape, which is simply a container for a list of coordinates:
class Shape(object):
"""Cell shape base class."""
def __init__(self, sequence):
self.points = sequence
#property
def points(self):
return self._points
#points.setter
def points(self, sequence):
# Error checking goes here, e.g. check that `sequence` is a
# sequence of numeric values.
self._points = sequence
Ideally I want to be able to define, say, a Square class, where the points.setter method checks that sequence is of length four. Furthermore I would like a user to not be able to instantiate Shape. Is there a way I can define Shape to be an abstract base class? I have tried changing the definition of shape to the following:
import abc
class Shape(object):
"""Cell shape base class."""
__metaclass__ = abc.ABCMeta
def __init__(self, sequence):
self.points = sequence
#abc.abstractproperty
def npoints(self):
pass
#property
def points(self):
return self._points
#points.setter
def points(self, sequence):
# Error checking goes here...
if len(sequence) != self.npoints:
raise TypeError('Some descriptive error message!')
self._points = sequence
This requires subclasses to define the property npoints. I can then define a class Square as
class Square(Shape):
#property
def npoints(self):
return 4
However, this would be rather tedious to implement for a large number of sublcasses (and with more than one property to implement). I was hoping to define a class factory which would create my subclasses for me, something along the lines of:
def Factory(name, npoints):
return type(name, (Shape,), dict(npoints=npoints))
Triangle = Factory('Triangle', 3)
Square = Factory('Square', 4)
# etc...
Is this class factory function a valid approach to take, or am I clobbering the npoints property? Is it better to replace the call to type with something more verbose like:
def Factory(name, _npoints):
class cls(Shape):
#property
def npoints(self):
return _npoints
cls.__name__ = name
return cls
An alternative approach would be to define a class attribute _NPOINTS and change the npoints
property of Shape to
#property
def npoints(self):
return _NPOINTS
However, then I loose the benefit of using an abstract base class since:
I can't see how to define a class attribute using type, and
I don't know how to define an abstract class attribute.
Does anyone have any thoughts on the best way to implement this abstract base class and class factory function, or even an altogether better design?
Without knowing more about your project, I cannot give specific advice on the general design. I will just provide a few more general hints and thoughts.
Dynamically generated classes are often a sign that you don't need separate classes at all – simply write a single class that incorparates all the functionality. What's the problem with a Shape class that gets it's properties at instantiation time? (Of course there are reasons to use dynamically generated classes – the namedtuple() factory function is one example. I couldn't find any specific reasons in your question, however.)
Instead of using abstract base classes, you often simply document the intended interface, and than write classes conforming to this interface. Due to the dynamic nature of Python, you don't strictly need a common base class. There are often other advantages to a common base class – for example shared functionality.
Only check for application code errors if not doing so leads to strange errors in unrelated places. If, say, your function expects an iterable, simply assume you got an iterable. If the user passed in something else, you code will fail when it tries to iterate the passed in object anyway, and the error message will usually be enough for the application developer to understand the error.
This article has a snippet showing usage of __bases__ to dynamically change the inheritance hierarchy of some Python code, by adding a class to an existing classes collection of classes from which it inherits. Ok, that's hard to read, code is probably clearer:
class Friendly:
def hello(self):
print 'Hello'
class Person: pass
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
That is, Person doesn't inherit from Friendly at the source level, but rather this inheritance relation is added dynamically at runtime by modification of the __bases__attribute of the Person class. However, if you change Friendly and Person to be new style classes (by inheriting from object), you get the following error:
TypeError: __bases__ assignment: 'Friendly' deallocator differs from 'object'
A bit of Googling on this seems to indicate some incompatibilities between new-style and old style classes in regards to changing the inheritance hierarchy at runtime. Specifically: "New-style class objects don't support assignment to their bases attribute".
My question, is it possible to make the above Friendly/Person example work using new-style classes in Python 2.7+, possibly by use of the __mro__ attribute?
Disclaimer: I fully realise that this is obscure code. I fully realize that in real production code tricks like this tend to border on unreadable, this is purely a thought experiment, and for funzies to learn something about how Python deals with issues related to multiple inheritance.
Ok, again, this is not something you should normally do, this is for informational purposes only.
Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:
setattr(Person, '__mro__', (Person, Friendly, object))
The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.
A possible workaround is to simply redefine the class:
def modify_Person_to_be_friendly():
# so that we're modifying the global identifier 'Person'
global Person
# now just redefine the class using type(), specifying that the new
# class should inherit from Friendly and have all attributes from
# our old Person class
Person = type('Person', (Friendly,), dict(Person.__dict__))
def main():
modify_Person_to_be_friendly()
p = Person()
p.hello() # works!
What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):
def main():
oldperson = Person()
ModifyPersonToBeFriendly()
p = Person()
p.hello()
# works! But:
oldperson.hello()
# does not
If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.
I've been struggling with this too, and was intrigued by your solution, but Python 3 takes it away from us:
AttributeError: attribute '__dict__' of 'type' objects is not writable
I actually have a legitimate need for a decorator that replaces the (single) superclass of the decorated class. It would require too lengthy a description to include here (I tried, but couldn't get it to a reasonably length and limited complexity -- it came up in the context of the use by many Python applications of an Python-based enterprise server where different applications needed slightly different variations of some of the code.)
The discussion on this page and others like it provided hints that the problem of assigning to __bases__ only occurs for classes with no superclass defined (i.e., whose only superclass is object). I was able to solve this problem (for both Python 2.7 and 3.2) by defining the classes whose superclass I needed to replace as being subclasses of a trivial class:
## T is used so that the other classes are not direct subclasses of object,
## since classes whose base is object don't allow assignment to their __bases__ attribute.
class T: pass
class A(T):
def __init__(self):
print('Creating instance of {}'.format(self.__class__.__name__))
## ordinary inheritance
class B(A): pass
## dynamically specified inheritance
class C(T): pass
A() # -> Creating instance of A
B() # -> Creating instance of B
C.__bases__ = (A,)
C() # -> Creating instance of C
## attempt at dynamically specified inheritance starting with a direct subclass
## of object doesn't work
class D: pass
D.__bases__ = (A,)
D()
## Result is:
## TypeError: __bases__ assignment: 'A' deallocator differs from 'object'
I can not vouch for the consequences, but that this code does what you want at py2.7.2.
class Friendly(object):
def hello(self):
print 'Hello'
class Person(object): pass
# we can't change the original classes, so we replace them
class newFriendly: pass
newFriendly.__dict__ = dict(Friendly.__dict__)
Friendly = newFriendly
class newPerson: pass
newPerson.__dict__ = dict(Person.__dict__)
Person = newPerson
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
We know that this is possible. Cool. But we'll never use it!
Right of the bat, all the caveats of messing with class hierarchy dynamically are in effect.
But if it has to be done then, apparently, there is a hack that get's around the "deallocator differs from 'object" issue when modifying the __bases__ attribute for the new style classes.
You can define a class object
class Object(object): pass
Which derives a class from the built-in metaclass type.
That's it, now your new style classes can modify the __bases__ without any problem.
In my tests this actually worked very well as all existing (before changing the inheritance) instances of it and its derived classes felt the effect of the change including their mro getting updated.
I needed a solution for this which:
Works with both Python 2 (>= 2.7) and Python 3 (>= 3.2).
Lets the class bases be changed after dynamically importing a dependency.
Lets the class bases be changed from unit test code.
Works with types that have a custom metaclass.
Still allows unittest.mock.patch to function as expected.
Here's what I came up with:
def ensure_class_bases_begin_with(namespace, class_name, base_class):
""" Ensure the named class's bases start with the base class.
:param namespace: The namespace containing the class name.
:param class_name: The name of the class to alter.
:param base_class: The type to be the first base class for the
newly created type.
:return: ``None``.
Call this function after ensuring `base_class` is
available, before using the class named by `class_name`.
"""
existing_class = namespace[class_name]
assert isinstance(existing_class, type)
bases = list(existing_class.__bases__)
if base_class is bases[0]:
# Already bound to a type with the right bases.
return
bases.insert(0, base_class)
new_class_namespace = existing_class.__dict__.copy()
# Type creation will assign the correct ‘__dict__’ attribute.
del new_class_namespace['__dict__']
metaclass = existing_class.__metaclass__
new_class = metaclass(class_name, tuple(bases), new_class_namespace)
namespace[class_name] = new_class
Used like this within the application:
# foo.py
# Type `Bar` is not available at first, so can't inherit from it yet.
class Foo(object):
__metaclass__ = type
def __init__(self):
self.frob = "spam"
def __unicode__(self): return "Foo"
# … later …
import bar
ensure_class_bases_begin_with(
namespace=globals(),
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
Use like this from within unit test code:
# test_foo.py
""" Unit test for `foo` module. """
import unittest
import mock
import foo
import bar
ensure_class_bases_begin_with(
namespace=foo.__dict__,
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
class Foo_TestCase(unittest.TestCase):
""" Test cases for `Foo` class. """
def setUp(self):
patcher_unicode = mock.patch.object(
foo.Foo, '__unicode__')
patcher_unicode.start()
self.addCleanup(patcher_unicode.stop)
self.test_instance = foo.Foo()
patcher_frob = mock.patch.object(
self.test_instance, 'frob')
patcher_frob.start()
self.addCleanup(patcher_frob.stop)
def test_instantiate(self):
""" Should create an instance of `Foo`. """
instance = foo.Foo()
The above answers are good if you need to change an existing class at runtime. However, if you are just looking to create a new class that inherits by some other class, there is a much cleaner solution. I got this idea from https://stackoverflow.com/a/21060094/3533440, but I think the example below better illustrates a legitimate use case.
def make_default(Map, default_default=None):
"""Returns a class which behaves identically to the given
Map class, except it gives a default value for unknown keys."""
class DefaultMap(Map):
def __init__(self, default=default_default, **kwargs):
self._default = default
super().__init__(**kwargs)
def __missing__(self, key):
return self._default
return DefaultMap
DefaultDict = make_default(dict, default_default='wug')
d = DefaultDict(a=1, b=2)
assert d['a'] is 1
assert d['b'] is 2
assert d['c'] is 'wug'
Correct me if I'm wrong, but this strategy seems very readable to me, and I would use it in production code. This is very similar to functors in OCaml.
This method isn't technically inheriting during runtime, since __mro__ can't be changed. But what I'm doing here is using __getattr__ to be able to access any attributes or methods from a certain class. (Read comments in order of numbers placed before the comments, it makes more sense)
class Sub:
def __init__(self, f, cls):
self.f = f
self.cls = cls
# 6) this method will pass the self parameter
# (which is the original class object we passed)
# and then it will fill in the rest of the arguments
# using *args and **kwargs
def __call__(self, *args, **kwargs):
# 7) the multiple try / except statements
# are for making sure if an attribute was
# accessed instead of a function, the __call__
# method will just return the attribute
try:
return self.f(self.cls, *args, **kwargs)
except TypeError:
try:
return self.f(*args, **kwargs)
except TypeError:
return self.f
# 1) our base class
class S:
def __init__(self, func):
self.cls = func
def __getattr__(self, item):
# 5) we are wrapping the attribute we get in the Sub class
# so we can implement the __call__ method there
# to be able to pass the parameters in the correct order
return Sub(getattr(self.cls, item), self.cls)
# 2) class we want to inherit from
class L:
def run(self, s):
print("run" + s)
# 3) we create an instance of our base class
# and then pass an instance (or just the class object)
# as a parameter to this instance
s = S(L) # 4) in this case, I'm using the class object
s.run("1")
So this sort of substitution and redirection will simulate the inheritance of the class we wanted to inherit from. And it even works with attributes or methods that don't take any parameters.
I have a number of atomic classes (Components/Mixins, not really sure what to call them) in a library I'm developing, which are meant to be subclassed by applications. This atomicity was created so that applications can only use the features that they need, and combine the components through multiple inheritance.
However, sometimes this atomicity cannot be ensured because some component may depend on another one. For example, imagine I have a component that gives a graphical representation to an object, and another component which uses this graphical representation to perform some collision checking. The first is purely atomic, however the latter requires that the current object already subclassed this graphical representation component, so that its methods are available to it. This is a problem, because we have to somehow tell the users of this library, that in order to use a certain Component, they also have to subclass this other one. We could make this collision component sub class the visual component, but if the user also subclasses this visual component, it wouldn't work because the class is not on the same level (unlike a simple diamond relationship, which is desired), and would give the cryptic meta class errors which are hard to understand for the programmer.
Therefore, I would like to know if there is any cool way, through maybe metaclass redefinition or using class decorators, to mark these unatomic components, and when they are subclassed, the additional dependency would be injected into the current object, if its not yet available. Example:
class AtomicComponent(object):
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent(object):
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
Can someone give me an hint on how I can do this? or if it is even possible...
edit:
Since it is debatable that the meta class solution is the best one, I'll leave this unaccepted for 2 days.
Other solutions might be to improve error messages, for example, doing something like UserClass2 would give an error saying that UnAtomicComponent already extends this component. This however creates the problem that it is impossible to use two UnAtomicComponents, given that they would subclass object on different levels.
"Metaclasses"
This is what they are for! At time of class creation, the class parameters run through the
metaclass code, where you can check the bases and change then, for example.
This runs without error - though it does not preserve the order of needed classes
marked with the "depends" decorator:
class AutoSubclass(type):
def __new__(metacls, name, bases, dct):
new_bases = set()
for base in bases:
if hasattr(base, "_depends"):
for dependence in base._depends:
if not dependence in bases:
new_bases.add(dependence)
bases = bases + tuple(new_bases)
return type.__new__(metacls, name, bases, dct)
__metaclass__ = AutoSubclass
def depends(*args):
def decorator(cls):
cls._depends = args
return cls
return decorator
class AtomicComponent:
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent:
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
(I removed inheritance from "object", as I declared a global __metaclass__ variable. All classs will still be new style class and have this metaclass. Inheriting from object or another class does override the global __metaclass__variable, and a class level __metclass__ will have to be declared)
-- edit --
Without metaclasses, the way to go is to have your classes to properly inherit from their dependencies. Tehy will no longer be that "atomic", but, since they could not work being that atomic, it may be no matter.
In the example bellow, classes C and D would be your User classes:
>>> class A(object): pass
...
>>> class B(A, object): pass
...
>>>
>>> class C(B): pass
...
>>> class D(B,A): pass
...
>>>