I'm seeking advice about design of my code.
Introduction
I have several classes, each represents one file type, eg: MediaImageFile, MediaAudioFile and generic (and also base class) MediaGenericFile.
Each file have two variants: Master and Version, so I created these classes to define theirs specific behaviour. EDIT: Version represents resized/cropped/trimmed/etc variant of Master file. It's used mainly for previews.
EDIT: The reason, why I want to do it dynamically is that this app should be reusable (it's Django-app) and therefore it should be easy to implement other MediaGenericFile subclass without modifying original code.
What I want to do
First of all, user should be able to register own MediaGenericFile subclasses without affecting original code.
Whether file is version or master is easily (one regexp) recognizable from filename.
/path/to/master.jpg -- master
/path/to/.versions/master_version.jpg -- version
Master/Version classes use some methods/properties of MediaGenericFile, like filename (you need to know filename to generate new version).
MediaGenericFile extends LazyFile, which is just lazy File object.
Now I need to put it together…
Used design
Before I start coding 'versions' feature, I had factory class MediaFile, which returns appropriate file type class according to extension:
>>> MediaFile('path/to/image.jpg')
<<< <MediaImageFile 'path/to/image.jpg'>
Classes Master and Version define new methods which use methods and attributes of MediaGenericFile and etc.
Approach 1
One approach is create dynamically new type, which inherits Master (or Version) and MediaGenericFile (or subclass).
class MediaFile(object):
def __new__(cls, *args, **kwargs):
... # decision about klass
if version:
bases = (Version, klass)
class_name = '{0}Version'.format(klass.__name__)
else:
bases = (Master, klass)
class_name = '{0}Master'.format(klass.__name__)
new_class = type(class_name, bases, {})
...
return new_class(*args, **kwargs)
Approach 2
Second approach is create method 'contribute_to_instance' in Master/Version and call it after creating new_class, but that's more tricky than I thought:
classs Master(object):
#classmethod
def contribute_to_instance(cls, instance):
methods = (...)
for m in methods:
setattr(instance, m, types.MethodType(getattr(cls, m), instance))
class MediaFile(object):
def __new__(*args, **kwargs):
... # decision about new_class
obj = new_class(*args, **kwargs)
if version:
version_class = Version
else:
version_class = Master
version_class.contribute_to_instance(obj)
...
return obj
However, this doesn't work. There are still problems with calling Master/Version's methods.
Questions
What would be good way to implement this multiple inheritance?
How is this problem called? :) I was trying to find some solutions, but I simply don't know how to name this problem.
Thanks in advance!
Note to answers
ad larsmans
Comparison and instance check wouldn't be problem for my case, because:
Comparisons are redefined anyway
class MediaGenericFile(object):
def __eq__(self, other):
return self.name == other.name
I never need to check isinstance(MediaGenericFileVersion, instance). I'm using isinstance(MediaGenericFile, instance) and isinstance(Version, instance) and both works as expected.
Nevertheless, creating new type per instance sounds to me as considerable defect.
Well, I could create both variations dynamically in metaclass and then use them, something like:
>>> MediaGenericFile.version_class
<<< <class MediaGenericFileVersion>
>>> MediaGenericFile.master_class
<<< <class MediaGenericFileMaster>
And then:
class MediaFile(object):
def __new__(cls, *args, **kwargs):
... # decision about klass
if version:
attr_name = 'version_class'
else:
attr_name = 'master_class'
new_class = getattr(klass, attr_name)
...
return new_class(*args, **kwargs)
Final solution
Finally the design pattern is factory class. MediaGenericFile subclasses are statically typed, users can implement and register their own. Master/Version variants are created dynamically (glued together from several mixins) in metaclass and stored in 'cache' to avoid perils mentioned by larsmans.
Thanks everyone for their suggestions. Finally I understand the metaclass concept. Well, at least I think that I understand it. Push origin master…
I'd certainly advise against the first approach of constructing classes in __new__. The problem with it is that you create a new type per instance, which causes overhead and worse, causes type comparisons to fail:
>>> Ham1 = type("Ham", (object,), {})
>>> Ham2 = type("Ham", (object,), {})
>>> Ham1 == Ham2
False
>>> isinstance(Ham1(), Ham2)
False
>>> isinstance(Ham2(), Ham1)
False
This violates the principle of least surprise because the classes may seem entirely identical:
>>> Ham1
<class '__main__.Ham'>
>>> Ham2
<class '__main__.Ham'>
You can get approach 1 to work properly, though, if you construct the classes at the module level, outside of MediaFile:
classes = {}
for klass in [MediaImageFile, MediaAudioFile]:
for variant in [Master, Version]:
# I'd actually do this the other way around,
# making Master and Version mixins
bases = (variant, klass)
name = klass.__name__ + variant.__name__
classes[name] = type(name, bases, {})
then, in MediaFile.__new__, look the required class up by name in classes. (Alternatively, set the newly constructed classes on the module instead of in a dict.)
I'm not sure how dynamic you want it to be, but using a "factory pattern" (here using a class factory), is fairly readable and understandable and may do what you want. This could serve as a base... MediaFactory could be smarter, and you could register multiple other classes, instead of hard-coding MediaFactoryMaster etc...
class MediaFactory(object):
__items = {}
#classmethod
def make(cls, item):
return cls.__items[item]
#classmethod
def register(cls, item):
def func(kls):
cls.__items[item] = kls
return kls
return func
class MediaFactoryMaster(MediaFactory, Master): pass
class MediaFactoryVersion(MediaFactory, Version): pass
class MediaFile(object):
pass
#MediaFactoryMaster.register('jpg') # adapt to take ['jpg', 'gif', 'png'] ?
class MediaFileImage(MediaFile):
pass
#MediaFactoryVersion.register('mp3') # adapt to take ['mp3', 'ogg', 'm4a'] ?
class MediaFileAudio(MediaFile):
pass
other possible MediaFactory.make
#classmethod
def make(cls, fname):
name, ext = somefunc(fname)
kls = cls.__items[ext]
other = Version if Version else Master
return type('{}{}'.format(kls.__name__,other.__name__), (kls, other), {})
How come you're not using inheritance but are playing around with __new__?
class GenericFile(File):
"""Base class"""
class Master(object):
"""Master Mixin"""
class Versioned(object):
"""Versioning mixin"""
class ImageFile(GenericFile):
"""Image Files"""
class MasterImage(ImageFile, Master):
"""Whatever"""
class VersionedImage(ImageFile, Versioned):
"""Blah blah blah"""
...
It's not clear why you're doing this though. I think there's a weird code smell here. I'd recommend fewer classes with a consistent interface (duck-typing) rather than a dozen classes and isinstance checks throughout the code to make it all work.
Perhaps you can update your question with what you'd like to do in your code and folks can help either identify the real pattern or a suggest a more idiomatic solution.
You do not have to create a new class for each instance. Don't create the new classes in __new__ create them in __metaclass__. define a metaclass in the base or in the base_module. The two "variant" subclasses are easily saved as as class attributes of their genaric parent and then __new__ just looks at the filename according to it's own rules and decides which subclass to return.
Watch out for __new__ that returns a class other than the one "nominated" during the constructor call. You may have to take steps to invoke __init__ from withing __new__
Subclasses will either have to:
"register" themselves with a factory or parent to be found
be imported and then have the parent or factory find them through a recursive search of cls.__subclasses (might have to happen once per creation but that's probably not a problem for file handeling)
found through the use of "setuptools" entry_points type tools but that requires more effort and coordination by the user
The OOD question you should be asking is "do the various classes of my proposed inheritance share any properties at all?"
The purpose of inheritance is to share common data or methods that the instances naturally have in common. Aside from both being files, what do Image files and Audio files have in common? If you really want to stretch your metaphors, you could conceivably have AudioFile.view() which could present — for example — a visualization of the power spectra of the audio data, but ImageFile.listen() makes even less sense.
I think your question side-steps this language independent conceptual issue in favor of the Python dependent mechanics of an object factory. I don't think you have a proper case of inheritance here, or you've failed to explain what common features your Media objects need to share.
Related
Let's say I want to create a registry of subclasses of a certain class. Now there are two approaches I can think of and while I'm aware of (some of) their differences, I'd love to learn more about the topic.
class Base:
pass
class DerivedA(Base):
pass
class DerivedB(Base):
pass
__subclasses__()
If I have the situation above, I can simply get the list of subclasses of Base like this:
>>> [cmd.__name__ for cmd in Base.__subclasses__()]
['DerivedA', 'DerivedB']
Now I'm aware that if I add a third class that is not directly subclassing Base like this:
class DerivedC(DerivedA):
pass
I will not see this one in the list:
>>> [cmd.__name__ for cmd in Base.__subclasses__()]
['DerivedA', 'DerivedB']
Also I can't filter the subclasses and for example ignore a particular subclass for any reason.
__init_subclass__()
Since Python 3.6 there is a nice hook into class creating process and more advanced things can be done without writing one's own metaclass. Thus I can also do something like this...
_registry = []
class Base:
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
_registry.append(cls.__name__)
class DerivedA(Base):
pass
class DerivedB(Base):
pass
class DerivedC(DerivedA):
pass
And then simply access _registry:
>>> _registry
['DerivedA', 'DerivedB', 'DerivedC']
I can also modify Base to ignore certain subclasses if I wanted:
_registry = []
class Base:
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
if cls.__name__ != 'DerivedB':
_registry.append(cls.__name__)
class DerivedA(Base):
pass
class DerivedB(Base):
pass
class DerivedC(DerivedA):
pass
>>> _registry
['DerivedA', 'DerivedC']
Why use the latter?
Now let's say that I don't want to filter the subclasses and I'm only interested in direct subclasses. The former approach seems to be simpler (subjective, I know). What are other differences and maybe what are the advantages of the latter approach?
Thanks!
The obvious gain of writing __init_subclass__ in a base class in this case is that you can automatically get to the subclasses that do not inherit directly from your base class, as you put it.
If you only need the classes that inherit directly from your base, then it is ready in the __subclasses__ method, and the major advantage is that you don't need to write a single line of code, and not even keep a separate registry, as the __subclasses__ will do that for you.
However, unless you are writing a relatively small app, or are dealing with a feature that just needs a small fixed number of these subclasses to be looked-up, relying in __subclasses__ is not enough - if you simply need, or want, another level of classes in your hierarchy, it will stop working, and you have to resort to a true registry anyway.
Prior to having the __init_subclass__ hook, one would have to write a proper metaclass to keep this registry, feeding it on the metaclass __init__ method, or do a complicated recursive query like:
def check_subclass(base, candidate):
for cls in base.__subclasses__():
if cls is candidate:
return True
if check_subclass(cls, candidate):
return True
return False
And, although it should go without saying, the __init_subclass__ method can do a lot more than simply keep a registry - as it can run any code. It could check against the DB layer if the fields mapped to that subclass are up to date, and warn of a needed migration - or even perform the DB migration itself, or initialise any resources that instances of the class will need to find ready when they are created, such as logger-handlers, thread-pools, db-connection pools, you name it.
TL;DR: If you just need the direct subclasses of a class, go with __subclasses__. The catch is exactly that it just annotates the direct subclasses.
I would like to create a class which defines a particular interface, and then require all subclasses to conform to this interface. For example, I would like to define a class
class Interface:
def __init__(self, arg1):
pass
def foo(self, bar):
pass
and then be assured that if I am holding any element a which has type A, a subclass of Interface, then I can call a.foo(2) it will work.
It looked like this question almost addressed the problem, but in that case it is up to the subclass to explicitly change it's metaclass.
Ideally what I'm looking for is something similar to Traits and Impls from Rust, where I can specify a particular Trait and a list of methods that trait needs to define, and then I can be assured that any object with that Trait has those methods defined.
Is there any way to do this in Python?
So, first, just to state the obvious - Python has a built-in mechanism to test for the existence of methods and attributes in derived classes - it just does not check their signature.
Second, a nice package to look at is zope.interface. Despte the zope namespace, it is a complete stand-alone package that allows really neat methods of having objects that can expose multiple interfaces, but just when needed - and then frees-up the namespaces. It sure involve some learning until one gets used to it, but it can be quite powerful and provide very nice patterns for large projects.
It was devised for Python 2, when Python had a lot less features than nowadays - and I think it does not perform automatic interface checking (one have to manually call a method to find-out if a class is compliant) - but automating this call would be easy, nonetheless.
Third, the linked accepted answer at How to enforce method signature for child classes? almost works, and could be good enough with just one change. The problem with that example is that it hardcodes a call to type to create the new class, and do not pass type.__new__ information about the metaclass itself. Replace the line:
return type(name, baseClasses, d)
for:
return super().__new__(cls, name, baseClasses, d)
And then, make the baseclass - the one defining your required methods use the metaclass - it will be inherited normally by any subclasses. (just use Python's 3 syntax for specifying metaclasses).
Sorry - that example is Python 2 - it requires change in another line as well, I better repost it:
from types import FunctionType
# from https://stackoverflow.com/a/23257774/108205
class SignatureCheckerMeta(type):
def __new__(mcls, name, baseClasses, d):
#For each method in d, check to see if any base class already
#defined a method with that name. If so, make sure the
#signatures are the same.
for methodName in d:
f = d[methodName]
for baseClass in baseClasses:
try:
fBase = getattr(baseClass, methodName)
if not inspect.getargspec(f) == inspect.getargspec(fBase):
raise BadSignatureException(str(methodName))
except AttributeError:
#This method was not defined in this base class,
#So just go to the next base class.
continue
return super().__new__(mcls, name, baseClasses, d)
On reviewing that, I see that there is no mechanism in it to enforce that a method is actually implemented. I.e. if a method with the same name exists in the derived class, its signature is enforced, but if it does not exist at all in the derived class, the code above won't find out about it (and the method on the superclass will be called - that might be a desired behavior).
The answer:
Fourth -
Although that will work, it can be a bit rough - since it does any method that override another method in any superclass will have to conform to its signature. And even compatible signatures would break. Maybe it would be nice to build upon the ABCMeta and #abstractmethod existind mechanisms, as those already work all corner cases. Note however that this example is based on the code above, and check signatures at class creation time, while the abstractclass mechanism in Python makes it check when the class is instantiated. Leaving it untouched will enable you to work with a large class hierarchy, which might keep some abstractmethods in intermediate classes, and just the final, concrete classes have to implement all methods.
Just use this instead of ABCMeta as the metaclass for your interface classes, and mark the methods you want to check the interface as #abstractmethod as usual.
class M(ABCMeta):
def __init__(cls, name, bases, attrs):
errors = []
for base_cls in bases:
for meth_name in getattr(base_cls, "__abstractmethods__", ()):
orig_argspec = inspect.getfullargspec(getattr(base_cls, meth_name))
target_argspec = inspect.getfullargspec(getattr(cls, meth_name))
if orig_argspec != target_argspec:
errors.append(f"Abstract method {meth_name!r} not implemented with correct signature in {cls.__name__!r}. Expected {orig_argspec}.")
if errors:
raise TypeError("\n".join(errors))
super().__init__(name, bases, attrs)
You could follow the pyspark pattern, where the method of the base class performs (optional) argument validity checking, and then calls a "non-public" method of the subclass, for example:
class Regressor():
def fit(self, X, y):
self._check_arguments(X, y)
self._fit(X, y)
def _check_arguments(self, X, y):
if True:
pass
else:
raise ValueError('Invalid arguments.')
class LinearRegressor(Regressor):
def _fit(self, X, y):
# code here
Say I have a third-party library where a metaclass requires me to implement something. But I want to have an intermediate "abstract" subclass that doesn't. How can I do this?
Consider this to be a very minimal example of what third-party library has:
class ServingMeta(type):
def __new__(cls, name, bases, classdict):
if any(isinstance(b, ServingMeta) for b in bases):
if "name" not in classdict:
# Actual code fails for a different reason,
# but the logic is the same.
raise TypeError(f"Class '{name}' has no 'name' attribute")
return super().__new__(cls, name, bases, classdict)
class Serving(object, metaclass=ServingMeta):
def shout_name(self):
return self.name.upper()
I cannot modify the code above. It's an external dependency (and I don't want to fork it).
The code is meant to be used this way:
class Spam(Serving):
name = "SPAM"
spam = Spam()
print(spam.shout_name())
However, I happen to have a lot of spam, and I want to introduce a base class with the common helper methods. Something like this:
class Spam(Serving):
def thrice(self):
return " ".join([self.shout_name()] * 3)
class LovelySpam(Spam):
name = "lovely spam"
class WonderfulSpam(Spam):
name = "wonderful spam"
Obviously, this doesn't work and fails with the well-expected TypeError: Class 'SpamBase' has no 'name' attribute declared. Would third-party library had a SpamBase class without a metaclass, I could've subclassed that - but no such luck this time (I've mentioned this inconvenience to the library authors).
I can make it a mixin:
class SpamMixin(object):
def thrice(self):
return " ".join([self.shout_name()] * 3)
class LovelySpam(SpamMixin, Serving):
name = "lovely spam"
class WonderfulSpam(SpamMixin, Serving):
name = "wonderful spam"
However, this makes me and my IDE cringe a little, as it quickly becomes cumbersome to repeat SpamMixin everywhere and also because object has no shout_name attribute (and I don't want to silence analysis tools). In short, I just don't like this approach.
What else can I do?
Is there a way to get a metaclass-less version of Serving? I think of something like this:
ServingBase = remove_metaclass(Serving)
class Spam(ServingBase, metaclass=ServingMeta):
...
But don't know how to actually implement remove_metaclass and whenever it's even reasonably possible (of course, it must be doable, with some introspection, but it could require more arcane magic than I can cast).
Any other suggestions are also welcomed. Basically, I want to have my code DRY (one base class to rule them all), and have my linter/code analysis icons all green.
The mixin approach is the correct way to go. If your IDE "cringe" that is a deffect on that tool - just disable a little of the "features" that are obviously incorrect tunning when coding for a dynamic language like Python.
And this is not even about creating things dynamically, it is merely multiple-inheritance, which is supported by the language since forever. And one of the main uses of multiple-inheritance is exactly being able to create mixins just as this one you need.
Another inheritance-based workaround is to make your hierarchy one level deeper, and just introduce the metaclass after you come up with your mixin methods:
class Mixin(object):
def mimixin(self): ...
class SpamBase(Mixin, metaclass=ServingMeta):
name = "stub"
Or just addd the mixin in an intermediate subclass:
class Base(metaclass=Serving Meta):
name = "stub"
class MixedBase(Mixin, Base):
name = "stub"
class MyLovingSpam(MixedBase):
name = "MyLovingSpam"
If you don't want to be repeating the mixin=-base name in every class, that is the way to go.
"Removing" a metaclass just for the sake of having a late mixin is way over the top. Really. Broken. The way to do it wol e re-create the class dynamically, just as #vaultah mentions in the other answer, but doing that in an intermediate class is a thing you should not do. Doing that to please the IDE is something you should not do twice: messing with metaclasses is hard enough already. Removing things on inheritance/class creation that the language puts there naturally is something nasty (cf. this answer: How to make a class attribute exclusive to the super class ) . On the other hand, mixins and multiple inheritance are just natural.
Are you still there? I told you not to do so:
Now, onto your question - instead of "supressing the metaclass" in an intermediate class, it would be more feasible to inherit the metaclass you have there and change its behavior - so that it does not check for the constraints in specially marked classes - create an attribute for your use, like _skip_checking
class MyMeta(ServingMeta):
def __new__(metacls, name, bases, namespace):
if namespace.get("_skip_checking", False):
# hardcode call to "type" metaclass:
del namespace["_skip_checking"]
cls = type.__new__(metacls, name, bases, namespace)
else:
cls = super().__new__(metacls, name, bases, namespace)
return cls
# repeat for __init__ if needed.
class Base(metaclass=MyMeta):
_skip_checking = True
# define mixin methods
class LoveSpam(Base):
name = "LoveSpam"
There's really no direct way to remove the metaclass from a Python class, because the metaclass created that class. What you can try is re-create the class using a different metaclass, which doesn't have unwanted behaviour. For example, you could use type (the default metaclass).
In [6]: class Serving(metaclass=ServingMeta):
...: def shout_name(self):
...: return self.name.upper()
...:
In [7]: ServingBase = type('ServingBase', Serving.__bases__, dict(vars(Serving)))
Basically this takes the __bases__ tuple and the namespace of the Serving class, and uses them to create a new class ServingBase. N.B. this means that ServingBase will receive all bases and methods/attributes from Serving, some of which may have been added by ServingMeta.
I have gone through this: What is a metaclass in Python?
But can any one explain more specifically when should I use the meta class concept and when it's very handy?
Suppose I have a class like below:
class Book(object):
CATEGORIES = ['programming','literature','physics']
def _get_book_name(self,book):
return book['title']
def _get_category(self, book):
for cat in self.CATEGORIES:
if book['title'].find(cat) > -1:
return cat
return "Other"
if __name__ == '__main__':
b = Book()
dummy_book = {'title':'Python Guide of Programming', 'status':'available'}
print b._get_category(dummy_book)
For this class.
In which situation should I use a meta class and why is it useful?
Thanks in advance.
You use metaclasses when you want to mutate the class as it is being created. Metaclasses are hardly ever needed, they're hard to debug, and they're difficult to understand -- but occasionally they can make frameworks easier to use. In our 600Kloc code base we've used metaclasses 7 times: ABCMeta once, 4x models.SubfieldBase from Django, and twice a metaclass that makes classes usable as views in Django. As #Ignacio writes, if you don't know that you need a metaclass (and have considered all other options), you don't need a metaclass.
Conceptually, a class exists to define what a set of objects (the instances of the class) have in common. That's all. It allows you to think about the instances of the class according to that shared pattern defined by the class. If every object was different, we wouldn't bother using classes, we'd just use dictionaries.
A metaclass is an ordinary class, and it exists for the same reason; to define what is common to its instances. The default metaclass type provides all the normal rules that make classes and instances work the way you're used to, such as:
Attribute lookup on an instance checks the instance followed by its class, followed by all superclasses in MRO order
Calling MyClass(*args, **kwargs) invokes i = MyClass.__new__(MyClass, *args, **kwargs) to get an instance, then invokes i.__init__(*args, **kwargs) to initialise it
A class is created from the definitions in a class block by making all the names bound in the class block into attributes of the class
Etc
If you want to have some classes that work differently to normal classes, you can define a metaclass and make your unusual classes instances of the metaclass rather than type. Your metaclass will almost certainly be a subclass of type, because you probably don't want to make your different kind of class completely different; just as you might want to have some sub-set of Books behave a bit differently (say, books that are compilations of other works) and use a subclass of Book rather than a completely different class.
If you're not trying to define a way of making some classes work differently to normal classes, then a metaclass is probably not the most appropriate solution. Note that the "classes define how their instances work" is already a very flexible and abstract paradigm; most of the time you do not need to change how classes work.
If you google around, you'll see a lot of examples of metaclasses that are really just being used to go do a bunch of stuff around class creation; often automatically processing the class attributes, or finding new ones automatically from somewhere. I wouldn't really call those great uses of metaclasses. They're not changing how classes work, they're just processing some classes. A factory function to create the classes, or a class method that you invoke immediately after class creation, or best of all a class decorator, would be a better way to implement this sort of thing, in my opinion.
But occasionally you find yourself writing complex code to get Python's default behaviour of classes to do something conceptually simple, and it actually helps to step "further out" and implement it at the metaclass level.
A fairly trivial example is the "singleton pattern", where you have a class of which there can only be one instance; calling the class will return an existing instance if one has already been created. Personally I am against singletons and would not advise their use (I think they're just global variables, cunningly disguised to look like newly created instances in order to be even more likely to cause subtle bugs). But people use them, and there are huge numbers of recipes for making singleton classes using __new__ and __init__. Doing it this way can be a little irritating, mainly because Python wants to call __new__ and then call __init__ on the result of that, so you have to find a way of not having your initialisation code re-run every time someone requests access to the singleton. But wouldn't be easier if we could just tell Python directly what we want to happen when we call the class, rather than trying to set up the things that Python wants to do so that they happen to do what we want in the end?
class Singleton(type):
def __init__(self, *args, **kwargs):
super(Singleton, self).__init__(*args, **kwargs)
self.__instance = None
def __call__(self, *args, **kwargs):
if self.__instance is None:
self.__instance = super(Singleton, self).__call__(*args, **kwargs)
return self.__instance
Under 10 lines, and it turns normal classes into singletons simply by adding __metaclass__ = Singleton, i.e. nothing more than a declaration that they are a singleton. It's just easier to implement this sort of thing at this level, than to hack something out at the class level directly.
But for your specific Book class, it doesn't look like you have any need to do anything that would be helped by a metaclass. You really don't need to reach for metaclasses unless you find the normal rules of how classes work are preventing you from doing something that should be simple in a simple way (which is different from "man, I wish I didn't have to type so much for all these classes, I wonder if I could auto-generate the common bits?"). In fact, I have never actually used a metaclass for something real, despite using Python every day at work; all my metaclasses have been toy examples like the above Singleton or else just silly exploration.
A metaclass is used whenever you need to override the default behavior for classes, including their creation.
A class gets created from the name, a tuple of bases, and a class dict. You can intercept the creation process to make changes to any of those inputs.
You can also override any of the services provided by classes:
__call__ which is used to create instances
__getattribute__ which is used to lookup attributes and methods on a class
__setattr__ which controls setting attributes
__repr__ which controls how the class is diplayed
In summary, metaclasses are used when you need to control how classes are created or when you need to alter any of the services provided by classes.
If you for whatever reason want to do stuff like Class[x], x in Class etc., you have to use metaclasses:
class Meta(type):
def __getitem__(cls, x):
return x ** 2
def __contains__(cls, x):
return int(x ** (0.5)) == x ** 0.5
# Python 2.x
class Class(object):
__metaclass__ = Meta
# Python 3.x
class Class(metaclass=Meta):
pass
print Class[2]
print 4 in Class
check the link Meta Class Made Easy to know how and when to use meta class.
I have a number of atomic classes (Components/Mixins, not really sure what to call them) in a library I'm developing, which are meant to be subclassed by applications. This atomicity was created so that applications can only use the features that they need, and combine the components through multiple inheritance.
However, sometimes this atomicity cannot be ensured because some component may depend on another one. For example, imagine I have a component that gives a graphical representation to an object, and another component which uses this graphical representation to perform some collision checking. The first is purely atomic, however the latter requires that the current object already subclassed this graphical representation component, so that its methods are available to it. This is a problem, because we have to somehow tell the users of this library, that in order to use a certain Component, they also have to subclass this other one. We could make this collision component sub class the visual component, but if the user also subclasses this visual component, it wouldn't work because the class is not on the same level (unlike a simple diamond relationship, which is desired), and would give the cryptic meta class errors which are hard to understand for the programmer.
Therefore, I would like to know if there is any cool way, through maybe metaclass redefinition or using class decorators, to mark these unatomic components, and when they are subclassed, the additional dependency would be injected into the current object, if its not yet available. Example:
class AtomicComponent(object):
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent(object):
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
Can someone give me an hint on how I can do this? or if it is even possible...
edit:
Since it is debatable that the meta class solution is the best one, I'll leave this unaccepted for 2 days.
Other solutions might be to improve error messages, for example, doing something like UserClass2 would give an error saying that UnAtomicComponent already extends this component. This however creates the problem that it is impossible to use two UnAtomicComponents, given that they would subclass object on different levels.
"Metaclasses"
This is what they are for! At time of class creation, the class parameters run through the
metaclass code, where you can check the bases and change then, for example.
This runs without error - though it does not preserve the order of needed classes
marked with the "depends" decorator:
class AutoSubclass(type):
def __new__(metacls, name, bases, dct):
new_bases = set()
for base in bases:
if hasattr(base, "_depends"):
for dependence in base._depends:
if not dependence in bases:
new_bases.add(dependence)
bases = bases + tuple(new_bases)
return type.__new__(metacls, name, bases, dct)
__metaclass__ = AutoSubclass
def depends(*args):
def decorator(cls):
cls._depends = args
return cls
return decorator
class AtomicComponent:
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent:
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
(I removed inheritance from "object", as I declared a global __metaclass__ variable. All classs will still be new style class and have this metaclass. Inheriting from object or another class does override the global __metaclass__variable, and a class level __metclass__ will have to be declared)
-- edit --
Without metaclasses, the way to go is to have your classes to properly inherit from their dependencies. Tehy will no longer be that "atomic", but, since they could not work being that atomic, it may be no matter.
In the example bellow, classes C and D would be your User classes:
>>> class A(object): pass
...
>>> class B(A, object): pass
...
>>>
>>> class C(B): pass
...
>>> class D(B,A): pass
...
>>>