I'm trying to get a python2 program working in python3, it has the following Meta class definition. Which works just fine on Py2. What's the "best" way to have this be compatible with both py2 and py3?
It's failing in the unit test where it does:
try:
raise Actor.DoesNotExist
except Actor.DoesNotExist:
pass
Failure is:
AttributeError: type object 'Actor' has no attribute 'DoesNotExist'
The base meta class definition is:
class MetaDocument(type):
def __new__(meta,name,bases,dct):
class DoesNotExist(BaseException):
pass
class MultipleDocumentsReturned(BaseException):
pass
dct['DoesNotExist'] = DoesNotExist
dct['MultipleDocumentsReturned'] = MultipleDocumentsReturned
class_type = type.__new__(meta, name, bases, dct)
if not class_type in document_classes:
if name == 'Document' and bases == (object,):
pass
else:
document_classes.append(class_type)
return class_type
class Document(object):
__metaclass__ = MetaDocument
You could use the MetaDocument() metaclass as a factory to produce a class replacing your Document class, re-using the class attributes:
class Document(object):
# various and sundry methods and attributes
body = vars(Document).copy()
body.pop('__dict__', None)
body.pop('__weakref__', None)
Document = MetaDocument(Document.__name__, Document.__bases__, body)
This doesn't require you to build the 3rd argument, the class body, manually.
You can turn this into a class decorator:
def with_metaclass(mcls):
def decorator(cls):
body = vars(cls).copy()
# clean out class body
body.pop('__dict__', None)
body.pop('__weakref__', None)
return mcls(cls.__name__, cls.__bases__, body)
return decorator
then use as:
#with_metaclass(MetaDocument)
class Document(object):
# various and sundry methods and attributes
Alternatively, use the six library for this:
#six.add_metaclass(MetaDocument)
class Document(object):
where the #six.add_metaclass() decorator also takes care of any __slots__ you may have defined; my simpler version above doesn't.
six also has a six.with_metaclass() base-class factory:
class Document(six.with_metaclass(MetaDocument)):
which injects an extra base class into the MRO.
six has a utility for this.
class Document(six.with_metaclass(MetaDocument, object)):
# class definition, without the __metaclass__
The only side effect is that the class hierarchy changes from
>>> Document.__mro__
(<class 'test.Document'>, <type 'object'>)
to
>>> Document.__mro__
(<class 'test.Document'>, <class 'test.NewBase'>, <type 'object'>)
because with_metaclass actually returns a new class with the appropriate metaclass.
Related
For the below code in python 3,
class Spam(object):
def __init__(self,name):
self.name = name
def bar(self):
print('Am Spam.bar')
metaclass for Spam is type and base class for Spam is object.
My understanding is,
the purpose of base class is to inherit the properties. Metaclass is to construct the given class definition, as shown below,
body= \
"""
def __init__(self, name):
self.name = name
def bar(self):
print('Am ', self.name)
"""
clsdict = type.__prepare__('type', 'Spam', (object,))
exec(body, globals(), clsdict)
Spam = type('Spam', (object,), clsdict)
s = Spam('xyz')
s.bar()
Code is tested here.
With the given syntax def __prepare__(metacls, name, bases) to use,
Does __prepare__() require passing 'type' as first argument?
type.__prepare__ is a bit special, in that it ignores all and any arguments passed to it and returns an empty dict.
>>> type.__prepare__()
{}
That said, you are not calling __prepare__ correctly. It is called with: the name of the class to be created, its bases and any keyword arguments the class is being created with. __prepare__ is called as metaclass.__prepare__(name, bases, **kwds) Thus,
class MyClass(SomeBase, arg="value", metaclass=MyMeta):
...
will have __prepare__ called as:
MyMeta.__prepare__("MyClass", (SomeBase,), arg="value")
However, most user defined meta classes define their __prepare__ as a classmethod meaning the metaclass is implicitly passed. Meaning your __prepare__ definition can look like:
#classmethod
def __prepare__(metaclass, name, bases, **kwargs):
...
But __prepare__ is still called in the same way as before. It is through the magic of descriptors that the metaclass argument is added.
I have spent a lot of time researching this, but none of the answers seem to work how I would like.
I have an abstract class with a class attribute I want each subclass to be forced to implement
class AbstractFoo():
forceThis = 0
So that when I do this
class RealFoo(AbstractFoo):
pass
it throws an error telling me it can't create the class until I implement forceThis.
How can I do that?
(I don't want the attribute to be read-only, but if that's the only solution, I'll accept it.)
For a class method, I've discovered I can do
from abc import ABCMeta, abstractmethod
class AbstractFoo(metaclass=ABCMeta):
#classmethod
#abstractmethod
def forceThis():
"""This must be implemented"""
so that
class RealFoo(AbstractFoo):
pass
at least throws the error TypeError: Can't instantiate abstract class EZ with abstract methods forceThis
(Although it doesn't force forceThis to be a class method.)
How can I get a similar error to pop up for the class attribute?
You can do this by defining your own metaclass. Something like:
class ForceMeta(type):
required = ['foo', 'bar']
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
for prop in mcls.required:
if not hasattr(cls, prop):
raise NotImplementedError('must define {}'.format(prop))
return cls
Now you can use this as the metaclass of your own classes:
class RequiredClass(metaclass=ForceMeta):
foo = 1
which will raise the error 'must define bar'.
I came up with a solution based on those posted earlier. (Thank you #Daniel Roseman and #martineau)
I created a metaclass called ABCAMeta (the last 'A' stands for 'Attributes').
The class has two ways of working.
A class which just uses ABCAMeta as a metaclass must have a property called required_attributes which should contain a list of the names of all the attributes you want to require on future subclasses of that class
A class whose parent's metaclass is ABCAMeta must have all the required attributes specified by its parent class(es).
For example:
class AbstractFoo(metaclass=ABCAMeta):
required_attributes = ['force_this']
class RealFoo(AbstractFoo):
pass
will throw an error:
NameError: Class 'RealFoo' has not implemented the following attributes: 'force_this'
Exactly how I wanted.
from abc import ABCMeta
class NoRequirements(RuntimeError):
def __init__(self, message):
RuntimeError.__init__(self, message)
class ABCAMeta(ABCMeta):
def __init__(mcls, name, bases, namespace):
ABCMeta.__init__(mcls, name, bases, namespace)
def __new__(mcls, name, bases, namespace):
def get_requirements(c):
"""c is a class that should have a 'required_attributes' attribute
this function will get that list of required attributes or
raise a NoRequirements error if it doesn't find one.
"""
if hasattr(c, 'required_attributes'):
return c.required_attributes
else:
raise NoRequirements(f"Class '{c.__name__}' has no 'required_attributes' property")
cls = super().__new__(mcls, name, bases, namespace)
# true if no parents of the class being created have ABCAMeta as their metaclass
basic_metaclass = True
# list of attributes the class being created must implement
# should stay empty if basic_metaclass stays True
reqs = []
for parent in bases:
parent_meta = type(parent)
if parent_meta==ABCAMeta:
# the class being created has a parent whose metaclass is ABCAMeta
# the class being created must contain the requirements of the parent class
basic_metaclass=False
try:
reqs.extend(get_requirements(parent))
except NoRequirements:
raise
# will force subclasses of the created class to define
# the attributes listed in the required_attributes attribute of the created class
if basic_metaclass:
get_requirements(cls) # just want it to raise an error if it doesn't have the attributes
else:
missingreqs = []
for req in reqs:
if not hasattr(cls, req):
missingreqs.append(req)
if len(missingreqs)!=0:
raise NameError(f"Class '{cls.__name__}' has not implemented the following attributes: {str(missingreqs)[1:-1]}")
return cls
Any suggestions for improvement are welcome in the comments.
Although you can do something very similar with a metaclass, as illustrated in #Daniel Roseman's answer, it can also be done with a class decorator. A couple of advantages they have are that errors will occur when the class is defined, instead of when an instance of one is created, and the syntax for specifying them is the same in both Python 2 and 3. Some folks also find them simpler and easier to understand.
def check_reqs(cls):
requirements = 'must_have',
missing = [req for req in requirements if not hasattr(cls, req)]
if missing:
raise NotImplementedError(
'class {} did not define required attribute{} named {}'.format(
cls.__name__, 's' if len(missing) > 1 else '',
', '.join('"{}"'.format(name) for name in missing)))
return cls
#check_reqs
class Foo(object): # OK
must_have = 42
#check_reqs
class Bar(object): # raises a NotImplementedError
pass
I have a master class for a planet:
class Planet:
def __init__(self,name):
self.name = name
(...)
def destroy(self):
(...)
I also have a few classes that inherit from Planet and I want to make one of them unable to be destroyed (not to inherit the destroy function)
Example:
class Undestroyable(Planet):
def __init__(self,name):
super().__init__(name)
(...)
#Now it shouldn't have the destroy(self) function
So when this is run,
Undestroyable('This Planet').destroy()
it should produce an error like:
AttributeError: Undestroyable has no attribute 'destroy'
The mixin approach in other answers is nice, and probably better for most cases. But nevertheless, it spoils part of the fun - maybe obliging you to have separate planet-hierarchies - like having to live with two abstract classes each ancestor of "destroyable" and "non-destroyable".
First approach: descriptor decorator
But Python has a powerful mechanism, called the "descriptor protocol", which is used to retrieve any attribute from a class or instance - it is even used to ordinarily retrieve methods from instances - so, it is possible to customize the method retrieval in a way it checks if it "should belong" to that class, and raise attribute error otherwise.
The descriptor protocol mandates that whenever you try to get any attribute from an instance object in Python, Python will check if the attribute exists in that object's class, and if so, if the attribute itself has a method named __get__. If it has, __get__ is called (with the instance and class where it is defined as parameters) - and whatever it returns is the attribute. Python uses this to implement methods: functions in Python 3 have a __get__ method that when called, will return another callable object that, in turn, when called will insert the self parameter in a call to the original function.
So, it is possible to create a class whose __get__ method will decide whether to return a function as a bound method or not depending on the outer class been marked as so - for example, it could check an specific flag non_destrutible. This could be done by using a decorator to wrap the method with this descriptor functionality
class Muteable:
def __init__(self, flag_attr):
self.flag_attr = flag_attr
def __call__(self, func):
"""Called when the decorator is applied"""
self.func = func
return self
def __get__(self, instance, owner):
if instance and getattr(instance, self.flag_attr, False):
raise AttributeError('Objects of type {0} have no {1} method'.format(instance.__class__.__name__, self.func.__name__))
return self.func.__get__(instance, owner)
class Planet:
def __init__(self, name=""):
pass
#Muteable("undestroyable")
def destroy(self):
print("Destroyed")
class BorgWorld(Planet):
undestroyable = True
And on the interactive prompt:
In [110]: Planet().destroy()
Destroyed
In [111]: BorgWorld().destroy()
...
AttributeError: Objects of type BorgWorld have no destroy method
In [112]: BorgWorld().destroy
AttributeError: Objects of type BorgWorld have no destroy method
Perceive that unlike simply overriding the method, this approach raises the error when the attribute is retrieved - and will even make hasattr work:
In [113]: hasattr(BorgWorld(), "destroy")
Out[113]: False
Although, it won't work if one tries to retrieve the method directly from the class, instead of from an instance - in that case the instance parameter to __get__ is set to None, and we can't say from which class it was retrieved - just the owner class, where it was declared.
In [114]: BorgWorld.destroy
Out[114]: <function __main__.Planet.destroy>
Second approach: __delattr__ on the metaclass:
While writting the above, it occurred me that Pythn does have the __delattr__ special method. If the Planet class itself implements __delattr__ and we'd try to delete the destroy method on specifc derived classes, it wuld nt work: __delattr__ gards the attribute deletion of attributes in instances - and if you'd try to del the "destroy" method in an instance, it would fail anyway, since the method is in the class.
However, in Python, the class itself is an instance - of its "metaclass". That is usually type . A proper __delattr__ on the metaclass of "Planet" could make possible the "disinheitance" of the "destroy" method by issuing a `del UndestructiblePlanet.destroy" after class creation.
Again, we use the descriptor protocol to have a proper "deleted method on the subclass":
class Deleted:
def __init__(self, cls, name):
self.cls = cls.__name__
self.name = name
def __get__(self, instance, owner):
raise AttributeError("Objects of type '{0}' have no '{1}' method".format(self.cls, self.name))
class Deletable(type):
def __delattr__(cls, attr):
print("deleting from", cls)
setattr(cls, attr, Deleted(cls, attr))
class Planet(metaclass=Deletable):
def __init__(self, name=""):
pass
def destroy(self):
print("Destroyed")
class BorgWorld(Planet):
pass
del BorgWorld.destroy
And with this method, even trying to retrieve or check for the method existense on the class itself will work:
In [129]: BorgWorld.destroy
...
AttributeError: Objects of type 'BorgWorld' have no 'destroy' method
In [130]: hasattr(BorgWorld, "destroy")
Out[130]: False
metaclass with a custom __prepare__ method.
Since metaclasses allow one to customize the object that contains the class namespace, it is possible to have an object that responds to a del statement within the class body, adding a Deleted descriptor.
For the user (programmer) using this metaclass, it is almost the samething, but for the del statement been allowed into the class body itself:
class Deleted:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
raise AttributeError("No '{0}' method on class '{1}'".format(self.name, owner.__name__))
class Deletable(type):
def __prepare__(mcls,arg):
class D(dict):
def __delitem__(self, attr):
self[attr] = Deleted(attr)
return D()
class Planet(metaclass=Deletable):
def destroy(self):
print("destroyed")
class BorgPlanet(Planet):
del destroy
(The 'deleted' descriptor is the correct form to mark a method as 'deleted' - in this method, though, it can't know the class name at class creation time)
As a class decorator:
And given the "deleted" descriptor, one could simply inform the methods to be removed as a class decorator - there is no need for a metaclass in this case:
class Deleted:
def __init__(self, cls, name):
self.cls = cls.__name__
self.name = name
def __get__(self, instance, owner):
raise AttributeError("Objects of type '{0}' have no '{1}' method".format(self.cls, self.name))
def mute(*methods):
def decorator(cls):
for method in methods:
setattr(cls, method, Deleted(cls, method))
return cls
return decorator
class Planet:
def destroy(self):
print("destroyed")
#mute('destroy')
class BorgPlanet(Planet):
pass
Modifying the __getattribute__ mechanism:
For sake of completeness - what really makes Python reach methods and attributes on the super-class is what happens inside the __getattribute__ call. n the object version of __getattribute__ is where the algorithm with the priorities for "data-descriptor, instance, class, chain of base-classes, ..." for attribute retrieval is encoded.
So, changing that for the class is an easy an unique point to get a "legitimate" attribute error, without need for the "non-existent" descritor used on the previous methods.
The problem is that object's __getattribute__ does not make use of type's one to search the attribute in the class - if it did so, just implementing the __getattribute__ on the metaclass would suffice. One have to do that on the instance to avoid instance lookp of an method, and on the metaclass to avoid metaclass look-up. A metaclass can, of course, inject the needed code:
def blocker_getattribute(target, attr, attr_base):
try:
muted = attr_base.__getattribute__(target, '__muted__')
except AttributeError:
muted = []
if attr in muted:
raise AttributeError("object {} has no attribute '{}'".format(target, attr))
return attr_base.__getattribute__(target, attr)
def instance_getattribute(self, attr):
return blocker_getattribute(self, attr, object)
class M(type):
def __init__(cls, name, bases, namespace):
cls.__getattribute__ = instance_getattribute
def __getattribute__(cls, attr):
return blocker_getattribute(cls, attr, type)
class Planet(metaclass=M):
def destroy(self):
print("destroyed")
class BorgPlanet(Planet):
__muted__=['destroy'] # or use a decorator to set this! :-)
pass
If Undestroyable is a unique (or at least unusual) case, it's probably easiest to just redefine destroy():
class Undestroyable(Planet):
# ...
def destroy(self):
cls_name = self.__class__.__name__
raise AttributeError("%s has no attribute 'destroy'" % cls_name)
From the point of view of the user of the class, this will behave as though Undestroyable.destroy() doesn't exist … unless they go poking around with hasattr(Undestroyable, 'destroy'), which is always a possibility.
If it happens more often that you want subclasses to inherit some properties and not others, the mixin approach in chepner's answer is likely to be more maintainable. You can improve it further by making Destructible an abstract base class:
from abc import abstractmethod, ABCMeta
class Destructible(metaclass=ABCMeta):
#abstractmethod
def destroy(self):
pass
class BasePlanet:
# ...
pass
class Planet(BasePlanet, Destructible):
def destroy(self):
# ...
pass
class IndestructiblePlanet(BasePlanet):
# ...
pass
This has the advantage that if you try to instantiate the abstract class Destructible, you'll get an error pointing you at the problem:
>>> Destructible()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Destructible with abstract methods destroy
… similarly if you inherit from Destructible but forget to define destroy():
class InscrutablePlanet(BasePlanet, Destructible):
pass
>>> InscrutablePlanet()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class InscrutablePlanet with abstract methods destroy
Rather than remove an attribute that is inherited, only inherit destroy in the subclasses where it is applicable, via a mix-in class. This preserves the correct "is-a" semantics of inheritance.
class Destructible(object):
def destroy(self):
pass
class BasePlanet(object):
...
class Planet(BasePlanet, Destructible):
...
class IndestructiblePlanet(BasePlanet): # Does *not* inherit from Destructible
...
You can provide suitable definitions for destroy in any of Destructible, Planet, or any class that inherits from Planet.
Metaclasses and descriptor protocols are fun, but perhaps overkill. Sometimes, for raw functionality, you can't beat good ole' __slots__.
class Planet(object):
def __init__(self, name):
self.name = name
def destroy(self):
print("Boom! %s is toast!\n" % self.name)
class Undestroyable(Planet):
__slots__ = ['destroy']
def __init__(self,name):
super().__init__(name)
print()
x = Planet('Pluto') # Small, easy to destroy
y = Undestroyable('Jupiter') # Too big to fail
x.destroy()
y.destroy()
Boom! Pluto is toast!
Traceback (most recent call last):
File "planets.py", line 95, in <module>
y.destroy()
AttributeError: destroy
You cannot inherit only a portion of a class. Its all or nothing.
What you can do is to put the destroy function in a second level of the class, such you have the Planet-class without the destry-function, and then you make a DestroyablePlanet-Class where you add the destroy-function, which all the destroyable planets use.
Or you can put a flag in the construct of the Planet-Class which determines if the destroy function will be able to succeed or not, which is then checked in the destroy-function.
I want to add an attribute for every class created by a metaclass. For example, when a class named C is created, I want add an attribute C._C__sup whose value is the descriptor super(C).
Here is what I've tried:
class Meta(type):
def __init__(cls, name, bases, dict): # Not overriding type.__new__
cls.__dict__['_' + name + '__sup'] = super(cls)
# Not calling type.__init__; do I need it?
class C(object):
__metaclass__ = Meta
c = C()
print c._C__sup
This gives me:
TypeError: Error when calling the metaclass bases
'dictproxy' object does not support item assignment
Some background information:
(You don't have to read this part)
Inspired by this article, what I'm doing is trying to avoid "hardcoding" the class name when using super:
The idea there is to use the unbound super objects as private
attributes. For instance, in our example, we could define the private
attribute __sup in the class C as the unbound super object
super(C):
>>> C._C__sup = super(C)
With this definition inside the methods the syntax self.__sup.meth
can be used as an alternative to super(C, self).meth. The advantage
is that you avoid to repeat the name of the class in the calling
syntax, since that name is hidden in the mangling mechanism of private
names. The creation of the __sup attributes can be hidden in a
metaclass and made automatic. So, all this seems to work: but
actually this not the case.
Use setattr instead of assignment to cls.__dict__:
class Meta(type):
def __init__(cls, name, bases, clsdict): # Not overriding type.__new__
setattr(cls, '_' + name + '__sup', super(cls))
super(Meta, cls).__init__(name, bases, clsdict)
class C(object):
__metaclass__ = Meta
def say(self):
return 'wow'
class D(C):
def say(self):
return 'bow' + self.__sup.say()
c = C()
print(c._C__sup)
# <super: <class 'C'>, <C object>>
d = D()
print(d.say())
prints
bowwow
By the way, it is a good idea to call
super(Meta, cls).__init__(name, bases, clsdict)
inside Meta.__init__ to allow Meta to participate in class hierarchies which
might need super to properly call a chain of __init__s. This seems
particularly appropriate since you are building a metaclass to assist with the
use of super.
So I have a .py file containing a class where its subclasses can be accessed as properties. All these subclasses are defined beforehand. I also need all the subclasses to have the same ability (having their own subclasses be accessible as properties). The biggest problem I've been facing is that I don't know how to access the current class within my implementation of __getattr__(), so that'd be a good place to start.
Here's some Python+Pseudocode with what I've tried so far. I'm pretty sure it won't work since __getattr__() seems to be only working with instances of a class. If that is case, sorry, I am not as familiar with OOP in Python as I would like.
class A(object):
def __getattr__(self, name):
subclasses = [c.__name__ for c in current_class.__subclasses__()]
if name in subclasses:
return name
raise AttributeError
If I've understood your question properly, you can do what you want by using a custom metaclass that adds a classmethod to its instances. Here's an example:
class SubclassAttributes(type):
def __getattr__(cls, name): # classmethod of instances
for subclass in cls.__subclasses__():
if subclass.__name__ == name:
return subclass
else:
raise TypeError('Class {!r} has no subclass '
'named {!r}'.format(cls.__name__, name))
class Base(object):
__metaclass__ = SubclassAttributes # Python 2 metaclass syntax
#class Base(object, metaclass=SubclassAttributes): # Python 3 metaclass syntax
# """ nothing to see here """
class Derived1(Base): pass
class Derived2(Base): pass
print(Base.Derived1) # -> <class '__main__.Derived1'>
print(Base.Derived2) # -> <class '__main__.Derived2'>
print(Base.Derived3) # -> TypeError: Class 'Base' has no subclass named 'Derived3'
For something that works in both Python 2 and 3, define the class as shown below. Derives Base from a class that has SubclassAttributes as its metaclass. The is similar to what the six module's with_metaclass() function does:
class Base(type.__new__(type('TemporaryMeta', (SubclassAttributes,), {}),
'TemporaryClass', (), {})): pass
class A(object):
def __getattr__(self, key):
for subclass in self.__class__.__subclasses__():
if (subclass.__name__ == key):
return subclass
raise AttributeError, key
Out of curiosity, what is this designed to be used for?
>>> class A(object):
... pass
...
>>> foo = A()
>>> foo.__class__
<class '__main__.A'>