So I have a .py file containing a class where its subclasses can be accessed as properties. All these subclasses are defined beforehand. I also need all the subclasses to have the same ability (having their own subclasses be accessible as properties). The biggest problem I've been facing is that I don't know how to access the current class within my implementation of __getattr__(), so that'd be a good place to start.
Here's some Python+Pseudocode with what I've tried so far. I'm pretty sure it won't work since __getattr__() seems to be only working with instances of a class. If that is case, sorry, I am not as familiar with OOP in Python as I would like.
class A(object):
def __getattr__(self, name):
subclasses = [c.__name__ for c in current_class.__subclasses__()]
if name in subclasses:
return name
raise AttributeError
If I've understood your question properly, you can do what you want by using a custom metaclass that adds a classmethod to its instances. Here's an example:
class SubclassAttributes(type):
def __getattr__(cls, name): # classmethod of instances
for subclass in cls.__subclasses__():
if subclass.__name__ == name:
return subclass
else:
raise TypeError('Class {!r} has no subclass '
'named {!r}'.format(cls.__name__, name))
class Base(object):
__metaclass__ = SubclassAttributes # Python 2 metaclass syntax
#class Base(object, metaclass=SubclassAttributes): # Python 3 metaclass syntax
# """ nothing to see here """
class Derived1(Base): pass
class Derived2(Base): pass
print(Base.Derived1) # -> <class '__main__.Derived1'>
print(Base.Derived2) # -> <class '__main__.Derived2'>
print(Base.Derived3) # -> TypeError: Class 'Base' has no subclass named 'Derived3'
For something that works in both Python 2 and 3, define the class as shown below. Derives Base from a class that has SubclassAttributes as its metaclass. The is similar to what the six module's with_metaclass() function does:
class Base(type.__new__(type('TemporaryMeta', (SubclassAttributes,), {}),
'TemporaryClass', (), {})): pass
class A(object):
def __getattr__(self, key):
for subclass in self.__class__.__subclasses__():
if (subclass.__name__ == key):
return subclass
raise AttributeError, key
Out of curiosity, what is this designed to be used for?
>>> class A(object):
... pass
...
>>> foo = A()
>>> foo.__class__
<class '__main__.A'>
Related
I'm creating some custom classes in Python and I was wondering if there was any way to define an attribute on a class without having all of its instances inherit the attribute.
For example:
class Foo():
def bar():
pass
Foo.bar # would return `bar` function
instanceOfFoo = Foo()
instanceOfFoo.bar # would raise an AttributeError
I know I could subclass Foo, override bar, and manually raise an AttributeError as a #property to give the "appearance" that bar doesn't exist, but is there any way to do this without subclasses?
Context: I'm trying to replicate the Date class from JavaScript, where calling Date.parse (on the Date class itself) will do something, but calling parse on a Date instance will not do anything because instances don't inherit the parse function from Date. (They inherit all their properties from Date.prototype which is why it works there.)
I found what I was looking for using metaclasses.
class MetaFoo(type):
def bar(self):
pass
class Foo(metaclasses=MetaFoo):
pass
Foo.bar # returns `bar` method
instanceOfFoo = Foo()
instanceOfFoo.bar # raises AttributeError
Foo is being created using the MetaFoo class and is inheriting the bar method in the process. Since inheritance only works with direct instances of a class, Foo (an instance of MetaFoo) inherits the bar method, but instanceOfFoo (an instance of Foo but not MetaFoo) does not inherit the method.
There is one work-around for your question. Note that in python You can't call Foo.bar(), unless bar() is declared either as #staticmethod or #classmethod.
class Foo(object):
def __init__(self):
self.bar = self.foo
#classmethod
def bar(cls):
return 1
#classmethod
def foo(cls):
raise AttributeError("Method not implemented for instances of a class.")
print(Foo.bar()) # --> 1
t = Foo()
print(t.bar()) # --> AttributeError: Method not implemented for instances of a class.
I have spent a lot of time researching this, but none of the answers seem to work how I would like.
I have an abstract class with a class attribute I want each subclass to be forced to implement
class AbstractFoo():
forceThis = 0
So that when I do this
class RealFoo(AbstractFoo):
pass
it throws an error telling me it can't create the class until I implement forceThis.
How can I do that?
(I don't want the attribute to be read-only, but if that's the only solution, I'll accept it.)
For a class method, I've discovered I can do
from abc import ABCMeta, abstractmethod
class AbstractFoo(metaclass=ABCMeta):
#classmethod
#abstractmethod
def forceThis():
"""This must be implemented"""
so that
class RealFoo(AbstractFoo):
pass
at least throws the error TypeError: Can't instantiate abstract class EZ with abstract methods forceThis
(Although it doesn't force forceThis to be a class method.)
How can I get a similar error to pop up for the class attribute?
You can do this by defining your own metaclass. Something like:
class ForceMeta(type):
required = ['foo', 'bar']
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
for prop in mcls.required:
if not hasattr(cls, prop):
raise NotImplementedError('must define {}'.format(prop))
return cls
Now you can use this as the metaclass of your own classes:
class RequiredClass(metaclass=ForceMeta):
foo = 1
which will raise the error 'must define bar'.
I came up with a solution based on those posted earlier. (Thank you #Daniel Roseman and #martineau)
I created a metaclass called ABCAMeta (the last 'A' stands for 'Attributes').
The class has two ways of working.
A class which just uses ABCAMeta as a metaclass must have a property called required_attributes which should contain a list of the names of all the attributes you want to require on future subclasses of that class
A class whose parent's metaclass is ABCAMeta must have all the required attributes specified by its parent class(es).
For example:
class AbstractFoo(metaclass=ABCAMeta):
required_attributes = ['force_this']
class RealFoo(AbstractFoo):
pass
will throw an error:
NameError: Class 'RealFoo' has not implemented the following attributes: 'force_this'
Exactly how I wanted.
from abc import ABCMeta
class NoRequirements(RuntimeError):
def __init__(self, message):
RuntimeError.__init__(self, message)
class ABCAMeta(ABCMeta):
def __init__(mcls, name, bases, namespace):
ABCMeta.__init__(mcls, name, bases, namespace)
def __new__(mcls, name, bases, namespace):
def get_requirements(c):
"""c is a class that should have a 'required_attributes' attribute
this function will get that list of required attributes or
raise a NoRequirements error if it doesn't find one.
"""
if hasattr(c, 'required_attributes'):
return c.required_attributes
else:
raise NoRequirements(f"Class '{c.__name__}' has no 'required_attributes' property")
cls = super().__new__(mcls, name, bases, namespace)
# true if no parents of the class being created have ABCAMeta as their metaclass
basic_metaclass = True
# list of attributes the class being created must implement
# should stay empty if basic_metaclass stays True
reqs = []
for parent in bases:
parent_meta = type(parent)
if parent_meta==ABCAMeta:
# the class being created has a parent whose metaclass is ABCAMeta
# the class being created must contain the requirements of the parent class
basic_metaclass=False
try:
reqs.extend(get_requirements(parent))
except NoRequirements:
raise
# will force subclasses of the created class to define
# the attributes listed in the required_attributes attribute of the created class
if basic_metaclass:
get_requirements(cls) # just want it to raise an error if it doesn't have the attributes
else:
missingreqs = []
for req in reqs:
if not hasattr(cls, req):
missingreqs.append(req)
if len(missingreqs)!=0:
raise NameError(f"Class '{cls.__name__}' has not implemented the following attributes: {str(missingreqs)[1:-1]}")
return cls
Any suggestions for improvement are welcome in the comments.
Although you can do something very similar with a metaclass, as illustrated in #Daniel Roseman's answer, it can also be done with a class decorator. A couple of advantages they have are that errors will occur when the class is defined, instead of when an instance of one is created, and the syntax for specifying them is the same in both Python 2 and 3. Some folks also find them simpler and easier to understand.
def check_reqs(cls):
requirements = 'must_have',
missing = [req for req in requirements if not hasattr(cls, req)]
if missing:
raise NotImplementedError(
'class {} did not define required attribute{} named {}'.format(
cls.__name__, 's' if len(missing) > 1 else '',
', '.join('"{}"'.format(name) for name in missing)))
return cls
#check_reqs
class Foo(object): # OK
must_have = 42
#check_reqs
class Bar(object): # raises a NotImplementedError
pass
I have a large Python 2.3 based installation with 200k LOC. As part of a migration project I need to intercept all attribute lookups of all old-style class.
Old legacy code:
class Foo(Bar):
...
My idea is to inject a common mixin class like
class Foo(Bar, Mixin):
...
class Mixin:
def __getattr__(self, k)
print repr(self), k
return Foo.__getattr__(self, k)
However I am running always into a recursion because Foo.__getattr__ resolves
to Mixin.__getattr__.
Is there any way to fix the code for Python 2.3 old-style classes?
If you are already injecting mixins, why not add object as parent, to make them new style
class Foo(Mixin, Bar, object):
...
And then use super
class Mixin(object):
def __getattr__(self, k)
print repr(self), k
return super(Mixin, self).__getattr__(k)
Assuming that none of the classes in your code base implement __setattr__ or __getattr__ then one approach is to intercept __setattr__ in your Mixin, writing the value to another reserved attribute, then read it back in __getattr__
class Mixin:
def __setattr__(self, attr, value):
# write the value into some special reserved space
namespace = self.__dict__.setdefault("_namespace", {})
namespace[attr] = value
def __getattr__(self, attr):
# reject special methods so e.g. __repr__ can't recurse
if attr.startswith("__") and attr.endswith("__"):
raise AttributeError
# do whatever you wish to do here ...
print repr(self), attr
# read the value from the reserved space
namespace = self.__dict__.get("_namespace", {})
return namespace[attr]
Example:
class Foo(Mixin):
def __init__(self):
self.x = 1
Then
>>> Foo().x
<__main__.Foo instance at 0x10c4dad88> x
Clearly this won't work if any of your Foo classes implement __setattr__ or __getattr__ themselves.
I'm trying to get a python2 program working in python3, it has the following Meta class definition. Which works just fine on Py2. What's the "best" way to have this be compatible with both py2 and py3?
It's failing in the unit test where it does:
try:
raise Actor.DoesNotExist
except Actor.DoesNotExist:
pass
Failure is:
AttributeError: type object 'Actor' has no attribute 'DoesNotExist'
The base meta class definition is:
class MetaDocument(type):
def __new__(meta,name,bases,dct):
class DoesNotExist(BaseException):
pass
class MultipleDocumentsReturned(BaseException):
pass
dct['DoesNotExist'] = DoesNotExist
dct['MultipleDocumentsReturned'] = MultipleDocumentsReturned
class_type = type.__new__(meta, name, bases, dct)
if not class_type in document_classes:
if name == 'Document' and bases == (object,):
pass
else:
document_classes.append(class_type)
return class_type
class Document(object):
__metaclass__ = MetaDocument
You could use the MetaDocument() metaclass as a factory to produce a class replacing your Document class, re-using the class attributes:
class Document(object):
# various and sundry methods and attributes
body = vars(Document).copy()
body.pop('__dict__', None)
body.pop('__weakref__', None)
Document = MetaDocument(Document.__name__, Document.__bases__, body)
This doesn't require you to build the 3rd argument, the class body, manually.
You can turn this into a class decorator:
def with_metaclass(mcls):
def decorator(cls):
body = vars(cls).copy()
# clean out class body
body.pop('__dict__', None)
body.pop('__weakref__', None)
return mcls(cls.__name__, cls.__bases__, body)
return decorator
then use as:
#with_metaclass(MetaDocument)
class Document(object):
# various and sundry methods and attributes
Alternatively, use the six library for this:
#six.add_metaclass(MetaDocument)
class Document(object):
where the #six.add_metaclass() decorator also takes care of any __slots__ you may have defined; my simpler version above doesn't.
six also has a six.with_metaclass() base-class factory:
class Document(six.with_metaclass(MetaDocument)):
which injects an extra base class into the MRO.
six has a utility for this.
class Document(six.with_metaclass(MetaDocument, object)):
# class definition, without the __metaclass__
The only side effect is that the class hierarchy changes from
>>> Document.__mro__
(<class 'test.Document'>, <type 'object'>)
to
>>> Document.__mro__
(<class 'test.Document'>, <class 'test.NewBase'>, <type 'object'>)
because with_metaclass actually returns a new class with the appropriate metaclass.
I have the following python code:
class FooMeta(type):
def __setattr__(self, name, value):
print name, value
return super(FooMeta, self).__setattr__(name, value)
class Foo(object):
__metaclass__ = FooMeta
FOO = 123
def a(self):
pass
I would have expected __setattr__ of the meta class being called for both FOO and a. However, it is not called at all. When I assign something to Foo.whatever after the class has been defined the method is called.
What's the reason for this behaviour and is there a way to intercept the assignments that happen during the creation of the class? Using attrs in __new__ won't work since I'd like to check if a method is being redefined.
A class block is roughly syntactic sugar for building a dictionary, and then invoking a metaclass to build the class object.
This:
class Foo(object):
__metaclass__ = FooMeta
FOO = 123
def a(self):
pass
Comes out pretty much as if you'd written:
d = {}
d['__metaclass__'] = FooMeta
d['FOO'] = 123
def a(self):
pass
d['a'] = a
Foo = d.get('__metaclass__', type)('Foo', (object,), d)
Only without the namespace pollution (and in reality there's also a search through all the bases to determine the metaclass, or whether there's a metaclass conflict, but I'm ignoring that here).
The metaclass' __setattr__ can control what happens when you try to set an attribute on one of its instances (the class object), but inside the class block you're not doing that, you're inserting into a dictionary object, so the dict class controls what's going on, not your metaclass. So you're out of luck.
Unless you're using Python 3.x! In Python 3.x you can define a __prepare__ classmethod (or staticmethod) on a metaclass, which controls what object is used to accumulate attributes set within a class block before they're passed to the metaclass constructor. The default __prepare__ simply returns a normal dictionary, but you could build a custom dict-like class that doesn't allow keys to be redefined, and use that to accumulate your attributes:
from collections import MutableMapping
class SingleAssignDict(MutableMapping):
def __init__(self, *args, **kwargs):
self._d = dict(*args, **kwargs)
def __getitem__(self, key):
return self._d[key]
def __setitem__(self, key, value):
if key in self._d:
raise ValueError(
'Key {!r} already exists in SingleAssignDict'.format(key)
)
else:
self._d[key] = value
def __delitem__(self, key):
del self._d[key]
def __iter__(self):
return iter(self._d)
def __len__(self):
return len(self._d)
def __contains__(self, key):
return key in self._d
def __repr__(self):
return '{}({!r})'.format(type(self).__name__, self._d)
class RedefBlocker(type):
#classmethod
def __prepare__(metacls, name, bases, **kwargs):
return SingleAssignDict()
def __new__(metacls, name, bases, sad):
return super().__new__(metacls, name, bases, dict(sad))
class Okay(metaclass=RedefBlocker):
a = 1
b = 2
class Boom(metaclass=RedefBlocker):
a = 1
b = 2
a = 3
Running this gives me:
Traceback (most recent call last):
File "/tmp/redef.py", line 50, in <module>
class Boom(metaclass=RedefBlocker):
File "/tmp/redef.py", line 53, in Boom
a = 3
File "/tmp/redef.py", line 15, in __setitem__
'Key {!r} already exists in SingleAssignDict'.format(key)
ValueError: Key 'a' already exists in SingleAssignDict
Some notes:
__prepare__ has to be a classmethod or staticmethod, because it's being called before the metaclass' instance (your class) exists.
type still needs its third parameter to be a real dict, so you have to have a __new__ method that converts the SingleAssignDict to a normal one
I could have subclassed dict, which would probably have avoided (2), but I really dislike doing that because of how the non-basic methods like update don't respect your overrides of the basic methods like __setitem__. So I prefer to subclass collections.MutableMapping and wrap a dictionary.
The actual Okay.__dict__ object is a normal dictionary, because it was set by type and type is finicky about the kind of dictionary it wants. This means that overwriting class attributes after class creation does not raise an exception. You can overwrite the __dict__ attribute after the superclass call in __new__ if you want to maintain the no-overwriting forced by the class object's dictionary.
Sadly this technique is unavailable in Python 2.x (I checked). The __prepare__ method isn't invoked, which makes sense as in Python 2.x the metaclass is determined by the __metaclass__ magic attribute rather than a special keyword in the classblock; which means the dict object used to accumulate attributes for the class block already exists by the time the metaclass is known.
Compare Python 2:
class Foo(object):
__metaclass__ = FooMeta
FOO = 123
def a(self):
pass
Being roughly equivalent to:
d = {}
d['__metaclass__'] = FooMeta
d['FOO'] = 123
def a(self):
pass
d['a'] = a
Foo = d.get('__metaclass__', type)('Foo', (object,), d)
Where the metaclass to invoke is determined from the dictionary, versus Python 3:
class Foo(metaclass=FooMeta):
FOO = 123
def a(self):
pass
Being roughly equivalent to:
d = FooMeta.__prepare__('Foo', ())
d['Foo'] = 123
def a(self):
pass
d['a'] = a
Foo = FooMeta('Foo', (), d)
Where the dictionary to use is determined from the metaclass.
There are no assignments happening during the creation of the class. Or: they are happening, but not in the context you think they are. All class attributes are collected from class body scope and passed to metaclass' __new__, as the last argument:
class FooMeta(type):
def __new__(self, name, bases, attrs):
print attrs
return type.__new__(self, name, bases, attrs)
class Foo(object):
__metaclass__ = FooMeta
FOO = 123
Reason: when the code in the class body executes, there's no class yet. Which means there's no opportunity for metaclass to intercept anything yet.
Class attributes are passed to the metaclass as a single dictionary and my hypothesis is that this is used to update the __dict__ attribute of the class all at once, e.g. something like cls.__dict__.update(dct) rather than doing setattr() on each item. More to the point, it's all handled in C-land and simply wasn't written to call a custom __setattr__().
It's easy enough to do whatever you want to the attributes of the class in your metaclass's __init__() method, since you're passed the class namespace as a dict, so just do that.
During the class creation, your namespace is evaluated to a dict and passed as an argument to the metaclass, together with the class name and base classes. Because of that, assigning a class attribute inside the class definition wouldn't work the way you expect. It doesn't create an empty class and assign everything. You also can't have duplicated keys in a dict, so during class creation attributes are already deduplicated. Only by setting an attribute after the class definition you can trigger your custom __setattr__.
Because the namespace is a dict, there's no way for you to check duplicated methods, as suggested by your other question. The only practical way to do that is parsing the source code.