I wonder if there is a way to get the class name automatically upon defining class attribute before initialization
class MyClass(object):
attribute1 = 1
attribute2 = 2 # This a simple example, MyClass has many other attributes
print className # normally one would use self.__class__.__name__ but self or cls are not defined at the level
def __init__(self):
a = 1
Purpose
In the framework I am working in, attribute1 and attribute2 are object instances (you will say everything is object oriented in python :) ) I want to set the class name to those attributes prior to MyClass initialization. MyClass get initialized a lot of times and it has much more than 2 attributes which makes the operation very time consuming to do at every initialization
A metaclass can be used to set an attribute with the class name before the class initialises:
class MyClassMeta(type):
def __init__(self, name, bases, attrs):
super(MyClassMeta, self).__init__(name, bases, attrs)
# Set class_name to the name of our class; in this case "MyClass"
self.class_name = name
class MyClass(object):
__metaclass__ = MyClassMeta
def __init__(self):
# Prints "MyClass"
print(self.class_name)
See this answer for a detailed explanation of metaclasses.
Related
This may have been answered somewhere else, but I was wondering if there was any way to remove an attribute/method decorated with #property in a subclass.
Example:
from datetime import datetime
class A():
def __init__(self, num):
self._num = num
#property
def id(self):
return self._num * datetime.now().timestamp()
class B(A):
def __init__(self, id, num):
super().__init__(num)
self.id = id
The above code does not run if you attempt to create an instance of class B. AttributeError: can't set attribute
The base class uses a property because it needs to evaluate its ID on the fly, while my sub class is able to know its ID when it is created. The id attribute is accessed OFTEN, and I am seeing a significant performance hit because I have to use a property to serve this attribute, instead of just accessing it directly. (From what I have read, properties increase time-to-access by 5x). My application is currently spending around 10% of runtime getting this property.
Is there any way I can short-circuit the property in a sub class?
I'm going to go through several possibilities here. Some of them do what you literally asked. Some of them don't, but they may be better options anyway.
First, your example base class changes the value of obj.id on every access due to the passage of time. That's really bizarre and doesn't seem like a useful concept of "ID". If your real use case has a stable obj.id return value, then you can cache it to avoid the expense of recomputation:
def __init__(self):
...
self._id = None
#property
def id(self):
if self._id is not None:
return self._id
retval = self._id = expensive_computation()
return retval
This may mitigate the expense of the property. If you need more mitigation, look for places where you access id repeatedly, and instead, access it once and save it in a variable. Local variable lookup outperforms attribute access no matter how the attribute is implemented. (Of course, if you actually do have weird time-variant IDs, then this sort of refactoring may not be valid.)
Second, you can't override a property with a "regular" attribute, but you can create your own version of property that can be overridden this way. Your property blocks attribute setting, and takes priority over "regular" attributes even if you force an entry into the instance __dict__, because property has a __set__ method (even if you don't write a setter). Writing your own descriptor without a __set__ would allow overriding. You could do it with a generic LowPriorityProperty:
class LowPriorityProperty(object):
"""
Like #property, but no __set__ or __delete__, and does not take priority
over the instance __dict__.
"""
def __init__(self, fget):
self.fget = fget
def __get__(self, instance, owner=None):
if instance is None:
return self
return self.fget(instance)
class Foo(object):
...
#LowPriorityProperty
def id(self):
...
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...
Or with a role-specific descriptor class:
class IDDescriptor(object):
def __get__(self, instance, owner=None):
if instance is None:
return self
# Remember, self is the descriptor. instance is the object you're
# trying to compute the id attribute of.
return whatever(instance)
class Foo(object):
id = IDDescriptor()
...
class Bar(Foo):
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...
The role-specific descriptor performs better than the generic LowPriorityProperty, but both perform worse than property due to implementing more logic in Python instead of C.
Finally, you can't override a property with a "regular" attribute, but you can override it with another descriptor, such as another property, or such as the descriptors created for __slots__. If you're really, really pressed for performance, __slots__ is probably more performant than any descriptor you could implement manually, but the interaction between __slots__ and the property is weird and obscure and you'll probably want to leave a comment explaining what you're doing.
class Foo(object):
#property
def id(self):
...
class Bar(Foo):
__slots__ = ('id',)
def __init__(self):
super(Bar, self).__init__()
self.id = whatever
...
add a class C as common ancestor, without id. inherit A and B from it and implement id there as needed. Python wont care that id doesn’t exist on C.
refactor non-id code/attributes from A to C.
Suitability depends on whether OP controls class hierarchy and instantiation mechanisms.
I also found a workaround to get it working as is:
from datetime import datetime
class A():
def __init__(self, num):
self._num = num
#property
def id(self):
return self._num * datetime.now().timestamp()
class B(A):
#this fixes the problem
id = None
def __init__(self, id, num):
super().__init__(num)
self.id = id
b = B("id", 3)
print(vars(b))
This will output:
{'_num': 3, 'id': 'id'}
The trick is id = None on class B. Basically, Python's attribute/method lookup mechanism will stop at the first class with id as an attribute in the MRO. With id = None on class B, the lookup stops there and it never gets as far as that pesky #property on A.
If I comment it back out, as per the OP:
self.id = id
AttributeError: can't set attribute
I have spent a lot of time researching this, but none of the answers seem to work how I would like.
I have an abstract class with a class attribute I want each subclass to be forced to implement
class AbstractFoo():
forceThis = 0
So that when I do this
class RealFoo(AbstractFoo):
pass
it throws an error telling me it can't create the class until I implement forceThis.
How can I do that?
(I don't want the attribute to be read-only, but if that's the only solution, I'll accept it.)
For a class method, I've discovered I can do
from abc import ABCMeta, abstractmethod
class AbstractFoo(metaclass=ABCMeta):
#classmethod
#abstractmethod
def forceThis():
"""This must be implemented"""
so that
class RealFoo(AbstractFoo):
pass
at least throws the error TypeError: Can't instantiate abstract class EZ with abstract methods forceThis
(Although it doesn't force forceThis to be a class method.)
How can I get a similar error to pop up for the class attribute?
You can do this by defining your own metaclass. Something like:
class ForceMeta(type):
required = ['foo', 'bar']
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
for prop in mcls.required:
if not hasattr(cls, prop):
raise NotImplementedError('must define {}'.format(prop))
return cls
Now you can use this as the metaclass of your own classes:
class RequiredClass(metaclass=ForceMeta):
foo = 1
which will raise the error 'must define bar'.
I came up with a solution based on those posted earlier. (Thank you #Daniel Roseman and #martineau)
I created a metaclass called ABCAMeta (the last 'A' stands for 'Attributes').
The class has two ways of working.
A class which just uses ABCAMeta as a metaclass must have a property called required_attributes which should contain a list of the names of all the attributes you want to require on future subclasses of that class
A class whose parent's metaclass is ABCAMeta must have all the required attributes specified by its parent class(es).
For example:
class AbstractFoo(metaclass=ABCAMeta):
required_attributes = ['force_this']
class RealFoo(AbstractFoo):
pass
will throw an error:
NameError: Class 'RealFoo' has not implemented the following attributes: 'force_this'
Exactly how I wanted.
from abc import ABCMeta
class NoRequirements(RuntimeError):
def __init__(self, message):
RuntimeError.__init__(self, message)
class ABCAMeta(ABCMeta):
def __init__(mcls, name, bases, namespace):
ABCMeta.__init__(mcls, name, bases, namespace)
def __new__(mcls, name, bases, namespace):
def get_requirements(c):
"""c is a class that should have a 'required_attributes' attribute
this function will get that list of required attributes or
raise a NoRequirements error if it doesn't find one.
"""
if hasattr(c, 'required_attributes'):
return c.required_attributes
else:
raise NoRequirements(f"Class '{c.__name__}' has no 'required_attributes' property")
cls = super().__new__(mcls, name, bases, namespace)
# true if no parents of the class being created have ABCAMeta as their metaclass
basic_metaclass = True
# list of attributes the class being created must implement
# should stay empty if basic_metaclass stays True
reqs = []
for parent in bases:
parent_meta = type(parent)
if parent_meta==ABCAMeta:
# the class being created has a parent whose metaclass is ABCAMeta
# the class being created must contain the requirements of the parent class
basic_metaclass=False
try:
reqs.extend(get_requirements(parent))
except NoRequirements:
raise
# will force subclasses of the created class to define
# the attributes listed in the required_attributes attribute of the created class
if basic_metaclass:
get_requirements(cls) # just want it to raise an error if it doesn't have the attributes
else:
missingreqs = []
for req in reqs:
if not hasattr(cls, req):
missingreqs.append(req)
if len(missingreqs)!=0:
raise NameError(f"Class '{cls.__name__}' has not implemented the following attributes: {str(missingreqs)[1:-1]}")
return cls
Any suggestions for improvement are welcome in the comments.
Although you can do something very similar with a metaclass, as illustrated in #Daniel Roseman's answer, it can also be done with a class decorator. A couple of advantages they have are that errors will occur when the class is defined, instead of when an instance of one is created, and the syntax for specifying them is the same in both Python 2 and 3. Some folks also find them simpler and easier to understand.
def check_reqs(cls):
requirements = 'must_have',
missing = [req for req in requirements if not hasattr(cls, req)]
if missing:
raise NotImplementedError(
'class {} did not define required attribute{} named {}'.format(
cls.__name__, 's' if len(missing) > 1 else '',
', '.join('"{}"'.format(name) for name in missing)))
return cls
#check_reqs
class Foo(object): # OK
must_have = 42
#check_reqs
class Bar(object): # raises a NotImplementedError
pass
I want to enforce a Child class to set several attributes(instance variables), to achieve I am using abstractproperty decorator in python 2.7. Here is the sample code:
from abc import ABCMeta, abstractproperty
class Parent(object):
__metaclass__ = ABCMeta
#abstractproperty
def name(self):
pass
class Child1(Parent):
pass
class Child2(Parent):
name = 'test'
class Child3(Parent):
def __init__(self):
self.name = 'test'
obj_child1 = Child1()
Child1 object creation gives an error as expected.
obj_child2 = Child2()
Child2 object creation works fine as because abstract attribute 'name' is set
However
obj_child3 = Child3()
gives TypeError: Can't instantiate abstract class Child3 with abstract methods name
I did not understand:although the attribute 'name' is set in the init
method, why the Child3 object creation is throwing type error.
My requirement is set attributes inside a method of child class. An explanation of what is wrong with child2 class and if there is a better way to enforce a child class to set attributes in a method will help me. Thanks in advance.
A simpler solution will be
class Parent(object):
name = None
def __init__(self):
if self.name == None:
raise NotImplementedError('Subclasses must define bar')
class Child1(Parent):
pass
class Child2(Parent):
name = 'test'
class Child3(Parent):
def __init__(self)
self.name = 'test'
obj1 = Child1() raises NotImplementedError
obj2 = Child2() works fine
obj3 = Child3() works fine. This is what you need to enforce a child class to set attribute name and set the attribute from a method.
Here's what I believe is happening.
When you try to instantiate Child1, you have not provided an implementation for the abstract property name. You may have done so like this:
class Child1(Parent):
#property
def name(self):
return "foo"
Since the class does not provide an implementation for all abstract methods inherited from its parents, it is effectively an abstract class itself, and therefore cannot be instantiated. Trying to instantiate it gives you your TypeError.
In the case of Child2, you define it as:
class Child2(Parent):
name = 'test'
In this case, you're overwriting the name property of the class so that it's no longer referencing an abstract property that needs to be implemented. This means that Child2 is not abstract and can be instantiated.
One difference I noticed was that when name = 'test' as you've implemented it, vars(Child2) returns output like this:
{..., '__abstractmethods__': frozenset(), ..., 'name': 'test'}
However, when you change this to something like foo = 'test', you get this:
{..., '__abstractmethods__': frozenset({'name'}), ..., 'foo': 'test'}
Notice that the __abstractmethods__ property is an empty set in the case that you define name = 'test' i.e. when Child2 can be instantiated.
Finally, in the case of Child3 you have to bear in mind that the abstract property is stored as a property of the class itself, which you don't redefine anywhere.
As soon as you try to create an instance of it, the interpreter will see that it's missing an implementation for at least one abstract method and throw the TypeError you see. It doesn't even reach the assignment self.name = 'test' in the constructor.
To answer your second part about how to enforce children to always provide an implementation for abstract properties/methods when they can do something like you did - I'm not actually sure if it's possible. Python is a language of consenting adults.
You can do some like this,
parent class:
class Parent(object):
__metaclass__ = ABCMeta
#abstractproperty
def name(self):
pass
Child class:
class Child(Parent):
name = None
def __init__(self):
self.name = 'test'
Now
obj = Child()
obj.name
gives the required output of 'test'.
By doing this you can enforce a class to set an attribute of parent class and in child class you can set them from a method.
It is not a perfect solution that you require, but it is close. The only problem with this solution is you will need to define all the abstractproperty of parent as None in child class and them set them using a method.
I want to add an attribute for every class created by a metaclass. For example, when a class named C is created, I want add an attribute C._C__sup whose value is the descriptor super(C).
Here is what I've tried:
class Meta(type):
def __init__(cls, name, bases, dict): # Not overriding type.__new__
cls.__dict__['_' + name + '__sup'] = super(cls)
# Not calling type.__init__; do I need it?
class C(object):
__metaclass__ = Meta
c = C()
print c._C__sup
This gives me:
TypeError: Error when calling the metaclass bases
'dictproxy' object does not support item assignment
Some background information:
(You don't have to read this part)
Inspired by this article, what I'm doing is trying to avoid "hardcoding" the class name when using super:
The idea there is to use the unbound super objects as private
attributes. For instance, in our example, we could define the private
attribute __sup in the class C as the unbound super object
super(C):
>>> C._C__sup = super(C)
With this definition inside the methods the syntax self.__sup.meth
can be used as an alternative to super(C, self).meth. The advantage
is that you avoid to repeat the name of the class in the calling
syntax, since that name is hidden in the mangling mechanism of private
names. The creation of the __sup attributes can be hidden in a
metaclass and made automatic. So, all this seems to work: but
actually this not the case.
Use setattr instead of assignment to cls.__dict__:
class Meta(type):
def __init__(cls, name, bases, clsdict): # Not overriding type.__new__
setattr(cls, '_' + name + '__sup', super(cls))
super(Meta, cls).__init__(name, bases, clsdict)
class C(object):
__metaclass__ = Meta
def say(self):
return 'wow'
class D(C):
def say(self):
return 'bow' + self.__sup.say()
c = C()
print(c._C__sup)
# <super: <class 'C'>, <C object>>
d = D()
print(d.say())
prints
bowwow
By the way, it is a good idea to call
super(Meta, cls).__init__(name, bases, clsdict)
inside Meta.__init__ to allow Meta to participate in class hierarchies which
might need super to properly call a chain of __init__s. This seems
particularly appropriate since you are building a metaclass to assist with the
use of super.
If you have multiple layers of inheritance and know that a particular variable exists, is there a way to trace back to where the variable originated? Without having to navigate backwards by looking through each file and classes. Possibly calling some sort of function that will do it?
Example:
parent.py
class parent(object):
def __init__(self):
findMe = "Here I am!"
child.py
from parent import parent
class child(parent):
pass
grandson.py
from child import child
class grandson(child):
def printVar(self):
print self.findMe
Try to locate where the findMe variable came from with a function call.
If the "variable" is an instance variable - , so , if at any point in chain of __init__ methods you do:
def __init__(self):
self.findMe = "Here I am!"
It is an instance variable from that point on, and cannot, for all effects, be made distinct of any other instance variable. (Unless you put in place a mechanism, like a class with a special __setattr__ method, that will keep track of attributes changing, and introspect back which part of the code set the attribute - see last example on this answer)
Please also note that on your example,
class parent(object):
def __init__(self):
findMe = "Here I am!"
findMe is defined as a local variable to that method and does not even exist after __init__ is finished.
Now, if your variable is set as a class attribute somewhere on the inheritance chain:
class parent(object):
findMe = False
class childone(parent):
...
It is possible to find the class where findMe is defined by introspecting each class' __dict__ in the MRO (method resolution order) chain . Of course, there is no way, and no sense, in doing that without introspecting all classes in the MRO chain - except if one keeps track of attributes as defined, like in the example bellow this - but introspecting the MRO itself is a oneliner in Python:
def __init__(self):
super().__init__()
...
findme_definer = [cls for cls in self.__class__.__mro__ if "findMe" in cls.__dict__][0]
Again - it would be possible to have a metaclass to your inheritance chain which would keep track of all defined attributes in the inheritance tree, and use a dictionary to retrieve where each attribute is defined. The same metaclass could also auto-decorate all __init__ (or all methods), and set a special __setitem__ so that it could track instance attributes as they are created, as listed above.
That can be done, is a bit complicated, would be hard to maintain, and probably is a signal you are taking the wrong approach to your problem.
So, the metaclass to record just class attributes could simply be (python3 syntax - define a __metaclass__ attribute on the class body if you are still using Python 2.7):
class MetaBase(type):
definitions = {}
def __init__(cls, name, bases, dct):
for attr in dct.keys():
cls.__class__.definitions[attr] = cls
class parent(metaclass=MetaBase):
findMe = 5
def __init__(self):
print(self.__class__.definitions["findMe"])
Now, if one wants to find which of the superclasses defined an attribute of the currentclass, just a "live" tracking mechanism, wrapping each method in each class can work - it is a lot trickier.
I've made it - even if you won't need this much, this combines both methods - keeping track of class attributes in the class'class definitions and on an instance _definitions dictionary - since in each created instance an arbitrary method might have been the last to set a particular instance attribute: (This is pure Python3, and maybe not that straighforward porting to Python2 due to the "unbound method" that Python2 uses, and is a simple function in Python3)
from threading import current_thread
from functools import wraps
from types import MethodType
from collections import defaultdict
def method_decorator(func, cls):
#wraps(func)
def wrapper(self, *args, **kw):
self.__class__.__class__.current_running_class[current_thread()].append(cls)
result = MethodType(func, self)(*args, **kw)
self.__class__.__class__.current_running_class[current_thread()].pop()
return result
return wrapper
class MetaBase(type):
definitions = {}
current_running_class = defaultdict(list)
def __init__(cls, name, bases, dct):
for attrname, attr in dct.items():
cls.__class__.definitions[attr] = cls
if callable(attr) and attrname != "__setattr__":
setattr(cls, attrname, method_decorator(attr, cls))
class Base(object, metaclass=MetaBase):
def __setattr__(self, attr, value):
if not hasattr(self, "_definitions"):
super().__setattr__("_definitions", {})
self._definitions[attr] = self.__class__.current_running_class[current_thread()][-1]
return super().__setattr__(attr,value)
Example Classes for the code above:
class Parent(Base):
def __init__(self):
super().__init__()
self.findMe = 10
class Child1(Parent):
def __init__(self):
super().__init__()
self.findMe1 = 20
class Child2(Parent):
def __init__(self):
super().__init__()
self.findMe2 = 30
class GrandChild(Child1, Child2):
def __init__(self):
super().__init__()
def findall(self):
for attr in "findMe findMe1 findMe2".split():
print("Attr '{}' defined in class '{}' ".format(attr, self._definitions[attr].__name__))
And on the console one will get this result:
In [87]: g = GrandChild()
In [88]: g.findall()
Attr 'findMe' defined in class 'Parent'
Attr 'findMe1' defined in class 'Child1'
Attr 'findMe2' defined in class 'Child2'