Develop Reference

Identifier for each subclass

I would like to create a metaclass/descriptor in order to have a identifier attribute which counts through each instances. Furthermore I would like to have that the identifier attribute cannot be changed. This is what I have done so far:
class Identifier(object):
"""Iterator for Transformation Identification
"""
def __init__(self, current):
self.current = current
def __iter__(self):
return self
def __next__(self):
self.current += 1
return self.current
def __get__(self, instance, owner):
return self
def __set__(self, instance, value):
raise Exception('')
class InheritDecoratorsMixin:
_iter = Identifier(0)
def __init_subclass__(cls, *args, **kwargs):
setattr(cls, 'id', next(cls._iter))
super().__init_subclass__(*args, **kwargs)
Here we have an example:
class Test1(InheritDecoratorsMixin):
pass
class Test2(Test1):
pass
class Test3(Test1):
pass
class Test4(Test3):
pass
a = Test2()
print(a.id)
b = Test3()
print(b.id)
c = Test4()
print(c.id)
print(c.id)
c.id = 5
print(c.id)
The output is
2
3
4
4
5
This is fine except for the fact that the attribute can be changed. When I changed the code to:
class Identifier(object):
"""Iterator for Transformation Identification
"""
def __init__(self, current):
self.current = current
def __iter__(self):
return self
def __next__(self):
self.current += 1
return self
def __get__(self, instance, owner):
return self
def __set__(self, instance, value):
raise Exception('')
where the next returns 'self' which is basically the iterator then the value cannot be set to '5' but the attribute 'id' is not an integer anymore but an instance of the iterator class. Is there a way to combine both versions, where we both have an integer as id attribute as well as the restriction that the attribute cannot be changed?
Note: in the output there are different values for each subclass but it remains the same for each instance.
Thanks in advance for any help.

You don't actually need a __init_subclass__ method there: the descriptor itself can work to keep track of the instances for each class it is set on.
And then you can assign the descriptor to .id itself:
class Identifier:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
inner_attr = "_" + self.name
id = instance.__dict__.get(inner_attr)
if id is not None:
return id
counter = owner.__dict__.get(inner_attr, 0)
counter += 1
setattr(instance, inner_attr, counter)
setattr(owner, inner_attr, counter)
return counter
def __set__(self, instance, value):
raise AttributeError("read only attribute")
class A:
id = Identifier()
And testing it on an interactive session:
In [35]: a = A()
In [36]: a.id
Out[36]: 1
In [37]: a.id = 3
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [37], in <cell line: 1>()
----> 1 a.id = 3
Input In [34], in Identifier.__set__(self, instance, value)
19 def __set__(self, instance, value):
---> 20 raise AttributeError("read only attribute")
AttributeError: read only attribute
In [38]: class B(A): pass
In [39]: B().id
Out[39]: 1
In [40]: B().id
Out[40]: 2
In [41]: B().id
Out[41]: 3
In [42]: A().id
Out[42]: 2
So, this won't need metaclasses or __init_subclass__ at all.
As for the unerscored attribute _id: it will remain writable, and it is important that you understand that it is impossible, in Python to prevent the inner state of being modifiable. The "public" attribute, ".id" is not changeable, and that is what maytters. It is possible to "hide away" the _id attribute - as an attribute of the descriptor instead of in the class and instances dictionary, for example, but ultimately, if someone 'wants' to change it, there will always be a way to do so. The documentation contract that it is a readonly, public attribute named "id" and whatever else exists is private have to be enough.

You implement a descriptor protocol but do so on the class you are using an infinite counter essentially. That __set__ you write to block setting id will only work if you try to set _iter.
from typing import Any, NoReturn
from typing_extensions import Self
class Identifier:
"""Descriptor for Transformation Identification
"""
def __init__(self, starting_id: int) -> None:
self.current: int = starting_id
def __set_name__(self, owner: type, name: str) -> None:
self.name = name
self.id_map: dict[owner, int] = {}
def __get__(self, instance: Any, owner: type) -> Self|int:
if instance is None:
return self
else:
instance_id = self.id_map.get(instance, None)
if instance_id is None:
self.current += 1
instance_id = self.current
self.id_map[instance] = instance_id
return instance_id
def __set__(self, instance: Any, value: Any) -> NoReturn:
raise Exception(f'Cannot set {self.name}')
class InheritDecoratorsMixin:
id = Identifier(0)
class Test1(InheritDecoratorsMixin):
pass
class Test2(Test1):
pass
class Test3(Test1):
pass
class Test4(Test3):
pass
a = Test2()
print(a.id)
b = Test3()
print(b.id)
b2 = Test3()
print(b2.id)
c = Test4()
print(c.id)
print(c.id)
c.id = 5
And the output in ipython
1
2
3
4
4
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
Input In [4], in <module>
59 print(c.id)
60 print(c.id)
---> 61 c.id = 5
Input In [4], in Identifier.__set__(self, instance, value)
27 def __set__(self, instance: Any, value: Any) -> NoReturn:
---> 28 raise Exception(f"Cannot set {self.name}")
Exception: Cannot set id

Prohibit class member from being set to None in Python

Is there a way in Python to prohibit class members from being set to None outside of __init__?
class Dummy:
def __init__(self, x):
if x is not None:
self.x = x
else:
raise Exception("x cannot be None")
d = Dummy("foo")
d.x = None
In my code I have type hints, but these are not enforced, so saying x can only be str does not really change anything in terms of what's allowed.

You'll need a #property:
class Dummy:
def __init__(self, x):
self.x = x
#property
def x(self):
return self._x
#x.setter
def x(self, value):
if value is None:
raise Exception("x cannot be None")
self._x = value
d = Dummy(8)
d.x = 16
d.x = None # Raises

Note: I have interpreted the question here as meaning preventing all attributes from being set to None outside of __init__, although have added below an option to protect just certain specified attributes.
How about this? Have __setattr__ method as wrapper to object.__setattr__.
class Dummy:
def __init__(self, x):
self._None_forbidden = False
self.z = None # this will work
self._None_forbidden = True
def __setattr__(self, k, v):
if v is None and self._None_forbidden:
raise ValueError("cannot set attribute to None")
object.__setattr__(self, k, v)
d = Dummy("foo")
print(d.z) # None
d.y = 2
print(d.y) # 2
d.x = None # raises ValueError
If you just wanted to protect certain attributes, you could make self._None_forbidden into a set of attribute names that are not allowed to be set to None. For example:
class Dummy:
def __init__(self, x):
self._None_forbidden = set()
self.z = None # this will work
self._None_forbidden.add("z")
def __setattr__(self, k, v):
if v is None and k in self._None_forbidden:
raise ValueError("cannot set attribute to None")
object.__setattr__(self, k, v)
d = Dummy("foo")
print(d.z) # None
d.y = 2
print(d.y) # 2
d.x = None
print(d.x) # None
d.z = 4
print(d.z) # 4
d.z = None # raises ValueError
Obviously if the caller manipulated _None_forbidden, then this can be circumvented, but then they should know that they are doing something unsupported.

Initialize property on first use

I'm curious on what might be a better pattern for creating a property attribute that initialises its value on 1st use. Below is a class with several variations on a theme.
def some_initializer(s):
return f"Value: {s}"
class Foo(object):
""" Initialize on first use properties """
def __init__(self):
self._prop1 = None
#property
def prop1(self):
""" Existing private member attribute """
if not self._prop1:
self._prop1 = some_initializer("prop1")
return self._prop1
#property
def prop2(self):
""" Create private attribute on demand """
if not hasattr(self, "_prop2"):
self._prop2 = some_initializer("prop2")
return self._prop2
#property
def prop3(self):
""" Create private attribute on demand - shorter """
self._prop3 = getattr(self, "_prop3", some_initializer("prop3"))
return self._prop3
#property
def prop4(self):
""" Stash value in attribute with same name as property """
_prop4 = self.__dict__.get('_prop4')
if _prop4 is not None:
return _prop4
self._prop4 = _prop4 = some_initializer("prop4")
return _prop4
>> f = Foo()
>> print(f.prop1)
>> print(f.prop2)
>> print(f.prop3)
>> print(f.prop4)
Value: prop1
Value: prop2
Value: prop3
Value: prop4
In the past I've used variations prop1, prop2 and prop3. Recently I was introduced to the prop4 variation that I feel is quite confusing although perhaps technically correct. Any cons with these variations or maybe there are better ways?
Edit: Ideally, it would be nice to maintain compatibility with property setter and deleter decorators too.

i would just write a custom descriptor and use that instead:
class cached_property:
def __init__(self, f):
self.f = f
def __get__(self, instance, owner):
if not instance:
return self
res = instance.__dict__[self.f.__name__] = self.f(instance)
return res
usage:
class C:
#cached_property
def prop(self):
print('you will see me once')
return 4

You could use functools.lru_cache to memoize the property value:
from functools import lru_cache
class Foo(object):
#property
#lru_cache()
def prop(self):
print("called once")
return 42
foo = Foo()
print(foo.prop)
print(foo.prop)

I also thought of a descriptor but came up with this approach
from weakref import WeakKeyDictionary
def five():
return 5
def six():
return 6
def seven():
return 7
class FirstInit:
def __init__(self, initializer):
self.initializer = initializer
self.data = WeakKeyDictionary()
def __get__(self, instance, owner):
try:
value = self.data[instance]
except KeyError as e:
value = self.initializer()
self.data[instance] = value
return self.data[instance]
Usage:
class F:
a = FirstInit(five)
b = FirstInit(six)
c = FirstInit(seven)
def __init__(self,name):
self.name = f'{name}:{self.c}'
>>> f = F('foo')
>>> f.name
'foo:7'
>>> f.a, f.b
(5, 6)
>>> f.a = 'sixteen'
>>> f.a, f.b
('sixteen', 6)
>>> f.b += 13
>>> f.a, f.b
('sixteen', 19)
>>>
For an initializer that takes an argument:
d = {'P1':5, 'P2':6, 'P3':7}
def initializer(which):
return d[which]
class FirstInit:
def __init__(self, initializer, prop):
self.initializer = initializer
self.prop = prop
self.data = WeakKeyDictionary()
def __get__(self, instance, owner):
try:
value = self.data[instance]
except KeyError as e:
value = self.initializer(self.prop)
self.data[instance] = value
return self.data[instance]
class G:
a = FirstInit(initializer, 'P1')
b = FirstInit(initializer, 'P2')
c = FirstInit(initializer, 'P3')
def __init__(self,name):
self.name = f'{name}:{self.c}'
...
>>> g = G('foo')
>>> g.name
'foo:7'
>>> g.b += 16
>>> g.a,g.b
(5, 22)
>>> g.a = 'four'
>>> g.a,g.b
('four', 22)
>>>

How to avoid creating objects with same values?

I need to create a class whose instances can't have same values. If you create instance with value that have already been used you'll get old same instance.
I did it using special class method:
class A():
instances = []
def __init__(self, val):
self.val = val
#classmethod
def new(cls, val):
"""
Return instance with same value or create new.
"""
for ins in cls.instances:
if ins.val == val:
return ins
new_ins = A(val)
cls.instances.append(new_ins)
return new_ins
a1 = A.new("x")
a2 = A.new("x")
a3 = A.new("y")
print a1 # <__main__.A instance at 0x05B7FD00> S\ /M\
print a2 # <__main__.A instance at 0x05B7FD00> \A/ \E
print a3 # <__main__.A instance at 0x05B7FD28>
Is there a way to do it more elegant, without using .new method?

You could try functools.lru_cache.
For example:
from functools import lru_cache
class A:
#lru_cache()
def __new__(cls, arg):
return super().__new__(cls)
def __init__(self, arg):
self.n = arg
Sample usage:
>>> a1 = A('1')
>>> a2 = A('1')
>>> a1 is a2
True
>>> a1.n
'1'
>>> a2.n
'1'
Alternatively you could try building a custom caching class, as pointed out by Raymond Hettinger in this tweet: https://twitter.com/raymondh/status/977613745634471937.

This can be done by overriding the __new__ method, which is responsible for creating new instances of a class. Whenever you create a new instance you store it in a dict, and if the dict contains a matching instance then you return it instead of creating a new one:
class A:
instances = {}
def __new__(cls, val):
try:
return cls.instances[val]
except KeyError:
pass
obj = super().__new__(cls)
cls.instances[val] = obj
return obj
def __init__(self, val):
self.val = val
a = A(1)
b = A(2)
c = A(1)
print(a is b) # False
print(a is c) # True
One downside of this solution is that the __init__ method will be called regardless of whether the instance is a newly created one or one that's been stored in the dict. This can cause problems if your constructor has undesired side effects:
class A:
...
def __init__(self, val):
self.val = val
self.foo = 'foo'
a = A(1)
a.foo = 'bar'
b = A(1)
print(a.foo) # output: foo
Notice how a's foo attribute changed from "bar" to "foo" when b was created.
Another option is to use a metaclass and override its __call__ method:
class MemoMeta(type):
def __new__(mcs, name, bases, attrs):
cls = super().__new__(mcs, name, bases, attrs)
cls.instances = {}
return cls
def __call__(cls, val):
try:
return cls.instances[val]
except KeyError:
pass
obj = super().__call__(val)
cls.instances[val] = obj
return obj
class A(metaclass=MemoMeta):
def __init__(self, val):
self.val = val
self.foo = 'foo'
This bypasses the problem with __init__ being called on existing instances:
a = A(1)
a.foo = 'bar'
b = A(1)
print(a.foo) # output: bar

If you really want to make it more elegant, implement the duplicate check in __new__, so it will be performed when you call A(something).
Just do it in __new__:
def __new__(cls, val=None):
for i in cls.instances:
if val == i.val:
return i
return object.__new__(cls)

Descriptor class design in Python (with inheritance)

I'm trying to design a descriptor class which I can use through other class which is a subclass of a class which is a subclass of a class.
class MyDescriptorClass(object):
def __init__(self, owner, name, activates = 0):
self.value = None
self.name = name
self.owner = owner
self.activates = 0
self.connects = []
def __set__(self, obj, val):
self.set(val)
def __get__(self, instance, owner):
return self.value
def set(self, value):
if self.value == value:
return 0
self.value = value
if self.activates:
self.owner.evaluate()
def connect(self, inputs):
if not isinstance(inputs, list):
inputs = list(inputs)
for input in inputs:
self.connects.append(input)
class ParentClass(object):
def __init__(self, name):
self.states = {}
self.name = name
self.A = MyDescriptorClass(self, name, activates = 1)
self.B = MyDescriptorClass(self, name, activates = 1)
self.states.setDefault('A', self.A)
self.states.setDefault('B', self.B)
class ChildClass1(ParentClass):
def __init__(self, name)
super(ChildClass1, self).__init__(name)
self.ans = None
def evaluate(self):
self.ans = self.A.value + self.B.value
class ChildClass2(ParentClass):
def __init__(self, name)
super(ChildClass1, self).__init__(name)
self.ans = None
def evaluate(self):
self.ans = self.A.value * self.B.value
self.A = MyDescriptorClass() will not work according to the python docs
so the only way is that I declate A = MyDescriptorClass() in the ParentClass as
class ParentClass(object):
A = MyDescriptorClass() # here I am unable to pass the owner
And since, I'm using a child class, super call skips this part and starts directly with __init__
Is there any way in which I can modify the design so as to set the value of ChildClass1.A instance directly?
c = ChildClass1("c1")
c.A = 10 # I directly want to set this value instead of using c.A.set(10)
c.B = 20
c.evaluate()
print c.ans # 30
c.B = 40
print c.ans # 50

Try not to put information which is specific to instances in the descriptor. Keep information specific to instances in instance attributes, and keep information specific to the descriptor (like activates) in the descriptor:
class MyDescriptorClass(object):
def __init__(self, activates = 0):
self.value = None
self.activates = activates
self.connects = []
def __set__(self, instance, val): # 1
if self.value == val:
return 0
self.value = val
if self.activates:
instance.evaluate()
def __get__(self, instance, instcls): # 1
return self.value
Note that the __set__ and __get__ methods are passed the
instance which is accessing the descriptor. Therefore, you do not
need to store the owner in MyDescriptor. The instance is the
owner.
Given the clarification of the problem in the comments below, here is how I would implement the descriptor.
class GateInput(object):
def __init__(self, index):
self.index = index # 4
def __get__(self, inst, instcls):
return inst.inputs[self.index].ans # 5
def __set__(self, inst, val):
if isinstance(val, (float, int)):
inst.inputs[self.index] = Constant(val)
else:
inst.inputs[self.index] = val
class Constant(object):
def __init__(self, val):
self.ans = val
class Gate(object):
A = GateInput(0) # 1
B = GateInput(1) # 1
def __init__(self, name):
self.name = name
self.inputs = [Constant(0), Constant(0)] # 2
class Adder(Gate):
#property
def ans(self):
result = 0
for gate in self.inputs:
result += gate.ans # 3
return result
class Multiplier(Gate):
#property
def ans(self):
result = 1
for gate in self.inputs:
result *= gate.ans
return result
b = Multiplier('b1')
b.A = 2
b.B = 3
print(b.A)
# 2
print(b.ans)
# 6
c = Adder('c1')
c.A = 10
print(c.ans)
# 10
# This connects output of b to an input of c
c.B = b
print(c.ans)
# 16
Descriptors have to be defined as class attributes, not instance
attributes. Since the descriptor is accessed by all instances, you
probably do not want the descriptor to change merely because an
instance is being created. Therefore, do not instantiate the
descriptor in __init__.
Each instance of Gate has a list of inputs. The items self.inputs
are instances of Constant or Gate.
Here we see the purpose of the Constant class. For every gate,
gate.ans needs to return a value.
The index records which item in inst.inputs the GateInput is
connected to.
inst is an instance of Gate. For example, c.A causes Python to
call GateInput.__get__(self, c, type(c)). Thus, inst is c
here.

As it is int he comments:
descriptors must be class attributes, not instance attributes in order to work -
so, to start with:
class ParentClass(object):
A = MyDescriptorClass()
B = MyDescriptorClass()
def __init__(self, name):
self.states = {}
self.name = name
self.A.configure(self, name, activates = 1)
self.B.configure(self, name, activates = 1)
self.states.setDefault('A', self.A)
self.states.setDefault('B', self.B)
And then you fix your Descriptor class accordingly:
either have then keeping all data refering to an instance in the instance itself
(that is why __get__ and __set__ receive the object itself) - or have
each descriptor instance have a dictionary where they can annotate data related
to the instances of the class they belong too, by, for example, object ID.
Your descriptor class could be more or less along these lines:
class MyDescriptorClass(object):
def __init__(self):
self.data = defaultDict(dict)
def configure(self, owner, name, activates = 0):
container = self.data(id(owner))
container["value"] = None
container["name"] = name
...
def __set__(self, owner, value):
# implemnt your previous "set" method straight here
...
...