Develop Reference

Using #classmethod with #property [duplicate]

This question already has answers here:
Using property() on classmethods
(19 answers)
Closed 3 years ago.
In python I can add a method to a class with the #classmethod decorator. Is there a similar decorator to add a property to a class? I can better show what I'm talking about.
class Example(object):
the_I = 10
def __init__( self ):
self.an_i = 20
#property
def i( self ):
return self.an_i
def inc_i( self ):
self.an_i += 1
# is this even possible?
#classproperty
def I( cls ):
return cls.the_I
#classmethod
def inc_I( cls ):
cls.the_I += 1
e = Example()
assert e.i == 20
e.inc_i()
assert e.i == 21
assert Example.I == 10
Example.inc_I()
assert Example.I == 11
Is the syntax I've used above possible or would it require something more?
The reason I want class properties is so I can lazy load class attributes, which seems reasonable enough.

Here's how I would do this:
class ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.fget.__get__(obj, klass)()
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
type_ = type(obj)
return self.fset.__get__(obj, type_)(value)
def setter(self, func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
self.fset = func
return self
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class Bar(object):
_bar = 1
#classproperty
def bar(cls):
return cls._bar
#bar.setter
def bar(cls, value):
cls._bar = value
# test instance instantiation
foo = Bar()
assert foo.bar == 1
baz = Bar()
assert baz.bar == 1
# test static variable
baz.bar = 5
assert foo.bar == 5
# test setting variable on the class
Bar.bar = 50
assert baz.bar == 50
assert foo.bar == 50
The setter didn't work at the time we call Bar.bar, because we are calling
TypeOfBar.bar.__set__, which is not Bar.bar.__set__.
Adding a metaclass definition solves this:
class ClassPropertyMetaClass(type):
def __setattr__(self, key, value):
if key in self.__dict__:
obj = self.__dict__.get(key)
if obj and type(obj) is ClassPropertyDescriptor:
return obj.__set__(self, value)
return super(ClassPropertyMetaClass, self).__setattr__(key, value)
# and update class define:
# class Bar(object):
# __metaclass__ = ClassPropertyMetaClass
# _bar = 1
# and update ClassPropertyDescriptor.__set__
# def __set__(self, obj, value):
# if not self.fset:
# raise AttributeError("can't set attribute")
# if inspect.isclass(obj):
# type_ = obj
# obj = None
# else:
# type_ = type(obj)
# return self.fset.__get__(obj, type_)(value)
Now all will be fine.

If you define classproperty as follows, then your example works exactly as you requested.
class classproperty(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, owner):
return self.f(owner)
The caveat is that you can't use this for writable properties. While e.I = 20 will raise an AttributeError, Example.I = 20 will overwrite the property object itself.

[answer written based on python 3.4; the metaclass syntax differs in 2 but I think the technique will still work]
You can do this with a metaclass...mostly. Dappawit's almost works, but I think it has a flaw:
class MetaFoo(type):
#property
def thingy(cls):
return cls._thingy
class Foo(object, metaclass=MetaFoo):
_thingy = 23
This gets you a classproperty on Foo, but there's a problem...
print("Foo.thingy is {}".format(Foo.thingy))
# Foo.thingy is 23
# Yay, the classmethod-property is working as intended!
foo = Foo()
if hasattr(foo, "thingy"):
print("Foo().thingy is {}".format(foo.thingy))
else:
print("Foo instance has no attribute 'thingy'")
# Foo instance has no attribute 'thingy'
# Wha....?
What the hell is going on here? Why can't I reach the class property from an instance?
I was beating my head on this for quite a while before finding what I believe is the answer. Python #properties are a subset of descriptors, and, from the descriptor documentation (emphasis mine):
The default behavior for attribute access is to get, set, or delete the
attribute from an object’s dictionary. For instance, a.x has a lookup chain
starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing
through the base classes of type(a) excluding metaclasses.
So the method resolution order doesn't include our class properties (or anything else defined in the metaclass). It is possible to make a subclass of the built-in property decorator that behaves differently, but (citation needed) I've gotten the impression googling that the developers had a good reason (which I do not understand) for doing it that way.
That doesn't mean we're out of luck; we can access the properties on the class itself just fine...and we can get the class from type(self) within the instance, which we can use to make #property dispatchers:
class Foo(object, metaclass=MetaFoo):
_thingy = 23
#property
def thingy(self):
return type(self).thingy
Now Foo().thingy works as intended for both the class and the instances! It will also continue to do the right thing if a derived class replaces its underlying _thingy (which is the use case that got me on this hunt originally).
This isn't 100% satisfying to me -- having to do setup in both the metaclass and object class feels like it violates the DRY principle. But the latter is just a one-line dispatcher; I'm mostly okay with it existing, and you could probably compact it down to a lambda or something if you really wanted.

If you use Django, it has a built in #classproperty decorator.
from django.utils.decorators import classproperty
For Django 4, use:
from django.utils.functional import classproperty

I think you may be able to do this with the metaclass. Since the metaclass can be like a class for the class (if that makes sense). I know you can assign a __call__() method to the metaclass to override calling the class, MyClass(). I wonder if using the property decorator on the metaclass operates similarly.
Wow, it works:
class MetaClass(type):
def getfoo(self):
return self._foo
foo = property(getfoo)
#property
def bar(self):
return self._bar
class MyClass(object):
__metaclass__ = MetaClass
_foo = 'abc'
_bar = 'def'
print MyClass.foo
print MyClass.bar
Note: This is in Python 2.7. Python 3+ uses a different technique to declare a metaclass. Use: class MyClass(metaclass=MetaClass):, remove __metaclass__, and the rest is the same.

As far as I can tell, there is no way to write a setter for a class property without creating a new metaclass.
I have found that the following method works. Define a metaclass with all of the class properties and setters you want. IE, I wanted a class with a title property with a setter. Here's what I wrote:
class TitleMeta(type):
#property
def title(self):
return getattr(self, '_title', 'Default Title')
#title.setter
def title(self, title):
self._title = title
# Do whatever else you want when the title is set...
Now make the actual class you want as normal, except have it use the metaclass you created above.
# Python 2 style:
class ClassWithTitle(object):
__metaclass__ = TitleMeta
# The rest of your class definition...
# Python 3 style:
class ClassWithTitle(object, metaclass = TitleMeta):
# Your class definition...
It's a bit weird to define this metaclass as we did above if we'll only ever use it on the single class. In that case, if you're using the Python 2 style, you can actually define the metaclass inside the class body. That way it's not defined in the module scope.

def _create_type(meta, name, attrs):
type_name = f'{name}Type'
type_attrs = {}
for k, v in attrs.items():
if type(v) is _ClassPropertyDescriptor:
type_attrs[k] = v
return type(type_name, (meta,), type_attrs)
class ClassPropertyType(type):
def __new__(meta, name, bases, attrs):
Type = _create_type(meta, name, attrs)
cls = super().__new__(meta, name, bases, attrs)
cls.__class__ = Type
return cls
class _ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, owner):
if self in obj.__dict__.values():
return self.fget(obj)
return self.fget(owner)
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
return self.fset(obj, value)
def setter(self, func):
self.fset = func
return self
def classproperty(func):
return _ClassPropertyDescriptor(func)
class Bar(metaclass=ClassPropertyType):
__bar = 1
#classproperty
def bar(cls):
return cls.__bar
#bar.setter
def bar(cls, value):
cls.__bar = value
bar = Bar()
assert Bar.bar==1
Bar.bar=2
assert bar.bar==2
nbar = Bar()
assert nbar.bar==2

I happened to come up with a solution very similar to #Andrew, only DRY
class MetaFoo(type):
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.thingy})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
#property
def thingy(cls):
if not inspect.isclass(cls):
cls = type(cls)
return cls._thingy
#thingy.setter
def thingy(cls, value):
if not inspect.isclass(cls):
cls = type(cls)
cls._thingy = value
class Foo(metaclass=MetaFoo):
_thingy = 23
class Bar(Foo)
_thingy = 12
This has the best of all answers:
The "metaproperty" is added to the class, so that it will still be a property of the instance
Don't need to redefine thingy in any of the classes
The property works as a "class property" in for both instance and class
You have the flexibility to customize how _thingy is inherited
In my case, I actually customized _thingy to be different for every child, without defining it in each class (and without a default value) by:
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.services, '_thingy': None})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)

If you only need lazy loading, then you could just have a class initialisation method.
EXAMPLE_SET = False
class Example(object):
#classmethod
def initclass(cls):
global EXAMPLE_SET
if EXAMPLE_SET: return
cls.the_I = 'ok'
EXAMPLE_SET = True
def __init__( self ):
Example.initclass()
self.an_i = 20
try:
print Example.the_I
except AttributeError:
print 'ok class not "loaded"'
foo = Example()
print foo.the_I
print Example.the_I
But the metaclass approach seems cleaner, and with more predictable behavior.
Perhaps what you're looking for is the Singleton design pattern. There's a nice SO QA about implementing shared state in Python.

Overwrite several methods of a class with a loop

I would like to know if there is an easy way to do some identical edits on several methods of a class. An example :
class Dog():
def __init__(self):
self.name = 'abc'
self.age = 1
def setName(self, newValue):
self.name = newValue
def setAge(self, newValue):
self.age = newValue
class TalkingDog(Dog):
def __init__(self):
super().__init__()
# The end is in pseudo code :
for method in TalkingDog.allMethods :
method = method + 'print('I have been edited !')'
I know that I can also overwrite each method but in a situation with tens of methods, that will be a little boring...
So I tried this :
class TalkingDog(Dog):
def __init__(self):
super().__init__()
for method in self.__dir__():
if method.startswith('set'):
oldMethod = getattr(self, method)
def _newMethod(newValue):
oldMethod(newValue)
print('I have been edited !')
setattr(self, method, _newMethod)
a = TalkingDog()
print(a.setName) >>> <function TalkingDog.__init__.<locals>._newMethod at 0x0000000002C350D0>
That almost works but setName is not anymore a method. It's an attribute which contains a function. I completely understand why but I'm trying to get a cleaner result. With that result, I risk of having problems later. For example I can't use the library pickle with that object (got the error _pickle.PicklingError: Can't pickle <function TalkingDog.__init__.<locals>._newMethod at 0x00000000003DCBF8>: attribute lookup _newMethod on __main__ failed).

The Pythonic way to do this is probably to use the descriptor protocol, which is also what properties use:
class VocalAttribute:
def __init__(self, name, feedback):
"""Called when you first create the descriptor."""
self.name = name # the name of the attribute 'behind' the property
self.feedback = feedback # the feedback to show when the value changes
def __get__(self, obj):
"""Called when you get the descriptor value."""
return getattr(obj, self.name)
def __set__(self, obj, value):
"""Called when you set the descriptor value."""
prev = getattr(obj, self.name, None)
if value != prev:
setattr(obj, self.name, value)
print(self.feedback)
def __delete__(self, obj):
"""Called when you delete the descriptor value."""
delattr(obj, self.name)
class Foo:
bar = VocalAttribute('_bar', 'I have been edited!')
foo = Foo()
print('1.')
foo.bar = 'hello'
print('2.')
foo.bar = 'hello'
print('3.')
foo.bar = 'world'
Output:
1.
I have been edited!
2.
3.
I have been edited!
Note that this only gives feedback when the new value is different to the old one - you can tweak the behaviour as needed in __set__. It also means you can directly read from and assign to foo.bar, rather than needing to call getters and setters (what is this, Java?)

since decorator could explicit called here a way to use it:
def updater(obj, call_back, call_back_args=(), call_back_kw=None, replace=False):
# ability to be called on the fly with different args and kw for the callback
# now it returns the updated obj (instance or class)
# but could a be factory returning a new obj in this case make a copy of obj, update this coy and return it
def update_function(fn, *args, **kw):
def wrapper(*args, **kw):
if replace:
# call only the callback
res = call_back(*call_back_args, **call_back_kw)
else:
res = fn(*args, **kw)
call_back(*call_back_args, **call_back_kw)
return res
return wrapper
# get all methods of the obj
# and apply update_function (a decorator) to all methods
for name, m in inspect.getmembers(
obj, predicate=lambda x: inspect.isfunction(x) or inspect.ismethod(x)):
# make the selection here
# could be made on the name for instance
if not name.startswith('_'):
new_m = update_function(m)
setattr(obj, name, new_m)
return obj
# declare a callback
def call_back(*args, **kw):
# simple callback
print("I have been edited and called with %r args and %r kw " % (args, kw))
a = Dog()
# could be called on instance or class
# apply the callback on all "public" methods
updater(
a,
call_back,
call_back_args=(2, 3, 4),
call_back_kw={"kw1": "v_1"}
)

python __setattr__ to act like __getattr__

I have a class that contains a dictionary, i use __getattr__(key) to get nicer access to the dictionary[key] now i would like to be able to set things in the dictionary with the same access style.
Class foo(object):
def __init__(self, name):
self.props = {"name":name}
def __getattr__(self, attribute):
return self.props[attribute]
This is so i can access it in this way
f = foo("test")
print f.name
I would like the ability to set the attributes aswell, however using setattr is proving problematic due to it being called before anything else fails. Is there a way to make it act like getattr?

__setattr__ is fine, but you need to protect yourself from case when __setattr__ is called before self.props is set (RuntimeError: maximum recursion depth exceeded)
class foo(object):
# List of properties which are not stored in the props dict
__slots__ = ('props', 'other_property')
def __init__(self, name):
self.props = {"name":name}
self.other_property = 2
def __getattr__(self, attribute):
return self.props[attribute]
def __setattr__(self, name, value):
if name in self.__slots__:
super(foo, self).__setattr__(name, value)
else:
self.props[name] = value
f = foo("name")
print f.name
f.value = 2
f.name = "TEST"
print f.value
print f.props

How to use __setattr__ correctly, avoiding infinite recursion

I want to define a class containing read and write methods, which can be called as follows:
instance.read
instance.write
instance.device.read
instance.device.write
To not use interlaced classes, my idea was to overwrite the __getattr__ and __setattr__ methods and to check, if the given name is device to redirect the return to self. But I encountered a problem giving infinite recursions. The example code is as follows:
class MyTest(object):
def __init__(self, x):
self.x = x
def __setattr__(self, name, value):
if name=="device":
print "device test"
else:
setattr(self, name, value)
test = MyTest(1)
As in __init__ the code tried to create a new attribute x, it calls __setattr__, which again calls __setattr__ and so on. How do I need to change this code, that, in this case, a new attribute x of self is created, holding the value 1?
Or is there any better way to handle calls like instance.device.read to be 'mapped' to instance.read?
As there are always questions about the why: I need to create abstractions of xmlrpc calls, for which very easy methods like myxmlrpc.instance,device.read and similar can be created. I need to 'mock' this up to mimic such multi-dot-method calls.

You must call the parent class __setattr__ method:
class MyTest(object):
def __init__(self, x):
self.x = x
def __setattr__(self, name, value):
if name=="device":
print "device test"
else:
super(MyTest, self).__setattr__(name, value)
# in python3+ you can omit the arguments to super:
#super().__setattr__(name, value)
Regarding the best-practice, since you plan to use this via xml-rpc I think this is probably better done inside the _dispatch method.
A quick and dirty way is to simply do:
class My(object):
def __init__(self):
self.device = self

Or you can modify self.__dict__ from inside __setattr__():
class SomeClass(object):
def __setattr__(self, name, value):
print(name, value)
self.__dict__[name] = value
def __init__(self, attr1, attr2):
self.attr1 = attr1
self.attr2 = attr2
sc = SomeClass(attr1=1, attr2=2)
sc.attr1 = 3

You can also use object.
class TestClass:
def __init__(self):
self.data = 'data'
def __setattr__(self, name, value):
print("Attempt to edit the attribute %s" %(name))
object.__setattr__(self, name, value)

or you can just use #property:
class MyTest(object):
def __init__(self, x):
self.x = x
#property
def device(self):
return self

If you don't want to specify which attributes can or cannot be set, you can split the class to delay the get/set hooks until after initialization:
class MyTest(object):
def __init__(self, x):
self.x = x
self.__class__ = _MyTestWithHooks
class _MyTestWithHooks(MyTest):
def __setattr__(self, name, value):
...
def __getattr__(self, name):
...
if __name__ == '__main__':
a = MyTest(12)
...
As noted in the code you'll want to instantiate MyTest, since instantiating _MyTestWithHooks will result in the same infinite recursion problem as before.

Using both __setattr__ and descriptors for a python class

I'm writing a python class that uses __setattr__ and __getattr__ to provide custom attribute access.
However, some attributes can't be handled in a generic way, so I was hoping to use descriptors for those.
A problem arises in that for a descriptor, the descriptor's __get__ will be invoked in favour of the instances __getattr__, but when assigning to an attribute, __setattr__ will be invoked in favour of the descriptors __set__.
An example:
class MyDesc(object):
def __init__(self):
self.val = None
def __get__(self, instance, owner):
print "MyDesc.__get__"
return self.val
def __set__(self, instance, value):
print "MyDesc.__set__"
self.val = value
class MyObj(object):
foo = MyDesc()
def __init__(self, bar):
object.__setattr__(self, 'names', dict(
bar=bar,
))
object.__setattr__(self, 'new_names', dict())
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
self.new_names[name] = value
def __getattr__(self, name):
print "MyObj.__getattr__ for %s" % name
if name in self.new_names:
return self.new_names[name]
if name in self.names:
return self.names[name]
raise AttributeError(name)
if __name__ == "__main__":
o = MyObj('bar-init')
o.bar = 'baz'
print o.bar
o.foo = 'quux'
print o.foo
prints:
MyObj.__setattr__ for bar
MyObj.__getattr__ for bar
baz
MyObj.__setattr__ for foo
MyDesc.__get__
None
The descriptor's __set__ is never called.
Since the __setattr__ definition isn't just overriding behaviour for a limited set of names, there's no clear place that it can defer to object.__setattr__
Is there a recommended way to have assigning to attributes use the descriptor, if available, and __setattr__ otherwise?

I think I'd approach this by having a mechanism to automatically mark which are the
descriptors in each class, and wrap the __setattr__ in a way that it'd call
object's normal behavior for those names.
This can be easily achieved with a metaclass (and a decorator for __setattr__
def setattr_deco(setattr_func):
def setattr_wrapper(self, attr, value):
if attr in self._descriptors:
return object.__setattr__(self, attr, value)
return setattr_func(self, attr, value)
return setattr_wrapper
class MiscSetattr(type):
def __new__(metacls, name, bases, dct):
descriptors = set()
for key, obj in dct.items():
if key == "__setattr__":
dct[key] = setattr_deco(obj)
elif hasattr(obj, "__get__"):
descriptors.add(key)
dct["_descriptors"] = descriptors
return type.__new__(metacls, name, bases, dct)
# and use MiscSetattr as metaclass for your classes

One of possible ways:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
for cls in self.__class__.__mro__ + (self, ):
if name in cls.__dict__:
return object.__setattr__(self, name, value)
print 'New name', name, value
self.new_names[name] = value
It checks if name already defined in class, base classes or instance and then it calls object.__setattr__ which will execute descriptor __set__.
Another way:
def __setattr__(self, name, value):
print "MyObj.__setattr__ for %s" % name
try:
object.__getattribute__(self, name)
except AttributeError:
print 'New name', name, value
self.new_names[name] = value
else:
object.__setattr__(self, name, value)
But it will call descriptor's __get__.
P.S.
I'm not sure about need to check all __mro__ members since MyObj will contain inherited class members in __dict__.
Maybe for cls in (self.__class__, self):... will be enough.