In python classes, the #property is a nice decorator that avoids using explicit setter and getter functions. However, it comes at a cost of an overhead 2-5 times that of a "classical" class function. In my case, this is quite OK in the case of setting a property, where the overhead is insignificant compared to the processing that needs to be done when setting.
However, I need no processing when getting the property. It is always just "return self.property". Is there an elegant way to use the setter but not using the getter, without needing to use a different internal variable?
Just to illustrate, the class below has the property "var" which refers to the internal variable "_var". It takes longer to call "var" than "_var" but it would be nice if developers and users alike could just use "var" without having to keep track of "_var" too.
class MyClass(object):
def __init__(self):
self._var = None
# the property "var". First the getter, then the setter
#property
def var(self):
return self._var
#var.setter
def var(self, newValue):
self._var = newValue
#... and a lot of other stuff here
# Use "var" a lot! How to avoid the overhead of the getter and not to call self._var!
def useAttribute(self):
for i in xrange(100000):
self.var == 'something'
For those interested, on my pc calling "var" takes 204 ns on average while calling "_var" takes 44 ns on average.
Don't use a property in this case. A property object is a data descriptor, which means that any access to instance.var will invoke that descriptor and Python will never look for an attribute on the instance itself.
You have two options: use the .__setattr__() hook or build a descriptor that only implements .__set__.
Using the .__setattr__() hook
class MyClass(object):
var = 'foo'
def __setattr__(self, name, value):
if name == 'var':
print "Setting var!"
# do something with `value` here, like you would in a
# setter.
value = 'Set to ' + value
super(MyClass, self).__setattr__(name, value)
Now normal attribute lookups are used when reading .var but when assigning to .var the __setattr__ method is invoked instead, letting you intercept value and adjust it as needed.
Demo:
>>> mc = MyClass()
>>> mc.var
'foo'
>>> mc.var = 'bar'
Setting var!
>>> mc.var
'Set to bar'
A setter descriptor
A setter descriptor would only intercept variable assignment:
class SetterProperty(object):
def __init__(self, func, doc=None):
self.func = func
self.__doc__ = doc if doc is not None else func.__doc__
def __set__(self, obj, value):
return self.func(obj, value)
class Foo(object):
#SetterProperty
def var(self, value):
print 'Setting var!'
self.__dict__['var'] = value
Note how we need to assign to the instance .__dict__ attribute to prevent invoking the setter again.
Demo:
>>> f = Foo()
>>> f.var = 'spam'
Setting var!
>>> f.var = 'ham'
Setting var!
>>> f.var
'ham'
>>> f.var = 'biggles'
Setting var!
>>> f.var
'biggles'
property python docs: https://docs.python.org/2/howto/descriptor.html#properties
class MyClass(object):
def __init__(self):
self._var = None
# only setter
def var(self, newValue):
self._var = newValue
var = property(None, var)
c = MyClass()
c.var = 3
print ('ok')
print (c.var)
output:
ok
Traceback (most recent call last):
File "Untitled.py", line 15, in <module>
print c.var
AttributeError: unreadable attribute
The #WeizhongTu answer
class MyClass(object):
def __init__(self):
self._var = None
# only setter
def var(self, newValue):
self._var = newValue
var = property(None, var)
c = MyClass()
c.var = 3
print ('ok')
print (c.var)
Is fine, except from the fact that is making the variable ungettable...
A similar solution but preserving getter is with
var = property(lambda self: self._var, var)
instead of
var = property(None, var)
The accepted answer's setter descriptor would be probably more convenient if it set the property by itself:
A setter descriptor (alt.)
class setter:
def __init__(self, func, doc=None):
self.func = func
self.__doc__ = doc or func.__doc__
def __set__(self, obj, value):
obj.__dict__[self.func.__name__] = self.func(obj, value)
class Foo:
#setter
def var(self, value):
print('Setting var!')
# validations and/or operations on received value
if not isinstance(value, str):
raise ValueError('`var` must be a string')
value = value.capitalize()
# returns property value
return value
Demo:
>>> f = Foo()
>>> f.var = 'spam'
Setting var!
>>> f.var = 'ham'
Setting var!
>>> f.var
'Ham'
>>> f.var = 'biggles'
Setting var!
>>> f.var
'Biggles'
>>> f.var = 3
ValueError: `var` must be a string
Related
In this example, I would like to avoid the # Oops eventuality.
def foo():
return "foo"
class MyClass(object):
def __init__(self):
setattr(self, 'foo', foo)
def bar(self):
return "bar"
-
>>> x = MyClass()
>>> x.foo()
>>> x.foo = 2 # Oops
>>> x.foo()
TypeError: 'int' object is not callable
How can I prevent my methods to be overwritten by mistake?
Make x.foo a property, without specifying a setter. However it is quite tricky to do it dynamically:
def add_property(inst, name, method):
'''
Adds a property to a class instance.
Property must be added to the CLASS.
'''
cls = type(inst)
if not hasattr(cls, '__perinstance'):
cls = type(cls.__name__, (cls,), {})
cls.__perinstance = True
inst.__class__ = cls
setattr(cls, name, property(method))
And then instead of just doing setattr do it like this:
class MyClass(object):
def __init__(self):
add_property(self, 'foo', lambda _ : 2)
(for more realistic use, replace the lambda function with the
function or method returning the value for foo)
Output:
>>> o=MyClass()
>>> o.foo
2
>>> o.foo=3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>>
You can check if passed to setattr attribute name is already exists in class (and instance if needed) __dict__ and do not rewrite it in that case:
class MyClass(object):
def __setattr__(self, name, value):
if name not in self.__class__.__dict__ and name not in self.__dict__:
super(MyClass, self).__setattr__(name, value)
test:
>>> x = MyClass()
>>> x.foo = foo # equal to your __init__ setattr call
>>> x.foo()
'foo'
>>> x.foo = 2
>>> x.foo()
'foo'
I am getting through the property to study descriptor protocol, and I am writing my own property like this:
class my_property(object):
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
if doc is None and fget is not None:
doc = fget.__doc__
self.__doc__ = doc
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError("unreadable attribute")
return self.fget(obj)
def setter(self, fset):
return type(self)(self.fget, fset, self.fdel)
class test_my_property(object):
def __init__(self, value):
self._val = value
#my_property
def val(self):
return self._val
#val.setter
def val(self, value):
self._val = value
def main():
c = test_my_property(5)
print c.val
c.val = 10
print c.val
print type(c).__dict__['val'].__set__
And I get:
5
10
AttributeError: 'my_property' object has no attribute '__set__'
My question is, since "__set__" is not defined, then how "c.val = 10" can work?
if "__set__" is inherited from object by my_property, then, why it report the AttributeError?
__set__ is not inherited from object. Getting and setting val attribute works because when accessing an attribute of an object, first the instance will be checked, then the class. Since you set instance attribute val, it uses that. I think this is especially clear if you're looking at a simple example of this with no descriptors,
>>> class Foo(object):
... val = 5
...
>>> f = Foo()
>>> f.val # f doesn't have val so fallback on Foo
5
>>> f.val = 10
>>> f.val # f now has val so use it
10
>>> del f.val # oops what now
>>> f.val # class again
5
The only difference between the above example and yours is that your class val is (when you finish) a property.
With all that said, you generally don't want to be naming your property the same thing as the instance attribute that will hold it's contents. The usual formulation is something like this,
class Foo(object):
def __init__(self, value):
self._val = value
#property
def val(self):
return self._val
#Jared's answer is correct. __set__ is not inherited from object. I'll try to explain in another way, which might be clearer.
First, as you already understand, if your descriptor do have a __set__ method, it gets called when running c.val=10. That means the interpreter looks for the __set__ method, and if it founds it, it treats it as a descriptor, by calling it.
Now, since my_property does not have a __set__ method, it won't get the descriptor treatment when running c.val=10. The interpreter falls back to the "standard" treatment, which is roughly equivalent to c.__dict__['val']=10.
You can easily verify that using:
print c.__dict__ # no 'val'
c.val = 10
print c.__dict__ # 'val' was added
Now, the 'val' in c.__dict__ (at object level) overshaddows your property (which is defined at class level), and will get used when accessing c.val.
If you want to forbid assignment to your property, you'd need to do it explicitly. You'd need to define a __set__ method and raise an error in it.
Class Foo has a bar, and it is not loaded until it is accessed. Further accesses to bar should incur no overhead.
class Foo(object):
def get_bar(self):
print "initializing"
self.bar = "12345"
self.get_bar = self._get_bar
return self.bar
def _get_bar(self):
print "accessing"
return self.bar
Is it possible to do something like this using properties or, better yet, attributes, instead of using a getter method?
The goal is to lazy load without overhead on all subsequent accesses...
There are some problems with the current answers. The solution with a property requires that you specify an additional class attribute and has the overhead of checking this attribute on each look up. The solution with __getattr__ has the issue that it hides this attribute until first access. This is bad for introspection and a workaround with __dir__ is inconvenient.
A better solution than the two proposed ones is utilizing descriptors directly. The werkzeug library has already a solution as werkzeug.utils.cached_property. It has a simple implementation so you can directly use it without having Werkzeug as dependency:
_missing = object()
class cached_property(object):
"""A decorator that converts a function into a lazy property. The
function wrapped is called the first time to retrieve the result
and then that calculated result is used the next time you access
the value::
class Foo(object):
#cached_property
def foo(self):
# calculate something important here
return 42
The class has to have a `__dict__` in order for this property to
work.
"""
# implementation detail: this property is implemented as non-data
# descriptor. non-data descriptors are only invoked if there is
# no entry with the same name in the instance's __dict__.
# this allows us to completely get rid of the access function call
# overhead. If one choses to invoke __get__ by hand the property
# will still work as expected because the lookup logic is replicated
# in __get__ for manual invocation.
def __init__(self, func, name=None, doc=None):
self.__name__ = name or func.__name__
self.__module__ = func.__module__
self.__doc__ = doc or func.__doc__
self.func = func
def __get__(self, obj, type=None):
if obj is None:
return self
value = obj.__dict__.get(self.__name__, _missing)
if value is _missing:
value = self.func(obj)
obj.__dict__[self.__name__] = value
return value
Sure, just have your property set an instance attribute that is returned on subsequent access:
class Foo(object):
_cached_bar = None
#property
def bar(self):
if not self._cached_bar:
self._cached_bar = self._get_expensive_bar_expression()
return self._cached_bar
The property descriptor is a data descriptor (it implements __get__, __set__ and __delete__ descriptor hooks), so it'll be invoked even if a bar attribute exists on the instance, with the end result that Python ignores that attribute, hence the need to test for a separate attribute on each access.
You can write your own descriptor that only implements __get__, at which point Python uses an attribute on the instance over the descriptor if it exists:
class CachedProperty(object):
def __init__(self, func, name=None):
self.func = func
self.name = name if name is not None else func.__name__
self.__doc__ = func.__doc__
def __get__(self, instance, class_):
if instance is None:
return self
res = self.func(instance)
setattr(instance, self.name, res)
return res
class Foo(object):
#CachedProperty
def bar(self):
return self._get_expensive_bar_expression()
If you prefer a __getattr__ approach (which has something to say for it), that'd be:
class Foo(object):
def __getattr__(self, name):
if name == 'bar':
bar = self.bar = self._get_expensive_bar_expression()
return bar
return super(Foo, self).__getattr__(name)
Subsequent access will find the bar attribute on the instance and __getattr__ won't be consulted.
Demo:
>>> class FooExpensive(object):
... def _get_expensive_bar_expression(self):
... print 'Doing something expensive'
... return 'Spam ham & eggs'
...
>>> class FooProperty(FooExpensive):
... _cached_bar = None
... #property
... def bar(self):
... if not self._cached_bar:
... self._cached_bar = self._get_expensive_bar_expression()
... return self._cached_bar
...
>>> f = FooProperty()
>>> f.bar
Doing something expensive
'Spam ham & eggs'
>>> f.bar
'Spam ham & eggs'
>>> vars(f)
{'_cached_bar': 'Spam ham & eggs'}
>>> class FooDescriptor(FooExpensive):
... bar = CachedProperty(FooExpensive._get_expensive_bar_expression, 'bar')
...
>>> f = FooDescriptor()
>>> f.bar
Doing something expensive
'Spam ham & eggs'
>>> f.bar
'Spam ham & eggs'
>>> vars(f)
{'bar': 'Spam ham & eggs'}
>>> class FooGetAttr(FooExpensive):
... def __getattr__(self, name):
... if name == 'bar':
... bar = self.bar = self._get_expensive_bar_expression()
... return bar
... return super(Foo, self).__getatt__(name)
...
>>> f = FooGetAttr()
>>> f.bar
Doing something expensive
'Spam ham & eggs'
>>> f.bar
'Spam ham & eggs'
>>> vars(f)
{'bar': 'Spam ham & eggs'}
Sure it is, try:
class Foo(object):
def __init__(self):
self._bar = None # Initial value
#property
def bar(self):
if self._bar is None:
self._bar = HeavyObject()
return self._bar
Note that this is not thread-safe. cPython has GIL, so it's a relative issue, but if you plan to use this in a true multithread Python stack (say, Jython), you might want to implement some form of lock safety.
I am using a code snippet from here along with my own modifications in ironpython which works extremly well:
from System.ComponentModel import INotifyPropertyChanged, PropertyChangedEventArgs
from Library.pyevent import make_event
class Notify_property(property):
''' defines a notifiable property
'''
def __init__(self, getter):
def newgetter(slf):
#return None when the property does not exist yet
try:
return getter(slf)
except AttributeError:
return None
super(Notify_property, self).__init__(newgetter)
def setter(self, setter):
def newsetter(slf, newvalue):
# do not change value if the new value is the same
# trigger PropertyChanged event when value changes
oldvalue = self.fget(slf)
if oldvalue != newvalue:
setter(slf, newvalue)
slf.OnPropertyChanged(setter.__name__)
return property(
fget=self.fget,
fset=newsetter,
fdel=self.fdel,
doc=self.__doc__)
class NotifyPropertyChangedBase(INotifyPropertyChanged):
''' The base of the MVVM view model
Here the bound properties are added in addition with its
handlers.
'''
# handlers which get fired on any change register here
PropertyChanged = None
''' handlers that only get fired on their property change register here
they are organized in a dictionary with the property name as key and
a list of handlers as value
'''
_property_handlers = {}
def __init__(self):
''' we create an event for the property changed event
'''
self.PropertyChanged, self._propertyChangedCaller = make_event()
def add_PropertyChanged(self, value):
''' helper function to wrap the += behaviour
'''
self.PropertyChanged += value
def remove_PropertyChanged(self, value):
''' helper function to wrap the -= behaviour
'''
self.PropertyChanged -= value
def OnPropertyChanged(self, propertyName):
''' gets fired on an property changed event
'''
if self.PropertyChanged is not None:
self._propertyChangedCaller(self, PropertyChangedEventArgs(propertyName))
try:
for property_handler in self._property_handlers[propertyName]:
property_handler(propertyName,PropertyChangedEventArgs(propertyName))
except KeyError:
pass
def add_notifiable_property(self, notifiable_property):
self.add_handled_property((notifiable_property,None))
def add_notifiable_property_list(self, *symbols):
for symbol in symbols:
self.add_notifiable_property(symbol)
def add_handled_property_list(self, *symbols):
for symbol in symbols:
self.add_handled_property(symbol)
def add_handled_property(self, notifiable_property):
symbol = notifiable_property[0]
if notifiable_property[1] is not None:
self._property_handlers[notifiable_property[0]] = notifiable_property[1]
dnp = """
import sys
sys.path.append(__file__)
#Notify_property
def {0}(self):
return self._{0}
#{0}.setter
def {0}(self, value):
self._{0} = value
""".format(symbol)
d = globals()
exec dnp.strip() in d
setattr(self.__class__, symbol, d[symbol])
Now I must admit that I not fully understand all of the code. Mainly the use of the Notify_property class is an enigma to me. To get a better understanding of the code I tried to remove a property. Calling from my MainViewModel which subclasses the above class I can define a property via:
add_notifiable_property('TestProperty')
or
add_handled_property((TestProperty,[handler1,handler2])
I can also delete handlers (not yet implemeted) but how to I remove a property again?
del self.TestProperty
excepts with
undeletable attribute
and
delattr(self,'TestProperty')
excepts with
delattr takes exactly 2 arguments 2 given
hmm very strange.
I also tried to add a function to my base class:
def remove_notifiable_property(self,propertyname):
''' removes a notifiable property
'''
self._property_handlers.pop(propertyname,None)
exec "del self.{0}".format(propertyname)
but get the same error about an undeletable attribute.
How can I remove a set property again?
EDIT: I found out I was missing the deleter function. Adding this code to the above dnp string now leads to a new error:
#{0}.deleter
def {0}(self):
del self._{0}
with new error:
Derived calss has no attribute _TestProperty
with TestProperty being the name I added. Still stuck.
EDIT2:
I tracked it down to the following:
class C(object):
def __init__(self):
pass#self._x = None
#property
def x(self):
"""I'm the 'x' property."""
return self._x
#x.setter
def x(self, value):
self._x = value
#x.deleter
def x(self):
del self._x
c = C()
print dir(c)
c.x = 'A'
print c.x
print dir(c)
del c.x
print dir (c)
shows the same behavior. The error no _ came from missing initializing the attribute. Adding an:
exec """self._{0} = None""".format(symbol)
to the last line of the add_handled_property fixes it.
But still the attribute itself is shown with dir, also it is not in the class anymore. Is this a bug in python?
You should be able to do
delattr(self.__class__, 'TestProperty')
because the properties are in the class's __dict__. See the last line:
setattr(self.__class__, symbol, d[symbol])
Example of how properties work in Python using the class A, instance a and property p in the class A:
>>> class A(object):
class Property(object):
def __get__(*args):
print 'get:', args
def __set__(*args):
print 'set:', args
def __delete__(*args):
print 'del:', args
p = Property()
>>> A.p
get: (<__main__.Property object at 0x7f3e16da4690>, None, <class '__main__.A'>)
>>> a = A()
>>> a.p
get: (<__main__.Property object at 0x7f3e16da4690>, <__main__.A object at 0x7f3e16da4910>, <class '__main__.A'>)
>>> a.p = 3
set: (<__main__.Property object at 0x7f3e16da4690>, <__main__.A object at 0x7f3e16da4910>, 3)
>>> del a.p
del: (<__main__.Property object at 0x7f3e16da4690>, <__main__.A object at 0x7f3e16da4910>)
you can replace them in the class
>>> A.p = 2
>>> a.p
2
or delete them from the class
>>> A.p = A.Property()
>>> del A.p
What I would like to do there is declaring class variables, but actually use them as vars of the instance. I have a class Field and a class Thing, like this:
class Field(object):
def __set__(self, instance, value):
for key, v in vars(instance.__class__).items():
if v is self:
instance.__dict__.update({key: value})
def __get__(self, instance, owner):
for key, v in vars(instance.__class__).items():
if v is self:
try:
return instance.__dict__[key]
except:
return None
class Thing(object):
foo = Field()
So when I instantiate a thing and set attribute foo, it will be added to the instance, not the class, the class variable is never actually re-set.
new = Thing()
new.foo = 'bar'
# (foo : 'bar') is stored in new.__dict__
This works so far, but the above code for Field is rather awkward. It has too look for the Field object instance in the classes props, otherwise there seems no way of knowing the name of the property (foo) in __set__ and __get__. Is there another, more straight forward way to accomplish this?
Every instance of Field (effectively) has a name. Its name is the attribute name (or key) which references it in Thing. Instead of having to look up the key dynamically, you could instantiate Fields with the name at the time the class attribute is set in Thing:
class Field(object):
def __init__(self, name):
self.name = name
def __set__(self, instance, value):
instance.__dict__.update({self.name: value})
def __get__(self, instance, owner):
if instance is None:
return self
try:
return instance.__dict__[self.name]
except KeyError:
return None
def make_field(*args):
def wrapper(cls):
for arg in args:
setattr(cls, arg, Field(arg))
return cls
return wrapper
#make_field('foo')
class Thing(object):
pass
And it can be used like this:
new = Thing()
Before new.foo is set, new.foo returns None:
print(new.foo)
# None
After new.foo is set, 'foo' is an instance attribute of new:
new.foo = 'bar'
print(new.__dict__)
# {'foo': 'bar'}
You can access the descriptor (the Field instance itself) with Thing.foo:
print(Thing.foo)
# <__main__.Field object at 0xb76cedec>
PS. I'm assuming you have a good reason why
class Thing(object):
foo = None
does not suffice.
Reread your question and realized I had it wrong:
You don't need to override the default python behavior to do this. For example, you could do the following:
class Thing(object):
foo = 5
>>> r = Thing()
>>> r.foo = 10
>>> s = Thing()
>>> print Thing.foo
5
>>> print r.foo
10
>>> print s.foo
5
If you want the default to be 'None' for a particular variable, you could just set the class-wide value to be None. That said, you would have to declare it specifically for each variable.
The easiest way would be to call the attribute something else than the name of the descriptor variable - preferably starting with _ to signal its an implementation detail. That way, you end up with:
def __set__(self, instance, value):
instance._foo = value
def __get__(self, instance, owner):
return getattr(instance, '_foo', None)
The only drawback of this is that you can't determine the name of the key from the one used for the descriptor. If that increased coupling isn't a problem compared to the loop, you could just use a property:
class Thing:
#property
def foo(self):
return getattr(self, '_foo', None)
#foo.setter
def foo(self, value):
self._foo = value
otherwise, you could pass the name of the variable into the descriptor's __init__, so that you have:
class Thing:
foo = Field('_foo')
Of course, all this assumes that the simplest and most Pythonic way - use a real variable Thing().foo that you set to None in Thing.__init__ - isn't an option for some reason. If that way will work for you, you should prefer it.