Class variable in python context class - python

I am writing a series of nested contexts, and need to keep trace of their relative relationship. But I ran in the following behavior that I did not expect:
class NestedContext():
_current = None
def __enter__(self):
self.parent = self._current
self._current = self
def __exit__(self, _type, _value, _tb):
self._current = self.parent
with NestedContext():
assert NestedContext._current is not None # fails
with NestedContext():
pass
The assert fail because the class variable _current is None, which is unexpected, since I thought __enter__ just set it to self. This seems to be related some deeper behavior of Python context that I don't understand. Can anyone explain? Thanks.

You are testing the class attribute. You set self._current, an instance attribute.
If you wanted to set the class attribute, you need to qualify it as such:
class NestedContext(object):
_current = None
def __enter__(self):
self.parent = self._current
NestedContext._current = self
def __exit__(self, _type, _value, _tb):
NestedContext._current = self.parent
When reading an attribute, Python searches through the layers of instance, class, and base classes. But when setting an attribute, no such layering takes place. So self._current will fall back to NestedContext if there is no such attribute on self, but self._current = some_value sets that attribute on self, not on the class.
I also used object as the parent class for NestedContext; unless there are compelling reasons not to (e.g. you are working with legacy code that relies on the old behaviours), you always want to get new style classes.
Demo (with addition of a _name attribute and a __repr__ method):
>>> class NestedContext(object):
... _current = None
... def __init__(self, name):
... self._name = name
... def __repr__(self):
... return '<NestedContext({._name!r})>'.format(self)
... def __enter__(self):
... self.parent = self._current
... NestedContext._current = self
... def __exit__(self, _type, _value, _tb):
... NestedContext._current = self.parent
...
>>> with NestedContext('outer'):
... print NestedContext._current
... with NestedContext('inner'):
... print NestedContext._current
... print NestedContext._current.parent
...
<NestedContext('outer')>
<NestedContext('inner')>
<NestedContext('outer')>

Related

How can the output of a `type()` function have a new input?

I'm reading an docs about Descriptors in this doc
The doc explains property function in terms of descriptor protocols.
And it emulates the PyProperty_Type() in python.
class Property:
"Emulate PyProperty_Type() in Objects/descrobject.c"
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
if doc is None and fget is not None:
doc = fget.__doc__
self.__doc__ = doc
self._name = ''
def __set_name__(self, owner, name):
self._name = name
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError(f"property '{self._name}' has no getter")
return self.fget(obj)
def __set__(self, obj, value):
if self.fset is None:
raise AttributeError(f"property '{self._name}' has no setter")
self.fset(obj, value)
def __delete__(self, obj):
if self.fdel is None:
raise AttributeError(f"property '{self._name}' has no deleter")
self.fdel(obj)
def getter(self, fget):
prop = type(self)(fget, self.fset, self.fdel, self.__doc__)
prop._name = self._name
return prop
def setter(self, fset):
prop = type(self)(self.fget, fset, self.fdel, self.__doc__)
prop._name = self._name
return prop
def deleter(self, fdel):
prop = type(self)(self.fget, self.fset, fdel, self.__doc__)
prop._name = self._name
return pro
I have a question about type() in getter(), setter() and deleter().
How does the type() function get inputs like that?
As far as I know, type() only return the type of the input, literally. My expectation of type(self) is a type class (e.g. '<class 'type'>') not a class or instance itself.
~But here the type(self) get inputs same as the original class, which seems weird. I believe here type(self) should be change to self.~ This is a wrong statement.
For example,
l = [1,2,3]
x = type(l)
print(x)
print(type(x))
the results of above code are shown below.
<class 'list'>
<class 'type'>
And if I add input like below,
l = [1,2,3]
x = type(l)([1,2])
print(x)
output is
[1, 2]
Hopely someone anwsers this question. Thanks.
type(self) returns the type of the current object referred by self - that is the class of the object - in this code, it would be the same as using Property - and then you simply call the object class - i.e. property, with the new parameters.
In other words so that it is clear:
type(self) returns the class of "self", in this case "Property". It also could be written as self.__class__ or spelled out explicitly as Property.
The immediately following parenthesis group, as in the line prop = type(self)(fget, self.fset, self.fdel, self.__doc__) will simply call the class with this new set of parameters, producing a new instance. (In Python the syntax and semantics for creating a new instance and calling a class are the same).
The reason for using it in this way rather than just hardcoding the name "Property" is because in this way it will work if one subclasses the class "Property" defined here, and the code will work as is with any subclasses, as type(self) will return the subclass, which will then be used to create the new instance.
An advice I can give you: Python is a great language to learn if you use it a bit in the interactive mode, rather than relying solely on the output of complete programs - if you can go into an interactive mode and type the following, and see this happening by yourself, it might be much easier to understand these concepts:
>>> class A:
... def __init__(self):
... print("creating new instance of A")
...
>>> a = A()
creating new instance of A
>>> type(a)
<class '__main__.A'>
>>> type(a)()
creating new instance of A
<__main__.A object at 0x...>

Why would calling the super class `__getattribute__` cause an infinite recursion?

clearly the following will result in an infinite loop:
class Klaus:
def __getattribute__(self, key):
return getattr(type(self), key)
However, I don't see why calling the super class __getattribute__ would:
class Parent:
def __init__(self):
print("begin __init__")
self._x = 3
print("end __init__")
def __getattribute__(self, attr_name):
superk = super(type(self), self)
boohl = superk.__getattribute__ == self.__getattribute__
print("with age comes wisdom", boohl)
return superk.__getattribute__(attr_name)
class Child(Parent):
def __getattribute__(self, attr_name):
superk = super(type(self), self)
boohl = superk.__getattribute__ == self.__getattribute__
print("this booger is green!", boohl)
return super(type(self), self).__getattribute__(attr_name)
obj = Child()
print("lambda")
print(obj._x)
print("anonymous")
Because type(self) is always Child. Even though you're in a method of Parent, self is still an instance of Child.
This is why when you use the long form of super you must always explicitly use the current class, rather than this. Of course, in Python 3 you can use super() without any parameters anyway.

How to specify numba jitclass when the class' attribute contains an instance of self

I have a simple Python class:
class Node(object):
def __init__(self):
self._left = self
self._right = self
#property
def left(self):
return self._left
#left.setter
def left(self, value):
self._left = value
#property
def right(self):
return self._right
#right.setter
def right(self, value):
self._right = value
I would like to jit this class but it isn't clear how to define the types when self._left and self._right are of Node object type. I came across this other answer that uses .class_type.instance_type but that example only refers to a class attribute that contains a different class instance. In my case, since Node is not defined yet, I cannot declare the following spec:
spec=[("_left", Node.class_type.instance_type),
("_right", Node.class_type.instance_type),
]
#jitclass(spec)
class Node(object):
Based on this example - numba has a deferred_type function for this case.
node_type = numba.deferred_type()
spec=[
("_left", node_type),
("_right", node_type),
]
#jitclass(spec)
class Node:
# omitted
# define the deferred type
node_type.define(Node.class_type.instance_type)
# class can be used!
a = Node()
b = Node()
a.left = b

Using #classmethod with #property [duplicate]

This question already has answers here:
Using property() on classmethods
(19 answers)
Closed 3 years ago.
In python I can add a method to a class with the #classmethod decorator. Is there a similar decorator to add a property to a class? I can better show what I'm talking about.
class Example(object):
the_I = 10
def __init__( self ):
self.an_i = 20
#property
def i( self ):
return self.an_i
def inc_i( self ):
self.an_i += 1
# is this even possible?
#classproperty
def I( cls ):
return cls.the_I
#classmethod
def inc_I( cls ):
cls.the_I += 1
e = Example()
assert e.i == 20
e.inc_i()
assert e.i == 21
assert Example.I == 10
Example.inc_I()
assert Example.I == 11
Is the syntax I've used above possible or would it require something more?
The reason I want class properties is so I can lazy load class attributes, which seems reasonable enough.
Here's how I would do this:
class ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.fget.__get__(obj, klass)()
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
type_ = type(obj)
return self.fset.__get__(obj, type_)(value)
def setter(self, func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
self.fset = func
return self
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class Bar(object):
_bar = 1
#classproperty
def bar(cls):
return cls._bar
#bar.setter
def bar(cls, value):
cls._bar = value
# test instance instantiation
foo = Bar()
assert foo.bar == 1
baz = Bar()
assert baz.bar == 1
# test static variable
baz.bar = 5
assert foo.bar == 5
# test setting variable on the class
Bar.bar = 50
assert baz.bar == 50
assert foo.bar == 50
The setter didn't work at the time we call Bar.bar, because we are calling
TypeOfBar.bar.__set__, which is not Bar.bar.__set__.
Adding a metaclass definition solves this:
class ClassPropertyMetaClass(type):
def __setattr__(self, key, value):
if key in self.__dict__:
obj = self.__dict__.get(key)
if obj and type(obj) is ClassPropertyDescriptor:
return obj.__set__(self, value)
return super(ClassPropertyMetaClass, self).__setattr__(key, value)
# and update class define:
# class Bar(object):
# __metaclass__ = ClassPropertyMetaClass
# _bar = 1
# and update ClassPropertyDescriptor.__set__
# def __set__(self, obj, value):
# if not self.fset:
# raise AttributeError("can't set attribute")
# if inspect.isclass(obj):
# type_ = obj
# obj = None
# else:
# type_ = type(obj)
# return self.fset.__get__(obj, type_)(value)
Now all will be fine.
If you define classproperty as follows, then your example works exactly as you requested.
class classproperty(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, owner):
return self.f(owner)
The caveat is that you can't use this for writable properties. While e.I = 20 will raise an AttributeError, Example.I = 20 will overwrite the property object itself.
[answer written based on python 3.4; the metaclass syntax differs in 2 but I think the technique will still work]
You can do this with a metaclass...mostly. Dappawit's almost works, but I think it has a flaw:
class MetaFoo(type):
#property
def thingy(cls):
return cls._thingy
class Foo(object, metaclass=MetaFoo):
_thingy = 23
This gets you a classproperty on Foo, but there's a problem...
print("Foo.thingy is {}".format(Foo.thingy))
# Foo.thingy is 23
# Yay, the classmethod-property is working as intended!
foo = Foo()
if hasattr(foo, "thingy"):
print("Foo().thingy is {}".format(foo.thingy))
else:
print("Foo instance has no attribute 'thingy'")
# Foo instance has no attribute 'thingy'
# Wha....?
What the hell is going on here? Why can't I reach the class property from an instance?
I was beating my head on this for quite a while before finding what I believe is the answer. Python #properties are a subset of descriptors, and, from the descriptor documentation (emphasis mine):
The default behavior for attribute access is to get, set, or delete the
attribute from an object’s dictionary. For instance, a.x has a lookup chain
starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing
through the base classes of type(a) excluding metaclasses.
So the method resolution order doesn't include our class properties (or anything else defined in the metaclass). It is possible to make a subclass of the built-in property decorator that behaves differently, but (citation needed) I've gotten the impression googling that the developers had a good reason (which I do not understand) for doing it that way.
That doesn't mean we're out of luck; we can access the properties on the class itself just fine...and we can get the class from type(self) within the instance, which we can use to make #property dispatchers:
class Foo(object, metaclass=MetaFoo):
_thingy = 23
#property
def thingy(self):
return type(self).thingy
Now Foo().thingy works as intended for both the class and the instances! It will also continue to do the right thing if a derived class replaces its underlying _thingy (which is the use case that got me on this hunt originally).
This isn't 100% satisfying to me -- having to do setup in both the metaclass and object class feels like it violates the DRY principle. But the latter is just a one-line dispatcher; I'm mostly okay with it existing, and you could probably compact it down to a lambda or something if you really wanted.
If you use Django, it has a built in #classproperty decorator.
from django.utils.decorators import classproperty
For Django 4, use:
from django.utils.functional import classproperty
I think you may be able to do this with the metaclass. Since the metaclass can be like a class for the class (if that makes sense). I know you can assign a __call__() method to the metaclass to override calling the class, MyClass(). I wonder if using the property decorator on the metaclass operates similarly.
Wow, it works:
class MetaClass(type):
def getfoo(self):
return self._foo
foo = property(getfoo)
#property
def bar(self):
return self._bar
class MyClass(object):
__metaclass__ = MetaClass
_foo = 'abc'
_bar = 'def'
print MyClass.foo
print MyClass.bar
Note: This is in Python 2.7. Python 3+ uses a different technique to declare a metaclass. Use: class MyClass(metaclass=MetaClass):, remove __metaclass__, and the rest is the same.
As far as I can tell, there is no way to write a setter for a class property without creating a new metaclass.
I have found that the following method works. Define a metaclass with all of the class properties and setters you want. IE, I wanted a class with a title property with a setter. Here's what I wrote:
class TitleMeta(type):
#property
def title(self):
return getattr(self, '_title', 'Default Title')
#title.setter
def title(self, title):
self._title = title
# Do whatever else you want when the title is set...
Now make the actual class you want as normal, except have it use the metaclass you created above.
# Python 2 style:
class ClassWithTitle(object):
__metaclass__ = TitleMeta
# The rest of your class definition...
# Python 3 style:
class ClassWithTitle(object, metaclass = TitleMeta):
# Your class definition...
It's a bit weird to define this metaclass as we did above if we'll only ever use it on the single class. In that case, if you're using the Python 2 style, you can actually define the metaclass inside the class body. That way it's not defined in the module scope.
def _create_type(meta, name, attrs):
type_name = f'{name}Type'
type_attrs = {}
for k, v in attrs.items():
if type(v) is _ClassPropertyDescriptor:
type_attrs[k] = v
return type(type_name, (meta,), type_attrs)
class ClassPropertyType(type):
def __new__(meta, name, bases, attrs):
Type = _create_type(meta, name, attrs)
cls = super().__new__(meta, name, bases, attrs)
cls.__class__ = Type
return cls
class _ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, owner):
if self in obj.__dict__.values():
return self.fget(obj)
return self.fget(owner)
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
return self.fset(obj, value)
def setter(self, func):
self.fset = func
return self
def classproperty(func):
return _ClassPropertyDescriptor(func)
class Bar(metaclass=ClassPropertyType):
__bar = 1
#classproperty
def bar(cls):
return cls.__bar
#bar.setter
def bar(cls, value):
cls.__bar = value
bar = Bar()
assert Bar.bar==1
Bar.bar=2
assert bar.bar==2
nbar = Bar()
assert nbar.bar==2
I happened to come up with a solution very similar to #Andrew, only DRY
class MetaFoo(type):
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.thingy})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
#property
def thingy(cls):
if not inspect.isclass(cls):
cls = type(cls)
return cls._thingy
#thingy.setter
def thingy(cls, value):
if not inspect.isclass(cls):
cls = type(cls)
cls._thingy = value
class Foo(metaclass=MetaFoo):
_thingy = 23
class Bar(Foo)
_thingy = 12
This has the best of all answers:
The "metaproperty" is added to the class, so that it will still be a property of the instance
Don't need to redefine thingy in any of the classes
The property works as a "class property" in for both instance and class
You have the flexibility to customize how _thingy is inherited
In my case, I actually customized _thingy to be different for every child, without defining it in each class (and without a default value) by:
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.services, '_thingy': None})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
If you only need lazy loading, then you could just have a class initialisation method.
EXAMPLE_SET = False
class Example(object):
#classmethod
def initclass(cls):
global EXAMPLE_SET
if EXAMPLE_SET: return
cls.the_I = 'ok'
EXAMPLE_SET = True
def __init__( self ):
Example.initclass()
self.an_i = 20
try:
print Example.the_I
except AttributeError:
print 'ok class not "loaded"'
foo = Example()
print foo.the_I
print Example.the_I
But the metaclass approach seems cleaner, and with more predictable behavior.
Perhaps what you're looking for is the Singleton design pattern. There's a nice SO QA about implementing shared state in Python.

python: set read-only attribute for class objects

I've created a class object called 'Node'. I then created two subclasses 'Beetle' and 'Dakota'. You'll notice there is an attr called 'superclass' which is part of the baseclass. I want this attribute to be set for each subclass and once it's set it should never be changed. It's a read-only attribute. I was wondering how to set this attribute up correctly in order to be a read only attribute?
class Node(object):
def __init__(self, name, superclass, attributes, children):
self.name = name
self.superclass = superclass
self.attributes = attributes if attributes is not None else {}
self.children = children if children is not None else []
class Beetle(Node):
def __init__(self, name="", superclass="Cars", attributes=None, children=None, enabled=True):
super(Beetle, self).__init__(name=name, superclass=superclass, attributes=attributes, children=children)
self.enabled = enabled
class Dakota(Node):
def __init__(self, name="", superclass="Trucks", attributes=None, children=None, enabled=True):
super(Dakota, self).__init__(name=name, superclass=superclass, attributes=attributes, children=children)
self.enabled = enabled
Rename you variable to add __ to the begining
self.__superclass = superclass
you can't access self.__superclass with something like Dakota().__superclass
if you want to get value of __superclass add a function in Node class to return it.
def getsuperclass(self):
return self.__superclass
...
Dakota().getsuperclass()
To have a 'read-only' property in a class you can make use of the #property decoration.
An example:
class Dakota(object):
def __init__(self, superclass):
self.__superclass = superclass
#property
def superclass(self):
return self.__superclass
Usage:
>>> a = Dakota('lol')
>>> a.superclass
'lol'
>>> a.superclass = 'hah'
AttributeError...
AttributeError: can't set attribute
You can refer to this other answer thread

Categories

Resources