I have a situation, where getattribute fallbacks to getattr and then again getattribute gets called.
How current getattribute gets called again? I am confused.
class Count(object):
def __init__(self,mymin,mymax):
self.mymin=mymin
self.mymax=mymax
self.current=None
def __getattr__(self, item):
print("akhjhd")
self.__dict__[item]=0
return 0
def __getattribute__(self, item):
print("this is called first")
if item.startswith('cur'):
print("this raised an error")
raise AttributeError
print("This will execute as well")
return object.__getattribute__(self,item)
obj1 = Count(1,10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.current)
Console Output:
this is called first
This will execute as well
1
this is called first
This will execute as well
10
this is called first
this raised an error
akhjhd
this is called first
This will execute as well
0
getattr is called because getattribute raises AttributeError
self.__dict__ invokes the "second" call to getattribute
Clean the code and add print(item) to make this clearer:
class Count(object):
def __init__(self):
self.current = None
def __getattr__(self, item):
print("in getattr")
self.__dict__[item] = 0
return 0
def __getattribute__(self, item):
print(item)
print("in __getattribute__ 1")
if item.startswith('cur'):
print("starts with 'cur'")
raise AttributeError
print("in __getattribute__ 2")
return object.__getattribute__(self, item)
obj1 = Count()
print(obj1.current)
Outputs
current
in __getattribute__ 1
starts with 'cur'
in getattr
__dict__
in __getattribute__ 1
in __getattribute__ 2
0
You need to consult with python Data model
Excerpts for __getattribute__:
Called unconditionally to implement attribute accesses for instances of the class. If the class also defines __getattr__(), the latter will not be called unless __getattribute__() either calls it explicitly or raises an AttributeError.
I see in your code:
if item.startswith('cur'):
print("this raised an error")
raise AttributeError
So I think you did it intentionally
Related
class Test():
def __init__(self,age):
self.age=age
def __getattribute__(self,attribute):
print("Initializing getattribute")
return 6
def __setattr__(self,attribute,value):
print("Initializing setattr")
return object.__setattr__(self,attribute,value)
test=Test(4)
test.age
print(test.age)
From the code above the result is :
Initializing setattr
Initializing getattribute
Initializing getattribute
6
I understand where each dunder method is called, but what do they really do?In the previous example getattribute dictate the attribute value and if I delete the line :
return object.__setattr__(self,attribute,value)
Nothing changes.
So what does __setattr__ do?
__getattribute__ is called before any other attempt is made to access an attribute. No matter what __setattr__ does, test.age is handled by test.__getattribute__("age"), which returns 6 whether or not there is an attribute named age.
If you get rid of __getattribute__:
class Test():
def __init__(self,age):
self.age=age
def __setattr__(self,attribute,value):
print("Initializing setattr")
return object.__setattr__(self,attribute,value)
test=Test(4)
test.age
print(test.age)
The class behaves normally, setting test.age to 4. If you further get rid of the call to object.__setattr__, then you'll get an AttributeError because self.age = age will never actually create or set the age attribute; it just prints the initialization message and returns:
class Test():
def __init__(self,age):
self.age=age
def __setattr__(self,attribute,value):
print("Initializing setattr")
test=Test(4)
test.age
print(test.age)
results in
Initializing setattr
Traceback (most recent call last):
File "/Users/chepner/tmp.py", line 11, in <module>
test.age
AttributeError: 'Test' object has no attribute 'age'
I have the following class that tries to define a custom __getattribute__:
class Item:
def __init__(self):
self.x = 1
def __dir__(self):
return ['x']
def __getattribute__(self, i):
if i not in dir(self):
return "NOT DEFINED!"
else:
return super().__getattribute__(a)
And then I can run it fine:
>>> i=Item()
>>> i.x
1
>>> i.a
'NOT DEFINED!'
However, if I change this line:
if i not in dir(self):
To:
if i not in self.__dir__():
I get a RecursionError as it seems to be calling itself. Why is this so, and why does dir() and differently than __dir__()?
The __getattribute__ method is called whenever the dot notation is used on an object, so self.__dir__() would call the __getattribute__ method of self with '__dir__' as the argument, which, with your code, would then call self.__dir__() again, which calls the __getattribute__ method again with the same argument, resulting in endless recursions.
In the official doc, __getattr__ will be called when __getattribute__ raise an AttributeError.
But the code did not come into an infinite loop, why?
class A:
def __getattr__(self, item):
return self.__getattribute__(item)
a = A()
print(a.d)
It just throw a AttributeError, how to explain it?
This debugging print will help you to understand what is going on:
class A:
def __getattr__(self, item):
print('__getattr__ called for item {}'.format(item))
return self.__getattribute__(item)
def __getattribute__(self, item):
print('__getattribute__ called for item {}'.format(item))
return object.__getattribute__(self, item)
a = A()
print(a.d)
Output:
__getattribute__ called for item d
__getattr__ called for item d
__getattribute__ called for item __getattribute__
__getattribute__ called for item d
AttributeError: 'A' object has no attribute 'd'
__getattribute__ is called first.
After failure of __getattribute__ lookup algorithm calls __getattr__.
Inside it in self.__getattribute__ also __getattribute__ is called for item __getattribute__.
Now __getattribute__ is called for item d, but not in the flow of lookup algorithm because it was called directly, so __getattr__ will not be called.
object.__getattribute__(self, item) can't find d so error is raised.
class Foo(object):
def __init__(self):
self.a = 1
self.c = 0
def __getattr__(self, name):
self.c += 1
print('getattribute')
if hasattr(self, name) is False:
print("No")
return None
return super(Foo, self).__getattr__(self, name)
fo = Foo()
print(fo.a)
print(fo.b)
print(fo.c)
Running the above programs prints "getattribute" and "no" multiple times. __getattr__ is called multiple times. 333 to be exact. self.c prints 333.
What I want to achieve is to have a class that doesn't raise an error if a class variable or method is not declared in the class.
Whats the possible reason for this?
hasattr just tries to get the attribute and returns False if it can’t. Whether it can’t is determined in Python 3 by an attempt throwing an AttributeError, and in Python 2 by an attempt throwing any error. (This includes RecursionErrors and is why it fails silently after 333 calls. Python 2 is not a sane language; upgrade to 3 if possible.)
Instead, you can return the alternative value on an AttributeError yourself:
def __getattr__(self, name):
try:
return super(Foo, self).__getattr__(self, name)
except AttributeError:
return None
This can potentially hide other AttributeErrors, but it’s difficult to avoid doing that just by the nature of Python.
hasattr is a shortcut to call getattr and see if it raises an exception (which means attribute does not exist) or not (which means it exists)
cf : https://docs.python.org/3/library/functions.html#hasattr
getattr calls __getattr__ so you are doing a recursive call
I think a possible workaround would be to use:
name in self.__dict__
instead of:
hasattr(self, name)
This is because running hasattr(self, name) calls self.__getattr__(name) (aka getattr(self, name)) - reference.
So when doing hasattr(self, name) within __getattr__ it calls self.__getattr__(name), here comes the unwanted recursion.
I would fix it with:
class Foo(object):
def __init__(self):
self.a = 1
self.c = 0
def __getattr__(self, name):
self.__dict__['c'] += 1
print('getattribute')
try:
return self.__dict__[name]
except KeyError:
print('No')
return None
fo = Foo()
print(fo.a)
print(fo.b)
print(fo.c)
The problem comes from print(fo.b).
Since b is not defined as a member of Foo, fo.b results in a call to fo.__getattr__('b').
Then, hasattr(self, name), which is tantamount to hasattr(fo, 'b') calls itself gettatr(fo, 'b'), as stated in hte documentation.
Hence an infinite recursion, resulting in a RecursionError in Python 3.
Since getting fo.b does not really make sense if you know that Foo does not have a b member, a first fix I would suggest is to define that member.
class Foo(object):
def __init__(self):
...
self.b = 1
...
Your code then outputs
1
1
0
A more clever fix would be to check if the name argument passed to __getattr__ is 'b', or depending to your needs, different from 'a' and 'c'.
In this situation, you can force __getattr__ to define the requested unfound member.
class Foo(object):
...
def __getattr__(self, name):
if name == 'b':
self.b = 2
return self.b
else:
...
Alternatively:
class Foo(object):
...
def __getattr__(self, name):
if name not in ('a', 'c'):
self.b = 2
return self.b
else:
...
class Shadow(object):
pass
class Test(object):
a = 1
b = 2
_shadow = Shadow()
def __getattribute__(self, name):
try:
return object.__getattribute__(self._shadow, name)
except: print "not shadowed"
return object.__getattribute__(self, name)
With the above piece of code I would like to implement the following behavior:
>>>t = Test()
>>>t.a
1
>>>t._shadow.a = 17
>>>t.a
17
>>>t.b
2
The code works, but it will print "not shadowed" M-times (until recursion depth is reached). The question is why, there shouldn't be any recursion involved, I'm calling object.__getattribute__ and not self.__getattribute__.
__getattribute__ is called for all attribute access, including for self._shadow. But since you have __getattribute__ overridden, self._shadow triggers an infinite recursion.
The only work-around for that is to use object.__getattribute__, or better, super(Test, self).__getattribute__, to retrieve the _shadow attribute:
class Test(object):
a = 1
b = 2
_shadow = Shadow()
def __getattribute__(self, name):
shadow = super(Test, self).__getattribute__('_shadow')
try:
return getattr(shadow, name)
except AttributeError:
print "not shadowed"
return super(Test, self).__getattribute__(name)
There is no need to use object.__getattribute__ for the attribute access on the shadow object. Don't use Pokemon-style exception handling (you don't want to catch them all); catch only the specific AttributeError exception here.
Demo:
>>> t = Test()
>>> t.a
not shadowed
1
>>> t._shadow.a = 42
not shadowed
>>> t.a
42
Note that here too, accessing t._shadow triggers the 'not shadowed' message as it goes through the __getattribute__ handler.