I am in the process of learning Python 3 and just ran into the getattr function. From what I can tell, it is invoked when the attribute call is not found in the class definition as a function or a variable.
In order to understand the behaviour, I wrote the following test class (based on what I've read):
class Test(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def __getattr__(self, itm):
if itm is 'test':
return lambda x: "%s%s" % (x.foo, x.bar)
raise AttributeError(itm)
And I then initate my object and call the non-existent function test which, expectedly, returns the reference to the function:
t = Test("Foo", "Bar")
print(t.test)
<function Test.__getattr__.<locals>.<lambda> at 0x01A138E8>
However, if I call the function, the result is not the expected "FooBar", but an error:
print(t.test())
TypeError: <lambda>() missing 1 required positional argument: 'x'
In order to get my expected results, I need to call the function with the same object as the first parameter, like this:
print(t.test(t))
FooBar
I find this behaviour rather strange, as when calling p.some_function(), is said to add p as the first argument.
I would be grateful if someone could shine some light over this headache of mine. I am using PyDev in Eclipse.
__getattr__ return values are "raw", they don't behave like class attributes, invoking the descriptor protocol that plain methods involve that causes the creation of bound methods (where self is passed implicitly). To bind the function as a method, you need to perform the binding manually:
import types
...
def __getattr__(self, itm):
if itm is 'test': # Note: This should really be == 'test', not is 'test'
# Explicitly bind function to self
return types.MethodType(lambda x: "%s%s" % (x.foo, x.bar), self)
raise AttributeError(itm)
types.MethodType is poorly documented (the interactive help is more helpful), but basically, you pass it a user-defined function and an instance of a class and it returns a bound method that, when called, implicitly passes that instance as the first positional argument (the self argument).
Note that in your specific case, you could just rely on closure scope to make a zero-argument function continue to work:
def __getattr__(self, itm):
if itm is 'test': # Note: This should really be == 'test', not is 'test'
# No binding, but referring to self captures it in closure scope
return lambda: "%s%s" % (self.foo, self.bar)
raise AttributeError(itm)
Now it's not a bound method at all, just a function that happens to have captured self from the scope in which it was defined (the __getattr__ call). Which solution is best depends on your needs; creating a bound method is slightly slower, but gets a true bound method, while relying on closure scope is (trivially, ~10ns out of >400ns) faster, but returns a plain function (which may be a problem if, for example, it's passed as a callback to code that assumes it's a bound method and can have __self__ and __func__ extracted separately for instance).
To get what you want, you need a lambda that doesn't take arguments:
return lambda: "%s%s" % (self.foo, self.bar)
But you should really use a property for this, instead.
class Test(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
#property
def test(self):
return "{}{}".format(self.foo, self.bar)
t = Test("Foo", "Bar")
print(t.test)
# FooBar
Note the lack of parentheses.
If you're absolutely determined that it must be a function, do this:
class Test(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
#property
def test(self):
return lambda: "{}{}".format(self.foo, self.bar)
t = Test("Foo", "Bar")
print(t.test())
# FooBar
You need to create something that behaves like a bound method, you could simply use functools.partial to bind the instance to the function:
from functools import partial
class Test(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def __getattr__(self, itm):
if itm == 'test': # you shouldn't use "is" for comparisons!
return partial(lambda x: "%s%s" % (x.foo, x.bar), self)
raise AttributeError(itm)
The test:
t = Test("Foo", "Bar")
print(t.test)
# functools.partial(<function Test.__getattr__.<locals>.<lambda> at 0x0000020C70CA6510>, <__main__.Test object at 0x0000020C7217F8D0>)
print(t.test())
# FooBar
"I find this behaviour rather strange, as when calling
p.some_function(), is said to add p as the first argument."
some_function is actually a method, which is why it gets passed an instance implicitly when the method is "bound to an object." But plain functions don't work that way, only functions defined in the class body have this magic applied to them automagically. And actually, unbound methods (accessed via the class directly) function the same as normal functions! The terminology "bound and unbound" methods no longer applies, because in Python 3 we only have methods and functions (getting rid of the distinction between unbound methods and plain functions). When an instance is instantiated, accessing the attribute returns a method which implicitly calls the instance on invocation.
>>> class A:
... def method(self, x):
... return x
...
>>> a.method
<bound method A.method of <__main__.A object at 0x101a5b3c8>>
>>> type(a.method)
<class 'method'>
However, if you access the attribute of the class you'll see it's just a function:
>>> A.method
<function A.method at 0x101a64950>
>>> type(A.method)
<class 'function'>
>>> a = A()
Now, observe:
>>> bound = a.method
>>> bound(42)
42
>>> unbound = A.method
>>> unbound(42)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() missing 1 required positional argument: 'x'
But this is the magic of classes. Note, you can even add functions to classes dynamically, and they get magically turned into methods when you invoke them on an instance:
>>> A.method2 = lambda self, x: x*2
>>> a2 = A()
>>> a2.method2(4)
8
And, as one would hope, the behavior still applies to objects already created!
>>> a.method2(2)
4
Note, this doesn't work if you dynamically add to an instance:
>>> a.method3 = lambda self, x: x*3
>>> a.method3(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: <lambda>() missing 1 required positional argument: 'x'
You have to do the magic yourself:
>>> from types import MethodType
>>> a.method4 = MethodType((lambda self, x: x*4), a)
>>> a.method4(4)
16
>>>
Notice that if you do print(t.__getattr__) you get something like <bound method Test.__getattr__ of <__main__.Test object at 0x00000123FBAE4DA0>>. The key point is that methods defined on an object are said to be 'bound' and so always take the object as the first parameter. Your lambda function is just an anonymous function not 'bound' to anything, so for it to access the object it needs to be explicitly passed in.
I presume you are only doing this to experiment with using `__getattr__', as what you are doing could be much more easily achieved by making your lambda a method on the object.
I was trying to store reference to unbound method and noticed that it is being automatically bound. See example below. Is there more elegant way to store unbound method in the class without binding it?
def unbound_method():
print("YEAH!")
class A:
bound_method = unbound_method
unbound_methods = [unbound_method]
a = A()
a.unbound_methods[0]() # succeeds
a.bound_method() # fails
# TypeError: unbound_method() takes 0 positional arguments but 1 was given
This is not a standard "do you know about #staticmethod?" question.
What I'm trying to achieve is provide a way for children of the class provide another handler or certain situations. I do not control the unbound_method itself, it is provided from some library.
def unbound_method_a():
print("YEAH!")
def unbound_method_b():
print("WAY MORE YEAH!")
class A:
bound_method = unbound_method_a
class B(A):
bound_method = unbound_method_b
a = A()
a.bound_method() #fails
# print("YEAH!")
b = B()
b.bound_method() #fails
# print("WAY MORE YEAH!")
It can be achieved by wrapping the unbound method some dummy object like array, or in a bound method, just to drop self reference like this:
def unbound_method_a():
print("YEAH!")
def unbound_method_b():
print("WAY MORE YEAH!")
class A:
def call_unbound_method(self):
return unbound_method_a()
class B(A):
def call_unbound_method(self):
return unbound_method_b()
a = A()
a.call_unbound_method()
# print("YEAH!")
b = B()
b.call_unbound_method()
# print("WAY MORE YEAH!")
Not as far as I know. Would it be so bad if you just replace
a.bound_method()
with
A.bound_method()
?
I can't think of a situation in which the first one can't be replaced by the second one.
class A:
def foo(self):
print "foo()"
getattr(A, foo) # True
A.foo() # error
getattr(A(), foo) # True
A().foo() # prints "foo()"
That being said, here is my problem:
I wish to store test case meta information as attributes of the Test Case class objects themselves, not on instances of them.
I have a list of attribute names to extract, but if there is an instance method of the same name, then getattr(class_obj, attr) will return True, but getattr(class_obj, attr)() raises an Error.
Is there a way to tell getattr not to include attributes of the instantiated class and only of the class object itself?
EDIT: I tried accessing class_obj.__dict__ directly (which I understand is bad practice), but it does not include some attributes like __name__
EDIT: Rephrase of the question. Is there a way to differentiate between methods of the class obj and the methods of an instance of the class?
Is this good enough?
import types
class Test(object):
#staticmethod
def foo():
print 'foo'
def bar(self):
print 'bar'
In combination with:
>>>(isinstance(getattr(Test, 'foo'), types.FunctionType),
isinstance(getattr(Test, 'bar'), types.FunctionType))
True, False
You can also use the inspect module:
>>> inspect.isfunction(Test.foo)
True
>>> inspect.isfunction(Test.bar)
False
With a little additional work you can even distinguish class methods from instance methods and static methods:
import inspect
def get_type(cls, attr):
try:
return [a.kind for a in inspect.classify_class_attrs(cls) if a.name == attr][0]
except IndexError:
return None
class Test(object):
#classmethod
def foo(cls):
print 'foo'
def bar(self):
print 'bar'
#staticmethod
def baz():
print 'baz'
You can use it as:
>>> get_type(Test, 'foo')
'class method'
>>> get_type(Test, 'bar')
'method'
>>> get_type(Test, 'baz')
'static method'
>>> get_type(Test, 'nonexistant')
None
Your results from an incorrect definition of foo, not any underlying semantics of class attributes. By default, a function declared inside a class is an instance method, which must take at least one argument, an instance of the class. Conventionally, it is referred to as self:
class A:
def foo(self):
print "foo()"
Normally, you would call such a method like this:
a = A()
a.foo() # passes the object 'a' implicitly as the value of the parameter 'self'
but this is legal as well
a = A()
A.foo(a) # pass the object 'a' explicitly as the value of the parameter 'self'
In order to define a function inside a class that doesn't take any such implicit arguments, you need to decorate it with the #staticmethod decorator:
class A:
#staticmethod
def foo():
print "foo()"
Now, you can call foo the way you tried to previously:
>>> A.foo()
foo()
You want something like this:
from inspect import ismethod
from collections import Callable
def can_i_call(func):
if not isinstance(func, Callable):
# not a callable at all
return False
if not ismethod(func):
# regular function or class or whatever
return True
# func is a method
return func.im_self is not None
Note: this will only test whether or not an attempt to call will error out because you're calling an unbound method without a self. It doesn't guarantee that func() will succeed, i.e. not fail for any other reason.
class Test:
def func():
print('func')
test1 = Test()
test2 = Test()
test1.func() #TypeError: fun1() takes no arguments (1 given)
test2.newfunc = Test.func
test2.newfunc()#It goes well
# update part
def foo(self):
pass
Test.foo = foo
test1.foo # it is bound method
test2.foo = foo
test2.foo # it is function
# end
Is there any difference between the two ways ?
Thanks.
# update part
def foo(self):
pass
Test.foo = foo
test1.foo # it is bound method
test2.foo = foo
test2.foo # it is function
# end
Note that what's important is that the retrieval should take place in class instead of instance.
Methods of a class that are called on an instance of the class are passed the instance reference as an argument automatically. Thus, they're usually declared with a self argument:
class Test:
def func(self):
print('func')
test1 = Test()
test1.func() # self = test1
A bit of investigation:
>>> test1.func
<bound method Test.func of <__main__.Test instance at 0x800f211b8>>
>>> Test.func
<unbound method Test.func>
Then user-defined bound method (test1.func in our case) is called, this call is actually performed as Test.func(test1) - class instance is always passed as first argument.
See much more on this topic in Python Data model.
Edit
Output above is from Python 2.6. Since Test.func() works for you, I assume that you are using Python 3. In this case output will be the next:
>>> test1.func
<bound method Test.func of <__main__.Test object at 0xb747624c>>
>>> Test.func
<function func at 0xb74719ec>
As you see, Test.func is a simple function, so no 'magic' will be added while calling it.
I was wondering if anyone could explain and offer a solution to this issue:
$ cat object-override-methods.py
class A:
def foo(self):
return 1
class B:
def foo(self):
return 1
for klass in A, B:
orig_foo = klass.foo
def foo(self):
return orig_foo(self) * 2
klass.foo = foo
A().foo()
B().foo()
$ python object-override-methods.py
Traceback (most recent call last):
File "object-override-methods.py", line 15, in <module>
A().foo()
File "object-override-methods.py", line 12, in foo
return orig_foo(self) * 2
TypeError: unbound method foo() must be called with B instance as first argument (got A instance instead)
Thanks in advance.
orig_foo is a global variable which changes value with each pass through the loop. After the loop is done, orig_foo refers to B.foo.
The inner functions foo (one or each pass through the loop) both use the global value for orig_foo when they are called. So they both call B.foo(self).
When calling an "unbound method" like orig_foo, Python2 checks that the first argument is an instance of the appropriate class. A().foo() does not pass this check. (Interestingly, this check was removed in Python3, so there would be no TypeError raised, and this bug may become harder to find.)
To fix this, you must bind the value of orig_foo to the appropriate klass.
You can do that by making orig_foo a local variable of foo. One way to do that is to make orig_foo an argument of foo with a default value. Python binds default values at the time a function is defined. So orig_foo=orig_foo binds the local variable orig_foo to the current value of the klass.foo:
for klass in A, B:
orig_foo = klass.foo
def foo(self, orig_foo=orig_foo):
return orig_foo(self) * 2
klass.foo = foo
Because orig_foo is defined at global scope, you're trampling on its value each time round the loop. That trampled value is then shared by each of your new foo methods.
A simple fix is to move the code into a function, like this:
def rebind_foo(klass):
orig_foo = klass.foo
def foo(self):
return orig_foo(self) * 2
klass.foo = foo
for klass in A, B:
rebind_foo(klass)
That ensures that each new foo method gets its own value of orig_foo.