Whenever I decorate a method of a class outside the class definition and use it, it throws me a TypeError saying unbound method with class instance as first argument .
I am setting back the decorated method using setattr() method. For eg:
class A(object):
#classmethod
def demo_method(cls, a):
print a
def decorator(function):
from functools import wraps
#wraps(function)
def wrapper(*args, **kwargs):
return_value = function(*args, **kwargs)
return return_value
return wrapper
setattr(A, 'demo_method', decorator(A.demo_method))
A.demo_method(1)
It throws following error:
TypeErrorTraceback (most recent call last)
<ipython-input-13-447ee51677f4> in <module>()
17 setattr(A, 'demo_method', decorator(A.demo_method))
18
---> 19 A.demo_method(1)
20
TypeError: unbound method demo_method() must be called with A instance as first argument (got int instance instead)
The behavior you're seeing is actually not related to the use of decorator itself. You can also say just this:
class C(object):
pass
setattr(C, 'm', lambda a: a) # actually also just: C.m = lambda a: a
print(C.m(1))
And you would still get essentially the same error. You can read up a bit more in Python 2 documentation for User-defined methods, namely these bits:
Note that the transformation from function object to (unbound or bound) method object happens each time the attribute is retrieved from the class or instance. ... Also notice that this transformation only happens for user-defined functions; other callable objects (and all non-callable objects) are retrieved without transformation. It is also important to note that user-defined functions which are attributes of a class instance are not converted to bound methods; this only happens when the function is an attribute of the class.
And the exception you're hitting because:
When an unbound user-defined method object is called, the underlying function (im_func) is called, with the restriction that the first argument must be an instance of the proper class (im_class) or of a derived class thereof.
If you had closer look at C.m, you would see:
>>> print(C.m)
<unbound method C.<lambda>>
>>> print(C().m)
<bound method C.<lambda> of <__main__.C object at 0x7f6f6c5fa850>>
Whereas with Python 3 (as was hinted in a comment this construct would have passed) the behavior would be:
>>> print(C.m)
<function <lambda> at 0x7f69fbe8d0d0>
>>> print(C().m)
<bound method <lambda> of <__main__.C object at 0x7f69fbe89940>>
Note C.m in the latter case is still accessed as a (plain) function and not an (unbound) method.
I am not entirely sure what exactly you're ultimately going for and what would be the most helpful direction, but to just get the same behavior you'd get in Python 3 for this code, you could change (not that I'd recommend it) C.m() call to C.m.__func__() one to access the underlying function directly. Or with your example:
A.demo_method.__func__(1)
I was actually going through descriptors python docs and came across this example
>>> class D(object):
def f(self, x):
return x
>>> d = D()
>>> D.__dict__['f'] # Stored internally as a function
<function f at 0x00C45070>
>>> id(D.__dict__['f']) # Memory location
49294384
>>> D.f # Get from a class becomes an unbound method
<unbound method D.f>
>>> id(D.f )
48549440
>>> d.f # Get from an instance becomes a bound method
<bound method D.f of <__main__.D object at 0x00B18C90>>
>>> id(d.f)
48549440
So from the above code, I understood that python stores the function definition/declaration of a class as a separate object internally inside class __dict__ variable, when we access directly using __dict__ variable it has memory location as 49294384
But why does it is showing as different function/method object with different memory location 48549440 when accessed through Class or Object? like D.f and d.f
was it not supposed to refer to the same object when we access using the __dict__ variable?. If so why?
D.f is a function taking one argument (self)
x = D.f
x(d)
d.f is a "bound method", i.e. a function where the self argument has already been filled in. You can say
x = d.f
x()
Therefor it cannot be the same thing as D.f, and has to be on a different location.
xtofi explained the difference between descriptor objects (unbound) and bound methods.
I think the missing part is that bound methods are not kept in memory, and they are actually created every time you access them. (You may get the same memory location, but it's not the same object).
Why?
Because on every call on a descriptor may result in a different behavior. Here is an example to explain this idea.
class A(object):
i = 'a'
#property
def name(self):
if A.i == 'a':
return self.fa()
else:
return self.fb()
def fa(self):
print 'one function'
def fb(self):
print 'another function'
Calling the function name on an instance of A results in different function calls.
What is the difference between the following class methods?
Is it that one is static and the other is not?
class Test(object):
def method_one(self):
print "Called method_one"
def method_two():
print "Called method_two"
a_test = Test()
a_test.method_one()
a_test.method_two()
In Python, there is a distinction between bound and unbound methods.
Basically, a call to a member function (like method_one), a bound function
a_test.method_one()
is translated to
Test.method_one(a_test)
i.e. a call to an unbound method. Because of that, a call to your version of method_two will fail with a TypeError
>>> a_test = Test()
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
You can change the behavior of a method using a decorator
class Test(object):
def method_one(self):
print "Called method_one"
#staticmethod
def method_two():
print "Called method two"
The decorator tells the built-in default metaclass type (the class of a class, cf. this question) to not create bound methods for method_two.
Now, you can invoke static method both on an instance or on the class directly:
>>> a_test = Test()
>>> a_test.method_one()
Called method_one
>>> a_test.method_two()
Called method_two
>>> Test.method_two()
Called method_two
Methods in Python are a very, very simple thing once you understood the basics of the descriptor system. Imagine the following class:
class C(object):
def foo(self):
pass
Now let's have a look at that class in the shell:
>>> C.foo
<unbound method C.foo>
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
As you can see if you access the foo attribute on the class you get back an unbound method, however inside the class storage (the dict) there is a function. Why's that? The reason for this is that the class of your class implements a __getattribute__ that resolves descriptors. Sounds complex, but is not. C.foo is roughly equivalent to this code in that special case:
>>> C.__dict__['foo'].__get__(None, C)
<unbound method C.foo>
That's because functions have a __get__ method which makes them descriptors. If you have an instance of a class it's nearly the same, just that None is the class instance:
>>> c = C()
>>> C.__dict__['foo'].__get__(c, C)
<bound method C.foo of <__main__.C object at 0x17bd4d0>>
Now why does Python do that? Because the method object binds the first parameter of a function to the instance of the class. That's where self comes from. Now sometimes you don't want your class to make a function a method, that's where staticmethod comes into play:
class C(object):
#staticmethod
def foo():
pass
The staticmethod decorator wraps your class and implements a dummy __get__ that returns the wrapped function as function and not as a method:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Hope that explains it.
When you call a class member, Python automatically uses a reference to the object as the first parameter. The variable self actually means nothing, it's just a coding convention. You could call it gargaloo if you wanted. That said, the call to method_two would raise a TypeError, because Python is automatically trying to pass a parameter (the reference to its parent object) to a method that was defined as having no parameters.
To actually make it work, you could append this to your class definition:
method_two = staticmethod(method_two)
or you could use the #staticmethod function decorator.
>>> class Class(object):
... def __init__(self):
... self.i = 0
... def instance_method(self):
... self.i += 1
... print self.i
... c = 0
... #classmethod
... def class_method(cls):
... cls.c += 1
... print cls.c
... #staticmethod
... def static_method(s):
... s += 1
... print s
...
>>> a = Class()
>>> a.class_method()
1
>>> Class.class_method() # The class shares this value across instances
2
>>> a.instance_method()
1
>>> Class.instance_method() # The class cannot use an instance method
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method instance_method() must be called with Class instance as first argument (got nothing instead)
>>> Class.instance_method(a)
2
>>> b = 0
>>> a.static_method(b)
1
>>> a.static_method(a.c) # Static method does not have direct access to
>>> # class or instance properties.
3
>>> Class.c # a.c above was passed by value and not by reference.
2
>>> a.c
2
>>> a.c = 5 # The connection between the instance
>>> Class.c # and its class is weak as seen here.
2
>>> Class.class_method()
3
>>> a.c
5
method_two won't work because you're defining a member function but not telling it what the function is a member of. If you execute the last line you'll get:
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
If you're defining member functions for a class the first argument must always be 'self'.
Accurate explanation from Armin Ronacher above, expanding on his answers so that beginners like me understand it well:
Difference in the methods defined in a class, whether static or instance method(there is yet another type - class method - not discussed here so skipping it), lay in the fact whether they are somehow bound to the class instance or not. For example, say whether the method receives a reference to the class instance during runtime
class C:
a = []
def foo(self):
pass
C # this is the class object
C.a # is a list object (class property object)
C.foo # is a function object (class property object)
c = C()
c # this is the class instance
The __dict__ dictionary property of the class object holds the reference to all the properties and methods of a class object and thus
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
the method foo is accessible as above. An important point to note here is that everything in python is an object and so references in the dictionary above are themselves pointing to other objects. Let me call them Class Property Objects - or as CPO within the scope of my answer for brevity.
If a CPO is a descriptor, then python interpretor calls the __get__() method of the CPO to access the value it contains.
In order to determine if a CPO is a descriptor, python interpretor checks if it implements the descriptor protocol. To implement descriptor protocol is to implement 3 methods
def __get__(self, instance, owner)
def __set__(self, instance, value)
def __delete__(self, instance)
for e.g.
>>> C.__dict__['foo'].__get__(c, C)
where
self is the CPO (it could be an instance of list, str, function etc) and is supplied by the runtime
instance is the instance of the class where this CPO is defined (the object 'c' above) and needs to be explicity supplied by us
owner is the class where this CPO is defined(the class object 'C' above) and needs to be supplied by us. However this is because we are calling it on the CPO. when we call it on the instance, we dont need to supply this since the runtime can supply the instance or its class(polymorphism)
value is the intended value for the CPO and needs to be supplied by us
Not all CPO are descriptors. For example
>>> C.__dict__['foo'].__get__(None, C)
<function C.foo at 0x10a72f510>
>>> C.__dict__['a'].__get__(None, C)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__get__'
This is because the list class doesnt implement the descriptor protocol.
Thus the argument self in c.foo(self) is required because its method signature is actually this C.__dict__['foo'].__get__(c, C) (as explained above, C is not needed as it can be found out or polymorphed)
And this is also why you get a TypeError if you dont pass that required instance argument.
If you notice the method is still referenced via the class Object C and the binding with the class instance is achieved via passing a context in the form of the instance object into this function.
This is pretty awesome since if you chose to keep no context or no binding to the instance, all that was needed was to write a class to wrap the descriptor CPO and override its __get__() method to require no context.
This new class is what we call a decorator and is applied via the keyword #staticmethod
class C(object):
#staticmethod
def foo():
pass
The absence of context in the new wrapped CPO foo doesnt throw an error and can be verified as follows:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Use case of a static method is more of a namespacing and code maintainability one(taking it out of a class and making it available throughout the module etc).
It maybe better to write static methods rather than instance methods whenever possible, unless ofcourse you need to contexualise the methods(like access instance variables, class variables etc). One reason is to ease garbage collection by not keeping unwanted reference to objects.
that is an error.
first of all, first line should be like this (be careful of capitals)
class Test(object):
Whenever you call a method of a class, it gets itself as the first argument (hence the name self) and method_two gives this error
>>> a.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
The second one won't work because when you call it like that python internally tries to call it with the a_test instance as the first argument, but your method_two doesn't accept any arguments, so it wont work, you'll get a runtime error.
If you want the equivalent of a static method you can use a class method.
There's much less need for class methods in Python than static methods in languages like Java or C#. Most often the best solution is to use a method in the module, outside a class definition, those work more efficiently than class methods.
The call to method_two will throw an exception for not accepting the self parameter the Python runtime will automatically pass it.
If you want to create a static method in a Python class, decorate it with the staticmethod decorator.
Class Test(Object):
#staticmethod
def method_two():
print "Called method_two"
Test.method_two()
Please read this docs from the Guido First Class everything Clearly explained how Unbound, Bound methods are born.
Bound method = instance method
Unbound method = static method.
The definition of method_two is invalid. When you call method_two, you'll get TypeError: method_two() takes 0 positional arguments but 1 was given from the interpreter.
An instance method is a bounded function when you call it like a_test.method_two(). It automatically accepts self, which points to an instance of Test, as its first parameter. Through the self parameter, an instance method can freely access attributes and modify them on the same object.
Unbound Methods
Unbound methods are methods that are not bound to any particular class instance yet.
Bound Methods
Bound methods are the ones which are bound to a specific instance of a class.
As its documented here, self can refer to different things depending on the function is bound, unbound or static.
Take a look at the following example:
class MyClass:
def some_method(self):
return self # For the sake of the example
>>> MyClass().some_method()
<__main__.MyClass object at 0x10e8e43a0># This can also be written as:>>> obj = MyClass()
>>> obj.some_method()
<__main__.MyClass object at 0x10ea12bb0>
# Bound method call:
>>> obj.some_method(10)
TypeError: some_method() takes 1 positional argument but 2 were given
# WHY IT DIDN'T WORK?
# obj.some_method(10) bound call translated as
# MyClass.some_method(obj, 10) unbound method and it takes 2
# arguments now instead of 1
# ----- USING THE UNBOUND METHOD ------
>>> MyClass.some_method(10)
10
Since we did not use the class instance — obj — on the last call, we can kinda say it looks like a static method.
If so, what is the difference between MyClass.some_method(10) call and a call to a static function decorated with a #staticmethod decorator?
By using the decorator, we explicitly make it clear that the method will be used without creating an instance for it first. Normally one would not expect the class member methods to be used without the instance and accesing them can cause possible errors depending on the structure of the method.
Also, by adding the #staticmethod decorator, we are making it possible to be reached through an object as well.
class MyClass:
def some_method(self):
return self
#staticmethod
def some_static_method(number):
return number
>>> MyClass.some_static_method(10) # without an instance
10
>>> MyClass().some_static_method(10) # Calling through an instance
10
You can’t do the above example with the instance methods. You may survive the first one (as we did before) but the second one will be translated into an unbound call MyClass.some_method(obj, 10) which will raise a TypeError since the instance method takes one argument and you unintentionally tried to pass two.
Then, you might say, “if I can call static methods through both an instance and a class, MyClass.some_static_method and MyClass().some_static_method should be the same methods.” Yes!
I noticed that if I define a class attribute equal to a function when I create an instance of that class the attribute becomes a bound method. Can someone explain me the reason of this behaviour?
In [9]: def func():
...: pass
...:
In [10]: class A(object):
....: f = func
....:
In [11]: a = A()
In [12]: a.f
Out[12]: <bound method A.func of <__main__.A object at 0x104add190>>
In [13]: a.f()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-13-19134f1ad9a8> in <module>()
----> 1 a.f()
global a.f = <bound method A.func of <__main__.A object at 0x104add190>>
TypeError: func() takes no arguments (1 given)
You assigned a function to the attribute A.f (the attribute f of the class A). The attribute A.f was defined as part of the class. It is a function, so it is by default an instance method of that class.
Creating an instance (named a) of class A causes that instance to have an attribute f, and you access that by the name a.f. This is a bound method (cause it's bounded to the object a; further explanation here).
Every instance method, when it is called, automatically receives the instance as its first argument (conventionally named self). Other types of method are possible: - see class methods and static methods.
For this reason the error says that func takes no arguments (as it's defined as def func():) but received 1 (self).
To do what you want, you should tell python that you're using a static method
def func():
pass
class A(object):
f = staticmethod(func)
Python is not a message based OO system1. Instead, similar to JavaScript, properties are resolved to first-class functions and then invoked; the behavior differs a bit in the mechanics of such, as discovered.
In Python the requirement is that methods have at least one parameter, normally called self, that will be automatically supplied the associated instance when it is invoked as a method.
Furthermore (and perhaps to the point of the question), Python does not differentiate between using def f.. or f = some_func() when establishing instance member bindings; arguably this matches behavior outside of classes.
In the example, assigning the function to the instance 'makes it expect to be treated like an instance method'. It is the exact same - parameterless - function called in both cases; only the future usage of such is relevant.
Now, unlike JavaScript, Python handles methods and object association through the concept of bound methods - functions resolved as methods are always 'bound'.
The behavior of a.f returning a bound method - function that will automatically supply the bound object to the first parameter as self - is done independently of the source of the function. In this case that means the parameterless function cannot be used when it is 'bound' as it does not accept a self parameter.
As a demonstration, the following will fail in the same way because the source underlying method does not meet the minimum requirements of accepting the instance as an argument:
g = a.f
g()
In this case calling g() is equivalent to calling func(a).
1 For comparison, Java, C#, Ruby, and SmallTalk are message based OO systems - in these an object is told to invoke a method by a 'name', instead of resolving a method (or function) as a value which can be invoked.
A little late to the party, but another viable solution is in storing the function as dictionary, which is an attribute of the class.
# Some arbitrary function
def f(x):
return x
# A class, which will wrap the function as a dictionary object, which is a class attr
class Test:
def __init__(self,f):
self.f = {'f':f}
def func(self,x):
return self.f['f'](x)
# Now let's test the Test object
t = Test(f=f)
t.func(3)
>>>
3
This approach is more verbose than the accepted answer, however, it doesn't get into staticmethod, decorators, or other advanced topics. So, this will work in a pinch if you're intimidated by the other, preferred approach.
Edit: If you want this to robust enough to handle keyword arguments:
class Test:
def __init__(self,f):
self.f = {'f':f}
def func(self,p):
return self.f['f'](**p)
def f(**p):
return p['x'] * p['y']
t = Test(f=f)
t.func({'x':2,'y':3})
>>>
6
So I know in python everything is an 'object' meaning that it can be passed as an argument to a method. But I'm trying to understand how exactly does this work. So I was trying out the following example:
class A:
def __init__(self):
self.value = 'a'
def my_method(self)
print self.value
class B:
def __init__(self):
self.values = 'b'
def my_method(self, method):
method()
a = A()
b = B()
b.my_method(a.my_method)
Now first this was written just to see how things work. I know I should for example check if my_method 's argument is callable. Now my question is:
How exactly is the method passed here? I mean the output I'm recieving is 'a' so I'm guessing that when a object method is passed as parameter so is the actual object ? In this case when I pass a.my_method the instance a is also passed ?
When you access a.my_method Python sees that it is an attribute of the class and that A.my_method has a method __get__() so it calls A.my_method.__get__(a), that method creates a new object (the 'bound method') which contains both a reference to A.my_method and a reference to a itself. When you call the bound method it passes the call back through to the underlying method.
This happens every time you call any method whether you call it directly or, as in your code, delay the actual call until later. You can find more description of the descriptor protocol as it is known at http://docs.python.org/reference/datamodel.html#descriptors
How exactly is the method passed here?
This is easily answered: in Python, everything is an object :) Also functions/methods, which are specific function objects which can be passed as parameters.
In this case when I pass a.my_method the instance a is also passed?
Yes, in this case the instance a is also passed, though embedded in the function object and cannot be retrieved. On the contrary you could pass the function itself and supply it with an instance. Consider the following example:
b = B()
func = A.my_method # Attention, UPPERCASE A
func(b)
This would call the A classes method my_method with an instance of B.
Functions are 'first-class objects' in Python. What that means is that a function is an object as much as a list or an integer. When you define a function, you are actually binding the function object to the name you use in the def.
Functions can be bound to other names or passed as parameters to functions (or stored in lists, or...).
source: http://mail.python.org/pipermail/tutor/2005-October/042616.html
a.my_method returns not exactly the function you defined in the class A (you can get it via A.my_method) but an object called a 'bound method' which contains both the original function (or 'unbound method' in Python 2, IIRC) and the object from which it was retrieved (the self argument).