Could somebody please elaborate this paragraph to me:
If you still don’t understand how methods work, a look at the
implementation can perhaps clarify matters. When an instance attribute
is referenced that isn’t a data attribute, its class is searched. If
the name denotes a valid class attribute that is a function object, a
method object is created by packing (pointers to) the instance object
and the function object just found together in an abstract object:
this is the method object. When the method object is called with an
argument list, a new argument list is constructed from the instance
object and the argument list, and the function object is called with
this new argument list.
Especially the part:
If the name denotes a valid class attribute that is a function object,
a method object is created by packing (pointers to) the instance
object and the function object just found together in an abstract
object: this is the method object.
Difference between Method Object and Function Object?
Thanks.
A method object is a wrapper for a function object that allows the function to be attached (bound) to an instance. The method object knows what instance it is bound to, and, when it is called, it calls the wrapped function, passing the instance as the first argument. In other words, the method object is responsible for the explicit passing of self that is characteristic of Python.
Related
Let's say we have a class, and we are calling it's method before creating an object:
class ParentClass:
def MyMethod(self):
self.a=5
print("I am Method")
ParentClass.MyMethod(ParentClass)
Why do we get a result?
Also, why hasattr(ParentClass,'a') is showing that instance variable is created?
You get a result because you ParentClass.MyMethod evaluates to a regular function that takes one argument, which you supplied. The function doesn't care what the type of self is, as long as you can define an attribute named a for it. The result of ParentClass.MyMethod(ParentClass) is to add an attribute named a to ParentClass.
ParentClass().MyMethod, on the other hand, produces an instance of method that wraps the function, due to the descriptor protocol. The method, when called, simply calls the function with the instance as the first argument and the rest of its own arguments. Because the function doesn't expect any more arguments, you would get a TypeError.
I understand the #staticmethod decorator in practice. But a bug in mocking a static method led me down the Python semantics rabbit hole. This description in The standard type hierarchy section is confusing me:
Static method objects provide a way of defeating the transformation
of function objects to method objects described above.
A static method object is a wrapper around any other object, usually a
user-defined method object. When a static method object is retrieved
from a class or a class instance, the object actually returned is the
wrapped object, which is not subject to any further transformation.
Static method objects are not themselves callable, although the
objects they wrap usually are. Static method objects are created by
the built-in staticmethod() constructor.
The staticmethod() constructor takes a function object as sole argument. How can it wrap any other object than a function object? Even if this doesn't fail, how does it make any sense?
How is it usually a wrapper around a user-defined method object instead of a function object? User-defined method objects, when called, add the object they're called on to the start of the argument list, then call the function object stored on the class (ignoring all the various special cases).
How is it that static method objects are not themselves callable? How do calls to these work, then?
You can see that staticmethod can take any argument:
>>> x = staticmethod(3)
and that it is, indeed, not callable:
>>> x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'staticmethod' object is not callable
staticmethod doesn't do much more than store a reference to its argument. The "magic" happens when you try to access a staticmethod object as the attribute of a class object or an instance of a class. When you do, you get the result of the staticmethod method's __get__ method, which is... the thing you originally wrapped.
>>> x.__get__(x)
3
Don't worry about why we passed x as an argument; suffice it to say, staticmethod.__get__ mostly ignores its argument(s).
When you wrap a function in a class statement, the staticmethod saves a reference to the function, to be called later when you ask for it.
>>> class Foo(object):
... #staticmethod
... def x():
... pass
...
>>> type(Foo.__dict__['x'])
<type 'staticmethod'>
>>> type(Foo.x)
<type 'function'>
Instance methods work the way they do because function.__get__ returns an instance of method, which is in some sense just the original function partially applied the instance that invokes it. You may have seen that x.foo() is the same as type(x).foo(x). The reason that is true is because x.foo first resolves to type(x).foo, which itself evaluates to type(x).__dict__['foo'].__get__(x, type(x). The return value of function.__get__ is basically a wrapper around the function foo, but with x already supplied as the first argument.
staticmethod's main purpose is to provide a different __get__ method.
Incidentally, classmethod serves the same purpose. classmethod.__get__ returns something that calls the wrapped function with the class as the first argument, whether you invoke the class method from an instance of the class or the class itself.
How can it wrap any other object than a function object?
Pretty easily.
class Example(object):
example = staticmethod(5)
print(Example.example) # prints 5
You can pass anything you want to the staticmethod constructor.
Even if this doesn't fail, how does it make any sense?
It usually doesn't, but staticmethod doesn't check.
How is it usually a wrapper around a user-defined method object instead of a function object?
It's not. That part's just wrong.
How is it that static method objects are not themselves callable? How do calls to these work, then?
The descriptor protocol. Static method objects have a __get__ method that returns whatever object they wrap. Attribute access invokes this __get__ method and returns what __get__ returns.
I'm unclear as to what this one paragraph in the python tutorial documentation is saying.
(found here: https://docs.python.org/3/tutorial/classes.html#method-objects)
When an instance attribute is referenced that isn’t a data attribute, its class is searched. If the name denotes a valid class attribute that is a function object, a method object is created by packing (pointers to) the instance object and the function object just found together in an abstract object: this is the method object. When the method object is called with an argument list, a new argument list is constructed from the instance object and the argument list, and the function object is called with this new argument list.
From my current understanding, I think what it's saying is that whenever you reference an attribute of an instance of a class like in the 8th line of this little snippet here:
class MyClass():
attribute = "I am an attribute"
def func(self):
return "I am a function"
instance = MyClass()
print(instance.func())
When python sees
instance.func()
what it's really doing isn't looking for a method func "owned by" instance, it's looking for a function func owned by MyClass, then calling that function owned by MyClass with instance as the self parameter.
so basically it's the same thing as:
MyClass.func(instance)
I feel like I'm missing something subtle though. I don't understand what it means by
... a method object is created by packing (pointers to) the instance object and the function object just found together in an abstract object: this is the method object.
What is an abstract object?
What does it mean to "pack" a pointer?
What does it mean to "pack" multiple pointers?
Why even have a method object for instance if python is just going to look at MyClass's function object?
Why doesn't python just make methods be "owned by" their instances? Why even go through the whole process of calling MyClass's func instead of instance's func?
Why did the designers of the language decide to make it be this way?
"Abstract object" means that there isn't necessarily a real Python object being created, it's just a way of describing what's happening behind the scenes as if there were some object being created.
"packing" means that it's just collecting these things together into this abstract object.
So when you write
instance.func()
it internally creates something that represents the function combined with the instance. When this is called, the method function is called as you described, with the instance passed as the first argument (conventionally named self).
Why do this? So that you can pass these things around:
foo = instance.func
foo()
The value of foo contains that abstract object that represents the function combined with the instance.
Methods are owned by classes so that all instances of a class automatically get the same method. This is the essence of OO programming and the basis of inheritance among classes.
I did not understand the following sentence from Python tutorial:
there are no shorthands for referencing the object's members from it's methods.The method function is declared with an explicit 1st argument representing the object, which is provided implicitly by the call.
Does this mean, in every method of a class, its object is hidden along with the data members? Can this be explained with sample code?
This is just referring to the fact that in an object's methods, the object itself is always the first argument (i.e., the "self" in def method(self, arg1, arg2)), and that you need to explicitly say that you want self.var in the method rather than just saying var.
For more background on why that decision was made, see https://docs.python.org/2/faq/design.html#why-must-self-be-used-explicitly-in-method-definitions-and-calls from the docs or http://neopythonic.blogspot.com.au/2008/10/why-explicit-self-has-to-stay.html for some examples.
>>> class A(object): pass
>>> def func(cls): pass
>>> A.func = func
>>> A.func
<unbound method A.func>
How does this assignment create a method? It seems unintuitive that assignment does the following for classes:
Turn functions into unbound instance methods
Turn functions wrapped in classmethod() into class methods (actually, this is pretty intuitive)
Turn functions wrapped in staticmethod() into functions
It seems that for the first, there should be an instancemethod(), and for the last one, there shouldn't be a wrapper function at all. I understand that these are for uses within a class block, but why should they apply outside of it?
But more importantly, how exactly does assignment of the function into a class work? What magic happens that resolves those 3 things?
Even more confusing with this:
>>> A.func
<unbound method A.func>
>>> A.__dict__['func']
<function func at 0x...>
But I think this is something to do with descriptors, when retrieving attributes. I don't think it has much to do with the setting of attributes here.
You're right that this has something to do with descriptor protocol. Descriptors are how passing the receiver object as the first parameter of a method is implemented in Python. You can read more detail about Python attribute lookup from here. The following shows on a bit lower level, what is happening when you do A.func = func; A.func:
# A.func = func
A.__dict__['func'] = func # This just sets the attribute
# A.func
# The __getattribute__ method of a type object calls the __get__ method with
# None as the first parameter and the type as the second.
A.__dict__['func'].__get__(None, A) # The __get__ method of a function object
# returns an unbound method object if the
# first parameter is None.
a = A()
# a.func()
# The __getattribute__ method of object finds an attribute on the type object
# and calls the __get__ method of it with the instance as its first parameter.
a.__class__.__dict__['func'].__get__(a, a.__class__)
# This returns a bound method object that is actually just a proxy for
# inserting the object as the first parameter to the function call.
So it's the looking up of the function on a class or an instance that turns it into a method, not assigning it to a class attribute.
classmethod and staticmethod are just slightly different descriptors, classmethod returning a bound method object bound to a type object and staticmethod just returns the original function.
Descriptors are the magic1 that turns an ordinary function into a bound or unbound method when you retrieve it from an instance or class, since they’re all just functions that need different binding strategies. The classmethod and staticmethod decorators implement other binding strategies, and staticmethod actually just returns the raw function, which is the same behavior you get from a non-function callable object.
See “User-defined methods” for some gory details, but note this:
Also notice that this transformation only happens for user-defined functions; other callable objects (and all non-callable objects) are retrieved without transformation.
So if you wanted this transformation for your own callable object, you could just wrap it in a function, but you could also write a descriptor to implement your own binding strategy.
Here’s the staticmethod decorator in action, returning the underlying function when it’s accessed.
>>> #staticmethod
... def f(): pass
>>> class A(object): pass
>>> A.f = f
>>> A.f
<function f at 0x100479398>
>>> f
<staticmethod object at 0x100492750>
Whereas a normal object with a __call__ method doesn’t get transformed:
>>> class C(object):
... def __call__(self): pass
>>> c = C()
>>> A.c = c
>>> A.c
<__main__.C object at 0x10048b890>
>>> c
<__main__.C object at 0x10048b890>
1 The specific function is func_descr_get in Objects/funcobject.c.
What you have to consider is that in Python everything is an object. By establishing that it is easier to understand what is happening. If you have a function def foo(bar): print bar, you can do spam = foo and call spam(1), getting of course, 1.
Objects in Python keep their instance attributes in a dictionary called __dict__ with a "pointer" to other objects. As functions in Python are objects as well, they can be assigned and manipulated as simple variables, passed around to other functions, etc. Python's implementation of object orientation takes advantage of this, and treats methods as attributes, as functions that are in the __dict__ of the object.
Instance methods' first parameter is always the instance object itself, generally called self (but this could be called this or banana). When a method is called directly on the class, it is unbound to any instance, so you have to give it an instance object as the first parameter (A.func(A())). When you call a bound function (A().func()), the first parameter of the method, self, is implicit, but behind the curtains Python does exactly the same as calling directly on the unbound function and passing the instance object as the first parameter.
If this is understood, the fact that assigning A.func = func (which behind the curtains is doing A.__dict__["func"] = func) leaves you with an unbound method, is unsurprising.
In your example the cls in def func(cls): pass actually what will be passed on is the instance (self) of type A. When you apply the classmethod or staticmethod decorators do nothing more than take the first argument obtained during the call of the function/method, and transform it into something else, before calling the function.
classmethod takes the first argument, gets the class object of the instance, and passes that as the first argument to the function call, while staticmethod simply discards the first parameter and calls the function without it.
Point 1: The function func you defined exists as a First-Class Object in Python.
Point 2: Classes in Python store their attributes in their __dict__.
So what happens when you pass a function as the value of a class attribute in Python? That function is stored in the class' __dict__, making it a method of that class accessed by calling the attribute name you assigned it to.
Relating to MTsoul's comment to Gabriel Hurley's answer:
What is different is that func has a __call__() method, making it "callable", i.e. you can apply the () operator to it. Check out the Python docs (search for __call__ on that page).