Develop Reference

How do python handles "self" attribute of a class internally, when the method is assigned to a variable

Recently, I was looking into the "getattr" method, and one interesting behaviour is that it could return a method of a class instance and assign it to a variable. Here's the example I played with:
class Example:
def some_method(self, param1):
print(param1)
example = Example()
method1 = getattr(example, "some_method")
print(method1)
method1("hi")
And here's the output:
<bound method Example.some_method of <__main__.Example object at 0x7f82cbcb2850>>
hi
I do understand for the first line of the output that the method1 is a "bound method" type related to the actual instance example. But for the function call involving the method method1("hi"), the "self" parameter is omitted, only the "param1" value is given. I wonder how this function call is processed successfully and where the "self" parameter is actually stored internally.

Methods rely on the mechanism called descriptors. Any type can be made a descriptor by implementing methods such as __get__. For example:
class D:
def __get__(self, obj, type=None):
return f'Some value from {obj.__class__.__name__}'
class A:
d = D()
print(A().d) # 'Some value from A'
Python attribute access mechanism will recognize the descriptor and delegate the access logic to its __get__(), __set__(), etc. Now, the plot twist is that Python functions also implement the descriptor protocol, that defines how they behave when accessed as methods. For example, if we have a method A.f():
class A:
def __init__(self, x):
self.x = x
def f(self, y):
return self.x + y
then this code:
a = A(2)
print(a.f(3)) # prints 5
in fact runs as:
a = A(2)
bound_f = A.__get__(a)
print(bound_f(3)) # also prints 5
So when a function is accessed as a non-static method, it intercepts the access via te descriptor protocol and returns a bound method object, with attribute __self__ set to the object on which it was accessed:
>>> print(bound_f)
<bound method A.f of <__main__.A object at 0x7fc13db99ac0>>
>>> print(bound_f.__self__)
<__main__.A at 0x7fc13db99ac0>
Static, class methods and properties are also implemented using descriptors.

Redefine Method of an Object

I've got a class, where a method should only run once. Of course, it could easily be done with artificial has_executed = True/False flag, but why use it, if you can just delete the method itself? python's a duck-typed language, everything is a reference, bla-bla-bla, what can go wrong?
At least it was the thought. I couldn't actually do it:
class A:
def b(self):
print("empty")
self.__delattr__('b')
a = A()
a.b()
raises AttributeError: b. However, executing self.__getattribute__('b') returns <bound method A.b of <__main__.A object at 0x000001CDC6742FD0>>, which sounds stupid to me: why is a method any different from an attribute, since everything in python is just a reference to an object? And why can I __getattribute__, but not __delattr__?
The same goes to redefinition. I can easily set any attribute, but methods are a no-no?
class A:
def b(self):
print("first")
self.__setattr__('b', lambda self: print(f"second"))
a = A()
a.b()
a.b()
results into TypeError: <lambda>() missing 1 required positional argument: 'self'. Which, of course, means, that now python isn't using dot-notation as intended. Of course, we could ditch the self attribute in the lambda altogether, considering we've got the reference to it already in b. But isn't it incorrect by design?
The further I'm trying to take python to the limit, the more frustrated I become. Some imposed limitations (or seemingly imposed?) seem so unnatural, considering the way the language is marketed. Shouldn't it allow this? Why doesn't it work?
UPD
Ok, consider this:
class A:
def __init__(self):
self.variable = 1
def b(self):
print("old")
self.variable += 1
def new_b():
print("new")
self.variable += 15
self.__setattr__('b', new_b)
It will work and do what we want: none of other objects will have their A.b method redefined once one object kind of overlays its b definition. (overlays, since everyone so far says that you cannot redefine a method for an object, but instead only kind of hide it from the caller behind another attribute with the same name, as far as I understand).
Is this good?

It doesn't work because b isn't an attribute belonging to the instance, it belongs to the class. So you can't delete it on the instance because it isn't there to be deleted.
>>> a = A()
>>> list(a.__dict__)
[]
>>> list(A.__dict__)
['__module__', 'b', '__dict__', '__weakref__', '__doc__']
When a.b is evaluated, Python will see that a has no instance attribute named b and fall back to the class. (It's a little more complicated because when falling back to the class, it will not simply return the method itself, but a version of the method which is bound to the instance a.)
Since you don't want to delete the method on the class, the way to go is to replace the method on the instance. I don't know why you tried to do this with __setattr__ - there is no need for that, simply assign self.b = ... as normal. The reason your attempt failed is because your lambda requires a positional parameter named self, but this parameter will not be automatically bound to the instance when you look it up, because it is an instance attribute, not a class attribute.
class A:
def b(self):
print('first')
self.b = lambda: print('second')
Usage:
>>> a = A()
>>> a.b()
first
>>> a.b()
second

Well in python you have 2 types of attributes
A class attribute is a variable that belongs to a certain class, and not a particular object. Every instance of this class shares the same variable. These attributes are usually defined outside the init constructor
An instance/object attribute is a variable that belongs to one (and only one) object. Every instance of a class points to its own attributes variables. These attributes are defined within the init constructor.
In case of a class attribute its part of the class descriptor, so you cannot delete it from the object attributes like self.__deleteattr__ or add new one with __setattr__ as it alters the class descriptor and reflects on all objects. Such an operation can have devastating effects.
Its very similar to a class variable as well. You can however change the behavior with overriding or reassigning like below
class A:
def b(self):
print("empty")
A.b = lambda self: print(f"second")
a = A()
a.b()
a.b()

Why do different classes take the same amount of memory?

Why does foo not take more memory if it has another variable?
class Foo():
a = 1
b = 1
class Bar():
a = 1
import sys
foo = Foo()
print(sys.getsizeof(foo)) # 56
bar = Bar()
print(sys.getsizeof(bar)) # 56

First, methods are stored with the type of the object, not the object itself.
However, you'll notice that if you ask for the class sizes, they are equal as well.
>>> sys.getsizeof(Foo)
1064
>>> sys.getsizeof(Bar)
1064
Each class object has a __dict__ attribute that stores references to its methods and other attributes.
>>> Foo.__dict__['why_does_this_not_use_take_memory']
<function Foo.why_does_this_not_use_take_memory at 0x103609ef0>
However, sys.getsizeof doesn't recurse into that dict; it only returns the memory used by the class object itself, not including objects that you can reach via the class and its attributes.
Each method is a class attribute. Without going too much into the descriptor protocol, in general, something like foo.why_does_this_not_use_take_memory is really just shorthand for
Foo.why_does_this_not_use_take_memory.__get__(foo, Foo)
That is, the function-valued attribute is retrieved, but then its __get__ method is called to return a method object specific to foo. The method is essentially just a wrapper around the function which, when called, passes foo and its own arguments to Foo.why_does_this_not_use_take_memory (which is how self gets bound to foo).

Why does `instance_of_object.foo is instance_of_object.foo` evaluate False? [duplicate]

This question already has an answer here:
Two methods inherited from one method in class are different in instances, aren't they?
(1 answer)
Closed 4 years ago.
If I have a
class A:
def foo(self):
pass
this evaluates to True:
getattr(A, 'foo') is A.foo
but this evaluates to False:
a = A()
getattr(a, 'foo') is a.foo
as does
a.foo is a.foo
Why?
I found that getattr(a, 'foo') and a.foo both are represented by
<bound method A.foo of <__main__.A object at 0x7a2c4de10d50>>)
So no hint there....

At least in CPython, bound methods are implemented as an instance of a class method. Every time you ask for the value of a bound function, you get a new instance of this class.
x = a.foo
y = a.foo
assert x is not y
id(x) # E.g. 139664144271304
id(y) # E.g. 139664144033992
type(x) # <class 'method'>
type(y) # <class 'method'>
All this class does is store a reference to the instance and the unbound function, and when you call the class it calls the unbound function with the stored instance (along with your other arguments).
Unbound functions, like A.foo, are just regular old functions - no new instances of proxy classes are being constructed, so identity works as you expect.
The reason for this difference is that the semantic meaning of a.foo depends on two things, the value of a and the value of A.foo. In order to be able to get this meaning at any point in time later, both of these values need to be stored. This is what the method class does.
Conversely, the meaning of A.foo depends only on a single value: A.foo. So no additional work is required to store anything, and the value itself is used.
You might consider the idea of pre-allocating bound method instances, so that a.foo always returns the same immutable object - but given the dynamic nature of Python, it is simpler and cheaper to just construct a new one each time, even if they could be the same.

To add to #GManNickG answer:
getattr(a, 'foo').__func__ is a.foo.__func__
will return True.

Some objects stored in classes are descriptors, which don't follow normal rules for object lookups. The foo method you're dealing with in your example is one (function objects are descriptors).
A descriptor is an instance of a class that defines a __get__ (and optionally __set__ and __delete__) method(s). Those methods control what happens when you look up the desciptor on an instance of the class it's stored in.
I think an example will make this more clear:
class DescriptorClass:
def __get__(*args):
print("__get__ was called")
return "whatever"
class OtherClass:
descriptor_instance = DescriptorClass() # the descriptor instance is a class variable
other_instance = OtherClass()
# this calls the descriptor's __get__ method, which prints "__get__ was called"
result = other_instance.descriptor_instance
print(result) # will print "whatever", since that's what the __get__ method returned
A __get__ method doesn't need to return the same thing every time it's called. In fact, it usually won't. In the specific case of functions being used as descriptors (i.e. methods), a new "bound method" object will be created each time you look the function up. Thus the is operator will not see multiple bound methods as the same object, even though they may be binding the same function to the same instance.

How does assignment of a function as a class attribute become a method in Python?

>>> class A(object): pass
>>> def func(cls): pass
>>> A.func = func
>>> A.func
<unbound method A.func>
How does this assignment create a method? It seems unintuitive that assignment does the following for classes:
Turn functions into unbound instance methods
Turn functions wrapped in classmethod() into class methods (actually, this is pretty intuitive)
Turn functions wrapped in staticmethod() into functions
It seems that for the first, there should be an instancemethod(), and for the last one, there shouldn't be a wrapper function at all. I understand that these are for uses within a class block, but why should they apply outside of it?
But more importantly, how exactly does assignment of the function into a class work? What magic happens that resolves those 3 things?
Even more confusing with this:
>>> A.func
<unbound method A.func>
>>> A.__dict__['func']
<function func at 0x...>
But I think this is something to do with descriptors, when retrieving attributes. I don't think it has much to do with the setting of attributes here.

You're right that this has something to do with descriptor protocol. Descriptors are how passing the receiver object as the first parameter of a method is implemented in Python. You can read more detail about Python attribute lookup from here. The following shows on a bit lower level, what is happening when you do A.func = func; A.func:
# A.func = func
A.__dict__['func'] = func # This just sets the attribute
# A.func
# The __getattribute__ method of a type object calls the __get__ method with
# None as the first parameter and the type as the second.
A.__dict__['func'].__get__(None, A) # The __get__ method of a function object
# returns an unbound method object if the
# first parameter is None.
a = A()
# a.func()
# The __getattribute__ method of object finds an attribute on the type object
# and calls the __get__ method of it with the instance as its first parameter.
a.__class__.__dict__['func'].__get__(a, a.__class__)
# This returns a bound method object that is actually just a proxy for
# inserting the object as the first parameter to the function call.
So it's the looking up of the function on a class or an instance that turns it into a method, not assigning it to a class attribute.
classmethod and staticmethod are just slightly different descriptors, classmethod returning a bound method object bound to a type object and staticmethod just returns the original function.

Descriptors are the magic1 that turns an ordinary function into a bound or unbound method when you retrieve it from an instance or class, since they’re all just functions that need different binding strategies. The classmethod and staticmethod decorators implement other binding strategies, and staticmethod actually just returns the raw function, which is the same behavior you get from a non-function callable object.
See “User-defined methods” for some gory details, but note this:
Also notice that this transformation only happens for user-defined functions; other callable objects (and all non-callable objects) are retrieved without transformation.
So if you wanted this transformation for your own callable object, you could just wrap it in a function, but you could also write a descriptor to implement your own binding strategy.
Here’s the staticmethod decorator in action, returning the underlying function when it’s accessed.
>>> #staticmethod
... def f(): pass
>>> class A(object): pass
>>> A.f = f
>>> A.f
<function f at 0x100479398>
>>> f
<staticmethod object at 0x100492750>
Whereas a normal object with a __call__ method doesn’t get transformed:
>>> class C(object):
... def __call__(self): pass
>>> c = C()
>>> A.c = c
>>> A.c
<__main__.C object at 0x10048b890>
>>> c
<__main__.C object at 0x10048b890>
1 The specific function is func_descr_get in Objects/funcobject.c.

What you have to consider is that in Python everything is an object. By establishing that it is easier to understand what is happening. If you have a function def foo(bar): print bar, you can do spam = foo and call spam(1), getting of course, 1.
Objects in Python keep their instance attributes in a dictionary called __dict__ with a "pointer" to other objects. As functions in Python are objects as well, they can be assigned and manipulated as simple variables, passed around to other functions, etc. Python's implementation of object orientation takes advantage of this, and treats methods as attributes, as functions that are in the __dict__ of the object.
Instance methods' first parameter is always the instance object itself, generally called self (but this could be called this or banana). When a method is called directly on the class, it is unbound to any instance, so you have to give it an instance object as the first parameter (A.func(A())). When you call a bound function (A().func()), the first parameter of the method, self, is implicit, but behind the curtains Python does exactly the same as calling directly on the unbound function and passing the instance object as the first parameter.
If this is understood, the fact that assigning A.func = func (which behind the curtains is doing A.__dict__["func"] = func) leaves you with an unbound method, is unsurprising.
In your example the cls in def func(cls): pass actually what will be passed on is the instance (self) of type A. When you apply the classmethod or staticmethod decorators do nothing more than take the first argument obtained during the call of the function/method, and transform it into something else, before calling the function.
classmethod takes the first argument, gets the class object of the instance, and passes that as the first argument to the function call, while staticmethod simply discards the first parameter and calls the function without it.

Point 1: The function func you defined exists as a First-Class Object in Python.
Point 2: Classes in Python store their attributes in their __dict__.
So what happens when you pass a function as the value of a class attribute in Python? That function is stored in the class' __dict__, making it a method of that class accessed by calling the attribute name you assigned it to.

Relating to MTsoul's comment to Gabriel Hurley's answer:
What is different is that func has a __call__() method, making it "callable", i.e. you can apply the () operator to it. Check out the Python docs (search for __call__ on that page).