Develop Reference

Invisible argument python [duplicate]

This question already has answers here:
What is the purpose of the `self` parameter? Why is it needed?
(26 answers)
Closed 6 months ago.
When defining a method on a class in Python, it looks something like this:
class MyClass(object):
def __init__(self, x, y):
self.x = x
self.y = y
But in some other languages, such as C#, you have a reference to the object that the method is bound to with the "this" keyword without declaring it as an argument in the method prototype.
Was this an intentional language design decision in Python or are there some implementation details that require the passing of "self" as an argument?

I like to quote Peters' Zen of Python. "Explicit is better than implicit."
In Java and C++, 'this.' can be deduced, except when you have variable names that make it impossible to deduce. So you sometimes need it and sometimes don't.
Python elects to make things like this explicit rather than based on a rule.
Additionally, since nothing is implied or assumed, parts of the implementation are exposed. self.__class__, self.__dict__ and other "internal" structures are available in an obvious way.

It's to minimize the difference between methods and functions. It allows you to easily generate methods in metaclasses, or add methods at runtime to pre-existing classes.
e.g.
>>> class C:
... def foo(self):
... print("Hi!")
...
>>>
>>> def bar(self):
... print("Bork bork bork!")
...
>>>
>>> c = C()
>>> C.bar = bar
>>> c.bar()
Bork bork bork!
>>> c.foo()
Hi!
>>>
It also (as far as I know) makes the implementation of the python runtime easier.

I suggest that one should read Guido van Rossum's blog on this topic - Why explicit self has to stay.
When a method definition is decorated, we don't know whether to automatically give it a 'self' parameter or not: the decorator could turn the function into a static method (which has no 'self'), or a class method (which has a funny kind of self that refers to a class instead of an instance), or it could do something completely different (it's trivial to write a decorator that implements '#classmethod' or '#staticmethod' in pure Python). There's no way without knowing what the decorator does whether to endow the method being defined with an implicit 'self' argument or not.
I reject hacks like special-casing '#classmethod' and '#staticmethod'.

Python doesn't force you on using "self". You can give it whatever name you want. You just have to remember that the first argument in a method definition header is a reference to the object.

Also allows you to do this: (in short, invoking Outer(3).create_inner_class(4)().weird_sum_with_closure_scope(5) will return 12, but will do so in the craziest of ways.
class Outer(object):
def __init__(self, outer_num):
self.outer_num = outer_num
def create_inner_class(outer_self, inner_arg):
class Inner(object):
inner_arg = inner_arg
def weird_sum_with_closure_scope(inner_self, num)
return num + outer_self.outer_num + inner_arg
return Inner
Of course, this is harder to imagine in languages like Java and C#. By making the self reference explicit, you're free to refer to any object by that self reference. Also, such a way of playing with classes at runtime is harder to do in the more static languages - not that's it's necessarily good or bad. It's just that the explicit self allows all this craziness to exist.
Moreover, imagine this: We'd like to customize the behavior of methods (for profiling, or some crazy black magic). This can lead us to think: what if we had a class Method whose behavior we could override or control?
Well here it is:
from functools import partial
class MagicMethod(object):
"""Does black magic when called"""
def __get__(self, obj, obj_type):
# This binds the <other> class instance to the <innocent_self> parameter
# of the method MagicMethod.invoke
return partial(self.invoke, obj)
def invoke(magic_self, innocent_self, *args, **kwargs):
# do black magic here
...
print magic_self, innocent_self, args, kwargs
class InnocentClass(object):
magic_method = MagicMethod()
And now: InnocentClass().magic_method() will act like expected. The method will be bound with the innocent_self parameter to InnocentClass, and with the magic_self to the MagicMethod instance. Weird huh? It's like having 2 keywords this1 and this2 in languages like Java and C#. Magic like this allows frameworks to do stuff that would otherwise be much more verbose.
Again, I don't want to comment on the ethics of this stuff. I just wanted to show things that would be harder to do without an explicit self reference.

I think it has to do with PEP 227:
Names in class scope are not accessible. Names are resolved in the
innermost enclosing function scope. If a class definition occurs in a
chain of nested scopes, the resolution process skips class
definitions. This rule prevents odd interactions between class
attributes and local variable access. If a name binding operation
occurs in a class definition, it creates an attribute on the resulting
class object. To access this variable in a method, or in a function
nested within a method, an attribute reference must be used, either
via self or via the class name.

I think the real reason besides "The Zen of Python" is that Functions are first class citizens in Python.
Which essentially makes them an Object. Now The fundamental issue is if your functions are object as well then, in Object oriented paradigm how would you send messages to Objects when the messages themselves are objects ?
Looks like a chicken egg problem, to reduce this paradox, the only possible way is to either pass a context of execution to methods or detect it. But since python can have nested functions it would be impossible to do so as the context of execution would change for inner functions.
This means the only possible solution is to explicitly pass 'self' (The context of execution).
So i believe it is a implementation problem the Zen came much later.

As explained in self in Python, Demystified
anything like obj.meth(args) becomes Class.meth(obj, args). The calling process is automatic while the receiving process is not (its explicit). This is the reason the first parameter of a function in class must be the object itself.
class Point(object):
def __init__(self,x = 0,y = 0):
self.x = x
self.y = y
def distance(self):
"""Find distance from origin"""
return (self.x**2 + self.y**2) ** 0.5
Invocations:
>>> p1 = Point(6,8)
>>> p1.distance()
10.0
init() defines three parameters but we just passed two (6 and 8). Similarly distance() requires one but zero arguments were passed.
Why is Python not complaining about this argument number mismatch?
Generally, when we call a method with some arguments, the corresponding class function is called by placing the method's object before the first argument. So, anything like obj.meth(args) becomes Class.meth(obj, args). The calling process is automatic while the receiving process is not (its explicit).
This is the reason the first parameter of a function in class must be the object itself. Writing this parameter as self is merely a convention. It is not a keyword and has no special meaning in Python. We could use other names (like this) but I strongly suggest you not to. Using names other than self is frowned upon by most developers and degrades the readability of the code ("Readability counts").
...
In, the first example self.x is an instance attribute whereas x is a local variable. They are not the same and lie in different namespaces.
Self Is Here To Stay
Many have proposed to make self a keyword in Python, like this in C++ and Java. This would eliminate the redundant use of explicit self from the formal parameter list in methods. While this idea seems promising, it's not going to happen. At least not in the near future. The main reason is backward compatibility. Here is a blog from the creator of Python himself explaining why the explicit self has to stay.

The 'self' parameter keeps the current calling object.
class class_name:
class_variable
def method_name(self,arg):
self.var=arg
obj=class_name()
obj.method_name()
here, the self argument holds the object obj. Hence, the statement self.var denotes obj.var

There is also another very simple answer: according to the zen of python, "explicit is better than implicit".

How to check if a python function is a method or not before it is bounded to an object?

I have a class like this
class A(object):
def __init__(self, name):
self.name = name
def run(self):
pass
if we look at the type of run it is a function. I am now writing a decorator and this decorator should be used with either a stand alone function or a method but has different behavior if the function it is decorating is a method. When registering the method run, the decorator cannot really tell if the function is a method because it has not been bounded to an object yet. I have tried inspect.ismethod and it also does not work. Is there a way that I can detect run is a method in my decorator instead of a standalone function? Thanks!
To add a bit more info:
Basically I am logging something out. If it is decorating an object method, I need the name of the class of that object and the method name, if it is the decorating a function, I just need the function name.

As mentionned by chepner, a function only becomes a method when it's used as one - ie when it's looked up on an instance and resolved on the class. What you are decorating is and will always be a function (well, unless you already decorated it with something that returns another callable type of course, cf the classmethod type).
At this point you have two options: the safe and explicit one, and the unsafe guessing game one.
The safe and explicit solution is, simply, to have two distinct decorators, one for plain functions, and another for "functions to be used as methods".
The unsafe guessing game one is to inspect the function's first arg name (using inspect.getargspecs()) and consider it's a "function to be used as method" if the first argument is named "self".
Obviously the safe and explicit solution is also much simpler ;-)

Why we need corresponding class function with each method in python?

I am learning OOP in python and following this and this stackoverflow answers and this post
I understood how class works and how method called and all things but i have some doubts:
Consider this fragment of code:
class Point(object):
def __init__(self,x,y):
self.x = x
self.y = y
def distance(self):
print (self.x)
def bye(self):
print(self.y)
a=Point(1,2)
a.distance()
a.bye()
As i read in tutorial :
when we call a method with some arguments, the corresponding class
function is called by placing the method's object before the first
argument. So, anything like obj.meth(args) becomes Class.meth(obj,
args).
when ObjectA.methodA(arg1, arg2) is called, python internally converts
it for you as:
ClassA.methodA(ObjectA, arg1, arg2)
Now my confusion is why program need to call class with each method ?
Class.meth(obj, args) ??
like when we call a.distance it become Point.distance(a) causes of "self"
when we called a.bye it become Point.bye(a) causes of "self" .
when Point class is necessery with each method if we don't use Point class with each method what will happen?
why can't simply meth(obj, args) works ?
My main doubt is why its called class.some_method with each method when we called with attribute of method . why its needs calls with each one?
#if i am understanding right then its necessary because so that each method can access other methods data like variables and stuff?

The key is
python internally converts it for you
From your standpoint:
meth(self, args) is the syntax you use to define member functions; and
obj.meth(args) is the syntax you use to call member functions.
The meth(obj,args) option is the way procedural languages work. That is often how the implementation works, but expressing the call as obj.meth(args) keeps focus on the object and makes it easier to read which data values (object instances) are being used.
Edit 1 If I understand your question correctly, you are asking why Python needs to know the class when it already has the instance available, and instances know their own types. In fact, Python fetches methods based on the instance all the time. I think the point the tutorial is making is that in Python, the class is the primary place the functions are defined. This is different from some object-oriented languages, in which each instance has its own methods, and they may be completely different from each other. So the tutorial is contrasting the usual approach in Python:
class Foo:
def bar(self):
pass
with an alternative (possible in Python, but not typical):
foo = object() # an empty instance
foo.bar = lambda self: pass
Edit 2 Python methods normally live in the classes, not in the instances. Even if you create 1000 Point objects, there is only one copy of the actual instruction bytes for Point.distance. Those instruction bytes are executed anytime <some point variable>.distance() is called. You are correct that the self parameter is how those instruction bytes know what instance to work on, and how the method can access other data in the passed instance.
Edit 3 self isn't exactly a namespace in the way that local vs. global is. However, it is fair to say that self.foo refers to a foo that is indeed accessible to all the methods of this instance of the current class. Given
a = Point(1,2)
b = Point(3,4)
inside a Point.distance call, self refers to a or b, but not both. So when you call a.distance(), the self.x will be a.x, not b.x. But all methods of Point can access self.x to get whatever the current point's x is.
Edit 4 Suppose you weren't using objects, but instead dictionaries:
a = {'x':1, 'y':2} # make a "point"
b = {'x':3, 'y':4} # make another
def point_distance(point):
print (point['x'])
then you could say:
point_distance(a)
to get the effect of
print (a['x'])
Classes do basically that, with cleaner syntax and some nice benefits. But just as the point parameter to point_distance() refers to one and only one point-like dictionary each time you call point_distance(), the self parameter to Point.distance() refers to one and only one Point instance each time you call <whatever point>.distance().

Because you can have the same method name in different classes, and it needs to call the appropriate one. So if you have
class Class1:
def meth():
print "This is Class 1"
class Class2:
def meth():
print "This is Class 2"
c1 = Class1()
c2 = Class2()
c1.meth() # equivalent to Class1.meth(c1)
c2.meth() # equivalent to Class2.meth(c2)
If it translated c1.meth() to meth(c1), there's no way for the system to know which meth() function to call.

Classes define what is common to all instances of them. Usually this is the code comprising each of its methods. To apply this code to the correct instance object, the language interprets
instance.method(arg1, arg2, ...)
as
class_of_instance.method(instance, arg1, arg2, ...)
so the code is applied to the proper class instance.

Why do we use #staticmethod?

I just can't see why do we need to use #staticmethod. Let's start with an exmaple.
class test1:
def __init__(self,value):
self.value=value
#staticmethod
def static_add_one(value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
a=test1(3)
print(a.new_val) ## >>> 4
class test2:
def __init__(self,value):
self.value=value
def static_add_one(self,value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
b=test2(3)
print(b.new_val) ## >>> 4
In the example above, the method, static_add_one , in the two classes do not require the instance of the class(self) in calculation.
The method static_add_one in the class test1 is decorated by #staticmethod and work properly.
But at the same time, the method static_add_one in the class test2 which has no #staticmethod decoration also works properly by using a trick that provides a self in the argument but doesn't use it at all.
So what is the benefit of using #staticmethod? Does it improve the performance? Or is it just due to the zen of python which states that "Explicit is better than implicit"?

The reason to use staticmethod is if you have something that could be written as a standalone function (not part of any class), but you want to keep it within the class because it's somehow semantically related to the class. (For instance, it could be a function that doesn't require any information from the class, but whose behavior is specific to the class, so that subclasses might want to override it.) In many cases, it could make just as much sense to write something as a standalone function instead of a staticmethod.
Your example isn't really the same. A key difference is that, even though you don't use self, you still need an instance to call static_add_one --- you can't call it directly on the class with test2.static_add_one(1). So there is a genuine difference in behavior there. The most serious "rival" to a staticmethod isn't a regular method that ignores self, but a standalone function.

Today I suddenly find a benefit of using #staticmethod.
If you created a staticmethod within a class, you don't need to create an instance of the class before using the staticmethod.
For example,
class File1:
def __init__(self, path):
out=self.parse(path)
def parse(self, path):
..parsing works..
return x
class File2:
def __init__(self, path):
out=self.parse(path)
#staticmethod
def parse(path):
..parsing works..
return x
if __name__=='__main__':
path='abc.txt'
File1.parse(path) #TypeError: unbound method parse() ....
File2.parse(path) #Goal!!!!!!!!!!!!!!!!!!!!
Since the method parse is strongly related to the classes File1 and File2, it is more natural to put it inside the class. However, sometimes this parse method may also be used in other classes under some circumstances. If you want to do so using File1, you must create an instance of File1 before calling the method parse. While using staticmethod in the class File2, you may directly call the method by using the syntax File2.parse.
This makes your works more convenient and natural.

I will add something other answers didn't mention. It's not only a matter of modularity, of putting something next to other logically related parts. It's also that the method could be non-static at other point of the hierarchy (i.e. in a subclass or superclass) and thus participate in polymorphism (type based dispatching). So if you put that function outside the class you will be precluding subclasses from effectively overriding it. Now, say you realize you don't need self in function C.f of class C, you have three two options:
Put it outside the class. But we just decided against this.
Do nothing new: while unused, still keep the self parameter.
Declare you are not using the self parameter, while still letting other C methods to call f as self.f, which is required if you wish to keep open the possibility of further overrides of f that do depend on some instance state.
Option 2 demands less conceptual baggage (you already have to know about self and methods-as-bound-functions, because it's the more general case). But you still may prefer to be explicit about self not being using (and the interpreter could even reward you with some optimization, not having to partially apply a function to self). In that case, you pick option 3 and add #staticmethod on top of your function.

Use #staticmethod for methods that don't need to operate on a specific object, but that you still want located in the scope of the class (as opposed to module scope).
Your example in test2.static_add_one wastes its time passing an unused self parameter, but otherwise works the same as test1.static_add_one. Note that this extraneous parameter can't be optimized away.
One example I can think of is in a Django project I have, where a model class represents a database table, and an object of that class represents a record. There are some functions used by the class that are stand-alone and do not need an object to operate on, for example a function that converts a title into a "slug", which is a representation of the title that follows the character set limits imposed by URL syntax. The function that converts a title to a slug is declared as a staticmethod precisely to strongly associate it with the class that uses it.

In python, how can I tell which class a super object is wrapping?

Ie, if I have a class MyClass, and I do super(MyClass).init, how can I tell which class's init is actually going to be called?
Some code to illustrate:
class MyClass(OtherClass, ThirdClass):
def __init__(self):
mySuper = super(MyClass)
if mySuper == SomeClass:
# doesn't work - mySuper is a super object (not a normal class object)
pass
if mySuper.__init__ == SomeClass.__init__:
# doesn't work - mySuper.__init__ is a super-method-wrapper object
pass
if mySuper.__thisclass__ == SomeClass:
# doesn't work - __thisclass__ is set to be MyClass, not the "parent" class
pass
Any ideas?
EDIT:
If I hadn't already awarded points here, I would probably delete this question, as it's not really very useful as posed, and could potentially encourage bad habits.
As sven-marnach notes, I'm using the one-arg version, super(MyClass), instead of the more useful two-arg version, super(MyClass, self)... and now, I have no idea why I would have wanted to do that. My best guess is that I was still unclear on the proper usage of super at the time.
If you're using the two-arg version, then the second check works - with the caveat that you would need to get .im_func, ie:
if mySuper.__init__.im_func == SomeClass.__init__.im_func:
See Determine whether super().__new__ will be object.__new__ in Python 3? for an example of why this sort of check is useful...

You can extract the wrapped class using
mro = my_super.__self_class__.mro()
wrapped_class = mro[mro.index(my_super.__thisclass__) + 1]
This looks complex, but I also think it is rather pointless to do this.
Edit: I just noticed you don't pass self to super(). For that case, you could use
wrapped_class = my_super.__thisclass__.mro()[1]
The question that remains is: why would you want to do this?