I'm making a program in python in which specific instances of an object must be decorated with new functions built at runtime.
I've seen very simple examples of adding functions to objects through MethodType:
import types
def foo():
print("foo")
class A:
bar = "bar"
a = A()
a.foo = types.MethodType(foo, a)
But none of the examples I've seen show how a function added in this manner can reference to the new owner's attributes. As far as I know, even though this binds the foo() function to the instance a, foo() must still be a pure function, and cannot contain references to anything local.
In my case, I need functions to change attributes of the object they are added to. Here are two examples of the kind of thing I need to be able to do:
class A:
foo = "foo"
def printme():
print(foo)
def nofoo():
foo = "bar"
def printBar():
if foo != "foo"
self.printme()
I would then need a way to add a copy of a nofoo() or printBar() to an A object in such a way that they can access the object attributes named foo and the function named printme() correctly.
So, is this possible? Is there a way to do this kind of programming in vanilla Python? or at least Is there a programming pattern that achieves this kind of behavior?
P.S.: In my system, I also add attributes dynamically to objects. Your first thought then might be "How can I ever be sure that the object I'm adding the nofoo() function to actually has an attribute named foo?", but I also have a fairly robust tag system that makes sure that I never try to add a nofoo() function to an object that hasn't a foo variable. The reason I mention this is that solutions that look at the class definition aren't very useful to me.
As said in the comments, your function actually must take at least one parameter: self, the instance the method is being called on. The self parameter can be used as it would be used in a normal instance method. Here is an example:
>>> from types import MethodType
>>>
>>> class Class:
def method(self):
print('method run')
>>> cls = Class()
>>>
>>> def func(self): # must accept one argument, `self`
self.method()
>>> cls.func = MethodType(func, cls)
>>> cls.func()
method run
>>>
Without your function accepting self, an exception would be raised:
>>> def func():
self.method()
>>> cls.func = MethodType(func, cls)
>>> cls.func()
Traceback (most recent call last):
File "<pyshell#21>", line 1, in <module>
cls.func()
TypeError: func() takes 0 positional arguments but 1 was given
>>>
class A:
def __init__(self):
self.foo = "foo"
def printme(self):
print(self.foo)
def nofoo(self):
self.foo = "bar"
a.nofoo = types.MethodType(nofoo, a)
a.nofoo()
a.printme()
prints
bar
It's not entirely clear what you're trying to do, and I'm worried that whatever it is may be a bad idea. However, I can explain how to do what you're asking, even if it isn't what you want, or should want. I'll point out that it's very uncommon to want to do the second version below, and even rarer to want to do the third version, but Python does allow them both, because "even rarer than very uncommon" still isn't "never". And, in the same spirit…
The short answer is "yes". A dynamically-added method can access the owner object exactly the same way a normal method can.
First, here's a normal, non-dynamic method:
class C:
def meth(self):
return self.x
c = C()
c.x = 3
c.meth()
Obviously, with a normal method like this, when you call c.meth(), the c ends up as the value of the self parameter, so self.x is c.x, which is 3.
Now, here's how you dynamically add a method to a class:
class C:
pass
c = C()
c.x = 3
def meth(self):
print(self.x)
C.meth = meth
c.meth()
This is actually doing exactly the same thing. (Well, we've left another name for the same function object sitting around in globals, but that's the only difference) If C.meth is the same function it was in the first version, then obviously whatever magic made c.meth() work in the first version will do the exact same thing here.
(This used to be slightly more complicated in Python 2, because of unbound methods, and classic classes too… but fortunately you don't have to worry about that.)
Finally, here's how you dynamically add a method to an instance:
class C:
pass
c = C()
c.x = 3
def meth(self):
print(self.x)
c.meth = types.MethodType(meth, c)
c.meth()
Here, you actually have to know the magic that makes c.meth() work in the first two cases. So read the Descriptor HOWTO. After that, it should be obvious.
But if you just want to pretend that Guido is a wizard (Raymond definitely is a wizard) and it's magic… Well, in the first two versions, Guido's magic wand creates a special bound method object whenever you ask for c.meth, but even he isn't magical enough to do that when C.meth doesn't exist. But we can painstakingly create that same bound method object and store it as c.meth. After that, we're going to get the same thing we stored whenever we ask for c.meth, which we explicitly built as the same thing we got in the first two examples, so it'll obviously do the same thing.
But what if we did this:
class C:
pass
c = C()
c.x = 3
def meth(self):
print(self.x)
c.meth = meth
c.meth(c)
Here, you're not letting Guido do his descriptor magic to create c.meth, and you're not doing it manually, you're just sticking a regular function there. Which means if you want anything to show up as the self parameter, you have to explicitly pass it as an argument, as in that silly c.meth(c) line at the end. But if you're willing to do that, then even this one works. No matter how self ends up as c, self.x is going to be c.x.
Related
I've got a class, where a method should only run once. Of course, it could easily be done with artificial has_executed = True/False flag, but why use it, if you can just delete the method itself? python's a duck-typed language, everything is a reference, bla-bla-bla, what can go wrong?
At least it was the thought. I couldn't actually do it:
class A:
def b(self):
print("empty")
self.__delattr__('b')
a = A()
a.b()
raises AttributeError: b. However, executing self.__getattribute__('b') returns <bound method A.b of <__main__.A object at 0x000001CDC6742FD0>>, which sounds stupid to me: why is a method any different from an attribute, since everything in python is just a reference to an object? And why can I __getattribute__, but not __delattr__?
The same goes to redefinition. I can easily set any attribute, but methods are a no-no?
class A:
def b(self):
print("first")
self.__setattr__('b', lambda self: print(f"second"))
a = A()
a.b()
a.b()
results into TypeError: <lambda>() missing 1 required positional argument: 'self'. Which, of course, means, that now python isn't using dot-notation as intended. Of course, we could ditch the self attribute in the lambda altogether, considering we've got the reference to it already in b. But isn't it incorrect by design?
The further I'm trying to take python to the limit, the more frustrated I become. Some imposed limitations (or seemingly imposed?) seem so unnatural, considering the way the language is marketed. Shouldn't it allow this? Why doesn't it work?
UPD
Ok, consider this:
class A:
def __init__(self):
self.variable = 1
def b(self):
print("old")
self.variable += 1
def new_b():
print("new")
self.variable += 15
self.__setattr__('b', new_b)
It will work and do what we want: none of other objects will have their A.b method redefined once one object kind of overlays its b definition. (overlays, since everyone so far says that you cannot redefine a method for an object, but instead only kind of hide it from the caller behind another attribute with the same name, as far as I understand).
Is this good?
It doesn't work because b isn't an attribute belonging to the instance, it belongs to the class. So you can't delete it on the instance because it isn't there to be deleted.
>>> a = A()
>>> list(a.__dict__)
[]
>>> list(A.__dict__)
['__module__', 'b', '__dict__', '__weakref__', '__doc__']
When a.b is evaluated, Python will see that a has no instance attribute named b and fall back to the class. (It's a little more complicated because when falling back to the class, it will not simply return the method itself, but a version of the method which is bound to the instance a.)
Since you don't want to delete the method on the class, the way to go is to replace the method on the instance. I don't know why you tried to do this with __setattr__ - there is no need for that, simply assign self.b = ... as normal. The reason your attempt failed is because your lambda requires a positional parameter named self, but this parameter will not be automatically bound to the instance when you look it up, because it is an instance attribute, not a class attribute.
class A:
def b(self):
print('first')
self.b = lambda: print('second')
Usage:
>>> a = A()
>>> a.b()
first
>>> a.b()
second
Well in python you have 2 types of attributes
A class attribute is a variable that belongs to a certain class, and not a particular object. Every instance of this class shares the same variable. These attributes are usually defined outside the init constructor
An instance/object attribute is a variable that belongs to one (and only one) object. Every instance of a class points to its own attributes variables. These attributes are defined within the init constructor.
In case of a class attribute its part of the class descriptor, so you cannot delete it from the object attributes like self.__deleteattr__ or add new one with __setattr__ as it alters the class descriptor and reflects on all objects. Such an operation can have devastating effects.
Its very similar to a class variable as well. You can however change the behavior with overriding or reassigning like below
class A:
def b(self):
print("empty")
A.b = lambda self: print(f"second")
a = A()
a.b()
a.b()
I am learning OOP in python and following this and this stackoverflow answers and this post
I understood how class works and how method called and all things but i have some doubts:
Consider this fragment of code:
class Point(object):
def __init__(self,x,y):
self.x = x
self.y = y
def distance(self):
print (self.x)
def bye(self):
print(self.y)
a=Point(1,2)
a.distance()
a.bye()
As i read in tutorial :
when we call a method with some arguments, the corresponding class
function is called by placing the method's object before the first
argument. So, anything like obj.meth(args) becomes Class.meth(obj,
args).
when ObjectA.methodA(arg1, arg2) is called, python internally converts
it for you as:
ClassA.methodA(ObjectA, arg1, arg2)
Now my confusion is why program need to call class with each method ?
Class.meth(obj, args) ??
like when we call a.distance it become Point.distance(a) causes of "self"
when we called a.bye it become Point.bye(a) causes of "self" .
when Point class is necessery with each method if we don't use Point class with each method what will happen?
why can't simply meth(obj, args) works ?
My main doubt is why its called class.some_method with each method when we called with attribute of method . why its needs calls with each one?
#if i am understanding right then its necessary because so that each method can access other methods data like variables and stuff?
The key is
python internally converts it for you
From your standpoint:
meth(self, args) is the syntax you use to define member functions; and
obj.meth(args) is the syntax you use to call member functions.
The meth(obj,args) option is the way procedural languages work. That is often how the implementation works, but expressing the call as obj.meth(args) keeps focus on the object and makes it easier to read which data values (object instances) are being used.
Edit 1 If I understand your question correctly, you are asking why Python needs to know the class when it already has the instance available, and instances know their own types. In fact, Python fetches methods based on the instance all the time. I think the point the tutorial is making is that in Python, the class is the primary place the functions are defined. This is different from some object-oriented languages, in which each instance has its own methods, and they may be completely different from each other. So the tutorial is contrasting the usual approach in Python:
class Foo:
def bar(self):
pass
with an alternative (possible in Python, but not typical):
foo = object() # an empty instance
foo.bar = lambda self: pass
Edit 2 Python methods normally live in the classes, not in the instances. Even if you create 1000 Point objects, there is only one copy of the actual instruction bytes for Point.distance. Those instruction bytes are executed anytime <some point variable>.distance() is called. You are correct that the self parameter is how those instruction bytes know what instance to work on, and how the method can access other data in the passed instance.
Edit 3 self isn't exactly a namespace in the way that local vs. global is. However, it is fair to say that self.foo refers to a foo that is indeed accessible to all the methods of this instance of the current class. Given
a = Point(1,2)
b = Point(3,4)
inside a Point.distance call, self refers to a or b, but not both. So when you call a.distance(), the self.x will be a.x, not b.x. But all methods of Point can access self.x to get whatever the current point's x is.
Edit 4 Suppose you weren't using objects, but instead dictionaries:
a = {'x':1, 'y':2} # make a "point"
b = {'x':3, 'y':4} # make another
def point_distance(point):
print (point['x'])
then you could say:
point_distance(a)
to get the effect of
print (a['x'])
Classes do basically that, with cleaner syntax and some nice benefits. But just as the point parameter to point_distance() refers to one and only one point-like dictionary each time you call point_distance(), the self parameter to Point.distance() refers to one and only one Point instance each time you call <whatever point>.distance().
Because you can have the same method name in different classes, and it needs to call the appropriate one. So if you have
class Class1:
def meth():
print "This is Class 1"
class Class2:
def meth():
print "This is Class 2"
c1 = Class1()
c2 = Class2()
c1.meth() # equivalent to Class1.meth(c1)
c2.meth() # equivalent to Class2.meth(c2)
If it translated c1.meth() to meth(c1), there's no way for the system to know which meth() function to call.
Classes define what is common to all instances of them. Usually this is the code comprising each of its methods. To apply this code to the correct instance object, the language interprets
instance.method(arg1, arg2, ...)
as
class_of_instance.method(instance, arg1, arg2, ...)
so the code is applied to the proper class instance.
Is there an effective way to debug side-effects in Python?
For example, list (or any different mutable object) passed as argument to function.
>>> some_list = [1,2]
>>> some_function_with_huge_call_tree(some_list)
>>> print some_list
[2,1]
How now to determine where in the program the list has been reversed?
One more example, class instance passed as argument:
>>> print obj.x
foo
>>> some_function_with_super_huge_call_tree(obj)
>>> print obj.x
bar
Where a member of the class instance has been changed?
In both cases i want something like this:
pdb.break_on_change(some_list) # case 1
pdb.break_on_change(obj.x) # case 2
Unfortunately, the PDB does not have such a function.
In other words, I'm trying to find a common solution for all cases.
You could put in a mock object that tells you what is done with it.
Here is a (too) minimal example to trace down where a mutating function attribute is called.
def throw(*args, **kwargs):
raise AssertionError()
class Mock(object):
def __init__(self, inner):
self.__inner = inner
def __getattr__(self, name):
if name == 'mutate':
return throw
return getattr(self.__inner, name)
Then use it as in
some_function(Mock(obj))
to find out whether some_function is mutating obj.
This example lacks many features that you would rightfully expect from a mock infrastructure. Probably worst, it doesn't support magic. Maybe you can expand it to fit your needs. Actually, I would be surprised if there wouldn't already be a library that does this.
This question already has answers here:
What is the purpose of the `self` parameter? Why is it needed?
(26 answers)
Closed 6 years ago.
I'm pretty much ignorant of OOP jargon and concepts. I know conceptually what an object is, and that objects have methods. I even understand that in python, classes are objects! That's cool, I just don't know what it means. It isn't clicking with me.
I'm currently trying to understand a few detailed answers that I think will illuminate my understanding of python:
What does the "yield" keyword do in Python?
What is a metaclass in Python?
In the first answer, the author uses the following code as an example:
>>> class Bank(): # let's create a bank, building ATMs
... crisis = False
... def create_atm(self) :
... while not self.crisis :
... yield "$100"
I don't immediately grok what self is pointing to. This is definitely a symptom of not understanding classes, which I will work on at some point. To clarify, in
>>> def func():
... for i in range(3):
... print i
I understand that i points to an item in the list range(3) which, since it is in a function, isn't global. But what does self "point to"?
I'll try to clear up some confusion about classes and objects for you first. Lets look at this block of code:
>>> class Bank(): # let's create a bank, building ATMs
... crisis = False
... def create_atm(self) :
... while not self.crisis :
... yield "$100"
The comment there is a bit deceptive. The above code does not "create" a bank. It defines what a bank is. A bank is something which has a property called crisis, and a function create_atm. That's what the above code says.
Now let's actually create a bank:
>>> x = Bank()
There, x is now a bank. x has a property crisis and a function create_atm. Calling x.create_atm(); in python is the same as calling Bank.create_atm(x);, so now self refers to x. If you add another bank called y, calling y.create_atm() will know to look at y's value of crisis, not x's since in that function self refers to y.
self is just a naming convention, but it is very good to stick with it. It's still worth pointing out that the code above is equivalent to:
>>> class Bank(): # let's create a bank, building ATMs
... crisis = False
... def create_atm(thisbank) :
... while not thisbank.crisis :
... yield "$100"
It may help you to think of the obj.method(arg1, arg2) invocation syntax as purely syntactic sugar for calling method(obj, arg1, arg2) (except that method is looked up via obj's type, and isn't global).
If you view it that way, obj is the first argument to the function, which traditionally is named self in the parameter list. (You can, in fact, name it something else, and your code will work correctly, but other Python coders will frown at you.)
"self" is the instance object automatically passed to the class instance's method when called, to identify the instance that called it. "self" is used to access other attributes or methods of the object from inside the method. (methods are basically just functions that belong to a class)
"self" does not need to be used when calling a method when you already have an available instance.
Accessing the "some_attribute" attribute from inside a method:
class MyClass(object):
some_attribute = "hello"
def some_method(self, some_string):
print self.some_attribute + " " + some_string
Accessing the "some_attribute" attribute from an existing instance:
>>> # create the instance
>>> inst = MyClass()
>>>
>>> # accessing the attribute
>>> inst.some_attribute
"hello"
>>>
>>> # calling the instance's method
>>> inst.some_method("world") # In addition to "world", inst is *automatically* passed here as the first argument to "some_method".
hello world
>>>
Here is a little code to demonstrate that self is the same as the instance:
>>> class MyClass(object):
>>> def whoami(self, inst):
>>> print self is inst
>>>
>>> local_instance = MyClass()
>>> local_instance.whoami(local_instance)
True
As mentioned by others, it's named "self" by convention, but it could be named anything.
self refers to the current instance of Bank. When you create a new Bank, and call create_atm on it, self will be implicitly passed by python, and will refer to the bank you created.
I don't immediately grok what self is pointing to. This is definitely a symptom of not understanding classes, which I will work on at some point.
self is an argument passed in to the function. In Python, this first argument is implicitly the object that the method was invoked on. In other words:
class Bar(object):
def someMethod(self):
return self.field
bar = Bar()
bar.someMethod()
Bar.someMethod(bar)
These last two lines have equivalent behavior. (Unless bar refers to an object of a subclass of Bar -- then someMethod() might refer to a different function object.)
Note that you can name the "special" first argument anything you want -- self is just a convention for methods.
I understand that i points to an item in the list range(3) which, since it is in a function, isn't global. But what does self "point to"?
The name self does not exist in the context of that function. Attempting to use it would raise a NameError.
Example transcript:
>>> class Bar(object):
... def someMethod(self):
... return self.field
...
>>> bar = Bar()
>>> bar.field = "foo"
>>> bar.someMethod()
'foo'
>>> Bar.someMethod(bar)
'foo'
>>> def fn(i):
... return self
...
>>> fn(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in fn
NameError: global name 'self' is not defined
The reason "self" is there (by convention) is that when the Python runtime sees a call of the form Object.Method(Param1,Param2), it calls Method with parameters (Object,Param1,Param2). So if you call that first parameter "self", everyone will know what you are talking about.
The reason you have to do this is the subject of another question.
As far as a metaclass, it's something rarely used. You might want to look at:
http://python-history.blogspot.com/2009/04/metaclasses-and-extension-classes-aka.html, the original author and current Benevolent Dictator For Life of Python explains what this is, and how it came to be. He also has a nice article on some possible uses, but most people never directly use it at all.
One Rubyist's perspective (Ruby is my first programming language so I apologize for whatever oversimplified, potentially wrong abstractions I'm about to use)
as far as I can tell, the dot operator, for example:
os.path
is such that os gets passed into path() as its first variable "invisibly"
It is as if os.path is REALLY this:
path(os)
If there were a daisy chain I'd imagine that this:
os.path.filename
Would be sort of like this in reality*:
filename(path(os))
Here comes the offensive part
So with the self variable all that's doing is allowing the CLASS METHOD (from a rubyist perspective python 'instance methods' appear to be class methods...) to act as an instance method by getting an instance passed into it as its first variable (via the "sneaky" dot method above) which is called self by convention. Were self not an instance
c = ClassName()
c.methodname
but the class itself:
ClassName.methodname
the class would get passed in rather than the instance.
OK, also important to remember is that the __init__ method is called "magic" by some. So don't worry about what gets passed into generate a new instance. To be honest its probably nil.
self refers to an instance of the class.
Alright, so I was taking a look at some source when I came across this:
>>> def __parse(self, filename):
... "parse ID3v1.0 tags from MP3 file"
... self.clear()
... try:
... fsock = open(filename, "rb", 0)
... try:
... fsock.seek(-128, 2)
... tagdata = fsock.read(128)
... finally:
... fsock.close()
... if tagdata[:3] == 'TAG':
... for tag, (start, end, parseFunc) in self.tagDataMap.items():
... self[tag] = parseFunc(tagdata[start:end])
... except IOError:
... pass
...
So, I decided to test it out.
>>> __parse("blah.mp3")
And, I received this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __parse() takes exactly 2 arguments (1 given)
This wouldn't be the first time I've encountered this, I keep thinking I'm meant to include self in the argument parameter list, but I know that that's not right. Could someone explain to me why this happens a lot with code I try to play around with, I suppose its due to my level of understanding about the terms, I barely even understand what init or self does, or why it's relevant. def x(b): print b is the same as def x(self, b): self.b = b print self.b isn't it? Why does it matter so much!
I just want a basic explanation, so I can get this out of my mind,thanks.
The def __parse was inside some class definition.
You can't pull the method defs out of the class definitions. The method function definition is part of the class.
Look at these two examples:
def add( a, b ):
return a + b
And
class Adder( object ):
def __init__( self ):
self.grand_total = 0
def add( self, a, b ):
self.grand_total += a+b
return a+b
Notes.
The function does not use self.
The class method does use self. Generally, all instance methods will use self, unless they have specific decorators like #classmethod that say otherwise.
The function doesn't depend on anything else else.
The class method depends on being called by an instance of the class Adder; further, it depends on that instance of the class Adder having been initialized correctly. In this case, the initialization function (__init__) has assured that each instance of Adder always has an instance variable named grand_total and that instance variable has an initial value of 0.
You can't pull the add method function out of the Adder class and use it separately. It is not a stand-alone function. It was defined inside the class and has certain expectations because of that location inside the class.
Functions/methods can be written outside of a class and then used for a technique in Python called monkeypatching:
class C(object):
def __init__(self):
self.foo = 'bar'
def __output(self):
print self.foo
C.output = __output
c = C()
c.output()
Looks like you're a bit confused about classes and object-oriented programming. The 'self' thing is one of the gotchas in python for people coming from other programming languages. IMO the official tutorial doesn't handle it too well. This tutorial seems quite good.
If you've ever learnt java, self in python is very similar to this in java. The difference is that python requires you to list self as the first argument to every function in a class definition.
If python is your first programming language (or your first object-oriented language), you could remember this as a simple rule-of-thumb: if you're defining a function that's part of a class, you need to include self as the first argument. If you're defining a function that's not part of a class, you shouldn't include self in the arguments. You can't take a class function and make it stand-alone without doing some (or possibly a lot of) extra coding. Finally, never include self as an argument when calling a function.
There are exceptions to those rules, but they're not worth worrying about now.
self is passed in automatically by the instancemethod wrapper on classes. This function isn't wrapped; it's not a method, it's just a function. It doesn't even make sense without being attached to a class, since it needs the self parameter.
As an aside, it is possible to create static methods of a class in Python. The simplest way to do this is via decorators (e.g. #staticmethod). I suspect this kind of thing is usually not the Pythonic solution though.