what gets returned when you return 'self' inside a python class? where do we exactly use return 'self'? In the below example what does self exactly returns
class Fib:
'''iterator that yields numbers in the Fibonacci sequence'''
def __init__(self, max):
self.max = max
def __iter__(self):
self.a = 0
self.b = 1
return self
def __next__(self):
fib = self.a
if fib > self.max:
raise StopIteration
self.a, self.b = self.b, self.a + self.b
print(self.a,self.b,self.c)
return fib
Python treats method calls like object.method() approximately like method(object). The docs say that "call x.f() is exactly equivalent to MyClass.f(x)". This means that a method will receive the object as the first argument. By convention in the definition of methods, this first argument is called self.
So self is the conventional name of the object owning the method.
Now, why would we want to return self? In your particular example, it is because the object implements the iterator protocol, which basically means it has __iter__ and __next__ methods. The __iter__ method must (according to the docs) "Return the iterator object itself", which is exactly what is happening here.
As an aside, another common reason for returning self is to support method chaining, where you would want to do object.method1().method2().method3() where all those methods are defined in the same class. This pattern is quite common in libraries like pandas.
The keyword self is used to refer to the instance that you are calling the method from.
This is particularly useful for chaining. In your example, let's say we want to call __next__() on an initialized Fib instance. Since __iter__() returns self, the following are equivalent :
obj = Fib(5)
obj.__iter__() # Initialize obj
obj.__next__()
And
obj = Fib(5).__iter__() # Create AND initialize obj
obj.__next__()
In your particular example, the self keyword returns the instance of the Fib class from which you are calling __iter__() (called obj in my small snippet).
Hope it'll be helpful.
Partial Answer:
When you return self, you return the class instance. For example:
class Foo:
def __init__(self, a):
self.a = a
def ret_self(self):
return self
If I create an instance and run ret_self, you will see that they both refer to the same instance:
>>> x = Foo("a")
>>> x
<__main__.Foo instance at 0x0000000002823D48>
>>> x.ret_self()
<__main__.Foo instance at 0x0000000002823D48>
In other words, both x and x.ret_self() return the same reference to that class instance.
self is actually another way of saying "this instance of Foo". Hence, instance variables are self.a in the class.
When will you need this? I don't have the experience to tell you and I do not want to give possibly misleading information that I am unsure of. I will leave it to someone else to expound on this answer.
Please do not accept this answer.
Related
I am making a constructor in Python. When called with an existing object as its input, it should set the "new" object to that same object. Here is a 10 line demonstration:
class A:
def __init__(self, value):
if isinstance(value, A):
self = value
else:
self.attribute = value
a = A(1)
b = A(a)#a and b should be references to the same object
print("b is a", b is a)#this should be true: the identities should be the same
print("b == a", b == a)#this should be true: the values should be the same
I want the object A(a) constructed from the existing object a to be a. Why is it not? To be clear, I want A(a) to reference the same object as a, NOT a copy.
self, like any other argument, is among the local variables of a function or method. Assignment to the bare name of a local variable never affects anything outside of that function or method, it just locally rebinds that name.
As a comment rightly suggests, it's unclear why you wouldn't just do
b = a
Assuming you have a sound reason, what you need to override is not __init__, but rather __new__ (then take some precaution in __init__ to avoid double initialization). It's not an obvious course so I'll wait for you to explain what exactly you're trying to accomplish.
Added: having clarified the need I agree with the OP that a factory function (ideally, I suggest, as a class method) is better -- and clearer than __new__, which would work (it is a class method after all) but in a less-sharply-clear way.
So, I would code as follows:
class A(object):
#classmethod
def make(cls, value):
if isinstance(value, cls): return value
return cls(value)
def __init__(self, value):
self.attribute = value
Now,
a = A.make(1)
b = A.make(a)
accomplishes the OP's desires, polymorphically over the type of argument passed to A.make.
The only way to make it work exactly as you have it is to implement __new__, the constructor, rather than __init__, the initialiser (the behaviour can get rather complex if both are implemented). It would also be wise to implement __eq__ for equality comparison, although this will fall back to identity comparison. For example:
>>> class A(object):
def __new__(cls, value):
if isinstance(value, cls):
return value
inst = super(A, cls).__new__(cls)
inst.attribute = value
return inst
def __eq__(self, other):
return self.attribute == other.attribute
>>> a = A(1)
>>> b = A(a)
>>> a is b
True
>>> a == b
True
>>> a == A(1)
True # also equal to other instance with same attribute value
You should have a look at the data model documentation, which explains the various "magic methods" available and what they do. See e.g. __new__.
__init__ is an initializer, not a constructor. You would have to mess around with __new__ to do what you want, and it's probably not a good idea to go there.
Try
a = b = A(1)
instead.
If you call a constructor, it's going to create a new object. The simplest thing is to do what hacatu suggested and simply assign b to a's value. If not, perhaps you could have an if statement checking if the value passed in is equal to the object you want referenced and if it is, simply return that item before ever calling the constructor. I haven't tested so I'm not sure if it'd work.
I want B to be a subclass of some class A, and I want to override A.__new__1. Typically, my code for this would have this basic structure:
class B(A):
def __new__(cls, ...):
ret = super(B, cls).__new__(cls, ...)
# modify ret as needed
return ret # type(ret) is cls
# ...
In this way, type(B(...)) is indeed B. (Note: the ellipses in super(B, cls).__new__(cls, ...) need not represent the same items as those represented by the ellipses in B.__new__'s signature.)
But suppose now that I want to use the value returned by some factory method/function A_Factory (which returns an object of type A) as the starting value for the variable ret in the constructor. If I simply coded this:
class B(A):
def __new__(cls, ...):
ret = A_Factory(...)
# modify ret as needed
return ret # type(ret) is A
...then type(B(...)) would be A, not B.
What's the correct way to set class of ret in the second version of B.__new__ above so that, type(B(...)) is B, and, more generally, so that, for any direct or indirect subclass C of B, type(C(...)) is C?
1 I know that one normally does not need to override a parent class's __new__, but here I'm interested in the uncommon case where one needs to.
You can set the __class__ attribute of the instance to be the cls passed into __new__:
class A(object):
def __init__(self):
self.x = 2
def A_Factory():
return A()
class B(A):
def __new__(cls):
ret = A_Factory()
ret.__class__ = cls # Override the class.
return ret # type(ret) is cls
class C(B):
pass
if __name__ == "__main__":
b = B()
print(b.x)
print(type(b))
print(isinstance(b, B))
print(isinstance(b, A))
c = C()
print(type(C))
Output:
2
<class '__main__.B'>
True
True
<class '__main__.C'>
There are some limitations to this, though. The commit that added the ability to assign to __class__ lists them:
Make __class__ assignment possible, when the object structures are the
same. I hope the test for structural equivalence is stringent enough.
It only allows the assignment if the old and new types:
have the same basic size
have the same item size
have the same dict offset
have the same weaklist offset
have the same GC flag bit
have a common base that is the same except for maybe the dict and weaklist (which may have been added separately at the same offsets
in both types)
They also need to have the identical __slots__, if you have that defined on either class. In practice, this usually just means that both classes need to be implemented in Python. Trying to do it with a regular Python class and int or tuple or something else implemented in C isn't likely to work.
Given two non-related classes A and B, how to call A.method with an object of B as self?
class A:
def __init__(self, x):
self.x = x
def print_x(self):
print self.x
class B:
def __init__(self, x):
self.x = x
a = A('spam')
b = B('eggs')
a.print_x() #<-- spam
<magic>(A.print_x, b) #<-- 'eggs'
In Python 3.x you can simply do what you want:
A.print_x(b) #<-- 'eggs'
If you only have an instance of 'A', then get the class first:
a.__class__.print_x(b) #<-- 'eggs'
In Python 2.x (which the OP uses) this doesn't work, as noted by the OP and explained by Amber in the comments:
This is a difference between Python 2.x and Python 3.x - methods in
3.x don't enforce being passed the same class.
More details (OP edit)
In python 2, A.print_x returns an "unbound method", which cannot be directly applied to other classes' objects:
When an unbound user-defined method object is called, the underlying function (im_func) is called, with the restriction that the first argument must be an instance of the proper class (im_class) or of a derived class thereof. >> http://docs.python.org/reference/datamodel.html
To work around this restriction, we first have to obtain a "raw" function from a method, via im_func or __func__ (2.6+), which then can be called passing an object. This works on both classes and instances:
# python 2.5-
A.print_x.im_func(b)
a.print_x.im_func(b)
# python 2.6+
A.print_x.__func__(b)
a.print_x.__func__(b)
In python 3 there's no such thing anymore as unbound method.
Unbound methods are gone for good. ClassObject.method returns an
ordinary function object, instance.method still returns a bound
method object. >> http://www.python.org/getit/releases/3.0/NEWS.txt
Hence, in python 3, A.print_x is just a function, and can be called right away and a.print_x still has to be unbounded:
# python 3.0+
A.print_x(b)
a.print_x.__func__(b)
You don't (well, it's not that you can't throw enough magic at it to make it work, it's that you just shouldn't). If the function is supposed to work with more than one type, make it... a function.
# behold, the magic and power of duck typing
def fun(obj):
print obj.x
class A:
x = 42
class B:
x = 69
fun(A())
fun(B())
I don't know why you would really want to do this, but it is possible:
>>> class A(object):
... def foo(self):
... print self.a
...
>>> class B(object):
... def __init__(self):
... self.a = "b"
...
>>> x = A()
>>> y = B()
>>> x.foo.im_func(y)
b
>>> A.foo.im_func(y)
b
An instance method (a class instance's bound method) has a property called im_func which refers to the actual function called by the instance method, without the instance/class binding. The class object's version of the method also has this property.
I have a class that need to make some magic with every operator, like __add__, __sub__ and so on.
Instead of creating each function in the class, I have a metaclass which defines every operator in the operator module.
import operator
class MetaFuncBuilder(type):
def __init__(self, *args, **kw):
super().__init__(*args, **kw)
attr = '__{0}{1}__'
for op in (x for x in dir(operator) if not x.startswith('__')):
oper = getattr(operator, op)
# ... I have my magic replacement functions here
# `func` for `__operators__` and `__ioperators__`
# and `rfunc` for `__roperators__`
setattr(self, attr.format('', op), func)
setattr(self, attr.format('r', op), rfunc)
The approach works fine, but I think It would be better if I generate the replacement operator only when needed.
Lookup of operators should be on the metaclass because x + 1 is done as type(x).__add__(x,1) instead of x.__add__(x,1), but it doesn't get caught by __getattr__ nor __getattribute__ methods.
That doesn't work:
class Meta(type):
def __getattr__(self, name):
if name in ['__add__', '__sub__', '__mul__', ...]:
func = lambda:... #generate magic function
return func
Also, the resulting "function" must be a method bound to the instance used.
Any ideas on how can I intercept this lookup? I don't know if it's clear what I want to do.
For those questioning why do I need to this kind of thing, check the full code here.
That's a tool to generate functions (just for fun) that could work as replacement for lambdas.
Example:
>>> f = FuncBuilder()
>>> g = f ** 2
>>> g(10)
100
>>> g
<var [('pow', 2)]>
Just for the record, I don't want to know another way to do the same thing (I won't declare every single operator on the class... that will be boring and the approach I have works pretty fine :). I want to know how to intercept attribute lookup from an operator.
Some black magic let's you achieve your goal:
operators = ["add", "mul"]
class OperatorHackiness(object):
"""
Use this base class if you want your object
to intercept __add__, __iadd__, __radd__, __mul__ etc.
using __getattr__.
__getattr__ will called at most _once_ during the
lifetime of the object, as the result is cached!
"""
def __init__(self):
# create a instance-local base class which we can
# manipulate to our needs
self.__class__ = self.meta = type('tmp', (self.__class__,), {})
# add operator methods dynamically, because we are damn lazy.
# This loop is however only called once in the whole program
# (when the module is loaded)
def create_operator(name):
def dynamic_operator(self, *args):
# call getattr to allow interception
# by user
func = self.__getattr__(name)
# save the result in the temporary
# base class to avoid calling getattr twice
setattr(self.meta, name, func)
# use provided function to calculate result
return func(self, *args)
return dynamic_operator
for op in operators:
for name in ["__%s__" % op, "__r%s__" % op, "__i%s__" % op]:
setattr(OperatorHackiness, name, create_operator(name))
# Example user class
class Test(OperatorHackiness):
def __init__(self, x):
super(Test, self).__init__()
self.x = x
def __getattr__(self, attr):
print "__getattr__(%s)" % attr
if attr == "__add__":
return lambda a, b: a.x + b.x
elif attr == "__iadd__":
def iadd(self, other):
self.x += other.x
return self
return iadd
elif attr == "__mul__":
return lambda a, b: a.x * b.x
else:
raise AttributeError
## Some test code:
a = Test(3)
b = Test(4)
# let's test addition
print(a + b) # this first call to __add__ will trigger
# a __getattr__ call
print(a + b) # this second call will not!
# same for multiplication
print(a * b)
print(a * b)
# inplace addition (getattr is also only called once)
a += b
a += b
print(a.x) # yay!
Output
__getattr__(__add__)
7
7
__getattr__(__mul__)
12
12
__getattr__(__iadd__)
11
Now you can use your second code sample literally by inheriting from my OperatorHackiness base class. You even get an additional benefit: __getattr__ will only be called once per instance and operator and there is no additional layer of recursion involved for the caching. We hereby circumvent the problem of method calls being slow compared to method lookup (as Paul Hankin noticed correctly).
NOTE: The loop to add the operator methods is only executed once in your whole program, so the preparation takes constant overhead in the range of milliseconds.
The issue at hand is that Python looks up __xxx__ methods on the object's class, not on the object itself -- and if it is not found, it does not fall back to __getattr__ nor __getattribute__.
The only way to intercept such calls is to have a method already there. It can be a stub function, as in Niklas Baumstark's answer, or it can be the full-fledged replacement function; either way, however, there must be something already there or you will not be able to intercept such calls.
If you are reading closely, you will have noticed that your requirement for having the final method be bound to the instance is not a possible solution -- you can do it, but Python will never call it as Python is looking at the class of the instance, not the instance, for __xxx__ methods. Niklas Baumstark's solution of making a unique temp class for each instance is as close as you can get to that requirement.
It looks like you are making things too complicated. You can define a mixin class and inherit from it. This is both simpler than using metaclasses and will run faster than using __getattr__.
class OperatorMixin(object):
def __add__(self, other):
return func(self, other)
def __radd__(self, other):
return rfunc(self, other)
... other operators defined too
Then every class you want to have these operators, inherit from OperatorMixin.
class Expression(OperatorMixin):
... the regular methods for your class
Generating the operator methods when they're needed isn't a good idea: __getattr__ is slow compared to regular method lookup, and since the methods are stored once (on the mixin class), it saves almost nothing.
If you want to achieve your goal without metaclasses, you can append the following to your code:
def get_magic_wrapper(name):
def wrapper(self, *a, **kw):
print('Wrapping')
res = getattr(self._data, name)(*a, **kw)
return res
return wrapper
_magic_methods = ['__str__', '__len__', '__repr__']
for _mm in _magic_methods:
setattr(ShowMeList, _mm, get_magic_wrapper(_mm))
It reroutes the methods in _magic_methods to the self._data object, by adding these attributes to the class iteratively. To check if it works:
>>> l = ShowMeList(range(8))
>>> len(l)
Wrapping
8
>>> l
Wrapping
[0, 1, 2, 3, 4, 5, 6, 7]
>>> print(l)
Wrapping
[0, 1, 2, 3, 4, 5, 6, 7]
Consider the following (broken) code:
import functools
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partial(f,1)
x=Foo()
x.g(2)
What I want to do is take the function f and partially apply it, resulting in a function g(self,b). I would like to use this function as a method, however this does not currently work and instead I get the error
Traceback (most recent call last):
File "test.py", line 8, in <module>
x.g(2)
TypeError: f() takes exactly 3 arguments (2 given)
Doing x.g(x,2) however works, so it seem the issue is that g is considered a "normal" function instead of a method of the class. Is there a way to get x.g to behave like a method (i.e implicitly pass the self parameter) instead of a function?
There are two issues at hand here. First, for a function to be turned into a method it must be stored on the class, not the instance. A demonstration:
class Foo(object):
def a(*args):
print 'a', args
def b(*args):
print 'b', args
Foo.b = b
x = Foo()
def c(*args):
print 'c', args
x.c = c
So a is a function defined in the class definition, b is a function assigned to the class afterwards, and c is a function assigned to the instance. Take a look at what happens when we call them:
>>> x.a('a will have "self"')
a (<__main__.Foo object at 0x100425ed0>, 'a will have "self"')
>>> x.b('as will b')
b (<__main__.Foo object at 0x100425ed0>, 'as will b')
>>> x.c('c will only recieve this string')
c ('c will only recieve this string',)
As you can see there is little difference between a function defined along with the class, and one assigned to it later. I believe there is actually no difference as long as there is no metaclass involved, but that is for another time.
The second problem comes from how a function is actually turned into a method in the first place; the function type implements the descriptor protocol. (See the docs for details.) In a nutshell, the function type has a special __get__ method which is called when you perform an attribute lookup on the class itself. Instead of you getting the function object, the __get__ method of that function object is called, and that returns a bound method object (which is what supplies the self argument).
Why is this a problem? Because the functools.partial object is not a descriptor!
>>> import functools
>>> def f(*args):
... print 'f', args
...
>>> g = functools.partial(f, 1, 2, 3)
>>> g
<functools.partial object at 0x10042f2b8>
>>> g.__get__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'functools.partial' object has no attribute '__get__'
There are a number of options you have at this point. You can explicitly supply the self argument to the partial:
import functools
class Foo(object):
def __init__(self):
def f(self, a, b):
print a + b
self.g = functools.partial(f, self, 1)
x = Foo()
x.g(2)
...or you would imbed the self and value of a in a closure:
class Foo(object):
def __init__(self):
a = 1
def f(b):
print a + b
self.g = f
x = Foo()
x.g(2)
These solutions are of course assuming that there is an as yet unspecified reason for assigning a method to the class in the constructor like this, as you can very easily just define a method directly on the class to do what you are doing here.
Edit: Here is an idea for a solution assuming the functions may be created for the class, instead of the instance:
class Foo(object):
pass
def make_binding(name):
def f(self, *args):
print 'Do %s with %s given %r.' % (name, self, args)
return f
for name in 'foo', 'bar', 'baz':
setattr(Foo, name, make_binding(name))
f = Foo()
f.foo(1, 2, 3)
f.bar('some input')
f.baz()
Gives you:
Do foo with <__main__.Foo object at 0x10053e3d0> given (1, 2, 3).
Do bar with <__main__.Foo object at 0x10053e3d0> given ('some input',).
Do baz with <__main__.Foo object at 0x10053e3d0> given ().
This will work. But I'm not sure if this is what you are looking for
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partial(f,1, self) # <= passing `self` also.
x = Foo()
x.g(2)
this is simply a concrete example of what i believe is the most correct (and therefore pythonic :) way to solve -- as the best solution (definition on a class!) was never revealed -- #MikeBoers explanations are otherwise solid.
i've used this pattern quite a bit (recently for an proxied API), and it's survived untold production hours without the slightest irregularity.
from functools import update_wrapper
from functools import partial
from types import MethodType
class Basic(object):
def add(self, **kwds):
print sum(kwds.values())
Basic.add_to_one = MethodType(
update_wrapper(partial(Basic.add, a=1), Basic.add),
None,
Basic,
)
x = Basic()
x.add(a=1, b=9)
x.add_to_one(b=9)
...yields:
10
10
...the key take-home-point here is MethodType(func, inst, cls), which creates an unbound method from another callable (you can even use this to chain/bind instance methods to unrelated classes... when instantiated+called the original instance method will receive BOTH self objects!)
note the exclusive use of keyword arguments! while there might be a better way to handle, args are generally a PITA because the placement of self becomes less predictable. also, IME anyway, using *args, and **kwds in the bottom-most function has proven very useful later on.
functools.partialmethod() is available since python 3.4 for this purpose.
import functools
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partialmethod(f,1)
x=Foo()
x.g(2)