Develop Reference

How does Python turn a function into a method?

I know that functions are just descriptors, like this:
def func(self):
print(self.name)
class C:
def __init__(self, name):
self.name = name
C.func = func
c = C("foo")
c.func()
I thought at first that c.func equals C.func.__get__(c)，yes，C.func.__get__(c) return a bound method. But when I set the __get__ of func to None, c.func still returns a bound method.
def func(self):
print(self.name)
class C:
def __init__(self, name):
self.name = name
func.__get__ = None
C.func = func
c = C("foo")
c.func
output:
<bound method func of <__main__.C object at 0x0000027EB23BF088>>
So I'm confused. Moreover, I found that when calling a function from an instance, Python actually calls the class's ___getAttribute__ method, which returns a bound method.
def func(self):
print(self.name)
func.__get__ = None
class C:
def __getattribute__(self, name):
r = super().__getattribute__(name)
print(r) # r is a bound method already
return r
def __init__(self, name):
self.name = name
C.func = func
c = C("foo")
c.func
output:
<bound method func of <__main__.C object at 0x0000027EB243D1C8>>
func.__get__ doesn't seem to have any effect. So, What happended in __getattribute__? How does Python turn a function into a method? I've Googled and done some research, but I still can't find the answer.
Maybe I'm making things complicated, In my understanding, function is itself a descriptor, but just like the code below, I set the func to None, it works normally:
class C:
def func(self):
print('hello world')
func.__get__ = None
c = C()
c.func()
but if it's a descriptor, it will raise TypeError:
class C:
class D:
def __get__(self, inst, cls):
if inst is None:
return self
return 'hello world'
D.__get__ = None
func = D()
c = C()
c.func

Well, if I understand correctly from what I found. (Since I didn't know the descriptors, that's exactly why I like to help, still learning)
First, let's look at __getattr__ and __getattribute__.
Let's have an empty class A
class A:
pass
If I initialize an object and try to call a property, because there is none at the moment, we get AttributeError.
a = A()
a.some_property
The following occurs:
Simple check of flow:
class FlowDemo:
def __init__(self):
self.inited_property = True
def __getattribute__(self, item):
if item in ('__class__', '__len__') : # For less spam of getting this attribute, if you want, you can remove condition.
print('Get Attribute', item)
# Call default behavior
return super().__getattribute__(item)
def __getattr__(self, item):
print('Get Attr', item)
if item == 'some_magic_name':
return "It's magic!"
raise AttributeError
fd = FlowDemo()
fd.inited_property
# Get Attribute inited_property
# True
fd.some_magic_property
# Get Attribute some_magic_name
# Get Attr some_magic_name
# "It's magic!"
fd.some_property
# Get Attribute some_property
# Get Attr some_property
# Traceback (most recent call last):
# File "<input>", line 1, in <module>
# File "stack-class-property-and-descriptors.py", line 67, in # __getattr__
# raise AttributeError
# AttributeError
This is probably understandable, including the use. But to be sure, I'll give an example. This logic is used as a dynamic representation of the result from the databases (mapping of attributes to ordinary dict, list, etc.).
But it can also be just logic for accessing an attribute (property), such as an access counter or validation (but this applies to __setattr__ and __setattribute__)
And what about descriptors?
First let's look at data-descriptors, they are easier for me to understand.
This is a class or decoder that has __get__ and one or both of __set__ and __delete__.
Once this is defined, python, when used in the property definition with it and then does not return a class but the value it obtains through __get__, does not overwrite an already declared class when declaring a value, but uses its __set__.
Example:
class WeekDayDescriptor:
def __init__(self):
self.__week_day = 0
def __get__(self, instance, owner=None):
return self.__week_day
def __set__(self, instance, value):
if not isinstance(value, int):
raise TypeError('Value must be int')
if not (0 <= value < 6):
raise ValueError('Value must be in range 0 - 6')
self.__week_day = value
class Calendar:
week_day = WeekDayDescriptor()
def __init__(self, week_day):
self.week_day = week_day
Demo:
c = Calendar(9)
# ValueError: Value must be in range 0-6
c = Calendar('6')
# TypeError: Value must be int
c = Calendar(3)
c.week_day = 6
c.week_day = 10
# ValueError: Value must be in range 0-6
c.week_day = 'monday'
# TypeError: Value must be int
Decorator descriptor:
class Calendar:
#property
def week_day(self):
return self.__week_day
#week_day.setter
def week_day(self, week_day):
if not isinstance(week_day, int):
raise TypeError('Value must be int')
if not (0 <= week_day < 6):
raise ValueError('Value must be in range 0 - 6')
self.__week_day = week_day
def __init__(self, week_day):
self.week_day = week_day
pass
And now for non-data descriptors...
A non-data descriptor is one that has only __get__.
As I understand it, each method automatically has its own descriptor, thanks to which the functions get references to the object - self.
We can write our own descriptor for a function / method, but it's not that straightforward, we have to help ourselves and get around it a bit.
def function_as_method(self, value):
print(self, value)
class HelperDescriptor:
def __get__(self, instance, owner):
def wrapper(*args, **kwargs):
return function_as_method(instance, *args, **kwargs)
return wrapper
class Foo:
baz = HelperDescriptor()
>>> bar = Foo()
>>> bar.baz(1)
<__main__.Foo object at 0x7f64f7768b70> 1
Source of last code block, but in czech lang.
And finally, your mentioned problem, when we set __get__ to None and you still get a reference to the function.
It's simple, python doesn't directly distinguish between primitive data types and functions, it's all a variable (or attribute / property) that has a value. Whether it's just value or it's callable is a different matter.
def f(): return True
print(type(f), f())
# <class 'function'> True
f = 123
print(type(f), f)
# <class 'int'> 123
Therefore, when we ask for the obj.func method or call it obj.func() directly, the first two changed magic is called first - __getattribute__ and __getattr__.
And if we call a method, it is called only after we get a reference to a function in memory.
Again a simple example:
def func(self, value):
print('Printing:', value)
class PrintDescriptor:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
def wrapper(*args, **kwargs):
print(f"Calling '{self.name}' method")
return func(instance, *args, **kwargs)
return wrapper
class B:
foo = PrintDescriptor('foo')
bar = PrintDescriptor('bar')
def __getattribute__(self, item):
if item not in ('__len__', '__class__', '__dict__'):
print('Get Attribute', item)
return super().__getattribute__(item)
Demo:
b = B()
b.foo
# Get Attribute foo
# <function PrintDescriptor.__get__.<locals>.wrapper at 0x7f774a782ee0>
b.foo(2)
# Get Attribute foo
# Calling 'foo' method
# Printing: 2
b.bar(4)
# Get Attribute bar
# Calling 'bar' method
# Printing: 4
Sources:
https://www.datacamp.com/community/tutorials/python-descriptors#above1
https://blog.milde.cz/post/319-pokrocile-techniky-v-pythonu-deskriptory/
Python Doc, __get__
Python Docs, __getattribute__
Python Docs, __getattr__

How to avoid creating objects with same values?

I need to create a class whose instances can't have same values. If you create instance with value that have already been used you'll get old same instance.
I did it using special class method:
class A():
instances = []
def __init__(self, val):
self.val = val
#classmethod
def new(cls, val):
"""
Return instance with same value or create new.
"""
for ins in cls.instances:
if ins.val == val:
return ins
new_ins = A(val)
cls.instances.append(new_ins)
return new_ins
a1 = A.new("x")
a2 = A.new("x")
a3 = A.new("y")
print a1 # <__main__.A instance at 0x05B7FD00> S\ /M\
print a2 # <__main__.A instance at 0x05B7FD00> \A/ \E
print a3 # <__main__.A instance at 0x05B7FD28>
Is there a way to do it more elegant, without using .new method?

You could try functools.lru_cache.
For example:
from functools import lru_cache
class A:
#lru_cache()
def __new__(cls, arg):
return super().__new__(cls)
def __init__(self, arg):
self.n = arg
Sample usage:
>>> a1 = A('1')
>>> a2 = A('1')
>>> a1 is a2
True
>>> a1.n
'1'
>>> a2.n
'1'
Alternatively you could try building a custom caching class, as pointed out by Raymond Hettinger in this tweet: https://twitter.com/raymondh/status/977613745634471937.

This can be done by overriding the __new__ method, which is responsible for creating new instances of a class. Whenever you create a new instance you store it in a dict, and if the dict contains a matching instance then you return it instead of creating a new one:
class A:
instances = {}
def __new__(cls, val):
try:
return cls.instances[val]
except KeyError:
pass
obj = super().__new__(cls)
cls.instances[val] = obj
return obj
def __init__(self, val):
self.val = val
a = A(1)
b = A(2)
c = A(1)
print(a is b) # False
print(a is c) # True
One downside of this solution is that the __init__ method will be called regardless of whether the instance is a newly created one or one that's been stored in the dict. This can cause problems if your constructor has undesired side effects:
class A:
...
def __init__(self, val):
self.val = val
self.foo = 'foo'
a = A(1)
a.foo = 'bar'
b = A(1)
print(a.foo) # output: foo
Notice how a's foo attribute changed from "bar" to "foo" when b was created.
Another option is to use a metaclass and override its __call__ method:
class MemoMeta(type):
def __new__(mcs, name, bases, attrs):
cls = super().__new__(mcs, name, bases, attrs)
cls.instances = {}
return cls
def __call__(cls, val):
try:
return cls.instances[val]
except KeyError:
pass
obj = super().__call__(val)
cls.instances[val] = obj
return obj
class A(metaclass=MemoMeta):
def __init__(self, val):
self.val = val
self.foo = 'foo'
This bypasses the problem with __init__ being called on existing instances:
a = A(1)
a.foo = 'bar'
b = A(1)
print(a.foo) # output: bar

If you really want to make it more elegant, implement the duplicate check in __new__, so it will be performed when you call A(something).
Just do it in __new__:
def __new__(cls, val=None):
for i in cls.instances:
if val == i.val:
return i
return object.__new__(cls)

Why does my __init__ function need to be #classmethod?

Here is a code snippet I wrote as a test. I noticed that if I don't define the init method as a classmethod, the code doesn't run:
class A(object):
def __init__(self):
self.value = 0
self.add(1)
#classmethod
def add(self, arg):
self.value += arg
class B(A):
#classmethod
def add(self, arg):
self.value += arg * 2
if __name__ == '__main__':
a = A()
b = B()
print a.value
print b.value
This outputs:
Traceback (most recent call last):
File "inherit.py", line 17, in <module>
a = A()
File "inherit.py", line 4, in __init__
self.add(1)
File "inherit.py", line 8, in add
self.value += arg
AttributeError: type object 'A' has no attribute 'value'
However if I change my init function to be #classmethod, the code works as intended:
class A(object):
#classmethod
def __init__(self):
self.value = 0
self.add(1)
#classmethod
def add(self, arg):
self.value += arg
class B(A):
#classmethod
def add(self, arg):
self.value += arg * 2
if __name__ == '__main__':
a = A()
b = B()
print a.value
print b.value
Output:
1
2
I was under the impression init was by default a class method whose first argument must be self. What is going on?

The problem is that you have add marked as a classmethod, but it isn't. Take out the #classmethod from the adds and it should work.

Implementing Class.magic_class_method

Python has a magic __getattr__ method that allows custom values to be returned:
class A(object):
def __getattr__(self, name):
return name
B = A()
print B.foo # 'foo'
However, calling A.foo has no similar effect, because A is not an instance.
Using metaclasses, Google App Engine raises this error on instantiation:
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 913, in __init__
key_name.__class__.__name__)
BadKeyError: Name must be string type, not tuple
Assuming the referenced question is correctly implemented, what other ways can a magic class __getattr__ be implemented?

The metaclass solution should work, here is an example:
class GetAttrMeta(type):
def __getattr__(self, name):
return name
class A(object):
__metaclass__ = GetAttrMeta
print A.foo # 'foo'
Or with Python 3.x:
class GetAttrMeta(type):
def __getattr__(self, name):
return name
class A(object, metaclass=GetAttrMeta):
pass
print(A.foo) # 'foo'

Not sure if this answers your question, but maybe checkout property descriptors ..
class RevealAccess(object):
"""A data descriptor that sets and returns values
normally and prints a message logging their access.
"""
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print 'Retrieving', self.name
return self.val
def __set__(self, obj, val):
print 'Updating' , self.name
self.val = val
>>> class MyClass(object):
x = RevealAccess(10, 'var "x"')
y = 5
>>> MyClass.x
Retrieving var "x"
10
>>> MyClass().x
Retrieving var "x"
10
>>>
>>> m = MyClass()
>>> m.x
Retrieving var "x"
10
>>> m.x = 20
Updating var "x"
>>> m.x
Retrieving var "x"
20
>>> m.y
5

Python : How to "merge" two class

I want to add some attributes and methods into various class. The methods and attributes that I have to add are the same but not the class to assign them, so I want to construct a class who assign new methods and attributes for a class given in argument.
I try this but it's not working:
(I know that is a very wrong way to try to assign something to self, it's just to show what I want to do)
class A:
def __init__(self):
self.a = 'a'
def getattA(self):
return self.a
class B:
def __init__(self, parent) :
self = parent
# This is working :
print self.getattA()
def getattB(self):
return self.getattA()
insta = A()
instb = B(insta)
# This is not working :
print instb.getattB()
The result is :
a
Traceback (most recent call last):
File "D:\Documents and settings\Bureau\merge.py", line 22, in <module>
print instb.getattB()
File "D:\Documents and settings\Bureau\merge.py", line 16, in getattB
return self.getattA()
AttributeError: B instance has no attribute 'getattA'
And I expected to got 'a' for the call of instb.gettattB()
To resume I want to inherit class B from class A giving class A in argument of class B because my class B will be a subclass of various class, not always A.

The Best answer is in the comments, it was useful for me so I decided to show it in an answer (thank to sr2222):
The way to dynamicaly declare inherance in Python is the type() built-in function.
For my example :
class A(object) :
def __init__(self, args):
self.a = 'a'
self.args = args
def getattA(self):
return self.a, self.args
class B(object) :
b = 'b'
def __init__(self, args) :
self.b_init = args
def getattB(self):
return self.b
C = type('C', (A,B), dict(c='c'))
instc = C('args')
print 'attributes :', instc.a, instc.args, instc.b, instc.c
print 'methodes :', instc.getattA(), instc.getattB()
print instc.b_init
The code return :
attributes : a args b c
methodes : ('a', 'args') b
Traceback (most recent call last):
File "D:\Documents and settings\Bureau\merge2.py", line 24, in <module>
print instc.b_init
AttributeError: 'C' object has no attribute 'b_init'
My class C inerhite attributes and methods of class A and class B and we add c attribute. With the instanciation of C (instc = C('args')) The init for A is call but not for B.
Very useful for me because I have to add some attributes and methodes (the same) on different class.

I was having trouble with calling different constructors, using super doesn't necessarily make sense in a case like this, I opted to inherit and call each constructor on the current object manually:
class Foo(object):
def __init__(self, foonum):
super(Foo, self).__init__()
self.foonum = foonum
class Bar(object):
def __init__(self, barnum):
super(Bar, self).__init__()
self.barnum = barnum
class DiamondProblem(Foo, Bar):
# Arg order don't matter, since we call the `__init__`'s ourself.
def __init__(self, barnum, mynum, foonum):
Foo.__init__(self, foonum)
Bar.__init__(self, barnum)
self.mynum = mynum

How about this?
class A:
def __init__(self):
self.a = 'a'
def getatt(self):
return self.a
class B:
def __init__(self, parent) :
self.parent = parent
def __getattr__(self, attr):
return getattr(self.parent, attr)
def getattB(self):
return self.parent.getatt()
insta = A()
instb = B(insta)
print instb.getattB()
print instb.getatt()
But method in class A can not access attr in class B.
Another way:
import functools
class A:
def __init__(self):
self.a = 'a'
def getatt(self):
return self.a
class B:
def __init__(self, parent):
for attr, val in parent.__dict__.iteritems():
if attr.startswith("__"): continue
self.__dict__[attr] = val
for attr, val in parent.__class__.__dict__.iteritems():
if attr.startswith("__"): continue
if not callable(val): continue
self.__dict__[attr] = functools.partial(val, self)
def getattB(self):
return self.getatt()
insta = A()
instb = B(insta)
print instb.__dict__
print instb.getattB()
print instb.getatt()
Slow with init but call fast.

Since B is not a subclass of A, there is no path in B to getatt() in A

I guess i have a easier method
class fruit1:
def __init__(self):
self.name = "apple"
self.color = "blue"
class fruit2:
def __init__(self):
self.name = "banana"
self.size = 100
def merge(ob1, ob2):
ob1.__dict__.update(ob2.__dict__)
return ob1
f1 = fruit1()
f2 = fruit2()
fruit = merge(f1, f2)
print("name:",fruit.name," color:",fruit.color, " size:",fruit.size)
#output: name: banana color: blue size: 100

I'm not certain what you are trying to do, but the code below is giving my the output I think you are expecting. notice:
a is initialized outside the constructor in A
B is declared as a subclass of A
Code:
class A:
a='' #Initialize a
def __init__(self):
self.a = 'a'
def getatt(self):
return self.a
class B(A): #Declare B as subclass
def __init__(self, parent) :
self = parent
print self.getatt()
def getattB(self):
return self.getatt()
insta = A()
instb = B(insta)
print instb.getattB()

Helper function below conducts the merge of the dataclass instances, the attributes orders is derived from *args order:
from dataclasses import dataclass
#dataclass
class A:
foo: str
bar: str
def merge_dataclasses(*args):
if len({e.__class__.__name__ for e in args}) > 1:
raise NotImplementedError('Merge of non-homogeneous entries no allowed.')
data = {}
for entry in args[::-1]:
data.update(vars(entry))
return entry.__class__(**data)
print(merge_dataclasses(A(foo='f', bar='bar'), A(foo='b_foo', bar='b_bar')))

One easy way to merge two or more classes is through the tool set dyndesign:
from dyndesign import mergeclasses
class Base:
def __init__(self, init_value):
self.param = init_value
def m1(self):
print(f"Method `m1` of class `Base`, and {self.param=}")
def m2(self):
print(f"Method `m2` of class `Base`")
class Ext:
def m1(self):
print(f"Method `m1` of class `Ext`, and {self.param=}")
MergedClass = mergeclasses(Base, Ext)
merged_instance = MergedClass("INITIAL VALUE")
merged_instance.m1()
# Method `m1` of class `Ext`, and self.param='INITIAL VALUE'
merged_instance.m2()
# Method `m2` of class `Base`

Emphasizing ThorSummoner's's answer and Hong's comment; this method appears to be cleaner than the excepted answer. Notice Hong's use of super().init(self) in all but the last object added to the merge class.
class Foo(object):
def __init__(self, foonum):
super(Foo, self).__init__(self)
self.foonum = foonum
class Bar(object):
def __init__(self, barnum):
super(Bar, self).__init__(self)
self.barnum = barnum
class Oops(object):
def __init__(self, oopsnum):
super(Oops, self).__init__()
self.oopsnum = oopsnum
class DiamondProblem(Foo, Bar, Oops):
def __init__(self, mynum, foonum, barnum, oopsnum):
Foo.__init__(self, foonum)
Bar.__init__(self, barnum)
Oops.__init__(self, oopsnum)
self.mynum = mynum
def main():
dia = DiamondProblem(1, 10, 20, 30)
print(f"mynum: {dia.mynum}")
print(f"foonum: {dia.foonum}")
print(f"barnum: {dia.barnum}")
print(f"oopsnum: {dia.oopsnum}")