class A():
...
I want class A that is initialized with any object as parameter, but when __init__ goes with A type, it should leave/return old A object, dont create wrapper A(A()).
I guess that overwrite __new__ method is solution, but how there stop creating object creator and return given parameter as a result.
My workaround now is just recommendation:
def ToA(it):
if type(it) == A:
return it
else:
return A(it)
But I need block somehow A to prevent direct use of A(A()).
I am not at all sure I understand what you want, but something like this may be it. You can't have an __init__ with this technique, because if you do, it will be called regardless of whether __new__ returned a new object.
class A(object):
def __new__(cls, obj):
if isinstance(obj, cls):
return obj
rv = object.__new__(cls)
# everything you would normally put in __init__
# goes here instead
rv._obj = obj
return rv
# for instance:
def __repr__(self):
return "A({})".format(repr(self._obj))
>>> A()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __new__() takes exactly 2 arguments (1 given)
>>> A(1)
A(1)
>>> A(A(1))
A(1)
Not being able to use __init__ is troublesome, particularly if you need to subclass A. Here's a way to work around that:
class A(object):
def __new__(cls, obj):
if isinstance(obj, cls):
return obj
rv = object.__new__(cls)
rv._initialized = False
return rv
def __init__(self, obj):
if self._initialized: return
self._obj = obj
self._initialized = True
class B(A):
def __init__(self, obj):
if self._initialized: return
A.__init__(self, obj)
self._otherthing = "foo"
You have to check self._initialized in every subclass's __init__ method, unfortunately.
Related
I know that functions are just descriptors, like this:
def func(self):
print(self.name)
class C:
def __init__(self, name):
self.name = name
C.func = func
c = C("foo")
c.func()
I thought at first that c.func equals C.func.__get__(c)，yes，C.func.__get__(c) return a bound method. But when I set the __get__ of func to None, c.func still returns a bound method.
def func(self):
print(self.name)
class C:
def __init__(self, name):
self.name = name
func.__get__ = None
C.func = func
c = C("foo")
c.func
output:
<bound method func of <__main__.C object at 0x0000027EB23BF088>>
So I'm confused. Moreover, I found that when calling a function from an instance, Python actually calls the class's ___getAttribute__ method, which returns a bound method.
def func(self):
print(self.name)
func.__get__ = None
class C:
def __getattribute__(self, name):
r = super().__getattribute__(name)
print(r) # r is a bound method already
return r
def __init__(self, name):
self.name = name
C.func = func
c = C("foo")
c.func
output:
<bound method func of <__main__.C object at 0x0000027EB243D1C8>>
func.__get__ doesn't seem to have any effect. So, What happended in __getattribute__? How does Python turn a function into a method? I've Googled and done some research, but I still can't find the answer.
Maybe I'm making things complicated, In my understanding, function is itself a descriptor, but just like the code below, I set the func to None, it works normally:
class C:
def func(self):
print('hello world')
func.__get__ = None
c = C()
c.func()
but if it's a descriptor, it will raise TypeError:
class C:
class D:
def __get__(self, inst, cls):
if inst is None:
return self
return 'hello world'
D.__get__ = None
func = D()
c = C()
c.func
Well, if I understand correctly from what I found. (Since I didn't know the descriptors, that's exactly why I like to help, still learning)
First, let's look at __getattr__ and __getattribute__.
Let's have an empty class A
class A:
pass
If I initialize an object and try to call a property, because there is none at the moment, we get AttributeError.
a = A()
a.some_property
The following occurs:
Simple check of flow:
class FlowDemo:
def __init__(self):
self.inited_property = True
def __getattribute__(self, item):
if item in ('__class__', '__len__') : # For less spam of getting this attribute, if you want, you can remove condition.
print('Get Attribute', item)
# Call default behavior
return super().__getattribute__(item)
def __getattr__(self, item):
print('Get Attr', item)
if item == 'some_magic_name':
return "It's magic!"
raise AttributeError
fd = FlowDemo()
fd.inited_property
# Get Attribute inited_property
# True
fd.some_magic_property
# Get Attribute some_magic_name
# Get Attr some_magic_name
# "It's magic!"
fd.some_property
# Get Attribute some_property
# Get Attr some_property
# Traceback (most recent call last):
# File "<input>", line 1, in <module>
# File "stack-class-property-and-descriptors.py", line 67, in # __getattr__
# raise AttributeError
# AttributeError
This is probably understandable, including the use. But to be sure, I'll give an example. This logic is used as a dynamic representation of the result from the databases (mapping of attributes to ordinary dict, list, etc.).
But it can also be just logic for accessing an attribute (property), such as an access counter or validation (but this applies to __setattr__ and __setattribute__)
And what about descriptors?
First let's look at data-descriptors, they are easier for me to understand.
This is a class or decoder that has __get__ and one or both of __set__ and __delete__.
Once this is defined, python, when used in the property definition with it and then does not return a class but the value it obtains through __get__, does not overwrite an already declared class when declaring a value, but uses its __set__.
Example:
class WeekDayDescriptor:
def __init__(self):
self.__week_day = 0
def __get__(self, instance, owner=None):
return self.__week_day
def __set__(self, instance, value):
if not isinstance(value, int):
raise TypeError('Value must be int')
if not (0 <= value < 6):
raise ValueError('Value must be in range 0 - 6')
self.__week_day = value
class Calendar:
week_day = WeekDayDescriptor()
def __init__(self, week_day):
self.week_day = week_day
Demo:
c = Calendar(9)
# ValueError: Value must be in range 0-6
c = Calendar('6')
# TypeError: Value must be int
c = Calendar(3)
c.week_day = 6
c.week_day = 10
# ValueError: Value must be in range 0-6
c.week_day = 'monday'
# TypeError: Value must be int
Decorator descriptor:
class Calendar:
#property
def week_day(self):
return self.__week_day
#week_day.setter
def week_day(self, week_day):
if not isinstance(week_day, int):
raise TypeError('Value must be int')
if not (0 <= week_day < 6):
raise ValueError('Value must be in range 0 - 6')
self.__week_day = week_day
def __init__(self, week_day):
self.week_day = week_day
pass
And now for non-data descriptors...
A non-data descriptor is one that has only __get__.
As I understand it, each method automatically has its own descriptor, thanks to which the functions get references to the object - self.
We can write our own descriptor for a function / method, but it's not that straightforward, we have to help ourselves and get around it a bit.
def function_as_method(self, value):
print(self, value)
class HelperDescriptor:
def __get__(self, instance, owner):
def wrapper(*args, **kwargs):
return function_as_method(instance, *args, **kwargs)
return wrapper
class Foo:
baz = HelperDescriptor()
>>> bar = Foo()
>>> bar.baz(1)
<__main__.Foo object at 0x7f64f7768b70> 1
Source of last code block, but in czech lang.
And finally, your mentioned problem, when we set __get__ to None and you still get a reference to the function.
It's simple, python doesn't directly distinguish between primitive data types and functions, it's all a variable (or attribute / property) that has a value. Whether it's just value or it's callable is a different matter.
def f(): return True
print(type(f), f())
# <class 'function'> True
f = 123
print(type(f), f)
# <class 'int'> 123
Therefore, when we ask for the obj.func method or call it obj.func() directly, the first two changed magic is called first - __getattribute__ and __getattr__.
And if we call a method, it is called only after we get a reference to a function in memory.
Again a simple example:
def func(self, value):
print('Printing:', value)
class PrintDescriptor:
def __init__(self, name):
self.name = name
def __get__(self, instance, owner):
def wrapper(*args, **kwargs):
print(f"Calling '{self.name}' method")
return func(instance, *args, **kwargs)
return wrapper
class B:
foo = PrintDescriptor('foo')
bar = PrintDescriptor('bar')
def __getattribute__(self, item):
if item not in ('__len__', '__class__', '__dict__'):
print('Get Attribute', item)
return super().__getattribute__(item)
Demo:
b = B()
b.foo
# Get Attribute foo
# <function PrintDescriptor.__get__.<locals>.wrapper at 0x7f774a782ee0>
b.foo(2)
# Get Attribute foo
# Calling 'foo' method
# Printing: 2
b.bar(4)
# Get Attribute bar
# Calling 'bar' method
# Printing: 4
Sources:
https://www.datacamp.com/community/tutorials/python-descriptors#above1
https://blog.milde.cz/post/319-pokrocile-techniky-v-pythonu-deskriptory/
Python Doc, __get__
Python Docs, __getattribute__
Python Docs, __getattr__
I would like to allow my class constructor to accept an instance of this class and in that case to return this same instance instead of creating a new object. Like what tuple does:
>>> t = (1, 2, 3)
>>> tuple(t) is t
True
I imagine I need to override the __new__ method for this, and additionally take care of this special case in the __init__ method. Are there any recipes for this?
I would have preferred to completely skip __init__ when the constructor is given a class instance that it is going to return unchanged, but I see no way. I am also a bit suspicious about the triple use of cls in the line with super:
class C:
#staticmethod
def __new__(cls, x=None):
if isinstance(x, cls):
return x
else:
return super(cls, cls).__new__(cls)
def __init__(self, x=None):
if x is self: return # Can I just skip __init__ instead?
self.x = x
(I know about super() without arguments, but I do not like inconsistent magic.)
After learning more about super and about MRO in Python, i've figure out that this code is not good. For example, subclassing C results in
>>> class D(C): pass
>>> d = D(1)
......
RecursionError: maximum recursion depth exceeded while calling a Python object
I am so terrified by super() without arguments that magically extracts its arguments from the (lexical or dynamic?) context, that rather than using it I've decided to add a "callback" to finalise the class definition:
class C2:
#classmethod
def finalise(this_cls_yes_this_one):
#staticmethod
def __new__(cls, x=None):
if isinstance(x, cls):
return x
else:
return super(this_cls_yes_this_one, cls).__new__(cls)
this_cls_yes_this_one.__new__ = __new__
del this_cls_yes_this_one.finalise
def __init__(self, x=None):
if x is self: return
self.x = x
C2.finalise()
You can use a metaclass. The __init__ and __new__ methods are called in __call__ method of the metacalss whenever an instance is call. You can handle both by overriding the metaclasses __call__ function.
class MyMetaClass(type):
def __call__(cls, obj, *args, **kwargs):
if (isinstance(obj, cls)):
return obj
else:
self = cls.__new__(cls, obj, *args, **kwargs)
cls.__init__(self, obj, *args, **kwargs)
return self
# Note: In order to be sure that you don't miss anything
# It's better to do super().__call__(obj, *args, **kwargs) here
class A(metaclass=MyMetaClass):
def __init__(self, obj, *args, **kwargs):
self.val = obj
Demo:
a = A(10)
b = A(a)
c = A(40)
print(b is a)
print(a.val)
print(b.val)
print(c is a)
print(c.val)
# out:
True
10
10
False
40
I am trying to implement a metaclass that initializes class variables when a first its instance is being created. I want to keep a new magic method __load__ that should be called as a classmethod (like __new__). So I implemented it like this:
class StaticLoad(type):
__loaded_classes = set()
def __call__(cls, *args, **kwargs):
if cls not in cls.__loaded_classes:
if hasattr(cls, '__load__'):
cls.__load__()
cls.__loaded_classes.add(cls)
return super().__call__(*args, **kwargs)
class BaseClass(metaclass=StaticLoad):
s = 0
class MyClass(BaseClass):
#classmethod
def __load__(cls):
print("Loading", cls.__name__, "...")
cls.s += 1
obj1 = MyClass()
obj2 = MyClass()
print(MyClass.s)
It works fine and gives the correct result:
Loading MyClass ...
1
Now I want to implement the method __load__ as a classmethod by default like __new__ (without the need to type #classmethod above each time). I tried this:
class StaticLoad(type):
__loaded_classes = set()
def __call__(cls, *args, **kwargs):
if cls not in cls.__loaded_classes:
if hasattr(cls, '__load__'):
# I try to apply classmethod routine to make
# cls.__load__ a classmethod
classmethod(cls.__load__)()
cls.__loaded_classes.add(cls)
return super().__call__(*args, **kwargs)
class BaseClass(metaclass=StaticLoad):
s = 0
class MyClass(BaseClass):
# #classmethod line was deleted
def __load__(cls):
print("Loading", cls.__name__, "...")
cls.s += 1
obj1 = MyClass()
obj2 = MyClass()
print(MyClass.s)
I got the error:
Traceback (most recent call last):
File "example.py", line 22, in <module>
obj1 = MyClass()
File "example.py", line 7, in __call__
classmethod(cls.__load__)()
TypeError: 'classmethod' object is not callable
It looks like classmethod routine is correctly available only inside a class definition.
How should I improve my metaclass to make it work fine? I would like to keep the content of classes BaseClass and MyClass as I wrote above, placing all magic into StaticLoad.
With the help of #AnttiHaapala the solution is simple. Instead of calling
classmethod(cls.__load__)()
I had to call
cls.__load__(cls)
If you want to perform transforms on the certain methods and attributes of a class creation, you do that on the metaclass' __new__ function.
Since yu already have a metaclass, all you have to do is to implement its __new__ method to convert any __load__ methods in a classmethod:
class StaticLoad(type):
__loaded_classes = set()
def __new__(metacls, name, bases, namespace):
if "__load__" in namespace and not isinstance(namespace["__load__"], classmethod):
namespace["__load__"] = classmethod(namespace["load"])
return super().__new__(metacls, name, bases, namespace)
def __call__(cls, *args, **kwargs):
if cls not in cls.__class__.__loaded_classes:
if hasattr(cls, '__load__'):
cls.__load__()
type(cls).__loaded_classes.add(cls)
return super().__call__(*args, **kwargs)
(the other change I made was to make explict that "__loaded_classes" should be accessed on the metaclass, not on the class itself).
There is an answered question about classmethod and property combined together: Using property() on classmethods
I still don't understand the cause of the problem, please help.
My understanding of classmethod was that it simply replaces self with cls. With this in mind I wrote several classmethods during the past few years and now I see I was wrong all that time.
So what is the difference between #classmethod and #cm from the code below?
def cm(func):
def decorated(self, *args, **kwargs):
return func(self.__class__, *args, **kwargs)
return decorated
class C:
V = 0
#property
#classmethod
def inc1(cls):
cls.V += 1
print("V1 =", cls.V)
#property
#cm
def inc3(cls):
cls.V += 3
print("V3 =", cls.V)
c = C()
#c.inc1 # fails with: TypeError: 'classmethod' object is not callable
c.inc3 # works
inc3 with cm works, but inc1 with classmethod does not.
what is the difference between #classmethod and #cm from the code below?
decorator is calling during class creation time before an instance is created.
In your case, since #cm returns func(self.__class__, *args, **kwargs), which is relied on self, it should be used as a instance method.
On the other hand, #classmethod is able to use before an instance is created.
def cm(func):
def decorated(self, *args, **kwargs):
return func(self.__class__, *args, **kwargs)
return decorated
class C:
#classmethod
def inc1(cls):
(blablabla)
#cm
def inc3(cls):
(blablabla)
C().inc1() # works as a instance method
C.inc1() # works as a classmethod
C().inc3() # works as a instance method
C.inc3() # TypeError: unbound method decorated() must be called with C instance as first argument (got nothing instead)
For a combination of classmethod and property, it could be done by return an customized object. Reference
class ClassPropertyDescriptor(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.f.__get__(obj, klass)()
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class C:
#classproperty
def inc1(cls):
(blablabla)
C.inc1 # works as a classmethod property
[Edit]
Q. What does the classmethod() call do with the method it decorates to achieve that?
The implementation can be done by using descriptor
class ClassMethodDescriptor(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
def myclassmethod(func):
return ClassMethodDescriptor(func)
class C:
#myclassmethod
def inc1(cls):
(blablabla)
C.inc1() # works as a classmethod
Q. Why is the result not callable?
Because the implementation of ClassMethodDescriptor does not define __call__ function. Once using #property, it will return ClassMethodDescriptor which is not callable.
The difference is that classmethod is not callable, and cm method is callable. This means that when the property(class) makes a call to the inputed func(which it is supposed to do), it works as you'll except for cm, but will not work for classmethod since classmethod does not have a call implemented.
class method does not know anything about instance and does not require it.
instance method knows about it's instance and it's class.
class Foo:
some = 'some'
class Bar(Foo):
def __init__(self):
self.some = 'not some'
#classmethod
def cls_some(cls):
print(cls.some)
def instance_some(self):
print(self.some)
Bar.cls_some()
>>>some
Bar().instance_some()
>>>not some
Also as you can see you don't need an instance to call classmethod.
I'm having a hard time understanding what happens when I try to nest descriptors/decorators. I'm using python 2.7.
For example, let's take the following simplified versions of property and classmethod:
class MyProperty(object):
def __init__(self, fget):
self.fget = fget
def __get__(self, obj, objtype=None):
print 'IN MyProperty.__get__'
return self.fget(obj)
class MyClassMethod(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, objtype=None):
print 'IN MyClassMethod.__get__'
def f(*args, **kwargs):
return self.f(objtype, *args, **kwargs)
return f
Trying to nest them:
class A(object):
# doesn't work:
#MyProperty
#MyClassMethod
def klsproperty(cls):
return 555
# works:
#MyProperty
def prop(self):
return 111
# works:
#MyClassMethod
def klsmethod(cls, x):
return x**2
% print A.klsproperty
IN MyProperty.__get__
...
TypeError: 'MyClassMethod' object is not callable
The __get__ method of the inner descriptor MyClassMethod is not getting called.
Failing to figure out why, I tried throwing in (what I think is) a no-op descriptor:
class NoopDescriptor(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, objtype=None):
print 'IN NoopDescriptor.__get__'
return self.f.__get__(obj, objtype=objtype)
Trying to use the no-op descriptor/decorator in nesting:
class B(object):
# works:
#NoopDescriptor
#MyProperty
def prop1(self):
return 888
# doesn't work:
#MyProperty
#NoopDescriptor
def prop2(self):
return 999
% print B().prop1
IN NoopDescriptor.__get__
IN MyProperty.__get__
888
% print B().prop2
IN MyProperty.__get__
...
TypeError: 'NoopDescriptor' object is not callable
I don't understand why B().prop1 works and B().prop2 does not.
Questions:
What am I doing wrong? Why am I getting a object is not callable error?
What's the right way? e.g. what is the best way to define MyClassProperty while re-using MyClassMethod and MyProperty (or classmethod and property)
In this case, when the decorators are used without parameters, a decorator is called with the function it decorates as its parameter. The decorator's return value is used instead of the decorated function. So:
#MyProperty
def prop(self):
...
is equivalent to:
def prop(self):
...
prop = MyProperty(prop)
Since MyProperty implements the descriptor protocol, accessing A.prop will actually call A.prop.__get__(), and you've defined __get__ to call the object which was decorated (in this case, the original function/method), so everything works fine.
Now, in the nested case:
#MyProperty
#MyClassMethod
def prop(self):
...
The equivalent is:
def prop(self):
...
prop = MyClassMethod(prop) # prop is now instance of MyClassMethod
prop = MyProperty(prop) # prop is now instance of MyProperty
# (with fget == MyClassMethod instance)
Now, as before, accessing A.prop will actually call A.prop.__get__() (in MyProperty) which then tries to call the instance of MyClassMethod (the object which was decorated and stored in the fget attribute).
But the MyClassMethod does not have a __call__ method defined, so you get the error MyClassMethod is not callable.
And to address your second question: A property is already a class attribute - in your example, accessing A.prop will return the value of the property in the class object and A().prop will return the value of the property in an instance object (which can be the same as the class object if the instance did not override it).
You can make your code work if you make MyProperty apply the descriptor protocol to its wrapped object:
class MyProperty(object):
def __init__(self, fget):
self.fget = fget
def __get__(self, obj, objtype=None):
print('IN MyProperty.__get__')
try:
return self.fget.__get__(obj, objtype)()
except AttributeError: # self.fget has no __get__ method
return self.fget(obj)
Now your example code works:
class A(object):
#MyProperty
#MyClassMethod
def klsproperty(cls):
return 555
print(A.klsproperty)
The output is:
IN MyProperty.__get__
IN MyClassMethod.__get__
555
I found the definitive answer to my own old question in Graham Dumpleton's fascinating blog.
In short, the decorators I wrote do not honour the descriptors protocol, by trying to call the wrapped function/object directly, instead of first giving them a chance to perform their "descriptor magic" (by calling their __get__() first).