I have this code:
from typing import Callable, Any
class Test(classmethod):
def __init__(self, f: Callable[..., Any]):
super().__init__(f)
def __get__(self,*args,**kwargs):
print(args) # why out put is (None, <class '__main__.A'>) where form none why no parameter 123
# where was it called
return super().__get__(*args,**kwargs)
class A:
#Test
def b(cls,v_b):
print(cls,v_b)
A.b(123)
Why the output is (None, <class '__main__.A'>)? Where did None come form and why is it not the parameter 123, which is the value I called it with?
The __get__ method is called when the method b is retrieved from the A class. It has nothing to do with the actual calling of b.
To illustrate this, separate the access to b from the actual call of b:
print("Getting a reference to method A.b")
method = A.b
print("I have a reference to the method now. Let's call it.")
method()
This results in this output:
Getting a reference to method A.b
(None, <class '__main__.A'>)
I have a reference to the method now. Let's call it.
<class '__main__.A'> 123
So you see, it is normal that the output in __get__ does not show anything about the argument you call b with, because you haven't made the call yet.
The output None, <class '__main__.A'> is in line with the Python documentation on __get__:
object.__get__(self, instance, owner=None)
Called to get the attribute of the owner class (class attribute access) or of an instance of that class (instance attribute access). The optional owner argument is the owner class, while instance is the instance that the attribute was accessed through, or None when the attribute is accessed through the owner.
In your case you are using it for accessing an attribute (b) of a class (A) -- not of an instance of A -- so that explains the instance argument is None and the owner argument is your class A.
The second output, made with print(cls,v_b), will print <class '__main__.A'> for cls, because that is what happens when you call class methods (as opposed to instance methods). Again, from the documentation:
When a class attribute reference (for class C, say) would yield a class method object, it is transformed into an instance method object whose __self__ attribute is C.
Your case is described here, where A is the class, and so the first parameter (which you called cls) will get as value A.
You can apply multiple decorators on the same function, for example,
first (and outer) decorator could be a classmethod
and the second (doing your stuff) could define a wrapper, where you could accept your arguments as usual
In [4]: def test_deco(func):
...: def wrapper(cls, *args, **kwds):
...: print("cls is", cls)
...: print("That's where 123 should appear>>>", args, kwds)
...: return func(cls, *args, **kwds)
...:
...: return wrapper
...:
...:
...: class A:
...: #classmethod
...: #test_deco
...: def b(cls, v_b):
...: print("That's where 123 will appear as well>>>", v_b)
...:
...:
...: A.b(123)
cls is <class '__main__.A'>
That's where 123 should appear>>> (123,) {}
That's where 123 will appear as well>>> 123
In [5]:
It's too much trouble to use two at a time i want only use one like it
It is possible to define a decorator applying a couple of other decorators:
def my_super_decorator_doing_everything_at_once(func):
return classmethod(my_small_decorator_doing_almost_everything(func))
That works because decorators notation
#g
#f
def x(): ...
is a readable way to say
def x(): ...
x = g(f(x))
Related
I encoutered a weird case of class inheritance in Python 3.8.
I can create a class by inheriting from an instance which is not itself a type. I expected a TypeError in that case, but that is not what happens.
class B:
def __new__(cls, *args, **kwargs):
print(args, kwargs)
return super().__new__(cls)
b = B() # prints: () {}
class A(b): # I expected a TypeError here
x = "foo"
# prints: ('A', (<__main__.B object at 0x7f32bd102390>,),
# {'__module__': '__main__', '__qualname__': 'A', 'x': 'foo'}) {}
print(A) # <__main__.B object at 0x7f20556c93d0>
print(isinstance(A, B)) # True
print(isinstance(A, type)) # False
It looks like when I give B() as base for A, then B is being used as the metaclass.
Is my understanding correct? Why is that the case? And I cannot seem to find anything regarding this behaviour, is it documented or is it an implementation quirk specific to cPython?
Basically, the type constructor will check the bases to guess the most derived metaclass. It will then see the instance of A, and use its class as the "most derived metaclass". So, yes, the class of things that go in "bases" is used as the metaclass.
From that point, it will call the retrieved "metaclass" to build the B class itself. As it is a class, it is called, this triggers the class' __new__ as usual. Since it returns an instance of "A" that is what you get as being your "B": Python does not get in the way to verify if whatever the metaclass callable returned is itself a "class" - it can be any object. This gives one the flexibility to abuse the class statement to build other kind of objects, given appropriate super-classes or metaclasses. In this case, it just happened to not raise a TypeError and return you an instance of "A". (but you had to adjust the class' __new__ to avoid the TypeError)
In [16]: class A:
...: def __new__(cls, *args, **kw):
...: print(args, kw)
...: return super().__new__(cls)
...:
...:
In [17]: class B(A()): pass
() {}
('B', (<__main__.A object at 0x7f26862395b0>,), {'__module__': '__main__', '__qualname__': 'B'}) {}
In [18]: type(B)
Out[18]: __main__.A
In [19]: isinstance(B, A)
Out[19]: True
The Python docs, specify that attribute access is done via the objects __dict__ attribute.
The default behavior for attribute access is to get, set, or delete the attribute from an object’s dictionary. For instance, a.x has a lookup chain starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing through the base classes of type(a) excluding metaclasses. If the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead. Where this occurs in the precedence chain depends on which descriptor methods were defined.
In this example, I override the object's __dict__, but python can still manage to get its attribute, how does that work?:
In [1]: class Foo:
...: def __init__(self):
...: self.a = 5
...: self.b = 3
...: def __dict__(self):
...: return 'asdasd'
...:
In [2]: obj_foo = Foo()
In [3]: obj_foo.__dict__
Out[3]: <bound method Foo.__dict__ of <__main__.Foo object at 0x111056850>>
In [4]: obj_foo.__dict__()
Out[4]: 'asdasd'
In [5]: obj_foo.b
Out[5]: 3
You are implementing the __dict__ method.
If you override the __dict__ attribute you will succeed in breaking Python:
class Foo:
def __init__(self):
self.a = 1
self.__dict__ = {}
Foo().a
AttributeError: 'Foo' object has no attribute 'a'
Interestingly enough, if both are overridden everything is back to normal:
class Foo:
def __init__(self):
self.a = 1
self.__dict__ = {}
def __dict__(self):
pass
print(Foo().a)
Outputs
1
Making __dict__ a property also does not seem to break anything:
class Foo:
def __init__(self):
self.a = 1
#property
def __dict__(self):
return {}
print(Foo().a)
Also outputs 1.
I suspect the answer to the question *why* lies (probably) deep in the CPython implementation of `object`, more digging needs to be done.
##### Case 1, use property #####
class Case1:
# ignore getter and setter for property
var = property(getter, setter, None, None)
##### Case 2, use equivalent methods ####
class Descriptor:
def __get__(self, obj, type=None):
return None
def __set__(self, obj, val):
pass
class Case2:
var = Descriptor()
My question is:
When I use 'property' to control the access of one variable,
instance.var will correctly return the real value,
while Class.var will return the property object itself (e.g. property object at 0x7fc5215ac788)
But when I use equivalent methods (e.g. descriptor) and override __get__ and __set__ methods,
both instance.var and Class.var can return the real value instead of the object itself.
So why they behave so differently?
I guess it is because some of default functions implemented in the my descriptor make the magic, so what are they?
update:
The reason for the above question is that __get__ function implemented in the property will determine if it is called by instance or Class, and when it is called by Class, it will return the object itself (i.e. self).
But as __set__ function does not have type or cls parameter, and based on my test, Class.var = 5 cannot be caught by __set__ function.
Therefore, I wonder what hooks we can use to customize the class variable level assignment Class.var = value?
When you do MyClass.some_descriptor, there's (obviously) no instance to be passed to the descriptor, so it is invoked with obj=None:
>>> class Desc(object):
... def __get__(self, obj, cls=None):
... print "obj : {} - cls : {}".format(obj, cls)
... return 42
...
>>> class Foo(object):
... bar = Desc()
...
>>> Foo.bar
obj : None - cls : <class '__main__.Foo'>
42
>>> Foo().bar
obj : <__main__.Foo object at 0x7fd285cf4a90> - cls : <class '__main__.Foo'>
42
>>>
In most cases (and specially with the generic property descriptor) the goal is to compute the return value based on instance attributes so there's not much you can return without the instance. In this case, most authors choose to return the descriptor instance itself so it can be correctly identified for what it is when inspecting the class.
If you want this behaviour (which makes sense for most descriptors), you just have to test obj against None and return self:
>>> class Desc2(object):
... def __get__(self, obj, cls=None):
... if obj is None:
... return self
... return 42
...
>>> Foo.baaz = Desc2()
>>> Foo.baaz
<__main__.Desc2 object at 0x7fd285cf4b10>
>>> Foo().baaz
42
>>>
And that's all the "magic" involved .
Now if you wonder why this is not the default: there are use cases for returning something else for a descriptor looked up on a class - methods for example (yes, Python functions are descriptors - their __get__ method returns a method object, which is actually a callable wrapper around the instance (if any), class and function):
>>> Foo.meth = lambda self: 42
>>> Foo.meth
<unbound method Foo.<lambda>>
>>> Foo().meth
<bound method Foo.<lambda> of <__main__.Foo object at 0x7fd285cf4bd0>>
>>> Foo.meth(Foo())
42
A colleague of mine wrote code analogous to the following today, asked me to have a look, and it took me a while to spot the mistake:
class A():
def __init__(self):
print('A')
class B(A):
def __init__(self):
super(B).__init__()
b = B()
The problem here is that there's no self parameter to super() in B's constructor. What surprised me is that absolutely nothing happens in this case, i.e. no error, nothing. What does the super object created by super(B) contain? As an object, it clearly has a constructor, so that's what gets called, but how is that object related to B? In particular, why is this valid code and doesn't throw an exception somewhere? Is super(B) an object with some actual use and what would that be?
The only thing that causes all these ambiguities is that "why obj = super(B).__init__() works?". That's because super(B).__self_class__ returns None and in that case you're calling the None objects' __init__ like following which returns None:
In [40]: None.__init__()
Regarding the rest of the cases, you can simply check the difference by calling the super's essential attributes in both cases:
In [36]: class B(A):
def __init__(self):
obj = super(B, self)
print(obj.__class__)
print(obj.__thisclass__)
print(obj.__self_class__)
print(obj.__self__)
....:
In [37]: b = B()
<class 'super'>
<class '__main__.B'>
<class '__main__.B'>
<__main__.B object at 0x7f15a813f940>
In [38]:
In [38]: class B(A):
def __init__(self):
obj = super(B)
print(obj.__class__)
print(obj.__thisclass__)
print(obj.__self_class__)
print(obj.__self__)
....:
In [39]: b = B()
<class 'super'>
<class '__main__.B'>
None
None
For the rest of the things I recommend you to read the documentation thoroughly. https://docs.python.org/3/library/functions.html#super and this article by Raymond Hettinger https://rhettinger.wordpress.com/2011/05/26/super-considered-super/.
Moreover, If you want to know why super(B) doesn't work outside of the class and generally why calling the super() without any argument works inside a class you can read This comprehensive answer by Martijn https://stackoverflow.com/a/19609168/2867928.
A short description of the solution:
As mentioned in the comments by #Nathan Vērzemnieks you need to call the initializer once to get the super() object work. The reason is laid behind the magic of new super object that is explained in aforementioned links.
In [1]: class A:
...: def __init__(self):
...: print("finally!")
...:
In [2]: class B(A):
...: def __init__(self):
...: sup = super(B)
...: print("Before: {}".format(sup))
...: sup.__init__()
...: print("After: {}".format(sup))
...: sup.__init__()
...:
In [3]: B()
Before: <super: <class 'B'>, NULL>
After: <super: <class 'B'>, <B object>>
finally!
The confusion here comes from the fact that (in a class definition context) super() gives a bound super object which then delegates __init__ to its __self_class__, while super(B) creates an unbound super object which, because its __self_class__ is None, does not delegate.
In [41]: class Test(int):
...: def __init__(self):
...: print(super().__self_class__)
...: print(super().__init__)
...: print(super(Test).__self_class__)
...: print(super(Test).__init__)
...:
In [42]: Test()
<class '__main__.Test'>
<method-wrapper '__init__' of Test object at 0x10835c9c8>
None
<method-wrapper '__init__' of super object at 0x10835c3c8>
So when you call super(B).__init__(), it creates an unbound super but then immediately calls __init__ on it; that, because of the magic described in the various links in this other answer, binds that unbound super. There are no references to it, so it disappears, but that's what's happening under the hood.
I currently have this code:
class Generator(object):
def __getattr__(self, name):
def f(self):
return ("Result of"+name, self)
f.__name__ = name
return f
def foo(self):
pass
g = Generator()
print g.foo
print Generator.foo
print g.bar
print Generator.bar
Which gives:
<bound method Generator.foo of <__main__.Generator object at 0x00B62D30>>
<unbound method Generator.foo>
<function bar at 0x00A9DE70>
AttributeError: type object 'Generator' has no attribute 'bar'
What do I have to do to make it give:
<bound method Generator.foo of <__main__.Generator object at 0x00B62D30>>
<unbound method Generator.foo>
<bound method Generator.bar of <__main__.Generator object at 0x00B62D30>>
<unbound method Generator.bar>
Here's a metaclass that adds the __getattr__ function from the class definition back to the metaclass itself. This avoids having to define the function in multiple places, or as a separate global function defined beforehand and added individually to the metaclass and class.
class Meta(type):
def __new__(mcls, name, bases, dikt):
fgetattr = dikt.get('__getattr__')
if fgetattr is not None:
setattr(mcls, '__getattr__', fgetattr)
return super(Meta, mcls).__new__(mcls, name, bases, dikt)
class Generator(object):
__metaclass__ = Meta
def __getattr__(obj, name):
def f(self):
return "Result of %s for %r" % (name, self)
f.__name__ = name
if isinstance(obj, type):
setattr(obj, name, f)
else:
setattr(type(obj), name, f)
return getattr(obj, name)
Rather than directly create the method via the dynamic function's __get__ descriptor method, I think it's better to store the function in the class dict and rely on getattr to return the proper bound/unbound method. Subsequent attribute access will use the function from the class. Since the same __getattr__ function is used for both the class and the instance, an isinstance check is required to ensure the dynamic function gets stored to the class and not the instance.
In Python 3, getting the function as an attribute of the class merely returns the function since unbound methods were removed from the language. Also, the metaclass syntax has changed to a keyword argument in the class definition line.
Test:
>>> g = Generator()
>>> g.foo
<bound method Generator.foo of <__main__.Generator object at 0xb7248c2c>>
>>> Generator.foo
<unbound method Generator.foo>
>>> g.bar
<bound method Generator.bar of <__main__.Generator object at 0xb7248c2c>>
>>> Generator.bar
<unbound method Generator.bar>
>>> g.foo()
'Result of foo for <__main__.Generator object at 0xb7248c2c>'
>>> Generator.foo(g)
'Result of foo for <__main__.Generator object at 0xb7248c2c>'
>>> 'foo' in vars(Generator), 'bar' in vars(Generator)
(True, True)
__getattr__() only works on instances, you will need to make the function on a metaclass on Generator to get the behavior your want.
You have two problems.
You only set the dynamic function on the instance g (not the class Generator), so Generator.bar is not defined.
You don't wrap it in a MethodType, so you get a function instead of a method.
For 1, if you always call g.foo before you call Generator.foo, you can just add the line
setattr(self.__class__, name, f)
inside __getattr__, which will bind the name as a method on the class. Otherwise, you will need a custom __getattr__ on the type object, which means that you have to make it an instance of a custom class i.e. write your own metaclass.
For 2, see #thg435's answer. Note that this is icky because of backwards compatibility in Python 2, and has been neatened considerably in Py3k -- now what you are doing would basically work. It's because of the automatic injection of self.
For the first case (g.bar), replace return f with return MethodType(f, self).