Develop Reference

What are the differences between a `classmethod` and a metaclass method?

In Python, I can create a class method using the #classmethod decorator:
>>> class C:
... #classmethod
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> C.f()
f called with cls=<class '__main__.C'>
Alternatively, I can use a normal (instance) method on a metaclass:
>>> class M(type):
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> class C(metaclass=M):
... pass
...
>>> C.f()
f called with cls=<class '__main__.C'>
As shown by the output of C.f(), these two approaches provide similar functionality.
What are the differences between using #classmethod and using a normal method on a metaclass?

As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the abc.ABCMeta.register method.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict).
Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
#classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in #classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a #property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.

When you phrase it like you did in the question, the #classmethod and metaclasses may look similar but they have rather different purposes. The class that is injected in the #classmethod's argument is usually used for constructing an instance (i.e. an alternative constructor). On the other hand, the metaclasses are usually used to modify the class itself (e.g. like what Django does with its models DSL).
That is not to say that you can't modify the class inside a classmethod. But then the question becomes why didn't you define the class in the way you want to modify it in the first place? If not, it might suggest a refactor to use multiple classes.
Let's expand the first example a bit.
class C:
#classmethod
def f(cls):
print(f'f called with cls={cls}')
Borrowing from the Python docs, the above will expand to something like the following:
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
class C:
def f(cls):
print(f'f called with cls={cls}')
f = ClassMethod(f)
Note how __get__ can take either an instance or the class (or both), and thus you can do both C.f and C().f. This is unlike the metaclass example you give which will throw an AttributeError for C().f.
Moreover, in the metaclass example, f does not exist in C.__dict__. When looking up the attribute f with C.f, the interpreter looks at C.__dict__ and then after failing to find, looks at type(C).__dict__ (which is M.__dict__). This may matter if you want the flexibility to override f in C, although I doubt this will ever be of practical use.

In your example, the difference would be in some other classes that will have M set as their metaclass.
class M(type):
def f(cls):
pass
class C(metaclass=M):
pass
class C2(metaclass=M):
pass
C.f()
C2.f()
class M(type):
pass
class C(metaclass=M):
#classmethod
def f(cls):
pass
class C2(metaclass=M):
pass
C.f()
# C2 does not have 'f'
Here is more on metaclasses
What are some (concrete) use-cases for metaclasses?

Both #classmethod and Metaclass are different.
Everything in python is an object. Every thing means every thing.
What is Metaclass ?
As said every thing is an object. Classes are also objects in fact classes are instances of other mysterious objects formally called as meta-classes. Default metaclass in python is "type" if not specified
By default all classes defined are instances of type.
Classes are instances of Meta-Classes
Few important points are to understand metioned behaviour
As classes are instances of meta classes.
Like every instantiated object, like objects(instances) get their attributes from class. Class will get it's attributes from Meta-Class
Consider Following Code
class Meta(type):
def foo(self):
print(f'foo is called self={self}')
print('{} is instance of {}: {}'.format(self, Meta, isinstance(self, Meta)))
class C(metaclass=Meta):
pass
C.foo()
Where,
class C is instance of class Meta
"class C" is class object which is instance of "class Meta"
Like any other object(instance) "class C" has access it's attributes/methods defined in it's class "class Meta"
So, decoding "C.foo()" . "C" is instance of "Meta" and "foo" is method calling through instance of "Meta" which is "C".
First argument of method "foo" is reference to instance not class unlike "classmethod"
We can verify as if "class C" is instance of "Class Meta
isinstance(C, Meta)
What is classmethod?
Python methods are said to be bound. As python imposes the restriction that method has to be invoked with instance only.
Sometimes we might want to invoke methods directly through class without any instance (much like static members in java) with out having to create any instance.By default instance is required to call method. As a workaround python provides built-in function classmethod to bind given method to class instead of instance.
As class methods are bound to class. It takes at least one argument which is reference to class itself instead of instance (self)
if built-in function/decorator classmethod is used. First argument
will be reference to class instead of instance
class ClassMethodDemo:
#classmethod
def foo(cls):
print(f'cls is ClassMethodDemo: {cls is ClassMethodDemo}')
As we have used "classmethod" we call method "foo" without creating any instance as follows
ClassMethodDemo.foo()
Above method call will return True. Since first argument cls is indeed reference to "ClassMethodDemo"
Summary:
Classmethod's receive first argument which is "a reference to class(traditionally referred as cls) itself"
Methods of meta-classes are not classmethods. Methods of Meta-classes receive first argument which is "a reference to instance(traditionally referred as self) not class"

Is it right to say that setter function for property in python is like overloading the assignment operator?

I just started to study the subject of OOP in python and and I got into a bit of trouble subject of decorators/properties and all "private" methods tricks in python..
Is it okay to say that when using #property is like using an attribute, but behind the scenes there is a function that do something? (for example, checks the input)
In addition, is using the #is_barking.setter is like overloading the assignment operator in other languages (let's say C++)? because i can also check the input and things like that
This is the code:
class Dog():
def __init__(self,name):
self.name = name
#property
def is_barking(self):
try:
return self._is_barking
except AttributeError as error:
self._is_barking = False
return self._is_barking
#is_barking.setter
def is_barking(self,value):
self._is_barking = value
def main():
rexi = Dog("rexi")
print(rexi.is_barking)
rexi.is_barking = True
print(rexi.is_barking)
main()
>> False
>> True
Thank you very much!

Yes, that's sort of correct. The setter lets you intercept assignment to that attribute name alone.
It doesn't overload assignment, it is simply facilitated for by the descriptor protocol at runtime, and property objects are data descriptor objects; they get to intercept all attribute access (getting, setting and deleting) on instances of the class they are a member of, and so can veto any of those operations. This is different from C++ assignment overloading, which is handled at compile time (IIRC) and operates on whole instances, not just attributes on instances.
What really happens is that attribute assignment is handled by the class object whose instance the attribute is being assigned to, in the object.__setattr__ special method. The class will check if the given attribute name is covered by an object on the class that is a descriptor object with a __set__ or __delete__ method.
So general attribute assignment can be hooked into via the __setattr__ special method, Python delegates attribute setting to the parent type of instances:
# foo.attr = bar -> Python essentially calls __setattr__ on the class
type(foo).__setattr__(foo, "attr", bar)
and in the regular, simple case that then becomes
foo.__dict__["attr"] = bar
but in reality, there is a search for a data descriptor first, on type(foo) and its parent classes. If such an object exist, then it is tasked with handling attribute setting:
obj = None
for cls in reversed(type(foo).__mro__):
if "attr" in cls.__dict__:
obj = cls.__dict__["attr"]
break
if hasattr(obj, "__set__") or hasattr(obj, "__delete__"): # data descriptor?
try:
obj.__set__(foo, bar)
except AttributeError:
raise AttributeError("can't set attribute")
else:
foo.__dict__["attr"] = bar
property objects implement __set__, and this implementation will call your setter (if one is set).

Python set attributes during object creation in __new__

When using __new__ to customize the creation of a metaclass, we can pass attributes to the type().__new__ method which will be set on the object before it is returned, e.g.
class Foo(type):
def __new__(cls, name, bases, attrs):
attrs['customAttr'] = 'someVal'
return type.__new__(cls, name, bases, attrs)
So that:
>> Foo.__dict__
{'customeAttr': 'someVal', ...}
However I don't know how to do the same for a normal (non-meta) class, which causes a problem when using __setattr__:
class Bar(object):
def __new__(cls, someVal):
obj = object().__new__(cls) # cant pass custom attrs
obj.customAttr = someVal # obj is already a Bar and invokes __setattr__
return obj
def __setattr__(*args): raise Exception('read-only class')
So that unfortunately:
>>> Bar(42)
...
Exception: read-only class
In the __new__ of Bar I get back a fully fledged class instance from object() and any attribute access goes through normal lookup rules, in this case invoking __setattr__. Metaclass Foo avoids this as type() will set attributes before returning the instance during low-level creation whereas object() will not.
Is there a way of passing attributes to object() or is another another type I can use as the instance returned from __new__ that does allow attributes to be set before it becomes a full class instance? I am not interesting in solutions like setting __class__ after instance creation.

You have to explictly bypass your own class's __setattr__ by calling the super or root object __setattr__. So you'd change:
obj.customAttr = someVal
to:
object.__setattr__(obj, 'customAttr', someVal)
A less general approach (doesn't apply to __slots__ based classes) is to directly assign to __dict__ using dict operations:
obj.__dict__['customAttr'] = someVal # Equivalently: vars(obj)['customAttr'] = someVal
The first approach is what the newly __slots__-ed uuid.UUID now uses; before it became __slots__-ed, it used the second approach. In both cases this was needed because they used the same __setattr__ trick to make the type as immutable as possible (without going to the trouble of subclassing tuple, a la typing.NamedTuple).

Way to have a class level type descriptor with __set__

I've got data descriptors working for objects with both __set__ and __get__.
It seems however, that class descriptors do not support __set__. Doing so instead replaces the descriptor object itself with the value assigned.
The following code demonstrates this
from __future__ import print_function
class Descriptor(object):
def __get__(self, obj, cls):
print('__get__')
def __set__(self, obj, value):
print('__set__')
class Class(object):
descriptor = Descriptor()
print('Object')
a = Class()
a.descriptor
a.descriptor = 1
print('Class')
Class.descriptor
Class.descriptor = 2
Which outputs
Object
__get__
__set__
Class
__get__
As you can see, the class level __set__ did not get called.
Is there some workaround or hack (no matter how horrible) that will permit a __set__ data descriptor on a class?
Just to be clear, I don't want calling code to have to implement any 'hack'. I want the calling code to work as expected above, but any hack to be 'behind the scenes'.
Using Python 2.7

I won't get into the whole Descriptor protocol. I don't fully understand it myself; in fact, you've reminded me that I need to quit being lazy and really dive into it. For the mean time, I'll say this:
What I do understand is that descriptors will only work their magic on instances. Now, you probably already knew this, which is why you're wondering if there's a hack to go around this limitation.
If you have the slightest familiarity with metaclasses, you'll know that classes are instances too. Classes can be instances of a class, which can also be an instance and so forth. This is great because what you're asking would look something like this:
class Descriptor(object):
def __get__(self, obj, cls):
print('__get__')
def __set__(self, obj, value):
print('__set__')
class MetaClass(type):
descriptor = Descriptor()
class Class(object):
__metaclass__ = MetaClass
# This will work fine when you do Class.descriptor, as you asked
# but it will raise an AttributeError if you do
# a = Class()
# a.descriptor
# Read on for the full explanation...
The descriptor variable defined in MetaClass is only visible to the Class class. Any instances of Class that try to call it will give you an AttributeError. This is because instances of classes, when searching for an attribute, search their own __dict__ before searching the class's __dict__, but it won't search as far as the class's __metaclass__. Now, if you want to have it both ways and use the same variable name for both the class and its instances (although I wouldn't recommend it as it would invite confusion), you could do this:
class Descriptor(object):
def __get__(self, obj, cls):
print('__get__')
def __set__(self, obj, value):
print('__set__')
class MetaClass(type):
descriptor = Descriptor()
class Class(object):
__metaclass__ = MetaClass
descriptor = Descriptor()
At this point you may be wondering: if an instance searches its own __dict__ before searching its class, how is it that calling 'Class.descriptor' won't pick the same descriptor that 'a.descriptor' uses (which, as you've observed, wouldn't work properly) if Class.descriptor is essentially its own instance variable (from a metaclass POV) ?
The answer is that data descriptors (descriptors which have both __get__ and __set__ defined), as opposed to non-data descriptors (which have only defined __get__), have priority over instance variables. In other words, the descriptor variable in MetaClass is the one Class will pick up because it has priority over Class's own descriptor variable. The same goes for the Class instance, which automatically picks up the descriptor variable defined in Class.
I hope I haven't confused you. This stuff is easy to forget, I think doubly so because it's not very common nor necessary to understand this level of magic most of the time. I had to refresh my memory on this one! Good question :)

'super' object not calling __getattr__

I have one object wrapped inside another.
The "Wrapper" accesses the attributes from the "Wrapped" object by overriding __getattr__.
This works well until I need to override an atribute on a sub class, and then access the attribute from the base class using super().
I can still access the attribute directly from __getattr__ but why does super() not work?
class Wrapped(object):
def __init__(self, value):
self.value = value
def hello_world(self):
print 'hello world', self.value
class Wrapper(object):
def __init__(self, obj):
self.wrapped_obj = obj
def __getattr__(self, name):
if name in self.__dict__:
return getattr(self, name)
else:
return getattr(self.wrapped_obj, name)
class Subclass(Wrapper):
def __init__(self, obj):
super(Subclass, self).__init__(obj)
def hello_world(self):
# this works
func = super(Subclass, self).__getattr__('hello_world')()
# this doesn't
super(Subclass, self).hello_world()
a = Wrapped(2)
b = Subclass(a)
b.hello_world()

According to this, super does not allow implicit calls of "hook" functions such as __getattr__. I'm not sure why it is implemented this way (there's probably a good reason and things are already confusing enough since the super object has custom __getattribute__ and __get__ methods as it is), but it seems like it's just the way things are.
Edit: This post appears to clear things up a little. It looks like the problem is the extra layer of indirection caused by __getattribute__ is ignored when calling functions implicitly. Doing foo.x is equivalent to
foo.__getattr__(x)
(Assuming no __getattribute__ method is defined and x is not in foo.__dict__)
However, it is NOT equivalent to
foo.__getattribute__('__getattr__')(x)
Since super returns a proxy object, it has an extra layer of indirection which causes things to fail.
P.S. The self.__dict__ check in your __getattr__ function is completely unnecessary. __getattr__ is only called if the attribute doesn't already exist in your dict. (Use __getattribute__ if you want it to always be called, but then you have to be very careful, because even something simple like if name in self.__dict__ will cause infinite recursion.