Python (2.6) seems to be derping for no reason, can anyone see a problem with this code?
class DB ():
def doSomething (self, str):
print str
class A ():
__db = DB()
#staticmethod
def getDB ():
return A.__db
db = property(getDB)
A.db.doSomething("blah")
Fails with the exception:
AttributeError: 'property' object has no attribute 'doSomething'
It was my understanding that a property would automatically run its getter when accessed, so why is it complaining about a property object, and why isn't it finding my clearly available method?
In addition to needing to inherit from object, properties only work on instances.
a = A()
a.db.doSomething("blah")
To make a property work on the class, you can define a metaclass. (A class is an instance of a metaclass, so properties defined on the metaclass work on the class, just as properties defined on a class work on an instance of that class.)
You aren't using classes correctly. A class is (normally) two things:
A factory for creating a family of related objects
A definition of the common behaviour of those objects
These related objects are the instances of the class. Normal methods are invoked on instances of the class, not on the class itself. If you want methods that can be invoked from the class, without an instance, you need to label the methods with #classmethod (or #staticmethod).
However I don't actually know whether properties work when retrieved from a class object. I can't check right now, but I don't think so. The error you are getting is that A.db is retrieving the property object which defines the property itself, it isn't "evaluating" the property to get A.__db. Property objects have no doSomething attribute. Properties are designed to be created in classes as descriptions of how the instances of those classes work.
If you did intend to be working with an instance of A, then you'll need to create one:
my_a = A()
my_a.db.doSomething("blah")
However, this will also fail. You have not correctly written getDB as any kind of method. Normal methods need an argument to represent the instance it was invoked on (traditionally called self):
def getDB(self):
...
Static methods don't, but need a decorator to label them as static:
#staticmethod
def getDB():
...
Class methods need both an argument to receive the class they were invoked on, and a decorator:
#classmethod
def getDB(cls):
...
You don't need getters in Python:
class B(object):
def do_something(self, str):
print str
class A(object):
db = B()
A.db.do_something("blah")
(I also PEP8:ed the code)
Related
I am trying to gain a better understanding of class variables and the #classmethod decorator in python. I've done a lot of googling but I am having difficulty grasping basic OOP concepts. Take the following class:
class Repository:
repositories = []
repository_count = 0
def __init__(self):
self.update_repositories()
Repository.repository_count += 1
#classmethod
def update_repositories(cls):
if not cls.repositories:
print('appending repository')
cls.repositories.append('twenty')
else:
print('list is full')
a = Repository()
b = Repository()
print(Repository.repository_count)
Output:
appending repository
list is full
2
In the __init__ method, why does self.update_repositories() successfully call the update_repositories class method? I thought that self in this case refers to the instantiated object, not the class?
The code works without using the #classmethod decorator. Why?
In the __init__ method why do I need to use the keyword Repository in Repository.repository_count += 1? Am I doing this correctly or is there a better practice?
Class methods can be called from an instance. Look at the documentation here.
A class method can be called either on the class (such as C.f()) or on an instance (such as C().f()). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument.
The function works without the decorator, but it is not a class method. The cls and self parameter names are simply convention. You can put anything in the place of cls or self. For example:
class Demo:
def __init__(self):
pass
def instance_method(test):
print(test)
#classmethod
def class_method(test):
print(test)
demo = Demo()
This results in:
demo.instance_method()
>>> <__main__.Demo object at 0x7facd8e34510>
demo.class_method()
>>> <class '__main__.Demo'>
So all non decorated methods in a class are a considered instance
methods and all methods decorated with #classmethod are
class methods. Naming your parameters cls, self or
anything else for that matter does not effect the functionality, but I
would strongly advice sticking with convention.
In your case specifcally removing the #classmethod decorator turns the method into an instance method and cls is now actually what self would normally be, a reference to the class's instance. Since class methods and attributes can be called from an instance cls.update_repositories still points to the class variable.
Depends on what you are trying to do. Generally if you want to access a class variable or method inside a class, but outside a class method, your approach is correct.
In Python, I can create a class method using the #classmethod decorator:
>>> class C:
... #classmethod
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> C.f()
f called with cls=<class '__main__.C'>
Alternatively, I can use a normal (instance) method on a metaclass:
>>> class M(type):
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> class C(metaclass=M):
... pass
...
>>> C.f()
f called with cls=<class '__main__.C'>
As shown by the output of C.f(), these two approaches provide similar functionality.
What are the differences between using #classmethod and using a normal method on a metaclass?
As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the abc.ABCMeta.register method.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict).
Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
#classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in #classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a #property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.
When you phrase it like you did in the question, the #classmethod and metaclasses may look similar but they have rather different purposes. The class that is injected in the #classmethod's argument is usually used for constructing an instance (i.e. an alternative constructor). On the other hand, the metaclasses are usually used to modify the class itself (e.g. like what Django does with its models DSL).
That is not to say that you can't modify the class inside a classmethod. But then the question becomes why didn't you define the class in the way you want to modify it in the first place? If not, it might suggest a refactor to use multiple classes.
Let's expand the first example a bit.
class C:
#classmethod
def f(cls):
print(f'f called with cls={cls}')
Borrowing from the Python docs, the above will expand to something like the following:
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
class C:
def f(cls):
print(f'f called with cls={cls}')
f = ClassMethod(f)
Note how __get__ can take either an instance or the class (or both), and thus you can do both C.f and C().f. This is unlike the metaclass example you give which will throw an AttributeError for C().f.
Moreover, in the metaclass example, f does not exist in C.__dict__. When looking up the attribute f with C.f, the interpreter looks at C.__dict__ and then after failing to find, looks at type(C).__dict__ (which is M.__dict__). This may matter if you want the flexibility to override f in C, although I doubt this will ever be of practical use.
In your example, the difference would be in some other classes that will have M set as their metaclass.
class M(type):
def f(cls):
pass
class C(metaclass=M):
pass
class C2(metaclass=M):
pass
C.f()
C2.f()
class M(type):
pass
class C(metaclass=M):
#classmethod
def f(cls):
pass
class C2(metaclass=M):
pass
C.f()
# C2 does not have 'f'
Here is more on metaclasses
What are some (concrete) use-cases for metaclasses?
Both #classmethod and Metaclass are different.
Everything in python is an object. Every thing means every thing.
What is Metaclass ?
As said every thing is an object. Classes are also objects in fact classes are instances of other mysterious objects formally called as meta-classes. Default metaclass in python is "type" if not specified
By default all classes defined are instances of type.
Classes are instances of Meta-Classes
Few important points are to understand metioned behaviour
As classes are instances of meta classes.
Like every instantiated object, like objects(instances) get their attributes from class. Class will get it's attributes from Meta-Class
Consider Following Code
class Meta(type):
def foo(self):
print(f'foo is called self={self}')
print('{} is instance of {}: {}'.format(self, Meta, isinstance(self, Meta)))
class C(metaclass=Meta):
pass
C.foo()
Where,
class C is instance of class Meta
"class C" is class object which is instance of "class Meta"
Like any other object(instance) "class C" has access it's attributes/methods defined in it's class "class Meta"
So, decoding "C.foo()" . "C" is instance of "Meta" and "foo" is method calling through instance of "Meta" which is "C".
First argument of method "foo" is reference to instance not class unlike "classmethod"
We can verify as if "class C" is instance of "Class Meta
isinstance(C, Meta)
What is classmethod?
Python methods are said to be bound. As python imposes the restriction that method has to be invoked with instance only.
Sometimes we might want to invoke methods directly through class without any instance (much like static members in java) with out having to create any instance.By default instance is required to call method. As a workaround python provides built-in function classmethod to bind given method to class instead of instance.
As class methods are bound to class. It takes at least one argument which is reference to class itself instead of instance (self)
if built-in function/decorator classmethod is used. First argument
will be reference to class instead of instance
class ClassMethodDemo:
#classmethod
def foo(cls):
print(f'cls is ClassMethodDemo: {cls is ClassMethodDemo}')
As we have used "classmethod" we call method "foo" without creating any instance as follows
ClassMethodDemo.foo()
Above method call will return True. Since first argument cls is indeed reference to "ClassMethodDemo"
Summary:
Classmethod's receive first argument which is "a reference to class(traditionally referred as cls) itself"
Methods of meta-classes are not classmethods. Methods of Meta-classes receive first argument which is "a reference to instance(traditionally referred as self) not class"
How can I override the metaclass of a Python class, with a unittest.mock.MagicMock instance instead?
I have a function whose job involves working with the metaclass of an argument:
# lorem.py
class Foo(object):
pass
def quux(existing_class):
…
metaclass = type(existing_class)
new_class = metaclass(…)
The unit tests for this function will need to assert that the calls to
the metaclass go as expected, without actually calling a real class
object.
Note: The test case does not care about the metaclass's behaviour; it cares that quux retrieves that metaclass (using type(existing_class)) and calls the metaclass with the correct arguments.
So to write a unit test for this function, I want to pass a class object whose metaclass is a mock object instead. This will allow, for example, making assertions about how the metaclass was called, and ensuring no unwanted side effects.
# test_lorem.py
import unittest
import unittest.mock
import lorem
class stub_metaclass(type):
def __new__(metaclass, name, bases, namespace):
return super().__new__(metaclass, name, bases, namespace)
class quux_TestCase(unittest.TestCase):
#unittest.mock.patch.object(
lorem.Foo, '__class__', side_effect=stub_metaclass)
def test_calls_expected_metaclass_with_class_name(
self,
mock_foo_metaclass,
):
expected_name = 'Foo'
expected_bases = …
expected_namespace = …
lorem.quux(lorem.Foo)
mock_foo_metaclass.assert_called_with(
expected_name, expected_bases, expected_namespace)
When I try to mock the __class__ attribute of an existing class, though, I get this error:
File "/usr/lib/python3/dist-packages/mock/mock.py", line 1500, in start
result = self.__enter__()
File "/usr/lib/python3/dist-packages/mock/mock.py", line 1460, in __enter__
setattr(self.target, self.attribute, new_attr)
TypeError: __class__ must be set to a class, not 'MagicMock' object
This is telling me that unittest.mock.patch is attempting to set the __class__ attribute temporarily to a MagicMock instance, as I want; but Python is refusing that with a TypeError.
But placing a mock object as the metaclass is exactly what I'm trying to do: put a unittest.mock.MagicMock instance in the __class__ attribute in order that the mock object will do all that it does: record calls, pretend valid behaviour, etc.
How can I set a mock object in place of the Foo class's __class__ attribute, in order to instrument Foo and test that my code uses Foo's metaclass correctly?
You can't do exactly what you want. As you can see an object's __class__ attribute is very special in Python, and even for ordinary instances there are checks in runtime to verify it is assigned to a proper type.
When you get down to a class's __class__, that is even more strict.
Possible approach:
One thing to do in there is not pass a class to your test - but an object that is an instance from a crafted ordinary class, which will have an artificial __class__ attribute. Even them, you will have to change your code from calling type(existing_class) to do existing_class.__class__ directly. For an instance object to "falsify" its __class__ anyway, you have to implement __class__ as a property on its class (or override __getattribute__; (the class itself will report its true metaclass, but an instance can return whatever is coded on the __class__ property.
class Foo:
#property
def __class__(self):
return stub_metaclass
Actual suggestion:
But then, since you are at it, maybe the simplest thing is to mock type instead on the target module where quux is defined.
class MockType:
def __init__(self):
self.mock = mock.Mock()
def __call__(self, *args):
return self.mock
...
class ...:
...
def test_calls_expected_metaclass_with_class_name(
self,
):
try:
new_type = MockType()
# This creates "type" on the module "lorem" namespace
# as a global variable. It will then override the built-in "type"
lorem.type = new_type
lorem.quux(lorem.Foo)
finally:
del lorem.type # un-shadows the built-in type on the module
new_type.mock.assert_called_with(
'Foo', unittest.mock.ANY, unittest.mock.ANY)
Still another approach
Another thing that can be done is to craft a full "MockMetaclass" in the "old fashion": without unittest.magicmock at all, instead, with intrumented __new__ and other relevant methods that will record the called parameters, and function as a true metaclass for a class you pass in as parameter.
Considerations on what is being done
People reaching here, please note that one should not test the class creation (and metaclass) mechanisms themselves. One can just assume the Python runtime have these working and tested already.
Recent I study Python,but I have a question about __slots__. In my opinion, it is for limiting parameters in Class, but also limiting the method in Class?
For example:
from types import MethodType
Class Student(object):
__slots__=('name','age')
When I run the code:
def set_age(self,age):
self.age=age
stu=Student()
stu.set_age=MethodType(set_age,stu,Student)
print stu.age
An error has occurred:
stu.set_age=MethodType(set_age,stu,Student)
AttributeError: 'Student' object has no attribute 'set_age'
I want to know, why not use set_age for this class?
Using __slots__ means you don't get a __dict__ with each class instance, and so each instance is more lightweight. The downside is that you cannot modify the methods and cannot add attributes. And you cannot do what you attempted to do, which is to add methods (which would be adding attributes).
Also, the pythonic approach is not to instantiate a MethodType, but to simply create the function in the class namespace. If you're attempting to add or modify the function on the fly, as in monkey-patching, then you simply assign the function to the class, as in:
Student.set_age = set_age
Assigning it to the instance, of course, you can't do if it uses __slots__.
Here's the __slots__ docs:
https://docs.python.org/2/reference/datamodel.html#slots
In new style classes, methods are not instance attributes. Instead, they're class attributes that follow the descriptor protocol by defining a __get__ method. The method call obj.some_method(arg) is equivalent to obj.__class__.method.__get__(obj)(arg), which is in turn, equivalent to obj.__class__.method(obj, arg). The __get__ implementation does the instance binding (sticking obj in as the first argument to method when it is called).
In your example code, you're instead trying to put a hand-bound method as an instance variable of the already-existing instance. This doesn't work because your __slots__ declaration prevents you from adding new instance attributes. However, if you wrote to the class instead, you'd have no problem:
class Foo(object):
__slots__ = () # no instance variables!
def some_method(self, arg):
print(arg)
Foo.some_method = some_method # this works!
f = Foo()
f.some_method() # so does this
This code would also work if you created the instance before adding the method to its class.
Your attribute indeed doesn't have an attribute set_age since you didn't create a slot for it. What did you expect?
Also, it should be __slots__ not __slots (I imagine this is right in your actual code, otherwise you wouldn't be getting the error you're getting).
Why aren't you just using:
class Student(object):
__slots__ = ('name','age')
def set_age(self,age):
self.age = age
where set_age is a method of the Student class rather than adding the function as a method to an instance of the Student class.
Instead of __slots__, I'm using the following method. It allow the use of only a predefined set of parameters:
class A(object):
def __init__(self):
self.__dict__['a']=''
self.__dict__['b']=''
def __getattr__(self,name):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
return d.__dict__[attr]
else:
raise AttributeError
def __setattr__(self,name,value):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
d[name] = value
else:
raise AttributeError
The use of getattr(..) is to avoid recursion.
There are some merits usin __slots__ vs __dict__ in term of memory and perhaps speed but this is easy to implement and read.
I'm trying to wrap my brain around something here... Let's say that I have a simple setup of class inheritance:
class SuperClass(object):
def __init__(self):
self.var = 1
class SubClass(SuperClass):
def getVar(self):
return self.var
If I have an existing instance of SuperClass, is there a way that I can 'cast' it as SubClass, without creating a new instance?
In Python you don't have a cast operator - you can get around this by assigning the type to the instance's __class__ attribute:
>>> super_instance = SuperClass()
>>> super_instance.__class__ = SubClass
>>> print super_instance.getVar()
1
However, this is more error prone than cast in many other languages as the validity and safety of your "cast" is not verified by the compiler.
For example, if SubClass had a method that accessed an attribute that was not available on SuperClass then attempting to call that method on super_instance would result in an error at run time, even though it appears to be a valid instance of SubClass.