Recent I study Python,but I have a question about __slots__. In my opinion, it is for limiting parameters in Class, but also limiting the method in Class?
For example:
from types import MethodType
Class Student(object):
__slots__=('name','age')
When I run the code:
def set_age(self,age):
self.age=age
stu=Student()
stu.set_age=MethodType(set_age,stu,Student)
print stu.age
An error has occurred:
stu.set_age=MethodType(set_age,stu,Student)
AttributeError: 'Student' object has no attribute 'set_age'
I want to know, why not use set_age for this class?
Using __slots__ means you don't get a __dict__ with each class instance, and so each instance is more lightweight. The downside is that you cannot modify the methods and cannot add attributes. And you cannot do what you attempted to do, which is to add methods (which would be adding attributes).
Also, the pythonic approach is not to instantiate a MethodType, but to simply create the function in the class namespace. If you're attempting to add or modify the function on the fly, as in monkey-patching, then you simply assign the function to the class, as in:
Student.set_age = set_age
Assigning it to the instance, of course, you can't do if it uses __slots__.
Here's the __slots__ docs:
https://docs.python.org/2/reference/datamodel.html#slots
In new style classes, methods are not instance attributes. Instead, they're class attributes that follow the descriptor protocol by defining a __get__ method. The method call obj.some_method(arg) is equivalent to obj.__class__.method.__get__(obj)(arg), which is in turn, equivalent to obj.__class__.method(obj, arg). The __get__ implementation does the instance binding (sticking obj in as the first argument to method when it is called).
In your example code, you're instead trying to put a hand-bound method as an instance variable of the already-existing instance. This doesn't work because your __slots__ declaration prevents you from adding new instance attributes. However, if you wrote to the class instead, you'd have no problem:
class Foo(object):
__slots__ = () # no instance variables!
def some_method(self, arg):
print(arg)
Foo.some_method = some_method # this works!
f = Foo()
f.some_method() # so does this
This code would also work if you created the instance before adding the method to its class.
Your attribute indeed doesn't have an attribute set_age since you didn't create a slot for it. What did you expect?
Also, it should be __slots__ not __slots (I imagine this is right in your actual code, otherwise you wouldn't be getting the error you're getting).
Why aren't you just using:
class Student(object):
__slots__ = ('name','age')
def set_age(self,age):
self.age = age
where set_age is a method of the Student class rather than adding the function as a method to an instance of the Student class.
Instead of __slots__, I'm using the following method. It allow the use of only a predefined set of parameters:
class A(object):
def __init__(self):
self.__dict__['a']=''
self.__dict__['b']=''
def __getattr__(self,name):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
return d.__dict__[attr]
else:
raise AttributeError
def __setattr__(self,name,value):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
d[name] = value
else:
raise AttributeError
The use of getattr(..) is to avoid recursion.
There are some merits usin __slots__ vs __dict__ in term of memory and perhaps speed but this is easy to implement and read.
Related
I am trying to gain a better understanding of class variables and the #classmethod decorator in python. I've done a lot of googling but I am having difficulty grasping basic OOP concepts. Take the following class:
class Repository:
repositories = []
repository_count = 0
def __init__(self):
self.update_repositories()
Repository.repository_count += 1
#classmethod
def update_repositories(cls):
if not cls.repositories:
print('appending repository')
cls.repositories.append('twenty')
else:
print('list is full')
a = Repository()
b = Repository()
print(Repository.repository_count)
Output:
appending repository
list is full
2
In the __init__ method, why does self.update_repositories() successfully call the update_repositories class method? I thought that self in this case refers to the instantiated object, not the class?
The code works without using the #classmethod decorator. Why?
In the __init__ method why do I need to use the keyword Repository in Repository.repository_count += 1? Am I doing this correctly or is there a better practice?
Class methods can be called from an instance. Look at the documentation here.
A class method can be called either on the class (such as C.f()) or on an instance (such as C().f()). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument.
The function works without the decorator, but it is not a class method. The cls and self parameter names are simply convention. You can put anything in the place of cls or self. For example:
class Demo:
def __init__(self):
pass
def instance_method(test):
print(test)
#classmethod
def class_method(test):
print(test)
demo = Demo()
This results in:
demo.instance_method()
>>> <__main__.Demo object at 0x7facd8e34510>
demo.class_method()
>>> <class '__main__.Demo'>
So all non decorated methods in a class are a considered instance
methods and all methods decorated with #classmethod are
class methods. Naming your parameters cls, self or
anything else for that matter does not effect the functionality, but I
would strongly advice sticking with convention.
In your case specifcally removing the #classmethod decorator turns the method into an instance method and cls is now actually what self would normally be, a reference to the class's instance. Since class methods and attributes can be called from an instance cls.update_repositories still points to the class variable.
Depends on what you are trying to do. Generally if you want to access a class variable or method inside a class, but outside a class method, your approach is correct.
In Python, I can create a class method using the #classmethod decorator:
>>> class C:
... #classmethod
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> C.f()
f called with cls=<class '__main__.C'>
Alternatively, I can use a normal (instance) method on a metaclass:
>>> class M(type):
... def f(cls):
... print(f'f called with cls={cls}')
...
>>> class C(metaclass=M):
... pass
...
>>> C.f()
f called with cls=<class '__main__.C'>
As shown by the output of C.f(), these two approaches provide similar functionality.
What are the differences between using #classmethod and using a normal method on a metaclass?
As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the abc.ABCMeta.register method.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict).
Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
#classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in #classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a #property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.
When you phrase it like you did in the question, the #classmethod and metaclasses may look similar but they have rather different purposes. The class that is injected in the #classmethod's argument is usually used for constructing an instance (i.e. an alternative constructor). On the other hand, the metaclasses are usually used to modify the class itself (e.g. like what Django does with its models DSL).
That is not to say that you can't modify the class inside a classmethod. But then the question becomes why didn't you define the class in the way you want to modify it in the first place? If not, it might suggest a refactor to use multiple classes.
Let's expand the first example a bit.
class C:
#classmethod
def f(cls):
print(f'f called with cls={cls}')
Borrowing from the Python docs, the above will expand to something like the following:
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
class C:
def f(cls):
print(f'f called with cls={cls}')
f = ClassMethod(f)
Note how __get__ can take either an instance or the class (or both), and thus you can do both C.f and C().f. This is unlike the metaclass example you give which will throw an AttributeError for C().f.
Moreover, in the metaclass example, f does not exist in C.__dict__. When looking up the attribute f with C.f, the interpreter looks at C.__dict__ and then after failing to find, looks at type(C).__dict__ (which is M.__dict__). This may matter if you want the flexibility to override f in C, although I doubt this will ever be of practical use.
In your example, the difference would be in some other classes that will have M set as their metaclass.
class M(type):
def f(cls):
pass
class C(metaclass=M):
pass
class C2(metaclass=M):
pass
C.f()
C2.f()
class M(type):
pass
class C(metaclass=M):
#classmethod
def f(cls):
pass
class C2(metaclass=M):
pass
C.f()
# C2 does not have 'f'
Here is more on metaclasses
What are some (concrete) use-cases for metaclasses?
Both #classmethod and Metaclass are different.
Everything in python is an object. Every thing means every thing.
What is Metaclass ?
As said every thing is an object. Classes are also objects in fact classes are instances of other mysterious objects formally called as meta-classes. Default metaclass in python is "type" if not specified
By default all classes defined are instances of type.
Classes are instances of Meta-Classes
Few important points are to understand metioned behaviour
As classes are instances of meta classes.
Like every instantiated object, like objects(instances) get their attributes from class. Class will get it's attributes from Meta-Class
Consider Following Code
class Meta(type):
def foo(self):
print(f'foo is called self={self}')
print('{} is instance of {}: {}'.format(self, Meta, isinstance(self, Meta)))
class C(metaclass=Meta):
pass
C.foo()
Where,
class C is instance of class Meta
"class C" is class object which is instance of "class Meta"
Like any other object(instance) "class C" has access it's attributes/methods defined in it's class "class Meta"
So, decoding "C.foo()" . "C" is instance of "Meta" and "foo" is method calling through instance of "Meta" which is "C".
First argument of method "foo" is reference to instance not class unlike "classmethod"
We can verify as if "class C" is instance of "Class Meta
isinstance(C, Meta)
What is classmethod?
Python methods are said to be bound. As python imposes the restriction that method has to be invoked with instance only.
Sometimes we might want to invoke methods directly through class without any instance (much like static members in java) with out having to create any instance.By default instance is required to call method. As a workaround python provides built-in function classmethod to bind given method to class instead of instance.
As class methods are bound to class. It takes at least one argument which is reference to class itself instead of instance (self)
if built-in function/decorator classmethod is used. First argument
will be reference to class instead of instance
class ClassMethodDemo:
#classmethod
def foo(cls):
print(f'cls is ClassMethodDemo: {cls is ClassMethodDemo}')
As we have used "classmethod" we call method "foo" without creating any instance as follows
ClassMethodDemo.foo()
Above method call will return True. Since first argument cls is indeed reference to "ClassMethodDemo"
Summary:
Classmethod's receive first argument which is "a reference to class(traditionally referred as cls) itself"
Methods of meta-classes are not classmethods. Methods of Meta-classes receive first argument which is "a reference to instance(traditionally referred as self) not class"
I want to initialize class member with use class method but not know how to call it.
Can you suggest some solution - maybe it is very trivial but I can not find solution?
This code will not work - I do not why?
class X(object):
#staticmethod
def __Y():
return 1
CONSTANT = __Y()
x = X()
print x.CONSTANT
This will work but I need use call method to initialize class members.
class X(object):
CONSTANT = 1
x = X()
print x.CONSTANT
Note, I am not want initialize object variables but class variable.
Drop the #staticmethod decorator and the first approach will work as well. You don't need staticmethod just to call a function inside a class statement.
Since that way the function will not be usable when called from class instances, it is an idiom to also remove it after use. In your example:
class X(object):
def __y():
return 1
CONSTANT = __y()
# ... other uses of __y, if any
del __y
To understand why your approach didn't work, consider what staticmethod does. It wraps a normal function into a descriptor object that, when retrieved from the class, produces the original function unchanged, i.e. without the usual "bound method" semantics. (Retrieving a normal def function from an instance or a class would get you a bound method that automagically inserts self as the first argument.)
However, the descriptor returned by staticmethod is itself not callable, its sole function is to produce the callable object when accessed through the class or instance. Proposals to make the staticmethod descriptor callable were rejected because such use of staticmethod is erroneous in the first place.
Python (2.6) seems to be derping for no reason, can anyone see a problem with this code?
class DB ():
def doSomething (self, str):
print str
class A ():
__db = DB()
#staticmethod
def getDB ():
return A.__db
db = property(getDB)
A.db.doSomething("blah")
Fails with the exception:
AttributeError: 'property' object has no attribute 'doSomething'
It was my understanding that a property would automatically run its getter when accessed, so why is it complaining about a property object, and why isn't it finding my clearly available method?
In addition to needing to inherit from object, properties only work on instances.
a = A()
a.db.doSomething("blah")
To make a property work on the class, you can define a metaclass. (A class is an instance of a metaclass, so properties defined on the metaclass work on the class, just as properties defined on a class work on an instance of that class.)
You aren't using classes correctly. A class is (normally) two things:
A factory for creating a family of related objects
A definition of the common behaviour of those objects
These related objects are the instances of the class. Normal methods are invoked on instances of the class, not on the class itself. If you want methods that can be invoked from the class, without an instance, you need to label the methods with #classmethod (or #staticmethod).
However I don't actually know whether properties work when retrieved from a class object. I can't check right now, but I don't think so. The error you are getting is that A.db is retrieving the property object which defines the property itself, it isn't "evaluating" the property to get A.__db. Property objects have no doSomething attribute. Properties are designed to be created in classes as descriptions of how the instances of those classes work.
If you did intend to be working with an instance of A, then you'll need to create one:
my_a = A()
my_a.db.doSomething("blah")
However, this will also fail. You have not correctly written getDB as any kind of method. Normal methods need an argument to represent the instance it was invoked on (traditionally called self):
def getDB(self):
...
Static methods don't, but need a decorator to label them as static:
#staticmethod
def getDB():
...
Class methods need both an argument to receive the class they were invoked on, and a decorator:
#classmethod
def getDB(cls):
...
You don't need getters in Python:
class B(object):
def do_something(self, str):
print str
class A(object):
db = B()
A.db.do_something("blah")
(I also PEP8:ed the code)
Look at this code:
class MyClass():
# Why does this give me "NameError: name 'self' is not defined":
mySelf = self
# But this does not?
def myFunction(self):
mySelf2 = self
Basically I want a way for a class to refer to itself without needing to name itself specifically, hence I want self to work for the class, not just methods/functions. How can I achieve this?
EDIT: The point of this is that I'm trying to refer to the class name from inside the class itself with something like self.class._name_ so that the class name isn't hardcoded anywhere in the class's code, and thus it's easier to re-use the code.
EDIT 2: From what I've learned from the answers below, what I'm trying to do is impossible. I'll have to find a different way. Mission abandoned.
EDIT 3: Here is specifically what I'm trying to do:
class simpleObject(object):
def __init__(self, request):
self.request = request
#view_defaults(renderer='string')
class Test(simpleObject):
# this line throws an error because of self
myClassName = self.__class__.__name__
#view_config(route_name=myClassName)
def activateTheView(self):
db = self.request.db
foo = 'bar'
return foo
Note that self is not defined at the time when you want the class to refer to itself for the assignment to work. This is because (in addition to being named arbitrarily), self refers to instances and not classes. At the time that the suspect line of code attempts to run, there is as of yet no class for it to refer to. Not that it would refer to the class if there was.
In a method, you can always use type(self). That will get the subclass of MyClass that created the current instance. If you want to hard-code to MyClass, that name will be available in the global scope of the methods. This will allow you to do everything that your example would allow if it actually worked. E.g, you can just do MyClass.some_attribute inside your methods.
You probably want to modify the class attributes after class creation. This can be done with decorators or on an ad-hoc basis. Metaclasses may be a better fit. Without knowing what you actually want to do though, it's impossible to say.
UPDATE:
Here's some code to do what you want. It uses a metaclass AutoViewConfigMeta and a new decorator to mark the methods that you want view_config applied to. I spoofed the view_config decorator. It prints out the class name when it's called though to prove that it has access to it. The metaclass __new__ just loops through the class dictionary and looks for methods that were marked by the auto_view_config decorator. It cleans off the mark and applies the view_config decorator with the appropriate class name.
Here's the code.
# This just spoofs the view_config decorator.
def view_config(route=''):
def dec(f):
def wrapper(*args, **kwargs):
print "route={0}".format(route)
return f(*args, **kwargs)
return wrapper
return dec
# Apply this decorator to methods for which you want to call view_config with
# the class name. It will tag them. The metaclass will apply view_config once it
# has the class name.
def auto_view_config(f):
f.auto_view_config = True
return f
class AutoViewConfigMeta(type):
def __new__(mcls, name, bases, dict_):
#This is called during class creation. _dict is the namespace of the class and
# name is it's name. So the idea is to pull out the methods that need
# view_config applied to them and manually apply them with the class name.
# We'll recognize them because they will have the auto_view_config attribute
# set on them by the `auto_view_config` decorator. Then use type to create
# the class and return it.
for item in dict_:
if hasattr(dict_[item], 'auto_view_config'):
method = dict_[item]
del method.auto_view_config # Clean up after ourselves.
# The next line is the manual form of applying a decorator.
dict_[item] = view_config(route=name)(method)
# Call out to type to actually create the class with the modified dict.
return type.__new__(mcls, name, bases, dict_)
class simpleObject(object):
__metaclass__ = AutoViewConfigMeta
class Test(simpleObject):
#auto_view_config
def activateTheView(self):
foo = 'bar'
print foo
if __name__=='__main__':
t = Test()
t.activateTheView()
Let me know if you have any questions.
Python has an "explict is better than implicit" design philosophy.
Many languages have an implicit pointer or variable in the scope of a method that (e.g. this in C++) that refers to the object through which the method was invoked. Python does not have this. Here, all bound methods will have an extra first argument that is the object through which the method was invoked. You can call it anything you want (self is not a keyword like this in C++). The name self is convention rather than a syntactic rule.
Your method myFunction defines the variable self as a parameter so it works. There's no such variable at the class level so it's erroring out.
So much for the explanation. I'm not aware of a straightforward way for you to do what you want and I've never seen such requirement in Python. Can you detail why you want to do such a thing? Perhaps there's an assumption that you're making which can be handled in another way using Python.
self is just a name, your self in this case is a class variable and not this for the object using which it is called,
self is treated as a normal variable and it is not defined, where as the self in the function comes from the object used for calling.
you want to treat the object reference in self as a class variable which is not possible.
self isn't a keyword, it's just a convention. The methods are attributes of the class object (not the instance), but they receive the instance as their first argument. You could rename the argument to xyzzy if you wanted and it would still work the same way.
But (as should be obvious) you can't refer to a method argument outside the body of the method. Inside a class block but outside of any method, self is undefined. And the concept wouldn't even make sense -- at the time the class block is being evaluated, no instance of the class can possibly exist yet.
Because the name self is explicitly defined as part of the arguments to myFunction. The first argument to a method is the instance that the method was called on; in the class body, there isn't an "instance we're dealing with", because the class body deals with every possible instance of the class (including ones that don't necessarily exist yet) - so, there isn't a particular object that could be called self.
If you want to refer to the class itself, rather than some instance of it, this is spelled self.__class__ (or, for new-style classes in Py2 and all classes in Py3, type(self)) anywhere self exists. If you want to be able to deal with this in situations where self doesn't exist, then you may want to look at class methods which aren't associated with any particular instance, and so take the class itself in place of self. If you really need to do this in the class body (and, you probably don't), you'll just have to call it by name.
You can't refer to the class itself within the class body because the class doesn't exist at the time that the class body is executed. (If the previous sentence is confusing, reading up about metaclasses will either clear this up or make you more confused.)
Within an instance method, you can refer to the class of the instance with self.__class__, but be careful here. This will be the instance's actual class, which through the power of inheritance might not be the class in which the method was defined.
Within a class method, the class is passed in as the first argument, much like instances are the first argument to instance methods:
class MyClass(object):
#classmethod
def foo(cls):
print cls.__name__
MyClass.foo() # Should print "MyClass"
As with instance methods, the actual class might differ due to inheritance.
class OtherClass(MyClass):
pass
OtherClass.foo() # Should print "OtherClass"
If you really need to refer to MyClass within a method of MyClass, you're pretty much going to have to refer to it as MyClass unless you use magic. This sort of magic is more trouble than it is worth.