I am writing a framework, and I want my base class to use different functions for renaming in the child classes. I figured the best way would be to use a class attribute, like in case of A, but I got TypeErrors when running it like in rename_columns(). However it worked with implementation like B
import pandas as pd
class A:
my_func_mask = str.lower
foo = 'bar'
def rename_columns(self, data):
return data.rename(columns=self.my_func_mask)
class B(A):
def rename_columns(self, data):
return data.rename(columns=self.__class__.my_func_mask)
So I experimented with the above a bit, and I get the following:
a = A()
a.foo # Works fine, gives back 'bar'
a.__class__.my_func_mask # Works as expected `a.__class__.my_func_mask is str.lower` is true
a.my_func_mask # throws TypeError: descriptor 'lower' for 'str' objects doesn't apply to 'A' object
My questions would be why can I use regular typed (int, str, etc.) values as class attributes and access them on the instance as well, while I cannot do that for functions?
What happens during the attribute lookup in these cases? What is the difference in the attribute resolution process?
Actually both foo and my_func_mask is in __class__.__dict__ so I am a bit puzzled. Thanks for the clarifications!
You are storing an unbound built-in method on your class, meaning it is a descriptor object. When you then try to access that on self, descriptor binding applies but the __get__ method called to complete the binding tells you that it can't be bound to your custom class instances, because the method would only work on str instances. That's a strict limitation of most methods of built-in types.
You need to store it in a different manner; putting it inside another container, such as a list or dictionary, would avoid binding. Or you could wrap it in a staticmethod descriptor to have it be bound and return the original. Another option is to not store this as a class attribute, and simply create an instance attribute in __init__.
But in this case, I'd not store str.lower as an attribute value, at all. I'd store None and fall back to str.lower when you still encounter None:
return data.rename(columns=self.my_func_mask or str.lower)
Setting my_func_mask to None is a better indicator that a default is going to be used, clearly distinguishable from explicitly setting str.lower as the mask.
You need to declare staticmethod.
class A:
my_func_mask = staticmethod(str.lower)
foo = 'bar'
>>> A().my_func_mask is str.lower
>>> True
Everything that is placed in the class definition is bound to the class, but you can't bind a built-in to your own class.
Essentially, all code that you place in a class is executed when the class is created. All items in locals() are then bound to your class at the end of the class. That's why this also works to bind a method to your class:
def abc(self):
print('{} from outside the class'.format(self))
class A:
f1 = abc
f2 = lambda self: print('{} from lambda'.format(self))
def f3(self):
print('{} from method'.format(self))
To not have the function bound to your class, you have to place it in the __init__ method of your class:
class A:
def __init__(self):
self.my_func_mask = str.lower
Related
I've got a class, where a method should only run once. Of course, it could easily be done with artificial has_executed = True/False flag, but why use it, if you can just delete the method itself? python's a duck-typed language, everything is a reference, bla-bla-bla, what can go wrong?
At least it was the thought. I couldn't actually do it:
class A:
def b(self):
print("empty")
self.__delattr__('b')
a = A()
a.b()
raises AttributeError: b. However, executing self.__getattribute__('b') returns <bound method A.b of <__main__.A object at 0x000001CDC6742FD0>>, which sounds stupid to me: why is a method any different from an attribute, since everything in python is just a reference to an object? And why can I __getattribute__, but not __delattr__?
The same goes to redefinition. I can easily set any attribute, but methods are a no-no?
class A:
def b(self):
print("first")
self.__setattr__('b', lambda self: print(f"second"))
a = A()
a.b()
a.b()
results into TypeError: <lambda>() missing 1 required positional argument: 'self'. Which, of course, means, that now python isn't using dot-notation as intended. Of course, we could ditch the self attribute in the lambda altogether, considering we've got the reference to it already in b. But isn't it incorrect by design?
The further I'm trying to take python to the limit, the more frustrated I become. Some imposed limitations (or seemingly imposed?) seem so unnatural, considering the way the language is marketed. Shouldn't it allow this? Why doesn't it work?
UPD
Ok, consider this:
class A:
def __init__(self):
self.variable = 1
def b(self):
print("old")
self.variable += 1
def new_b():
print("new")
self.variable += 15
self.__setattr__('b', new_b)
It will work and do what we want: none of other objects will have their A.b method redefined once one object kind of overlays its b definition. (overlays, since everyone so far says that you cannot redefine a method for an object, but instead only kind of hide it from the caller behind another attribute with the same name, as far as I understand).
Is this good?
It doesn't work because b isn't an attribute belonging to the instance, it belongs to the class. So you can't delete it on the instance because it isn't there to be deleted.
>>> a = A()
>>> list(a.__dict__)
[]
>>> list(A.__dict__)
['__module__', 'b', '__dict__', '__weakref__', '__doc__']
When a.b is evaluated, Python will see that a has no instance attribute named b and fall back to the class. (It's a little more complicated because when falling back to the class, it will not simply return the method itself, but a version of the method which is bound to the instance a.)
Since you don't want to delete the method on the class, the way to go is to replace the method on the instance. I don't know why you tried to do this with __setattr__ - there is no need for that, simply assign self.b = ... as normal. The reason your attempt failed is because your lambda requires a positional parameter named self, but this parameter will not be automatically bound to the instance when you look it up, because it is an instance attribute, not a class attribute.
class A:
def b(self):
print('first')
self.b = lambda: print('second')
Usage:
>>> a = A()
>>> a.b()
first
>>> a.b()
second
Well in python you have 2 types of attributes
A class attribute is a variable that belongs to a certain class, and not a particular object. Every instance of this class shares the same variable. These attributes are usually defined outside the init constructor
An instance/object attribute is a variable that belongs to one (and only one) object. Every instance of a class points to its own attributes variables. These attributes are defined within the init constructor.
In case of a class attribute its part of the class descriptor, so you cannot delete it from the object attributes like self.__deleteattr__ or add new one with __setattr__ as it alters the class descriptor and reflects on all objects. Such an operation can have devastating effects.
Its very similar to a class variable as well. You can however change the behavior with overriding or reassigning like below
class A:
def b(self):
print("empty")
A.b = lambda self: print(f"second")
a = A()
a.b()
a.b()
Recent I study Python,but I have a question about __slots__. In my opinion, it is for limiting parameters in Class, but also limiting the method in Class?
For example:
from types import MethodType
Class Student(object):
__slots__=('name','age')
When I run the code:
def set_age(self,age):
self.age=age
stu=Student()
stu.set_age=MethodType(set_age,stu,Student)
print stu.age
An error has occurred:
stu.set_age=MethodType(set_age,stu,Student)
AttributeError: 'Student' object has no attribute 'set_age'
I want to know, why not use set_age for this class?
Using __slots__ means you don't get a __dict__ with each class instance, and so each instance is more lightweight. The downside is that you cannot modify the methods and cannot add attributes. And you cannot do what you attempted to do, which is to add methods (which would be adding attributes).
Also, the pythonic approach is not to instantiate a MethodType, but to simply create the function in the class namespace. If you're attempting to add or modify the function on the fly, as in monkey-patching, then you simply assign the function to the class, as in:
Student.set_age = set_age
Assigning it to the instance, of course, you can't do if it uses __slots__.
Here's the __slots__ docs:
https://docs.python.org/2/reference/datamodel.html#slots
In new style classes, methods are not instance attributes. Instead, they're class attributes that follow the descriptor protocol by defining a __get__ method. The method call obj.some_method(arg) is equivalent to obj.__class__.method.__get__(obj)(arg), which is in turn, equivalent to obj.__class__.method(obj, arg). The __get__ implementation does the instance binding (sticking obj in as the first argument to method when it is called).
In your example code, you're instead trying to put a hand-bound method as an instance variable of the already-existing instance. This doesn't work because your __slots__ declaration prevents you from adding new instance attributes. However, if you wrote to the class instead, you'd have no problem:
class Foo(object):
__slots__ = () # no instance variables!
def some_method(self, arg):
print(arg)
Foo.some_method = some_method # this works!
f = Foo()
f.some_method() # so does this
This code would also work if you created the instance before adding the method to its class.
Your attribute indeed doesn't have an attribute set_age since you didn't create a slot for it. What did you expect?
Also, it should be __slots__ not __slots (I imagine this is right in your actual code, otherwise you wouldn't be getting the error you're getting).
Why aren't you just using:
class Student(object):
__slots__ = ('name','age')
def set_age(self,age):
self.age = age
where set_age is a method of the Student class rather than adding the function as a method to an instance of the Student class.
Instead of __slots__, I'm using the following method. It allow the use of only a predefined set of parameters:
class A(object):
def __init__(self):
self.__dict__['a']=''
self.__dict__['b']=''
def __getattr__(self,name):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
return d.__dict__[attr]
else:
raise AttributeError
def __setattr__(self,name,value):
d=getattr(self,'__dict__')
if d.keys().__contains__(name):
d[name] = value
else:
raise AttributeError
The use of getattr(..) is to avoid recursion.
There are some merits usin __slots__ vs __dict__ in term of memory and perhaps speed but this is easy to implement and read.
I want to initialize class member with use class method but not know how to call it.
Can you suggest some solution - maybe it is very trivial but I can not find solution?
This code will not work - I do not why?
class X(object):
#staticmethod
def __Y():
return 1
CONSTANT = __Y()
x = X()
print x.CONSTANT
This will work but I need use call method to initialize class members.
class X(object):
CONSTANT = 1
x = X()
print x.CONSTANT
Note, I am not want initialize object variables but class variable.
Drop the #staticmethod decorator and the first approach will work as well. You don't need staticmethod just to call a function inside a class statement.
Since that way the function will not be usable when called from class instances, it is an idiom to also remove it after use. In your example:
class X(object):
def __y():
return 1
CONSTANT = __y()
# ... other uses of __y, if any
del __y
To understand why your approach didn't work, consider what staticmethod does. It wraps a normal function into a descriptor object that, when retrieved from the class, produces the original function unchanged, i.e. without the usual "bound method" semantics. (Retrieving a normal def function from an instance or a class would get you a bound method that automagically inserts self as the first argument.)
However, the descriptor returned by staticmethod is itself not callable, its sole function is to produce the callable object when accessed through the class or instance. Proposals to make the staticmethod descriptor callable were rejected because such use of staticmethod is erroneous in the first place.
Normally a descriptor is used on a class attribute like so:
class Owner(object):
attr = Attr()
When getting Owner.attr, Attr.__get__(self, instance, owner) is called where self = Owner.attr, instance = None and owner = Owner.
When Owner is instantiated instance will be the instance of Owner.
Now I would like to apply this concept to method parameters instead of class attributes.
How it would look in practice (let's assume that the functionality of Attr is to wrap a string with a given string):
class Example(object):
def funct(self, param=Attr('t')):
return param == 'test' # < param calls the descriptor here
e = Example()
e.funct('es') # < is True because 'es' wrapped with 't' becomes 'test'.
When accessing param, Attr.__get__(self, instance, owner) will be called with self = funct.param, instance = funct and owner = funct (although it doesn't make sense to have owner and instance the same, might be None?).
But since funct is not a class, this will not work. How can I get something similar to work?
A decorator on the function will be processing the parameters, so this might add to the solution I think.
The decorator must be, for example, be able to change the wrapper string.
Functions actually are first class objects in Python, but you are correct in saying that the syntax you describe would not work as you want. You could potentially do something like this with a decorator that inspects the passed attributes for characteristics that would enable this sort of functionality though. However, you'd probably be better off implementing a callable object, then attaching descriptors to that and creating instances of the callable rather than functions.
Why are constructors indeed called "Constructors"? What is their purpose and how are they different from methods in a class?
Also, can there be more that one __init__ in a class? I tried the following, can someone please explain the result?
>>> class test:
def __init__(self):
print "init 1"
def __init__(self):
print "init 2"
>>> s=test()
init 2
Finally, is __init__ an operator overloader?
There is no function overloading in Python, meaning that you can't have multiple functions with the same name but different arguments.
In your code example, you're not overloading __init__(). What happens is that the second definition rebinds the name __init__ to the new method, rendering the first method inaccessible.
As to your general question about constructors, Wikipedia is a good starting point. For Python-specific stuff, I highly recommend the Python docs.
Why are constructors indeed called "Constructors" ?
The constructor (named __new__) creates and returns a new instance of the class. So the C.__new__ class method is the constructor for the class C.
The C.__init__ instance method is called on a specific instance, after it is created, to initialise it before being passed back to the caller. So that method is the initialiser for new instances of C.
How are they different from methods in a class?
As stated in the official documentation __init__ is called after the instance is created. Other methods do not receive this treatment.
What is their purpose?
The purpose of the constructor C.__new__ is to define custom behaviour during construction of a new C instance.
The purpose of the initialiser C.__init__ is to define custom initialisation of each instance of C after it is created.
For example Python allows you to do:
class Test(object):
pass
t = Test()
t.x = 10 # here you're building your object t
print t.x
But if you want every instance of Test to have an attribute x equal to 10, you can put that code inside __init__:
class Test(object):
def __init__(self):
self.x = 10
t = Test()
print t.x
Every instance method (a method called on a specific instance of a class) receives the instance as its first argument. That argument is conventionally named self.
Class methods, such as the constructor __new__, instead receive the class as their first argument.
Now, if you want custom values for the x attribute all you have to do is pass that value as argument to __init__:
class Test(object):
def __init__(self, x):
self.x = x
t = Test(10)
print t.x
z = Test(20)
print t.x
I hope this will help you clear some doubts, and since you've already received good answers to the other questions I will stop here :)
Classes are simply blueprints to create objects from. The constructor is some code that are run every time you create an object. Therefor it does'nt make sense to have two constructors. What happens is that the second over write the first.
What you typically use them for is create variables for that object like this:
>>> class testing:
... def __init__(self, init_value):
... self.some_value = init_value
So what you could do then is to create an object from this class like this:
>>> testobject = testing(5)
The testobject will then have an object called some_value that in this sample will be 5.
>>> testobject.some_value
5
But you don't need to set a value for each object like i did in my sample. You can also do like this:
>>> class testing:
... def __init__(self):
... self.some_value = 5
then the value of some_value will be 5 and you don't have to set it when you create the object.
>>> testobject = testing()
>>> testobject.some_value
5
the >>> and ... in my sample is not what you write. It's how it would look in pyshell...
coonstructors are called automatically when you create a new object, thereby "constructing" the object. The reason you can have more than one init is because names are just references in python, and you are allowed to change what each variable references whenever you want (hence dynamic typing)
def func(): #now func refers to an empty funcion
pass
...
func=5 #now func refers to the number 5
def func():
print "something" #now func refers to a different function
in your class definition, it just keeps the later one
There is no notion of method overloading in Python. But you can achieve a similar effect by specifying optional and keyword arguments