I know I should have come up with a better title, but anyway...
Say I make a class inherited from int in python:
class Foo(int):
def is_even(self):
return self%2 == 0
and do something like this
a = Foo(3)
b = Foo(5)
print(type(a+b)) #=> <class 'int'>
I understand this behaviour is not surprising at all, as __add___ called here is defined to return int instances. But I would like to create a class so that a+b returns Foo(8). In other words, I'd like the result a+b to have the is_even method.
Is there any way I can achieve this conveniently? Or do I have to overwrite __add__ and everything?
Background information: I'm trying to write an interpreter for an esoteric programming language called Grass . In that attempt, I want to have a class that behaves like 'callable-int' (actually, numpy.uint8), whose __call__ would be like
def __call__(self, other):
if self == other:
return lambda x: lambda y: x
else:
return lambda x: lambda y: y
.
There are tricks that you could do with metaclasses (__metaclass__ class variable) or the __getattribute__ special method. But the documentation states:
Bypassing the getattribute() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter)
Which means that if you want to make sure that the parent class is never handled directly, you need to intercept everything. And for int, that is described as emulating numeric types (i.e.: implementing all those methods).
That said, I believe you could implement all those methods in your class quite easily by creating a lambda or generic method that takes two parameters and just calls super on them. And then assign that method to all the specific methods that you need to implement. So you implement once and reuse it.
Related
The following code is of course totally pointless; it's not supposed to
do anything but illustrate what I'm confused about:
class func():
def __call__(self, x):
raise Exception("func.__call__ error")
def double(x):
return 2*x
doubler = func()
doubler.__call__ = double
print doubler(2)
Can someone explain why this works? I would have expected that if I
wanted to set doubler.__call__ to something it would be a function
that takes two variables; I'd expect the code above to raise some sort
of too-many-parameters error. What gets passed to what, when?
(And then: How could I set doubler.__call__ to a function that
will actually have access to both "self" and "x"?)
(Context: An admittedly silly of-academic-interest example of why I might want to set an instance method this way: Each computable instance needs its own Approx method; creating a separate subclass for each instance seems "wrong"...)
Edit. Probably a better example, making it clear it has nothing
to do with magic-method magic:
class func():
def call(self, x):
raise Exception("func.call error")
def double(x):
return 2*x
doubler = func()
doubler.call = double
print doubler.call(2)
On third thought, probably the following is the right way to do it.
(i) Seems cleaner somehow, using the Python object model instead of
tinkering with it (ii) even 24 hours ago with my then much cruder
understanding I would have expected it to work; somehow in this
version it simply seems to make sense to me that the function passed
to the constructor should take only one variable (iii) it seems to
work regardless of whether I inherit from object, which I think means it would also work in 3.0.
class func3(object):
def __init__(self, f):
self.f = f
def __call__(self, x):
return self.f(x)
def double(x):
return 2.0*x
f3=func3(double)
print f3(2)
When you assign to doubler.__call__, you're binding an function to an instance attribute. This hides the class attribute of the same name that was created in the class statement.
Python's method binding only kicks in when you are looking up a class attribute via an instance. If the attribute's value is a descriptor (which functions are), then the descriptor's __get__ method gets called with appropriate parameters. For a function object, that binds the method to the instance (so self gets passed in automatically as the first argument).
Your first example wouldn't actually work in Python 3, only in Python 2. That's because in Python 2 you're creating an "old-style" class, which does all its method lookups on the instance. In new-style classes (which you can get in Python 2 by inheriting from object, or by default in Python 3), __special__ methods, when they're invoked by the interpreter (e.g. when you do doubler(2) to run doubler.__call__) are looked up only in the class, not in the instance's attributes. So your first example won't work with a new-style class, but the version that uses a normal method (call instead of __call__) would be fine.
This is something between an answer to the question and a continuation of the question. I was kindly referred to another thread where more or less the same question was answered. I didn't follow the answers in that thread very well, being ignorant of the things the people there are talking about, hence the Question: Is what I say below correct? (If yes then this is an answer to the question above; if no I'd appreciate someone explaining why not...)
(i) Since I assign a function to an instance of func instead of to the class, it is now an "instance method", as opposed to a "class method".
(ii) And that's why it's not passed the instance as the first parameter; that happens with class methods but not with instance methods...
If I want to change the behavior of an inherited method, I would do something like this:
class a:
def changeMe(self):
print('called a')
class b(a):
def changeMe(self):
print('called b')
I believe this is an example of overriding in Python.
However, if I want to overload an operator, I do something very similar:
class c:
aNumber = 0
def __add__(self, operand):
print("words")
return self.aNumber + operand.aNumber
a = c()
b = c()
a.aNumber += 1
b.aNumber += 2
print(a + b) # prints "words\n3"
I thought that maybe the operator methods are really overridden in python since we overload using optional parameters and we just call it operator overloading out of convention.
But it also couldn't be an override, since '__add__' in object.__dict__.keys() is False; a method needs to be a member of the parent class in order to be overridden (and all classes inherit from object when created).
Where is the gap in my understanding?
I guess since the original question specifically asked about the gap in my own understanding, I am best-positioned to answer it. Go figure.
What I failed to understand was that whereas overriding depends on inheritance, overloading does not. Rather, Python matches methods for overloading based on name only.
For a subclass to override a method, the method does indeed need to exist in the parent class. Therefore, the def __add__ portion is not an example of overriding.
(In this case, I also did not fully understand that if the interpreter sees a + operator, it will look to the class of the operands for a definition of the __add__ magic method.)
Because the + operator is essentially an alias for __add__(), the same name is being used. Operator overloading is in fact an example of overloading because we are changing the behavior of the name (+ or __add__) when it is called with novel parameters (in my example, objects of class c).
Overloading means 2 methods with the SAME Name and different signatures + return types. Overriding means 2 methods with the SAME name, wherein the sub method has different functionality.The main difference between overloading and overriding is that in overloading we can use same function name with different parameters for multiple times for different tasks with on a class. and overriding means we can use same name function name with same parameters of the base class in the derived class. this is also called as re usability of code in the program.
Was just thinking about Python's dict "function" and starting to realize that dict isn't really a function at all. For example, if we do dir(dict), we get all sorts of methods that aren't include in the usual namespace of an user defined function. Extending that thought, its similar to dir(list) and dir(len). They aren't function, but really types. But then I'm confused about the documentation page, http://docs.python.org/2/library/functions.html, which clearly says functions. (I guess it should really just says builtin callables)
So what gives? (Starting to seem that making the distinction of classes and functions is trivial)
It's a callable, as are classes in general. Calling dict() is effectively to call the dict constructor. It is like when you define your own class (C, say) and you call C() to instantiate it.
One way that dict is special, compared to, say, sum, is that though both are callable, and both are implemented in C (in cpython, anyway), dict is a type; that is, isinstance(dict, type) == True. This means that you can use dict as the base class for other types, you can write:
class MyDictSubclass(dict):
pass
but not
class MySumSubclass(sum):
pass
This can be useful to make classes that behave almost like a builtin object, but with some enhancements. For instance, you can define a subclass of tuple that implements + as vector addition instead of concatenation:
class Vector(tuple):
def __add__(self, other):
return Vector(x + y for x, y in zip(self, other))
Which brings up another interesting point. type is also implemented in C. It's also callable. Like dict (and unlike sum) it's an instance of type; isinstance(type, type) == True. Because of this weird, seemingly impossible cycle, type can be used to make new classes of classes, (called metaclasses). You can write:
class MyTypeSubclass(type):
pass
class MyClass(object):
__metaclass__ = MyTypeSubclass
or, in Python 3:
class MyClass(metaclass=MyTypeSubclass):
pass
Which give the interesting result that isinstance(MyClass, MyTypeSubclass) == True. How this is useful is a bit beyond the scope of this answer, though.
dict() is a constructor for a dict instance. When you do dir(dict) you're looking at the attributes of class dict. When you write a = dict() you're setting a to a new instance of type dict.
I'm assuming here that dict() is what you're referring to as the "dict function". Or are you calling an indexed instance of dict, e.g. a['my_key'] a function?
Note that calling dir on the constructor dict.__init__
dir(dict.__init__)
gives you what you would expect, including the same stuff as you'd get for any other function. Since a call to the dict() constructor results in a call to dict.__init__(instance), that explains where those function attributes went. (Of course there's a little extra behind-the-scenes work in any constructor, but that's the same for dicts as for any object.)
As the Python 2 documentation on __repr__ states:
If at all possible, this (i.e. __repr__) should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment).
So how come builtin __repr__ for classes does not act accordingly to that guideline?
Example
>>> class A(object):
... pass
>>> repr(A)
"<class 'A'>"
To meet the guideline, the default __repr__ should return "A", i.e. generally A.__name__. Why is it acting differently? It would be extra-easy to implement, I believe.
Edit: The scope of 'reproduction'
I can see in the answers that it is not clear in the discussion what repr should return. The way I see it, the repr function should return a string that allows you to reproduce the object:
in an arbitrary context and
automatically (i.e. not manually).
Ad.1. Take a look at a built-in class case (taken from this SO question):
>>> from datetime import date
>>>
>>> repr(date.today()) # calls date.today().__repr__()
'datetime.date(2009, 1, 16)'
Apparently, the assumed context is as if you use the basic form of import, i.e. import datetime, because if you would try eval(repr(date.today())), datetime would not be recognized. So the point is that __repr__ doesn't need to represent the object from scratch. It's enough if it is unambiguous in a context the community agreed upon, e.g. using direct module's types and functions. Sounds reasonable, right?
Ad.2. Giving an impression of how the object could be reconstructed is not enough for repr, I believe. Helpfulness in debugging is the purpose of str.
Conclusion
So what I expect from repr is allowing me to do eval on the result. And in the case of a class, I would not like to get the whole code that would reconstruct the class from scratch. Instead, I would like to have an unambiguous reference to a class visible in my scope. The "Module.Class" would suffice. No offence, Python, but "<class 'Module.Class'>" doesn't just cut it.
Consider a slightly more complicated class:
class B(object):
def __init__(self):
self.foo=3
repr would need to return something like
type("B", (object,), { "__init__": lambda self: setattr(self, "foo", 3) })
Notice one difficulty already: not all functions defined by the def statement can be translated into a single lambda expression. Change B slightly:
class B(object):
def __init__(self, x=2, y, **kwargs):
print "in B.__init__"
How do you write an expression that defines B.__init__? You can't use
lambda self: print "in B.__init__"
because lambda expressions cannot contain statements. For this simple class, it is already impossible to write a single expression that defines the class completely.
Because the default __repr__ cannot know what statements were used to create the class.
The documentation you quote starts with If at all possible. Since it is not possible to represent custom classes in a way that lets you recreate them, a different format is used, which follows the default for all things not easily recreated.
If repr(A) were to just return 'A', that'd be meaningless. You are not recreating A, you'd just be referencing it then. "type('A', (object,), {})" would be closer to reflecting the class constructor, but that'd be a) confusing for people not familiar with the fact python classes are instances of type and b) never able to reflect methods and attributes accurately.
Compare the output to that of repr(type) or repr(int) instead, these follow the same pattern.
I know this is an older question, but I found a way to do it.
The only way I know to do this is with a metaclass like so:
class A(object):
secret = 'a'
class _metaA(type):
#classmethod
def __repr__(cls):
return "<Repr for A: secret:{}>".format(A.secret)
__metaclass__ =_metaA
outputs:
>>> A
<Repr for A: secret:a>
Since neither "<class 'A'>" nor "A" can be used to re-create the class when its definition is not available, I think the question is moot.
I've been hacking classes in Python like this:
def hack(f,aClass) :
class MyClass(aClass) :
def f(self) :
f()
return MyClass
A = hack(afunc,A)
Which looks pretty clean to me. It takes a class, A, creates a new class derived from it that has an extra method, calling f, and then reassigns the new class to A.
How does this differ from metaclass hacking in Python? What are the advantages of using a metaclass over this?
The definition of a class in Python is an instance of type (or an instance of a subclass of type). In other words, the class definition itself is an object. With metaclasses, you have the ability to control the type instance that becomes the class definition.
When a metaclass is invoked, you have the ability to completely re-write the class definition. You have access to all the proposed attributes of the class, its ancestors, etc. More than just injecting a method or removing a method, you can radically alter the inheritance tree, the type, and pretty much any other aspect. You can also chain metaclasses together for a very dynamic and totally convoluted experience.
I suppose the real benefit, though is that the class's type remains the class's type. In your example, typing:
a_inst = A()
type(a_inst)
will show that it is an instance of MyClass. Yes, isinstance(a_inst, aClass) would return True, but you've introduced a subclass, rather than a dynamically re-defined class. The distinction there is probably the key.
As rjh points out, the anonymous inner class also has performance and extensibility implications. A metaclass is processed only once, and the moment that the class is defined, and never again. Users of your API can also extend your metaclass because it is not enclosed within a function, so you gain a certain degree of extensibility.
This slightly old article actually has a good explanation that compares exactly the "function decoration" approach you used in the example with metaclasses, and shows the history of the Python metaclass evolution in that context: http://www.ibm.com/developerworks/linux/library/l-pymeta.html
You can use the type callable as well.
def hack(f, aClass):
newfunc = lambda self: f()
return type('MyClass', (aClass,), {'f': newfunc})
I find using type the easiest way to get into the metaclass world.
A metaclass is the class of a class. IMO, the bloke here covered it quite serviceably, including some use-cases. See Stack Overflow question "MetaClass", "new", "cls" and "super" - what is the mechanism exactly?.