TL;DR: When decorating a class with #numba.jitclass special methods such as __add__ do not appear in instances of the class, while other methods work normally. Why does this happen?
Consider the following class declaration:
import numba as nb
dual_spec = [('x', nb.float64), ('y', nb.float64)]
#nb.jitclass(dual_spec)
class xy:
def __init__(self, x, y):
self.x = x
self.y = y
def addition(self, other):
return xy(self.x + other.x, self.y + other.y)
def __add__(self, other):
return xy(self.x + other.x, self.y + other.y)
Without the decorator the class works perfectly fine. Due to the __add__ method expressions like xy(1, 2) + xy(3, 4) are possible and return expected results. However, with the decorator I get the following error message:
>>> xy(1, 2) + xy(3, 4) # TypeError: unsupported operand type(s) for +: 'xy' and 'xy'
>>> xy(1, 2).addition(xy(3, 4)) # But this works nicely
It looks like the __add__ method is not present in xy objects:
>>> xy(1, 2).__add__ # AttributeError: 'xy' object has no attribute '__add__'
But the method is present in the class:
>>> xy.__add__ # <function __main__.xy.__add__>
What is numba doing to the __add__ method during instantiation? Is there another way to enable operators for jitted classes so that I can write xy(1, 2) + xy(3, 4)?
Currently (as of numba version 0.33) operator overloading on jitclasses is not supported, open issue here:
https://github.com/numba/numba/issues/1606#issuecomment-284552746
I don't know the exact internals, but it is likely the method is simply being discarded. Note that when you instantiate at a jitclass you are not directly instantiating the python class but instead are getting a wrapper around the low-level numba type.
v = xy(1, 2)
v
Out[8]: <numba.jitclass.boxing.xy at 0x2e700274950>
v._numba_type_
Out[9]: instance.jitclass.xy#2e77d394438<x:float64,y:float64>
Related
I have a vector class:
class Vector:
def __init__(self, x, y):
self.x, self.y = x, y
def __str__(self):
return '(%s,%s)' % (self.x, self.y)
def __add__(self, n):
if isinstance(n, (int, long, float)):
return Vector(self.x+n, self.y+n)
elif isinstance(n, Vector):
return Vector(self.x+n.x, self.y+n.y)
which works fine, i.e. I can write:
a = Vector(1,2)
print(a + 1) # prints (2,3)
However if the order of operation is reversed, then it fails:
a = Vector(1,2)
print(1 + a) # raises TypeError: unsupported operand type(s)
# for +: 'int' and 'instance'
I understand the error: the addition of an int object to an Vector object is undefined because I haven't defined it in the int class. Is there a way to work around this without defining it in the int (or parent of int) class?
You need to also define __radd__
Some operations do not necessarily evaluate like this a + b == b + a and that's why Python defines the add and radd methods.
Explaining myself better: it supports the fact that "int" does not define a + operation with class Vector instances as part of the operation. Therefore vector + 1 is not the same as 1 + vector.
When Python tries to see what the 1.__add__ method can do, an exception is raised. And Python goes and looks for Vector.__radd__ operation to try to complete it.
In the OP's case the evaluation is true and suffices with __radd__ = __add__
class Vector(object):
def __init__(self, x, y):
self.x, self.y = x, y
def __str__(self):
return '(%s,%s)' % (self.x, self.y)
def __add__(self, n):
if isinstance(n, (int, long, float)):
return Vector(self.x+n, self.y+n)
elif isinstance(n, Vector):
return Vector(self.x+n.x, self.y+n.y)
__radd__ = __add__
a = Vector(1, 2)
print(1 + a)
Which outputs:
(2,3)
The same applies to all number-like operations.
When you say x + y, Python calls x.__add__(y). If x does not implement __add__ (or that method returns NotImplemented), Python tries to call y.__radd__(x) as a fallback.
Thus all you have to do is to define the __radd__() method in your Vector class and 1 + y will work as you would expect.
Note: you would have to do similar for other operations too, e.g. implement __mul__() and __rmul__() pair, etc.
You might also want to look at this question, it explains the same principle in more details.
Update:
Depending on your use case, you might also want to implement the __iadd__() method (and its cousins) to override the += operator.
For example, if you say y += 1 (y being an instance of Vector here), you might want to modify the y instance itself, and not return a new Vector instance as a result, which what your __add__() method currently does.
Consider the following example of a 'wrapper' class to represent vectors:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __str__(self):
return str(self._vals)
The __add__ method takes care of adding two vectors as well as adding a vector with a scalar. However, the second case is not complete as the following examples show:
>>> a = Vector([1.2, 3, 4])
>>> print(a)
[1.2, 3, 4]
>>> print(a+a)
[2.4, 6, 8]
>>> print(a+5)
[6.2, 8, 9]
>>> print(5+a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'Vector'
To my understanding the reason is that the overloaded operator only tells Python what to do when it sees a + x where a is an instance of Vector, but there is no indication of what to do for x + a (with a an instance of Vector and x a scalar).
How one should overload the operators in such circumstances to cover all cases (i.e., to support the case that self is not an instance of Vector but other is)?
Ok. I guess I found the answer: one has to overload __radd__ operator as well:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __radd__(self, other):
return self + other
def __str__(self):
return str(self._vals)
Although to me this looks a bit redundant. (Why Python does not use the commutativity of addition by default, assuming __radd__(self, other) always returns self + other? Of course for special cases the user can override __radd__.)
You could define a Scalar class that has int as its base class.
Then override __add__ to do what you want.
class Scalar(int):
def __add__(self):
# do stuff
You already figured out you need to implement __radd__. This is an answer as to why this is so, and why you need to do this in addition to implementing __add__, as a Both quotes are taken from Python Docs (Data Model - 3.3.8 Emulating numeric types), starting with the obvious:
These methods are called to implement the binary arithmetic operations (+, -, *, #, /, //, %, divmod(), pow(), **, <<, >>, &, ^, |). For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called.
So order determines which object's implementation of __add__ is called. When the method doesn't support the operation with the passed argument NotImplemented should be returned. That's where the so-called "reflected methods" come into play:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types. For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented [sic].
Now, why wouldn't __radd__(self, other) just fall back to __add__(self, other)? While ring addition is always commutative (see this and this math.stackexchange answers), you could have algebraic structures that are do not satisfy this assumption (e.g., near-rings). But my guess as a non-mathematician would be that it's just desirable to have a consistent data model across different numerical methods. While addition might be commonly commutative, multiplication is less so. (Think matrices and vectors! Although, admittedly this is not the best example, given __matmul__). I also prefer to see there being no exceptions, especially if I had to read about rings, etc. in a language documentation.
I'm creating my own complex number class (not using Python's built in one) and I'm running into a problem when I try to add zero to my complex number. For reference this is the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "display.py", line 25, in __init__
t.color(mandelbrot(c).getColor())
File "C:\Users\Joe\Desktop\Python\mandelbrot.py", line 10, in __init__
z = z*z + self.__starting_value
TypeError: unsupported operand type(s) for +: 'int' and 'complex'
Where self.__starting_value is the complex number.
The addition definition goes as follows:
class complex:
def __init__(self, a = 0, b = 0):
self.__real = float(a)
self.__imag = float(b)
def __add__(self, other):
return complex(self.__real + other.__real, self.__imag + other.__imag)
The solution should be simple enough but I am still learning Python and could use the help.
Two problems:
You are only handling addition of a complex object to a complex object, when you want to also be able to handle complex object + int.
When adding in this order (int + custom type), you need to add a reflected addition method (__radd__)
class complex:
def __init__(self, a = 0, b = 0):
self.__real = float(a)
self.__imag = float(b)
def __add__(self, other):
if isinstance(other, complex):
return complex(self.__real + other.__real, self.__imag + other.__imag)
else:
return complex(self.__real + other, self.__imag)
def __radd__(self, other):
return self + other
N.B.: It is considered bad style to shadow built-in names (like complex).
int + complex will first try to useint.__add__. complex + int will first try to use complex.__add__.
You need to implement complex.__radd__. See the related note in Emulating numeric types:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types.
You will need to handle the case where other is int in both complex.__add__ and complex.__radd__.
__add__ is for addition when the left-hand-side operand is of the type you're writing. When Python fails to call the int __add__ method, it tries __radd__. If you define __radd__ (with the same behavior), you'll get the result you want.
I am trying to understand how operator overriding works for two operands of a custom class.
For instance, suppose I have the following:
class Adder:
def __init__(self, value=1):
self.data = value
def __add__(self,other):
print('using __add__()')
return self.data + other
def __radd__(self,other):
print('using __radd__()')
return other + self.data
I initialize the following variables:
x = Adder(5)
y = Adder(4)
And then proceed to do the following operations:
1 + x
using __radd__()
Out[108]: 6
x + 2
using __add__()
Out[109]: 7
The two operations above seem straigtforward. If a member of my custom class is to the right of the "+" in the addition expression, then __radd__ is used. If it is on the left, then __add__ is used. This works for expressions when one operand is of the Adder type and another one is something else.
When I do this, however, I get the following result:
x + y
using __add__()
using __radd__()
Out[110]: 9
As you can see, if both operands are of the custom class, then both __add__ and __radd__ are called.
My question is how does Python unravel this situation and how is it able to call both the right-hand-addition function, as well as the left-hand-addition function.
It's because inside your methods you add the data to other. This is itself an instance of Adder. So the logic goes:
call __add__ on x;
add x.data (an int) to y (an Adder instance)
ah, right-hand operand is an instance with a __radd__ method, so
call __radd__ on y;
add int to y.data (another int).
Usually you would check to see if other was an instance of your class, and if so add other.data rather than just other.
That's the because the implementation of your __add__ and __radd__ method do not give any special treatment to the instances of the Adder class. Therefore, each __add__ call leads to an integer plus Adder instance operation which further requires __radd__ due to the Adder instance on the right side.
You can resolve this by doing:
def __add__(self, other):
print('using __add__()')
if isinstance(other, Adder):
other = other.data
return self.data + other
def __radd__(self, other):
print('using __radd__()')
return self.__add__(other)
I have a class Vec3D (see http://pastebin.com/9Y7YbCZq)
Currently, I allow Vec3D(1,0,0) + 1.2 but I'm wondering how I should proceed to overload the + operator in such a way that I get the following output:
>>> 3.3 + Vec3D(1,0,0)
[4.3, 3.3 , 3.3]
Code is not required, but just a hint in which direction I should look. Something general will be more useful than a specific implementation as I need to implement the same thing for multiplication, subtraction etc.
You're looking for __radd__:
class MyClass(object):
def __init__(self, value):
self.value = value
def __radd__(self, other):
print other, "radd", self.value
return self.value + other
my = MyClass(1)
print 1 + my
# 1 radd 1
# 2
If the object on the left of the addition doesn't support adding the object on the right, the object on the right is checked for the __radd__ magic method.
You want to use the __add__ (and possibly __radd__ and __iadd__) methods. Check out http://docs.python.org/reference/datamodel.html#object.__add__ for more details.
implement __radd__ . When you call 3.3 + Vec3D(1,0,0), as long as float doesn't have method __add__(y) with y being Vec3D, your reflected version __radd__ will be called.