I want to overload the operator # in python for a class I have written. I know how to do operator overloading in general (i.e. by defining __add__ and __radd__ to overload +) but I could not find a way to overload #.
Why I know, that # can be overloaded: for numpy arrays, A#B gives the matrix product of A and B, while A*B gives the Hadamard (element-wise) product.
The methods you need to overload are the __matmul__ and __rmatmul__ methods. E.g. if you want to add the inner product to lists:
class Vector(list):
def __matmul__(self, other):
return sum(x * y for x, y in zip(self, other))
def __rmatmul__(self, other):
return self.__matmul__(other)
Related
Consider the following example of a 'wrapper' class to represent vectors:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __str__(self):
return str(self._vals)
The __add__ method takes care of adding two vectors as well as adding a vector with a scalar. However, the second case is not complete as the following examples show:
>>> a = Vector([1.2, 3, 4])
>>> print(a)
[1.2, 3, 4]
>>> print(a+a)
[2.4, 6, 8]
>>> print(a+5)
[6.2, 8, 9]
>>> print(5+a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'Vector'
To my understanding the reason is that the overloaded operator only tells Python what to do when it sees a + x where a is an instance of Vector, but there is no indication of what to do for x + a (with a an instance of Vector and x a scalar).
How one should overload the operators in such circumstances to cover all cases (i.e., to support the case that self is not an instance of Vector but other is)?
Ok. I guess I found the answer: one has to overload __radd__ operator as well:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __radd__(self, other):
return self + other
def __str__(self):
return str(self._vals)
Although to me this looks a bit redundant. (Why Python does not use the commutativity of addition by default, assuming __radd__(self, other) always returns self + other? Of course for special cases the user can override __radd__.)
You could define a Scalar class that has int as its base class.
Then override __add__ to do what you want.
class Scalar(int):
def __add__(self):
# do stuff
You already figured out you need to implement __radd__. This is an answer as to why this is so, and why you need to do this in addition to implementing __add__, as a Both quotes are taken from Python Docs (Data Model - 3.3.8 Emulating numeric types), starting with the obvious:
These methods are called to implement the binary arithmetic operations (+, -, *, #, /, //, %, divmod(), pow(), **, <<, >>, &, ^, |). For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called.
So order determines which object's implementation of __add__ is called. When the method doesn't support the operation with the passed argument NotImplemented should be returned. That's where the so-called "reflected methods" come into play:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types. For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented [sic].
Now, why wouldn't __radd__(self, other) just fall back to __add__(self, other)? While ring addition is always commutative (see this and this math.stackexchange answers), you could have algebraic structures that are do not satisfy this assumption (e.g., near-rings). But my guess as a non-mathematician would be that it's just desirable to have a consistent data model across different numerical methods. While addition might be commonly commutative, multiplication is less so. (Think matrices and vectors! Although, admittedly this is not the best example, given __matmul__). I also prefer to see there being no exceptions, especially if I had to read about rings, etc. in a language documentation.
This question already has an answer here:
Python commutative operator override
(1 answer)
Closed 5 years ago.
I have a simple class that helps with mathematical operations on vectors (i.e. lists of numbers). My Vector can be multiplied by other instances of Vector or a scalar (float or int).
In other, more strongly typed, languages I would create a method to multiply two vectors and a separate method to multiply a vector by and int/float. I'm still pretty new to Python and am not sure how I would implement this. The only way I can think of doing it is override __mul__() and test the incoming parameter:
class Vector(object):
...
def __mul__(self, rhs):
if isinstance(rhs, Vector):
...
if isinstance(rhs, int) or isinstance(rhs, float):
...
Even if I do it that way I would be forced to multiply a Vector by a scalar like this:
v = Vector([1,2,3])
result = v * 7
What if I wanted to reverse the order of the operands in the multiplication?
result = 7 * v
What is the right way to do that in Python?
You also need to implement __rmul__. When the initial call to int.__mul__(7, v) fails, Python will next try type(v).__rmul__(v, 7).
def __rmul__(self, lhs):
return self * lhs # Effectively, turn 7 * v into v * 7
As Rawing points out, you could simply write __rmul__ = __mul__ for this definition. __rmul__ exists to allow for non-commutative multiplication where simply deferring to __mul__ with the operands reversed isn't sufficient.
For instance, if you were writing a Matrix class and wanted to support multiplication by a nested list, e.g.,
m = Matrix(...) # Some 2 x 2 matrix
n = [[1, 2], [3,4]]
p = n * m
Here, the list class wouldn't know how to multiple a list by a Matrix instance, so when list.__mul__(n, m) fails, Python would next try Matrix.__rmul__(m, n). However, n * m and m * n are two different results in general, so Matrix.__rmul__(m, n) != Matrix.__mul__(m, n); __rmul__ has to do a little extra work to generate the right answer.
There are special methods for reversed operations:
__rmul__ for the reverse of __mul__
and __radd__ for __add__,
...
These are called when the left hand side operator returns NotImplemented for the normal operation (so the operation 2 + vector_instance will first try: (2).__add__(vector_instance) but if this returns NotImplemented then vector_instance.__radd__(2) is called).
However I wouldn't use isinstance checks in the arithmetic special methods, that will lead to a lot of code repetition.
You could actually create a special case in __init__ and implement a conversion from scalars to a Vector there:
class Vector(object):
def __init__(self, x, y=None, z=None):
if y is None and z is None:
if isinstance(x, Vector):
self.x, self.y, self.z = x.x, x.y, x.z
else:
self.x, self.y, self.z = x, x, x
elif y is None or z is None:
raise ValueError('Either x, y and z must be given or only x')
else:
self.x, self.y, self.z = x, y, z
def __mul__(self, other):
other = Vector(other)
return Vector(self.x*other.x, self.y*other.y, self.z*other.z)
__rmul__ = __mul__ # commutative operation
def __sub__(self, other):
other = Vector(other)
return Vector(self.x-other.x, self.y-other.y, self.z-other.z)
def __rsub__(self, other): # not commutative operation
other = Vector(other)
return other - self
def __repr__(self):
return 'Vector({self.x}, {self.y}, {self.z})'.format(self=self)
This should work as expected:
>>> 2 - Vector(1, 2, 3)
Vector(1, 0, -1)
>>> Vector(1, 2, 3) - 2
Vector(-1, 0, 1)
>>> Vector(1, 2, 3) * 2
Vector(2, 4, 6)
>>> 2 * Vector(1, 2, 3)
Vector(2, 4, 6)
Note that this was a quick and dirty draft (that could have several bugs). I just wanted to present the "general idea" how it could be solved without special casing the type in each arithmetic operation.
I have a class called MyData that has __mul__ and __rmul__ defined (along with all the other arithmetic operators). Whenever these methods are used, it should always return a value of type MyData. However, I discovered that a * myDataObj is not the same as myDataObj * a, depending on the type of a. Specifically, if a was an int, it worked fine, but if a was a float then the first configuration return an array (my object has a numpy array as a member, and MyData.__getitem__ returns slices of that array) and the second configuration returns the proper value of type MyData.
Is there any way to determine the calling order of the operator in an expression like this?
Is there any way to determine the calling order of the operator in an expression like this?
First, the exact rules are described in the Data model section of the language reference, specifically the "Emulating numeric types" subsection.
The __rfoo__ methods are described as follows:
These methods are called to implement the binary arithmetic operations (+, -, *, /, %, divmod(), pow(), **, <<, >>, &, ^, |) with reflected (swapped) operands. These functions are only called if the left operand does not support the corresponding operation and the operands are of different types. [2] For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented.
Note that ternary pow() will not try calling __rpow__() (the coercion rules would become too complicated).
Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.
Putting this into Pythonesque pseudocode, x * y is evaluated something like this:
if type(y) is type(x):
return x.__mul__(y)
elif type(y) is a subclass of type(x):
try y.__rmul__(x)
otherwise x.__mul__(y)
else:
try x.__mul__(y)
otherwise y.__rmul__(x)
Of course you can also determine the calling order dynamically by creating separate types whose methods just print their names and testing them:
class Base(object):
def __mul__(self, lhs): print('Base.mul')
def __rmul__(self, rhs): print('Base.rmul')
class Derived(Base):
def __mul__(self, lhs): print('Derived.mul')
def __rmul__(self, rhs): print('Derived.rmul')
class Unrelated(object):
def __mul__(self, lhs): print('Unrelated.mul')
def __rmul__(self, rhs): print('Unrelated.rmul')
print('Base * Base: ', end='')
Base() * Base()
for x, y in itertools.permutations((Base, Derived, Unrelated), 2):
print('{} * {}: '.format(x.__name__, y.__name__), end='')
x() * y()
What about with built in types as well?
Exactly the same rules. Since Base is not a subclass of either int or float, and neither int nor float knows how to multiply by it, they'll both call Base.__rmul__. And so will any other unrelated type you throw at it:
>>> Base() * 2
Base.mul
>>> 2 * Base()
Base.rmul
>>> Base() * 2.5
Base.mul
>>> 2.5 * Base()
Base.rmul
>>> 'sdfsdfsdfds' * Base()
Base.rmul
>>> (lambda: 23) * Base()
Base.rmul
My problem is that I'm getting different results from 1.5 * myObj and myObj * 1.5
There are a number of reasons for that:
Your __mul__ and __rmul__ code don't do the same thing.
You inherited from float.
You inherited from some builtin or extension type that handles float multiplication at the C-API level and isn't designed to allow overrides in subclasses.
You created a classic class instead of a new-style class.
You made a typo in one of the names.
…
I have a vector class and I defined the __mul__ method to multiply a vector by a number.
Here is the __mul__ method :
def __mul__(self, other):
x = self.x * other
y = self.y * other
new = Vector()
new.set_pos((x, y))
return new
My problem is that I don't know which is which between the number and the vector.
If self is the number, self.x raises an error. (I'm maybe mistaking on this point : Is "other" always a number ?)
So I found here : Python: multiplication override
that I could do :
__rmul__ = __mul__
but how can I do that in a class definition ?
Something like :
def __rmul__ = __mul__
self will never be the number in __mul__() because the object the method is attached to is not the number, it's the vector, and by definition it's the multiplicand.
other will be a number if your object is being multiplied by a number. Or it could be something else, such as another vector, which you could test for and handle.
When your object is the multiplier, __rmul__() is called if the multiplicand doesn't know how to handle the operation.
To handle the case in which __mul__ and __rmul__ should be the same method, because the operation is commutative, you can just do the assignment in your class definition.
class Vector(object):
def __mul__(self, other):
pass
__rmul__ = __mul__
Simply list it as an attribute:
__rmul__ = __mul__
This is the same way you'd create an alias of a function in a module; creating an alias of a method within a class body works the same.
The point is that in Python, you can tell objects how to multiply themselves by things. That means that
a * b
could either mean "tell a to multiply itself by b" or "tell b to multiply itself by a". In code, that translates to
a.__mul__(b)
or
b.__rmul__(a)
I'd like to combine a list of class instances of a class for which the __add__ method is defined.
i.e., I have a list of class instances L=[A,B,C,D] and I want their sum E = A+B+C+D, but generalized so that instead of the + syntax I could do something like E = sum(L).
What function should I use to do that? Is the __add__ method adequate, or do I need to define a different class method (e.g. __iadd__) in order to accomplish this?
(if this turns out to be a duplicate, how should I be asking the question?)
import operator
reduce(operator.add, L)
sum may want to add numerical values to instances of your class. Define __radd__ so for example int + Foo(1) will be defined:
class Foo(object):
def __init__(self, val):
self.val = val
def __add__(self, other):
return self.val + other.val
def __radd__(self, other):
return other + self.val
A = Foo(1)
B = Foo(2)
L = [A,B]
print(A+B)
# 3
print(sum(L))
# 3
Ignore my previous answer, it was wrong.
The reduce function allows you to apply any binary function or method to all the elements of a sequence. So, you could write:
reduce(YourClass.__add__, sequence)
If not all objects in the sequence are instances of the same class, then instead use this:
import operator
reduce(operator.add, sequence)
Or this:
reduce(lambda x, y: x + y, sequence)