I have a class called MyData that has __mul__ and __rmul__ defined (along with all the other arithmetic operators). Whenever these methods are used, it should always return a value of type MyData. However, I discovered that a * myDataObj is not the same as myDataObj * a, depending on the type of a. Specifically, if a was an int, it worked fine, but if a was a float then the first configuration return an array (my object has a numpy array as a member, and MyData.__getitem__ returns slices of that array) and the second configuration returns the proper value of type MyData.
Is there any way to determine the calling order of the operator in an expression like this?
Is there any way to determine the calling order of the operator in an expression like this?
First, the exact rules are described in the Data model section of the language reference, specifically the "Emulating numeric types" subsection.
The __rfoo__ methods are described as follows:
These methods are called to implement the binary arithmetic operations (+, -, *, /, %, divmod(), pow(), **, <<, >>, &, ^, |) with reflected (swapped) operands. These functions are only called if the left operand does not support the corresponding operation and the operands are of different types. [2] For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented.
Note that ternary pow() will not try calling __rpow__() (the coercion rules would become too complicated).
Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.
Putting this into Pythonesque pseudocode, x * y is evaluated something like this:
if type(y) is type(x):
return x.__mul__(y)
elif type(y) is a subclass of type(x):
try y.__rmul__(x)
otherwise x.__mul__(y)
else:
try x.__mul__(y)
otherwise y.__rmul__(x)
Of course you can also determine the calling order dynamically by creating separate types whose methods just print their names and testing them:
class Base(object):
def __mul__(self, lhs): print('Base.mul')
def __rmul__(self, rhs): print('Base.rmul')
class Derived(Base):
def __mul__(self, lhs): print('Derived.mul')
def __rmul__(self, rhs): print('Derived.rmul')
class Unrelated(object):
def __mul__(self, lhs): print('Unrelated.mul')
def __rmul__(self, rhs): print('Unrelated.rmul')
print('Base * Base: ', end='')
Base() * Base()
for x, y in itertools.permutations((Base, Derived, Unrelated), 2):
print('{} * {}: '.format(x.__name__, y.__name__), end='')
x() * y()
What about with built in types as well?
Exactly the same rules. Since Base is not a subclass of either int or float, and neither int nor float knows how to multiply by it, they'll both call Base.__rmul__. And so will any other unrelated type you throw at it:
>>> Base() * 2
Base.mul
>>> 2 * Base()
Base.rmul
>>> Base() * 2.5
Base.mul
>>> 2.5 * Base()
Base.rmul
>>> 'sdfsdfsdfds' * Base()
Base.rmul
>>> (lambda: 23) * Base()
Base.rmul
My problem is that I'm getting different results from 1.5 * myObj and myObj * 1.5
There are a number of reasons for that:
Your __mul__ and __rmul__ code don't do the same thing.
You inherited from float.
You inherited from some builtin or extension type that handles float multiplication at the C-API level and isn't designed to allow overrides in subclasses.
You created a classic class instead of a new-style class.
You made a typo in one of the names.
…
Related
I have a class called Time, and I need to implement a Frequency class. How can I implement dividing ints or floats by an instance of Time to get an instance of Frequency ?
I already know about __div__, __truediv__, __floordiv__ and other Python special methods, and I already use them in my code to divide instances of classes by numbers or instances of other classes, but I cannot find a way to divide a number by an instance of my class.
Is it possible to implement dividing a number by an instance of a class in Python ?
The __rtruediv__ method is what you're looking for.
When x / y is executed, if type(x) does not implement a __div__(self, other) method where other can be of class type(y), then type(y).__rtruediv__(y, x) is executed, and its result is returned.
Usage:
class Foo:
def __init__(self, x):
self.x = x
def __truediv__(self, other):
return self.x / other
def __rtruediv__(self, other):
return other / self.x
>>> f = Foo(10)
>>> f / 10
1.0
>>> 10 / f
1.0
Yes. You just have to make sure that Time.__rtruediv__() returns a Frequency instance when it receives a float or integer.
Usage:
>>> 100 / Time(2)
Frequency(50.0)
>>> 2.5 / Time(5)
Frequency(0.5)
Implementation:
class Time:
def __init__(self, value):
self.value = value
def __rtruediv__(self, other):
if not isinstance(other, (int, float)):
return NotImplemented
return Frequency(other / self.value)
class Frequency:
def __init__(self, value):
self.value = value
def __repr__(self):
return '{}({})'.format(self.__class__.__name__, self.value)
The python docs contains a full example on implementing the arithmetic operations for your custom classes.
The proper way to handle incompatible types is to return the special value NotImplemented.
NotImplemented
Special value which should be returned by the binary
special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.)
to indicate that the operation is not implemented with respect to the
other type
Suppose you try to use a unsupported complex number, returning NotImplemented will eventually cause a TypeError with a correct error message. (at least in python 3)
>>> 100j / Time(2)
Traceback (most recent call last):
File "python", line 1, in <module>
TypeError: unsupported operand type(s) for /: 'complex' and 'Time'
you need to implement __rtruediv__ and__rfloordiv__.
from the documentation
object.__radd__(self, other)
object.__rsub__(self, other)
object.__rmul__(self, other)
object.__rmatmul__(self, other)
object.__rtruediv__(self, other)
object.__rfloordiv__(self, other)
object.__rmod__(self, other)
object.__rdivmod__(self, other)
object.__rpow__(self, other)
object.__rlshift__(self, other)
object.__rrshift__(self, other)
object.__rand__(self, other)
object.__rxor__(self, other)
object.__ror__(self, other)
These methods are called to implement the binary arithmetic operations
(+, -, *, #, /, //, %, divmod(), pow(), **, <<, >>, &, ^, |) with
reflected (swapped) operands. These functions are only called if the
left operand does not support the corresponding operation [3] and the
operands are of different types. [4] For instance, to evaluate the
expression x - y, where y is an instance of a class that has an
__rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented.
Consider the following example of a 'wrapper' class to represent vectors:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __str__(self):
return str(self._vals)
The __add__ method takes care of adding two vectors as well as adding a vector with a scalar. However, the second case is not complete as the following examples show:
>>> a = Vector([1.2, 3, 4])
>>> print(a)
[1.2, 3, 4]
>>> print(a+a)
[2.4, 6, 8]
>>> print(a+5)
[6.2, 8, 9]
>>> print(5+a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'Vector'
To my understanding the reason is that the overloaded operator only tells Python what to do when it sees a + x where a is an instance of Vector, but there is no indication of what to do for x + a (with a an instance of Vector and x a scalar).
How one should overload the operators in such circumstances to cover all cases (i.e., to support the case that self is not an instance of Vector but other is)?
Ok. I guess I found the answer: one has to overload __radd__ operator as well:
class Vector:
def __init__(self, value):
self._vals = value.copy()
def __add__(self, other):
if isinstance(other, list):
result = [x+y for (x, y) in zip(self._vals, other)]
elif isinstance(other, Vector):
result = [x+y for (x, y) in zip(self._vals, other._vals)]
else:
# assume other is scalar
result = [x+other for x in self._vals]
return Vector(result)
def __radd__(self, other):
return self + other
def __str__(self):
return str(self._vals)
Although to me this looks a bit redundant. (Why Python does not use the commutativity of addition by default, assuming __radd__(self, other) always returns self + other? Of course for special cases the user can override __radd__.)
You could define a Scalar class that has int as its base class.
Then override __add__ to do what you want.
class Scalar(int):
def __add__(self):
# do stuff
You already figured out you need to implement __radd__. This is an answer as to why this is so, and why you need to do this in addition to implementing __add__, as a Both quotes are taken from Python Docs (Data Model - 3.3.8 Emulating numeric types), starting with the obvious:
These methods are called to implement the binary arithmetic operations (+, -, *, #, /, //, %, divmod(), pow(), **, <<, >>, &, ^, |). For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called.
So order determines which object's implementation of __add__ is called. When the method doesn't support the operation with the passed argument NotImplemented should be returned. That's where the so-called "reflected methods" come into play:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types. For instance, to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, y.__rsub__(x) is called if x.__sub__(y) returns NotImplemented [sic].
Now, why wouldn't __radd__(self, other) just fall back to __add__(self, other)? While ring addition is always commutative (see this and this math.stackexchange answers), you could have algebraic structures that are do not satisfy this assumption (e.g., near-rings). But my guess as a non-mathematician would be that it's just desirable to have a consistent data model across different numerical methods. While addition might be commonly commutative, multiplication is less so. (Think matrices and vectors! Although, admittedly this is not the best example, given __matmul__). I also prefer to see there being no exceptions, especially if I had to read about rings, etc. in a language documentation.
I am trying to understand how operator overriding works for two operands of a custom class.
For instance, suppose I have the following:
class Adder:
def __init__(self, value=1):
self.data = value
def __add__(self,other):
print('using __add__()')
return self.data + other
def __radd__(self,other):
print('using __radd__()')
return other + self.data
I initialize the following variables:
x = Adder(5)
y = Adder(4)
And then proceed to do the following operations:
1 + x
using __radd__()
Out[108]: 6
x + 2
using __add__()
Out[109]: 7
The two operations above seem straigtforward. If a member of my custom class is to the right of the "+" in the addition expression, then __radd__ is used. If it is on the left, then __add__ is used. This works for expressions when one operand is of the Adder type and another one is something else.
When I do this, however, I get the following result:
x + y
using __add__()
using __radd__()
Out[110]: 9
As you can see, if both operands are of the custom class, then both __add__ and __radd__ are called.
My question is how does Python unravel this situation and how is it able to call both the right-hand-addition function, as well as the left-hand-addition function.
It's because inside your methods you add the data to other. This is itself an instance of Adder. So the logic goes:
call __add__ on x;
add x.data (an int) to y (an Adder instance)
ah, right-hand operand is an instance with a __radd__ method, so
call __radd__ on y;
add int to y.data (another int).
Usually you would check to see if other was an instance of your class, and if so add other.data rather than just other.
That's the because the implementation of your __add__ and __radd__ method do not give any special treatment to the instances of the Adder class. Therefore, each __add__ call leads to an integer plus Adder instance operation which further requires __radd__ due to the Adder instance on the right side.
You can resolve this by doing:
def __add__(self, other):
print('using __add__()')
if isinstance(other, Adder):
other = other.data
return self.data + other
def __radd__(self, other):
print('using __radd__()')
return self.__add__(other)
I'm coming to Python from Racket. In Racket, I would define a Point structure like this:
(struct Point (x y) #:transparent)
A point is now a structure with two fields named x and y. I can compare two structures for (deep) equality by calling equal?.
What is the equivalent in Python? It looks to me like I have to write twelve lines:
class Point():
def __init__(self,x,y):
self.x = x;
self.y = y;
def __eq__(self, other):
return ((type(other) is Point)
and self.x == other.x
and self.y == other.y)
def __ne__(self, other):
return not(self == other)
... but surely there's an easier way?
Yes, well, if you need an entire class to represent your data type, then you will have to rely on the __eq__ and related dunder methods. However, in this particular case, a Pythonista would use a namedtuple:
from collections import namedtuple
Point = namedtuple('Point', ['x','y'])
Which will inherit all that from tuple.
If you don't need mutability, the simplest way to make basic classes of this sort is collections.namedtuple:
import collections
Point = collections.namedtuple('Point', 'x y')
That's it. You can just make Point objects with pt = Point(1, 2) or the like, and they work like two-tuples, but they also let you access them via named attributes, e.g. pt.x, pt.y.
The equality checking will be a little looser (Point(1, 2) == (1, 2) evaluates to True, because all namedtuples are subclasses of tuple and will compare using tuple rules, and in fact, different subclasses of tuple that don't override the comparison methods will compare equal to each other if they have the same values), but given that tuples are typically used as anonymous lightweight "classes", this is often what you want.
If you need to customize some behavior (adding functionality, or make the type comparisons stricter), you can make a custom class inherit from a namedtuple to get the basic features for free, then customize the bits you care about, e.g., to prevent it testing equal to non-Point types, you can do:
class Point(collections.namedtuple('PointBase', 'x y')):
def __eq__(self, other):
if not isinstance(other, Point):
return False
return super().__eq__(other)
# Sadly, tuple defines __ne__, so you must override it too to behave properly
# You don't need the canonical __ne__ implementation that handles NotImplemented
# though, since you're explicitly unfriendly to non-Point types
def __ne__(self, other): return not (self == other)
I have a vector class and I defined the __mul__ method to multiply a vector by a number.
Here is the __mul__ method :
def __mul__(self, other):
x = self.x * other
y = self.y * other
new = Vector()
new.set_pos((x, y))
return new
My problem is that I don't know which is which between the number and the vector.
If self is the number, self.x raises an error. (I'm maybe mistaking on this point : Is "other" always a number ?)
So I found here : Python: multiplication override
that I could do :
__rmul__ = __mul__
but how can I do that in a class definition ?
Something like :
def __rmul__ = __mul__
self will never be the number in __mul__() because the object the method is attached to is not the number, it's the vector, and by definition it's the multiplicand.
other will be a number if your object is being multiplied by a number. Or it could be something else, such as another vector, which you could test for and handle.
When your object is the multiplier, __rmul__() is called if the multiplicand doesn't know how to handle the operation.
To handle the case in which __mul__ and __rmul__ should be the same method, because the operation is commutative, you can just do the assignment in your class definition.
class Vector(object):
def __mul__(self, other):
pass
__rmul__ = __mul__
Simply list it as an attribute:
__rmul__ = __mul__
This is the same way you'd create an alias of a function in a module; creating an alias of a method within a class body works the same.
The point is that in Python, you can tell objects how to multiply themselves by things. That means that
a * b
could either mean "tell a to multiply itself by b" or "tell b to multiply itself by a". In code, that translates to
a.__mul__(b)
or
b.__rmul__(a)