Develop Reference

Equality Comparison with NumPy Instance Invokes `__bool__`

I have defined a class where its __ge__ method returns an instance of itself, and whose __bool__ method is not allowed to be invoked (similar to a Pandas Series).
Why is X.__bool__ invoked during np.int8(0) <= x, but not for any of the other examples? Who is invoking it? I have read the Data Model docs but I haven’t found my answer there.
import numpy as np
import pandas as pd
class X:
def __bool__(self):
print(f"{self}.__bool__")
assert False
def __ge__(self, other):
print(f"{self}.__ge__")
return X()
x = X()
np.int8(0) <= x
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5D90>.__bool__
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# File "<stdin>", line 4, in __bool__
# AssertionError
0 <= x
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5DF0>
x >= np.int8(0)
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5D30>
pd_ge = pd.Series.__ge__
def ge_wrapper(self, other):
print("pd.Series.__ge__")
return pd_ge(self, other)
pd.Series.__ge__ = ge_wrapper
pd_bool = pd.Series.__bool__
def bool_wrapper(self):
print("pd.Series.__bool__")
return pd_bool(self)
pd.Series.__bool__ = bool_wrapper
np.int8(0) <= pd.Series([1,2,3])
# Console output:
# pd.Series.__ge__
# 0 True
# 1 True
# 2 True
# dtype: bool

I suspect that np.int8.__le__ is defined so that instead of returning NotImplemented and letting X.__ge__ take over, it instead tries to return something like not (np.int(8) > x), and then np.int8.__gt__ raises NotImplemented. Once X.__gt__(x, np.int8(0)) returns an instance of X rather than a Boolean value, then we need to call x.__bool__() in order to compute the value of not x.
(Still trying to track down where int8.__gt__ is defined to confirm.)
(Update: not quite. int8 uses a single generic rich comparison function that simply converts the value to a 0-dimensional array, then returns the result of PyObject_RichCompare on the array and x.)
I did find this function that appears to ultimately implement np.int8.__le__:
static NPY_INLINE int
rational_le(rational x, rational y) {
return !rational_lt(y,x);
}
It's not clear to me how we avoid getting to this function if one of the arguments (like X) would not be a NumPy type. I think I give up.

TL;DR
X.__array_priority__ = 1000
The biggest hint is that it works with a pd.Series.
First I tried having X inherit from pd.Series. This worked (i.e. __bool__ no longer called).
To determine whether NumPy is using an isinstance check or duck-typing approach, I removed the explicit inheritance and added (based on this answer):
#property
def __class__(self):
return pd.Series
The operation no longer worked (i.e. __bool__ was called).
So now I think we can conclude NumPy is using a duck-typing approach. So I checked to see what attributes are being accessed on X.
I added the following to X:
def __getattribute__(self, item):
print("getattr", item)
return object.__getattribute__(self, item)
Again instantiating X as x, and invoking np.int8(0) <= x, we get:
getattr __array_priority__
getattr __array_priority__
getattr __array_priority__
getattr __array_struct__
getattr __array_interface__
getattr __array__
getattr __array_prepare__
<__main__.X object at 0x000002022AB5DBE0>.__ge__
<__main__.X object at 0x000002021A73BE50>.__bool__
getattr __array_struct__
getattr __array_interface__
getattr __array__
Traceback (most recent call last):
File "<stdin>", line 32, in <module>
np.int8(0) <= x
File "<stdin>", line 21, in __bool__
assert False
AssertionError
Ah-ha! What is __array_priority__? Who cares, really. With a little digging, all we need to know is that NDFrame (from which pd.Series inherits) sets this value as 1000.
If we add X.__array_priority__ = 1000, it works! __bool__ is no longer called.
What made this so difficult (I believe) is that the NumPy code didn't show up in the call stack because it is written in C. I could investigate further if I tried out the suggestion here.

int class constructor accepts bytes, or bytearray instance?

I am learning Python and I'm little confused about the int class constructor
Does the int class constructor accept bytes, or bytearray instance?
From the doc: https://docs.python.org/3/library/functions.html#int
class int([x])
class int(x, base=10)
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in radix base.
If I pass the bytes instance, then I am getting the below error.
i = 10
b = bytes(i)
val = int(b)
Output:
Traceback (most recent call last):
File "test.py", line 4, in <module>
val = int(b)
ValueError: invalid literal for int() with base 10: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

That refers to bytes of text:
>>> int(b'123')
123
You seem to be looking for int.from_bytes.

behavior of builtin functions when assigned as class attributes

I would like to assign a function as class attribute, and have it so when accessed through instance, it is still unbounded. I understand that this can be achieved with using staticmethod descriptor. But it seems the behavior is different for the builtin functions, and I would like to replicate that.
def abs_(value):
return abs(value)
class Test:
func_1 = abs
func_2 = len
func_3 = abs_
func_4 = staticmethod(abs_)
>>> test = Test()
>>> test.func_1
<built-in function abs>
>>> test.func_2
<built-in function len>
>>> test.func_3
<bound method abs_ of <__main__.Test object at 0x10436d910>>
In this case, the builtin function are unbound, and the defined function abs_ is bound to the instance. And obviously all functions work except func_3 since it is bound method.
>>> test.func_1(-1)
1
>>> test.func_3(-1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: abs_() takes 1 positional argument but 2 were given
How does builtin function achieve this, and is there a way to replicate the behavior (remain unbound)? Thank you!

Magic methods with > 1 operands between MyClass and standard Class

Imagine I want to define a new number class. Say, RationalFractions or GaussIntegers, whatever. Of course, I can easily define a + b for two objects of MyClass. But i would like to be able to add an object of MyClass with some existing, like "integer" or "float". With the result having a relevant type (from my point of view). E.g. the result of GaussInteger + float = GaussInteger, RationalFraction + integer = RationalFraction, etc.
I guess I should somehow alter add for Object class, or "integer", "float"? Or there is a way to do it without meddling with the existing classes?
Edit. So, an example:
class RatFr:
def __init__(self, m, n=1):
self.m = m
self.n = n
def __add__(self, other):
temp = (RatFr(other) if type(other) == int else other)
return RatFr(self.m * temp.n + self.n * temp.m, self.n * temp.n)
def __str__(self):
return f'{self.m}/{self.n}'
a = RatFr(5,3)
b = 1
print(a)
print(a + b)
print(b + a)
I get as a result:
5/3
8/3
Traceback (most recent call last):
File "/Users/aleksej/PycharmProjects/Alex2/playaround.py", line 19, in <module>
print(b + a)
TypeError: unsupported operand type(s) for +: 'int' and 'RatFr'
Trying to convert self does nothing good. As soon as the first operand is int, python obviously looks for integer add method.

Yes. You will want to override __add__ from object, taking self and say x as parameters. You can then deal with x according to its type. Here you have a few options. You can do explicit type checking, but this is not very Pythonic. I would probably make a conversion function to convert from int, float, etc to your type and call it on x. Then you can do whatever addition would do between two objects of your type. This sort of call to a conversion function before doing an operation is done in the mpmath library, in the backend. Remember that you will need to check if the thing you are converting is already the right type.

Disable silent conversions in numpy

Is there a way to disable silent conversions in numpy?
import numpy as np
a = np.empty(10, int)
a[2] = 4 # OK
a[3] = 4.9 # Will silently convert to 4, but I would prefer a TypeError
a[4] = 4j # TypeError: can't convert complex to long
Can numpy.ndarray objects be configured to return a TypeError when assigning any value which is not isinstance() of the ndarray type?
If not, would the best alternative be to subclass numpy.ndarray (and override __setattr__ or __setitem__)?

Unfortunately numpy doesn't offer this feature in array creation, you can set if casting is allowed only when you are converting an array (check the documentation for numpy.ndarray.astype).
You could use that feature, or subclass numpy.ndarray, but also consider using the array module offered by python itself to create a typed array:
from array import array
a = array('i', [0] * 10)
a[2] = 4 # OK
a[3] = 4.9 # TypeError: integer argument expected, got float

Just an idea.
#Python 2.7.3
>>> def test(value):
... if '.' in str(value):
... return str(value)
... else:
... return value
...
>>> a[3]=test(4.0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for long() with base 10: '4.0'