Implementing hierarchy for Enum members - python

I would like to establish a hierarchy for the members of my Enum. My (simplified) enum aims at representing different types of food. Of course, everyone knows a burger is "superior" to a pizza and my enum needs to convey this idea:
from functools import total_ordering
from enum import IntEnum, unique
#unique
#total_ordering
class FoodType(IntEnum):
PIZZA = 100
COOKIE = 200
STEAK = 300
BURGER = 400
def __lt__(self, other):
if self.__class__ is other.__class__:
return self.FOOD_HIERARCHY.index(self) < self.FOOD_HIERARCHY.index(other)
return NotImplemented
def __gt__(self, other):
if self.__class__ is other.__class__:
return self.FOOD_HIERARCHY.index(self) > self.FOOD_HIERARCHY.index(other)
return NotImplemented
def __eq__(self, other):
if self.__class__ is other.__class__:
return self.FOOD_HIERARCHY.index(self) == self.FOOD_HIERARCHY.index(other)
return NotImplemented
# Order is important here; smallest entity first
FoodType.FOOD_HIERARCHY = [
FoodType.COOKIE,
FoodType.STEAK,
FoodType.PIZZA,
FoodType.BURGER,
]
Here my food types are arbitrary integers. They need to be integers for reasons outside of the scope of this question. I also can't use the integer values for comparison, nor the order of definition of the food types. That is why I create the hierarchy of FoodType outside the enums, and make it an attribute of the Enum after the definition.
I would like to use the positions of the food types (aka indexes) to implement the comparison methods.
However when I run a simple comparison on two of the FoodType mentioned above, I get a recursion error:
In [2]: from test import FoodType
In [3]: FoodType.PIZZA < FoodType.BURGER
---------------------------------------------------------------------------
RecursionError Traceback (most recent call last)
<ipython-input-3-1880a19bb0cd> in <module>
----> 1 FoodType.PIZZA < FoodType.BURGER
~/projects/test.py in __lt__(self, other)
13 def __lt__(self, other):
14 if self.__class__ is other.__class__:
---> 15 return self.FOOD_HIERARCHY.index(self) < self.FOOD_HIERARCHY.index(other)
16 return NotImplemented
17
~/projects//test.py in __eq__(self, other)
23 def __eq__(self, other):
24 if self.__class__ is other.__class__:
---> 25 return self.FOOD_HIERARCHY.index(self) == self.FOOD_HIERARCHY.index(other)
26 return NotImplemented
27
... last 1 frames repeated, from the frame below ...
~/projects/test.py in __eq__(self, other)
23 def __eq__(self, other):
24 if self.__class__ is other.__class__:
---> 25 return self.FOOD_HIERARCHY.index(self) == self.FOOD_HIERARCHY.index(other)
26 return NotImplemented
27
RecursionError: maximum recursion depth exceeded while calling a Python object
I can't figure out why I get a recursion error. If I use the enum values to build the hierarchy and to look up the indexes, I can make this code work, but I would like to avoid that if possible.
Any idea why I get the recursion error and how I could make this code more elegant?
EDIT: as people mentioned in the comments, I do override __eq__, __lt__ and __gt__. I wouldn't have done it normally, but in my real life example I have two different hierarchies and some enum members can be in the two hierarchies. So I need to first check the 2 enum members I'm comparing are in the same hierarchy. That said, I can probably use __super()__. Thanks for the observation.
EDIT 2:
Base on #Ethan Furman's answer, here is what the final code looks like:
from enum import IntEnum, unique
def hierarchy(hierarchy_name, member_names):
def decorate(enum_cls):
for name in enum_cls.__members__:
if not hasattr(enum_cls[name], "ordering"):
enum_cls[name].ordering = {}
for i, name in enumerate(member_names.split()):
# FIXME, check if name in __members__
# FIXME, shouldn't exist yet, check!
enum_cls[name].ordering[hierarchy_name] = i
return enum_cls
return decorate
#hierarchy("food_hierarchy", "COOKIE STEAK PIZZA BURGER")
#unique
class FoodType(IntEnum):
PIZZA = 100
COOKIE = 200
STEAK = 300
BURGER = 400
def __lt__(self, other) -> bool:
if self.__class__ is other.__class__:
try:
hierarchy = (self.ordering.keys() & other.ordering.keys()).pop()
except KeyError:
raise ValueError("uncomparable, hierachies don't overlap")
return self.ordering[hierarchy] < other.ordering[hierarchy]
return NotImplemented
def __eq__(self, other) -> bool:
if self.__class__ is other.__class__:
return int(self) == int(other)
return NotImplemented

The recursion error is not important as your design is flawed:
total_ordering is useless/harmful because IntEnum is an int and ints already have total ordering
the food items, being ints will compare with other ints
not properly comparing with other ints will be a hard-to-find bug at some point
Possible solutions:
add an extra attribute to each member to control food ordering
(optional) make FoodType be a normal Enum and add an __int__ method to easily convert to int (and keep total_ordering)
The extra attribute can be done in one of two ways:
defined with the member
added afterwards
Defined with the member could easily be confusing:
class FoodType(IntEnum):
PIZZA = 100, 3
COOKIE = 200, 1
STEAK = 300, 2
BURGER = 400, 4
So I would do it as a decorator
#add_order('COOKIE STEAK PIZZA BURGER')
class FoodType(IntEnum):
PIZZA = 100
COOKIE = 200
STEAK = 300
BURGER = 400
If FoodType becomes an Enum you can still use total_ordering, otherwise you should use different methods for comparison; if you don't then you'll have 100 (PIZZA) not < 101 (a normal int) which will be a bug at some point -- an easy example being FoodTypes and ints both being keys in the same dict().
The decorator and __lt__ would look like:
def add_order(enum_cls, member_names):
for i, name in enumerate(member_names.split()):
enum_cls[name].order = i
class FoodType(IntEnum):
...
def __lt__(self, other):
if isinstance(other, self.__class__):
return self.order < other.order
return NotImplemented
N.B. total_ordering had a bug regarding NotImplemented which was fixed in 3.4, and somewhere in 2.7. Make sure your version works properly if using 2.7 (or just add the comparison methods yourself).

You get a recursion error because in order to determine the index the list elements need to compared for equality, which in turn will invoke __eq__.
Alternatively you could use a mapping from the enum members to some ordering, e.g.:
FoodType.FOOD_HIERARCHIES = [
{FoodType.COOKIE: 1, FoodType.PIZZA: 2, FoodType.BURGER: 3},
{FoodType.STEAK: 1, FoodType.BURGER: 2},
]
This requires to make the enum hashable:
def __hash__(self):
return hash(self._name_)
This works because the dictionary lookup checks for object identity before considering __eq__.
Since total_ordering won't replace the methods inherited from the base class, you'd need to override all comparison methods (or inherit from Enum instead of IntEnum):
from enum import IntEnum, unique
import operator
#unique
class FoodType(IntEnum):
PIZZA = 100
COOKIE = 200
STEAK = 300
BURGER = 400
def __hash__(self):
return hash(self._name_)
def __lt__(self, other):
return self._compare(other, operator.lt)
def __le__(self, other):
return self._compare(other, operator.le)
def __gt__(self, other):
return self._compare(other, operator.gt)
def __ge__(self, other):
return self._compare(other, operator.ge)
def __eq__(self, other):
return self._compare(other, operator.eq)
def __ne__(self, other):
return self._compare(other, operator.ne)
def _compare(self, other, op):
if self.__class__ is other.__class__:
hierarchy = next(h for h in self.FOOD_HIERARCHIES if self in h)
try:
return op(hierarchy[self], hierarchy[other])
except KeyError:
return False # or: return NotImplemented
return NotImplemented
FoodType.FOOD_HIERARCHIES = [
{FoodType.COOKIE: 1, FoodType.PIZZA: 2, FoodType.BURGER: 3},
{FoodType.STEAK: 1, FoodType.BURGER: 2},
]
print(FoodType.COOKIE < FoodType.BURGER) # True
print(FoodType.STEAK > FoodType.BURGER) # False
print(FoodType.STEAK < FoodType.PIZZA) # False

Related

return NotImplemented but when printed, I get False

Here is a minimal reproducible example:
class Attribut:
def __init__(
self,
name: str,
other_name: str,
):
self.name: str = name
self.other_name: str = other_name
def __eq__(self, other):
if isinstance(other, Attribut):
return self.name == other.name and self.other_name == other.other_name
else:
return NotImplemented
def __hash__(self):
return 0
If I try to do:
a = Attribut("lol", "a")
print(a==4)
I thought I would get NotImplemented, but instead I get False.
EDIT: (following chepner's answer)
Comparing one object from one class to an object to another class instead of comparing it to an integer:
class Attribut:
def __init__(
self,
name: str,
other_name: str,
):
self.name: str = name
self.other_name: str = other_name
def __eq__(self, other):
if isinstance(other, Attribut):
return self.name == other.name and self.other_name == other.other_name
else:
return NotImplemented
def __hash__(self):
return 0
class Attribut2:
def __init__(
self,
name: str,
other_name: str,
):
self.name: str = name
self.other_name: str = other_name
def __eq__(self, other):
if isinstance(other, Attribut2):
return self.name == other.name and self.other_name == other.other_name
else:
return NotImplemented
def __hash__(self):
return 1
a = Attribut("lol", "a")
b = Attribut2("lol", "b")
print(a==b)
I also get False.
Also, what is the point of overriding __hash__, I cannot find a situation where this is useful?
When a.__eq__(4) returns NotImplemented, you don't get the value back immediately. Instead, Python attempts to call the reflected version of __eq__ (__eq__ itself) with the other argument, namely (4).__eq__(a). It's this call that returns False.
By returning NotImplemented, you are not saying that self and other cannot be compared for equality (for that, raise a ValueError), but rather that Attribut.__eq__ does not know how to do so, but perhaps the other argument's __eq__ method does.
In your two-class example, we have the following:
Attribut.__eq__ returns NotImplemented, so Attribut2.__eq__ (since self and other have different types) is tried.
Attribut2.__eq__ returns NotImplemented, so what gets tried next?
Since calling Attribute.__eq__ again would just put us in an endless cycle, I think that Python falls back to object.__eq__ instead (which can compare any two objects via object identity), but this is not obvious from the descriptions of either NotImplemented or object.__eq__ itself. (This is supported by the fact that if you define both methods as
def __eq__(self, other):
return NotImplemented
and attempt to evaluate a == a, it evaluates to True.)
The closest thing to documentation for this that I can find is the following sentence from the description of NotImplemented:
(The interpreter will then try the reflected operation, or some other fallback, depending on the operator.)
I suspect that whether or not A.__eq__ is considered the reflection of B.__eq__ depends on the context in which B.__eq__ is called. That is, A.__eq__ is not the reflection of B.__eq__ in the case where B.__eq__ was just called as the reflection of A.__eq__.
Overriding hash method allows you to use other methods that would involve sorting a list of your Attribut objects.
To test it create a script in which you would create such a list
a = [Attribut("lol", "a"), Attribut("lol", "a"), Attribut("lol", "a")]
and then try to make a set out of it
set(a)
If the method hash is implemented in Attribut class then there would be no problem. If not then the program will return an error TypeError: unhashable type: 'Attribut'
However, hash is only being used in Python hash-table and it's sole purpose is to make a number out of a class object.
Refer to these articles for more information:
How does hash-table in set works in python?
What does "hashable" mean in Python?
Also, there can be more than one object that would have the same hash - the comparison goes deeper and checks each fields of a class in methods that use comparison between two instances (like set). To make it possible there must be implemented the first step - main class must be hashable.
The hash should also be equal for the objects that should be equal. When comparison appears, Python first checks if two objects have the same hash, if not then the comparison results in False, no matter if two objects should be 'equal'.
You can try this code and check this behaviour. Comment/uncomment function body in hash function to be same or different hash:
from random import random
class Attribut:
def __init__(
self,
name: str,
other_name: str,
):
self.name: str = name
self.other_name: str = other_name
def __eq__(self, other):
if isinstance(other, Attribut):
return self.name == other.name and self.other_name == other.other_name
else:
return NotImplemented
def __hash__(self):
# Same hash
# return 0
# Different hash
return hash(random())
c = [Attribut("lol", "a"), Attribut("lol", "a")]
print(set(c))

How can I make a dataclass hash the same as a string?

I want to replace string keys in dictionaries in my code with a dataclass so that I can provide meta data to the keys for debugging. However, I still want to be able to use a string to lookup dictionaries. I tried implementing a data-class with a replaced __hash__ function, however my code is not working as expected:
from dataclasses import dataclass
#dataclass(eq=True, frozen=True)
class Key:
name: str
def __hash__(self):
return hash(self.name)
k = "foo"
foo = Key(name=k)
d = {}
d[foo] = 1
print(d[k]) # Key Error
The two hash functions are the same:
print(hash(k) == hash(foo)) # True
So I don't understand why this doesn't work.
Two objects having different hashes guarantees that they're different, but two objects having the same hash doesn't in itself guarantee that they're the same (because hash collisions exist). If you want the Key to be considered equal to a corresponding str, implement that in __eq__:
def __eq__(self, other):
if isinstance(other, Key):
return self.name == other.name
if isinstance(other, str):
return self.name == other
return False
This fixes the KeyError you're encountering.
Adding my notes here from the comments on the answer above, as no one looks at those in any case, so those are likely to get swept under the rug at some point.
PyCharm also produces a helpful warning:
'eq' is ignored if the class already defines '__eq__' method.
I think this means to remove the eq=True usage as well, from the #dataclass(...) decorator.
technically, you could also remove the last if isinstance(..., str): as well as the last return statement. I'm not entirely sure what would be the implications of that, however.
Here then, is a slightly more optimized approach (timings with timeit module below):
class Key:
name: str
def __hash__(self):
return hash(self.name)
def __eq__(self, other):
return self.name == getattr(other, 'name', other)
Timings with timeit
from dataclasses import dataclass
from timeit import timeit
#dataclass(frozen=True)
class Key:
name: str
def __hash__(self):
return hash(self.name)
def __eq__(self, other):
if isinstance(other, Key):
return self.name == other.name
if isinstance(other, str):
return self.name == other
return False
class KeyTwo(Key):
def __eq__(self, other):
return self.name == getattr(other, 'name', other)
k = "foo"
foo = Key(name=k)
foo_two = KeyTwo(name=k)
print('__eq__() Timings --')
print('isinstance(): ', timeit("foo == k", globals=globals()))
print('getattr(): ', timeit("foo_two == k", globals=globals()))
assert foo == foo_two == k
Results on my M1 Mac:
__eq__() Timings --
isinstance(): 0.10553250007797033
getattr(): 0.08371329202782363

Python equality of class instances [duplicate]

When writing custom classes it is often important to allow equivalence by means of the == and != operators. In Python, this is made possible by implementing the __eq__ and __ne__ special methods, respectively. The easiest way I've found to do this is the following method:
class Foo:
def __init__(self, item):
self.item = item
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def __ne__(self, other):
return not self.__eq__(other)
Do you know of more elegant means of doing this? Do you know of any particular disadvantages to using the above method of comparing __dict__s?
Note: A bit of clarification--when __eq__ and __ne__ are undefined, you'll find this behavior:
>>> a = Foo(1)
>>> b = Foo(1)
>>> a is b
False
>>> a == b
False
That is, a == b evaluates to False because it really runs a is b, a test of identity (i.e., "Is a the same object as b?").
When __eq__ and __ne__ are defined, you'll find this behavior (which is the one we're after):
>>> a = Foo(1)
>>> b = Foo(1)
>>> a is b
False
>>> a == b
True
Consider this simple problem:
class Number:
def __init__(self, number):
self.number = number
n1 = Number(1)
n2 = Number(1)
n1 == n2 # False -- oops
So, Python by default uses the object identifiers for comparison operations:
id(n1) # 140400634555856
id(n2) # 140400634555920
Overriding the __eq__ function seems to solve the problem:
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return False
n1 == n2 # True
n1 != n2 # True in Python 2 -- oops, False in Python 3
In Python 2, always remember to override the __ne__ function as well, as the documentation states:
There are no implied relationships among the comparison operators. The
truth of x==y does not imply that x!=y is false. Accordingly, when
defining __eq__(), one should also define __ne__() so that the
operators will behave as expected.
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
return not self.__eq__(other)
n1 == n2 # True
n1 != n2 # False
In Python 3, this is no longer necessary, as the documentation states:
By default, __ne__() delegates to __eq__() and inverts the result
unless it is NotImplemented. There are no other implied
relationships among the comparison operators, for example, the truth
of (x<y or x==y) does not imply x<=y.
But that does not solve all our problems. Let’s add a subclass:
class SubNumber(Number):
pass
n3 = SubNumber(1)
n1 == n3 # False for classic-style classes -- oops, True for new-style classes
n3 == n1 # True
n1 != n3 # True for classic-style classes -- oops, False for new-style classes
n3 != n1 # False
Note: Python 2 has two kinds of classes:
classic-style (or old-style) classes, that do not inherit from object and that are declared as class A:, class A(): or class A(B): where B is a classic-style class;
new-style classes, that do inherit from object and that are declared as class A(object) or class A(B): where B is a new-style class. Python 3 has only new-style classes that are declared as class A:, class A(object): or class A(B):.
For classic-style classes, a comparison operation always calls the method of the first operand, while for new-style classes, it always calls the method of the subclass operand, regardless of the order of the operands.
So here, if Number is a classic-style class:
n1 == n3 calls n1.__eq__;
n3 == n1 calls n3.__eq__;
n1 != n3 calls n1.__ne__;
n3 != n1 calls n3.__ne__.
And if Number is a new-style class:
both n1 == n3 and n3 == n1 call n3.__eq__;
both n1 != n3 and n3 != n1 call n3.__ne__.
To fix the non-commutativity issue of the == and != operators for Python 2 classic-style classes, the __eq__ and __ne__ methods should return the NotImplemented value when an operand type is not supported. The documentation defines the NotImplemented value as:
Numeric methods and rich comparison methods may return this value if
they do not implement the operation for the operands provided. (The
interpreter will then try the reflected operation, or some other
fallback, depending on the operator.) Its truth value is true.
In this case the operator delegates the comparison operation to the reflected method of the other operand. The documentation defines reflected methods as:
There are no swapped-argument versions of these methods (to be used
when the left argument does not support the operation but the right
argument does); rather, __lt__() and __gt__() are each other’s
reflection, __le__() and __ge__() are each other’s reflection, and
__eq__() and __ne__() are their own reflection.
The result looks like this:
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return NotImplemented
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
x = self.__eq__(other)
if x is NotImplemented:
return NotImplemented
return not x
Returning the NotImplemented value instead of False is the right thing to do even for new-style classes if commutativity of the == and != operators is desired when the operands are of unrelated types (no inheritance).
Are we there yet? Not quite. How many unique numbers do we have?
len(set([n1, n2, n3])) # 3 -- oops
Sets use the hashes of objects, and by default Python returns the hash of the identifier of the object. Let’s try to override it:
def __hash__(self):
"""Overrides the default implementation"""
return hash(tuple(sorted(self.__dict__.items())))
len(set([n1, n2, n3])) # 1
The end result looks like this (I added some assertions at the end for validation):
class Number:
def __init__(self, number):
self.number = number
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return NotImplemented
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
x = self.__eq__(other)
if x is not NotImplemented:
return not x
return NotImplemented
def __hash__(self):
"""Overrides the default implementation"""
return hash(tuple(sorted(self.__dict__.items())))
class SubNumber(Number):
pass
n1 = Number(1)
n2 = Number(1)
n3 = SubNumber(1)
n4 = SubNumber(4)
assert n1 == n2
assert n2 == n1
assert not n1 != n2
assert not n2 != n1
assert n1 == n3
assert n3 == n1
assert not n1 != n3
assert not n3 != n1
assert not n1 == n4
assert not n4 == n1
assert n1 != n4
assert n4 != n1
assert len(set([n1, n2, n3, ])) == 1
assert len(set([n1, n2, n3, n4])) == 2
You need to be careful with inheritance:
>>> class Foo:
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
>>> class Bar(Foo):pass
>>> b = Bar()
>>> f = Foo()
>>> f == b
True
>>> b == f
False
Check types more strictly, like this:
def __eq__(self, other):
if type(other) is type(self):
return self.__dict__ == other.__dict__
return False
Besides that, your approach will work fine, that's what special methods are there for.
The way you describe is the way I've always done it. Since it's totally generic, you can always break that functionality out into a mixin class and inherit it in classes where you want that functionality.
class CommonEqualityMixin(object):
def __eq__(self, other):
return (isinstance(other, self.__class__)
and self.__dict__ == other.__dict__)
def __ne__(self, other):
return not self.__eq__(other)
class Foo(CommonEqualityMixin):
def __init__(self, item):
self.item = item
Not a direct answer but seemed relevant enough to be tacked on as it saves a bit of verbose tedium on occasion. Cut straight from the docs...
functools.total_ordering(cls)
Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:
The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.
New in version 2.7
#total_ordering
class Student:
def __eq__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
You don't have to override both __eq__ and __ne__ you can override only __cmp__ but this will make an implication on the result of ==, !==, < , > and so on.
is tests for object identity. This means a is b will be True in the case when a and b both hold the reference to the same object. In python you always hold a reference to an object in a variable not the actual object, so essentially for a is b to be true the objects in them should be located in the same memory location. How and most importantly why would you go about overriding this behaviour?
Edit: I didn't know __cmp__ was removed from python 3 so avoid it.
From this answer: https://stackoverflow.com/a/30676267/541136 I have demonstrated that, while it's correct to define __ne__ in terms __eq__ - instead of
def __ne__(self, other):
return not self.__eq__(other)
you should use:
def __ne__(self, other):
return not self == other
I think that the two terms you're looking for are equality (==) and identity (is). For example:
>>> a = [1,2,3]
>>> b = [1,2,3]
>>> a == b
True <-- a and b have values which are equal
>>> a is b
False <-- a and b are not the same list object
The 'is' test will test for identity using the builtin 'id()' function which essentially returns the memory address of the object and therefore isn't overloadable.
However in the case of testing the equality of a class you probably want to be a little bit more strict about your tests and only compare the data attributes in your class:
import types
class ComparesNicely(object):
def __eq__(self, other):
for key, value in self.__dict__.iteritems():
if (isinstance(value, types.FunctionType) or
key.startswith("__")):
continue
if key not in other.__dict__:
return False
if other.__dict__[key] != value:
return False
return True
This code will only compare non function data members of your class as well as skipping anything private which is generally what you want. In the case of Plain Old Python Objects I have a base class which implements __init__, __str__, __repr__ and __eq__ so my POPO objects don't carry the burden of all that extra (and in most cases identical) logic.
Instead of using subclassing/mixins, I like to use a generic class decorator
def comparable(cls):
""" Class decorator providing generic comparison functionality """
def __eq__(self, other):
return isinstance(other, self.__class__) and self.__dict__ == other.__dict__
def __ne__(self, other):
return not self.__eq__(other)
cls.__eq__ = __eq__
cls.__ne__ = __ne__
return cls
Usage:
#comparable
class Number(object):
def __init__(self, x):
self.x = x
a = Number(1)
b = Number(1)
assert a == b
This incorporates the comments on Algorias' answer, and compares objects by a single attribute because I don't care about the whole dict. hasattr(other, "id") must be true, but I know it is because I set it in the constructor.
def __eq__(self, other):
if other is self:
return True
if type(other) is not type(self):
# delegate to superclass
return NotImplemented
return other.id == self.id
I wrote a custom base with a default implementation of __ne__ that simply negates __eq__:
class HasEq(object):
"""
Mixin that provides a default implementation of ``object.__neq__`` using the subclass's implementation of ``object.__eq__``.
This overcomes Python's deficiency of ``==`` and ``!=`` not being symmetric when overloading comparison operators
(i.e. ``not x == y`` *does not* imply that ``x != y``), so whenever you implement
`object.__eq__ <https://docs.python.org/2/reference/datamodel.html#object.__eq__>`_, it is expected that you
also implement `object.__ne__ <https://docs.python.org/2/reference/datamodel.html#object.__ne__>`_
NOTE: in Python 3+ this is no longer necessary (see https://docs.python.org/3/reference/datamodel.html#object.__ne__)
"""
def __ne__(self, other):
"""
Default implementation of ``object.__ne__(self, other)``, delegating to ``self.__eq__(self, other)``.
When overriding ``object.__eq__`` in Python, one should also override ``object.__ne__`` to ensure that
``not x == y`` is the same as ``x != y``
(see `object.__eq__ <https://docs.python.org/2/reference/datamodel.html#object.__eq__>`_ spec)
:return: ``NotImplemented`` if ``self.__eq__(other)`` returns ``NotImplemented``, otherwise ``not self.__eq__(other)``
"""
equal = self.__eq__(other)
# the above result could be either True, False, or NotImplemented
if equal is NotImplemented:
return NotImplemented
return not equal
If you inherit from this base class, you only have to implement __eq__ and the base.
In retrospect, a better approach might have been to implement it as a decorator instead. Something like #functools.total_ordering

Accessing original int comparison from int-derived class with overloaded comparison operator

I have an int-derived class with overloaded comparison operator.
In the body of the overloaded methods I need to use the original operator.
The toy example:
>>> class Derived(int):
... def __eq__(self, other):
... return super(Derived, self).__eq__(other)
works fine with Python 3.3+, but fails with Python 2.7 with exception AttributeError: 'super' object has no attribute '__eq__'.
I can think about several walkarrounds, which I found not very clean:
return int(self) == other
requires creation of a new int object just to compare it, while
try:
return super(Derived, self).__eq__(other)
except AttributeError:
return super(Derived, self).__cmp__(other) == 0
splits the control flow based on the Python version, which I find terribly messy (so is inspecting the Python version explicitly).
How can I access the original integer comparison in an elegant way working with Python 2.7 and 3.3+?
Python 2 and 3 are significantly different from each other so I think you should bite the bullet and check versions. That is only to be expected if you're trying to write code that works on both (sooner or later in my experience you find something you have to patch). To avoid any performance impact you could do something like:
from six import PY2
class Derived(int):
if PY2:
def __eq__(self, other):
return super(Derived, self).__cmp__(other) == 0
else:
def __eq__(self, other):
return super(Derived, self).__eq__(other)
That's what I'd do. If I really wanted to subclass int...
If you really don't want to, perhaps you could try:
class Derived(int):
def __eq__(self, other):
return (self ^ other) == 0
Obviously if you care about performance you'll have to do some profiling with the rest of your code and find out if either of them is significantly worse...
Both versions implement an __xor__ method, you could try this:
class Derived(int):
def __eq__(self, other):
return not super(Derived, self).__xor__(other)
I believe that you should define the __eq__ in the int before defining the class. For example:
int = 5
def int.__eq__(self, other):
return self.real == other
IntDerived = Derived(int)
This should give the super class an __eq__ attribute.
EDITED
The main idea worked, but it has been brought to my attention that the code isn't working. So: improved code:
class Derived(int):
def __eq__(self, other):
return self.real == other
Int = 5
D = Derived(Int)
D.__eq__(4) #Output: False
D.__eq__(5) #Output: True
using hasattr avoids creating a new int object, catching an exception or explicitly checking for the Python version.
The below code works on both Python 2.7 and 3.3+:
class Derived(int):
def __eq__(self, other):
return super(Derived, self).__cmp__(other) == 0 if hasattr(Derived, "__cmp__") else super(Derived, self).__eq__(other)

Elegant ways to support equivalence ("equality") in Python classes

When writing custom classes it is often important to allow equivalence by means of the == and != operators. In Python, this is made possible by implementing the __eq__ and __ne__ special methods, respectively. The easiest way I've found to do this is the following method:
class Foo:
def __init__(self, item):
self.item = item
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def __ne__(self, other):
return not self.__eq__(other)
Do you know of more elegant means of doing this? Do you know of any particular disadvantages to using the above method of comparing __dict__s?
Note: A bit of clarification--when __eq__ and __ne__ are undefined, you'll find this behavior:
>>> a = Foo(1)
>>> b = Foo(1)
>>> a is b
False
>>> a == b
False
That is, a == b evaluates to False because it really runs a is b, a test of identity (i.e., "Is a the same object as b?").
When __eq__ and __ne__ are defined, you'll find this behavior (which is the one we're after):
>>> a = Foo(1)
>>> b = Foo(1)
>>> a is b
False
>>> a == b
True
Consider this simple problem:
class Number:
def __init__(self, number):
self.number = number
n1 = Number(1)
n2 = Number(1)
n1 == n2 # False -- oops
So, Python by default uses the object identifiers for comparison operations:
id(n1) # 140400634555856
id(n2) # 140400634555920
Overriding the __eq__ function seems to solve the problem:
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return False
n1 == n2 # True
n1 != n2 # True in Python 2 -- oops, False in Python 3
In Python 2, always remember to override the __ne__ function as well, as the documentation states:
There are no implied relationships among the comparison operators. The
truth of x==y does not imply that x!=y is false. Accordingly, when
defining __eq__(), one should also define __ne__() so that the
operators will behave as expected.
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
return not self.__eq__(other)
n1 == n2 # True
n1 != n2 # False
In Python 3, this is no longer necessary, as the documentation states:
By default, __ne__() delegates to __eq__() and inverts the result
unless it is NotImplemented. There are no other implied
relationships among the comparison operators, for example, the truth
of (x<y or x==y) does not imply x<=y.
But that does not solve all our problems. Let’s add a subclass:
class SubNumber(Number):
pass
n3 = SubNumber(1)
n1 == n3 # False for classic-style classes -- oops, True for new-style classes
n3 == n1 # True
n1 != n3 # True for classic-style classes -- oops, False for new-style classes
n3 != n1 # False
Note: Python 2 has two kinds of classes:
classic-style (or old-style) classes, that do not inherit from object and that are declared as class A:, class A(): or class A(B): where B is a classic-style class;
new-style classes, that do inherit from object and that are declared as class A(object) or class A(B): where B is a new-style class. Python 3 has only new-style classes that are declared as class A:, class A(object): or class A(B):.
For classic-style classes, a comparison operation always calls the method of the first operand, while for new-style classes, it always calls the method of the subclass operand, regardless of the order of the operands.
So here, if Number is a classic-style class:
n1 == n3 calls n1.__eq__;
n3 == n1 calls n3.__eq__;
n1 != n3 calls n1.__ne__;
n3 != n1 calls n3.__ne__.
And if Number is a new-style class:
both n1 == n3 and n3 == n1 call n3.__eq__;
both n1 != n3 and n3 != n1 call n3.__ne__.
To fix the non-commutativity issue of the == and != operators for Python 2 classic-style classes, the __eq__ and __ne__ methods should return the NotImplemented value when an operand type is not supported. The documentation defines the NotImplemented value as:
Numeric methods and rich comparison methods may return this value if
they do not implement the operation for the operands provided. (The
interpreter will then try the reflected operation, or some other
fallback, depending on the operator.) Its truth value is true.
In this case the operator delegates the comparison operation to the reflected method of the other operand. The documentation defines reflected methods as:
There are no swapped-argument versions of these methods (to be used
when the left argument does not support the operation but the right
argument does); rather, __lt__() and __gt__() are each other’s
reflection, __le__() and __ge__() are each other’s reflection, and
__eq__() and __ne__() are their own reflection.
The result looks like this:
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return NotImplemented
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
x = self.__eq__(other)
if x is NotImplemented:
return NotImplemented
return not x
Returning the NotImplemented value instead of False is the right thing to do even for new-style classes if commutativity of the == and != operators is desired when the operands are of unrelated types (no inheritance).
Are we there yet? Not quite. How many unique numbers do we have?
len(set([n1, n2, n3])) # 3 -- oops
Sets use the hashes of objects, and by default Python returns the hash of the identifier of the object. Let’s try to override it:
def __hash__(self):
"""Overrides the default implementation"""
return hash(tuple(sorted(self.__dict__.items())))
len(set([n1, n2, n3])) # 1
The end result looks like this (I added some assertions at the end for validation):
class Number:
def __init__(self, number):
self.number = number
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Number):
return self.number == other.number
return NotImplemented
def __ne__(self, other):
"""Overrides the default implementation (unnecessary in Python 3)"""
x = self.__eq__(other)
if x is not NotImplemented:
return not x
return NotImplemented
def __hash__(self):
"""Overrides the default implementation"""
return hash(tuple(sorted(self.__dict__.items())))
class SubNumber(Number):
pass
n1 = Number(1)
n2 = Number(1)
n3 = SubNumber(1)
n4 = SubNumber(4)
assert n1 == n2
assert n2 == n1
assert not n1 != n2
assert not n2 != n1
assert n1 == n3
assert n3 == n1
assert not n1 != n3
assert not n3 != n1
assert not n1 == n4
assert not n4 == n1
assert n1 != n4
assert n4 != n1
assert len(set([n1, n2, n3, ])) == 1
assert len(set([n1, n2, n3, n4])) == 2
You need to be careful with inheritance:
>>> class Foo:
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
>>> class Bar(Foo):pass
>>> b = Bar()
>>> f = Foo()
>>> f == b
True
>>> b == f
False
Check types more strictly, like this:
def __eq__(self, other):
if type(other) is type(self):
return self.__dict__ == other.__dict__
return False
Besides that, your approach will work fine, that's what special methods are there for.
The way you describe is the way I've always done it. Since it's totally generic, you can always break that functionality out into a mixin class and inherit it in classes where you want that functionality.
class CommonEqualityMixin(object):
def __eq__(self, other):
return (isinstance(other, self.__class__)
and self.__dict__ == other.__dict__)
def __ne__(self, other):
return not self.__eq__(other)
class Foo(CommonEqualityMixin):
def __init__(self, item):
self.item = item
Not a direct answer but seemed relevant enough to be tacked on as it saves a bit of verbose tedium on occasion. Cut straight from the docs...
functools.total_ordering(cls)
Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:
The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.
New in version 2.7
#total_ordering
class Student:
def __eq__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
You don't have to override both __eq__ and __ne__ you can override only __cmp__ but this will make an implication on the result of ==, !==, < , > and so on.
is tests for object identity. This means a is b will be True in the case when a and b both hold the reference to the same object. In python you always hold a reference to an object in a variable not the actual object, so essentially for a is b to be true the objects in them should be located in the same memory location. How and most importantly why would you go about overriding this behaviour?
Edit: I didn't know __cmp__ was removed from python 3 so avoid it.
From this answer: https://stackoverflow.com/a/30676267/541136 I have demonstrated that, while it's correct to define __ne__ in terms __eq__ - instead of
def __ne__(self, other):
return not self.__eq__(other)
you should use:
def __ne__(self, other):
return not self == other
I think that the two terms you're looking for are equality (==) and identity (is). For example:
>>> a = [1,2,3]
>>> b = [1,2,3]
>>> a == b
True <-- a and b have values which are equal
>>> a is b
False <-- a and b are not the same list object
The 'is' test will test for identity using the builtin 'id()' function which essentially returns the memory address of the object and therefore isn't overloadable.
However in the case of testing the equality of a class you probably want to be a little bit more strict about your tests and only compare the data attributes in your class:
import types
class ComparesNicely(object):
def __eq__(self, other):
for key, value in self.__dict__.iteritems():
if (isinstance(value, types.FunctionType) or
key.startswith("__")):
continue
if key not in other.__dict__:
return False
if other.__dict__[key] != value:
return False
return True
This code will only compare non function data members of your class as well as skipping anything private which is generally what you want. In the case of Plain Old Python Objects I have a base class which implements __init__, __str__, __repr__ and __eq__ so my POPO objects don't carry the burden of all that extra (and in most cases identical) logic.
Instead of using subclassing/mixins, I like to use a generic class decorator
def comparable(cls):
""" Class decorator providing generic comparison functionality """
def __eq__(self, other):
return isinstance(other, self.__class__) and self.__dict__ == other.__dict__
def __ne__(self, other):
return not self.__eq__(other)
cls.__eq__ = __eq__
cls.__ne__ = __ne__
return cls
Usage:
#comparable
class Number(object):
def __init__(self, x):
self.x = x
a = Number(1)
b = Number(1)
assert a == b
This incorporates the comments on Algorias' answer, and compares objects by a single attribute because I don't care about the whole dict. hasattr(other, "id") must be true, but I know it is because I set it in the constructor.
def __eq__(self, other):
if other is self:
return True
if type(other) is not type(self):
# delegate to superclass
return NotImplemented
return other.id == self.id
I wrote a custom base with a default implementation of __ne__ that simply negates __eq__:
class HasEq(object):
"""
Mixin that provides a default implementation of ``object.__neq__`` using the subclass's implementation of ``object.__eq__``.
This overcomes Python's deficiency of ``==`` and ``!=`` not being symmetric when overloading comparison operators
(i.e. ``not x == y`` *does not* imply that ``x != y``), so whenever you implement
`object.__eq__ <https://docs.python.org/2/reference/datamodel.html#object.__eq__>`_, it is expected that you
also implement `object.__ne__ <https://docs.python.org/2/reference/datamodel.html#object.__ne__>`_
NOTE: in Python 3+ this is no longer necessary (see https://docs.python.org/3/reference/datamodel.html#object.__ne__)
"""
def __ne__(self, other):
"""
Default implementation of ``object.__ne__(self, other)``, delegating to ``self.__eq__(self, other)``.
When overriding ``object.__eq__`` in Python, one should also override ``object.__ne__`` to ensure that
``not x == y`` is the same as ``x != y``
(see `object.__eq__ <https://docs.python.org/2/reference/datamodel.html#object.__eq__>`_ spec)
:return: ``NotImplemented`` if ``self.__eq__(other)`` returns ``NotImplemented``, otherwise ``not self.__eq__(other)``
"""
equal = self.__eq__(other)
# the above result could be either True, False, or NotImplemented
if equal is NotImplemented:
return NotImplemented
return not equal
If you inherit from this base class, you only have to implement __eq__ and the base.
In retrospect, a better approach might have been to implement it as a decorator instead. Something like #functools.total_ordering

Categories

Resources