Can magic methods be overridden outside of a class?
When I do something like this
def __int__(x):
return x + 5
a = 5
print(int(a))
it prints '5' instead of '10'. Do I do something wrong or magic methods just can't be overridden outside of a class?
Short answer; not really.
You cannot arbitrarily change the behaviour of int() a builtin function (*which internally calls __int__()) on arbitrary builtin types such as int(s).
You can however change the behaviour of custom objects like this:
Example:
class Foo(object):
def __init__(self, value):
self.value = value
def __add__(self, other):
self.value += other
def __repr__(self):
return "<Foo(value={0:d})>".format(self.value)
Demo:
>>> x = Foo(5)
>>> x + 5
>>> x
<Foo(value=10)>
This overrides two things here and implements two special methods:
__repr__() which get called by repr()
__add__() which get called by the + operator.
Update: As per the comments above; techincally you can redefine the builtin function int; Example:
def int(x):
return x + 5
int(5) # returns 10
However this is not recommended and does not change the overall behaviour of the object x.
Update #2: The reason you cannot change the behaviour of bultin types (without modifying the underlying source or using Cuthon or ctypes) is because builtin types in Python are not exposed or mutable to the user unlike Homoiconic Languages (See: Homoiconicity). -- Even then I'm not really sure you can with Cython/ctypes; but the reason question is "Why do you want to do this?"
Update #3: See Python's documentation on Data Model (object.__complex__ for example).
You can redefine a top-level __int__ function, but nobody ever calls that.
As implied in the Data Model documentation, when you write int(x), that calls x.__int__(), not __int__(x).
And even that isn't really true. First, __int__ is a special method, meaning it's allowed to call type(x).__int__(x) rather than x.__int__(), but that doesn't matter here. Second, it's not required to call __int__ unless you give it something that isn't already an int (and call it with the one-argument form). So, it could be as if it's was written like this:
def int(x, base=None):
if base is not None:
return do_basey_stuff(x, base)
if isinstance(x, int):
return x
return type(x).__int__(x)
So, there is no way to change what int(5) will do… short of just shadowing the builtin int function with a different builtin/global/local function of the same name, of course.
But what if you wanted to, say, change int(5.5)? That's not an int, so it's going to call float.__int__(5.5). So, all we have to do is monkeypatch that, right?
Well, yes, except that Python allows builtin types to be immutable, and most of the builtin types in CPython are. So, if you try it:
>>> _real_float_int = float.__int__
>>> def _float_int(self):
... return _real_float_int(self) + 5
>>> _float_int(5.5)
10
>>> float.__int__ = _float_int
TypeError: can't set attributes of built-in/extension type 'float'
However, if you're defining your own types, that's a different story:
>>> class MyFloat(float):
... def __int__(self):
... return super().__int__() + 5
>>> f = MyFloat(5.5)
>>> int(f)
10
Related
Why do the following lines give me the same result?
str.upper('hello')
and
'hello'.upper()
I tried to do the same with list.append but got a TypeError.
list.append([1])
Is the str type in Python overloaded? How can this be achieved by writing a class/function? I would appreciate an example.
list.append takes two arguments - the list to modify and the element to append. So you need to do it like this:
ls = [1]
list.append(ls, 2)
which is equivalent to the much more popular:
ls.append(2)
str.upper and list.append are both functions.
str.upper takes one argument.
>>> str.upper('test')
'TEST'
list.append takes two arguments.
>>> my_list = []
>>> list.append(my_list, 1)
>>> my_list
[1]
str.upper and list.append (like other functions) are also non-data-descriptors with a __get__ method which in this context has two implications:
When you access the function through the class via the dot notation (str.upper, list.append) the function's __get__ method (i.e. string.upper.__get__ and list.append.__get__) is called but it returns just the function itself.
When you access the function through an instance (my_string.upper, my_list.append) the function's __get__ method is called and it will return a new callable acting like the original function, but with whatever was "in front of the dot" automatically passed as the first argument. .
That's why you need to pass 1 - 1 = 0 arguments when calling my_string.upper() and 2 - 1 = 1 argument when calling my_list.append(1).
>>> 'my_string'.upper()
'MY_STRING'
>>>
>>> my_list = []
>>> my_list.append(1)
>>> my_list
[1]
You could even get these modified callables (methods) by explicitly calling __get__ and passing the argument to be bound (what has been before the dot) as its argument.
>>> my_string = 'my_string'
>>> upper_maker = str.upper.__get__(my_string)
>>> upper_maker()
'MY_STRING'
>>>
>>> my_list = []
>>> appender = list.append.__get__(my_list)
>>> appender(1)
>>> my_list
[1]
Finally, here's a short example demonstrating how descriptor instances can detect whether they are being accessed via their owner-class or via an instance.
class Descriptor:
def __get__(self, instance, owner_class):
if instance is None:
print('accessed through class')
# list.append.__get__ would return list.append here
else:
print('accessed through instance')
# list.append.__get__ would build a new callable here
# that takes one argument x and that internally calls
# list.append(instance, x)
class Class:
attribute = Descriptor()
Class.attribute # prints 'accessed through class'
instance = Class()
instance.attribute # prints 'accessed through instance'
Quoting Dave Kirbys answer from Relationship between string module and str:
There is some overlap between the string module and the str type,
mainly for historical reasons. In early versions of Python str objects
did not have methods, so all string manipulation was done with
functions from the string module. When methods were added to the str
type (in Python 1.5?) the functions were left in the string module for
compatibility, but now just forward to the equivalent str method.
However the string module also contains constants and functions that
are not methods on str, such as formatting, character translation etc.
There is nothing at all magical going on with str (except that we have a nice syntactic shortcut to creating one using ""). You can write a class that behaves like str and list to see more clearly what is happening here.
class MyClass():
def __init__(self, arg):
self.val=str(arg)
def do_thing(self):
self.val = "asdf"
def do_thing_with_arg(self, arg):
self.val = "asdf " + str(arg)
def __repr__(self):
return self.val
my_thing = MyClass("qwerty")
# this is like 'hello'.upper()
my_thing.do_thing()
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like str.upper('hello')
MyClass.do_thing(my_thing)
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like my_list.append('qwerty')
my_thing.do_thing_with_arg('zxcv')
print(my_thing)
# it prints 'asdf zxcv'
my_thing = MyClass("qwerty")
# this is like list.append(my_list, 'qwerty')
MyClass.do_thing_with_arg(my_thing, 'zxcv')
print(my_thing)
# it prints 'asdf zxcv'
The short version is, you're invoking what looks like an "instance method" on a class, but you are supplying the instance ('self') yourself as the first argument to the function call.
Say I have a Python class as follows:
class TestClass():
value = 20
def __str__(self):
return str(self.value)
The __str__ method will automatically be called any time I try to use an instance of TestClass as a string, like in print. Is there any equivalent for treating it as a number? For example, in
an_object = TestClass()
if an_object > 30:
...
where some hypothetical __num__ function would be automatically called to interpret the object as a number. How could this be easily done?
Ideally I'd like to avoid overloading every normal mathematical operator.
You can provide __float__(), __int__(), and/or __complex__() methods to convert objects to numbers. There is also a __round__() method you can provide for custom rounding. Documentation here. The __bool__() method technically fits here too, since Booleans are a subclass of integers in Python.
While Python does implicitly convert objects to strings for e.g. print(), it never converts objects to numbers without you saying to. Thus, Foo() + 42 isn't valid just because Foo has an __int__ method. You have to explicitly use int() or float() or complex() on them. At least that way, you know what you're getting just by reading the code.
To get classes to actually behave like numbers, you have to implement all the special methods for the operations that numbers participate in, including arithmetic and comparisons. As you note, this gets annoying. You can, however, write a mixin class so that at least you only have to write it once. Such as:
class NumberMixin(object):
def __eq__(self, other): return self.__num__() == self.__getval__(other)
# other comparison methods
def __add__(self, other): return self.__num__() + self.__getval__(other)
def __radd__(self, other): return self.__getval__(other) + self.__num__()
# etc., I'm not going to write them all out, are you crazy?
This class expects two special methods on the class it's mixed in with.
__num__() - converts self to a number. Usually this will be an alias for the conversion method for the most precise type supported by the object. For example, your class might have __int__() and __float__() methods, but __int__() will truncate the number, so you assign __num__ = __float__ in your class definition. On the other hand, if your class has a natural integral value, you might want to provide __float__ so it can also be converted to a float, but you'd use __num__ = __int__ since it should behave like an integer.
__getval__() - a static method that obtains the numeric value from another object. This is useful when you want to be able to support operations with objects other than numeric types. For example, when comparing, you might want to be able to compare to objects of your own type, as well as to traditional numeric types. You can write __getval__() to fish out the right attribute or call the right method of those other objects. Of course with your own instances you can just rely on float() to do the right thing, but __getval__() lets you be as flexible as you like in what you accept.
A simple example class using this mixin:
class FauxFloat(NumberMixin):
def __init__(self, value): self.value = float(value)
def __int__(self): return int(self.value)
def __float__(self): return float(self.value)
def __round__(self, digits=0): return round(self.value, digits)
def __str__(self): return str(self.value)
__repr__ = __str__
__num__ = __float__
#staticmethod
def __getval__(obj):
if isinstance(obj, FauxFloat):
return float(obj)
if hasattr(type(obj), "__num__") and callable(type(obj).__num__):
return type(obj).__num__(obj) # don't call dunder method on instance
try:
return float(obj)
except TypeError:
return int(obj)
ff = FauxFloat(42)
print(ff + 13) # 55.0
For extra credit, you could register your class so it'll be seen as a subclass of an appropriate abstract base class:
import numbers
numbers.Real.register(FauxFloat)
issubclass(FauxFloat, numbers.Real) # True
For extra extra credit, you might also create a global num() function that calls __num__() on objects that have it, otherwise falling back to the older methods.
In case of numbers it a bit more complicated. But its possible! You have to override your class operators to fit your needs.
operator.__lt__(a, b) # lower than
operator.__le__(a, b) # lower equal
operator.__eq__(a, b) # equal
operator.__ne__(a, b) # not equal
operator.__ge__(a, b) # greater equial
operator.__gt__(a, b) # greater than
Python Operators
Looks like you need __gt__ method.
class A:
val = 0
def __gt__(self, other):
return self.val > other
a = A()
a.val = 12
a > 10
If you just wanna cast object to int - you should define __int__ method (or __float__).
Without print (which I believe invokes str()) what happens when a variable is on a line by itself.
This is a bit contrived, I know, but I ran into this in a Jupyter notebook when testing a class I'm creating and now I'm curious. I can't seem to find the right set of Google search terms to find the answer in the docs.
I defined a class thusly:
class ExceptionList(BaseException):
pass
# I've implemented, in very standard ways, the following methods
# __str__()
# __getitem__()
# __delitem__()
# __repr__()
# I doubt any other specifics of the class are pertinent
EDIT
Here is the repr() implementation:
def __repr__(self):
return "{}({})".format(self.__class__.__name__, repr(self.__exception_list))
P.S. I coded that method based on http://brennerm.github.io/posts/python-str-vs-repr.html
EDIT
My implementation of repr() causes this behavior:
e = ExceptionList(["Oh, no"])
e
"ExceptionList(['Oh, no'])"
So consider:
e1 = Exception("Oh no!")
e2 = ExceptionList("Oh no!")
In separate notebook cells:
e1
Exception('Oh no!')
e2
__main__.ExceptionList()
Incidentally (maybe?) the output of:
e2.__class__
is close:
__main__.ExceptionList
Does it just have something to do with the scope in which the class was defined? Is it some special behavior of builtins?
Is this behavior this result of invoking some method that I'm unaware of? I tried implementing all of the methods produced with dir() though I'm willing to bet that's not exhaustive.
It probably doesn't matter for my implementation but now I need to know!
Boilerplate Hater Deterrent:
I don't know anything.
I'm a terrible programmer.
"I thought a python was a snake..."
I'm barely qualified to use a toaster.
Please forgive this post's pathetic usage of SO disk space.
If the sole content of a line is a variable or object, then that line is evaluated and the variable or object is returned and not stored to a variable.
a = 5 + 2 # 5+2 is evaluated and stored in a
5 + 2 # 5+2 is evaluated
a # a is evaluated
class SomeCustomClass:
def __init__(self, *args):
pass
scc = SomeCustomClass()
scc # returns a reference to this instance, but ref is not stored
Many python interfaces like Jupyter, IPython, IDLE will display the evaluation of the line.
>>> 5+2
7
>>> a=5+2
[Nothing]
>>> a
7
>>> scc
<__main__.SomeCustomClass instance at 0x7fb6d5943d40>
The <__main__.SomeCustomClass instance at 0x7fb6d5943d40> is called a representation of the class. If you look at Python's Data Model you will see that this representation is specified by the object's __repr__ method.
Modifying SomeCustomClass:
class SomeCustomClass:
def __init__(self, *args):
pass
def __repr__(self):
return "SomeCustomClass repr"
>>> scc = SomeCustomClass()
>>> scc
SomeCustomClass repr
>>> s = str(scc) #implicit call to __repr__
>>> s
SomeCustomClass repr
Now adding __str__ method:
class SomeCustomClass:
def __init__(self, *args):
pass
def __repr__(self):
return "SomeCustomClass repr"
def __str__(self):
return "SomeCustomClass str"
>>> scc = SomeCustomClass()
>>> scc
SomeCustomClass repr
>>> s = str(scc) #explicit call to __str__
>>> s
SomeCustomClass str
This question already has answers here:
Purpose of __repr__ method?
(6 answers)
Closed 5 years ago.
I am trying to learn python with my own and i stucked at __repr__ function. Though i have read lots of post on __repr__ along with the python document. so i have decided to ask this Question here. The code bellow explains my confusion.
class Point:
def __init__(self,x,y):
self.x, self.y = x,y
def __repr__(self):
return 'Point(x=%s, y=%s)'%(self.x, self.y)
def print_class(self):
return 'Point(x=%s, y=%s)'%(self.x, self.y)
p = Point(1,2)
print p
print p.print_class()
Point(x=1, y=2)
Point(x=1, y=2)
If a normal function can also perform similar task then what is the extra advantage of __repr__ over print_class() (in my case a normal function) function.
The __repr__ function is called by repr() internally. repr() is called when you are printing the object directly , and the class does not define a __str__() . From documentation -
object.__repr__(self)
Called by the repr() built-in function and by string conversions (reverse quotes) to compute the “official” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form <...some useful description...> should be returned. The return value must be a string object. If a class defines __repr__() but not __str__(), then __repr__() is also used when an “informal” string representation of instances of that class is required.
In your case for print_class() , you have to specifically call the method when printing the object. But in case of __repr__() , it gets internally called by print .
This is especially useful, when you are mixing different classes/types . For Example lets take a list which can have numbers and objects of your point class, now you want to print the elements of the list.
If you do not define the __repr__() or __str__() , you would have to first check the instance , whether its of type Point if so call print_class() , or if not directly print the number.
But when your class defines the __repr__() or __str__() , you can just directly call print on all the elements of the list, print statement would internally take care of printing the correct values.
Example , Lets assume a class which has print_class() method, but no __repr__() or __str__() , code -
>>> class CA:
... def __init__(self,x):
... self.x = x
... def print_class(self):
... return self.x
...
>>> l = [1,2,3,CA(4),CA(5)]
>>> for i in l:
... print(i)
...
1
2
3
<__main__.CA object at 0x00590F10>
<__main__.CA object at 0x005A5070>
SyntaxError: invalid syntax
>>> for i in l:
... if isinstance(i, CA):
... print(i.print_class())
... else:
... print(i)
...
1
2
3
4
5
As you can see, when we mix numbers and objects of type CA in the list, and then when we just did print(i) , it did not print what we wanted. For this to work correctly, we had to check the type of i and call the appropriate method (as done in second case).
Now lets assume a class that implements __repr__() instead of print_class() -
>>> class CA:
... def __init__(self,x):
... self.x = x
... def __repr__(self):
... return str(self.x)
...
>>>
>>> l = [1,2,3,CA(4),CA(5)]
>>> for i in l:
... print(i)
...
1
2
3
4
5
As you can see in second case, simply printing worked, since print internally calls __str__() first, and as that did not exist fell back to __repr__() .
And not just this, when we do str(list) , internally each list's element's __repr__() is called. Example -
First case (without __repr__() ) -
>>> str(l)
'[1, 2, 3, <__main__.CA object at 0x005AB3D0>, <__main__.CA object at 0x005AB410>]'
Second case (with __repr__() ) -
>>> str(l)
'[1, 2, 3, 4, 5]'
Also, in interactive interpreter, when you are directly using the object, it shows you the output of repr() function, Example -
>>> class CA:
... def __repr__(self):
... return "CA instance"
...
>>>
>>> c = CA()
>>> c
CA instance
The difference is that the __repr__ function is automatically called by Python in certain contexts, and is part of a predefined API with specific requirements. For instance, if you enter p by itself(not print p) in the interactive shell after creating your p object, its __repr__ will be called. It will also be used for print p if you don't define a __str__on p. (That is, you had to write print p.print_class(), but you didn't have to write print p.__repr__(); Python called __repr__ automatically for you.) The requirements for __repr__ are described in the documentation:
Called by the repr() built-in function and by string conversions (reverse quotes) to compute the “official” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form <...some useful description...> should be returned.
In short, if you write your own method called print_class you can make it do whatever you want and tell people how to use it, because it's your API. If you use __repr__ you're supposed to follow the conventions of Python's API. Either one may make sense depending on the context.
It helps you do more efficient coding work. even though you get same result using user define method like 'print_class()' as repr, but you don't need to type in '.print_class()' by repr method.
Say I have this class:
class MyString(str):
def someExtraMethod(self):
pass
I'd like to be able to do
a = MyString("Hello ")
b = MyString("World")
(a + b).someExtraMethod()
("a" + b).someExtraMethod()
(a + "b").someExtraMethod()
Running as is:
AttributeError: 'str' object has no attribute 'someExtraMethod'
Obviously that doesn't work. So I add this:
def __add__(self, other):
return MyString(super(MyString, self) + other)
def __radd__(self, other):
return MyString(other + super(MyString, self))
TypeError: cannot concatenate 'str' and 'super' objects
Hmmm, ok. super doesn't appear to respect operator overloading. Perhaps:
def __add__(self, other):
return MyString(super(MyString, self).__add__(other))
def __radd__(self, other):
return MyString(super(MyString, self).__radd__(other))
AttributeError: 'super' object has no attribute '__radd__'
Still no luck. What should I be doing here?
I usually use the super(MyClass, self).__method__(other) syntax, in your case this does not work because str does not provide __radd__. But you can convert the instance of your class into a string using str.
To who said that its version 3 works: it does not:
>>> class MyString(str):
... def __add__(self, other):
... print 'called'
... return MyString(super(MyString, self).__add__(other))
...
>>> 'test' + MyString('test')
'testtest'
>>> ('test' + MyString('test')).__class__
<type 'str'>
And if you implement __radd__ you get an AttributeError(see the note below).
Anyway, I'd avoid using built-ins as base type. As you can see some details may be tricky, and also you must redefine all the operations that they support, otherwise the object will become an instance of the built-in and not an instance of your class.
I think in most cases it's easier to use delegation instead of inheritance.
Also, if you simply want to add a single method, then you may try to use a function on plain strings instead.
I add here a bit of explanation about why str does not provide __radd__ and what's going on when python executes a BINARY_ADD opcode(the one that does the +).
The abscence of str.__radd__ is due to the fact that string objects implements the concatenation operator of sequences and not the numeric addition operation. The same is true for the other sequences such as list or tuple. These are the same at "Python level" but actually have two different "slots" in the C structures.
The numeric + operator, which in python is defined by two different methods(__add__ and __radd__) is actually a single C function that is called with swapped arguments to simulate a call to __radd__.
Now, you could think that implementing only MyString.__add__ would fix your problem, since str does not implement __radd__, but that's not true:
>>> class MyString(str):
... def __add__(self, s):
... print '__add__'
... return MyString(str(self) + s)
...
>>> 'test' + MyString('test')
'testtest'
As you can see MyString.__add__ is not called, but if we swap arguments:
>>> MyString('test') + 'test'
__add__
'testtest'
It is called, so what's happening?
The answer is in the documentation which states that:
For objects x and y, first x.__op__(y) is tried. If this is not
implemented or returns NotImplemented, y.__rop__(x) is tried. If this
is also not implemented or returns NotImplemented, a TypeError
exception is raised. But see the following exception:
Exception to the previous item: if the left operand is an instance of
a built-in type or a new-style class, and the right operand is an
instance of a proper subclass of that type or class and overrides the
base’s __rop__() method, the right operand’s __rop__() method is tried
before the left operand’s __op__() method.
This is done so that a subclass can completely override binary
operators. Otherwise, the left operand’s __op__() method would always
accept the right operand: when an instance of a given class is
expected, an instance of a subclass of that class is always
acceptable.
Which means you must implement all the methods of str and the __r*__ methods, otherwise you'll still have problems with argument order.
May be:
class MyString(str):
def m(self):
print(self)
def __add__(self, other):
return MyString(str(self) + other)
def __radd__(self, other):
return MyString(other + str(self))
a = MyString("Hello ")
b = MyString("World")
(a + b).m()
Update: you last version with super works for me