I would like to assign a function as class attribute, and have it so when accessed through instance, it is still unbounded. I understand that this can be achieved with using staticmethod descriptor. But it seems the behavior is different for the builtin functions, and I would like to replicate that.
def abs_(value):
return abs(value)
class Test:
func_1 = abs
func_2 = len
func_3 = abs_
func_4 = staticmethod(abs_)
>>> test = Test()
>>> test.func_1
<built-in function abs>
>>> test.func_2
<built-in function len>
>>> test.func_3
<bound method abs_ of <__main__.Test object at 0x10436d910>>
In this case, the builtin function are unbound, and the defined function abs_ is bound to the instance. And obviously all functions work except func_3 since it is bound method.
>>> test.func_1(-1)
1
>>> test.func_3(-1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: abs_() takes 1 positional argument but 2 were given
How does builtin function achieve this, and is there a way to replicate the behavior (remain unbound)? Thank you!
Related
how to call a function from a function object
a = fun_name
print(a)
like from this function object that I get how can I call the original function only from using this function object
<function fun_name at 0x00000265C9B0E320>
You just call the object, since it's just a function object, not really but it refers to it, example:
def a():
return 'a'
b = a
print(b())
You can call the original function like a() in this case.
I have defined a class where its __ge__ method returns an instance of itself, and whose __bool__ method is not allowed to be invoked (similar to a Pandas Series).
Why is X.__bool__ invoked during np.int8(0) <= x, but not for any of the other examples? Who is invoking it? I have read the Data Model docs but I haven’t found my answer there.
import numpy as np
import pandas as pd
class X:
def __bool__(self):
print(f"{self}.__bool__")
assert False
def __ge__(self, other):
print(f"{self}.__ge__")
return X()
x = X()
np.int8(0) <= x
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5D90>.__bool__
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# File "<stdin>", line 4, in __bool__
# AssertionError
0 <= x
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5DF0>
x >= np.int8(0)
# Console output:
# <__main__.X object at 0x000001BAC70D5C70>.__ge__
# <__main__.X object at 0x000001BAC70D5D30>
pd_ge = pd.Series.__ge__
def ge_wrapper(self, other):
print("pd.Series.__ge__")
return pd_ge(self, other)
pd.Series.__ge__ = ge_wrapper
pd_bool = pd.Series.__bool__
def bool_wrapper(self):
print("pd.Series.__bool__")
return pd_bool(self)
pd.Series.__bool__ = bool_wrapper
np.int8(0) <= pd.Series([1,2,3])
# Console output:
# pd.Series.__ge__
# 0 True
# 1 True
# 2 True
# dtype: bool
I suspect that np.int8.__le__ is defined so that instead of returning NotImplemented and letting X.__ge__ take over, it instead tries to return something like not (np.int(8) > x), and then np.int8.__gt__ raises NotImplemented. Once X.__gt__(x, np.int8(0)) returns an instance of X rather than a Boolean value, then we need to call x.__bool__() in order to compute the value of not x.
(Still trying to track down where int8.__gt__ is defined to confirm.)
(Update: not quite. int8 uses a single generic rich comparison function that simply converts the value to a 0-dimensional array, then returns the result of PyObject_RichCompare on the array and x.)
I did find this function that appears to ultimately implement np.int8.__le__:
static NPY_INLINE int
rational_le(rational x, rational y) {
return !rational_lt(y,x);
}
It's not clear to me how we avoid getting to this function if one of the arguments (like X) would not be a NumPy type. I think I give up.
TL;DR
X.__array_priority__ = 1000
The biggest hint is that it works with a pd.Series.
First I tried having X inherit from pd.Series. This worked (i.e. __bool__ no longer called).
To determine whether NumPy is using an isinstance check or duck-typing approach, I removed the explicit inheritance and added (based on this answer):
#property
def __class__(self):
return pd.Series
The operation no longer worked (i.e. __bool__ was called).
So now I think we can conclude NumPy is using a duck-typing approach. So I checked to see what attributes are being accessed on X.
I added the following to X:
def __getattribute__(self, item):
print("getattr", item)
return object.__getattribute__(self, item)
Again instantiating X as x, and invoking np.int8(0) <= x, we get:
getattr __array_priority__
getattr __array_priority__
getattr __array_priority__
getattr __array_struct__
getattr __array_interface__
getattr __array__
getattr __array_prepare__
<__main__.X object at 0x000002022AB5DBE0>.__ge__
<__main__.X object at 0x000002021A73BE50>.__bool__
getattr __array_struct__
getattr __array_interface__
getattr __array__
Traceback (most recent call last):
File "<stdin>", line 32, in <module>
np.int8(0) <= x
File "<stdin>", line 21, in __bool__
assert False
AssertionError
Ah-ha! What is __array_priority__? Who cares, really. With a little digging, all we need to know is that NDFrame (from which pd.Series inherits) sets this value as 1000.
If we add X.__array_priority__ = 1000, it works! __bool__ is no longer called.
What made this so difficult (I believe) is that the NumPy code didn't show up in the call stack because it is written in C. I could investigate further if I tried out the suggestion here.
I failed to phrase the question correctly online so I could not get an answer.
We have instance methods, static methods, and class methods.
What are functions called when they don't belong to a class?
They're just called functions.
In python, "function" refers to a type of callable procedure/block of code with its own localized namespace.
In contrast, "method" refers specifically to a kind of function that is bound to a class. We use "instance methods", "static methods", and "class methods" to differentiate between how those functions are bound to their respective classes, but in any case we call them methods because they are bound to their class.
So, we just call them functions, unless we have something more specific. If you must use some sort of qualifier, "unbound function" (alluding to the fact that it's not bound to any class) or "module function" (alluding to the fact that it's bound to the module it's defined in, though that's not a class), or even "static function" (but this could be confusing when trying to communicate to people who don't know the difference between functions and methods) or "free function" will probably work.
According to the section "Callable types" in the Python docs on its Data model
A user-defined function object is created by a function definition
So I guess one could say that everything that begins with def is a function.
In general, I think it depends a lot on the context, which term you want to use. For example, even though, to define a "static method", you'd write #staticmethod and everything, it's not called a "method" but a "function" in the context of the types module:
>>> class A:
... def f(self):
... print("Hello from method `f`")
... #staticmethod
... def g():
... print("Hello from function `g`")
...
>>> a = A()
>>> a.f()
Hello from method `f`
>>> a.g()
Hello from function `g`
>>> type(a.f)
<class 'method'>
>>> type(a.g)
<class 'function'>
Furthermore, looking at the docs of Python's types module reveals:
types.MethodType - The type of methods of user-defined class instances.
So methods are only found in instances. A method b.f of an instance b of a class B refers to the function B.f of the class:
>>> class B:
... def f(self):
... pass
...
>>> b1 = B()
>>> type(b1.f)
<class 'method'>
>>> type(B.f)
<class 'function'>
The methods are different objects for each instance:
>>> b2 = B()
>>> b1.f is b2.f
False
However, the methods refer to the same function B.f:
>>> b1.f.__func__
<function B.f at 0x7f166e31b2f0>
>>> b1.f.__func__ is b2.f.__func__
True
I imagine, this can be both useful or a pitfall, so it makes sense to know about it.
Here is an example, using a class C with function C.f and cache, an argument with a mutable default value:
>>> class C:
... def f(self, cache=[]):
... cache.append(cache[-1] + 1 if cache else 1)
... print(cache)
...
>>> c1 = C()
>>> c2 = C()
>>> c1.f()
[1]
>>> c2.f()
[1, 2]
>>> c3 = C()
>>> c3.f()
[1, 2, 3]
As you can see, all instances c1, c2 and c3 of class C share the same underlying function C.f with its argument cache.
This question already has answers here:
How should functions be tested for equality or identity?
(4 answers)
Closed 2 years ago.
I have a variable that contains a function.
def foo(x):
return x + 42
def bar(x):
return x - 42
my_var = foo
I want to check if that function is a certain function. Should I use is or ==?
my_var == foo
my_var == bar
and
my_var is foo
my_var is bar
both return what I expect.
They are the same thing for a function object. The == operator calls the __eq__ function to perform the comparison. The function object does not define an __eq__ method:
>>> def foo():
... pass
...
>>> foo.__eq__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute '__eq__'
Therefore the interpreter uses the default implementation in PyObject:
res = (v == w) ? Py_True : Py_False;
which is basically a pointer comparison, essentially the same as is.
No, you should use ==.
A good rule of thumb is only use is when doing is None or is not None and nowhere else. In this case, comparing functions with is works, but if you try to compare a method of an instance of a class with itself you'll get False (even on the same instance), whereas comparing with == returns what you expect:
>>> class A:
... def my_method(self):
... pass
...
>>> a = A()
>>> a.my_method is a.my_method
False
>>> a.my_method == a.my_method
True
Better to avoid having to remember this and always compare functions with ==.
See this question: Why don't methods have reference equality?
Is checks for the identity of an object. If you assign foo to myvar, then an alias is created and they both have the same id (in the case of functions at least).
Checking 2 functions for equivalence from a mathematical standpoint entails checking the equivalence of the domains and codomains of both functions.
So is is better.
Consider the following (broken) code:
import functools
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partial(f,1)
x=Foo()
x.g(2)
What I want to do is take the function f and partially apply it, resulting in a function g(self,b). I would like to use this function as a method, however this does not currently work and instead I get the error
Traceback (most recent call last):
File "test.py", line 8, in <module>
x.g(2)
TypeError: f() takes exactly 3 arguments (2 given)
Doing x.g(x,2) however works, so it seem the issue is that g is considered a "normal" function instead of a method of the class. Is there a way to get x.g to behave like a method (i.e implicitly pass the self parameter) instead of a function?
There are two issues at hand here. First, for a function to be turned into a method it must be stored on the class, not the instance. A demonstration:
class Foo(object):
def a(*args):
print 'a', args
def b(*args):
print 'b', args
Foo.b = b
x = Foo()
def c(*args):
print 'c', args
x.c = c
So a is a function defined in the class definition, b is a function assigned to the class afterwards, and c is a function assigned to the instance. Take a look at what happens when we call them:
>>> x.a('a will have "self"')
a (<__main__.Foo object at 0x100425ed0>, 'a will have "self"')
>>> x.b('as will b')
b (<__main__.Foo object at 0x100425ed0>, 'as will b')
>>> x.c('c will only recieve this string')
c ('c will only recieve this string',)
As you can see there is little difference between a function defined along with the class, and one assigned to it later. I believe there is actually no difference as long as there is no metaclass involved, but that is for another time.
The second problem comes from how a function is actually turned into a method in the first place; the function type implements the descriptor protocol. (See the docs for details.) In a nutshell, the function type has a special __get__ method which is called when you perform an attribute lookup on the class itself. Instead of you getting the function object, the __get__ method of that function object is called, and that returns a bound method object (which is what supplies the self argument).
Why is this a problem? Because the functools.partial object is not a descriptor!
>>> import functools
>>> def f(*args):
... print 'f', args
...
>>> g = functools.partial(f, 1, 2, 3)
>>> g
<functools.partial object at 0x10042f2b8>
>>> g.__get__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'functools.partial' object has no attribute '__get__'
There are a number of options you have at this point. You can explicitly supply the self argument to the partial:
import functools
class Foo(object):
def __init__(self):
def f(self, a, b):
print a + b
self.g = functools.partial(f, self, 1)
x = Foo()
x.g(2)
...or you would imbed the self and value of a in a closure:
class Foo(object):
def __init__(self):
a = 1
def f(b):
print a + b
self.g = f
x = Foo()
x.g(2)
These solutions are of course assuming that there is an as yet unspecified reason for assigning a method to the class in the constructor like this, as you can very easily just define a method directly on the class to do what you are doing here.
Edit: Here is an idea for a solution assuming the functions may be created for the class, instead of the instance:
class Foo(object):
pass
def make_binding(name):
def f(self, *args):
print 'Do %s with %s given %r.' % (name, self, args)
return f
for name in 'foo', 'bar', 'baz':
setattr(Foo, name, make_binding(name))
f = Foo()
f.foo(1, 2, 3)
f.bar('some input')
f.baz()
Gives you:
Do foo with <__main__.Foo object at 0x10053e3d0> given (1, 2, 3).
Do bar with <__main__.Foo object at 0x10053e3d0> given ('some input',).
Do baz with <__main__.Foo object at 0x10053e3d0> given ().
This will work. But I'm not sure if this is what you are looking for
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partial(f,1, self) # <= passing `self` also.
x = Foo()
x.g(2)
this is simply a concrete example of what i believe is the most correct (and therefore pythonic :) way to solve -- as the best solution (definition on a class!) was never revealed -- #MikeBoers explanations are otherwise solid.
i've used this pattern quite a bit (recently for an proxied API), and it's survived untold production hours without the slightest irregularity.
from functools import update_wrapper
from functools import partial
from types import MethodType
class Basic(object):
def add(self, **kwds):
print sum(kwds.values())
Basic.add_to_one = MethodType(
update_wrapper(partial(Basic.add, a=1), Basic.add),
None,
Basic,
)
x = Basic()
x.add(a=1, b=9)
x.add_to_one(b=9)
...yields:
10
10
...the key take-home-point here is MethodType(func, inst, cls), which creates an unbound method from another callable (you can even use this to chain/bind instance methods to unrelated classes... when instantiated+called the original instance method will receive BOTH self objects!)
note the exclusive use of keyword arguments! while there might be a better way to handle, args are generally a PITA because the placement of self becomes less predictable. also, IME anyway, using *args, and **kwds in the bottom-most function has proven very useful later on.
functools.partialmethod() is available since python 3.4 for this purpose.
import functools
class Foo(object):
def __init__(self):
def f(a,self,b):
print a+b
self.g = functools.partialmethod(f,1)
x=Foo()
x.g(2)