When is using __call__ a good idea? - python

What are peoples' opinions on using the __call__. I've only very rarely seen it used, but I think it's a very handy tool to use when you know that a class is going to be used for some default behaviour.

I think your intuition is about right.
Historically, callable objects (or what I've sometimes heard called "functors") have been used in the OO world to simulate closures. In C++ they're frequently indispensable.
However, __call__ has quite a bit of competition in the Python world:
A regular named method, whose behavior can sometimes be a lot more easily deduced from the name. Can convert to a bound method, which can be called like a function.
A closure, obtained by returning a function that's defined in a nested block.
A lambda, which is a limited but quick way of making a closure.
Generators and coroutines, whose bodies hold accumulated state much like a functor can.
I'd say the time to use __call__ is when you're not better served by one of the options above. Check the following criteria, perhaps:
Your object has state.
There is a clear "primary" behavior for your class that's kind of silly to name. E.g. if you find yourself writing run() or doStuff() or go() or the ever-popular and ever-redundant doRun(), you may have a candidate.
Your object has state that exceeds what would be expected of a generator function.
Your object wraps, emulates, or abstracts the concept of a function.
Your object has other auxilliary methods that conceptually belong with your primary behavior.
One example I like is UI command objects. Designed so that their primary task is to execute the comnand, but with extra methods to control their display as a menu item, for example, this seems to me to be the sort of thing you'd still want a callable object for.

Use it if you need your objects to be callable, that's what it's there for
I'm not sure what you mean by default behaviour
One place I have found it particularly useful is when using a wrapper or somesuch where the object is called deep inside some framework/library.

More generally, Python has a lot of double-underscore methods. They're there for a reason: they are the Python way of overloading operators. For instance, if you want a new class in which addition, I don't know, prints "foo", you define the __add__ and __radd__ methods. There's nothing inherently good or bad about this, any more than there's anything good or bad about using for loops.
In fact, using __call__ is often the more Pythonic approach, because it encourages clarity of code. You could replace MyCalculator.calculateValues( foo ) with MyCalculator( foo ), say.

Its usually used when class is used as function with some instance context, like some DecoratorClass which would be used as #DecoratorClass('some param'), so 'some param' would be stored in the instance's namespace and then instance being called as actual decorator.
It is not very useful when your class provides some different methods, since its usually not obvious what would the call do, and explicit is better than implicit in these cases.

Related

Python method call syntax shorthand

Is there a reason we call methods in python like object.method instead of Class.method(object)?
Maybe it isn't a strange choice, but personally it made understanding the self parameter much easier when I was shown the second way of calling a method.
Hardcoding the class name basically prevents you from using polymorphism. This is general OOP, not particularly a Python feature.
Your calling code should not need to know, nor care, which exact class object is.
This is immediately a problem for code where object can be a member of either Baseclass or Derivedclass, but much more complex inheritance and method overriding scenarios are possible, and sometimes necessary.

Is it conventional to say that functions are called and methods are invoked?

I’m reading Think Python: How to Think Like a Computer Scientist. The author uses “invoke” with methods and “call” with functions.
Is it a convention? And, if so, why is this distinction made? Why are functions said to be called, but methods are said to be invoked?
Not really, maybe it is easier for new readers to make an explicit distinction in order to understand that their invocation is slightly different. At least that why I suspect the author might have chosen different wording for each.
There doesn't seem to be a convention that dictates this in the Reference Manual for the Python language. What I seem them doing is choosing invoke when the call made to a function is implicit and not explicit.
For example, in the Callables section of the Standard Type Hierarchy you see:
[..] When an instance method object is called, the underlying function (__func__) is called, inserting the class instance (__self__) in front of the argument list. [...]
(Emphasis mine) Explicit call
Further down in Basic Customization and specifically for __new__ you can see:
Called to create a new instance of class cls. __new__() is a static method [...]
(Emphasis mine) Explicit call
While just a couple of sentences later you'll see how invoked is used because __new__ implicitly calls __init__:
If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked.
(Emphasis mine) Implicitly called
So no, no convention seems to be used, at least by the creators of the language. Simple is better than complex, I guess :-).
One good source for this would be the Python documentation. A simple text search through the section on Classes reveals the word "call" being used many times in reference to "calling methods", and the word "invoke" being used only once.
In my experience, the same is true: I regularly hear "call" used in reference to methods and functions, while I rarely hear "invoke" for either. However, I assume this is mainly a matter of personal preference and/or context (is the setting informal?, academic?, etc.).
You will also see places in the documentation where the word "invoke" is used in refernce to functions:
void Py_FatalError(const char *message)
Print a fatal error message
and kill the process. No cleanup is performed. This function should
only be invoked when a condition is detected that would make it
dangerous to continue using the Python interpreter; e.g., when the
object administration appears to be corrupted. On Unix, the standard C
library function abort() is called which will attempt to produce a
core file.
And from here:
void Py_DECREF(PyObject *o)
Decrement the reference count for object o. The object must not be NULL; if you aren’t sure that it isn’t NULL,
use Py_XDECREF(). If the reference count reaches zero, the object’s
type’s deallocation function (which must not be NULL) is invoked.
Although both these references are from the Python C API, so that may be significant.
To summerize:
I think it is safe to use either "invoke" or "call" in the context of functions or methods without sounding either like a noob or a showoff.
Note that I speak only of Python, and what I know from my own experience. I cannot speak to the difference between these terms in other languages.

Why `type(x).__enter__(x)` instead of `x.__enter__()` in Python standard contextlib?

In contextlib.py, I see the ExitStack class is calling __enter__() method via the type object (type(cm)) instead of direct method calls to the given object (cm).
I wonder why or why not.
e.g.,
does it give better exception traces when an error occurs?
is it just specific to some module author's coding style?
does it have any performance benefits?
does it avoid some artifacts/side-effects with complicated type hierarchies?
First of all, this is what happens when you do with something, it's not just contextlib that looks up special method on the type. Also, it's worth noting that the same happens with other special methods too: e.g. a + b results in type(a).__add__(a, b).
But why does it happen? This is a question that is often fired up on the python-dev and python-ideas mailing lists. And when I say "often", I mean "very often".
The last one were these: Missing Core Feature: + - * / | & do not call getattr and Eliminating special method lookup.
Here are some interesting points:
The current behaviour is by design - special methods are looked up as
slots on the object's class, not as instance attributes. This allows
the interpreter to bypass several steps in the normal instance
attribute lookup process.
(Source)
It is worth noting that the behavior is even more magical than this.
Even when looked up on the class, implicit special method lookup
bypasses __getattr__ and __getattribute__ of the metaclass. So the
special method lookup is not just an ordinary lookup that happens to
start on the class instead of the instance; it is a fully magic lookup
that does not engage the usual attribute-access-customization hooks at
any level.
(Source)
This behavior is also documented on the reference documentation: Special method lookup, which says:
Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).
In short, performance is the main concern. But let's take a closer look at this.
What's the difference between type(obj).__enter__() and obj.__enter__()?
When you write obj.attr, type(obj).__getattribute__('attr') gets called. The default implementation of __getattribute__() looks for attr into the instance dictionary (i.e. obj.__dict__) and into the class namespace and, failing that, calls type(obj).__getattr__('attr').
Now, this was a quick explanation and I have omitted some details, however it should give you an idea of how complicated an attribute lookup can be, and how slow it can become. Short circuiting special method lookup surely provides performance improvements, as looking up obj.__enter__() in the "classical" way may be too slow.

Should methods in a class be classmethod by default?

I was just working on a large class hierarchy and thought that probably all methods in a class should be classmethods by default.
I mean that it is very rare that one needs to change the actual method for an object, and whatever variables one needs can be passed in explicitly. Also, this way there would be lesser number of methods where people could change the object itself (more typing to do it the other way), and people would be more inclined to be "functional" by default.
But, I am a newb and would like to find out the flaws in my idea (if there are any :).
Having classmethods as a default is a well-known but outdated paradigm. It's called Modular Programming. Your classes become effectively modules this way.
The Object-Oriented Paradigm (OOP) is mostly considered superior to the Modular Paradigm (and it is younger). The main difference is exactly that parts of code are associated by default to a group of data (called an object) — and thus not classmethods.
It turns out in practice that this is much more useful. Combined with other OOP architectural ideas like inheritance this offers directer ways to represent the models in the heads of the developers.
Using object methods I can write abstract code which can be used for objects of various types; I don't have to know the type of the objects while writing my routine. E. g. I can write a max() routine which compares the elements of a list with each other to find the greatest. Comparing then is done using the > operator which is effectively an object method of the element (in Python this is __gt__(), in C++ it would be operator>() etc.). Now the object itself (maybe a number, maybe a date, etc.) can handle the comparison of itself with another of its type. In code this can be written as short as
a > b # in Python this calls a.__gt__(b)
while with only having classmethods you would have to write it as
type(a).__gt__(a, b)
which is much less readable.
If the method doesn't access any of an object's state, but is specific to that object's class, then it's a good candidate for being a classmethod.
Otherwise if it's more general, then just use a function defined at module level, no need to make it belong to a specific class.
I've found that classmethods are actually pretty rare in practice, and certainly not the default. There should be plenty of good code out there (on e.g. github) to get examples from.

What is the equivalent of passing functions as arguments using an object oriented approach

I have a program in python that includes a class that takes a function as an argument to the __init__ method. This function is stored as an attribute and used in various places within the class. The functions passed in can be quite varied, and passing in a key and then selecting from a set of predefined functions would not give the same degree of flexibility.
Now, apologies if a long list of questions like this is not cool, but...
Is their a standard way to achieve this in a language where functions aren't first class objects?
Do blocks, like in smalltalk or objective-C, count as functions in this respect?
Would blocks be the best way to do this in those languages?
What if there are no blocks?
Could you add a new method at runtime?
In which languages would this be possible (and easy)?
Or would it be better to create an object with a single method that performs the desired operation?
What if I wanted to pass lots of functions, would I create lots of singleton objects?
Would this be considered a more object oriented approach?
Would anyone consider doing this in python, where functions are first class objects?
I don't understand what you mean by "equivalent... using an object oriented approach". In Python, since functions are (as you say) first-class objects, how is it not "object-oriented" to pass functions as arguments?
a standard way to achieve this in a language where functions aren't first class objects?
Only to the extent that there is a standard way of functions failing to be first-class objects, I would say.
In C++, it is common to create another class, often called a functor or functionoid, which defines an overload for operator(), allowing instances to be used like functions syntactically. However, it's also often possible to get by with plain old function-pointers. Neither the pointer nor the pointed-at function is a first-class object, but the interface is rich enough.
This meshes well with "ad-hoc polymorphism" achieved through templates; you can write functions that don't actually care whether you pass an instance of a class or a function pointer.
Similarly, in Python, you can make objects register as callable by defining a __call__ method for the class.
Do blocks, like in smalltalk or objective-C, count as functions in this respect?
I would say they do. At least as much as lambdas count as functions in Python, and actually more so because they aren't crippled the way Python's lambdas are.
Would blocks be the best way to do this in those languages?
It depends on what you need.
Could you add a new method at runtime? In which languages would this be possible (and easy)?
Languages that offer introspection and runtime access to their own compiler. Python qualifies.
However, there is nothing about the problem, as presented so far, which suggests a need to jump through such hoops. Of course, some languages have more required boilerplate than others for a new class.
Or would it be better to create an object with a single method that performs the desired operation?
That is pretty standard.
What if I wanted to pass lots of functions, would I create lots of singleton objects?
You say this as if you might somehow accidentally create more than one instance of the class if you don't write tons of boilerplate in an attempt to prevent yourself from doing so.
Would this be considered a more object oriented approach?
Again, I can't fathom your understanding of the term "object-oriented". It doesn't mean "creating lots of objects".
Would anyone consider doing this in python, where functions are first class objects?
Not without a need for the extra things that a class can do and a function can't. With duck typing, why on earth would you bother?
I'm just going to answer some of your questions.
As they say in the Scheme community, "objects are a poor man's closures" (closures being first-class functions). Blocks are usually just syntactic sugar for closures. For languages that do not have closures, there exist various solutions.
One of the common solutions is to use operator overloading: C++ has a notion of function objects, which define a member operator() ("operator function call"). Python has a similar overloading mechanism, where you define __call__:
class Greeter(object):
def __init__(self, who):
self.who = who
def __call__(self):
print("Hello, %s!" % who)
hello = Greeter("world")
hello()
Yes, you might consider using this in Python instead of storing functions in objects, since functions can't be pickled.
In languages without operator overloading, you'll see things like Guava's Function interface.
You could use the strategy pattern. Basically you pass in an object with a known interface, but different behavior. It's like passing function but one that's wrapped up in an object.
In Smalltalk you'd mostly be using blocks. You can also create classes and instances at runtime.

Categories

Resources