Related
I have the following piece of code which fails with the following error:
RuntimeError: maximum recursion depth exceeded
I attempted to rewrite this to allow for tail recursion optimization (TCO). I believe that this code should have been successful if a TCO had taken place.
def trisum(n, csum):
if n == 0:
return csum
else:
return trisum(n - 1, csum + n)
print(trisum(1000, 0))
Should I conclude that Python does not do any type of TCO, or do I just need to define it differently?
No, and it never will since Guido van Rossum prefers to be able to have proper tracebacks:
Tail Recursion Elimination (2009-04-22)
Final Words on Tail Calls (2009-04-27)
You can manually eliminate the recursion with a transformation like this:
>>> def trisum(n, csum):
... while True: # Change recursion to a while loop
... if n == 0:
... return csum
... n, csum = n - 1, csum + n # Update parameters instead of tail recursion
>>> trisum(1000,0)
500500
I published a module performing tail-call optimization (handling both tail-recursion and continuation-passing style): https://github.com/baruchel/tco
Optimizing tail-recursion in Python
It has often been claimed that tail-recursion doesn't suit the Pythonic way of coding and that one shouldn't care about how to embed it in a loop. I don't want to argue with
this point of view; sometimes however I like trying or implementing new ideas
as tail-recursive functions rather than with loops for various reasons (focusing on the
idea rather than on the process, having twenty short functions on my screen in the same
time rather than only three "Pythonic" functions, working in an interactive session rather than editing my code, etc.).
Optimizing tail-recursion in Python is in fact quite easy. While it is said to be impossible
or very tricky, I think it can be achieved with elegant, short and general solutions; I even
think that most of these solutions don't use Python features otherwise than they should.
Clean lambda expressions working along with very standard loops lead to quick, efficient and
fully usable tools for implementing tail-recursion optimization.
As a personal convenience, I wrote a small module implementing such an optimization
by two different ways. I would like to discuss here about my two main functions.
The clean way: modifying the Y combinator
The Y combinator is well known; it allows to use lambda functions in a recursive
manner, but it doesn't allow by itself to embed recursive calls in a loop. Lambda
calculus alone can't do such a thing. A slight change in the Y combinator however
can protect the recursive call to be actually evaluated. Evaluation can thus be delayed.
Here is the famous expression for the Y combinator:
lambda f: (lambda x: x(x))(lambda y: f(lambda *args: y(y)(*args)))
With a very slight change, I could get:
lambda f: (lambda x: x(x))(lambda y: f(lambda *args: lambda: y(y)(*args)))
Instead of calling itself, the function f now returns a function performing the
very same call, but since it returns it, the evaluation can be done later from outside.
My code is:
def bet(func):
b = (lambda f: (lambda x: x(x))(lambda y:
f(lambda *args: lambda: y(y)(*args))))(func)
def wrapper(*args):
out = b(*args)
while callable(out):
out = out()
return out
return wrapper
The function can be used in the following way; here are two examples with tail-recursive
versions of factorial and Fibonacci:
>>> from recursion import *
>>> fac = bet( lambda f: lambda n, a: a if not n else f(n-1,a*n) )
>>> fac(5,1)
120
>>> fibo = bet( lambda f: lambda n,p,q: p if not n else f(n-1,q,p+q) )
>>> fibo(10,0,1)
55
Obviously recursion depth isn't an issue any longer:
>>> bet( lambda f: lambda n: 42 if not n else f(n-1) )(50000)
42
This is of course the single real purpose of the function.
Only one thing can't be done with this optimization: it can't be used with a
tail-recursive function evaluating to another function (this comes from the fact
that callable returned objects are all handled as further recursive calls with
no distinction). Since I usually don't need such a feature, I am very happy
with the code above. However, in order to provide a more general module, I thought
a little more in order to find some workaround for this issue (see next section).
Concerning the speed of this process (which isn't the real issue however), it happens
to be quite good; tail-recursive functions are even evaluated much quicker than with
the following code using simpler expressions:
def bet1(func):
def wrapper(*args):
out = func(lambda *x: lambda: x)(*args)
while callable(out):
out = func(lambda *x: lambda: x)(*out())
return out
return wrapper
I think that evaluating one expression, even complicated, is much quicker than
evaluating several simple expressions, which is the case in this second version.
I didn't keep this new function in my module, and I see no circumstances where it
could be used rather than the "official" one.
Continuation passing style with exceptions
Here is a more general function; it is able to handle all tail-recursive functions,
including those returning other functions. Recursive calls are recognized from
other return values by the use of exceptions. This solutions is slower than the
previous one; a quicker code could probably be written by using some special
values as "flags" being detected in the main loop, but I don't like the idea of
using special values or internal keywords. There is some funny interpretation
of using exceptions: if Python doesn't like tail-recursive calls, an exception
should be raised when a tail-recursive call does occur, and the Pythonic way will be
to catch the exception in order to find some clean solution, which is actually what
happens here...
class _RecursiveCall(Exception):
def __init__(self, *args):
self.args = args
def _recursiveCallback(*args):
raise _RecursiveCall(*args)
def bet0(func):
def wrapper(*args):
while True:
try:
return func(_recursiveCallback)(*args)
except _RecursiveCall as e:
args = e.args
return wrapper
Now all functions can be used. In the following example, f(n) is evaluated to the
identity function for any positive value of n:
>>> f = bet0( lambda f: lambda n: (lambda x: x) if not n else f(n-1) )
>>> f(5)(42)
42
Of course, it could be argued that exceptions are not intended to be used for intentionally
redirecting the interpreter (as a kind of goto statement or probably rather a kind of
continuation passing style), which I have to admit. But, again,
I find funny the idea of using try with a single line being a return statement: we try to return
something (normal behaviour) but we can't do it because of a recursive call occurring (exception).
Initial answer (2013-08-29).
I wrote a very small plugin for handling tail recursion. You may find it with my explanations there: https://groups.google.com/forum/?hl=fr#!topic/comp.lang.python/dIsnJ2BoBKs
It can embed a lambda function written with a tail recursion style in another function which will evaluate it as a loop.
The most interesting feature in this small function, in my humble opinion, is that the function doesn't rely on some dirty programming hack but on mere lambda calculus: the behaviour of the function is changed to another one when inserted in another lambda function which looks very like the Y combinator.
The word of Guido is at http://neopythonic.blogspot.co.uk/2009/04/tail-recursion-elimination.html
I recently posted an entry in my Python History blog on the origins of
Python's functional features. A side remark about not supporting tail
recursion elimination (TRE) immediately sparked several comments about
what a pity it is that Python doesn't do this, including links to
recent blog entries by others trying to "prove" that TRE can be added
to Python easily. So let me defend my position (which is that I don't
want TRE in the language). If you want a short answer, it's simply
unpythonic. Here's the long answer:
CPython does not and will probably never support tail call optimization based on Guido van Rossum's statements on the subject.
I've heard arguments that it makes debugging more difficult because of how it modifies the stack trace.
Try the experimental macropy TCO implementation for size.
Besides optimizing tail recursion, you can set the recursion depth manually by:
import sys
sys.setrecursionlimit(5500000)
print("recursion limit:%d " % (sys.getrecursionlimit()))
There is no built-in tail recursion optimization in Python. However, we can "rebuild" the function through the Abstract Syntax Tree( AST), eliminating the recursion there and replacing it with a loop. Guido was absolutely right, this approach has some limitations, so I can't recommend it for use.
However, I still wrote (rather as a training example) my own version of the optimizer, and you can even try how it works.
Install this package via pip:
pip install astrologic
Now you can run this sample code:
from astrologic import no_recursion
counter = 0
#no_recursion
def recursion():
global counter
counter += 1
if counter != 10000000:
return recursion()
return counter
print(recursion())
This solution is not stable, and you should never use it in production. You can read about some significant restrictions on the page in github (in Russian, sorry). However, this solution is quite "real", without interrupting the code and other similar tricks.
A tail call can never be optimized to a jump in Python. An optimization is a program transformation that preserves the program's meaning. Tail-call elimination doesn't preserve the meaning of Python programs.
One problem, often mentioned, is that tail-call elimination changes the call stack, and Python allows for runtime introspection of the stack. But there is another problem that is rarely mentioned. There is probably a lot of code like this in the wild:
def map_file(path):
f = open(path, 'rb')
return mmap.mmap(f.fileno())
The call to mmap.mmap is in tail position. If it were replaced by a jump, then the current stack frame would be discarded before control was passed to mmap. The current stack frame contains the only reference to the file object, so the file object could (and in CPython would) be freed before mmap is called, which would close the file descriptor, invalidating it before mmap sees it.
At best, the code would fail with an exception. At worst, the file descriptor could be reused in another thread, causing mmap to map the wrong file. So this "optimization" would be a potentially disastrous thing to unleash on the huge body of existing Python code.
The Python spec guarantees that such problems won't occur, so you can be sure that no conformant implementation will ever convert return f(args) into a jump—unless, perhaps, it has a sophisticated static analysis engine that can prove that discarding an object early will have no observable consequences in this case.
None of that would prevent Python from adding a syntax for explicit tail calls with jump semantics, such as
return from f(args)
That wouldn't break code that didn't use it, and it would probably be useful for autogenerated code and some algorithms. GvR is no longer BDFL, so it might happen, but I wouldn't hold my breath.
The second argument of the iter function is useful for looping over objects that don't define themselves as iterable, such as binary files:
response = b''
for block in iter(partial(f.read, 256), b''):
response += block
However in Python 3.8 we now have the “the walrus operator”, which in the What's new in Python 3.8 article is mentioned as way to solve the exact problem above:
# Loop over fixed length blocks
while (block := f.read(256)) != '':
process(block)
I wonder if the later is now considered "the right approach"? And if so, if there is ever any need for the second argument of iter, since any code on the form
for x in iter(f, y):
g(x)
might as well now be written:
while (x := f()) != y:
g(x)
I guess there might still be cases where we don't want to immediately loop the iterable, such as b''.join(iter(partial(f.read, 256), b'')) or some code (though it quickly gets pretty hairy).
Also a loop like for i, x in enumerate(iter(f, y)): might be hard to translate to the new syntax(?)
The PEP for walrus only mentions 2-arg iter in the example while h(x := f()): g(x), which it says can't trivially be translated to iter.
Python usually have pretty precise guidelines on these sorts of things, but I haven't been able to find any for this particular issue. Can you help me?
The assignment expression is useful if you are primarily interested in immediately iterating over the iterator, but it doesn't help you define an iterator to be used elsewhere.
For example, you may want to create an iterator that will first be wrapped in map, or filter, or itertools.islice, before finally iterating over the final result using a for loop.
This is not a duplicate of Assignment inside lambda expression in Python, i.e., I'm not asking how to trick Python into assigning in a lambda expression.
I have some λ-calculus background. Considering the following code, it
looks like Python is quite willing to perform side-effects in lambda
expressions:
#!/usr/bin/python
def applyTo42(f):
return f(42)
def double(x):
return x * 2
class ContainsVal:
def __init__(self, v):
self.v = v
def store(self, v):
self.v = v
def main():
print('== functional, no side effects')
print('-- print the double of 42')
print(applyTo42(double))
print('-- print 1000 more than 42')
print(applyTo42(lambda x: x + 1000))
print('-- print c\'s value instead of 42')
c = ContainsVal(23)
print(applyTo42(lambda x: c.v))
print('== not functional, side effects')
print('-- perform IO on 42')
applyTo42(lambda x: print(x))
print('-- set c\'s value to 42')
print(c.v)
applyTo42(lambda x: c.store(x))
print(c.v)
#print('== illegal, but why?')
#print(applyTo42(lambda x: c.v = 99))
if __name__ == '__main__':
main()
But if I uncomment the lines
print('== illegal, but why?')
print(applyTo42(lambda x: c.v = 99))
I'll get
SyntaxError: lambda cannot contain assignment
Why not? What is the deeper reason behind this?
As the code demonstrates, it cannot be about “purity” in a
functional sense.
The only explanation I can imagine is that assignemts do not
return anything, not even None. But that sounds lame and would
be easy to fix (one way: make lambda expressions return None if
body is a statement).
Not an answer:
Because it's defined that way (I want to know why it's defined that way).
Because it's in the grammar (see above).
Use def if you need statements (I did not ask for how to get
statements into a function).
“This would change syntax / the language / semantics” would be ok as an answer if you can come up with an example of such a change, and why it would be bad.
The entire reason lambda exists is that it's an expression.1 If you want something that's like lambda but is a statement, that's just def.
Python expressions cannot contain statements. This is, in fact, fundamental to the language, and Python gets a lot of mileage out of that decision. It's the reason indentation for flow control works instead of being clunky as in many other attempts (like CoffeeScript). It's the reason you can read off the state changes by skimming the first object in each line. It's even part of the reason the language is easy to parse, both for the compiler and for human readers.2
Changing Python to have some way to "escape" the statement-expression divide, except maybe in a very careful and limited way, would turn it into a completely different language, and one that no longer had many of the benefits that cause people to choose Python in the first place.
Changing Python to make most statements expressions (like, say, Ruby) would again turn it into a completely different language without Python's current benefits.
And if Python did make either of those changes, then there'd no longer be a reason for lambda in the first place;2,3 you could just use def statements inside an expression.
What about changing Python to instead make assignments expressions? Well, it should be obvious that would break "you can read off the state changes by skimming the first object in each line". Although Guido usually focuses on the fact that if spam=eggs is an error more often than a useful thing.
The fact that Python does give you ways to get around that when needed, like setattr or even explicitly calling __setitem__ on globals(), doesn't mean it's something that should have direct syntactic support. Something that's very rarely needed doesn't deserve syntactic sugar—and even more so for something that's unusual enough that it should raise eyebrows and/or red flags when it actually is done.
1. I have no idea whether that was Guido's understanding when he originally added lambda back in Python 1.0. But it's definitely the reason lambda wasn't removed in Python 3.0.
2. In fact, Guido has, multiple times, suggested that allowing an LL(1) parser that humans can run in their heads is sufficient reason for the language being statement-based, to the point that other benefits don't even need to be discussed. I wrote about this a few years ago if anyone's interested.
3. If you're wondering why so many languages do have a lambda expression despite already having def: In many languages, ranging from C++ to Ruby, function aren't first-class objects that can be passed around, so they had to invent a second thing that is first-class but works like a function. In others, from Smalltalk to Java, functions don't even exist, only methods, so again, they had to invent a second thing that's not a method but works like one. Python has neither of those problems.
4. A few languages, like C# and JavaScript, actually had perfectly working inline function definitions, but added some kind of lambda syntax as pure syntactic sugar, to make it more concise and less boilerplatey. That might actually be worth doing in Python (although every attempt at a good syntax so far has fallen flat), but it wouldn't be the current lambda syntax, which is nearly as verbose as def.
There is a syntax problem: an assignment is a statement, and the body of a lambda can only have expressions. Python's syntax is designed this way1. Check it out at https://docs.python.org/3/reference/grammar.html.
There is also a semantics problem: what does each statement return?
I don't think there is interest in changing this, as lambdas are meant for very simple and short code. Moreover, a statement would allow sequences of statements as well, and that's not desirable for lambdas.
It could be also fixed by selectively allowing certain statements in the lambda body, and specifying the semantics (e.g. an assignment returns None, or returns the assigned value; the latter makes more sense to me). But what's the benefit?
Lambdas and functions are interchangeable. If you really have a use-case for a particular statement in the body of a lambda, you can define a function that executes it, and your specific problem is solved.
Perhaps you can create a syntactic macro to allow that with MacroPy3 (I'm just guessing, as I'm a fan of the project, but still I haven't had the time to dive in it).
For example MacroPy would allow you to define a macro that transforms f[_ * _] into lambda a, b: a * b, so it should not be impossible to define the syntax for a lambda that calls a function you defined.
1 A good reason to not change it is that it would cripple the syntax, because a lambda can be in places where expressions can be. And statements should not. But that's a very subjective remark of my own.
My answer is based on chepner's comment above and doesn't draw from any other credible or official source, however I think that it will be useful.
If assignment was allowed in lambda expressions, then the error of confusing == (equality test) with = (assignment) would have more chances of escaping into the wild.
Example:
>>> # Correct use of equality test
... list(filter(lambda x: x==1, [0, 1, 0.0, 1.0, 0+0j, 1+0j]))
[1, 1.0, (1+0j)]
>>> # Suppose that assignment is used by mistake instead of equality testing
... # and the return value of an assignment expression is always None
... list(filter(lambda x: None, [0, 1, 0.0, 1.0, 0+0j, 1+0j]))
[]
>>> # Suppose that assignment is used by mistake instead of equality testing
... # and the return value of an assignment expression is the assigned value
... list(filter(lambda x: 1, [0, 1, 0.0, 1.0, 0+0j, 1+0j]))
[0, 1, 0.0, 1.0, 0j, (1+0j)]
As long as exec() (and eval()) is allowed inside lambda, you can do assignments inside lambda:
q = 3
def assign(var_str, val_str):
exec("global " + var_str + "; " +
var_str + " = " + val_str)
lambda_assign = lambda var_str, val_str: assign(var_str, val_str)
q ## gives: 3
lambda_assign("q", "100")
q ## gives: 100
## what would such expression be a win over the direct:
q = 100
## ? `lambda_assign("q", "100")` will be for sure slower than
## `q = 100` isn't it?
q_assign = lambda v: assign("q", v)
q_assign("33")
q ## 33
## but do I need lambda for q_assign?
def q_assign(v): assign("q", v)
## would do it, too, isn't it?
But since lambda expressions allow only 1 expression to be defined inside their body (at least in Python ...), what would be the point of to allow an assignment inside a lambda? Its net effect would be to assign directly (without using any lambda) q = 100, isn't it?
It would be even faster than doing it over a defined lambda, since you have at least one function lookup and execution less to execute ...
There's not really any deeper reasons, it has nothing to do with lambda or functional language designs, it's just to avoid programmers from mixing = and == operators, which is a very common mistake in other languages
IF there's more to this story, I assume like MAYBE because python bdfl GVR has expressed his unloving sides to lambda and other functional features and attempted(and conceded) to remove them from python 3 altogether https://www.artima.com/weblogs/viewpost.jsp?thread=98196
At the time of this writing the core devs were seen having a heated discussions recently on whether to include a limited name binding expression assignment, the debate is still on going so perhaps someday we may see it in lambda(unlikely)
As you said it yourself it is definitely not about side effects or purity, they just don't want lambda to be more than a single expression... ... ...
With that said, here's something about multi expressions assignments in lambda, read on if you're interested
It is not at all impossible in python, in fact it was sometimes necessary to capture variable and sidestep late bindings by (ab)using kwargs(keyword arguments)
edit:
code example
f = lambda x,a=1: (lambda c = a+2, b = a+1: (lambda e = x,d = c+1: print(a,b,c,d,e))())()
f("w")
# output 1 2 3 4 w
# expression assignment through an object's method call
if let(a=1) .a > 0 and let(b=let.a+1) .b != 1 and let(c=let.b+let.a) .c:
print(let.a, let.b, let.c)
# output 1 2 3
As it stands, Python was designed as a statement-based language. Therefore assignment and other name bindings are statements, and do not have any result.
The Python core developers are currently discussing PEP 572, which would introduce a name-binding expression.
I think all the fellows answered this already. We use mostly lambdas function when we just want to:
-create some simple functions that do the work perfectly in a specific place(most of the time hidden inside some other big functions
-The lambda function does not have a name
-Can be used with some other built-ins functions such as map, list and so forth ...
>>> Celsius = [39.2, 36.5, 37.3, 37.8]
>>> Fahrenheit = map(lambda x: (float(9)/5)*x + 32, Celsius) # mapping the list here
>>> print Fahrenheit
[102.56, 97.700000000000003, 99.140000000000001, 100.03999999999999]
Please visit this webpage , this could be useful.Keep it up !!!
https://www.python-course.eu/lambda.php
I am trying to calculate the length of a list. When I run it on cmd, I get:
RuntimeError: maximum recursion depth exceeded in comparison
I don't think there's anything wrong with my code:
def len_recursive(list):
if list == []:
return 0
else:
return 1 + len_recursive(list[1:])
Don't use recursion unless you can predict that it is not too deep. Python has quite small limit on recursion depth.
If you insist on recursion, the efficient way is:
def len_recursive(lst):
if not lst:
return 0
return 1 + len_recursive(lst[1::2]) + len_recursive(lst[2::2])
The recursion depth in Python is limited, but can be increased as shown in this post. If Python had support for the tail call optimization, this solution would work for arbitrary-length lists:
def len_recursive(lst):
def loop(lst, acc):
if not lst:
return acc
return loop(lst[1:], acc + 1)
return loop(lst, 0)
But as it is, you will have to use shorter lists and/or increase the maximum recursion depth allowed.
Of course, no one would use this implementation in real life (instead using the len() built-in function), I'm guessing this is an academic example of recursion, but even so the best approach here would be to use iteration, as shown in #poke's answer.
As others have explained, there are two problems with your function:
It's not tail-recursive, so it can only handle lists as long as sys.getrecursionlimit.
Even if it were tail-recursive, Python doesn't do tail recursion optimization.
The first is easy to solve. For example, see Óscar López's answer.
The second is hard to solve, but not impossible. One approach is to use coroutines (built on generators) instead of subroutines. Another is to not actually call the function recursively, but instead return a function with the recursive result, and use a driver that applies the results. See Tail Recursion in Python by Paul Butler for an example of how to implement the latter, but here's what it would look like in your case.
Start with Paul Butler's tail_rec function:
def tail_rec(fun):
def tail(fun):
a = fun
while callable(a):
a = a()
return a
return (lambda x: tail(fun(x)))
This doesn't work as a decorator for his case, because he has two mutually-recursive functions. But in your case, that's not an issue. So, using Óscar López's's version:
#tail_rec
def tail_len(lst):
def loop(lst, acc):
if not lst:
return acc
return lambda: loop(lst[1:], acc + 1)
return lambda: loop(lst, 0)
And now:
>>> print tail_len(range(10000))
10000
Tada.
If you actually wanted to use this, you might want to make tail_rec into a nicer decorator:
def tail_rec(fun):
def tail(fun):
a = fun
while callable(a):
a = a()
return a
return functools.update_wrapper(lambda x: tail(fun(x)), fun)
Imagine you're running this using a stack of paper. You want to count how many sheets you have. If someone gives you 10 sheets you take the first sheet, put it down on the table and grab the next sheet, placing it next to the first sheet. You do this 10 times and your desk is pretty full, but you've set out each sheet. You then start to count every page, recycling it as you count it up, 0 + 1 + 1 + ... => 10. This isn't the best way to count pages, but it mirrors the recursive approach and python's implementaion.
This works for small numbers of pages. Now imagine someone gives you 10000 sheets. Pretty soon there is no room on your desk to set out each page. This is essentially what the error message is telling you.
The Maximum Recursion depth is "how many sheets" can the table hold. Each time you call python needs to keep the "1 + result of recursive call" around so that when all the pages have been laid out it can come back and count them up. Unfortunately you're running out of space before the final counting-up occurs.
If you want to do this recursively to learn, since you're want to use len() in any reasonable situation, just use small lists, 25 should be fine.
Some systems could handle this for large lists if they support tail calls
Your exception message means that your method is called recursively too often, so it’s likely that your list is just too long to count the elements recursively like that. You could do it simply using a iterative solution though:
def len_iterative(lst):
length = 0
while lst:
length += 1
lst = lst[1:]
return length
Note that this will very likely still be a terrible solution as lst[1:] will keep creating copies of the list. So you will end up with len(lst) + 1 list instances (with lengths 0 to len(lst)). It is probably the best idea to just use the built-in len directly, but I guess it was an assignment.
Python isn't optimising tail recursion calls, so using such recursive algorythms isn't a good idea.
You can tweak stack with sys.setrecursionlimit(), but it's still not a good idea.
I can find lots of stuff showing me what a lambda function is, and how the syntax works and what not. But other than the "coolness factor" (I can make a function in middle a call to another function, neat!) I haven't seen something that's overwelmingly compelling to say why I really need/want to use them.
It seems to be more of a stylistic or structual choice in most examples I've seen. And kinda breaks the "Only one correct way to do something" in python rule. How does it make my programs, more correct, more reliable, faster, or easier to understand? (Most coding standards I've seen tend to tell you to avoid overly complex statements on a single line. If it makes it easier to read break it up.)
Here's a good example:
def key(x):
return x[1]
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key)
versus
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=lambda x: x[1])
From another angle: Lambda expressions are also known as "anonymous functions", and are very useful in certain programming paradigms, particularly functional programming, which lambda calculus provided the inspiration for.
http://en.wikipedia.org/wiki/Lambda_calculus
The syntax is more concise in certain situations, mostly when dealing with map et al.
map(lambda x: x * 2, [1,2,3,4])
seems better to me than:
def double(x):
return x * 2
map(double, [1,2,3,4])
I think the lambda is a better choice in this situation because the def double seems almost disconnected from the map that is using it. Plus, I guess it has the added benefit that the function gets thrown away when you are done.
There is one downside to lambda which limits its usefulness in Python, in my opinion: lambdas can have only one expression (i.e., you can't have multiple lines). It just can't work in a language that forces whitespace.
Plus, whenever I use lambda I feel awesome.
For me it's a matter of the expressiveness of the code. When writing code that people will have to support, that code should tell a story in as concise and easy to understand manner as possible. Sometimes the lambda expression is more complicated, other times it more directly tells what that line or block of code is doing. Use judgment when writing.
Think of it like structuring a sentence. What are the important parts (nouns and verbs vs. objects and methods, etc.) and how should they be ordered for that line or block of code to convey what it's doing intuitively.
Lambda functions are most useful in things like callback functions, or places in which you need a throwaway function. JAB's example is perfect - It would be better accompanied by the keyword argument key, but it still provides useful information.
When
def key(x):
return x[1]
appears 300 lines away from
[(1,2), (3,1), (5,10), (11,-3)].sort(key)
what does key do? There's really no indication. You might have some sort of guess, especially if you're familiar with the function, but usually it requires going back to look. OTOH,
[(1,2), (3,1), (5,10), (11,-3)].sort(lambda x: x[1])
tells you a lot more.
Sort takes a function as an argument
That function takes 1 parameter (and "returns" a result)
I'm trying to sort this list by the 2nd value of each of the elements of the list
(If the list were a variable so you couldn't see the values) this logic expects the list to have at least 2 elements in it.
There's probably some more information, but already that's a tremendous amount that you get just by using an anonymous lambda function instead of a named function.
Plus it doesn't pollute your namespace ;)
Yes, you're right — it is a structural choice. It probably does not make your programs more correct by just using lambda expressions. Nor does it make them more reliable, and this has nothing to do with speed.
It is only about flexibility and the power of expression. Like list comprehension. You can do most of that defining named functions (possibly polluting namespace, but that's again purely stylistic issue).
It can aid to readability by the fact, that you do not have to define a separate named function, that someone else will have to find, read and understand that all it does is to call a method blah() on its argument.
It may be much more interesting when you use it to write functions that create and return other functions, where what exactly those functions do, depends on their arguments. This may be a very concise and readable way of parameterizing your code behaviour. You can just express more interesting ideas.
But that is still a structural choice. You can do that otherwise. But the same goes for object oriented programming ;)
Ignore for a moment the detail that it's specifically anonymous functions we're talking about. functions, including anonymous ones, are assignable quantities (almost, but not really, values) in Python. an expression like
map(lambda y: y * -1, range(0, 10))
explicitly mentions four anonymous quantities: -1, 0, 10 and the result of the lambda operator, plus the implied result of the map call. it's possible to create values of anonymous types in some languages. so ignore the superficial difference between functions and numbers. the question when to use an anonymous function as opposed to a named one is similar to a question of when to put a naked number literal in the code and when to declare a TIMES_I_WISHED_I_HAD_A_PONY or BUFFER_SIZE beforehand. there are times when it's appropriate to use a (numeric, string or function) literal, and there are times when it's more appropriate to name such a thing and refer to it through its name.
see eg. Allen Holub's provocative, thought-or-anger-provoking book on Design Patterns in Java; he uses anonymous classes quite a bit.
Lambda, while useful in certain situations, has a large potential for abuse. lambda's almost always make code more difficult to read. And while it might feel satisfying to fit all your code onto a single line, it will suck for the next person who has to read your code.
Direct from PEP8
"One of Guido's key insights is that code is read much more often than it is written."
It is definitely true that abusing lambda functions often leads to bad and hard-to-read code. On the other hand, when used accurately, it does the opposite. There are already great answers in this thread, but one example I have come across is:
def power(n):
return lambda x: x**n
square = power(2)
cubic = power(3)
quadruple = power(4)
print(square(10)) # 100
print(cubic(10)) # 1000
print(quadruple(10)) # 10000
This simplified case could be rewritten in many other ways without the use of lambda. Still, one can infer how lambda functions can increase readability and code reuse in perhaps more complex cases and functions with this example.
Lambdas are anonymous functions (function with no name) that can be assigned to a variable or that can be passed as an argument to another function. The usefulness of lambda will be realized when you need a small piece of function that will be run once in a while or just once. Instead of writing the function in global scope or including it as part of your main program you can toss around few lines of code when needed to a variable or another function. Also when you pass the function as an argument to another function during the function call you can change the argument (the anonymous function) making the function itself dynamic. Suppose if the anonymous function uses variables outside its scope it is called closure. This is useful in callback functions.
One use of lambda function which I have learned, and where is not other good alternative or at least looks for me best is as default action in function parameter by
parameter=lambda x: x
This returns the value without change, but you can supply one function optionally to perform a transformation or action (like printing the answer, not only returning)
Also often it is useful to use in sorting as key:
key=lambda x: x[field]
The effect is to sort by fieldth (zero based remember) element of each item in sequence. For reversing you do not need lambda as it is clearer to use
reverse=True
Often it is almost as easy to do new real function and use that instead of lambda. If people has studied much Lisp or other functional programming, they also have natural tendency to use lambda function as in Lisp the function definitions are handled by lambda calculus.
Lambdas are objects, not methods, and they cannot be invoked in the same way that methods are.
for e.g
succ = ->(x){ x+1 }
succ mow holds a Proc object, which we can use like any other:
succ.call(2)
gives us an output = 3
I want to point out one situation other than list-processing where the lambda functions seems the best choice:
from tkinter import *
from tkinter import ttk
def callback(arg):
print(arg)
pass
root = Tk()
ttk.Button(root, text = 'Button1', command = lambda: callback('Button 1 clicked')).pack()
root.mainloop()
And if we drop lambda function here, the callback may only execute the callback once.
ttk.Button(root, text = 'Button1', command = callback('Button1 clicked')).pack()
Another point is that python does not have switch statements. Combining lambdas with dicts can be an effective alternative. e.g.:
switch = {
'1': lambda x: x+1,
'2': lambda x: x+2,
'3': lambda x: x+3
}
x = starting_val
ans = expression
new_ans = switch[ans](x)
In some cases it is much more clear to express something simple as a lambda. Consider regular sorting vs. reverse sorting for example:
some_list = [2, 1, 3]
print sorted(some_list)
print sorted(some_list, lambda a, b: -cmp(a, b))
For the latter case writing a separate full-fledged function just to return a -cmp(a, b) would create more misunderstanding then a lambda.
Lambdas allow you to create functions on the fly. Most of the examples I've seen don't do much more than create a function with parameters passed at the time of creation rather than execution. Or they simplify the code by not requiring a formal declaration of the function ahead of use.
A more interesting use would be to dynamically construct a python function to evaluate a mathematical expression that isn't known until run time (user input). Once created, that function can be called repeatedly with different arguments to evaluate the expression (say you wanted to plot it). That may even be a poor example given eval(). This type of use is where the "real" power is - in dynamically creating more complex code, rather than the simple examples you often see which are not much more than nice (source) code size reductions.
you master lambda, you master shortcuts in python.Here is why:
data=[(lambda x:x.text)(x.extract()) for x in soup.findAll('p') ]
^1 ^2 ^3 ^4
here we can see 4 parts of the list comprehension:
1: i finally want this
2: x.extract will perform some operation on x, here it pop the element from soup
3: x is the list iterable which is passed to the input of lambda at 2 along with extract operation
4: some arbitary list
i had found no other way to use 2 statements in lambda, but with this
kind of pipe-lining we can exploit the infinite potential of lambda.
Edit: as pointed out in the comments, by juanpa, its completely fine to use x.extract().text but the point was explaining the use of lambda pipe, ie passing the output of lambda1 as input to lambda2. via (lambda1 y:g(x))(lambda2 x:f(x))