Related
I've heard it said that multiline lambdas can't be added in Python because they would clash syntactically with the other syntax constructs in Python. I was thinking about this on the bus today and realized I couldn't think of a single Python construct that multiline lambdas clash with. Given that I know the language pretty well, this surprised me.
Now, I'm sure Guido had a reason for not including multiline lambdas in the language, but out of curiosity: what's a situation where including a multiline lambda would be ambiguous? Is what I've heard true, or is there some other reason that Python doesn't allow multiline lambdas?
Guido van Rossum (the inventor of Python) answers this exact question himself in an old blog post.
Basically, he admits that it's theoretically possible, but that any proposed solution would be un-Pythonic:
"But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption."
Look at the following:
map(multilambda x:
y=x+1
return y
, [1,2,3])
Is this a lambda returning (y, [1,2,3]) (thus map only gets one parameter, resulting in an error)? Or does it return y? Or is it a syntax error, because the comma on the new line is misplaced? How would Python know what you want?
Within the parens, indentation doesn't matter to python, so you can't unambiguously work with multilines.
This is just a simple one, there's probably more examples.
This is generally very ugly (but sometimes the alternatives are even more ugly), so a workaround is to make a braces expression:
lambda: (
doFoo('abc'),
doBar(123),
doBaz())
It won't accept any assignments though, so you'll have to prepare data beforehand.
The place I found this useful is the PySide wrapper, where you sometimes have short callbacks. Writing additional member functions would be even more ugly. Normally you won't need this.
Example:
pushButtonShowDialog.clicked.connect(
lambda: (
field1.clear(),
spinBox1.setValue(0),
diag.show())
A couple of relevant links:
For a while, I was following the development of Reia, which was initially going to have Python's indentation based syntax with Ruby blocks too, all on top of Erlang. But, the designer wound up giving up on indentation sensitivity, and this post he wrote about that decision includes a discussion about problems he ran into with indentation + multi-line blocks, and an increased appreciation he gained for Guido's design issues/decisions:
http://www.unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html
Also, here's an interesting proposal for Ruby-style blocks in Python I ran across where Guido posts a response w/o actually shooting it down (not sure whether there has been any subsequent shoot down, though):
http://tav.espians.com/ruby-style-blocks-in-python.html
Let me present to you a glorious but terrifying hack:
import types
def _obj():
return lambda: None
def LET(bindings, body, env=None):
'''Introduce local bindings.
ex: LET(('a', 1,
'b', 2),
lambda o: [o.a, o.b])
gives: [1, 2]
Bindings down the chain can depend on
the ones above them through a lambda.
ex: LET(('a', 1,
'b', lambda o: o.a + 1),
lambda o: o.b)
gives: 2
'''
if len(bindings) == 0:
return body(env)
env = env or _obj()
k, v = bindings[:2]
if isinstance(v, types.FunctionType):
v = v(env)
setattr(env, k, v)
return LET(bindings[2:], body, env)
You can now use this LET form as such:
map(lambda x: LET(('y', x + 1,
'z', x - 1),
lambda o: o.y * o.z),
[1, 2, 3])
which gives: [0, 3, 8]
[Edit Edit] Since this question is somehow still active 12 years after being asked. I will continue the tradition of amending my answer every 4 years or so.
Firstly, the question was how does multi-line lambda clash with Python. The accepted answer shows how with a simple example. The highly rated answer I linked below some years ago answers the question of "Why is it not a part of Python"--this answer is perhaps more satisfying to those who believe that the existing examples of "clashing" are not enough to make multi-line lambda impossible to implement in Python.
In previous iterations of this answer I discussed how to implement multi-line lambda into Python as is. I've since removed that part, because it was a flurry of bad practices. You may see it in the edit history of this answer if you wish.
However the answer to "Why not?", being "because Rossum said so" can still be a source of frustration. So lets see if it could be engineered around the counter example given by user balpha:
map(lambda x:
y=x+1 # <-- this line defines the outmost indent level*
for i in range(12):
y+=12
return y
, [1,2,3])
#*By convention it is always one-indent past the 'l' in lambda
As for the return value we have that the following is non-permissible in python:
def f():
return 3
, [1,2,3]
So by the same logic, "[1,2,3]" should not be part of the return value. Let's try it this way instead:
map(lambda x:
y=x+1 # part of lambda block
for i in range(12): # part of lambda block
y+=12 # part of lambda block
return y, [1,2,3]) # part of lambda block
This one's trickier, but since the lambda block has a clearly defined beginning (the token 'lambda') yet no clear ending, I would argue anything that is on the same line as part of a lambda block is also part of the lambda block.
One might imagine some features that can identify closing parenthesis or even inference based on the number of tokens expected by the enclosing element. In general, the above expression does not seem totally impossible to parse, but it may be a bit of a challenge.
To simplify things, you could separate all characters not intended to be part of the block:
map(lambda x:
y=x+1 # part of lambda block
for i in range(12): # part of lambda block
y+=12 # part of lambda block
return y # part of lambda block
, [1,2,3]) # argument separator, second argument, and closing paren for map
Back to where we were but this time it is unambiguous, because the last line is behind the lowest indent-depth for the lambda block.
Single line lambda would be a special case (identified by the lack of an immediate newline after the color), that behaves the same as it does now.
This is not to say that it necessarily should be a part of Python--but it is a quick illustration that is perhaps is possible with some changes in the language.
[Edit] Read this answer. It explains why multi-line lambda is not a thing.
Simply put, it's unpythonic. From Guido van Rossum's blog post:
I find any solution unacceptable that embeds an indentation-based block in the middle of an expression. Since I find alternative syntax for statement grouping (e.g. braces or begin/end keywords) equally unacceptable, this pretty much makes a multi-line lambda an unsolvable puzzle.
I'm guilty of practicing this dirty hack in some of my projects which is bit simpler:
lambda args...:( expr1, expr2, expr3, ...,
exprN, returnExpr)[-1]
I hope you can find a way to stay pythonic but if you have to do it this less painful than using exec and manipulating globals.
Let me also throw in my two cents about different workarounds.
How is a simple one-line lambda different from a normal function? I can think only of lack of assignments, some loop-like constructs (for, while), try-except clauses... And that's it? We even have a ternary operator - cool! So, let's try to deal with each of these problems.
Assignments
Some guys here have rightly noted that we should take a look at lisp's let form, which allows local bindings. Actually, all the non state-changing assignments can be performed only with let. But every lisp programmer knows that let form is absolutely equivalent to call to a lambda function! This means that
(let ([x_ x] [y_ y])
(do-sth-with-x-&-y x_ y_))
is the same as
((lambda (x_ y_)
(do-sth-with-x-&-y x_ y_)) x y)
So lambdas are more than enough! Whenever we want to make a new assignment we just add another lambda and call it. Consider this example:
def f(x):
y = f1(x)
z = f2(x, y)
return y,z
A lambda version looks like:
f = lambda x: (lambda y: (y, f2(x,y)))(f1(x))
You can even make the let function, if you don't like the data being written after actions on the data. And you can even curry it (just for the sake of more parentheses :) )
let = curry(lambda args, f: f(*args))
f_lmb = lambda x: let((f1(x),), lambda y: (y, f2(x,y)))
# or:
f_lmb = lambda x: let((f1(x),))(lambda y: (y, f2(x,y)))
# even better alternative:
let = lambda *args: lambda f: f(*args)
f_lmb = lambda x: let(f1(x))(lambda y: (y, f2(x,y)))
So far so good. But what if we have to make reassignments, i.e. change state? Well, I think we can live absolutely happily without changing state as long as task in question doesn't concern loops.
Loops
While there's no direct lambda alternative for loops, I believe we can write quite generic function to fit our needs. Take a look at this fibonacci function:
def fib(n):
k = 0
fib_k, fib_k_plus_1 = 0, 1
while k < n:
k += 1
fib_k_plus_1, fib_k = fib_k_plus_1 + fib_k, fib_k_plus_1
return fib_k
Impossible in terms of lambdas, obviously. But after writing a little yet useful function we're done with that and similar cases:
def loop(first_state, condition, state_changer):
state = first_state
while condition(*state):
state = state_changer(*state)
return state
fib_lmb = lambda n:\
loop(
(0,0,1),
lambda k, fib_k, fib_k_plus_1:\
k < n,
lambda k, fib_k, fib_k_plus_1:\
(k+1, fib_k_plus_1, fib_k_plus_1 + fib_k))[1]
And of course, one should always consider using map, reduce and other higher-order functions if possible.
Try-except and other control structs
It seems like a general approach to this kind of problems is to make use of lazy evaluation, replacing code blocks with lambdas accepting no arguments:
def f(x):
try: return len(x)
except: return 0
# the same as:
def try_except_f(try_clause, except_clause):
try: return try_clause()
except: return except_clause()
f = lambda x: try_except_f(lambda: len(x), lambda: 0)
# f(-1) -> 0
# f([1,2,3]) -> 3
Of course, this is not a full alternative to try-except clause, but you can always make it more generic. Btw, with that approach you can even make if behave like function!
Summing up: it's only natural that everything mentioned feels kinda unnatural and not-so-pythonically-beautiful. Nonetheless - it works! And without any evals and other trics, so all the intellisense will work. I'm also not claiming that you shoud use this everywhere. Most often you'd better define an ordinary function. I only showed that nothing is impossible.
Let me try to tackle #balpha parsing problem. I would use parentheses around the multiline lamda. If there is no parentheses, the lambda definition is greedy. So the lambda in
map(lambda x:
y = x+1
z = x-1
y*z,
[1,2,3]))
returns a function that returns (y*z, [1,2,3])
But
map((lambda x:
y = x+1
z = x-1
y*z)
,[1,2,3]))
means
map(func, [1,2,3])
where func is the multiline lambda that return y*z. Does that work?
(For anyone still interested in the topic.)
Consider this (includes even usage of statements' return values in further statements within the "multiline" lambda, although it's ugly to the point of vomiting ;-)
>>> def foo(arg):
... result = arg * 2;
... print "foo(" + str(arg) + ") called: " + str(result);
... return result;
...
>>> f = lambda a, b, state=[]: [
... state.append(foo(a)),
... state.append(foo(b)),
... state.append(foo(state[0] + state[1])),
... state[-1]
... ][-1];
>>> f(1, 2);
foo(1) called: 2
foo(2) called: 4
foo(6) called: 12
12
Here's a more interesting implementation of multi line lambdas. It's not possible to achieve because of how python use indents as a way to structure code.
But luckily for us, indent formatting can be disabled using arrays and parenthesis.
As some already pointed out, you can write your code as such:
lambda args: (expr1, expr2,... exprN)
In theory if you're guaranteed to have evaluation from left to right it would work but you still lose values being passed from one expression to an other.
One way to achieve that which is a bit more verbose is to have
lambda args: [lambda1, lambda2, ..., lambdaN]
Where each lambda receives arguments from the previous one.
def let(*funcs):
def wrap(args):
result = args
for func in funcs:
if not isinstance(result, tuple):
result = (result,)
result = func(*result)
return result
return wrap
This method let you write something that is a bit lisp/scheme like.
So you can write things like this:
let(lambda x, y: x+y)((1, 2))
A more complex method could be use to compute the hypotenuse
lst = [(1,2), (2,3)]
result = map(let(
lambda x, y: (x**2, y**2),
lambda x, y: (x + y) ** (1/2)
), lst)
This will return a list of scalar numbers so it can be used to reduce multiple values to one.
Having that many lambda is certainly not going to be very efficient but if you're constrained it can be a good way to get something done quickly then rewrite it as an actual function later.
In Python 3.8/3.9 there is Assignment Expression, so it could be used in lambda, greatly
expanding functionality
E.g., code
#%%
x = 1
y = 2
q = list(map(lambda t: (
tx := t*x,
ty := t*y,
tx+ty
)[-1], [1, 2, 3]))
print(q)
will print [3, 6, 9]
After Python3.8, there is another method for local binding
lambda x: (
y := x + 1,
y ** 2
)[-1]
For Loop
lambda x: (
y := x ** 2,
[y := y + x for _ in range(10)],
y
)[-1]
If Branch
lambda x: (
y := x ** 2,
x > 5 and [y := y + x for _ in range(10)],
y
)[-1]
Or
lambda x: (
y := x ** 2,
[y := y + x for _ in range(10)] if x > 5 else None,
y
)[-1]
While Loop
import itertools as it
lambda x: (
l := dict(y = x ** 2),
cond := lambda: l['y'] < 100,
body := lambda: l.update(y = l['y'] + x),
*it.takewhile(lambda _: cond() and (body(), True)[-1], it.count()),
l['y']
)[-1]
Or
import itertools as it
from types import SimpleNamespace as ns
lambda x: (
l := ns(y = x ** 2),
cond := lambda: l.y < 100,
body := lambda: vars(l).update(y = l.y + x),
*it.takewhile(lambda _: cond() and (body(), True)[-1], it.count()),
l.y
)[-1]
Or
import itertools as it
lambda x: (
y := x ** 2,
*it.takewhile(lambda t: t[0],
((
pred := y < 100,
pred and (y := y + x))
for _ in it.count())),
y
)[-1]
On the subject of ugly hacks, you can always use a combination of exec and a regular function to define a multiline function like this:
f = exec('''
def mlambda(x, y):
d = y - x
return d * d
''', globals()) or mlambda
You can wrap this into a function like:
def mlambda(signature, *lines):
exec_vars = {}
exec('def mlambda' + signature + ':\n' + '\n'.join('\t' + line for line in lines), exec_vars)
return exec_vars['mlambda']
f = mlambda('(x, y)',
'd = y - x',
'return d * d')
I know it is an old question, but for the record here is a kind of a solution to the problem of multiline lambda problem in which the result of one call is consumed by another call.
I hope it is not super hacky, since it is based only on standard library functions and uses no dunder methods.
Below is a simple example in which we start with x = 3 and then in the first line we add 1 and then in the second line we add 2 and get 6 as the output.
from functools import reduce
reduce(lambda data, func: func(data), [
lambda x: x + 1,
lambda x: x + 2
], 3)
## Output: 6
I was just playing a bit to try to make a dict comprehension with reduce, and come up with this one liner hack:
In [1]: from functools import reduce
In [2]: reduce(lambda d, i: (i[0] < 7 and d.__setitem__(*i[::-1]), d)[-1], [{}, *{1:2, 3:4, 5:6, 7:8}.items()])
Out[3]: {2: 1, 4: 3, 6: 5}
I was just trying to do the same as what was done in this Javascript dict comprehension: https://stackoverflow.com/a/11068265
You can simply use slash (\) if you have multiple lines for your lambda function
Example:
mx = lambda x, y: x if x > y \
else y
print(mx(30, 20))
Output: 30
I am starting with python but coming from Javascript the most obvious way is extract the expression as a function....
Contrived example, multiply expression (x*2) is extracted as function and therefore I can use multiline:
def multiply(x):
print('I am other line')
return x*2
r = map(lambda x : multiply(x), [1, 2, 3, 4])
print(list(r))
https://repl.it/#datracka/python-lambda-function
Maybe it does not answer exactly the question if that was how to do multiline in the lambda expression itself, but in case somebody gets this thread looking how to debug the expression (like me) I think it will help
One safe method to pass any number of variables between lambda items:
print((lambda: [
locals().__setitem__("a", 1),
locals().__setitem__("b", 2),
locals().__setitem__("c", 3),
locals().get("a") + locals().get("b") + locals().get("c")
])()[-1])
Output: 6
because a lambda function is supposed to be one-lined, as its the simplest form of a function, an entrance, then return
I have a list comprehension which approximates to:
[f(x) for x in l if f(x)]
Where l is a list and f(x) is an expensive function which returns a list.
I want to avoid evaluating f(x) twice for every non-empty occurance of f(x). Is there some way to save its output within the list comprehension?
I could remove the final condition, generate the whole list and then prune it, but that seems wasteful.
Edit:
Two basic approaches have been suggested:
An inner generator comprehension:
[y for y in (f(x) for x in l) if y]
or memoization.
I think the inner generator comprehension is elegant for the problem as stated. In actual fact I simplified the question to make it clear, I really want:
[g(x, f(x)) for x in l if f(x)]
For this more complicated situation, I think memoization produces a cleaner end result.
[y for y in (f(x) for x in l) if y]
Will do.
Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), it's possible to use a local variable within a list comprehension in order to avoid calling twice the same function:
In our case, we can name the evaluation of f(x) as a variable y while using the result of the expression to filter the list but also as the mapped value:
[y for x in l if (y := f(x))]
A solution (the best if you have repeated value of x) would be to memoize the function f, i.e. to create a wrapper function that saves the argument by which the function is called and save it, than return it if the same value is asked.
a really simple implementation is the following:
storage = {}
def memoized(value):
if value not in storage:
storage[value] = f(value)
return storage[value]
[memoized(x) for x in l if memoized(x)]
and then use this function in the list comprehension. This approach is valid under two condition, one theoretical and one practical. The first one is that the function f should be deterministic, i.e. returns the same results given the same input, and the other is that the object x can be used as a dictionary keys. If the first one is not valid than you should recompute f each timeby definition, while if the second one fails it is possible to use some slightly more robust approaches.
You can find a lot of implementation of memoization around the net, and I think that the new versions of python have something included in them too.
On a side note, never use the small L as a variable name, is a bad habit as it can be confused with an i or a 1 on some terminals.
EDIT:
as commented, a possible solution using generators comprehension (to avoid creating useless duplicate temporaries) would be this expression:
[g(x, fx) for x, fx in ((x,f(x)) for x in l) if fx]
You need to weight your choice given the computational cost of f, the number of duplication in the original list and memory at you disposition. Memoization make a space-speed tradeoff, meaning that it keep tracks of each result saving it, so if you have huge lists it can became costly on the memory occupation front.
You should use a memoize decorator. Here is an interesting link.
Using memoization from the link and your 'code':
def memoize(f):
""" Memoization decorator for functions taking one or more arguments. """
class memodict(dict):
def __init__(self, f):
self.f = f
def __call__(self, *args):
return self[args]
def __missing__(self, key):
ret = self[key] = self.f(*key)
return ret
return memodict(f)
#memoize
def f(x):
# your code
[f(x) for x in l if f(x)]
[y for y in [f(x) for x in l] if y]
For your updated problem, this might be useful:
[g(x,y) for x in l for y in [f(x)] if y]
Nope. There's no (clean) way to do this. There's nothing wrong with a good-old-fashioned loop:
output = []
for x in l:
result = f(x)
if result:
output.append(result)
If you find that hard to read, you can always wrap it in a function.
As the previous answers have shown, you can use a double comprehension or use memoization. For reasonably-sized problems it's a matter of taste (and I agree that memoization looks cleaner, since it hides the optimization). But if you're examining a very large list, there's a huge difference: Memoization will store every single value you've calculated, and can quickly blow out your memory. A double comprehension with a generator (round parens, not square brackets) only stores what you want to keep.
To come to your actual problem:
[g(x, f(x)) for x in series if f(x)]
To calculate the final value you need both x and f(x). No problem, pass them both like this:
[g(x, y) for (x, y) in ( (x, f(x)) for x in series ) if y ]
Again: this should be using a generator (round parens), not a list comprehension (square brackets). Otherwise you will build the whole list before you start filtering the results. This is the list comprehension version:
[g(x, y) for (x, y) in [ (x, f(x)) for x in series ] if y ] # DO NOT USE THIS
There have been a lot of answers regarding memoizing. The Python 3 standard library now has a lru_cache, which is a Last Recently Used Cache. So you can:
from functools import lru_cache
#lru_cache()
def f(x):
# function body here
This way your function will only be called once. You can also specify the size of the lru_cache, by default this is 128. The problem with the memoize decorators shown above is that the size of the lists can grow well out of hand.
You can use memoization. It is a technique which is used in order to avoid doing the same computation twice by saving somewhere the result for each calculated value.
I saw that there is already an answer that uses memoization, but I would like to propose a generic implementation, using python decorators:
def memoize(func):
def wrapper(*args):
if args in wrapper.d:
return wrapper.d[args]
ret_val = func(*args)
wrapper.d[args] = ret_val
return ret_val
wrapper.d = {}
return wrapper
#memoize
def f(x):
...
Now f is a memoized version of itself.
With this implementation you can memoize any function using the #memoize decorator.
Use map() !!
comp = [x for x in map(f, l) if x]
f is the function f(X), l is the list
map() will return the result of f(x) for each x in the list.
Here is my solution:
filter(None, [f(x) for x in l])
How about defining:
def truths(L):
"""Return the elements of L that test true"""
return [x for x in L if x]
So that, for example
> [wife.children for wife in henry8.wives]
[[Mary1], [Elizabeth1], [Edward6], [], [], []]
> truths(wife.children for wife in henry8.wives)
[[Mary1], [Elizabeth1], [Edward6]]
In Python 2, I can write:
In [5]: points = [ (1,2), (2,3)]
In [6]: min(points, key=lambda (x, y): (x*x + y*y))
Out[6]: (1, 2)
But that is not supported in 3.x:
File "<stdin>", line 1
min(points, key=lambda (x, y): (x*x + y*y))
^
SyntaxError: invalid syntax
The straightforward workaround is to index explicitly into the tuple that was passed:
>>> min(points, key=lambda p: p[0]*p[0] + p[1]*p[1])
(1, 2)
This is very ugly. If the lambda were a function, I could do
def some_name_to_think_of(p):
x, y = p
return x*x + y*y
But because the lambda only supports a single expression, it's not possible to put the x, y = p part into it.
How else can I work around this limitation?
No, there is no other way. You covered it all. The way to go would be to raise this issue on the Python ideas mailing list, but be prepared to argue a lot over there to gain some traction.
Actually, just not to say "there is no way out", a third way could be to implement one more level of lambda calling just to unfold the parameters - but that would be at once more inefficient and harder to read than your two suggestions:
min(points, key=lambda p: (lambda x,y: (x*x + y*y))(*p))
Python 3.8 update
Since the release of Python 3.8, PEP 572 — assignment expressions — have been available as a tool.
So, if one uses a trick to execute multiple expressions inside a lambda - I usually do that by creating a tuple and just returning the last component of it, it is possible to do the following:
>>> a = lambda p:(x:=p[0], y:=p[1], x ** 2 + y ** 2)[-1]
>>> a((3,4))
25
One should keep in mind that this kind of code will seldom be more readable or practical than having a full function. Still, there are possible uses - if there are various one-liners that would operate on this point, it could be worth to have a namedtuple, and use the assignment expression to effectively "cast" the incoming sequence to the namedtuple:
>>> from collections import namedtuple
>>> point = namedtuple("point", "x y")
>>> b = lambda s: (p:=point(*s), p.x ** 2 + p.y ** 2)[-1]
According to http://www.python.org/dev/peps/pep-3113/ tuple unpacking are gone, and 2to3 will translate them like so:
As tuple parameters are used by lambdas because of the single
expression limitation, they must also be supported. This is done by
having the expected sequence argument bound to a single parameter and
then indexing on that parameter:
lambda (x, y): x + y
will be translated into:
lambda x_y: x_y[0] + x_y[1]
Which is quite similar to your implementation.
I don't know any good general alternatives to the Python 2 arguments unpacking behaviour. Here's a couple of suggestion that might be useful in some cases:
if you can't think of a name; use the name of the keyword parameter:
def key(p): # more specific name would be better
x, y = p
return x**2 + y**3
result = min(points, key=key)
you could see if a namedtuple makes your code more readable if the list is used in multiple places:
from collections import namedtuple
from itertools import starmap
points = [ (1,2), (2,3)]
Point = namedtuple('Point', 'x y')
points = list(starmap(Point, points))
result = min(points, key=lambda p: p.x**2 + p.y**3)
While the destructuring arguments was removed in Python3, it was not removed from comprehensions. It is possible to abuse it to obtain similar behavior in Python 3.
For example:
points = [(1,2), (2,3)]
print(min(points, key=lambda y: next(x*x + y*y for (x,y) in [y])))
In comparison with the accepted answer of using a wrapper, this solution is able to completely destructure the arguments while the wrapper only destructures the first level. That is, you can do
values = [(('A',1),'a'), (('B',0),'b')]
print(min(values, key=lambda y: next(b for ((a,b),c) in (y,))))
In comparison to the accepted answer using an unwrapper lambda:
values = [(('A',1),'a'), (('B',0),'b')]
print(min(points, key=lambda p: (lambda a,b: (lambda x,y: (y))(*a))(*p)))
Alternatively one can also use a list instead of a tuple.
values = [(('A',1),'a'), (('B',0),'b')]
print(min(points, key=lambda y: next(b for (a,b),c in [y])))
This is just to suggest that it can be done, and should not be taken as a recommendation. However, IMO, this is better than the hack of using using multiple expressions in a tuple and returning the last one.
I think the better syntax is x * x + y * y let x, y = point, let keyword should be more carefully chosen.
The double lambda is the closest version.
lambda point: (lambda x, y: x * x + y * y)(*point)
High order function helper would be useful in case we give it a proper name.
def destruct_tuple(f):
return lambda args: f(*args)
destruct_tuple(lambda x, y: x * x + y * y)
Consider whether you need to unpack the tuple in the first place:
min(points, key=lambda p: sum(x**2 for x in p))
or whether you need to supply explicit names when unpacking:
min(points, key=lambda p: abs(complex(*p)**2)
Based on Cuadue suggestion and your comment on unpacking still being present in comprehensions, you can use, using numpy.argmin :
result = points[numpy.argmin(x*x + y*y for x, y in points)]
Another option is to write it into a generator producing a tuple where the key is the first element. Tuples are compared starting from beginning to end so the tuple with the smallest first element is returned. You can then index into the result to get the value.
min((x * x + y * y, (x, y)) for x, y in points)[1]
There may be a real solution to this, using PyFunctional!
Although not currently supported, I've submitted a tuple arg unpacking feature request to support:
(
seq((1, 2), (3, 4))
.map(unpack=lambda a, b: a + b)
) # => [3, 7]
Since questions on Stack Overflow are not supposed to contain the answer in the question, nor have explicit "update" sections, I am converting OP's original "updates" to a proper answer and making it community wiki.
OP originally claimed that this solution was "extending the idea in the answer". I cannot discern which answer that meant, or which idea. The idea is functionally the same as anthony.hl's answer, but that came years later. Considering the state of answers at the time, I think this qualifies as OP's original work.)
Make a wrapper function that generalizes the process of unpacking the arguments, like so:
def star(f):
return lambda args: f(*args)
Now we can use this to transform the lambda we want to write, into one that will receive the argument properly:
min(points, key=star(lambda x,y: (x*x + y*y))
We can further clean this up by using functools.wraps:
import functools
def star(f):
#functools.wraps(f)
def f_inner(args):
return f(*args)
return f_inner
In Haskell I would write:
main = do mapM_ print . map (\x -> x^2) . filter (\x -> (mod x 2) == 0) $ [1..20]
in Python I would have to use either many brackets or useless variables ... is there anything like . and $ in Python?
(I'm not familiar with Haskell, but if I understand your code snippet correctly...)
You can use a list comprehension to perform the filtering and exponentiation.
[i**2 for i in range(1,21) if i%2 == 0]
I would just use whatever idiomatic Python tools are available, such as list comprehensions, as others have pointed out, instead of trying to pretend you're writing Haskell, but if you really must, you could use compose combinator function even in Python:
# this is essentially just foldr (or right `reduce`) specialised on `compose2`
def compose(*args):
ret = identity
for f in reversed(args):
ret = compose2(f, ret)
return ret
def identity(x): return x
def compose2(f, g): return lambda x: f(g(x))
which you could use like this:
from functools import partial
# equiv. of: map (\x -> x^2) . filter (\x -> (mod x 2) == 0) $ [1..20]
compose(partial(map, lambda x: x**2), partial(filter, lambda x: x % 2 == 0))(range(1, 21))
which admittedly does work:
>>> compose(partial(map, lambda x: x**2), partial(filter, lambda x: x % 2 == 0))(range(1, 21))
[4, 16, 36, 64, 100, 144, 196, 256, 324, 400]
...but as you can see, Python lacks certain concepts such as currying and arbitrarily definable infix operators, so even though semantically, the above snippet of code is equivalent (even identical) to the Haskell snippet, it reads quite hellish.
As to the $ operator: it has little relevance in Python — its primary purpose in Haskell is related to operator precedence, which is a non-issue in Python because you can't really use operators most of the time anyway, and all of the built-in operators have predefined precedence.
And whereas $ can additionally be used as a higher order function in Haskell:
zipWith ($) [(3*), (4+), (5-)] [1,2,3]
...replicating this in Python with its (deprecated) apply "combinator" will, again, lead to code that is just ugly:
>>> list(starmap(apply, zip([lambda x: 3 * x, lambda x: 4 + x, lambda x: 5 - x], map(lambda x: [x], [1, 2, 3]))))
[3, 6, 2]
— again, several fundamental limitations of Python are at play here:
laziness isn't built-in and thus not handled automatically, so without "forcing" the starmap using list(), you don't get a "normal" list back;
apply is not (a -> b) -> a -> b but (a1 -> a2 -> ... -> aN -> b) -> (a1, a2, ..., aN) -> b, so you need to wrap the list elements with [] and use starmap not the normal map; this is also a result of the lack of currying;
lambda syntax is verbose because Guido's personal preference is against lambdas, map, reduce, and so on;
Ironically (since list comprehensions are something that Python borrowed from languages like Haskell), I'd probably write the code similarly in both languages:
# Python
for xsquared in [x**2 for x in range(1, 21) if x % 2 == 0]:
print(xsquared)
# legal, but not idiomatic; you don't construct a list just
# to throw it away.
# map(print, [x**2 for x in range(1, 21) if x % 2 == 0])
and
-- Haskell
main = (mapM_ print) [ x^2 | x <- [1..20], x `mod` 2 == 0 ]
or more briefly in each:
# Python
for xsquared in [x**2 for x in range(2, 21, 2)]:
print(xsquared)
-- Haskell
main = (mapM_ print) [x^2 | x <- [2,4..20]]
Functions in Python are more difficult to compose than in Haskell. A Haskell function takes one argument and returns one value. It's easy for the compiler to check that f . g makes sense given the defined type signatures for f and g. Python, however, has no such type signatures (even in 3.5, the type hinting is optional and only used during static analysis, not during runtime).
Further, Python functions can take an arbitrary number of arguments (no currying), and tuples are variable length, not fixed length. Suppose g returns a tuple. Should f ∘ g (my personal choice for a composition operator should such a thing ever be adopted, and Unicode operators be permitted) be equivalent to f(g(...)) or f(*g(...))? Both make sense, and point to the "need" for two different types of composition. What if g's return value has too many or too few values for f? What about keyword arguments to f? Should they be taken from a dictionary returned by g? What seems like a simple operation becomes quite complex to define in Python.
One other thing I may be completely wrong about. I get the impression that whereas each function in Python is compiled as a distinct piece of code, Haskell can compile optimized code for each composition, so that f . g isn't just naively converted to \x -> f (g x). At least in Python, for
def f(x):
return x + 5
def g(x):
return 3 * x
this is what the compiler could generate for f∘g
def fg(x):
return f(g(x))
which would be far less efficient than the equivalent of what I understand the Haskell compiler could generate:
def fg(x):
return 3*x + 5
For this case you should better use a list comprehension like #CoryKramer said.
To apply partial application in Python you should use functools.partial, would be something like this
from functools import partial
def compose(func1, *func2):
return func1 if not func2 else lambda x: func1(compose(*func2)(x))
myMap = partial(map, lambda x: x**2)
myFilter = partial(filter, lambda x: x%2 == 0)
myFunction = compose(myMap, myFilter)
myFunction(range(20))
Since map function returns list which is iterable and filter also you can nest them -
map(function1, (filter(function2,list)))
For more information I would recommend you read the map function documentation also filter function documentation
I've heard it said that multiline lambdas can't be added in Python because they would clash syntactically with the other syntax constructs in Python. I was thinking about this on the bus today and realized I couldn't think of a single Python construct that multiline lambdas clash with. Given that I know the language pretty well, this surprised me.
Now, I'm sure Guido had a reason for not including multiline lambdas in the language, but out of curiosity: what's a situation where including a multiline lambda would be ambiguous? Is what I've heard true, or is there some other reason that Python doesn't allow multiline lambdas?
Guido van Rossum (the inventor of Python) answers this exact question himself in an old blog post.
Basically, he admits that it's theoretically possible, but that any proposed solution would be un-Pythonic:
"But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption."
Look at the following:
map(multilambda x:
y=x+1
return y
, [1,2,3])
Is this a lambda returning (y, [1,2,3]) (thus map only gets one parameter, resulting in an error)? Or does it return y? Or is it a syntax error, because the comma on the new line is misplaced? How would Python know what you want?
Within the parens, indentation doesn't matter to python, so you can't unambiguously work with multilines.
This is just a simple one, there's probably more examples.
This is generally very ugly (but sometimes the alternatives are even more ugly), so a workaround is to make a braces expression:
lambda: (
doFoo('abc'),
doBar(123),
doBaz())
It won't accept any assignments though, so you'll have to prepare data beforehand.
The place I found this useful is the PySide wrapper, where you sometimes have short callbacks. Writing additional member functions would be even more ugly. Normally you won't need this.
Example:
pushButtonShowDialog.clicked.connect(
lambda: (
field1.clear(),
spinBox1.setValue(0),
diag.show())
A couple of relevant links:
For a while, I was following the development of Reia, which was initially going to have Python's indentation based syntax with Ruby blocks too, all on top of Erlang. But, the designer wound up giving up on indentation sensitivity, and this post he wrote about that decision includes a discussion about problems he ran into with indentation + multi-line blocks, and an increased appreciation he gained for Guido's design issues/decisions:
http://www.unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html
Also, here's an interesting proposal for Ruby-style blocks in Python I ran across where Guido posts a response w/o actually shooting it down (not sure whether there has been any subsequent shoot down, though):
http://tav.espians.com/ruby-style-blocks-in-python.html
Let me present to you a glorious but terrifying hack:
import types
def _obj():
return lambda: None
def LET(bindings, body, env=None):
'''Introduce local bindings.
ex: LET(('a', 1,
'b', 2),
lambda o: [o.a, o.b])
gives: [1, 2]
Bindings down the chain can depend on
the ones above them through a lambda.
ex: LET(('a', 1,
'b', lambda o: o.a + 1),
lambda o: o.b)
gives: 2
'''
if len(bindings) == 0:
return body(env)
env = env or _obj()
k, v = bindings[:2]
if isinstance(v, types.FunctionType):
v = v(env)
setattr(env, k, v)
return LET(bindings[2:], body, env)
You can now use this LET form as such:
map(lambda x: LET(('y', x + 1,
'z', x - 1),
lambda o: o.y * o.z),
[1, 2, 3])
which gives: [0, 3, 8]
[Edit Edit] Since this question is somehow still active 12 years after being asked. I will continue the tradition of amending my answer every 4 years or so.
Firstly, the question was how does multi-line lambda clash with Python. The accepted answer shows how with a simple example. The highly rated answer I linked below some years ago answers the question of "Why is it not a part of Python"--this answer is perhaps more satisfying to those who believe that the existing examples of "clashing" are not enough to make multi-line lambda impossible to implement in Python.
In previous iterations of this answer I discussed how to implement multi-line lambda into Python as is. I've since removed that part, because it was a flurry of bad practices. You may see it in the edit history of this answer if you wish.
However the answer to "Why not?", being "because Rossum said so" can still be a source of frustration. So lets see if it could be engineered around the counter example given by user balpha:
map(lambda x:
y=x+1 # <-- this line defines the outmost indent level*
for i in range(12):
y+=12
return y
, [1,2,3])
#*By convention it is always one-indent past the 'l' in lambda
As for the return value we have that the following is non-permissible in python:
def f():
return 3
, [1,2,3]
So by the same logic, "[1,2,3]" should not be part of the return value. Let's try it this way instead:
map(lambda x:
y=x+1 # part of lambda block
for i in range(12): # part of lambda block
y+=12 # part of lambda block
return y, [1,2,3]) # part of lambda block
This one's trickier, but since the lambda block has a clearly defined beginning (the token 'lambda') yet no clear ending, I would argue anything that is on the same line as part of a lambda block is also part of the lambda block.
One might imagine some features that can identify closing parenthesis or even inference based on the number of tokens expected by the enclosing element. In general, the above expression does not seem totally impossible to parse, but it may be a bit of a challenge.
To simplify things, you could separate all characters not intended to be part of the block:
map(lambda x:
y=x+1 # part of lambda block
for i in range(12): # part of lambda block
y+=12 # part of lambda block
return y # part of lambda block
, [1,2,3]) # argument separator, second argument, and closing paren for map
Back to where we were but this time it is unambiguous, because the last line is behind the lowest indent-depth for the lambda block.
Single line lambda would be a special case (identified by the lack of an immediate newline after the color), that behaves the same as it does now.
This is not to say that it necessarily should be a part of Python--but it is a quick illustration that is perhaps is possible with some changes in the language.
[Edit] Read this answer. It explains why multi-line lambda is not a thing.
Simply put, it's unpythonic. From Guido van Rossum's blog post:
I find any solution unacceptable that embeds an indentation-based block in the middle of an expression. Since I find alternative syntax for statement grouping (e.g. braces or begin/end keywords) equally unacceptable, this pretty much makes a multi-line lambda an unsolvable puzzle.
I'm guilty of practicing this dirty hack in some of my projects which is bit simpler:
lambda args...:( expr1, expr2, expr3, ...,
exprN, returnExpr)[-1]
I hope you can find a way to stay pythonic but if you have to do it this less painful than using exec and manipulating globals.
Let me also throw in my two cents about different workarounds.
How is a simple one-line lambda different from a normal function? I can think only of lack of assignments, some loop-like constructs (for, while), try-except clauses... And that's it? We even have a ternary operator - cool! So, let's try to deal with each of these problems.
Assignments
Some guys here have rightly noted that we should take a look at lisp's let form, which allows local bindings. Actually, all the non state-changing assignments can be performed only with let. But every lisp programmer knows that let form is absolutely equivalent to call to a lambda function! This means that
(let ([x_ x] [y_ y])
(do-sth-with-x-&-y x_ y_))
is the same as
((lambda (x_ y_)
(do-sth-with-x-&-y x_ y_)) x y)
So lambdas are more than enough! Whenever we want to make a new assignment we just add another lambda and call it. Consider this example:
def f(x):
y = f1(x)
z = f2(x, y)
return y,z
A lambda version looks like:
f = lambda x: (lambda y: (y, f2(x,y)))(f1(x))
You can even make the let function, if you don't like the data being written after actions on the data. And you can even curry it (just for the sake of more parentheses :) )
let = curry(lambda args, f: f(*args))
f_lmb = lambda x: let((f1(x),), lambda y: (y, f2(x,y)))
# or:
f_lmb = lambda x: let((f1(x),))(lambda y: (y, f2(x,y)))
# even better alternative:
let = lambda *args: lambda f: f(*args)
f_lmb = lambda x: let(f1(x))(lambda y: (y, f2(x,y)))
So far so good. But what if we have to make reassignments, i.e. change state? Well, I think we can live absolutely happily without changing state as long as task in question doesn't concern loops.
Loops
While there's no direct lambda alternative for loops, I believe we can write quite generic function to fit our needs. Take a look at this fibonacci function:
def fib(n):
k = 0
fib_k, fib_k_plus_1 = 0, 1
while k < n:
k += 1
fib_k_plus_1, fib_k = fib_k_plus_1 + fib_k, fib_k_plus_1
return fib_k
Impossible in terms of lambdas, obviously. But after writing a little yet useful function we're done with that and similar cases:
def loop(first_state, condition, state_changer):
state = first_state
while condition(*state):
state = state_changer(*state)
return state
fib_lmb = lambda n:\
loop(
(0,0,1),
lambda k, fib_k, fib_k_plus_1:\
k < n,
lambda k, fib_k, fib_k_plus_1:\
(k+1, fib_k_plus_1, fib_k_plus_1 + fib_k))[1]
And of course, one should always consider using map, reduce and other higher-order functions if possible.
Try-except and other control structs
It seems like a general approach to this kind of problems is to make use of lazy evaluation, replacing code blocks with lambdas accepting no arguments:
def f(x):
try: return len(x)
except: return 0
# the same as:
def try_except_f(try_clause, except_clause):
try: return try_clause()
except: return except_clause()
f = lambda x: try_except_f(lambda: len(x), lambda: 0)
# f(-1) -> 0
# f([1,2,3]) -> 3
Of course, this is not a full alternative to try-except clause, but you can always make it more generic. Btw, with that approach you can even make if behave like function!
Summing up: it's only natural that everything mentioned feels kinda unnatural and not-so-pythonically-beautiful. Nonetheless - it works! And without any evals and other trics, so all the intellisense will work. I'm also not claiming that you shoud use this everywhere. Most often you'd better define an ordinary function. I only showed that nothing is impossible.
Let me try to tackle #balpha parsing problem. I would use parentheses around the multiline lamda. If there is no parentheses, the lambda definition is greedy. So the lambda in
map(lambda x:
y = x+1
z = x-1
y*z,
[1,2,3]))
returns a function that returns (y*z, [1,2,3])
But
map((lambda x:
y = x+1
z = x-1
y*z)
,[1,2,3]))
means
map(func, [1,2,3])
where func is the multiline lambda that return y*z. Does that work?
(For anyone still interested in the topic.)
Consider this (includes even usage of statements' return values in further statements within the "multiline" lambda, although it's ugly to the point of vomiting ;-)
>>> def foo(arg):
... result = arg * 2;
... print "foo(" + str(arg) + ") called: " + str(result);
... return result;
...
>>> f = lambda a, b, state=[]: [
... state.append(foo(a)),
... state.append(foo(b)),
... state.append(foo(state[0] + state[1])),
... state[-1]
... ][-1];
>>> f(1, 2);
foo(1) called: 2
foo(2) called: 4
foo(6) called: 12
12
Here's a more interesting implementation of multi line lambdas. It's not possible to achieve because of how python use indents as a way to structure code.
But luckily for us, indent formatting can be disabled using arrays and parenthesis.
As some already pointed out, you can write your code as such:
lambda args: (expr1, expr2,... exprN)
In theory if you're guaranteed to have evaluation from left to right it would work but you still lose values being passed from one expression to an other.
One way to achieve that which is a bit more verbose is to have
lambda args: [lambda1, lambda2, ..., lambdaN]
Where each lambda receives arguments from the previous one.
def let(*funcs):
def wrap(args):
result = args
for func in funcs:
if not isinstance(result, tuple):
result = (result,)
result = func(*result)
return result
return wrap
This method let you write something that is a bit lisp/scheme like.
So you can write things like this:
let(lambda x, y: x+y)((1, 2))
A more complex method could be use to compute the hypotenuse
lst = [(1,2), (2,3)]
result = map(let(
lambda x, y: (x**2, y**2),
lambda x, y: (x + y) ** (1/2)
), lst)
This will return a list of scalar numbers so it can be used to reduce multiple values to one.
Having that many lambda is certainly not going to be very efficient but if you're constrained it can be a good way to get something done quickly then rewrite it as an actual function later.
In Python 3.8/3.9 there is Assignment Expression, so it could be used in lambda, greatly
expanding functionality
E.g., code
#%%
x = 1
y = 2
q = list(map(lambda t: (
tx := t*x,
ty := t*y,
tx+ty
)[-1], [1, 2, 3]))
print(q)
will print [3, 6, 9]
After Python3.8, there is another method for local binding
lambda x: (
y := x + 1,
y ** 2
)[-1]
For Loop
lambda x: (
y := x ** 2,
[y := y + x for _ in range(10)],
y
)[-1]
If Branch
lambda x: (
y := x ** 2,
x > 5 and [y := y + x for _ in range(10)],
y
)[-1]
Or
lambda x: (
y := x ** 2,
[y := y + x for _ in range(10)] if x > 5 else None,
y
)[-1]
While Loop
import itertools as it
lambda x: (
l := dict(y = x ** 2),
cond := lambda: l['y'] < 100,
body := lambda: l.update(y = l['y'] + x),
*it.takewhile(lambda _: cond() and (body(), True)[-1], it.count()),
l['y']
)[-1]
Or
import itertools as it
from types import SimpleNamespace as ns
lambda x: (
l := ns(y = x ** 2),
cond := lambda: l.y < 100,
body := lambda: vars(l).update(y = l.y + x),
*it.takewhile(lambda _: cond() and (body(), True)[-1], it.count()),
l.y
)[-1]
Or
import itertools as it
lambda x: (
y := x ** 2,
*it.takewhile(lambda t: t[0],
((
pred := y < 100,
pred and (y := y + x))
for _ in it.count())),
y
)[-1]
On the subject of ugly hacks, you can always use a combination of exec and a regular function to define a multiline function like this:
f = exec('''
def mlambda(x, y):
d = y - x
return d * d
''', globals()) or mlambda
You can wrap this into a function like:
def mlambda(signature, *lines):
exec_vars = {}
exec('def mlambda' + signature + ':\n' + '\n'.join('\t' + line for line in lines), exec_vars)
return exec_vars['mlambda']
f = mlambda('(x, y)',
'd = y - x',
'return d * d')
I know it is an old question, but for the record here is a kind of a solution to the problem of multiline lambda problem in which the result of one call is consumed by another call.
I hope it is not super hacky, since it is based only on standard library functions and uses no dunder methods.
Below is a simple example in which we start with x = 3 and then in the first line we add 1 and then in the second line we add 2 and get 6 as the output.
from functools import reduce
reduce(lambda data, func: func(data), [
lambda x: x + 1,
lambda x: x + 2
], 3)
## Output: 6
I was just playing a bit to try to make a dict comprehension with reduce, and come up with this one liner hack:
In [1]: from functools import reduce
In [2]: reduce(lambda d, i: (i[0] < 7 and d.__setitem__(*i[::-1]), d)[-1], [{}, *{1:2, 3:4, 5:6, 7:8}.items()])
Out[3]: {2: 1, 4: 3, 6: 5}
I was just trying to do the same as what was done in this Javascript dict comprehension: https://stackoverflow.com/a/11068265
You can simply use slash (\) if you have multiple lines for your lambda function
Example:
mx = lambda x, y: x if x > y \
else y
print(mx(30, 20))
Output: 30
I am starting with python but coming from Javascript the most obvious way is extract the expression as a function....
Contrived example, multiply expression (x*2) is extracted as function and therefore I can use multiline:
def multiply(x):
print('I am other line')
return x*2
r = map(lambda x : multiply(x), [1, 2, 3, 4])
print(list(r))
https://repl.it/#datracka/python-lambda-function
Maybe it does not answer exactly the question if that was how to do multiline in the lambda expression itself, but in case somebody gets this thread looking how to debug the expression (like me) I think it will help
One safe method to pass any number of variables between lambda items:
print((lambda: [
locals().__setitem__("a", 1),
locals().__setitem__("b", 2),
locals().__setitem__("c", 3),
locals().get("a") + locals().get("b") + locals().get("c")
])()[-1])
Output: 6
because a lambda function is supposed to be one-lined, as its the simplest form of a function, an entrance, then return