This question already has answers here:
What do lambda function closures capture?
(7 answers)
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 3 years ago.
I have a vector with some parameters and I would like to create a dictionary of anonymous (lambda) functions in Python3.
The goal of this is to create a callable function that gives values equal to the sum of these functions with the same argument.
I am struggling even to create a dictionary with the original lambda function objects and get them to behave consistently. I use the following code:
import numpy as np
a = np.linspace(0,2.0,10)
func = {}
for i,val in enumerate(a):
print(i)
func['f{}'.format(i)] = lambda x: x**val
print(func['f0'](0.5))
print(func['f1'](0.5))
print(func['f2'](0.5))
print(func['f3'](0.5))
The output of the final print statements gives the same value, whereas I would like it to give the values corresponding to x**val with the value of val coming from the originally constructed array a.
I guess what's happening is that the lambda functions always reference the "current" value of val, which, after the loop is executed is always the last value in the array? This makes sense because the output is:
0
1
2
3
4
5
6
7
8
9
0.25
0.25
0.25
0.25
The output makes sense because it is the result of 0.5**2.0 and the exponent is the last value that val takes on in the loop.
I don't really understand this because I would have thought val would go out of scope after the loop is run, but I'm assuming this is part of the "magic" of lambda functions in that they will keep variables that they need to compute the function in scope for longer.
I guess what I need to do is to insert the "literal" value of val at that point into the lambda function, but I've never done that and don't know how.
I would like to know how to properly insert the literal value of val into the lambda functions constructed at each iteration of the loop. I would also like to know if there is a better way to accomplish what I need to.
EDIT: it has been suggested that this question is a duplicate. I think it is a duplicate of the list comprehension post because the best answer is virtually identical and lambda functions are used.
I think it is not a duplicate of the lexical closures post, although I think it is important that this post was mentioned. That post gives a deeper understanding of the underlying causes for this behavior but the original question specifically states "mindful avoidance of lambda functions," which makes it a bit different. I'm not sure what the purpose of that mindful avoidance is, but the post did teach related lessons on scoping.
The problem with this approach is that val used inside your lambda function is the live variable, outside. When each lambda is called, the value used for val in the formula is the current value of val, therefore all your results are the same.
The solution is to "freeze" the value for val when creating each lambda function - the way that is easier to understand what is going on is to have an outer lambda function, that will take val as an input, and return your desided (inner) lambda - but with val frozen in a different scope. Note that the outer function is called and imedially discarded - its return value is the original function you had:
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = (lambda val: (lambda x: x**val))(val)
shorter version
Now, due to the way Python stores default arguments to functions, it is possible to store the "current val value" as a default argument in the lambda, and avoid the need for an outer function. But that spoils the lambda signature, and the "why" that value is there is harder to understand -
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = lambda x, val=val: x**val
Related
I have a tuple of functions that I want to pre-load with some data. Currently the way I am doing this is below. Essentially, I make a list of the new functions, and add the lambda functions to it one at a time, then reconvert to a tuple. However, when I use these functions in a different part of the code, every one of them acts as if it were the last one in the list.
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x: f(x, data)
newfuncs.append(newf)
return tuple(newfuncs)
Here is a simple example of the problem
funcs = (lambda x, y: x + y, lambda a, b: a - b)
funcs = newfuncs(10, funcs)
print(funcs[0](5))
print(funcs[1](5))
I would expect the number 15 to be printed, then -5. However, this code prints the number -5 twice. If anyone can help my understand why this is happening, it would be greatly appreciated. Thanks!
As mentioned, the issue is with the variable f, which is the same variable assigned to all lambda functions, so at the end of the loop, every lambda sees the same f.
The solution here is to either use functools.partial, or create a scoped default argument for the lambda:
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x, f=f: f(x, data) # note the f=f bit here
newfuncs.append(newf)
return tuple(newfuncs)
Calling these lambdas as before now gives:
15
-5
If you're using python3.x, make sure to take a look at this comment by ShadowRanger as a possible safety feature to the scoped default arg approach.
This is a well-known Python "issue," or should I say "this is just the way Python is."
You created the tuple:
( x => f(x, data), x => f(x, data) )
But what is f? f is not evaluated until you finally call the functions!
First, f was (x, y)=>x+y. Then in your for-loop, f was reassigned to (x, y)=>x-y.
When you finally get around to calling your functions, then, and only then, will the value of f be looked up. What is the value of f at this point? The value is (x, y)=>x-y for all of your functions. All of your functions do subtraction. This is because f is reassigned. There is ONLY ONE f. And the value of that one and only one f is set to the subtraction function, before any of your lambdas ever get called.
ADDENDUM
In case anyone is interested, different languages approach this problem in different ways. JavaScript is rather interesting (some would say confusing) here because it does things the Python way, which the OP found unexpected, as well as a different way, which the OP would have expected. In JavaScript:
> let funcs = []
> for (let f of [(x,y)=>x+y, (x,y)=>x-y]) {
funcs.push(x=>f(x,10));
}
> funcs[0](5)
15
funcs[1](5)
-5
However, if you change let to var above, it behaves like Python and you get -5 for both! This is because with let, you get a different f for each iteration of the for-loop; with var, the whole function shares the same f, which keeps getting reassigned. This is how Python works.
cᴏʟᴅsᴘᴇᴇᴅ has shown you that the way to do what you expect is to make sure you get that different f for each iteration, which Python allows you do in a pretty neat way, defaulting the second argument of the lambda to a f local to that iteration. It's pretty cool, so their answer should be accepted if it helps you.
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 years ago.
I want to create a list that contains the monomials up to degree n
basis = [lambda x:x**i for i in range(0,n+1)]
This however creates a list of n functions, but all the same one (of degree n) and not of degree 0,1,2,...n
I tried without list comprehension as well:
basis = []
for i in range(0,n+1):
basis.append(lambda x:x**i)
but with the same result. Substituting the lambda function by a classic function definition also did not fix it.
I checked Python lambdas and scoping, but that did not help, since I don't want to store function values, but the function itself. For example, I want to be able to call
basis[0](34)
and this should return
1
As i said in the comments, take a look at partial
def f(x, n):
return x**n
basis = [partial(f, n=i) for i in range(10)]
print(basis[0](34)) # 1
This entry sums it up perfectly
http://docs.python-guide.org/en/latest/writing/gotchas/#late-binding-closures :
... you can create a closure that binds immediately to its arguments by using a default arg like so:
def create_multipliers():
return [lambda x, i=i : i * x for i in range(5)]
You code would become:
basis = [lambda x,i=i:x**i for i in range(0,n+1)]
This is a bit of a hacky solution, but works. I still highly recommend to read the link provided as the 3 points that are made there are usual errors when you are new to Python.
This question already has answers here:
Syntax behind sorted(key=lambda: ...) [duplicate]
(10 answers)
Closed 6 years ago.
I have just finished my code for a school project, but have used one line of code from stack overflow that is slightly more advanced than the knowledge that's in my "skill comfort zone".
I do understand what it does, why it exists, etc.. But still cannot convert it into "human syntax" for my individual report. Could anyone give me a bit of their time and explain, as precisely as possible, the underlying mechanism in this line of code? Thanks in advance!
sites.sort(key = lambda x: x[0])
The role it has within my program is sorting a dictionary by the integers in its first column, from smallest to biggest.
Need to demonstrate that I fully understand this line of code, which I frankly do not.
Thanks!
A lambda function is basically like any other function, with some restrictions. It can only be one line, and it must be an expression. That expression is evaluated when the lambda function is called and the result of that expression is returned.
sites.sort(key = lambda x: x[0])
is the same as
def f(x):
return x[0]
sites.sort(key = f)
In this case, the lambda function is passed to the sort method as a key function. These are used in sorting to sort things based on something other than their value. Each element e of sites is passed to the key function, and then they are sorted based on f(e) or e[0] rather than the value of e.
When calling sort on an object, Python passes each of the elements contained inside that object to the function you specify as the key parameter.
A lambda function can be dissected in the following parts:
lambda <args>: <what to do with args>
In your lambda function you have a single arg x as the <args>, and your <what to do with args> is to get the zero element of it.
That element x[0] is going to be used in the comparisons sort performs.
In your example, sites must contain elements that themselves contain elements, for example, nested lists: l = [[1, 2, 3], [0, 1, 2, 3]].
Using this example, Python is first going to pass [1, 2, 3] to the lambda as the value of x and the lambda function is going to return x[0] == 1, then it will pass [0, 1, 2, 3] to it and get back x[0] == 0. These values are used during sorting to get the ordering as you require.
This question is distilled from the original application involving callback functions for Tkinter buttons. This is one line that illustrates the behavior.
lambdas = [lambda: i for i in range(3)]
if you then try invoking the lambda functions generated:
lambdas[0](), lambdas[1]() and lambdas[2]() all return 2.
The desired behavior was to have lambdas[0]() return 0, lambdas[1]() return 1, lambdas[2])() return 2.
I see that the index variable is interpreted by reference. The question is how to rephrase to have it treated by value.
Use a parameter with default value to bind the current value of i to a local variable. When the lambda gets called without an argument, the local variable i is assigned the default value:
In [110]: lambdas = [lambda i=i: i for i in range(3)]
In [111]: for lam in lambdas:
.....: print(lam())
.....:
0
1
2
When i is not a local variable, Python looks up its value in the enclosing scope. The value that is found is the last value i attained in the for-loop of the list comprehension. That is why, without the parameter with default value, each lambda returns 2 since the for-loop has completed by the time the lambdas get called.
Another common approach to solving this problem is to use a closure -- a function that can refer to environments that are no longer active such as the local namespace of an outer function, even after that function has returned.
def make_func(i):
return lambda: i
lambdas = [make_func(i) for i in range(3)]
for lam in lambdas:
print(lam())
prints
0
1
2
This works because when lam() gets called, since the i in the lambda
function's body is not a local variable, Python looks for the value of i in
the enclosing scope of the function make_func. Its local namespace is still
accessible to the closure, lam, even though make_func has already
completed. The value in that local namespace is the value which was passed to
make_func, which is, happily, the desired value of i.
As already mentioned by mkrieger1,
another way to create a new function with some argument values already supplied
is to use functools.partial:
lambdas = [functools.partial(lambda x: x, i) for i in range(3)]
The important thing to be understood here is, the functions are created during the evaluation of the list comprehension, but the value of i will be evaluated only during the execution of the functions.
So, in your case, you have created three functions and all of them refer i. When those functions are invoked at runtime, i will have the value 2, because that was the last value bound to i in the last iteration of the for loop.
Instead, you need to preserve the current value of i in each of the functions created. So, the common practice is to include a default parameter, like this
>>> lambdas = [lambda i=i: i for i in range(3)]
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Now, lambda i=i: i, creates a default parameter i which will have the current value of the loop variable i. So, when the functions are executed, the i referred is actually the parameter passed to the function, not the loop variable.
To avoid confusion, you can choose to use a different name for the variable, like this
>>> lambdas = [lambda t=i: t for i in range(3)]
If you are looking other ways to avoid this, you can use map function to apply the numbers to another function which will generate new functions with the current value, like this
>>> lambdas = map(lambda x: lambda: x, range(3))
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Here, lambda x: lambda: x creates an anonymous function which accepts a single parameter x, and returns another function which doesn't accept any parameters but returns the value x.
Note: Don't use this map form in actual code. It might reduce the readability of the code.
You can use functools.partial to create specialized functions from a more general one by partial application, which reduces the parameter count by one:
from functools import partial
lambdas = [partial(lambda x: x, i) for i in range(3)]
Here, lambda x: x is the general identity function taking one argument, and you create three specializations of it, taking no arguments, and returning fixed values.
Ahhh, further Googling found a solution (admittedly one I would not have stumbled upon myself). The desired behavior can be invoked by use of a default argument:
lambdas = [lambda i=i: i for i in range(3)]
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
What's going on here? I'm trying to create a list of functions:
def f(a,b):
return a*b
funcs = []
for i in range(0,10):
funcs.append(lambda x:f(i,x))
This isn't doing what I expect. I would expect the list to act like this:
funcs[3](3) = 9
funcs[0](5) = 0
But all the functions in the list seem to be identical, and be setting the fixed value to be 9:
funcs[3](3) = 27
funcs[3](1) = 9
funcs[2](6) = 54
Any ideas?
lambdas in python are closures.... the arguments you give it aren't going to be evaluated until the lambda is evaluated. At that time, i=9 regardless, because your iteration is finished.
The behavior you're looking for can be achieved with functools.partial
import functools
def f(a,b):
return a*b
funcs = []
for i in range(0,10):
funcs.append(functools.partial(f,i))
Yep, the usual "scoping problem" (actually a binding-later-than-you want problem, but it's often called by that name). You've already gotten the two best (because simplest) answers -- the "fake default" i=i solution, and functools.partial, so I'm only giving the third one of the classic three, the "factory lambda":
for i in range(0,10):
funcs.append((lambda i: lambda x: f(i, x))(i))
Personally I'd go with i=i if there's no risk of the functions in funcs being accidentally called with 2 parameters instead of just 1, but the factory function approach is worth considering when you need something a little bit richer than just pre-binding one arg.
There's only one i which is bound to each lambda, contrary to what you think. This is a common mistake.
One way to get what you want is:
for i in range(0,10):
funcs.append(lambda x, i=i: f(i, x))
Now you're creating a default parameter i in each lambda closure and binding to it the current value of the looping variable i.
All the lambdas end up being bound to the last one. See this question for a longer answer:
How do I create a list of Python lambdas (in a list comprehension/for loop)?
Considering the final value of i == 9
Like any good python function, it's going to use the value of the variable in the scope it was defined. Perhaps lambda: varname (being that it is a language construct) binds to the name, not the value, and evaluates that name at runtime?
Similar to:
i = 9
def foo():
print i
i = 10
foo()
I'd be quite interested in finding out of my answer is correct