I have a tuple of functions that I want to pre-load with some data. Currently the way I am doing this is below. Essentially, I make a list of the new functions, and add the lambda functions to it one at a time, then reconvert to a tuple. However, when I use these functions in a different part of the code, every one of them acts as if it were the last one in the list.
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x: f(x, data)
newfuncs.append(newf)
return tuple(newfuncs)
Here is a simple example of the problem
funcs = (lambda x, y: x + y, lambda a, b: a - b)
funcs = newfuncs(10, funcs)
print(funcs[0](5))
print(funcs[1](5))
I would expect the number 15 to be printed, then -5. However, this code prints the number -5 twice. If anyone can help my understand why this is happening, it would be greatly appreciated. Thanks!
As mentioned, the issue is with the variable f, which is the same variable assigned to all lambda functions, so at the end of the loop, every lambda sees the same f.
The solution here is to either use functools.partial, or create a scoped default argument for the lambda:
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x, f=f: f(x, data) # note the f=f bit here
newfuncs.append(newf)
return tuple(newfuncs)
Calling these lambdas as before now gives:
15
-5
If you're using python3.x, make sure to take a look at this comment by ShadowRanger as a possible safety feature to the scoped default arg approach.
This is a well-known Python "issue," or should I say "this is just the way Python is."
You created the tuple:
( x => f(x, data), x => f(x, data) )
But what is f? f is not evaluated until you finally call the functions!
First, f was (x, y)=>x+y. Then in your for-loop, f was reassigned to (x, y)=>x-y.
When you finally get around to calling your functions, then, and only then, will the value of f be looked up. What is the value of f at this point? The value is (x, y)=>x-y for all of your functions. All of your functions do subtraction. This is because f is reassigned. There is ONLY ONE f. And the value of that one and only one f is set to the subtraction function, before any of your lambdas ever get called.
ADDENDUM
In case anyone is interested, different languages approach this problem in different ways. JavaScript is rather interesting (some would say confusing) here because it does things the Python way, which the OP found unexpected, as well as a different way, which the OP would have expected. In JavaScript:
> let funcs = []
> for (let f of [(x,y)=>x+y, (x,y)=>x-y]) {
funcs.push(x=>f(x,10));
}
> funcs[0](5)
15
funcs[1](5)
-5
However, if you change let to var above, it behaves like Python and you get -5 for both! This is because with let, you get a different f for each iteration of the for-loop; with var, the whole function shares the same f, which keeps getting reassigned. This is how Python works.
cᴏʟᴅsᴘᴇᴇᴅ has shown you that the way to do what you expect is to make sure you get that different f for each iteration, which Python allows you do in a pretty neat way, defaulting the second argument of the lambda to a f local to that iteration. It's pretty cool, so their answer should be accepted if it helps you.
Related
This question already has answers here:
What do lambda function closures capture?
(7 answers)
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 3 years ago.
I have a vector with some parameters and I would like to create a dictionary of anonymous (lambda) functions in Python3.
The goal of this is to create a callable function that gives values equal to the sum of these functions with the same argument.
I am struggling even to create a dictionary with the original lambda function objects and get them to behave consistently. I use the following code:
import numpy as np
a = np.linspace(0,2.0,10)
func = {}
for i,val in enumerate(a):
print(i)
func['f{}'.format(i)] = lambda x: x**val
print(func['f0'](0.5))
print(func['f1'](0.5))
print(func['f2'](0.5))
print(func['f3'](0.5))
The output of the final print statements gives the same value, whereas I would like it to give the values corresponding to x**val with the value of val coming from the originally constructed array a.
I guess what's happening is that the lambda functions always reference the "current" value of val, which, after the loop is executed is always the last value in the array? This makes sense because the output is:
0
1
2
3
4
5
6
7
8
9
0.25
0.25
0.25
0.25
The output makes sense because it is the result of 0.5**2.0 and the exponent is the last value that val takes on in the loop.
I don't really understand this because I would have thought val would go out of scope after the loop is run, but I'm assuming this is part of the "magic" of lambda functions in that they will keep variables that they need to compute the function in scope for longer.
I guess what I need to do is to insert the "literal" value of val at that point into the lambda function, but I've never done that and don't know how.
I would like to know how to properly insert the literal value of val into the lambda functions constructed at each iteration of the loop. I would also like to know if there is a better way to accomplish what I need to.
EDIT: it has been suggested that this question is a duplicate. I think it is a duplicate of the list comprehension post because the best answer is virtually identical and lambda functions are used.
I think it is not a duplicate of the lexical closures post, although I think it is important that this post was mentioned. That post gives a deeper understanding of the underlying causes for this behavior but the original question specifically states "mindful avoidance of lambda functions," which makes it a bit different. I'm not sure what the purpose of that mindful avoidance is, but the post did teach related lessons on scoping.
The problem with this approach is that val used inside your lambda function is the live variable, outside. When each lambda is called, the value used for val in the formula is the current value of val, therefore all your results are the same.
The solution is to "freeze" the value for val when creating each lambda function - the way that is easier to understand what is going on is to have an outer lambda function, that will take val as an input, and return your desided (inner) lambda - but with val frozen in a different scope. Note that the outer function is called and imedially discarded - its return value is the original function you had:
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = (lambda val: (lambda x: x**val))(val)
shorter version
Now, due to the way Python stores default arguments to functions, it is possible to store the "current val value" as a default argument in the lambda, and avoid the need for an outer function. But that spoils the lambda signature, and the "why" that value is there is harder to understand -
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = lambda x, val=val: x**val
This question is distilled from the original application involving callback functions for Tkinter buttons. This is one line that illustrates the behavior.
lambdas = [lambda: i for i in range(3)]
if you then try invoking the lambda functions generated:
lambdas[0](), lambdas[1]() and lambdas[2]() all return 2.
The desired behavior was to have lambdas[0]() return 0, lambdas[1]() return 1, lambdas[2])() return 2.
I see that the index variable is interpreted by reference. The question is how to rephrase to have it treated by value.
Use a parameter with default value to bind the current value of i to a local variable. When the lambda gets called without an argument, the local variable i is assigned the default value:
In [110]: lambdas = [lambda i=i: i for i in range(3)]
In [111]: for lam in lambdas:
.....: print(lam())
.....:
0
1
2
When i is not a local variable, Python looks up its value in the enclosing scope. The value that is found is the last value i attained in the for-loop of the list comprehension. That is why, without the parameter with default value, each lambda returns 2 since the for-loop has completed by the time the lambdas get called.
Another common approach to solving this problem is to use a closure -- a function that can refer to environments that are no longer active such as the local namespace of an outer function, even after that function has returned.
def make_func(i):
return lambda: i
lambdas = [make_func(i) for i in range(3)]
for lam in lambdas:
print(lam())
prints
0
1
2
This works because when lam() gets called, since the i in the lambda
function's body is not a local variable, Python looks for the value of i in
the enclosing scope of the function make_func. Its local namespace is still
accessible to the closure, lam, even though make_func has already
completed. The value in that local namespace is the value which was passed to
make_func, which is, happily, the desired value of i.
As already mentioned by mkrieger1,
another way to create a new function with some argument values already supplied
is to use functools.partial:
lambdas = [functools.partial(lambda x: x, i) for i in range(3)]
The important thing to be understood here is, the functions are created during the evaluation of the list comprehension, but the value of i will be evaluated only during the execution of the functions.
So, in your case, you have created three functions and all of them refer i. When those functions are invoked at runtime, i will have the value 2, because that was the last value bound to i in the last iteration of the for loop.
Instead, you need to preserve the current value of i in each of the functions created. So, the common practice is to include a default parameter, like this
>>> lambdas = [lambda i=i: i for i in range(3)]
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Now, lambda i=i: i, creates a default parameter i which will have the current value of the loop variable i. So, when the functions are executed, the i referred is actually the parameter passed to the function, not the loop variable.
To avoid confusion, you can choose to use a different name for the variable, like this
>>> lambdas = [lambda t=i: t for i in range(3)]
If you are looking other ways to avoid this, you can use map function to apply the numbers to another function which will generate new functions with the current value, like this
>>> lambdas = map(lambda x: lambda: x, range(3))
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Here, lambda x: lambda: x creates an anonymous function which accepts a single parameter x, and returns another function which doesn't accept any parameters but returns the value x.
Note: Don't use this map form in actual code. It might reduce the readability of the code.
You can use functools.partial to create specialized functions from a more general one by partial application, which reduces the parameter count by one:
from functools import partial
lambdas = [partial(lambda x: x, i) for i in range(3)]
Here, lambda x: x is the general identity function taking one argument, and you create three specializations of it, taking no arguments, and returning fixed values.
Ahhh, further Googling found a solution (admittedly one I would not have stumbled upon myself). The desired behavior can be invoked by use of a default argument:
lambdas = [lambda i=i: i for i in range(3)]
I need to sort a dictionary file before I format it. I used list.sort and it put it
in ascii order (capital letters before lowercase). So I found this code online to do the sort. It works, but I don't fully understand how the lambda works with 2 variables and with cmp(). I am confused as to what cmp is comparing and which two variables lambda is using. Please explain how lambda works with cmp inside of the sort function.
f = open("swedish.txt", 'r')
f2 = open("swed.txt", 'w')
doc = f.read().split('\n')
doc.sort(lambda x, y: cmp(x.lower(), y.lower()))
for line in doc:
f2.write(line + '\n')
f.close()
f2.close()
To sort a list [b, a, c] one have to compare a with b and a with c and so on. Nothing else doest this lambda, it compares two components of the list with each other.
For your case, the key-argument is more suitable:
doc.sort(key=str.lower)
#Danial has the better answer for lower case compares. But since you asked about the lambda, in python 2.x, you can pass in a comparison function that takes 2 variables and returns -1, 0, 1 depending on whether var 1 is less than, equal to or greater than var 2. This feature has been removed in python 3, so consider it deprecated.
A lambda is just a function that hasn't been assigned to a variable and it is commonly used in cases like this where a call requires a function but you don't want to litter your code with small function definitions. lambda x,y: says you are defining a function with variables x and y, and cmp(x.lower(), y.lower()) is the implementation. In this case, you lower case the strings and cmp returns -1, 0 or 1 like sort requires.
Since you just want to lower case things, the cmp solution is much slower than the key solution. With key=, you only lower case the keys once, With the cmp function, you lower case two keys for every single compare.
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
What's going on here? I'm trying to create a list of functions:
def f(a,b):
return a*b
funcs = []
for i in range(0,10):
funcs.append(lambda x:f(i,x))
This isn't doing what I expect. I would expect the list to act like this:
funcs[3](3) = 9
funcs[0](5) = 0
But all the functions in the list seem to be identical, and be setting the fixed value to be 9:
funcs[3](3) = 27
funcs[3](1) = 9
funcs[2](6) = 54
Any ideas?
lambdas in python are closures.... the arguments you give it aren't going to be evaluated until the lambda is evaluated. At that time, i=9 regardless, because your iteration is finished.
The behavior you're looking for can be achieved with functools.partial
import functools
def f(a,b):
return a*b
funcs = []
for i in range(0,10):
funcs.append(functools.partial(f,i))
Yep, the usual "scoping problem" (actually a binding-later-than-you want problem, but it's often called by that name). You've already gotten the two best (because simplest) answers -- the "fake default" i=i solution, and functools.partial, so I'm only giving the third one of the classic three, the "factory lambda":
for i in range(0,10):
funcs.append((lambda i: lambda x: f(i, x))(i))
Personally I'd go with i=i if there's no risk of the functions in funcs being accidentally called with 2 parameters instead of just 1, but the factory function approach is worth considering when you need something a little bit richer than just pre-binding one arg.
There's only one i which is bound to each lambda, contrary to what you think. This is a common mistake.
One way to get what you want is:
for i in range(0,10):
funcs.append(lambda x, i=i: f(i, x))
Now you're creating a default parameter i in each lambda closure and binding to it the current value of the looping variable i.
All the lambdas end up being bound to the last one. See this question for a longer answer:
How do I create a list of Python lambdas (in a list comprehension/for loop)?
Considering the final value of i == 9
Like any good python function, it's going to use the value of the variable in the scope it was defined. Perhaps lambda: varname (being that it is a language construct) binds to the name, not the value, and evaluates that name at runtime?
Similar to:
i = 9
def foo():
print i
i = 10
foo()
I'd be quite interested in finding out of my answer is correct
I want to create a lambda function that takes two numpy arrays and sets a slice of the first to the second and returns the newly set numpy array.
Considering you can't assign things in lambda functions is there a way to do something similar to this?
The context of this is that I want to set the centre of a zeros array to another array in a single line, and the only solution I could come up with is to use reduce and lambda functions.
I.e. I'm thinking about the condensation of this (where b is given):
a = numpy.zeros( numpy.array(b.shape) + 2)
a[1:-1,1:-1] = b
Into a single line. Is this possible?
This is just an exercise in oneliners. I have the code doing what I want it to do, I'm just wondering about this for the fun of it :).
This is ugly; you should not use it. But it is oneline lambda as you've asked:
f = lambda b, a=None, s=slice(1,-1): f(b, numpy.zeros(numpy.array(b.shape) + 2))\
if a is None else (a.__setitem__([s]*a.ndim, b), a)[1]
What is __setitem__?
obj.__setitem__(index, value) is equivalent to obj[index] = value in this case. Example:
class A:
def __setitem__(self, index, value):
print 'index=%s, value=%s' % (index, value)
a = A()
a[1, 2] = 3
It prints:
index=(1, 2), value=3
Why does __setitem__() return None?
There is a general convention in Python that methods such as list.extend(), list.append() that modify an object in-place should return None. There are exceptions e.g., list.pop().
Y Combinator in Python
Here's blog post On writing Python one-liners which shows how write nameless recursive functions using lambdas (the link is suggested by #Peter Hansen).