Develop Reference

Out-print of a function-call [duplicate]

This question already has answers here:
Modifying a list inside a function
(4 answers)
Closed 2 years ago.
I am currently new to python and I'm still learning the basics, but there is one thing I just can't wrap my head around. Why is the code in Q.1 giving the out-print 3, while Q.2 is giving the out-print [4]?
When asked, I was told that the f(x)-line at the bottom of Q.1 isn't given any variable or box to hold the new return-value given from the def. and that's the reason why the out-print of x remain 3.
This made sense to me, but then why would the out-print in Q.2 equal the return-value of g(x)? There isn't any variable or box to contain the return-value of x in this code either..
Q.1:
def f(x):
x += 1
return x
x=3
f(x)
print(x)
Q.2:
def g(x):
x[0] = x[0] + 1
return x
x = [3]
g(x)
print(x)

A Python function takes arguments by reference (something that points to the real object) whenever that argument is "complex", and takes the argument by value (a copy of the item) when it's a simple thing.
Q.1 is taking an int as an argument, then the function creates an internal copy of it and thus does not modify the value of x outside the function.
Q.2 is taking a list as an argument, which is considered complex, in which case the function takes a reference to the original and whatever happens to that reference will also happen to the original.
You can find an explanation of pass-by-reference and pass-by-value with images here

In Python, lists are mutable objects, and as a result they are passed-by-reference when used in a function call. This means that when g(x) adds 1 to the first element of x, after the function exits, the original list x will contain those changes.
Note that the specific term used is not "pass-by-reference", but rather "pass-by-name", which comes with a couple different semantics, which you can learn more about if you wish.

The function defined is Q1 is returning a value of for. x contains 3 and is passed into the function, by calling it with f(x). It gets incremented and returned to the function call. But, the function call was not stored in a variable, so it was not saved into memory. Nothing was done with the returned value. Calling the function is only editing the variable x within the local scope (within the function). When you're using print(x) it is referencing the global variable of x, which still contains 3.
In Q2, lists are mutable. Editing them within a function, the changes persist in the global scope. Because that list is mutated in the global scope, using print(x) uses the updated global variable.
I hope this makes sense. Look into scope of variables in the documentation for more.

Q1
def f(x):
x += 1
return x
x=3
f(x)
print(x)
The reason this is returning 3 and not 4 is because you haven't rebound your variable to reference this new value. Instead of f(x) you can do x = f(x).
Q2
def g(var):
var[0] = var[0] + 1
return var
x = [3]
g(x)
print(x)
To answer this without making it confusing I've changed the local variable used in the function to var so you can see what I'm trying to explain to you easier.
First you are creating a list with a integer value 3 in the first spot in the list (element 0) and you make x reference this list.
When you call g() function and pass the list, the function sets var to reference the same list (not a different one, the same one). You then tell the function to increase the integer value in the first element by 1. Since lists are mutable with certain methods, you have already changed the list for both the local variable var, and the global variable x. This means that you actually don't need to use any return of the function because the list has been mutated in place.
Have a look at this video https://youtu.be/_AEJHKGk9ns where Ned Batchelder explains more about this.

creating a dictionary with anonymous functions in Python [duplicate]

This question already has answers here:
What do lambda function closures capture?
(7 answers)
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 3 years ago.
I have a vector with some parameters and I would like to create a dictionary of anonymous (lambda) functions in Python3.
The goal of this is to create a callable function that gives values equal to the sum of these functions with the same argument.
I am struggling even to create a dictionary with the original lambda function objects and get them to behave consistently. I use the following code:
import numpy as np
a = np.linspace(0,2.0,10)
func = {}
for i,val in enumerate(a):
print(i)
func['f{}'.format(i)] = lambda x: x**val
print(func['f0'](0.5))
print(func['f1'](0.5))
print(func['f2'](0.5))
print(func['f3'](0.5))
The output of the final print statements gives the same value, whereas I would like it to give the values corresponding to x**val with the value of val coming from the originally constructed array a.
I guess what's happening is that the lambda functions always reference the "current" value of val, which, after the loop is executed is always the last value in the array? This makes sense because the output is:
0
1
2
3
4
5
6
7
8
9
0.25
0.25
0.25
0.25
The output makes sense because it is the result of 0.5**2.0 and the exponent is the last value that val takes on in the loop.
I don't really understand this because I would have thought val would go out of scope after the loop is run, but I'm assuming this is part of the "magic" of lambda functions in that they will keep variables that they need to compute the function in scope for longer.
I guess what I need to do is to insert the "literal" value of val at that point into the lambda function, but I've never done that and don't know how.
I would like to know how to properly insert the literal value of val into the lambda functions constructed at each iteration of the loop. I would also like to know if there is a better way to accomplish what I need to.
EDIT: it has been suggested that this question is a duplicate. I think it is a duplicate of the list comprehension post because the best answer is virtually identical and lambda functions are used.
I think it is not a duplicate of the lexical closures post, although I think it is important that this post was mentioned. That post gives a deeper understanding of the underlying causes for this behavior but the original question specifically states "mindful avoidance of lambda functions," which makes it a bit different. I'm not sure what the purpose of that mindful avoidance is, but the post did teach related lessons on scoping.

The problem with this approach is that val used inside your lambda function is the live variable, outside. When each lambda is called, the value used for val in the formula is the current value of val, therefore all your results are the same.
The solution is to "freeze" the value for val when creating each lambda function - the way that is easier to understand what is going on is to have an outer lambda function, that will take val as an input, and return your desided (inner) lambda - but with val frozen in a different scope. Note that the outer function is called and imedially discarded - its return value is the original function you had:
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = (lambda val: (lambda x: x**val))(val)
shorter version
Now, due to the way Python stores default arguments to functions, it is possible to store the "current val value" as a default argument in the lambda, and avoid the need for an outer function. But that spoils the lambda signature, and the "why" that value is there is harder to understand -
for i,val in enumerate(a):
print(i)
func[f'f{i}'] = lambda x, val=val: x**val

Initialising a list of lambda functions in python

I have a tuple of functions that I want to pre-load with some data. Currently the way I am doing this is below. Essentially, I make a list of the new functions, and add the lambda functions to it one at a time, then reconvert to a tuple. However, when I use these functions in a different part of the code, every one of them acts as if it were the last one in the list.
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x: f(x, data)
newfuncs.append(newf)
return tuple(newfuncs)
Here is a simple example of the problem
funcs = (lambda x, y: x + y, lambda a, b: a - b)
funcs = newfuncs(10, funcs)
print(funcs[0](5))
print(funcs[1](5))
I would expect the number 15 to be printed, then -5. However, this code prints the number -5 twice. If anyone can help my understand why this is happening, it would be greatly appreciated. Thanks!

As mentioned, the issue is with the variable f, which is the same variable assigned to all lambda functions, so at the end of the loop, every lambda sees the same f.
The solution here is to either use functools.partial, or create a scoped default argument for the lambda:
def newfuncs(data, funcs):
newfuncs = []
for f in funcs:
newf = lambda x, f=f: f(x, data) # note the f=f bit here
newfuncs.append(newf)
return tuple(newfuncs)
Calling these lambdas as before now gives:
15
-5
If you're using python3.x, make sure to take a look at this comment by ShadowRanger as a possible safety feature to the scoped default arg approach.

This is a well-known Python "issue," or should I say "this is just the way Python is."
You created the tuple:
( x => f(x, data), x => f(x, data) )
But what is f? f is not evaluated until you finally call the functions!
First, f was (x, y)=>x+y. Then in your for-loop, f was reassigned to (x, y)=>x-y.
When you finally get around to calling your functions, then, and only then, will the value of f be looked up. What is the value of f at this point? The value is (x, y)=>x-y for all of your functions. All of your functions do subtraction. This is because f is reassigned. There is ONLY ONE f. And the value of that one and only one f is set to the subtraction function, before any of your lambdas ever get called.
ADDENDUM
In case anyone is interested, different languages approach this problem in different ways. JavaScript is rather interesting (some would say confusing) here because it does things the Python way, which the OP found unexpected, as well as a different way, which the OP would have expected. In JavaScript:
> let funcs = []
> for (let f of [(x,y)=>x+y, (x,y)=>x-y]) {
funcs.push(x=>f(x,10));
}
> funcs[0](5)
15
funcs[1](5)
-5
However, if you change let to var above, it behaves like Python and you get -5 for both! This is because with let, you get a different f for each iteration of the for-loop; with var, the whole function shares the same f, which keeps getting reassigned. This is how Python works.
cᴏʟᴅsᴘᴇᴇᴅ has shown you that the way to do what you expect is to make sure you get that different f for each iteration, which Python allows you do in a pretty neat way, defaulting the second argument of the lambda to a f local to that iteration. It's pretty cool, so their answer should be accepted if it helps you.

Python - appending a function to a list [duplicate]

This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 years ago.
I want to create a list that contains the monomials up to degree n
basis = [lambda x:x**i for i in range(0,n+1)]
This however creates a list of n functions, but all the same one (of degree n) and not of degree 0,1,2,...n
I tried without list comprehension as well:
basis = []
for i in range(0,n+1):
basis.append(lambda x:x**i)
but with the same result. Substituting the lambda function by a classic function definition also did not fix it.
I checked Python lambdas and scoping, but that did not help, since I don't want to store function values, but the function itself. For example, I want to be able to call
basis[0](34)
and this should return
1

As i said in the comments, take a look at partial
def f(x, n):
return x**n
basis = [partial(f, n=i) for i in range(10)]
print(basis[0](34)) # 1

This entry sums it up perfectly
http://docs.python-guide.org/en/latest/writing/gotchas/#late-binding-closures :
... you can create a closure that binds immediately to its arguments by using a default arg like so:
def create_multipliers():
return [lambda x, i=i : i * x for i in range(5)]
You code would become:
basis = [lambda x,i=i:x**i for i in range(0,n+1)]
This is a bit of a hacky solution, but works. I still highly recommend to read the link provided as the 3 points that are made there are usual errors when you are new to Python.

How to generate a list of different lambda functions with list comprehension?

This question is distilled from the original application involving callback functions for Tkinter buttons. This is one line that illustrates the behavior.
lambdas = [lambda: i for i in range(3)]
if you then try invoking the lambda functions generated:
lambdas[0](), lambdas[1]() and lambdas[2]() all return 2.
The desired behavior was to have lambdas[0]() return 0, lambdas[1]() return 1, lambdas[2])() return 2.
I see that the index variable is interpreted by reference. The question is how to rephrase to have it treated by value.

Use a parameter with default value to bind the current value of i to a local variable. When the lambda gets called without an argument, the local variable i is assigned the default value:
In [110]: lambdas = [lambda i=i: i for i in range(3)]
In [111]: for lam in lambdas:
.....: print(lam())
.....:
0
1
2
When i is not a local variable, Python looks up its value in the enclosing scope. The value that is found is the last value i attained in the for-loop of the list comprehension. That is why, without the parameter with default value, each lambda returns 2 since the for-loop has completed by the time the lambdas get called.
Another common approach to solving this problem is to use a closure -- a function that can refer to environments that are no longer active such as the local namespace of an outer function, even after that function has returned.
def make_func(i):
return lambda: i
lambdas = [make_func(i) for i in range(3)]
for lam in lambdas:
print(lam())
prints
0
1
2
This works because when lam() gets called, since the i in the lambda
function's body is not a local variable, Python looks for the value of i in
the enclosing scope of the function make_func. Its local namespace is still
accessible to the closure, lam, even though make_func has already
completed. The value in that local namespace is the value which was passed to
make_func, which is, happily, the desired value of i.
As already mentioned by mkrieger1,
another way to create a new function with some argument values already supplied
is to use functools.partial:
lambdas = [functools.partial(lambda x: x, i) for i in range(3)]

The important thing to be understood here is, the functions are created during the evaluation of the list comprehension, but the value of i will be evaluated only during the execution of the functions.
So, in your case, you have created three functions and all of them refer i. When those functions are invoked at runtime, i will have the value 2, because that was the last value bound to i in the last iteration of the for loop.
Instead, you need to preserve the current value of i in each of the functions created. So, the common practice is to include a default parameter, like this
>>> lambdas = [lambda i=i: i for i in range(3)]
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Now, lambda i=i: i, creates a default parameter i which will have the current value of the loop variable i. So, when the functions are executed, the i referred is actually the parameter passed to the function, not the loop variable.
To avoid confusion, you can choose to use a different name for the variable, like this
>>> lambdas = [lambda t=i: t for i in range(3)]
If you are looking other ways to avoid this, you can use map function to apply the numbers to another function which will generate new functions with the current value, like this
>>> lambdas = map(lambda x: lambda: x, range(3))
>>> lambdas[0]()
0
>>> lambdas[1]()
1
>>> lambdas[2]()
2
Here, lambda x: lambda: x creates an anonymous function which accepts a single parameter x, and returns another function which doesn't accept any parameters but returns the value x.
Note: Don't use this map form in actual code. It might reduce the readability of the code.

You can use functools.partial to create specialized functions from a more general one by partial application, which reduces the parameter count by one:
from functools import partial
lambdas = [partial(lambda x: x, i) for i in range(3)]
Here, lambda x: x is the general identity function taking one argument, and you create three specializations of it, taking no arguments, and returning fixed values.

Ahhh, further Googling found a solution (admittedly one I would not have stumbled upon myself). The desired behavior can be invoked by use of a default argument:
lambdas = [lambda i=i: i for i in range(3)]