This question already has answers here:
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
I was surprised that this assertion fails:
x = 42
x = lambda: x
assert x() == 42
It seems that x ends up recursively referring to itself, so that x(), x()(), etc. are all functions.
What is the rule used to parse this, and where is this documented?
By the way (not unexpectedly given the above), the original value of x has no references left after the lambda definition:
class X:
def __del__(self): print('deleting')
x = X()
x = lambda: x # 'deleting' is printed here
The variable x is created by the first assignment, and rebound with the second assignment.
Since the x in the lambda isn't evaluated until the lambda is called, calling it will evaluate to the most recently assigned value.
Note that this is not dynamic scoping - if it were dynamic, the following would print "99", but it prints "<function ...":
x = 42
x = lambda: x
def test(f):
x = 99
print(f())
test(x)
The first assignment is irrelevant; the x in the body of the lambda is bound late:
x = lambda: x # no need for a prior assignment
x = lambda: y # notice: no NameError occurs, *until it is called*
This is the same reason that creating lambdas in a loop is tricky, and is also used to make trees with the standard library defaultdict:
tree = lambda: defaultdict(tree)
t = tree()
t['foo']['bar']['baz'] = 'look ma, no intermediate steps'
A lambda is an anonymous function object. Python completely resolves whatever is on the right side of an equation to a single anonymous object and then resolves whatever is on the left side for assignment.
x = lambda: x
first compiles lambda: x into a function object that returns whatever happens to be in x at the time it is called. It then rebinds x with this function object, deleting whatever object happened to be there before.
Now x is a function that returns whatever is in x... which is a function that returns whatever is in x, etc... So you can write x()()()()()() as many times as you want, and still get that orginal lambda:x function object.
Python functions have a local namespace but only variables assigned in the function reside there. Since x isn't assigned in the lambda, it's resolved in the containing scope - that is, the module level "x". An identical piece of code is
def x():
return x
Contrast this with
def x():
x = 1
return x
Now, the parameter x is a local variable and is unrelated to the global x.
Related
This question already has answers here:
Why can a function modify some arguments as perceived by the caller, but not others?
(13 answers)
Closed 7 months ago.
Once I define a list in main() and passed it as a function parameter, I expected that its content will remain the same once we exit the function. I have the following code snippet:
def test(a):
a.append(1000)
return a
def main():
a = [1,2,3]
print(a, test(a))
main()
Output is clear.
# Output: [1,2,3,1000] [1,2,3,1000]
Even though the initial list contains 3 elements, function test() seems to modify its content. This thing does not happen to variables:
def test(a):
a = a + 10
return a
def main():
a = 5
print(a, test(a))
main()
# Output: 5 15
Why is that? Why does Python modify the initial lists and not variables?
a.append()
mutates the a which was passed by reference
a = a + 10
assigns a value to a new local variable in the function called a
Integers are immutable, but if there were some method that could be called on that a (on the right hand side) to mutate it, it would also be reflected in the caller.
Some objects in python (like lists, dicts and sets) are mutable, whereas others (like tuples, frozensets, strings and numbers) are immutable.
Operations on mutable objects modify the object themselves. Many immutable objects also allow operations, which in this case return a new object. (Internally this is implemented with the dunder "magic methods", which in this case just return a new object. x OPERATOR y actually calls a method on x(and/or y).)
Since in python x = y binds the name x to point to y, if y is mutable and you modify it, x will point to the modified y. But if y is immutable (but allows operations), operations on y will create a new y, to which x no longer points:
x = [] # mutable
y = x # y and x both point to a particular '[]'
x += ["hi"]
y == ["hi"] # True
x = 0
y = x # y and x both point to 0
x += 7 # x now points to 7
y == 7 # False
Exactly the same thing goes on when you parse an argument to a function---you bind a name in the function's scope pointing to that object. What then happens to the object pointed to by the name depends upon the type of object and what you do to it. If it's mutable and you mutate the object pointed to by the name in the function, you will mutate the object itself, and any other names bound to that object ouside the function will see the changed object.
This is the reason that we don't generally do this:
def f(x=[]):
...
Since f is evaluated once, when the module is first loaded, x will point to the same list for the duration of the function. Thus if the function does anything to x, x will be in that state next time it's called (which is rarely what you want).
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Lambda in a loop [duplicate]
(4 answers)
Closed last year.
I am trying to generate a list of lambdas that I will later apply to an object, but when I try to do it via a comprehension or a loop over a list, the reference to the variable is kept, rather than the value itself. Let me illustrate.
Assume your object class is something like this:
class Object:
def function(self, x):
print(x)
So when you create the object and invoke it you get something like this:
o = Object()
o.function(0)
>>> 0
Now, if I manually construct my list of lambdas it would look like this:
lambdas = [
lambda x: x.function(0),
lambda x: x.function(1),
lambda x: x.function(2)
]
Which I can then apply to my previously created object:
for l in lambdas:
l(o)
>>> 0
>>> 1
>>> 2
However, when I generate the lambda list from another list, I only get the reference to the latest element of the list:
lambdas = [lambda x: x.function(i) for i in range(2)]
for l in lambdas:
l(o)
>>> 2
>>> 2
>>> 2
On closer inspection I can see that each lambda has a different memory address, so they are NOT references to the same function.
So I can only assume that the lambda is keeping a reference to i which has a final value of 2 and therefore when invoked, it takes the value.
So my question is if its possible to set the value of the variable inside the lambda before invocation?
Note: The usa case for a list of lambdas is to pass to the agg function of a Pandas groupby on a DataFrame. I am not looking for a solution to the pandas problem, but curious about the general solution.
Generator Option
Just change lambdas to a generator instead of a list, this will cause it redefine i on every call:
lambdas = (lambda x: x.function(i) for i in range(2))
for l in lambdas:
print(l(o))
Full code:
class Object:
def function(self, x):
print(x)
o = Object()
o.function(0) #manual call
lambdas = (lambda x: x.function(i) for i in range(2))
for l in lambdas:
l(o)
Output:
0 #output from manual call
0 #output from generator
1 #output from generator
List Option
If you need a list for things like lambdas[0](o) you can send i to lambda each iteration by using i=i like so:
lambdas = [lambda x, i=i: x.function(i) for i in range(2)]
Example of second option:
class Object:
def function(self, x):
print(x)
o = Object()
lambdas = [lambda x, i=i: x.function(i) for i in range(2)] #notice the cahnge
for i in range(len(lambdas)):
lambdas[i](o) #notice the change
Output:
0
1
What takes place is that in this expression, the "living" (nonlocal) i variable is
used inside each lambda created. And at the end of the for loop, its value is the last value taken - which will be used when the lambdas are actually called.
lambdas = [lambda x: x.function(i) for i in range(2)]
The fix for that is to create an intermediary namespace which will "freeze" the nonlocal variable value at the time the lambda is created. This is usually done with another lambda:
lambdas = [(lambda i: (lambda x: x.function(i)))(i) for i in range(2)]
So, bear with me - in the above expression, for each execution of the for i loop, a new, disposable lambda i is created and called imediatelly with the current value of the i used in the for. Inside it, this value is bound to a local i variable, that is unique to this disposable lambda i (in Python internal workings, it gets its own "cell"). This unique iis then used in the second, permanent, lambda x expression. Whenever that one is called, it will use the i value persisted in the outter lambda i call. The external lambda i then returns the lambda x expression as its result, but its nonlocal i is bound to the value used inside the lambda i, not the one used in the for i.
This is a common problem in Python, but can't be fixed because it is part of how the language works.
There is a shorter, and working, form to "freeze" the i from for i when each lambda i is created, that does not require an outer function scope: when a function is created, the values passed as default for its parameters are stored along with the function. Then, if one stores the current value of i as a default value, it won't change when the variable i itself does:
lambdas = [lambda x, i=i: x.function(i) for i in range(2)]
Here, in the lambda x, i=i: snippet, the value of i in the scope the lambda is created is stored as the default value for the parameter i, which works as a local (in contrast with a nonlocal) variable inside the lambda function itself.
This question already has answers here:
Python functions call by reference [duplicate]
(12 answers)
Closed 6 years ago.
makebbb(a):
a = "bbb"
this function obviously fails to convert it's input to "bbb" as demonstrated by the following snippet
x = "notbbb"
makebbb(x)
print(x)
# Outputs "notbbb"
Now I understand the reason this fails, what I wish to find out is if there is anyway to make such a function? (I'm not just talking about assigning strings, but assignment with = in general)
Define the variable you want to change from a func as global ,
>>> def makebbb():
global x
x = "bbb"
>>> print x
notbbb
>>> makebbb()
>>> print x
bbb
>>>
why not simply return a value in your function and update 'a'?
def makebbb(a):
return "bbb"
x = "notbbb"
x = makebbb(x)
print(x)
def makebbb(a):
return "bbb"
x = "notbbb"
x = makebbb(x) #call function in the assignment to modify it
print(x)
Output:
bbb
If x exists outside the scope of the function, you can modify the variable directly (rebind values to the same variable in memory). Otherwise, a functional approach would be to just return a new string.
x = "notbbb"
makebbb() # Modify variable outside function scope.
print(x)
or
x = "notbbb"
print(makebbb(x)) # return new value.
The particular catch with this example is that you are trying to pass a an immutable value by reference. What happens instead is that a copy is created on argument passing. You can achieve the effect you want with mutable types such as lists though.
You can modify some arguments, those that are passed by reference instead of by value.
Arguments that are mutable objects can be changed, like so:
def change_list(l):
l.append(4)
k = [1, 2, 3]
change_list(k)
print(k) # Prints [1,2,3,4]
This, however, will not work:
def dont_change_list(l):
l = [5]
This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 9 years ago.
i need help-i try to send value to method like in c++ by ref/by ptr
how can i do it?
to exmple:
def test(x):
x=3
x=2
test(x)
print(x)
In this case x a local variable in test method and will not change the "original" X
so how can i change the "original" X?
thanks
In some ways, all calls in Python are called with references. In fact, all variables are references in a sense. But some types, like int from your example, cannot be changed.
In the case of, say, a list, the functionality you're looking for is trivial:
def change_it(some_list):
some_list.append("world")
foo = ["hello"]
change_it(foo)
print(foo) # prints ['hello', 'world']
Note, however, that reassigning the parameter variable some_list does not change the value in the calling context.
If you're asking this question, though, you're probably looking to do something like set two or three variables using one function. In that case, you're looking for something like this:
def foo_bar(x, y, z):
return 2*x, 3*y, 4*z
x = 3
y = 4
z = 5
x, y, z = foo_bar(x, y, z)
print(y) # prints 12
Of course, you can do anything in Python, but that doesn't mean you should. In the fashion of the TV show Mythbusters, here's something that does what you're looking for
import inspect
def foo(bar):
frame = inspect.currentframe()
outer = inspect.getouterframes(frame)[1][0]
outer.f_locals[bar] = 2 * outer.f_locals[bar]
a = 15
foo("a")
print(a) # prints 30
or even worse:
import inspect
import re
def foo(bar):
# get the current call stack
my_stack = inspect.stack()
# get the outer frame object off of the stack
outer = my_stack[1][0]
# get the calling line of code; see the inspect module documentation
# only works if the call is not split across multiple lines of code
calling_line = my_stack[1][4][0]
# get this function's name
my_name = my_stack[0][3]
# do a regular expression search for the function call in traditional form
# and extract the name of the first parameter
m = re.search(my_name + "\s*\(\s*(\w+)\s*\)", calling_line)
if m:
# finally, set the variable in the outer context
outer.f_locals[m.group(1)] = 2 * outer.f_locals[m.group(1)]
else:
raise TypeError("Non-traditional function call. Why don't you just"
" give up on pass-by-reference already?")
# now this works like you would expect
a = 15
foo(a)
print(a)
# but then this doesn't work:
baz = foo_bar
baz(a) # raises TypeError
# and this *really*, disastrously doesn't work
a, b = 15, 20
foo_bar, baz = str, foo_bar
baz(b) and foo_bar(a)
print(a, b) # prints 30, 20
Please, please, please, don't do this. I only put it in here to inspire the reader to look into some of the more obscure parts of Python.
As far as I am aware, this doesn't exist in Python (although a similar thing occurs if you pass mutable objects to a function). You would do either
def test():
global x
x = 3
test()
or
def test(x):
return 3
x = test(x)
The second of these is much preferred.
I started working in Python just recently and haven't fully learned all the nuts and bolts of it, but recently I came across this post that explains why python has closures, in there, there is a sample code that goes like this:
y = 0
def foo():
x = [0]
def bar():
print x[0], y
def change(z):
global y
x[0] = y = z
change(1)
bar()
change(2)
bar()
change(3)
bar()
change(4)
bar()
foo()
1 1
2 2
3 3
and basically I don't understand how it actually works, and what construct like x[0] does in this case, or actually I understand what it's doing, I just don't get how is it this :)
Before the nonlocal keyword was added in Python 3 (and still today, if you're stuck on 2.* for whatever reason), a nested function just couldn't rebind a local barename of its outer function -- because, normally, an assignment statement to a barename, such as x = 23, means that x is a local name for the function containing that statement. global exists (and has existed for a long time) to allow assignments to bind or rebind module-level barenames -- but nothing (except for nonlocal in Python 3, as I said) to allow assignments to bind or rebind names in the outer function.
The solution is of course very simple: since you cannot bind or rebind such a barename, use instead a name that is not bare -- an indexing or an attribute of some object named in the outer function. Of course, said object must be of a type that lets you rebind an indexing (e.g., a list), or one that lets you bind or rebind an attribute (e.g., a function), and a list is normally the simplest and most direct approach for this. x is exactly that list in this code sample -- it exists only in order to let nested function change rebind x[0].
It might be simpler to understand if you look at this simplified code where I have removed the global variable:
def foo():
x = [0]
def bar():
print x[0]
def change(z):
x[0] = z
change(1)
bar()
foo()
The first line in foo creates a list with one element. Then bar is defined to be a function which prints the first element in x and the function change modifies the first element of the list. When change(1) is called the value of x becomes [1].
This code is trying to explain when python creates a new variable, and when python reuses an existing variable. I rewrote the above code slightly to make the point more clear.
y = "lion"
def foo():
x = ["tiger"]
w = "bear"
def bar():
print y, x[0], w
def change(z):
global y
x[0] = z
y = z
w = z
bar()
change("zap")
bar()
foo()
This will produce this output:
lion tiger bear
zap zap bear
The point is that the inner function change is able to affect the variable y, and the elements of array x, but it does not change w (because it gets its own local variable w that is not shared).
x = [0] creates a new list with the value 0 in it. x[0] references the zero-eth element in the list, which also happens to be zero.
The example is referencing closures, or passable blocks of code within code.