Why lambda function to get the list of values i = 4 .During the call lambda, enclosing scope does not exist. The function f has finished work and returned control (the variable i does not exist).
def f():
L = []
for i in range(5):
L.append(lambda x: i ** x)
return L
L = f()
L[0]
def f1(N):
def f2(X):
return X**N
return f2
f=f1(2)
f (3)
9
g = f1(3)
g(3)
27
f(3)
9
Python uses closures to capture references to the original variable. The lambda objects retain a reference to the i name, through which the value can be accessed. This means that the i variable continues to live on after f completes.
You can introspect this closure in the .__closure__ tuple on the lambda objects; functions have the same attribute:
>>> L[0].__closure__
(<cell at 0x1077f8b78: int object at 0x107465880>,)
>>> L[0].__closure__[0]
<cell at 0x1077f8b78: int object at 0x107465880>
>>> L[0].__closure__[0].cell_contents
4
This is also why all lambdas in your list L refer to the value 4, and not to the numbers 0 through to 4. They all refer to the same closure:
>>> L[0].__closure__[0] is L[1].__closure__[0]
True
The closure refers to the variable, not to the value of that variable at the time the closure was defined. At the end of the loop i was last set to 4, so when looking up i in the lambda closure 4 will be found, for all lambdas in your list.
If you want your lambdas to refer to the value of i during the loop, capture it in a keyword argument:
def f():
L = []
for i in range(5):
L.append(lambda x, i=i: i ** x)
return L
Now i is a local variable to the lambda, not a closure.
Alternatively, create an entirely new scope from which to draw the closure:
def create_lambda(i):
return lambda x: i ** x
def f():
return [create_lambda(i) for i in range(5)]
Now create_lambda() is a new scope with it's own local i for the lambda closure to refer to. The lambdas then each have their own closures:
>>> L[0].__closure__[0] is L[1].__closure__[0]
False
Closures refer to a variable in a specific namespace; each time you call a function a new local namespace is created, so each closure refers to i in create_lambda in a separate namespace from other calls to create_lambda.
Related
def f():
print (x)
def g():
print (x)
x = 1
x = 3
f()
x = 3
g()
I am learning functions now, why are empty arguments used in these functions ?
No arguments are being used (as none are being passed).
Inside f, x is a free variable. There is no local variable by that name, so its value is looked up in the next enclosing scope, which in this case is the global scope.
Inside g, x is a local variable (by virtue of a value being assigned to it), but it is not yet defined when you call print(x). If you were to reverse the order of the assignment and the call to print, you would see 1 as the output, since the global variable by the same name is ignored.
If I have a function which has some non-local variable (in a closure), how do I access that variable? Can I modify it, and if so, how? Here's an example of such a function:
def outer():
x = 1
def inner(y):
nonlocal x
return x + y
return inner
inner = outer()
# how do I get / change the value of x inside inner?
(apologies if this is already answered elsewhere; I couldn't find it, so I thought I would share the answer once I worked it out)
A function's enclosed variables are stored as a tuple in the __closure__ attribute. The variables are stored as a cell type, which seems to just be a mutable container for the variable itself. You can access the variable a cell stores as cell.cell_contents. Because cells are mutable, you can change the values of a function's non-local variables by changing the cell contents. Here's an example:
def outer():
x = 1
def inner(y):
nonlocal x
return x + y
return inner
inner = outer()
print(inner(2)) # 3
print(inner.__closure__) # (<cell at 0x7f14356caf78: int object at 0x56487ab30380>,)
print(inner.__closure__[0].cell_contents) # 1
inner.__closure__[0].cell_contents = 10
print(inner(2)) # 12
print(inner.__closure__[0].cell_contents) # 10
EDIT - the above answer applies to Python 3.7+. For other Python versions, you can access the closure the same way, but you can't modify the enclosed variables (here's the Python issue that tracked setting cell values).
I'm trying to build a counter in python with the property of closure. The code in the following works:
def generate_counter():
CNT = [0]
def add_one():
CNT[0] = CNT[0] + 1
return CNT[0]
return add_one
However when I change the list CNT to a var, it did not work:
def generate_counter1():
x = 0
def add_one():
x = x + 1
return x
return add_one
when I print the closure property of an instance, I found the __closure__ of the second case is none:
>>> ct1 = generate_counter()
>>> ct2 = generate_counter1()
>>> print(ct1.__closure__[0])
<cell at 0xb723765c: list object at 0xb724370c>
>>> print(ct2.__closure__)
None
Just wondering why the index in outer function has to be a list?
Thanks for the answers! Found the docs which clearly explained this problem
https://docs.python.org/3/faq/programming.html#why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value
Python determines the scope of a name by looking at name binding behaviour; assignment is one such behaviour (function parameters, importing, the target in for target ... or while .. as target are other examples). A name you bind to in a function is considered local. See the Naming and Binding section of the reference documentation.
So the name x in your second example is a local variable, because you assigned directly to it:
x = x + 1
In fact, because you never gave that x a local value, you'll get an exception when you try to use that function; the local name is unbound when you try to read it:
>>> generate_counter1()()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in add_one
UnboundLocalError: local variable 'x' referenced before assignment
In your first example no such binding takes place; you are instead altering the contents of CNT, what that name references is not altered.
If you are using Python 3, you can override the decision to make a name local, by using a nonlocal statement:
def generate_counter2():
x = 0
def add_one():
nonlocal x
x = x + 1
return x
return add_one
By making x non-local, Python finds it in the parent context and creates a closure for it again.
>>> def generate_counter2():
... x = 0
... def add_one():
... nonlocal x
... x = x + 1
... return x
... return add_one
...
>>> generate_counter2().__closure__
(<cell at 0x1078c62e8: int object at 0x1072c8070>,)
nonlocal is new in Python 3; in Python 2 you are limited to tricks like using a mutable list object to evade the binding rule. Another trick would be to assign the counter to an attribute of the nested function; again, this avoids binding a name in the current scope:
def generate_counter3():
def add_one():
add_one.x += 1
return add_one.x
add_one.x = 0
return add_one
It doesn't have to be a list, it just has to be an mutable object which you mutate, not reassign.
From the docs:
If a variable is assigned a value anywhere within the function’s body, it’s assumed to be a local unless explicitly declared as global.
Thus, in your second example, x is considered local (to the inner function), and, after your first assignment, you're shadowing the outer 'x'.
On the other hand, in the first example (since you don't assign a value to CNT) it operates on the CNT defined in the outer function.
As a follow up to this question
Why can I change the global value of x.a within def A? I am guessing it has to do with the fact that it is a class since it would not work with a regular variable because it would be redefined in the local scope of def A, but I am not quite understanding what is happening.
Case 1
class X:
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
class Y:
def A(self):
print(x.a,x.b,x.c)
x.a = 4
x = X()
y = Y()
y.A()
print(x.a,x.b,x.c)
If you set x = Y() in A scope it would create a local x in that scope. In this case however, you are not setting x, you are setting x.a. Looking up variables takes into account global variables too. Imagine you are doing this instead setattr(x, "a", 4) and it will make more sense.
Also if I remember correctly you can "import" global variables into a function scope by using the global keyword. (see Use of "global" keyword in Python)
Global names can be read within functions:
x = 5
def read_x():
print(x)
Globals that reference mutable types can also be mutated within functions:
x = [1, 2, 3]
def mutate_x():
x[0] = 'One'
What you can't do to a global within the scope of a function is assignment:
x = 5
def set_x():
# this simply assigns a variable named x local to this function -- it doesn't modify the global x
x = 3
# now, back outside the scope of set_x, x remains 5
print(x)
5
Unless you explicitly declare the global within the scope of the function:
x = 5
def set_x():
global x
x = 3
# back outside the function's scope
print(x)
3
What you're doing in your example is "mutating" -- modifying an attribute of an object. Assigning a value to an attribute of a user-defined type is one example of mutation, just like modifying an element of a list or a dictionary. That's why this works.
The class A doesn't have an init method that defines x, so the method A, when you try to access the attribute a of x, try to find the x object in the local scope, and since it can't find it then it looks the outside scope where an object name x is present, it grabs that object and override the attribute a.
So basically what you are doing is modify the actual attribute a of the object x that you create before you call y.A().
It's the very basic foundation of a closure: access a variable that is defined outside the local scope.
I am reading this article about decorator.
At Step 8 , there is a function defined as:
def outer():
x = 1
def inner():
print x # 1
return inner
and if we run it by:
>>> foo = outer()
>>> foo.func_closure # doctest: +ELLIPSIS
it doesn't print x. According to the explanation :
Everything works according to Python’s scoping rules - x is a local
variable in our function outer. When inner prints x at point #1 Python
looks for a local variable to inner and not finding it looks in the
enclosing scope which is the function outer, finding it there.
But what about things from the point of view of variable lifetime? Our
variable x is local to the function outer which means it only exists
while the function outer is running. We aren’t able to call inner till
after the return of outer so according to our model of how Python
works, x shouldn’t exist anymore by the time we call inner and perhaps
a runtime error of some kind should occur.
However, I don't really understand what the second paragraph means.
I understand inner() does get the value of x but why it doesn't print x out?
thanks
UPDATE:
Thanks all for the answers. Now I understand the reason.
the "return inner" is just a pointer to inner() but it doesn't get executed, that is why inner() doesn't print x as it is not called at all
I understand inner() does get the value of x but why it doesn't print
x out?
It doesn't print out anything because you've not called the inner function yet.
>>> def outer():
x = 1
def inner():
print x # 1
return inner
...
>>> func = outer()
>>> func
<function inner at 0xb61e280c>
>>> func()
1
This is called a closure, i.e even though the outer function is not in stack(finished executing) anymore but still the inner function that was returned from it remembers it's state.(i.e value of x)
>>> def outer():
x = 1
y = 2
def inner():
z=3
print x
return inner
...
>>> func = outer()
>>> func.func_code.co_freevars #returns the variables that were used in closure
('x',)
From the source code on how python decides it's a closure or not:
459 if len(code.co_freevars) == 0:
460 closure = NULL
461 else:
462 len(closure) == len(code.co_freevars)
In py3.x you can also modify the value of x using nonlocal statement inside inner function.
>>> def outer():
x = 1
def inner():
nonlocal x
x += 1
print (x)
return inner
...
>>> func = outer()
>>> func()
2
>>> func()
3
>>> func()
4
You are not calling inner. You have called outer, which returns inner, but without calling it. If you want to call inner, do foo() (since you assinged the result of outer() to the name foo).
The paragraph you quoted is sort of tangential to this issue. You say you already understand why inner gets the value of x, which is what that paragraph is explaining. Basically, if a local variable is used in a nested function, and that nested function is returned, the value of the variable is stored along with the returned function, even if the scope where that variable was defined in no longer active. Normally x would be gone after outer finished, because x is just local to outer. But outer returns inner, which still needs access to x. So x gets wrapped up into what's called a closure, so it can still be accessed by inner later on.