I am using Python 3.7.0 and I am doing some experiments in order to comprehend the subtleties of variable scopes but I still don't understand this behaviour.
When I execute this code:
def f():
x=0
def g():y=x+1;x=y
return g
f()()
I get UnboundLocalError: local variable 'x' referenced before assignment
However when I execute this one:
def f():
x=0
def g():y=x+1
return g
f()()
It works fine. And even this one:
def f():
x=0
def g():x=1
return g
f()()
It also works fine. So I am confused. It seems to me that if assigning the value 1 to the nonlocal variable x in the g function works fine alone and on the other hand if assigning an expression containing x to a local variable y in the function g works also fine then the instructions y=x+1 and x=y should both work. I don't understand what is causing the error. I feel I am missing something fundamental in my understanding of Python's behaviour.
I'll take an ill-advised crack at this. In the following code:
def f():
x = 0 # assignment makes this x local to f()
def g():
y = x + 1
x = y # assignment makes this x local to g()
return g
f()()
The assignment of x inside g() forces it to be a variable local to g(). So the first line causes the error as you accessing the value of an unassigned local. I believe this "assignment makes it local" logic doesn't follow the flow of your program -- the function is viewed as whole to make this determination and then the code is looked at line by line.
You can access any variable in scope from a function, but you can only set locals. Unless you've declared the variable you're assigning as global or nonlocal:
def f():
x = 0 # assignment makes this x local to f()
def g():
nonlocal x
y = x + 1
x = y # x is declared to come from a nonlocal scope
return g
f()()
In your second example, it's not an issue as you're only looking at the value of x, not setting it, so it can come from any appropriate scope:
def f():
x = 0 # assignment makes this x local to f()
def g():
y = x + 1 # x comes from an outer scope
return g
f()()
Finally, your third example is the same as the first without the unssigned local variable usage error:
def f():
x = 0 # assignment makes this x local to f()
def g():
x = 1 # assignment makes this x local to g()
return g
f()()
It seems to me that if assigning the value 1 to the nonlocal variable
x in the g function works fine alone
How did you determine that the reassignment of x in f() worked fine?
def outside(x=1):
def printHam():
x = x+1
print x
return printHam
myfunc = outside(7)
myfunc()
This doesn't works gives error Local variable referenced before assignment error python
However this works
def outside(x=1):
def printHam():
print x + 1
return printHam
myfunc = outside(7)
myfunc()
Because you are assigning x in the first case, python will assume x is a local variable (which is the default). In the second case you aren't assigning it so it will check the global scope.
If you want this example to work, you have to pass the outer x into the inner function like so:
def outside(x=1):
def printHam(x=x):
x = x+1
print x
return printHam
That being said, this seems like a horribly contrived use case, so there's probably a better solution for your actual use case. But I can't tell you what it is without knowing more about what you're trying to do.
I am reading this article about decorator.
At Step 8 , there is a function defined as:
def outer():
x = 1
def inner():
print x # 1
return inner
and if we run it by:
>>> foo = outer()
>>> foo.func_closure # doctest: +ELLIPSIS
it doesn't print x. According to the explanation :
Everything works according to Python’s scoping rules - x is a local
variable in our function outer. When inner prints x at point #1 Python
looks for a local variable to inner and not finding it looks in the
enclosing scope which is the function outer, finding it there.
But what about things from the point of view of variable lifetime? Our
variable x is local to the function outer which means it only exists
while the function outer is running. We aren’t able to call inner till
after the return of outer so according to our model of how Python
works, x shouldn’t exist anymore by the time we call inner and perhaps
a runtime error of some kind should occur.
However, I don't really understand what the second paragraph means.
I understand inner() does get the value of x but why it doesn't print x out?
thanks
UPDATE:
Thanks all for the answers. Now I understand the reason.
the "return inner" is just a pointer to inner() but it doesn't get executed, that is why inner() doesn't print x as it is not called at all
I understand inner() does get the value of x but why it doesn't print
x out?
It doesn't print out anything because you've not called the inner function yet.
>>> def outer():
x = 1
def inner():
print x # 1
return inner
...
>>> func = outer()
>>> func
<function inner at 0xb61e280c>
>>> func()
1
This is called a closure, i.e even though the outer function is not in stack(finished executing) anymore but still the inner function that was returned from it remembers it's state.(i.e value of x)
>>> def outer():
x = 1
y = 2
def inner():
z=3
print x
return inner
...
>>> func = outer()
>>> func.func_code.co_freevars #returns the variables that were used in closure
('x',)
From the source code on how python decides it's a closure or not:
459 if len(code.co_freevars) == 0:
460 closure = NULL
461 else:
462 len(closure) == len(code.co_freevars)
In py3.x you can also modify the value of x using nonlocal statement inside inner function.
>>> def outer():
x = 1
def inner():
nonlocal x
x += 1
print (x)
return inner
...
>>> func = outer()
>>> func()
2
>>> func()
3
>>> func()
4
You are not calling inner. You have called outer, which returns inner, but without calling it. If you want to call inner, do foo() (since you assinged the result of outer() to the name foo).
The paragraph you quoted is sort of tangential to this issue. You say you already understand why inner gets the value of x, which is what that paragraph is explaining. Basically, if a local variable is used in a nested function, and that nested function is returned, the value of the variable is stored along with the returned function, even if the scope where that variable was defined in no longer active. Normally x would be gone after outer finished, because x is just local to outer. But outer returns inner, which still needs access to x. So x gets wrapped up into what's called a closure, so it can still be accessed by inner later on.
It's ok to get and print the outer function variable a
def outer():
a = 1
def inner():
print a
It's also ok to get the outer function array a and append something
def outer():
a = []
def inner():
a.append(1)
print a
However, it caused some trouble when I tried to increase the integer:
def outer():
a = 1
def inner():
a += 1 #or a = a + 1
print a
>> UnboundLocalError: local variable 'a' referenced before assignment
Why does this happen and how can I achieve my goal (increase the integer)?
In Python 3 you can do this with the nonlocal keyword. Do nonlocal a at the beginning of inner to mark a as nonlocal.
In Python 2 it is not possible.
Workaround for Python 2:
def outer():
a = [1]
def inner():
a[0] += 1
print a[0]
A generally cleaner way to do this would be:
def outer():
a = 1
def inner(b):
b += 1
return b
a = inner(a)
Python allows a lot, but non-local variables can be generally considered as "dirty" (without going into details here).
I'm trying to implement a closure in Python 2.6 and I need to access a nonlocal variable but it seems like this keyword is not available in python 2.x. How should one access nonlocal variables in closures in these versions of python?
Inner functions can read nonlocal variables in 2.x, just not rebind them. This is annoying, but you can work around it. Just create a dictionary, and store your data as elements therein. Inner functions are not prohibited from mutating the objects that nonlocal variables refer to.
To use the example from Wikipedia:
def outer():
d = {'y' : 0}
def inner():
d['y'] += 1
return d['y']
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
The following solution is inspired by the answer by Elias Zamaria, but contrary to that answer does handle multiple calls of the outer function correctly. The "variable" inner.y is local to the current call of outer. Only it isn't a variable, since that is forbidden, but an object attribute (the object being the function inner itself). This is very ugly (note that the attribute can only be created after the inner function is defined) but seems effective.
def outer():
def inner():
inner.y += 1
return inner.y
inner.y = 0
return inner
f = outer()
g = outer()
print(f(), f(), g(), f(), g()) #prints (1, 2, 1, 3, 2)
Rather than a dictionary, there's less clutter to a nonlocal class. Modifying #ChrisB's example:
def outer():
class context:
y = 0
def inner():
context.y += 1
return context.y
return inner
Then
f = outer()
assert f() == 1
assert f() == 2
assert f() == 3
assert f() == 4
Each outer() call creates a new and distinct class called context (not merely a new instance). So it avoids #Nathaniel's beware about shared context.
g = outer()
assert g() == 1
assert g() == 2
assert f() == 5
I think the key here is what you mean by "access". There should be no issue with reading a variable outside of the closure scope, e.g.,
x = 3
def outer():
def inner():
print x
inner()
outer()
should work as expected (printing 3). However, overriding the value of x does not work, e.g.,
x = 3
def outer():
def inner():
x = 5
inner()
outer()
print x
will still print 3. From my understanding of PEP-3104 this is what the nonlocal keyword is meant to cover. As mentioned in the PEP, you can use a class to accomplish the same thing (kind of messy):
class Namespace(object): pass
ns = Namespace()
ns.x = 3
def outer():
def inner():
ns.x = 5
inner()
outer()
print ns.x
There is another way to implement nonlocal variables in Python 2, in case any of the answers here are undesirable for whatever reason:
def outer():
outer.y = 0
def inner():
outer.y += 1
return outer.y
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
It is redundant to use the name of the function in the assignment statement of the variable, but it looks simpler and cleaner to me than putting the variable in a dictionary. The value is remembered from one call to another, just like in Chris B.'s answer.
Here's something inspired by a suggestion Alois Mahdal made in a comment regarding another answer:
class Nonlocal(object):
""" Helper to implement nonlocal names in Python 2.x """
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def outer():
nl = Nonlocal(y=0)
def inner():
nl.y += 1
return nl.y
return inner
f = outer()
print(f(), f(), f()) # -> (1 2 3)
There is a wart in python's scoping rules - assignment makes a variable local to its immediately enclosing function scope. For a global variable, you would solve this with the global keyword.
The solution is to introduce an object which is shared between the two scopes, which contains mutable variables, but is itself referenced through a variable which is not assigned.
def outer(v):
def inner(container = [v]):
container[0] += 1
return container[0]
return inner
An alternative is some scopes hackery:
def outer(v):
def inner(varname = 'v', scope = locals()):
scope[varname] += 1
return scope[varname]
return inner
You might be able to figure out some trickery to get the name of the parameter to outer, and then pass it as varname, but without relying on the name outer you would like need to use a Y combinator.
Another way to do it (although it's too verbose):
import ctypes
def outer():
y = 0
def inner():
ctypes.pythonapi.PyCell_Set(id(inner.func_closure[0]), id(y + 1))
return y
return inner
x = outer()
x()
>> 1
x()
>> 2
y = outer()
y()
>> 1
x()
>> 3
Extending Martineau elegant solution above to a practical and somewhat less elegant use case I get:
class nonlocals(object):
""" Helper to implement nonlocal names in Python 2.x.
Usage example:
def outer():
nl = nonlocals( n=0, m=1 )
def inner():
nl.n += 1
inner() # will increment nl.n
or...
sums = nonlocals( { k:v for k,v in locals().iteritems() if k.startswith('tot_') } )
"""
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def __init__(self, a_dict):
self.__dict__.update(a_dict)
Use a global variable
def outer():
global y # import1
y = 0
def inner():
global y # import2 - requires import1
y += 1
return y
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
Personally, I do not like the global variables. But, my proposal is based on https://stackoverflow.com/a/19877437/1083704 answer
def report():
class Rank:
def __init__(self):
report.ranks += 1
rank = Rank()
report.ranks = 0
report()
where user needs to declare a global variable ranks, every time you need to call the report. My improvement eliminates the need to initialize the function variables from the user.