This question already has answers here:
Is it possible to modify a variable in python that is in an outer (enclosing), but not global, scope?
(9 answers)
Closed 7 months ago.
The following code raises an UnboundLocalError:
def foo():
i = 0
def incr():
i += 1
incr()
print(i)
foo()
Is there a way to accomplish this?
Use nonlocal statement
def foo():
i = 0
def incr():
nonlocal i
i += 1
incr()
print(i)
foo()
For more information on this new statement added in python 3.x, go to https://docs.python.org/3/reference/simple_stmts.html#the-nonlocal-statement
You can use i as an argument like this:
def foo():
i = 0
def incr(i):
return i + 1
i = incr(i)
print(i)
foo()
See 9.2. Python Scopes and Namespaces:
if no global statement is in effect – assignments to names always go into the innermost scope.
Also:
The global statement can be used to indicate that particular variables live in the global scope and should be rebound there; the nonlocalstatement indicates that particular variables live in an enclosing scope and should be rebound there.
You have many solutions:
Pass i as an argument ✓ (I would go with this one)
Use nonlocal keyword
Note that in Python2.x you can access non-local variables but you can't change them.
People have answered your question but no one seems to address why exactly this is happening.
The following code raises an UnboundLocalError
So, why? Let's take a quote from the FAQ:
When you make an assignment to a variable in a scope, that variable becomes local to that scope and shadows any similarly named variable in the outer scope.
Inside your nested function you are performing an assignment with the += operator. What this means is that i += 1 will approximately perform i = i + 1 (from a binding perspective). As a result i in the expression i + 1 will be searched for in the local scope (because it is used in the assignment statement) for function incr where it will not be found, resulting in an UnboundLocalError: reference before assignment .
There's many ways you can tackle this and python 3 is more elegant in the approach you can take than python 2.
Python 3 nonlocal:
The nonlocal statements tells Python to look for a name in the enclosing scope (so in this case, in the scope of function foo()) for name references:
def foo():
i = 0
def incr():
nonlocal i
i +=1
incr()
print(i)
Function attributes Python 2.x and 3.x:
Remember that functions are first class objects, as such they can store state. Use function attributes to access and mutate function state, the good thing with this approach is it works on all pythons and doesn't require global, nonlocal statements.
def foo():
foo.i = 0
def incr():
foo.i +=1
incr()
print(foo.i)
The global Statement (Python 2.x, 3.x):
Really the ugliest of the bunch, but gets the job done:
i = 0
def foo():
def incr():
global i
i += 1
incr()
print(i)
foo()
The option of passing an argument to the function gets the same result but it doesn't relate to mutating the enclosing scope in the sense nonlocal and global are and is more like the function attributes presented. It is creating a new local variable in function inc and then re-binding the name to i with the i = incr(i) but, it does indeed get the job done.
In Python, int are immutable. So you could put your int in a mutable object.
def foo():
i = 0
obj = [i]
def incr(obj):
obj[0]+=1
incr(obj)
print(obj[0])
foo()
You can make i global and use it.
i = 0
def foo():
def incr():
global i
i += 1
incr()
print(i)
foo()
but the most preferred way is to pass i as a param to incr
def foo():
i = 0
def incr(arg):
arg += 1
return arg
i = incr(i)
print(i)
foo()
you could also use a lambda function:
def foo():
i=0
incr = lambda x: x+1
print incr(i)
foo()
I think the code is cleaner this way
Simple function attributes will not work in this case.
>>> def foo():
... foo.i = 0
... def incr():
... foo.i +=1
... incr()
... print(foo.i)
...
>>>
>>>
>>> foo()
1
>>> foo()
1
>>> foo()
1
You re assign your foo.i to 0 for every call of foo.
It better to use hassattr. But code become more complicated.
>>> def foo():
... if not hasattr(foo, 'i'):
... foo.i = 0
... def incr():
... foo.i += 1
... return foo.i
... i = incr()
... print(i)
...
>>>
>>>
>>> foo()
1
>>> foo()
2
>>> foo()
3
Also you can try this idea:
>>> def foo():
... def incr():
... incr.i += 1
... return incr.i
... incr.i = 0
... return incr
...
>>> a = foo()
>>> a()
1
>>> a()
2
>>> a()
3
>>> a()
4
May be it is more handy to wrap you incr into decorator.
Related
I am trying to better understanding scoping in python. I have the following toy example:
a = 1
print "in global: " + str(a)
def g():
a += 1
print "in g(): " + str(a)
def f():
a += 1
print "in f(): " + str(a)
g()
f()
I expected this to run and print out 1 then 2 then 2 again. However, instead I get en error:
UnboundLocalError: local variable 'a' referenced before assignment
I would have thought both g() and f() would pull a from the global scope. Incorrect?
UPDATED:
Thanks for the answers but what isn't clear is this: if I would like to just read the global variable a and assign it to a local variable that I create, also named a is that possible?
The reason I am doing this is I'm trying to figure out if g() inherits the scope of f() when it is called or the global scope where it is defined?
You are trying to change a in outer scope of functions, a is not in the scope of current functions, that's why you are getting such an error, because your functions know nothing about a. If you want to change a from within functions you need to use `global':
a = 1
print "in global: " + str(a)
def g():
global a
a += 1
print "in g(): " + str(a)
def f():
global a
a += 1
print "in f(): " + str(a)
g()
to print out the global a use it this way:
def f():
print a
new_a = a
print new_a
There is a good example about global here
And if you want to use local variable a with value from global a use globals:
def g():
a = globals()['a']
a += 1
print "in g(): " + str(a)
You need to use the global keyword.
>>> a = 1
>>> def g():
... global a
... a += 1
... print a
...
>>> g()
2
You are missing the global operative in the function g and f
like this:
a = 1
def g():
global a
a += 1
print "in g(): " + str(a)
def f():
global a
a += 1
print "in f(): " + str(a)
You can find more on global and local variables in the documentation here and from stack overflow here
When you ask for a variable value, Python follows LEGB rule: local, enclosing, global, built-in.
But when you assign a variable, it is created from scratch in the local scope (assuming the variable assignment is inside function), discarding whatever value was previously bound to it:
a = 5
def f(x):
print a # Prints `5`, because name `a` can be found following LEGB rule in Global scope.
print x # Prints whatever value `x` was having, found following LEGB rule in Local scope.
x = 123 # Forgets previous value of `x`, re-assign it to `123` in the Local scope.
Your problem with unbound variable appears here: a += 1 is simply a syntactic sugar for a = a + 1. When you do a = a + 1, Python catches that a is local , so it produces LOAD_FAST command in bytecode:
import dis
a = 1
def f():
a += 1
print dis.dis(f)
0 LOAD_FAST 0 (a)
3 LOAD_CONST 1 (1)
6 INPLACE_ADD
7 STORE_FAST 0 (a)
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
LOAD_FAST is used for local variable fetching. But you do not have a in local scope yet when LOAD_FAST is called, so the command fails.
Actually, if you do not assign to a, but want to modify it from local scope, you can do it via method calls. Unfortunately, int don't have modifying methods, so I'll give an example with class:
class Number(object):
def __init__(self, x):
self.x = x
def add_one(self):
self.x += 1
a = Number(1)
def f():
a.add_one()
print a.x
print a.x
f()
print a.x
1
2
2
Here you essentially do a=a+1, but in a more complicated manner. This is because calls to a are resolved as Global (and work fine) and you don't do any assignments inside f(). Method add_one is called on existing object - it does not care whether it was global or local in the caller.
TL;DR: I want a locals() that looks in a containing scope.
Hi, all.
I'm teaching a course on Python programming to some chemist friends, and I want to be sure I really understand scope.
Consider:
def a():
x = 1
def b():
print(locals())
print(globals())
b()
Locals prints an empty environment, and globals prints the usual globals. How do I get access to the environment where x is stored? Clearly the interpreter knows about it because I can refer to it.
Related: When does scoping happen? The following nameErrors on a = x+2 only if x=3 is included:
def a():
x = 1
def b():
a = x+2
x = 3
b()
If you comment out x=3, the code works. Does this mean that python makes a lexical-scope pass over the code before it interprets it?
What is happening in your code is that when python see's the x=3 line in your b() method, it is recreating x with a scope within the b function instead of using the x with it's scope in the a function.
because your code then goes:
a = x+2
x = 3
it is saying that you need to define the inner scoped x before referencing it.
however, when you do not have the assigning of x within the b function python does not attempt to make a lower scoped x and will not throw any errors.
The following code will illustrate this:
def a():
x = 1
def b():
x = 3
print (x)
def c():
print (x)
b()
c()
print (x)
a()
However if you were to declare x as global you could use it within a function, like so:
def a():
global x
x = 1
def d():
global x
x += 2
print (x)
d()
print (x)
a()
Python 3 has also added a nonlocal keyword that will let you access a variable from an enclosing scope, usage is like:
def a():
x = 1
def d():
nonlocal x
x += 2
print (x)
d()
print (x)
a()
Will print the same results as the global example.
In case I miss-read the question:
As per the answer to Get locals from calling namespace in Python:
you can use:
import inspect
def a():
x = 1
def d():
frame = inspect.currentframe()
try:
print (frame.f_back.f_locals)
finally:
del frame
d()
a()
to get the local scope of the functions caller.
The statement print(locals()) refers to nearest enclosing scope, that is the def b(): function. When calling b(), you will print the locals to this b function, and definition of x is outside the scope.
def a():
x = 1
def b():
print(locals())
print(globals())
b()
print(locals())
would print x as a local variable.
For your second question:
def a():
x = 1
def b():
#a = x+2
x = 3
b()
>>> a()
gives no error.
def a():
x = 1
def b():
a = x+2
#x = 3
b()
>>> a()
gives no error.
And
def a():
x = 1
def b():
a = x+2
x = 3
b()
>>> a()
gives following error: UnboundLocalError: local variable 'x' referenced before assignment
IMO (please check), in this third case, first statement in the body of b() looks for value of x in the most enclosing scope. And in this scope, x is assigned on the next line. If you only have statement a = x+2, x is not found in most enclosing scope, and is found in the 'next' enclosing scope, with x = 1.
python3 -c "help(locals)" says this about the locals function:
locals()
Return a dictionary containing the current scope's local variables.
That means that calling locals() in b() will only show you which variables are local to b(). That doesn't mean that b() can't see variables outside its scope -- but rather, locals() only returns information for variables local to b().
What you probably want is to make b() show information on the variables of the calling function. To do that, take this code (which show's b()'s local variables):
def a():
x = 1
def b():
print(locals())
b()
and replace it with this one (which uses the inspect module to get you the local information of the calling frame):
def a():
x = 1
def b():
import inspect
print(inspect.currentframe().f_back.f_locals)
b()
As for your other issue:
def a():
x = 1
def b():
a = x+2
x = 3
b()
The error you get is because Python 3 allows you to set/assign global variables and variables local to outside scopes provided you declare them with the global and nonlocal keywords. (I won't explain them here; I'll just say that you won't have any problem looking them up yourself.)
If you remove the x = 3 line (keeping the a = x+2 line), your code will run because x's value is being used, but not set. If instead you remove the a = x+2 line (keeping the x = 3 line), your code will run because it will create a new x inside b()'s scope.
If you keep both lines, Python doesn't know whether x is supposed to refer to an x outside the current scope (as a = x+2 seems to suggest) or if it's supposed to refer to an x local to b()'s scope (as x = 3 suggests).
If you want x to be local to b(), then you shouldn't have that a = x+2 line there, at least not before x is set with x = 3. But if you want x to be a()'s x, then you should declare x as nonlocal to b(), like this:
def a():
x = 1
def b():
nonlocal x # use a()'s x
a = x+2
x = 3
b()
If you're confused, remember that Python will let you use global variables and variables created outside of your current scope, but only if you don't assign to them. (It's often acceptable to read global variables, but it's frowned upon to set them.)
If you want to set a global variable or a non-local variable, you will have to declare those variables with global (for global variables) or nonlocal (for non-local variables), as shown in the above code.
I hope this helps.
It's ok to get and print the outer function variable a
def outer():
a = 1
def inner():
print a
It's also ok to get the outer function array a and append something
def outer():
a = []
def inner():
a.append(1)
print a
However, it caused some trouble when I tried to increase the integer:
def outer():
a = 1
def inner():
a += 1 #or a = a + 1
print a
>> UnboundLocalError: local variable 'a' referenced before assignment
Why does this happen and how can I achieve my goal (increase the integer)?
In Python 3 you can do this with the nonlocal keyword. Do nonlocal a at the beginning of inner to mark a as nonlocal.
In Python 2 it is not possible.
Workaround for Python 2:
def outer():
a = [1]
def inner():
a[0] += 1
print a[0]
A generally cleaner way to do this would be:
def outer():
a = 1
def inner(b):
b += 1
return b
a = inner(a)
Python allows a lot, but non-local variables can be generally considered as "dirty" (without going into details here).
Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()
I'm trying to implement a closure in Python 2.6 and I need to access a nonlocal variable but it seems like this keyword is not available in python 2.x. How should one access nonlocal variables in closures in these versions of python?
Inner functions can read nonlocal variables in 2.x, just not rebind them. This is annoying, but you can work around it. Just create a dictionary, and store your data as elements therein. Inner functions are not prohibited from mutating the objects that nonlocal variables refer to.
To use the example from Wikipedia:
def outer():
d = {'y' : 0}
def inner():
d['y'] += 1
return d['y']
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
The following solution is inspired by the answer by Elias Zamaria, but contrary to that answer does handle multiple calls of the outer function correctly. The "variable" inner.y is local to the current call of outer. Only it isn't a variable, since that is forbidden, but an object attribute (the object being the function inner itself). This is very ugly (note that the attribute can only be created after the inner function is defined) but seems effective.
def outer():
def inner():
inner.y += 1
return inner.y
inner.y = 0
return inner
f = outer()
g = outer()
print(f(), f(), g(), f(), g()) #prints (1, 2, 1, 3, 2)
Rather than a dictionary, there's less clutter to a nonlocal class. Modifying #ChrisB's example:
def outer():
class context:
y = 0
def inner():
context.y += 1
return context.y
return inner
Then
f = outer()
assert f() == 1
assert f() == 2
assert f() == 3
assert f() == 4
Each outer() call creates a new and distinct class called context (not merely a new instance). So it avoids #Nathaniel's beware about shared context.
g = outer()
assert g() == 1
assert g() == 2
assert f() == 5
I think the key here is what you mean by "access". There should be no issue with reading a variable outside of the closure scope, e.g.,
x = 3
def outer():
def inner():
print x
inner()
outer()
should work as expected (printing 3). However, overriding the value of x does not work, e.g.,
x = 3
def outer():
def inner():
x = 5
inner()
outer()
print x
will still print 3. From my understanding of PEP-3104 this is what the nonlocal keyword is meant to cover. As mentioned in the PEP, you can use a class to accomplish the same thing (kind of messy):
class Namespace(object): pass
ns = Namespace()
ns.x = 3
def outer():
def inner():
ns.x = 5
inner()
outer()
print ns.x
There is another way to implement nonlocal variables in Python 2, in case any of the answers here are undesirable for whatever reason:
def outer():
outer.y = 0
def inner():
outer.y += 1
return outer.y
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
It is redundant to use the name of the function in the assignment statement of the variable, but it looks simpler and cleaner to me than putting the variable in a dictionary. The value is remembered from one call to another, just like in Chris B.'s answer.
Here's something inspired by a suggestion Alois Mahdal made in a comment regarding another answer:
class Nonlocal(object):
""" Helper to implement nonlocal names in Python 2.x """
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def outer():
nl = Nonlocal(y=0)
def inner():
nl.y += 1
return nl.y
return inner
f = outer()
print(f(), f(), f()) # -> (1 2 3)
There is a wart in python's scoping rules - assignment makes a variable local to its immediately enclosing function scope. For a global variable, you would solve this with the global keyword.
The solution is to introduce an object which is shared between the two scopes, which contains mutable variables, but is itself referenced through a variable which is not assigned.
def outer(v):
def inner(container = [v]):
container[0] += 1
return container[0]
return inner
An alternative is some scopes hackery:
def outer(v):
def inner(varname = 'v', scope = locals()):
scope[varname] += 1
return scope[varname]
return inner
You might be able to figure out some trickery to get the name of the parameter to outer, and then pass it as varname, but without relying on the name outer you would like need to use a Y combinator.
Another way to do it (although it's too verbose):
import ctypes
def outer():
y = 0
def inner():
ctypes.pythonapi.PyCell_Set(id(inner.func_closure[0]), id(y + 1))
return y
return inner
x = outer()
x()
>> 1
x()
>> 2
y = outer()
y()
>> 1
x()
>> 3
Extending Martineau elegant solution above to a practical and somewhat less elegant use case I get:
class nonlocals(object):
""" Helper to implement nonlocal names in Python 2.x.
Usage example:
def outer():
nl = nonlocals( n=0, m=1 )
def inner():
nl.n += 1
inner() # will increment nl.n
or...
sums = nonlocals( { k:v for k,v in locals().iteritems() if k.startswith('tot_') } )
"""
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def __init__(self, a_dict):
self.__dict__.update(a_dict)
Use a global variable
def outer():
global y # import1
y = 0
def inner():
global y # import2 - requires import1
y += 1
return y
return inner
f = outer()
print(f(), f(), f()) #prints 1 2 3
Personally, I do not like the global variables. But, my proposal is based on https://stackoverflow.com/a/19877437/1083704 answer
def report():
class Rank:
def __init__(self):
report.ranks += 1
rank = Rank()
report.ranks = 0
report()
where user needs to declare a global variable ranks, every time you need to call the report. My improvement eliminates the need to initialize the function variables from the user.