TL;DR: I want a locals() that looks in a containing scope.
Hi, all.
I'm teaching a course on Python programming to some chemist friends, and I want to be sure I really understand scope.
Consider:
def a():
x = 1
def b():
print(locals())
print(globals())
b()
Locals prints an empty environment, and globals prints the usual globals. How do I get access to the environment where x is stored? Clearly the interpreter knows about it because I can refer to it.
Related: When does scoping happen? The following nameErrors on a = x+2 only if x=3 is included:
def a():
x = 1
def b():
a = x+2
x = 3
b()
If you comment out x=3, the code works. Does this mean that python makes a lexical-scope pass over the code before it interprets it?
What is happening in your code is that when python see's the x=3 line in your b() method, it is recreating x with a scope within the b function instead of using the x with it's scope in the a function.
because your code then goes:
a = x+2
x = 3
it is saying that you need to define the inner scoped x before referencing it.
however, when you do not have the assigning of x within the b function python does not attempt to make a lower scoped x and will not throw any errors.
The following code will illustrate this:
def a():
x = 1
def b():
x = 3
print (x)
def c():
print (x)
b()
c()
print (x)
a()
However if you were to declare x as global you could use it within a function, like so:
def a():
global x
x = 1
def d():
global x
x += 2
print (x)
d()
print (x)
a()
Python 3 has also added a nonlocal keyword that will let you access a variable from an enclosing scope, usage is like:
def a():
x = 1
def d():
nonlocal x
x += 2
print (x)
d()
print (x)
a()
Will print the same results as the global example.
In case I miss-read the question:
As per the answer to Get locals from calling namespace in Python:
you can use:
import inspect
def a():
x = 1
def d():
frame = inspect.currentframe()
try:
print (frame.f_back.f_locals)
finally:
del frame
d()
a()
to get the local scope of the functions caller.
The statement print(locals()) refers to nearest enclosing scope, that is the def b(): function. When calling b(), you will print the locals to this b function, and definition of x is outside the scope.
def a():
x = 1
def b():
print(locals())
print(globals())
b()
print(locals())
would print x as a local variable.
For your second question:
def a():
x = 1
def b():
#a = x+2
x = 3
b()
>>> a()
gives no error.
def a():
x = 1
def b():
a = x+2
#x = 3
b()
>>> a()
gives no error.
And
def a():
x = 1
def b():
a = x+2
x = 3
b()
>>> a()
gives following error: UnboundLocalError: local variable 'x' referenced before assignment
IMO (please check), in this third case, first statement in the body of b() looks for value of x in the most enclosing scope. And in this scope, x is assigned on the next line. If you only have statement a = x+2, x is not found in most enclosing scope, and is found in the 'next' enclosing scope, with x = 1.
python3 -c "help(locals)" says this about the locals function:
locals()
Return a dictionary containing the current scope's local variables.
That means that calling locals() in b() will only show you which variables are local to b(). That doesn't mean that b() can't see variables outside its scope -- but rather, locals() only returns information for variables local to b().
What you probably want is to make b() show information on the variables of the calling function. To do that, take this code (which show's b()'s local variables):
def a():
x = 1
def b():
print(locals())
b()
and replace it with this one (which uses the inspect module to get you the local information of the calling frame):
def a():
x = 1
def b():
import inspect
print(inspect.currentframe().f_back.f_locals)
b()
As for your other issue:
def a():
x = 1
def b():
a = x+2
x = 3
b()
The error you get is because Python 3 allows you to set/assign global variables and variables local to outside scopes provided you declare them with the global and nonlocal keywords. (I won't explain them here; I'll just say that you won't have any problem looking them up yourself.)
If you remove the x = 3 line (keeping the a = x+2 line), your code will run because x's value is being used, but not set. If instead you remove the a = x+2 line (keeping the x = 3 line), your code will run because it will create a new x inside b()'s scope.
If you keep both lines, Python doesn't know whether x is supposed to refer to an x outside the current scope (as a = x+2 seems to suggest) or if it's supposed to refer to an x local to b()'s scope (as x = 3 suggests).
If you want x to be local to b(), then you shouldn't have that a = x+2 line there, at least not before x is set with x = 3. But if you want x to be a()'s x, then you should declare x as nonlocal to b(), like this:
def a():
x = 1
def b():
nonlocal x # use a()'s x
a = x+2
x = 3
b()
If you're confused, remember that Python will let you use global variables and variables created outside of your current scope, but only if you don't assign to them. (It's often acceptable to read global variables, but it's frowned upon to set them.)
If you want to set a global variable or a non-local variable, you will have to declare those variables with global (for global variables) or nonlocal (for non-local variables), as shown in the above code.
I hope this helps.
Related
Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()
I am using Python 3.7.0 and I am doing some experiments in order to comprehend the subtleties of variable scopes but I still don't understand this behaviour.
When I execute this code:
def f():
x=0
def g():y=x+1;x=y
return g
f()()
I get UnboundLocalError: local variable 'x' referenced before assignment
However when I execute this one:
def f():
x=0
def g():y=x+1
return g
f()()
It works fine. And even this one:
def f():
x=0
def g():x=1
return g
f()()
It also works fine. So I am confused. It seems to me that if assigning the value 1 to the nonlocal variable x in the g function works fine alone and on the other hand if assigning an expression containing x to a local variable y in the function g works also fine then the instructions y=x+1 and x=y should both work. I don't understand what is causing the error. I feel I am missing something fundamental in my understanding of Python's behaviour.
I'll take an ill-advised crack at this. In the following code:
def f():
x = 0 # assignment makes this x local to f()
def g():
y = x + 1
x = y # assignment makes this x local to g()
return g
f()()
The assignment of x inside g() forces it to be a variable local to g(). So the first line causes the error as you accessing the value of an unassigned local. I believe this "assignment makes it local" logic doesn't follow the flow of your program -- the function is viewed as whole to make this determination and then the code is looked at line by line.
You can access any variable in scope from a function, but you can only set locals. Unless you've declared the variable you're assigning as global or nonlocal:
def f():
x = 0 # assignment makes this x local to f()
def g():
nonlocal x
y = x + 1
x = y # x is declared to come from a nonlocal scope
return g
f()()
In your second example, it's not an issue as you're only looking at the value of x, not setting it, so it can come from any appropriate scope:
def f():
x = 0 # assignment makes this x local to f()
def g():
y = x + 1 # x comes from an outer scope
return g
f()()
Finally, your third example is the same as the first without the unssigned local variable usage error:
def f():
x = 0 # assignment makes this x local to f()
def g():
x = 1 # assignment makes this x local to g()
return g
f()()
It seems to me that if assigning the value 1 to the nonlocal variable
x in the g function works fine alone
How did you determine that the reassignment of x in f() worked fine?
This question already has answers here:
In Python, variables inside if conditions hide the global scope even if they are not executed?
(4 answers)
Closed 4 years ago.
Today I'm reading python change log and meet the nonlocal keyword and did some experiment with it. I find a confusing situation where untriggered assignment will change the nonlocal keyword behavior, please see the example below.
def a():
x = 'a'
def b():
def c():
nonlocal x
x = 'c'
c()
b()
print(x)
a()
>>> python3 test.py
c
def a():
x = 'a'
def b():
def c():
nonlocal x
x = 'c'
c()
if False:
x = 'b'
b()
print(x)
a()
>>> python3 test2.py
a
You can saw that in test2.py, there is an untriggered assginment x = 'b' which changed the behavior of nonlocal.
Why this happened?
Python decides which variables are local to a function at compile time. x is assigned to within the function b, so it's local. That that branch is never actually reached at runtime is irrelevant and can't be decided in general.
So the x in c that is nonlocal is the next x in an outer scope, namely the one in b.
The alternative would be much more surprising -- consider what would happen if the if False: was instead if rand(10) == 6:. Then during the first call of b the nonlocal variable would refer to the outermost one, but randomly at some later call of b it would start referring to the other one!
Sorry if it is confusing; making an example.
def functionA():
x=10
b()
def functionB():
Y=22
return Y
Is there a way to access x from B, without pass it as parameter when I call B?
I would like to avoid to have to do something like
def functionB(var):
From my understanding, I can't call super because B is not part of A, it is just called from it. I would like to call B from A, and access in B a variable in A (or more than one; I am exploring the concept); but without pass it as parameter to B.
Is this even possible in Python?
Yes, you actually can:
import inspect
def A():
X = 42
B()
def B():
print(inspect.stack()[1][0].f_locals['X'])
A()
But you shouldn't.
Here come the downvotes.
This is how you can make sure you only execute it when the caller is "functionA" and you retrieve it's last local value before the call.
This is very sketchy though. Don't do this.
import sys
def functionA():
x=10
x = 5
functionB()
x = 3
def functionB():
frame = sys._getframe()
outer = frame.f_back
if outer.f_code.co_name == "functionA":
print outer.f_locals["x"]
Y=22
return Y
if both are just functions, no. but if you made both functions methods of a containing class, you could make a's variable accessible to fuction b's.
class Wrapper(object):
def __init__():
self.a_val = None
def functionA(self):
self.a_val = 123
self.functionB()
def functionB(self):
# you can access self.a_val here fine.
once you have this, you would call functionA as
Wrapper().functionA()
Well yes, what you want is a global variable:
def functionA():
global x
x=10
functionB()
def functionB():
Y=22
print 'X from B:', x
return Y
functionA()
Of course as any responsible programmer would tell you, be careful with global variables as they can easily turn your code into spaghetti.
The standard way to do that is, naturally, to use a class...
However, for argument sake, there is another way to do it, that will make that variable shared between those two functions and still not be global, using a closure:
def defineFuns():
global functionA, functionB
x = 10
def functionA():
print 'X from A:', x
functionB()
def functionB():
Y=22
print 'X from B:', x
return Y
defineFuns()
functionA()
Or playing with the function's dictionary...
def functionA():
x = 10
print 'X from A:', x
functionB.func_globals['x'] = x
functionB()
def functionB():
Y=22
print 'X from B:', x
return Y
functionA()
This is a perfect case for nested functions.
def a():
x = 10
def b():
y = 22 + x
return y
b()
Function b returns 32 (22+10). The x in function b is the same as the x in a. You can't call b from outside of a, but that makes perfect sense because the value of x would be undefined in that case.
Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()