If I have the following code:
var_global = "global"
def outer() :
var_outer = "outer"
def inner() :
var_inner = "inner"
# How to get here all variables available (it should contain
# var_inner, var_outer, var_global)?
print("inner", locals(), globals())
return inner
And I run:
outer()()
I would like to get all variables at print location. This should include var_outer as well. But neither locals nor globals provide it.
The closure variable mechanisms that would provide inner with access to var_outer are only triggered if inner actually contains accesses to the var_outer variable. Otherwise, var_outer is not saved, and no introspection will give you access to it.
var_outer truly ISN'T available in inner(). Making a variable available in a nested function requires that the variable be stored in a different way from the start; Python did not do so, due to lack of any reference to the variable from a nested function. If you did insert such a reference (such as the bare statement var_outer), it would show up in the output of locals().
var_outer can access var_inner by nonlocal.
var_global = "global"
def outer():
var_outer = "outer"
def inner():
var_inner = "inner"
print("inner", locals(), globals())
nonlocal var_outer
print(var_outer)
return inner
outer()()
Will print:
inner {'var_outer': 'outer', 'var_inner': 'inner'} [...]
Related
I have read Cannot change global variables in a function through an exec() statement?, but this is not my case.
When i run test(), it raises exception UnboundLocalError: local variable 'var' referenced before assignment, it looks like exec('global var', globals()) doesn't work.
var = 1
def test():
exec('global var', globals()) # The effect it achieves is different from `global var`, why?
var += 1
test()
print(var)
But if i change the function to the following, it worked:
def test():
exec('global var', globals())
exec('var += 1', globals())
I don't know why, can anyone explain?
The following quote from the language reference (section "Simple statements") seems relevant:
Programmer’s note: global is a directive to the parser. It applies only to code parsed at the same time as the global statement. In particular, a global statement contained in a string or code object supplied to the built-in exec() function does not affect the code block containing the function call, and code contained in such a string is unaffected by global statements in the code containing the function call. The same applies to the eval() and compile() functions.
I.e, in your first version, var is still local to the function.
'global var' only has meaning within the context of the dynamically created program (DCP).
Consider these two options for incrementing var. They may make it more obvious what's going on.
exec('var += 1', globals())
...or...
exec('global var\nvar += 1')
In the first case the Python runtime finds var because you passed the dictionary of globals to exec(). In the second case, you qualified var with 'global' so the runtime knows where to look. Note therefore that in the second case there's no need to pass globals()
I tested this out with a program I wrote myself:
>>> def f():
f=['f',1,2]
def g():
g=1
print('this prints out f from f(): ',f)
print("id",id(f))
def x():
x=1
print('this also prints out f from f():',f)
print('id',id(f))
x()
g()
>>> f()#output
this prints out f from f(): ['f', 1, 2]
id 140601546763464
this also prints out f from f(): ['f', 1, 2]
id 140601546763464
From what I learned, the innermost x() function can only access its own local namespace, the enclosing namespace, the global, and finally the built-in namespace. I initially thought that trying to access the list f declared in function f() from function x() would raise an error, as the f() function's namespace cannot be classified as any of the aforementioned elements. After running the program, I realized you indeed can access the list f from the function x(). I don't quite understand how this works though. My guess is that checking the enclosing namespace not only checks the local namespace of the enclosing function but the enclosing function for it as well, in a process that works almost recursively. Can somebody please explain how this works?
Python resolves names using LEGB rule:(LEGB means Local, Enclosing, Global, and Built-in)
Local scope:
contains the names that are defined inside the function.
visible only inside the function
created at function call(If we call the function multiple times each call creates new local scope)
will be destroyed once function return
Enclosing or nonlocal:
exists for nested functions
contains names defined in the enclosing function
visible in inner and enclosing functions.
Global:
contains all the names defined at the top level of a program
visible from everywhere inside the code.
exist throut the life of code.
Built-in:
created whenever we run a script
contains keywords, functions, exceptions, etc that are built into Python
visible everywhere in the code
The LEGB rule is a rule which determines the order in which Python looks up names.
i.e Python will look the name sequentially in the local, enclosing, global, and built-in scope. And inner scope codes can outer scope names but outer scope codes cannot access inner scope names.
When we use nested functions the scope resolving is as follows:
check the local scope(inside the function)
If not found check enclosing scopes of outer functions from the innermost scope to the outermost scope
If not found look the global scope
If not found look built-ins
Still not found raise error
A little easy problem:
exec("a=3")
print(a)
# This will print 3
If I use this:
def func():
exec("a=3")
print(a)
func()
# NameError: name 'a' is not defined.
What happened?How could I use exec() to assign it a value in a function?
Edit:I found a question with the same trouble but still didn't solved.
why do you want to do that?
I know using exec() is bad and unsafe.But recently I try to solve a OP's problem.I met it.
Python knows several kinds of scope: module global, function local, nonlocal closures, class body. Notably, scope resolution is defined statically at byte code compile time – most importantly, whether names refer to local/nonlocal or global scope cannot be changed.
Of these scopes, only global scope is guaranteed to behave similar to a dict, and as such writeable. The local/nonlocal scope is generally not writeable, and new variables cannot be added to it.
exec will write to the global scope if locals is not passed in; globals must then explicitly be set to its default of globals().
def func():
exec("a='exec'", globals()) # access only global scope
print(a)
a = 'global'
func() # prints exec
However, once a name is local to a function, exec cannot modify it.
def func():
a = 'local' # assignment makes name local
exec("a='exec global'", globals())
exec("a='exec locals'", globals(), locals())
print(a)
a = 'global'
func() # prints local
While a dict-like representation of local/nonlocal scope exists, the interpreter is not required to honour changes to it.
locals()
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks. Note that at the module level, locals() and globals() are the same dictionary.
Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
Even though exec does take locals as a dict, these are not treated like function locals/nonlocals. Attempts to modify the default locals (the result of locals()) are not defined.
exec()
... If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object. Remember that at module level, globals and locals are the same dictionary. If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition.
Note: The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. ...
execute help(exec) in your python3 REPL, you will get the following docuement:
Help on built-in function exec in module builtins:
exec(source, globals=None, locals=None, /)
Execute the given source in the context of globals and locals.
The source may be a string representing one or more Python statements
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.
so there are at least 2 options to provide the value of argument 'a':
assign a value to the variable 'a' in the module's global scope:
a = 1
def func():
exec("global a;a=3")
print(a)
pass a customized global or local context to exec:
def func():
my_context = {'a': 1}
exec("a=3", None, my_context)
print(my_context['a'])
NOTE: DONT USE eval or exec IN YOUR SERIOUS CODE UNLESS YOU KNOW WHAT YOU ARE DOING.
EDIT NOTE
the following solution(2th solution mentioned in the comments) wont work:
def func():
a = 1
exec("a=3")
print(a) # still get 1 here
The Python C API function PyEval_EvalCode let's you execute compiled Python code. I want to execute a block of Python code as if it were executing within the scope of a function, so that it has its own dictionary of local variables which don't affect the global state.
This seems easy enough to do, since PyEval_EvalCode lets you provide a Global and Local dictionary:
PyObject* PyEval_EvalCode(PyCodeObject *co, PyObject *globals, PyObject *locals)
The problem I run into has to do with how Python looks up variable names. Consider the following code, that I execute with PyEval_EvalCode:
myvar = 300
def func():
return myvar
func()
This simple code actually raises an error, because Python is unable to find the variable myvar from within func. Even though myvar is in the local dictionary in the outer scope, Python doesn't copy it into the local dictionary in the inner scope. The reason for this is as follows:
Whenever Python looks up a variable name, first it checks locals, then it checks globals, and finally it checks builtins. At module scope, locals and globals are the SAME dictionary object. So the statement x = 5 at module scope will place x in the the locals dictionary, which is also the globals dictionary. Now, a function defined at module scope which needs to lookup x won't find x within the function-scope locals, because Python doesn't copy module-scope locals into function-scope locals. But this normally isn't a problem, because it can find x in globals.
x = 5
def foo():
print(x) # This works because 'x' in globals() == True
It's only with nested functions, that Python seems to copy outer-scope locals into inner-scope locals. (It also seems to do so lazily, only if they are needed within the inner scope.)
def foo():
x = 5
def bar():
print(x) # Now 'x' in locals() == True
bar()
So the result of all this is that, when executing code at module scope, you HAVE to make sure that your global dictionary and local dictionary are the SAME object, otherwise module-scope functions won't be able to access module-scope variables.
But in my case, I don't WANT the global dictionary and local dictionary to be the same. So I need some way to tell the Python interpreter that I am executing code at function scope. Is there some way to do this? I looked at the PyCompileFlags as well as the additional arguments to PyEval_EvalCodeEx and can't find any way to do this.
Python doesn't actually copy outer-scope locals into inner-scope locals; the documentation for locals states:
Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
Here "free" variables refers to variables closed over by a nested function. It's an important distinction.
The simplest fix for your situation is just to pass the same dict object as globals and locals:
code = """
myvar = 300
def func():
return myvar
func()
"""
d = {}
eval(compile(code, "<str>", "exec"), d, d)
Otherwise, you can wrap your code in a function and extract it from the compiled object:
s = 'def outer():\n ' + '\n '.join(code.strip().split('\n'))
exec(compile(s, '<str>', 'exec').co_consts[0], {}, {})
I came across some code which kind of puzzled me. Here's a minimal example which shows this:
# of course, the ... are not part of the actual code
some_var = {"key1":"value1" ... "keyN":"valueN"}
def some_func():
v = some_var["key1"]
The code works, but the fact that I can access some_var directly confuses me. The last time I had to write some Python code, I remember having to write some_func like this:
def some_func():
global some_var
v = some_var["key1"]
I am using Python 2.7.1 on a Windows 7 PC. Did something change in the 2.7 release that allows for this?
No, you just can't reassign some_var in a local scope. Consider the following:
some_var = {}
def some_func():
# some_var[5] = 6
some_var = {1:2}
some_var[3] = 4
some_func()
print (repr(some_var)) # {}
You'll see the assignment in some_func actually creates a local variable which shadows the global one. Therefore, uncommenting the line would result in an UnboundLocalError - you can't access variables before they're defined.
There's a difference between using (e.g. calling or using in an expression) a name from an outer scope and assigning it (and there's a difference between assigning a bare variable and assigning a member of an object pointed to by a variable - x.y = ... and x[...] = ... count as method calls!).
You only need to declare that a variable is from an outer scope if you're re-assigning it. In Python 2, you can only do that with global variables (via global var), in Python 3 you can do it for abritarily nested scopes (e.g. closures) using nonlocal var.
Using a nonlocal variable as in you example doesn't require global. It does, however, as soon as you assign the variable (with the aforementioned definition of assignment) anywhere within that scope - so add a line some_var = ... after the line where you're using it and you'll get an UnboundLocalError. Refer to the documentation for the nitty gritty details.
You only have to use global if you want to assign a new value to that variable.
Nested scope was introduced in Python 2.1 (and enabled by default in Python 2.2) (emphasis mine):
Put simply, when a given variable name is not assigned a value within a function (by an assignment, or the def, class, or import statements), references to the variable will be looked up in the local namespace of the enclosing scope. A more detailed explanation of the rules, and a dissection of the implementation, can be found in the PEP.
You only need to use global if you intend to assign to the variable, for reading the variable this is not necessary. This difference is not arbitrary, though it may seem like it at first glance.
When reading a value the interpreter can just look for a local variable named some_var, if it cannot find it then it looks for a global variable of that name. These are simple and straight forward semantics.
When assigning values to a variable the interpreter needs to know whether you intend to assign to a local variable some_var or a global variable. The interpreter assumes that some_var = 2 when called within a function is assigning to a local variable, this makes sense since this is the most common case. For the relatively rare times when you want to assign to a global variable from within a function then you use the global modifier global some_var = 2.
It depends on the usage of the variable in the function
Python variable scope error
Assigning a value to a name makes the name local, unless the name is explicitly declared global.
a = 12
def foo():
a = 42
print a # uses local
foo()
>>> 42
def foo():
global a
a = 42
foo()
print a
>>> 42
If a name is not assigned to, it is global.
a = 12
def foo():
print a # uses global
foo()
>>> 12
In short, you only have to explicitly declare a name global if you will be assigning to it. If you are just reading from it, you can use it at will. However, if you ever assign to the variable, it will be considered local in that function unless you declared it global.
b = 5
def foo():
print b
b = 7
foo()
>>> ???
Since b is assigned to in foo() and not declared global, Python decides at compile time that b is a local name. Therefore b is a local name throughout the whole function, including at the print statement before the assignment.
Therefore the print statement gives you an error, because the local name b has not been defined!