Why does this code raise NameError?:
def func():
exec("my_var = 42")
print(my_var)
func()
There are many related questions, but I have not found a clear answer as to WHY. I don't want a workaround.
Also, I have noticed that the code works when I run:
exec("my_var = 42", globals(), globals())
But not really sure why.
The documentation for exec() states: "...if the optional parts are omitted [second and third arguments], the code is executed in the current scope.". The current scope is the func() function. Why can't I access my_var from this same scope?
The parser/compiler doesn't take the argument to exec() into account (since it could be a variable). So when it parses the function it doesn't see the assignment to my_var, so it treats it as a global variable. But the assignment in exec() will create a local variable. So the variable that was assigned is not the same one that it tries to print.
In addition, if you want changes to local variables to be visible, you have to pass the locals() dictionary explicitly. The [documentation] states:
Note: The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.
Related
I tested this out with a program I wrote myself:
>>> def f():
f=['f',1,2]
def g():
g=1
print('this prints out f from f(): ',f)
print("id",id(f))
def x():
x=1
print('this also prints out f from f():',f)
print('id',id(f))
x()
g()
>>> f()#output
this prints out f from f(): ['f', 1, 2]
id 140601546763464
this also prints out f from f(): ['f', 1, 2]
id 140601546763464
From what I learned, the innermost x() function can only access its own local namespace, the enclosing namespace, the global, and finally the built-in namespace. I initially thought that trying to access the list f declared in function f() from function x() would raise an error, as the f() function's namespace cannot be classified as any of the aforementioned elements. After running the program, I realized you indeed can access the list f from the function x(). I don't quite understand how this works though. My guess is that checking the enclosing namespace not only checks the local namespace of the enclosing function but the enclosing function for it as well, in a process that works almost recursively. Can somebody please explain how this works?
Python resolves names using LEGB rule:(LEGB means Local, Enclosing, Global, and Built-in)
Local scope:
contains the names that are defined inside the function.
visible only inside the function
created at function call(If we call the function multiple times each call creates new local scope)
will be destroyed once function return
Enclosing or nonlocal:
exists for nested functions
contains names defined in the enclosing function
visible in inner and enclosing functions.
Global:
contains all the names defined at the top level of a program
visible from everywhere inside the code.
exist throut the life of code.
Built-in:
created whenever we run a script
contains keywords, functions, exceptions, etc that are built into Python
visible everywhere in the code
The LEGB rule is a rule which determines the order in which Python looks up names.
i.e Python will look the name sequentially in the local, enclosing, global, and built-in scope. And inner scope codes can outer scope names but outer scope codes cannot access inner scope names.
When we use nested functions the scope resolving is as follows:
check the local scope(inside the function)
If not found check enclosing scopes of outer functions from the innermost scope to the outermost scope
If not found look the global scope
If not found look built-ins
Still not found raise error
A little easy problem:
exec("a=3")
print(a)
# This will print 3
If I use this:
def func():
exec("a=3")
print(a)
func()
# NameError: name 'a' is not defined.
What happened?How could I use exec() to assign it a value in a function?
Edit:I found a question with the same trouble but still didn't solved.
why do you want to do that?
I know using exec() is bad and unsafe.But recently I try to solve a OP's problem.I met it.
Python knows several kinds of scope: module global, function local, nonlocal closures, class body. Notably, scope resolution is defined statically at byte code compile time – most importantly, whether names refer to local/nonlocal or global scope cannot be changed.
Of these scopes, only global scope is guaranteed to behave similar to a dict, and as such writeable. The local/nonlocal scope is generally not writeable, and new variables cannot be added to it.
exec will write to the global scope if locals is not passed in; globals must then explicitly be set to its default of globals().
def func():
exec("a='exec'", globals()) # access only global scope
print(a)
a = 'global'
func() # prints exec
However, once a name is local to a function, exec cannot modify it.
def func():
a = 'local' # assignment makes name local
exec("a='exec global'", globals())
exec("a='exec locals'", globals(), locals())
print(a)
a = 'global'
func() # prints local
While a dict-like representation of local/nonlocal scope exists, the interpreter is not required to honour changes to it.
locals()
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks. Note that at the module level, locals() and globals() are the same dictionary.
Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
Even though exec does take locals as a dict, these are not treated like function locals/nonlocals. Attempts to modify the default locals (the result of locals()) are not defined.
exec()
... If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object. Remember that at module level, globals and locals are the same dictionary. If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition.
Note: The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. ...
execute help(exec) in your python3 REPL, you will get the following docuement:
Help on built-in function exec in module builtins:
exec(source, globals=None, locals=None, /)
Execute the given source in the context of globals and locals.
The source may be a string representing one or more Python statements
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.
so there are at least 2 options to provide the value of argument 'a':
assign a value to the variable 'a' in the module's global scope:
a = 1
def func():
exec("global a;a=3")
print(a)
pass a customized global or local context to exec:
def func():
my_context = {'a': 1}
exec("a=3", None, my_context)
print(my_context['a'])
NOTE: DONT USE eval or exec IN YOUR SERIOUS CODE UNLESS YOU KNOW WHAT YOU ARE DOING.
EDIT NOTE
the following solution(2th solution mentioned in the comments) wont work:
def func():
a = 1
exec("a=3")
print(a) # still get 1 here
I don't quite understand why the code
def f():
print(s)
s = "foo"
f()
runs perfectly fine but
def f():
print(s)
s = "bar"
s = "foo"
f()
gives me UnboundLocalError. I know that I can fix this by declaring s as a global variable inside the function or by simply passing s an an argument into the function.
Still I don't understand how python seemingly knows whether or not s is referenced inside the function before the line has been executed? Does python make some sort of list of all local variable references when the function is read into the global frame?
Other answers have focused on the practical aspects of this but have not actually answered the question you asked.
Yes, the Python compiler tracks which variables are assigned when it is compiling a code block (such as in a def). If a name is assigned to in a block, the compiler marks it as local.Take a look at function.__code__.co_varnames to see which variables the compiler has identified.
The nonlocal and global statements can override this.
Yes, Python will look-ahead to recover all variables declared in the local scope. These will then overshadow global variables.
So in your code:
def f():
print(s)
s = "foo"
f()
Python did not find s in the local scope, so it tries to recover it from the global scope and finds "foo".
Now in the other case the following happens:
def f():
print(s)
s = "bar
s = "foo"
f()
Python knows that s is a local variable because it did a look-ahead before runtime, but at runtime it was not assigned yet so it raised and exception.
Note that Python will even let you reference variables that have not been declared anywhere. If you do:
def foo():
return x
f()
You will get a NameError, because Python, when not finding, x as a local variable will just remember that at runtime it should look for a global variable named x and then fail if it does not exist.
So UnboundLocalError means that the variable may eventually be declared in scope but has not been yet. On the other hand NameError means that the variable will never be declared in the local scope, so Python tried to find it in the global scope, but it did not exist.
The python documentation states "execfile() cannot be used reliably to modify a function’s locals." on the page http://docs.python.org/2/library/functions.html#execfile
Can anyone provide any further details on this statement? The documentation is fairly minimal. The statement seems very contradictory to "If both dictionaries are omitted, the expression is executed in the environment where execfile() is called." which is also in the documentation. Is there a special case when excecfile is used within a function then execfile is then acting similar to a function in that it creates a new scoping level?
If I use execfile in a function such as
def testfun():
execfile('thefile.py',globals())
def testfun2():
print a
and there are objects created by the commands in 'thefile.py' (such as the object 'a'), how do I know if they are going to be local objects to testfun or global objects? So, in the function testfun2, 'a' will appear to be a global? If I omit globals() from the execfile statement, can anyone give a more detailed explanation why objects created by commands in 'thefile.py' are not available to 'testfun'?
In Python, the way names are looked up is highly optimized inside functions. One of the side effects is that the mapping returned by locals() gives you a copy of the local names inside a function, and altering that mapping does not actually influence the function:
def foo():
a = 'spam'
locals()['a'] = 'ham'
print(a) # prints 'spam'
Internally, Python uses the LOAD_FAST opcode to look up the a name in the current frame by index, instead of the slower LOAD_NAME, which would look for a local name (by name), then in the globals() mapping if not found in the first.
The python compiler can only emit LOAD_FAST opcodes for local names that are known at compile time; but if you allow the locals() to directly influence a functions' locals then you cannot know all the local names ahead of time. Nested functions using scoped names (free variables) complicates matters some more.
In Python 2, you can force the compiler to switch off the optimizations and use LOAD_NAME always by using an exec statement in the function:
def foo():
a = 'spam'
exec 'a == a' # a noop, but just the presence of `exec` is important
locals()['a'] = 'ham'
print(a) # prints 'ham'
In Python 3, exec has been replaced by exec() and the work-around is gone. In Python 3 all functions are optimized.
And if you didn't follow all this, that's fine too, but that is why the documentation glosses over this a little. It is all due to an implementation detail of the CPython compiler and interpreter that most Python users do not need to understand; all you need to know that using locals() to change local names in a function does not work, usually.
Locals are kind of weird in Python. Regular locals are generally accessed by index, not by name, in the bytecode (as this is faster), but this means that Python has to know all the local variables at compile time. And that means you can't add new ones at runtime.
Now, if you use exec in a function, in Python 2.x, Python knows not to do this and falls back to the slower method of accessing local variables by name, and you can make new ones programmatically. (This trick was removed in Python 3.) You'd think Python would also do this for execfile(), but it doesn't, because exec is a statement and execfile() is a function call, and the name execfile might not refer to the built-in function at runtime (it can be reassigned, after all).
What will happen in your example function? Well, try it and find out! As the documentation for execfile states, if you don't pass in a locals dict, the dict you pass in as globals will be used. You pass in globals() (your module's real global variables) so if it assigns to a, then a becomes a global.
Now you might try something like this:
def testfun():
execfile('thefile.py')
def testfun2():
print a
return testfun2
exec ""
The exec statement at the end forces testfun() to use the old-style name-based local variables. It doesn't even have to be executed, as it is not here; it just has to be in the function somewhere.
But this doesn't work either, because the name-based locals don't support nesting functions with free variables (a in this case). That functionality also requires Python know all the local variables at function definition time. You can't even define the above function—Python won't let you.
In short, trying to deal with local variables programmatically is a pain and the documentation is correct: execfile() cannot reliably be used to modify a function's locals.
A better solution, probably, is to just import the file as a module. You can do this within the function, then access values in the module the usual way.
def testfun():
import thefile
print thefile.a
If you won't know the name of the file to be imported until runtime, you can use __import__ instead. Also, you may need to modify sys.path to make sure the directory you want to import from is first in the path (and put it back afterward, probably).
You can also just pass in your own dictionary to execfile and afterward, access the variables from the executed file using myVarsDict['a'] and so on.
The Python C API function PyEval_EvalCode let's you execute compiled Python code. I want to execute a block of Python code as if it were executing within the scope of a function, so that it has its own dictionary of local variables which don't affect the global state.
This seems easy enough to do, since PyEval_EvalCode lets you provide a Global and Local dictionary:
PyObject* PyEval_EvalCode(PyCodeObject *co, PyObject *globals, PyObject *locals)
The problem I run into has to do with how Python looks up variable names. Consider the following code, that I execute with PyEval_EvalCode:
myvar = 300
def func():
return myvar
func()
This simple code actually raises an error, because Python is unable to find the variable myvar from within func. Even though myvar is in the local dictionary in the outer scope, Python doesn't copy it into the local dictionary in the inner scope. The reason for this is as follows:
Whenever Python looks up a variable name, first it checks locals, then it checks globals, and finally it checks builtins. At module scope, locals and globals are the SAME dictionary object. So the statement x = 5 at module scope will place x in the the locals dictionary, which is also the globals dictionary. Now, a function defined at module scope which needs to lookup x won't find x within the function-scope locals, because Python doesn't copy module-scope locals into function-scope locals. But this normally isn't a problem, because it can find x in globals.
x = 5
def foo():
print(x) # This works because 'x' in globals() == True
It's only with nested functions, that Python seems to copy outer-scope locals into inner-scope locals. (It also seems to do so lazily, only if they are needed within the inner scope.)
def foo():
x = 5
def bar():
print(x) # Now 'x' in locals() == True
bar()
So the result of all this is that, when executing code at module scope, you HAVE to make sure that your global dictionary and local dictionary are the SAME object, otherwise module-scope functions won't be able to access module-scope variables.
But in my case, I don't WANT the global dictionary and local dictionary to be the same. So I need some way to tell the Python interpreter that I am executing code at function scope. Is there some way to do this? I looked at the PyCompileFlags as well as the additional arguments to PyEval_EvalCodeEx and can't find any way to do this.
Python doesn't actually copy outer-scope locals into inner-scope locals; the documentation for locals states:
Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
Here "free" variables refers to variables closed over by a nested function. It's an important distinction.
The simplest fix for your situation is just to pass the same dict object as globals and locals:
code = """
myvar = 300
def func():
return myvar
func()
"""
d = {}
eval(compile(code, "<str>", "exec"), d, d)
Otherwise, you can wrap your code in a function and extract it from the compiled object:
s = 'def outer():\n ' + '\n '.join(code.strip().split('\n'))
exec(compile(s, '<str>', 'exec').co_consts[0], {}, {})