I was somehow surprised by the following behavior:
def main():
print "%s" % foo
if __name__ == "__main__":
foo = "bar"
main()
i.e. a module function has access to enclosing variables in the __main__. What's the explanation for it?
Variables in the current modules global scope are visible everywhere in the module -- this rule also holds for the __main__ module.
From Guido's tutorial:
At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:
the innermost scope, which is searched first, contains the local names
the scopes of any enclosing functions, which are searched starting with the nearest
enclosing scope, contains non-local, but also non-global names
the next-to-last scope contains the current module’s global names
the outermost scope (searched last) is the namespace containing built-in names
The thing here is that:
if __name__ == "__main__":
foo = "bar"
defines a global variable named foo in that script. so any function of that module will have access to it.
The piece of code listed above is global to the module and not inside any function.
foo is a module global variable (it's not in any function). All scopes within the module can access it.
In python there's the global scope, and functions have their own scopes. So it you define foo under the name==main, it's in the global scope. Also, it's not a mistake to use a variable which hasn't been declared yet, in a function, if it will be declared by the time the function will be called.
As sinelaw pointed out, the way out this annoyance and inadvertent bug/s is to use a function. This function can be within the 'if main:' like this:
if __name__ == "__main__":
def mainlet():
foo = "bar"
mainlet()
Related
I just bumped into an unexpected (at least, for me!) behavior, and I'm trying to understand it.
Let's say I have a main file:
main.py
from my_packages.module_00 import my_function
def main():
my_function()
if __name__ == "__main__":
main()
and, in the folder "my_packages", the module module_00 containing the function definition for "my_function" and a global variable:
module_00.py
global_var = 'global variable'
def my_function():
print(f'Do I know {global_var}???')
When I run main.py, it outputs:
Do I know global variable???
And I'm trying to figure out why it's working.
I would expect the variable global_var to have a scope limited only to the module where it's defined (the answer to this question seems to confirm it).
Basically, I assumed that importing my_function by
from my_packages.module_00 import my_function
was equivalent to copy/pasting the function definition in main.py. However, it seems that...the imported function somehow keeps track of the global variables declared in the module where the function itself has been defined?
Or am I missing something?
However, it seems that...the imported function somehow keeps track of the global variables declared in the module where the function itself has been defined?
That's exactly what it is doing.
>>> from module_00 import my_func
>>> my_func.__globals__['global_var']
'global variable'
>>> module_00.global_var is my_func.__globals__['global_var']
True
>>> module_00.global_var = 3
>>> my_func.__globals__['global_var']
3
__globals__ is a reference to the global namespace of the module where my_func was defined.
Fix a couple of typos and your code works as expected. Each module in Python has its own private symbol table for the module's global variables.
import in main.py must be my_func not my_function matching the function name as defined in module_00.py.
ref in f-string in module_00.py must be {global_var} not {global variable}.
main.py
from my_packages.module_00 import my_func
^^^^^
def main():
my_func() # my_func not my_function
if __name__ == "__main__":
main()
module_00.py
global_var = 'global variable'
def my_func():
print(f'Do I know {global_var}???')
^^^^^^^^^^
Output:
Do I know global variable???
If you want to access a module's global variable then you can import the module and access the variable with the syntax: package.module_name.variable_name; e.g. my_packages.module_00.global_var
I tested this out with a program I wrote myself:
>>> def f():
f=['f',1,2]
def g():
g=1
print('this prints out f from f(): ',f)
print("id",id(f))
def x():
x=1
print('this also prints out f from f():',f)
print('id',id(f))
x()
g()
>>> f()#output
this prints out f from f(): ['f', 1, 2]
id 140601546763464
this also prints out f from f(): ['f', 1, 2]
id 140601546763464
From what I learned, the innermost x() function can only access its own local namespace, the enclosing namespace, the global, and finally the built-in namespace. I initially thought that trying to access the list f declared in function f() from function x() would raise an error, as the f() function's namespace cannot be classified as any of the aforementioned elements. After running the program, I realized you indeed can access the list f from the function x(). I don't quite understand how this works though. My guess is that checking the enclosing namespace not only checks the local namespace of the enclosing function but the enclosing function for it as well, in a process that works almost recursively. Can somebody please explain how this works?
Python resolves names using LEGB rule:(LEGB means Local, Enclosing, Global, and Built-in)
Local scope:
contains the names that are defined inside the function.
visible only inside the function
created at function call(If we call the function multiple times each call creates new local scope)
will be destroyed once function return
Enclosing or nonlocal:
exists for nested functions
contains names defined in the enclosing function
visible in inner and enclosing functions.
Global:
contains all the names defined at the top level of a program
visible from everywhere inside the code.
exist throut the life of code.
Built-in:
created whenever we run a script
contains keywords, functions, exceptions, etc that are built into Python
visible everywhere in the code
The LEGB rule is a rule which determines the order in which Python looks up names.
i.e Python will look the name sequentially in the local, enclosing, global, and built-in scope. And inner scope codes can outer scope names but outer scope codes cannot access inner scope names.
When we use nested functions the scope resolving is as follows:
check the local scope(inside the function)
If not found check enclosing scopes of outer functions from the innermost scope to the outermost scope
If not found look the global scope
If not found look built-ins
Still not found raise error
This question already has answers here:
Visibility of global variables in imported modules
(9 answers)
Closed 7 years ago.
It appears that a class defined in a script has a different scope to one that is imported into the script. For example:
In a file foo.py:
class foo(object):
def __init__(self):
print globals()
In my main file:
from foo import foo
class bar(object):
def __init__(self):
print globals()
classimport = foo()
classinternal = bar()
The list of globals returned from foo and bar are different - why is this?
It is making life difficult as any class that needs to access the main globals() has to reside in the main file. How do I ensure that the imported class has the same global scope? Some things that I have tried after reading other posts here and here include:
module = __import__("foo", fromlist="foo")
globals()["foo"] = getattr(module, "foo")
and
__builtin__.foo = foo
Any help appreciated!
[EDIT] ---
So per the link above, this is answered in a duplicate article. It turns out that scope is not shared across modules. It mentions several ways around this, but in my case I need to actually create / read / write global variables. So I created a routine in the main script and pass it as an object when foo and bar are initialized. For example:
def PrintGlobals():
print globals()
class bar(object):
def __init__(self, PrintGlobals):
self.PrintGlobals = PrintGlobals
self.PrintGlobals()
classinternal = bar(PrintGlobals)
(Not my choice of how all this should work, its a hack until I get some time with the application devs :-)
Here's what the Python 3 FAQ has to say:
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a value anywhere within the function’s body, it’s assumed to be a local unless explicitly declared as global.
Though a bit surprising at first, a moment’s consideration explains this. On one hand, requiring global for assigned variables provides a bar against unintended side-effects. On the other hand, if global was required for all global references, you’d be using global all the time. You’d have to declare as global every reference to a built-in function or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for identifying side-effects.
To see globals in various scopes, try doing a print(globals()) at various points during your execution. For example: at the top-level module before any code is run, then in __init__.py if you have any code in there (because you import foo), at foo's module level, within each function, and before/after you modify any variables passed to the function.
This answer further explains:
I think the key thing you're missing here is that each module has its own "global" namespace. This can be a bit confusing at first, because in languages like C, there's a single global namespace shared by all external variables and functions. But once you get past that assumption, the Python way makes perfect sense.
Note however that all names assigned in a package __init__.py file are available in the package namespace when you import the package or a module in the package.
In Python 3.3.1, this works:
i = 76
def A():
global i
i += 10
print(i) # 76
A()
print(i) # 86
This also works:
def enclosing_function():
i = 76
def A():
nonlocal i
i += 10
print(i) # 76
A()
print(i) # 86
enclosing_function()
But this doesn't work:
i = 76
def A():
nonlocal i # "SyntaxError: no binding for nonlocal 'i' found"
i += 10
print(i)
A()
print(i)
The documentation for the nonlocal keyword states (emphasis added):
The nonlocal statement causes the listed identifiers to refer to
previously bound variables in the nearest enclosing scope.
In the third example, the "nearest enclosing scope" just happens to be the global scope. So why doesn't it work?
PLEASE READ THIS BIT
I do notice that the documentation goes on to state (emphasis added):
The [nonlocal] statement allows encapsulated code to
rebind variables outside of the local scope besides the global
(module) scope.
but, strictly speaking, this doesn't mean that what I'm doing in the third example shouldn't work.
The search order for names is LEGB, i.e Local, Enclosing, Global, Builtin. So the global scope is not an enclosing scope.
EDIT
From the docs:
The nonlocal statement causes the listed identifiers to refer to
previously bound variables in the nearest enclosing scope. This is
important because the default behavior for binding is to search the
local namespace first. The statement allows encapsulated code to
rebind variables outside of the local scope besides the global
(module) scope.
why is a module's scope considered global and not an enclosing one? It's still not global to other modules (well, unless you do from module import *), is it?
If you put some name into module's namespace; it is visible in any module that uses module i.e., it is global for the whole Python process.
In general, your application should use as few mutable globals as possible. See Why globals are bad?:
Non-locality
No Access Control or Constraint Checking
Implicit coupling
Concurrency issues
Namespace pollution
Testing and Confinement
Therefore It would be bad if nonlocal allowed to create globals by accident. If you want to modify a global variable; you could use global keyword directly.
global is the most destructive: may affect all uses of the module anywhere in the program
nonlocal is less destructive: limited by the outer() function scope (the binding is checked at compile time)
no declaration (local variable) is the least destructive option: limited by inner() function scope
You can read about history and motivation behind nonlocal in PEP: 3104
Access to Names in Outer Scopes.
It depends upon the Boundary cases:
nonlocals come with some senstivity areas which we need to be aware of. First, unlike the global statement, nonlocal names really must have previous been assigned in an enclosing def's scope when a nonlocal is evaluated or else you'll get an error-you cannot create them dynamically by assigning them anew in the enclosing scope. In fact, they are checked at function definition time before either or nested function is called
>>>def tester(start):
def nested(label):
nonlocal state #nonlocals must already exist in enclosing def!
state = 0
print(label, state)
return nested
SyntaxError: no binding for nonlocal 'state' found
>>>def tester(start):
def nested(label):
global state #Globals dont have to exits yet when declared
state = 0 #This creates the name in the module now
print(label, state)
return nested
>>> F = tester(0)
>>> F('abc')
abc 0
>>> state
0
Second, nonlocal restricts the scope lookup to just enclosing defs; nonlocals are not looked up in the enclosing module's global scope or the built-in scope outside all def's, even if they are already there:
for example:-
>>>spam = 99
>>>def tester():
def nested():
nonlocal spam #Must be in a def, not the module!
print('current=', spam)
spam += 1
return nested
SyntaxError: no binding for nonlocal 'spam' found
These restrictions make sense once you realize that python would not otherwise generally know enclosing scope to create a brand-new name in. In the prior listing, should spam be assigned in tester, or the module outside? Because this is ambiguous, Python must resolve nonlocals at function creation time, not function call time.
The answer is that the global scope does not enclose anything - it is global to everything. Use the global keyword in such a case.
Historical reasons
In 2.x, nonlocal didn't exist yet. It wasn't considered necessary to be able to modify enclosing, non-global scopes; the global scope was seen as a special case. After all, the concept of a "global variable" is a lot easier to explain than lexical closures.
The global scope works differently
Because functions are objects, and in particular because a nested function could be returned from its enclosing function (producing an object that persists after the call to the enclosing function), Python needs to implement lookup into enclosing scopes differently from lookup into either local or global scopes. Specifically, in the reference implementation of 3.x, Python will attach a __closure__ attribute to the inner function, which is a tuple of cell instances that work like references (in the C++ sense) to the closed-over variables. (These are also references in the reference-counting garbage-collection sense; they keep the call frame data alive so that it can be accessed after the enclosing function returns.)
By contrast, global lookup works by doing a chained dictionary lookup: there's a dictionary that implements the global scope, and if that fails, a separate dictionary for the builtin scope is checked. (Of course, writing a global only writes to the global dict, not the builtin dict; there is no builtin keyword.)
Theoretically, of course, there's no reason why the implementation of nonlocal couldn't fall back on a lookup in the global (and then builtin) scope, in the same way that a lookup in the global scope falls back to builtins. Stack Overflow is not the right place to speculate on the reason behind the design decision. I can't find anything relevant in the PEP, so it may simply not have been considered.
The best I can offer is: like with local variable lookup, nonlocal lookup works by determining at compile time what the scope of the variable will be. If you consider builtins as simply pre-defined, shadow-able globals (i.e. the only real difference between the actual implementation and just dumping them into the global scope ahead of time, is that you can recover access to the builtin with del), then so does global lookup. As they say, "simple is better than complex" and "special cases aren't special enough to break the rules"; so, no fallback behaviour.
first.py
myGlobal = "hello"
def changeGlobal():
myGlobal="bye"
second.py
from first import *
changeGlobal()
print myGlobal
The output I get is
hello
although I thought it should be
bye
Why doesn't the global variable myGlobal changes after the call to the changeGlobal() function?
Try:
def changeGlobal():
global myGlobal
myGlobal = "bye"
Actually, that doesn't work either. When you import *, you create a new local module global myGlobal that is immune to the change you intend (as long as you're not mutating the variable, see below). You can use this instead:
import nice
nice.changeGlobal()
print nice.myGlobal
Or:
myGlobal = "hello"
def changeGlobal():
global myGlobal
myGlobal="bye"
changeGlobal()
However, if your global is a mutable container, you're now holding a reference to a mutable and are able to see changes done to it:
myGlobal = ["hello"]
def changeGlobal():
myGlobal[0] = "bye"
I had once the same concern as yours and reading the following section from Norman Matloff's Quick and Painless Python Tutorial was really a good help. Here is what you need to understand (copied from Matloff's book):
Python does not truly allow global variables in the sense that C/C++ do. An imported Python module will not have direct access to the globals in the module which imports it, nor vice versa.
For instance, consider these two files, x.py,
# x.py
import y
def f():
global x
x = 6
def main():
global x
x = 3
f()
y.g()
if __name__ == ’__main__’:
main()
and y.py:
# y.py
def g():
global x
x += 1
The variable x in x.py is visible throughout the module x.py, but not in y.py. In fact, execution of the line
x += 1
in the latter will cause an error message to appear, “global name ’x’ is not defined.”
Indeed, a global variable in a module is merely an attribute (i.e. a member entity) of that module, similar to a class variable’s role within a class. When module B is imported by module A, B’s namespace is copied to A’s. If module B has a global variable X, then module A will create a variable of that name, whose initial value is whatever module B had for its variable of that name at the time of importing. But changes to X in one of the modules will NOT be reflected in the other.
Say X does change in B, but we want code in A to be able to get the latest value of X in B. We can do that by including a function, say named GetX() in B. Assuming that A imported everything from B, then A will get a function GetX() which is a copy of B’s function of that name, and whose sole purpose is to return the value of X. Unless B changes that function (which is possible, e.g. functions may be assigned), the functions in the two modules will always be the same, and thus A can use its function to get the value of X in B.
Python global variables are not global
As wassimans points out above they are essentially attributes within the scope of the module they are defined in (or the module that contains the function that defined them).
The first confusion(bug) people run into is not realizing that functions have a local name space and that setting a variable in a function makes it a local to the function even when they intended for it to change a (global) variable of the same name in the enclosing module. (declaring the name
in a 'global' statement in the function, or accessing the (global) variable before setting it.)
The second confusion(bug) people run into is that each module (ie imported file) contains its own so called 'global' name space. I guess python things the world(globe) is the module -- perhaps we are looking for 'universal' variables that span more than one globe.
The third confusion (that I'm starting to understand now) is where are the 'globals' in the __main__ module? Ie if you start python from the command line in interactive mode, or if you invoke python script (type the name of the foo.py from the command shell) -- there is no import of a module whose name you can use.
The contents of 'globals()' or globals().keys() -- which gives you a list of the globals -- seems to be accessible as: dir(sys.modules['__main__'])
It seems that the module for the loaded python script (or the interactive session with no loaded script), the one named in: __name__, has no global name, but is accessible as the module whose name is '__main__' in the system's list of all active modules, sys.modules