I read here a clear way to share variables across modules.
Accordingly, my code is set up like this:
config.py
def init():
global x
mod.py
import config
def set_globals(month):
config.x = month
main.py
import config
config.init()
import mod
mod.set_globals(2)
print(config.x)
Now, this is just a MWE. In the real case, I have lots of variables to share between modules - not just x. And mod.py actually reads a file to get these values and store them in variables.
main.py is designed set the environment and then drop the user into interactive mode, so that the user can play around with these functions and these variables.
My question is: can I simplify the variable names so that I don't need to use config.x but simply x? I know I can use something like this: import config as g and then refer to g.x. But I'd like to make it even simpler.
But when I try from config import *, I always get errors about variables not being defined. Is it possible?
And as a related question, given that global variables are somewhat evil™, is the method of sharing variables presented in the link in my top sentence also evil?
I understand that below are the scoping levels in python (listed in highest to lowest order)
Local(L): Defined inside function/class
Enclosed(E): Defined inside enclosing functions(Nested function concept)
Global(G): Defined at the uppermost level
Built-in(B): Reserved names in Python builtin modules
I understand this concept when we have a single file. I am trying to understand how this concept of namespace, local, enclosed, global, built-in works when using import in python.
I understand we can import using foll. approaches:
import modulename
import modulename.function(function/variable)
import package.module.function(function/variable)
from module import function(function/variable)
from package import module
from package.module import function(function/variable)
from module import *
from package import *
I know that in each of the above the entire module is loaded in the sys modules dictionary. But I am confused with respect to namespace, scope, global, local. I want to understand LEGB principle, namespace, global, localwith respect to the method of import vs from import. I have read that in case of point 1 (import modulename), the modulename is loaded in its own namespace and reference is available to the calling module with which it can change the variable values globally. Does this also mean it is in global namespace? Any if so then global with respect to what? Also I have read that from module import function/variable or from module import * brings function(s)/variable(s) in the calling modules namespace and any changes are not global. Does this mean that it is in local namespace? Local with respect to what?
Example:
Say module foo has a variable bar=10. When we use [import foo] - this brings foo in current namespace and allows us to change the value of bar via foo.bar=1000 and this change is visible to every piece of code that makes use of [import foo print(foo.bar)] after the above assignment was made. Where as in case of [from foo import bar], any change made to bar is only visible to this module and not any other module irrespective of whether they do an [import foo] or [from foo import bar] they would see value of bar as 10.
Any reference/links in this area will be very helpful. I want to understand the basics of how this works internally. Any information will be helpful to understand this. Articles that I found on this topic explain what is import vs from xx import yy. But they don't explain why/how the changes made become global (in case of import) vs local (in case of from import).
Some more examples:
In below code example, change made to a is only seen in the main.py file and does not affect the value of a in module1.py and subsequent modules that import module1.py will still see the value of a as 10.
module1.py:
a=10
main.py:
from module1 import a
print(a)
a=100
Whereas if we used import module1 instead of from module1 import a then the assignment would have changed value of a in module1.py and any subsequent import of module1.py would show the value of a as 100.
Note: Similarly we could also have functions inside the module.py that get and set the value of variable a. And using this approach, once we do a from module1 import getfn, setfn in main.py, we can use these functions which when called make the change to variable a and this change is globally visible.
You're over complicating things. From the point of view of imports, the only thing to know is that import foo brings foo into the current namespace, and from foo import bar brings bar into the current namespace.
Python has only 3 scopes: global local and builtin.
They are relative to the position in the code
Every variable whose value you can change is local
(note that changing global variable results in creating a new local variable instead,
so there are local and global variable of the same name)
Enclosed you speak of are local to that function
Every variable with unchangeable content whose value you can get is global
Builtin variables are exactly like global so they can even be considered global
to see what variables are global and what are local, put this line in your code
print("global:", globals(), "\n\nlocal:", locals())
I have three python files in a project:
'''lib.py
library file that first defines and also accesses a boolean flag'''
vflag = False
...
def lib_method()
global vflag
if not vflag:
do_something_here
'''app.py
main application that sets the boolean flag defined in lib.py'''
import lib
lib.vflag = method_that_sets_or_resets_vflag()
'''processor.py
does some processing and needs to access the bool flag'''
from lib import *
...
def processor_method()
global vflag
...
if vflag:
do_something_here
I am able to set/ reset the flag at app.py, but processor method is not able to get the correct value for this boolean variable.
It only gets whatever is set at the beginning of lib.py(not what is set by app.py). The requirement is to access the value set at runtime by app.py, NOT the value to which it was initialized by lib.py
I am importing the lib file in different ways at app.py and processor.py. Does that make a difference ?
It may be a fundamental mistake, so I will appreciate if some one can point me to a specific knowledge base.
Also, it is messing with my understanding of global variables. If I define a variable to be 'global', does that mean the variable stays to be a global variable in all the files that import this variable(or the file containing the variable definition)
When you use from lib import * in processor.py, you are getting a snapshot of what's going on in lib.py at that moment. The lib.py file is executed, and all of the functions and variables are copied and stored in the namespace of processor.py. You are not storing references to the original vflag from lib.py - you're storing an entirely new copy. So if you change vflag in lib.py, then processor.py will never know about it.
The better practice is to always use import lib, and just access the variable with lib.vflag. You don't even need to use the global keyword.
The best practice for this situation would be to pass vflag into the functions you need as a parameter and to not use global variables.
first.py
myGlobal = "hello"
def changeGlobal():
myGlobal="bye"
second.py
from first import *
changeGlobal()
print myGlobal
The output I get is
hello
although I thought it should be
bye
Why doesn't the global variable myGlobal changes after the call to the changeGlobal() function?
Try:
def changeGlobal():
global myGlobal
myGlobal = "bye"
Actually, that doesn't work either. When you import *, you create a new local module global myGlobal that is immune to the change you intend (as long as you're not mutating the variable, see below). You can use this instead:
import nice
nice.changeGlobal()
print nice.myGlobal
Or:
myGlobal = "hello"
def changeGlobal():
global myGlobal
myGlobal="bye"
changeGlobal()
However, if your global is a mutable container, you're now holding a reference to a mutable and are able to see changes done to it:
myGlobal = ["hello"]
def changeGlobal():
myGlobal[0] = "bye"
I had once the same concern as yours and reading the following section from Norman Matloff's Quick and Painless Python Tutorial was really a good help. Here is what you need to understand (copied from Matloff's book):
Python does not truly allow global variables in the sense that C/C++ do. An imported Python module will not have direct access to the globals in the module which imports it, nor vice versa.
For instance, consider these two files, x.py,
# x.py
import y
def f():
global x
x = 6
def main():
global x
x = 3
f()
y.g()
if __name__ == ’__main__’:
main()
and y.py:
# y.py
def g():
global x
x += 1
The variable x in x.py is visible throughout the module x.py, but not in y.py. In fact, execution of the line
x += 1
in the latter will cause an error message to appear, “global name ’x’ is not defined.”
Indeed, a global variable in a module is merely an attribute (i.e. a member entity) of that module, similar to a class variable’s role within a class. When module B is imported by module A, B’s namespace is copied to A’s. If module B has a global variable X, then module A will create a variable of that name, whose initial value is whatever module B had for its variable of that name at the time of importing. But changes to X in one of the modules will NOT be reflected in the other.
Say X does change in B, but we want code in A to be able to get the latest value of X in B. We can do that by including a function, say named GetX() in B. Assuming that A imported everything from B, then A will get a function GetX() which is a copy of B’s function of that name, and whose sole purpose is to return the value of X. Unless B changes that function (which is possible, e.g. functions may be assigned), the functions in the two modules will always be the same, and thus A can use its function to get the value of X in B.
Python global variables are not global
As wassimans points out above they are essentially attributes within the scope of the module they are defined in (or the module that contains the function that defined them).
The first confusion(bug) people run into is not realizing that functions have a local name space and that setting a variable in a function makes it a local to the function even when they intended for it to change a (global) variable of the same name in the enclosing module. (declaring the name
in a 'global' statement in the function, or accessing the (global) variable before setting it.)
The second confusion(bug) people run into is that each module (ie imported file) contains its own so called 'global' name space. I guess python things the world(globe) is the module -- perhaps we are looking for 'universal' variables that span more than one globe.
The third confusion (that I'm starting to understand now) is where are the 'globals' in the __main__ module? Ie if you start python from the command line in interactive mode, or if you invoke python script (type the name of the foo.py from the command shell) -- there is no import of a module whose name you can use.
The contents of 'globals()' or globals().keys() -- which gives you a list of the globals -- seems to be accessible as: dir(sys.modules['__main__'])
It seems that the module for the loaded python script (or the interactive session with no loaded script), the one named in: __name__, has no global name, but is accessible as the module whose name is '__main__' in the system's list of all active modules, sys.modules
The __debug__ variable is handy in part because it affects every module. If I want to create another variable that works the same way, how would I do it?
The variable (let's be original and call it 'foo') doesn't have to be truly global, in the sense that if I change foo in one module, it is updated in others. I'd be fine if I could set foo before importing other modules and then they would see the same value for it.
If you need a global cross-module variable maybe just simple global module-level variable will suffice.
a.py:
var = 1
b.py:
import a
print a.var
import c
print a.var
c.py:
import a
a.var = 2
Test:
$ python b.py
# -> 1 2
Real-world example: Django's global_settings.py (though in Django apps settings are used by importing the object django.conf.settings).
I don't endorse this solution in any way, shape or form. But if you add a variable to the __builtin__ module, it will be accessible as if a global from any other module that includes __builtin__ -- which is all of them, by default.
a.py contains
print foo
b.py contains
import __builtin__
__builtin__.foo = 1
import a
The result is that "1" is printed.
Edit: The __builtin__ module is available as the local symbol __builtins__ -- that's the reason for the discrepancy between two of these answers. Also note that __builtin__ has been renamed to builtins in python3.
I believe that there are plenty of circumstances in which it does make sense and it simplifies programming to have some globals that are known across several (tightly coupled) modules. In this spirit, I would like to elaborate a bit on the idea of having a module of globals which is imported by those modules which need to reference them.
When there is only one such module, I name it "g". In it, I assign default values for every variable I intend to treat as global. In each module that uses any of them, I do not use "from g import var", as this only results in a local variable which is initialized from g only at the time of the import. I make most references in the form g.var, and the "g." serves as a constant reminder that I am dealing with a variable that is potentially accessible to other modules.
If the value of such a global variable is to be used frequently in some function in a module, then that function can make a local copy: var = g.var. However, it is important to realize that assignments to var are local, and global g.var cannot be updated without referencing g.var explicitly in an assignment.
Note that you can also have multiple such globals modules shared by different subsets of your modules to keep things a little more tightly controlled. The reason I use short names for my globals modules is to avoid cluttering up the code too much with occurrences of them. With only a little experience, they become mnemonic enough with only 1 or 2 characters.
It is still possible to make an assignment to, say, g.x when x was not already defined in g, and a different module can then access g.x. However, even though the interpreter permits it, this approach is not so transparent, and I do avoid it. There is still the possibility of accidentally creating a new variable in g as a result of a typo in the variable name for an assignment. Sometimes an examination of dir(g) is useful to discover any surprise names that may have arisen by such accident.
Define a module ( call it "globalbaz" ) and have the variables defined inside it. All the modules using this "pseudoglobal" should import the "globalbaz" module, and refer to it using "globalbaz.var_name"
This works regardless of the place of the change, you can change the variable before or after the import. The imported module will use the latest value. (I tested this in a toy example)
For clarification, globalbaz.py looks just like this:
var_name = "my_useful_string"
You can pass the globals of one module to onother:
In Module A:
import module_b
my_var=2
module_b.do_something_with_my_globals(globals())
print my_var
In Module B:
def do_something_with_my_globals(glob): # glob is simply a dict.
glob["my_var"]=3
Global variables are usually a bad idea, but you can do this by assigning to __builtins__:
__builtins__.foo = 'something'
print foo
Also, modules themselves are variables that you can access from any module. So if you define a module called my_globals.py:
# my_globals.py
foo = 'something'
Then you can use that from anywhere as well:
import my_globals
print my_globals.foo
Using modules rather than modifying __builtins__ is generally a cleaner way to do globals of this sort.
You can already do this with module-level variables. Modules are the same no matter what module they're being imported from. So you can make the variable a module-level variable in whatever module it makes sense to put it in, and access it or assign to it from other modules. It would be better to call a function to set the variable's value, or to make it a property of some singleton object. That way if you end up needing to run some code when the variable's changed, you can do so without breaking your module's external interface.
It's not usually a great way to do things — using globals seldom is — but I think this is the cleanest way to do it.
I wanted to post an answer that there is a case where the variable won't be found.
Cyclical imports may break the module behavior.
For example:
first.py
import second
var = 1
second.py
import first
print(first.var) # will throw an error because the order of execution happens before var gets declared.
main.py
import first
On this is example it should be obvious, but in a large code-base, this can be really confusing.
I wondered if it would be possible to avoid some of the disadvantages of using global variables (see e.g. http://wiki.c2.com/?GlobalVariablesAreBad) by using a class namespace rather than a global/module namespace to pass values of variables. The following code indicates that the two methods are essentially identical. There is a slight advantage in using class namespaces as explained below.
The following code fragments also show that attributes or variables may be dynamically created and deleted in both global/module namespaces and class namespaces.
wall.py
# Note no definition of global variables
class router:
""" Empty class """
I call this module 'wall' since it is used to bounce variables off of. It will act as a space to temporarily define global variables and class-wide attributes of the empty class 'router'.
source.py
import wall
def sourcefn():
msg = 'Hello world!'
wall.msg = msg
wall.router.msg = msg
This module imports wall and defines a single function sourcefn which defines a message and emits it by two different mechanisms, one via globals and one via the router function. Note that the variables wall.msg and wall.router.message are defined here for the first time in their respective namespaces.
dest.py
import wall
def destfn():
if hasattr(wall, 'msg'):
print 'global: ' + wall.msg
del wall.msg
else:
print 'global: ' + 'no message'
if hasattr(wall.router, 'msg'):
print 'router: ' + wall.router.msg
del wall.router.msg
else:
print 'router: ' + 'no message'
This module defines a function destfn which uses the two different mechanisms to receive the messages emitted by source. It allows for the possibility that the variable 'msg' may not exist. destfn also deletes the variables once they have been displayed.
main.py
import source, dest
source.sourcefn()
dest.destfn() # variables deleted after this call
dest.destfn()
This module calls the previously defined functions in sequence. After the first call to dest.destfn the variables wall.msg and wall.router.msg no longer exist.
The output from the program is:
global: Hello world!
router: Hello world!
global: no message
router: no message
The above code fragments show that the module/global and the class/class variable mechanisms are essentially identical.
If a lot of variables are to be shared, namespace pollution can be managed either by using several wall-type modules, e.g. wall1, wall2 etc. or by defining several router-type classes in a single file. The latter is slightly tidier, so perhaps represents a marginal advantage for use of the class-variable mechanism.
This sounds like modifying the __builtin__ name space. To do it:
import __builtin__
__builtin__.foo = 'some-value'
Do not use the __builtins__ directly (notice the extra "s") - apparently this can be a dictionary or a module. Thanks to ΤΖΩΤΖΙΟΥ for pointing this out, more can be found here.
Now foo is available for use everywhere.
I don't recommend doing this generally, but the use of this is up to the programmer.
Assigning to it must be done as above, just setting foo = 'some-other-value' will only set it in the current namespace.
I use this for a couple built-in primitive functions that I felt were really missing. One example is a find function that has the same usage semantics as filter, map, reduce.
def builtin_find(f, x, d=None):
for i in x:
if f(i):
return i
return d
import __builtin__
__builtin__.find = builtin_find
Once this is run (for instance, by importing near your entry point) all your modules can use find() as though, obviously, it was built in.
find(lambda i: i < 0, [1, 3, 0, -5, -10]) # Yields -5, the first negative.
Note: You can do this, of course, with filter and another line to test for zero length, or with reduce in one sort of weird line, but I always felt it was weird.
I could achieve cross-module modifiable (or mutable) variables by using a dictionary:
# in myapp.__init__
Timeouts = {} # cross-modules global mutable variables for testing purpose
Timeouts['WAIT_APP_UP_IN_SECONDS'] = 60
# in myapp.mod1
from myapp import Timeouts
def wait_app_up(project_name, port):
# wait for app until Timeouts['WAIT_APP_UP_IN_SECONDS']
# ...
# in myapp.test.test_mod1
from myapp import Timeouts
def test_wait_app_up_fail(self):
timeout_bak = Timeouts['WAIT_APP_UP_IN_SECONDS']
Timeouts['WAIT_APP_UP_IN_SECONDS'] = 3
with self.assertRaises(hlp.TimeoutException) as cm:
wait_app_up(PROJECT_NAME, PROJECT_PORT)
self.assertEqual("Timeout while waiting for App to start", str(cm.exception))
Timeouts['WAIT_JENKINS_UP_TIMEOUT_IN_SECONDS'] = timeout_bak
When launching test_wait_app_up_fail, the actual timeout duration is 3 seconds.