I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
I want to know how to make variables from functions in imported modules available in the IPython interactive namespace.
I have some example code which I want to run from another script so I set it up as run_test.py:
def run_test():
a = 5
if __name__ == "__main__":
run_test()
I import the module as follows and call the function:
import run_test
run_test.run_test()
How do I make variable 'a' available in the interactive namespace? The only way I can do it is to make 'a' a global and run run_test.py directly rather than importing it and calling the function.
Any pointers appreciated.
I believe this can be accomplished by returning locals and then updating locals.
def main():
# do things
return locals()
if __name__ == '__main__':
new_locals = main()
locals().update(new_locals)
This seems to work, though it feels like a hack, so perhaps it won't always.
A reasonable example of where you want this: If you want access to all the variables in a function's namespace, but don't want that function's variables accessible to other parts of a script (i.e., other function definitions).
I'm developing a test engine with Python, but I'm facing some problems related to module loading and global functions.
The main idea of the framework I'm creating is to load a Python file containing functions and annotations "#thisisatest" to tell which functions are tests. I load this file with imp.load_source, and latter, I spawn threads that calls the function from the loaded module. It's something like this:
module = imp.load_source("test", "testdir/test.py")
function = module.testFunction
thread = threading.Thread(target=function)
thread.start()
Anyway, I want to connect to this test a "assertion function", doing something like:
module = imp.load_source("test", "testdir/test.py")
module.__globals__.assertAndTerminate = assertionFunction
function = module.testFunction
thread = threading.Thread(target=function)
thread.start()
And that's all right. The problem starts when the test.py imports another module that uses the assertAndTerminate function inside it. The module loaded by test.py is completely unaware from the __globals__ from test.py and don't know who's the assertAndTerminate I'm talking about (and that makes sense, since each module has its own __globals__).
Does anyone know a way I could set the same assertAndTerminate function for the test.py module and the modules loaded by it in a thread? I would prefer not searching for imports in a tree, is it possible?
Is there something like Thread(target=function, global_vars=["assertAndTerminate":assertionFunction])?
You need to set the attribute directly on the module; that is the global namespace for that module:
module = imp.load_source("test", "testdir/test.py")
module.assertAndTerminate = assertionFunction
You do have to set globals on a per-module basis. Globals from one module do no not propagate to other modules on import.
You can add to the __builtin__ module (builtin in Python 3):
import __builtin__
__builtin__.assertAndTerminate = assertionFunction
These are then visible in all modules:
>>> import __builtin__
>>> __builtin__.foobar = 'barbaz'
>>> foobar
'barbaz'
Generally speaking, you really want to avoid doing this. Find some other method to solve your problem. Import code instead of relying on globals being set.
I'm aware of the standard example: if you execute a module directly then it's __name__ global variable is defined as "__main__". However, nowhere in the documentation can I find a precise description of how __name__ is defined in the general case. The module documentation says...
Within a module, the module's name (as a string) is available as the value of the global variable __name__.
...but what does it mean by "the module's name"? Is it just the name of the module (the filename with .py removed), or does it include the fully-qualified package name as well?
How is the value of the __name__ variable in a Python module determined? For bonus points, indicate precisely where in the Python source code this operation is performed.
It is set to the absolute name of the module as imported. If you imported it as foo.bar, then __name__ is set to 'foo.bar'.
The name is determined in the import.c module, but because that module handles various different types of imports (including zip imports, bytecode-only imports and extension modules) there are several code paths to trace through.
Normally, import statements are translated to a call to __import__, which is by default implemented as a call to PyImport_ImportModuleLevelObject. See the __import__() documentation to get a feel for what the arguments mean. Within PyImport_ImportModuleLevelObject relative names are resolved, so you can chase down the name variables there if you want to.
The rest of the module handles the actual imports, with PyImport_AddModuleObject creating the actual namespace object and setting the name key, but you can trace that name value back to PyImport_ImportModuleLevelObject. By creating a module object, it's __name__ value is set in the moduleobject.c object constructor.
The __name__ variable is an attribute of the module that would be accessible by the name.
import os
assert os.__name__ == 'os'
Example self created module that scetches the import mechanism:
>>> import types
>>> m = types.ModuleType("name of module") # create new module with name
>>> exec "source_of_module = __name__" in m.__dict__ # execute source in module
>>> m.source_of_module
'name of module'
Lines from types module:
import sys
ModuleType = type(sys)
this is probably a dumb question, but wasnt too sure what else to do.
main.py
import module2
global x
hello="Hello"
x=module2.message()
x.say()
module2.py
class message:
def say():
print hello
When I print hello, i am referring to the hello variable in main.py however this method will return an error. Does anyone know the solution? (i would prefer not to pipe the hello variable into the function)
The only reliable solution is called encapsulation.
So, basically, you could change your code to look like that:
main.py
import module2
global x
hello="Hello"
x=module2.message(hello)
x.say()
module2.py
class message:
def __init__(self, hello):
self.hello = hello
def say():
print self.hello
Plus try to follow coding style of Python - life of you and future developers of your code will be easier.
Multiple options, but do note that one module cannot ever access the calling module directly.
Simply pass hello as a variable (def say(msg): print msg)
Pass all variables in main.py to module2: def say(g): print g['hello'] and say(globals())
Store it somewhere, then extract it when you need it.
Since main.py imports module2.py, you can access the globals defined in moule2 in main.
In your case since module2 is not importing main, so the globals in main is not accessed in module2.
one solution is that defined by #Tadeck
In this particular example, it's totally OK for module2.py to import main.py, there are some gotcha's though.
The most obvious is that main.py is probably being run from the command line, like python main.py, which has the effect of making that file think it's called __main__. You could import that module in module2, but that's sort of unusual; it's called "main.py" and you want to import main. Doing so will cause the module to be imported a second time.
For that to be OK, you have to arrange for importing the file to have no side effects unless it's imported as __main__. A very common idiom in python is to test that condition at the end of a module.
import module2
global x
hello="Hello"
def main():
x=module2.message()
x.say()
if __name__ == '__main__':
main()
And now it's just fine for module2.py to actually import main. On the other hand, importing variables from one module into another gets hard to predict when the imports can be recursive, you may not have that variable yet because the module is already trying to import you. On the other hand, it's always safe to refer to a variable in a module using dotted syntax. So your module2.py should be:
import main
class message:
def say():
print main.hello
which also makes it more obvious just where hello came from.