I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
In my python package I have an entry_point run.py file which takes the seed (e.g. 42) and the cuda device (e.g. "cuda:0") as command line argument.
Since both of these variables are used throughout the entire package at different places, I don't want to pass them as arguments from function to function. Hence, I did the following:
utils.py:
import random
import numpy as np
import torch
def set_device(device: str):
global _DEVICE
_DEVICE = torch.device(device)
def get_device() -> torch.device:
return _DEVICE
def set_seed_number(seed: int):
global _SEED
_SEED = seed
def set_seeds():
torch.manual_seed(_SEED)
random.seed(_SEED)
np.random.seed(_SEED)
And then within run.py I set these variables once by calling:
from package.utils import set_device, set_seed_number
...
set_device(device)
set_seed_number(seed=seed)
Now I can import and call the get_device()and set_seeds method from anywhere in my package and I don't have to pass these variables as arguments.
So far this approach works fine, but after reading that using globals in python is strongly discouraged I am wondering if there is a more pythonic way of achieving the above discribed goal?
I already thought of having a dedicated Singleton class, which dynamically would instantiate those constants but I am not exactly sure if and how that would work and if it would be considered more "pythonic" after all.
Thanks already for your answers and maybe you can point me to some patterns that seem applicable in this situation. I can only guess that I am not the first one trying to achieve the above discribed goal.
I can't honestly see a problem with global if it is used sparingly and only when there is a strong reason to do so. (I think the strong discouragement aganst global is because it is often abused.)
But as regards your proposed custom class, there is no need to instantiate it -- you can just set class variables.
main.py
import settings
settings.set_foo(3)
print(settings.Settings.foo)
settings.py
class Settings:
pass
def set_foo(x):
Settings.foo = x
This is no different in principle from putting your data items inside some other mutable collection e.g. a dictionary and then setting them inside functions in the module that defines it (or another that imports it).
main.py
import settings
settings.set_foo(3)
print(settings.settings['foo'])
settings.py
settings = {}
def set_foo(x):
settings['foo'] = x
I want to use a config.py file in which I will be declaring some constants to be used in mainFile.py.
I'll be defining these "constant variables" using Capital letters.
I want to make it easier to configure the mainFile.py parameters by defining the constants only in config.py.
Ofcourse, in mainFile.py, the values of these constant are by no means altered.
So, my question is :
If in mainFile.py I use "From config.py import *", are the "constant variables" defined in config.py used as global variables in mainFile.py?
E.g. inn functions defined within mainFile.py, do I need to re-define these variables as global in order to use them?
You can simply write import config, and all of your constants will be accessible to your code in mainFile.py.
There are two options:
As already pointed out by #Josh, in config.py you name your variables, such as m=10, t=52, and so on. Then on mainFile.py you import config and access your variables as config.m that will result in 10. Like:
y = config.m
y = 10
The second option is a bit longer. You create a function such as
def m():
m = 10
return m
Then on mainFile.py you import config and access the variable as config.m() that will yield 10. Like
z = config.m()
z = 10
global is required if you are modifying the imported variable in some case and you need to reflect that value on all other places where the variable is being accessed.
since you are just reading the value of the imported variable here, global is no required. also inside functions if any variable is read first it searches on local scope then on global scope. So no global required inside function for reading purpose.
I just finished the tutorial for making a rogue-like-game and I'm on my way to implement freatures.
The problem is, the whole game is a single file with 1k+ lines.
As you can see:
http://roguebasin.roguelikedevelopment.org/index.php?title=Complete_Roguelike_Tutorial,_using_python%2Blibtcod,_part_13_code
I want to divide it in different files/folders to handle the implements better.Maybe a file for each aspect of the game like map/player/npcs/items... but at least divide in Classes/Functions/Main.
The problem is, when I put this in the Main.py:
from Classes import *
from Functions import *
I get
NameError: name 'libtcod' is not defined
Which is a Module used in Main.
Then I tried to import the Main.py in the Classes.py and Functions.py
And get the
NameError: global name 'MAP_WIDTH' is not defined
MAP_WIDTH is a global variable in Main.py
I also tried to import the whole Main.py in Classes.py and Functions.py
But I get:
NameError: name 'Fighter' is not defined
Fighter is a Class inside Classes.py
Can anyone help me sort this out so I can start implement freatures.
EDIT: One simple example is:
Main.py
from Functions import *
def plus_one(ab):
return ab +1
a = 1
b = 2
c = add_things()
print plus_one(c)
Functions.py
from Main import *
def add_things():
global a,b
return a + b
It's a simple example, but in the project it get a lot of mutual dependecy between classes/functions and the main file.
There are many issues with your code and your planned program architecture. Please read my comment on your post. You need to shore up your knowledge of object oriented programming.
First, it is highly recommended to never use from Classes import *. You should use import Classes. Then to access functions or constants from the module you would use Classes.function_name or Classes.constant. See for more info on how to properly import in Python: http://effbot.org/zone/import-confusion.htm
Second, global variables are not recommended in Python. But if you do need them, you need to remember that in python a global variable means global to a module, not your entire program. Global variables in Python are a strange beast. If you need to read a global variable nothing special is required. However if you want to modify a global variable from within a function, then you must use the global keyword.
Thirdly, what you are doing is called a circle dependancy. Module A, imports Module B and Module B imports Module A. You can define shared functions, classes etc. in a third Module C. Then both A and B can import Module C. You can also defined your constants like MAP_WIDTH in module C and access them from A or B with C.MAP_WIDTH provided you have an import C.
A co-worker has a library that uses a hard-coded configuration defined in its own file. For instance:
constants.py:
API_URL="http://example.com/bogus"
Throughout the rest of the library, the configuration is accessed in the following manner.
from constants import API_URL
As you can imagine, this is not very flexible and causes problems during testing. If I want to change the configuration, I have to modify constants.py, which is in source code management.
Naturally, I'd much rather load the configuration from a JSON or YAML file. I can read the configuration into an object with no problems. Is there a way I can override the constants.py module without breaking the code, so that each global, e.g. API_URL is replaced by a value provided by my file?
I was thinking that after each from constants import ... I could add something like this:
from constants import * # existing configuration import
import json
new_config = json.load(open('config.json')) # load my config file into a dictionary
constants.__dict__.update(new_config) # override any constants with what I've loaded
The problem with this, of course, is that it's not very "DRY" and looks like it might be brittle.
Does anyone have a suggestion for doing this more cleanly? Thanks!
EDIT: looks like my approach doesn't work anyway. I guess "from import *" copies the values from the module into the current module's global scope?
DOUBLE EDIT: no, it does work; I'm just confused. But rather than doing this in X different files I'd like to have it work transparently if possible.
from module import <name> creates a reference in the importing module global namespace to the imported object. If that is an immutable, that means you now have to monkeypatch the value in the module that imported it.
Your only hope is to be the first to import constants and monkeypatch the names in that module. Subsequent imports will then use your monkeypatched values.
To patch the original module early, the following is enough:
import constants
for name, value in new_config.iteritems():
setattr(constants, name, value)