I'm bit confused about how the global variables work. I have a large project, with around 50 files, and I need to define global variables for all those files.
What I did was define them in my projects main.py file, as following:
# ../myproject/main.py
# Define global myList
global myList
myList = []
# Imports
import subfile
# Do something
subfile.stuff()
print(myList[0])
I'm trying to use myList in subfile.py, as following
# ../myproject/subfile.py
# Save "hey" into myList
def stuff():
globals()["myList"].append("hey")
An other way I tried, but didn't work either
# ../myproject/main.py
# Import globfile
import globfile
# Save myList into globfile
globfile.myList = []
# Import subfile
import subfile
# Do something
subfile.stuff()
print(globfile.myList[0])
And inside subfile.py I had this:
# ../myproject/subfile.py
# Import globfile
import globfile
# Save "hey" into myList
def stuff():
globfile.myList.append("hey")
But again, it didn't work. How should I implement this? I understand that it cannot work like that, when the two files don't really know each other (well subfile doesn't know main), but I can't think of how to do it, without using io writing or pickle, which I don't want to do.
The problem is you defined myList from main.py, but subfile.py needs to use it. Here is a clean way to solve this problem: move all globals to a file, I call this file settings.py. This file is responsible for defining globals and initializing them:
# settings.py
def init():
global myList
myList = []
Next, your subfile can import globals:
# subfile.py
import settings
def stuff():
settings.myList.append('hey')
Note that subfile does not call init()— that task belongs to main.py:
# main.py
import settings
import subfile
settings.init() # Call only once
subfile.stuff() # Do stuff with global var
print settings.myList[0] # Check the result
This way, you achieve your objective while avoid initializing global variables more than once.
See Python's document on sharing global variables across modules:
The canonical way to share information across modules within a single program is to create a special module (often called config or cfg).
config.py:
x = 0 # Default value of the 'x' configuration setting
Import the config module in all modules of your application; the module then becomes available as a global name.
main.py:
import config
print (config.x)
In general, don’t use from modulename import *. Doing so clutters the importer’s namespace, and makes it much harder for linters to detect undefined names.
You can think of Python global variables as "module" variables - and as such they are much more useful than the traditional "global variables" from C.
A global variable is actually defined in a module's __dict__ and can be accessed from outside that module as a module attribute.
So, in your example:
# ../myproject/main.py
# Define global myList
# global myList - there is no "global" declaration at module level. Just inside
# function and methods
myList = []
# Imports
import subfile
# Do something
subfile.stuff()
print(myList[0])
And:
# ../myproject/subfile.py
# Save "hey" into myList
def stuff():
# You have to make the module main available for the
# code here.
# Placing the import inside the function body will
# usually avoid import cycles -
# unless you happen to call this function from
# either main or subfile's body (i.e. not from inside a function or method)
import main
main.mylist.append("hey")
Using from your_file import * should fix your problems. It defines everything so that it is globally available (with the exception of local variables in the imports of course).
for example:
##test.py:
from pytest import *
print hello_world
and:
##pytest.py
hello_world="hello world!"
Hai Vu answer works great, just one comment:
In case you are using the global in other module and you want to set the global dynamically, pay attention to import the other modules after you set the global variables, for example:
# settings.py
def init(arg):
global myList
myList = []
mylist.append(arg)
# subfile.py
import settings
def print():
settings.myList[0]
# main.py
import settings
settings.init("1st") # global init before used in other imported modules
# Or else they will be undefined
import subfile
subfile.print() # global usage
Your 2nd attempt will work perfectly, and is actually a really good way to handle variable names that you want to have available globally. But you have a name error in the last line. Here is how it should be:
# ../myproject/main.py
# Import globfile
import globfile
# Save myList into globfile
globfile.myList = []
# Import subfile
import subfile
# Do something
subfile.stuff()
print(globfile.myList[0])
See the last line? myList is an attr of globfile, not subfile. This will work as you want.
Mike
I just came across this post and thought of posting my solution, just in case of anyone being in the same situation as me, where there are quite some files in the developed program, and you don't have the time to think through the whole import sequence of your modules (if you didn't think of that properly right from the start, such as I did).
In such cases, in the script where you initiate your global(s), simply code a class which says like:
class My_Globals:
def __init__(self):
self.global1 = "initial_value_1"
self.global2 = "initial_value_2"
...
and then use, instead of the line in the script where you initiated your globals, instead of
global1 = "initial_value_1"
use
globals = My_Globals()
I was then able to retrieve / change the values of any of these globals via
globals.desired_global
in any script, and these changes were automatically also applied to all the other scripts using them. All worked now, by using the exact same import statements which previously failed, due to the problems mentioned in this post / discussion here. I simply thought of global object's properties being changing dynamically without the need of considering / changing any import logic, in comparison to simple importing of global variables, and that definitely was the quickest and easiest (for later access) approach to solve this kind of problem for me.
Based on above answers and links within I created a new module called global_variables.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# ==============================================================================
#
# global_variables.py - Global variables shared by all modules.
#
# ==============================================================================
USER = None # User ID, Name, GUID varies by platform
def init():
""" This should only be called once by the main module
Child modules will inherit values. For example if they contain
import global_variables as g
Later on they can reference 'g.USER' to get the user ID.
"""
global USER
import getpass
USER = getpass.getuser()
# End of global_variables.py
Then in my main module I use this:
import global_variables as g
g.init()
In another child imported module I can use:
import global_variables as g
# hundreds of lines later....
print(g.USER)
I've only spent a few minutes testing in two different python multiple-module programs but so far it's working perfectly.
Namespace nightmares arise when you do from config import mySharedThing. That can't be stressed enough.
It's OK to use from in other places.
You can even have a config module that's totally empty.
# my_config.py
pass
# my_other_module.py
import my_config
def doSomething():
print(my_config.mySharedThing.message)
# main.py
from dataclasses import dataclass
from my_other_module import doSomething
import my_config
#dataclass
class Thing:
message: str
my_config.mySharedThing = Thing('Hey everybody!')
doSomething()
result:
$ python3 main.py
Hey everybody!
But using objects you pulled in with from will take you down a path of frustration.
# my_other_module.py
from my_config import mySharedThing
def doSomething():
print(mySharedThing.message)
result:
$ python3 main.py
ImportError: cannot import name 'mySharedThing' from 'my_config' (my_config.py)
And maybe you'll try to fix it like this:
# my_config.py
mySharedThing = None
result:
$ python3 main.py
AttributeError: 'NoneType' object has no attribute 'message'
And then maybe you'll find this page and try to solve it by adding an init() method.
But the whole problem is the from.
Related
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
I'm making a small game and wanted a settings page function that's in a seperate .py file to keep things more clean and easily editable. I have a global variable called textSpeed (which I use the global keyword to use properly in the function) which I change in this runSettings function, but when I print the variable back in my main file it hasn't changed.
The code in the main file (main.py)
from settings import runSettings
textSpeed = "not set"
runSettings()
print(textSpeed)
The code from the settings fuction file (settings.py)
def runSettings():
global textSpeed
textSpeed = input("select text speed. ")
print(textSpeed)
return textSpeed
textSpeed is a local variable - local to the main module.
You need to reference the variable from settings.
import settings
settings.textSpeed = "not set"
runSettings()
print(settings.textSpeed)
To avoid circular import, I advise you to create a third file if you wish to keep it this way. Let's call it varSpeed.py with the following code:
global textSpeed
textSpeed = "not set"
Then you can import varSpeed from both other files have access to that variable without the circular issue.
The runSettings function already retuns the value it sets as well. Instead of messing with mixing namespaces and importing global variables, just use the returned value in main.py:
textSpeed = runSettings()
the problem is that you are trying to change a variable from a different file, while you have not imported the file. I think the easiest way to handle this is to use a class variable like this:
Main file:
from class_file import MyClass
MyClass.run_settings()
print(MyClass.text_speed)
Settings file:
class MyClass:
text_speed = 'n/a'
#staticmethod
def run_settings():
MyClass.text_speed = input("Select text speed: ")
return MyClass.text_speed
I have an interesting application and I would like to access a global variable from a different module.
I currently have module x
from pkg.mod import *
in order to get the namespace foo.
where my pkg.mod looks like the following
import x
foo = 0
# do some stuff
foo = myClass()
That all works just great, however, I would like NOT to do that, mostly just because its bad.
If I try the following
from pkg import mod
I get a circular dependency issue. ImportError: cannot import name mod which makes sense to me.
What I want to do is
from pkg.mod import foo
but that also gives me an import name error: ImportError: cannot import name foo
Which I am not exactly sure why, except I dont think you can import variables specifically.
Is there anyway to import foo following correct practices?
---edit---
I have tried to remove the shared variable and put it in a third module in such a way
pkg.y
foo = 0
pkg.mod
from pkg.y import foo
foo = myClass()
x.dostuff()
x
from pkg.y import foo
def dostuff():
print(foo)
and now x thinks foo is still 0. (I can guarantee that mod is calling x, and thus happening first.)
When I need a true global variable I find it cleaner to create a module called trueglobals.py; it can remain empty but for proper documentation...
# -*- coding: utf-8 -*-
# Object references that are used extensively across many modules are
# stored here.
# someObject1 - instantiated in main.py
# someObject2 - instantiated in someothermodule.py
In the module instantiating the object...
import trueglobals
trueglobals.someObject1 = myClass(somevar)
In any module requiring the object...
import trueglobals
localVar = trueglobals.someObject1
I've literally been trying to understand Python imports for about a year now, and I've all but given up programming in Python because it just seems too obfuscated. I come from a C background, and I assumed that import worked like #include, yet if I try to import something, I invariably get errors.
If I have two files like this:
foo.py:
a = 1
bar.py:
import foo
print foo.a
input()
WHY do I need to reference the module name? Why not just be able to write import foo, print a? What is the point of this confusion? Why not just run the code and have stuff defined for you as if you wrote it in one big file? Why can't it work like C's #include directive where it basically copies and pastes your code? I don't have import problems in C.
To do what you want, you can use (not recommended, read further for explanation):
from foo import *
This will import everything to your current namespace, and you will be able to call print a.
However, the issue with this approach is the following. Consider the case when you have two modules, moduleA and moduleB, each having a function named GetSomeValue().
When you do:
from moduleA import *
from moduleB import *
you have a namespace resolution issue*, because what function are you actually calling with GetSomeValue(), the moduleA.GetSomeValue() or the moduleB.GetSomeValue()?
In addition to this, you can use the Import As feature:
from moduleA import GetSomeValue as AGetSomeValue
from moduleB import GetSomeValue as BGetSomeValue
Or
import moduleA.GetSomeValue as AGetSomeValue
import moduleB.GetSomeValue as BGetSomeValue
This approach resolves the conflict manually.
I am sure you can appreciate from these examples the need for explicit referencing.
* Python has its namespace resolution mechanisms, this is just a simplification for the purpose of the explanation.
Imagine you have your a function in your module which chooses some object from a list:
def choice(somelist):
...
Now imagine further that, either in that function or elsewhere in your module, you are using randint from the random library:
a = randint(1, x)
Therefore we
import random
You suggestion, that this does what is now accessed by from random import *, means that we now have two different functions called choice, as random includes one too. Only one will be accessible, but you have introduced ambiguity as to what choice() actually refers to elsewhere in your code.
This is why it is bad practice to import everything; either import what you need:
from random import randint
...
a = randint(1, x)
or the whole module:
import random
...
a = random.randint(1, x)
This has two benefits:
You minimise the risks of overlapping names (now and in future additions to your imported modules); and
When someone else reads your code, they can easily see where external functions come from.
There are a few good reasons. The module provides a sort of namespace for the objects in it, which allows you to use simple names without fear of collisions -- coming from a C background you have surely seen libraries with long, ugly function names to avoid colliding with anybody else.
Also, modules themselves are also objects. When a module is imported in more than one place in a python program, each actually gets the same reference. That way, changing foo.a changes it for everybody, not just the local module. This is in contrast to C where including a header is basically a copy+paste operation into the source file (obviously you can still share variables, but the mechanism is a bit different).
As mentioned, you can say from foo import * or better from foo import a, but understand that the underlying behavior is actually different, because you are taking a and binding it to your local module.
If you use something often, you can always use the from syntax to import it directly, or you can rename the module to something shorter, for example
import itertools as it
When you do import foo, a new module is created inside the current namespace named foo.
So, to use anything inside foo; you have to address it via the module.
However, if you use from from foo import something, you don't have use to prepend the module name, since it will load something from the module and assign to it the name something. (Not a recommended practice)
import importlib
# works like C's #include, you always call it with include(<path>, __name__)
def include(file, module_name):
spec = importlib.util.spec_from_file_location(module_name, file)
mod = importlib.util.module_from_spec(spec)
# spec.loader.exec_module(mod)
o = spec.loader.get_code(module_name)
exec(o, globals())
For example:
#### file a.py ####
a = 1
#### file b.py ####
b = 2
if __name__ == "__main__":
print("Hi, this is b.py")
#### file main.py ####
# assuming you have `include` in scope
include("a.py", __name__)
print(a)
include("b.py", __name__)
print(b)
the output will be:
1
Hi, this is b.py
2
Dir structure:
main.py
my_modules/
module1.py
module2.py
module1.py:
class fooBar():
....
class pew_pew_FooBarr()
....
...
How can I add all classes from module* to main without prefixes (i.e. to use them like foo = fooBar(), not foo = my_modules.module1.fooBar()).
An obvious decision is to write in main.py something like this:
from my_modules.module1 import *
from my_modules.module2 import *
from my_modules.module3 import *
...
But I don't want to change main.py when I create new moduleN. Is there solution for that?
I do know it's not a good idea to import classes like this, but I'm still curious about that.
UPD: This question differs from this one Loading all modules in a folder in Python, because my problem is to load modules without namespaces.
In the my_modules folder, add a __init__.py file to make it a proper package. In that file, you can inject the globals of each of those modules in the global scope of the __init__.py file, which makes them available as your module is imported (after you've also added the name of the global to the __all__ variable):
__all__ = []
import pkgutil
import inspect
for loader, name, is_pkg in pkgutil.walk_packages(__path__):
module = loader.find_module(name).load_module(name)
for name, value in inspect.getmembers(module):
if name.startswith('__'):
continue
globals()[name] = value
__all__.append(name)
Now, instead of doing:
from my_modules.class1 import Stuff
You can just do:
from my_modules import Stuff
Or to import everything into the global scope, which seems to be what you want to do:
from my_modules import *
The problem with this approach is classes overwrite one another, so if two modules provide Foo, you'll only be able to use the one that was imported last.