Let's say i have a really long script.(1000+ lines long, in my case) so i split it into sepperate files:
main.py #the file i execute
foo1.py #a file my main.py imports
foo2.py #a file imported by foo1.py
(note: main.py imports several files, not just the one)
Foo1.py holds Tkinter, and things related to it, while Foo2.py holds a huge object class with functions related to said class.
My problem is as follows:
Foo1 imports Foo2
Foo2 runs a function that calls another function from Foo1
Foo2 raises a 'global name ' is not defined' error
And also i can't import the function into Foo2, because Foo1 already has it and that raises an import error.
When two modules import each other there are a few things you need to keep in mind so everything is defined before it is needed.
First lets consider the mechanic of importing:
when a module is imported for the first time an entry is added to sys.modules and the defining file starts executing (pausing the execution of the import-er)
subsequent imports will simply use the entry in sys.modules - whether or not the file finished executing
So lets say module A is loaded first, imports module B which then imports module A, when this happens execution is as follows:
A is imported from something else for first time, A is added to sys.modules
A is executed up to importing B
B is added to sys.modules
B is executed:
when it imports A the partially loaded module is used from sys.modules
B runs completely before resuming
A resumes executing, having access to the complete module B
*1 so from A import x can only work if x is defined in A before import B, just using import A will give you the module object which is updated as the file executes, so while it may not have all the definitions right after import it will when the file has a chance to finish executing.
So the simplest way of solving this is to first not rely on the import for the execution of the module - meaning all the uses of the circular import is within def blocks that are not called from the module level of execution.
Related
I've read this post about circular imports in Python. It describes the following scenario and argues that this raises an error when run:
# module1
import module2
def function1():
module2.function2()
def function3():
print('Goodbye, World!')
# module2
import module1
def function2():
print('Hello, World!')
module1.function3()
# __init__.py
import module1
module1.function1()
But when I run this (Python 3.95), it runs perfectly fine. The post is pretty old and it doesn't specify the Python version it uses. Maybe there was some change in latter Pythons that support this?
Here's a simplified sequence of events that happen in the code in Python 3:
__init__.py starts running
An empty __main__ module is added to sys.modules
import module1 starts loading module1.py
An empty module1 module is added to sys.modules
import module2 starts loading module2.py
An empty module2 module is added to sys.modules
module2.function2 is created and added to module2.__dict__
The fact that function2 references names in module1 does not affect the creation of the function object in any way
module2 is fully loaded and execution returns to module1
module1.function1 and module1.function3 are created and added to module1.__dict__
Again, it does not matter what names the functions reference because they are not being called. AttributeError and NameError can be raised at runtime if necessary.
module1 is fully loaded and execution returns to __main__
module1.function runs successfully, since all the names it references are resolvable.
As you can see, there are no circular import issues in this particular sequence of imports because module1 and module2 do not attempt to call each other's functions. The current import system allows both modules to load before the functions are called.
The post you mention is from 2017, and must be using a version of python from before 3.0. A hint is found in the link in the following quote, which links to the python-2.x docs:
This approach doesn't contradict Python syntax, as the Python documentation says: "It is customary but not required to place all import statements at the beginning of a module (or script, for that matter)".
The paragraph after that is a bit misleading by the way:
The Python documentation also says that it is advisable to use import X, instead of other statements, such as from module import *, or from module import a,b,c.
While star imports are certainly discouraged, specific-name imports of the form from module import a,b,c are generally very much encouraged with few exceptions.
I have two modules as follows:
Module A - moda.py
import modb
x = None
def printx():
global x
print(x)
def main():
global x
x = 42
printx()
modb.printx()
printx()
if __name__ == '__main__':
main()
Module B - modb.py
import moda
def printx():
moda.printx()
print('modb imported')
When I run python moda.py, the output I get is:
modb imported
42
None
42
I don't understand why the second print (coming from modb.printx()) is None. I thought python modules behaved as singletons. What am I missing?
Can someone please explain why the module imported in modb is not same as the original module moda?
When an import statement is encountered, the interpreter looks for a corresponding key in sys.modules. If a key is found, it is bound to the name you requested. If not, a new empty module object is createdplaced in sys.modules, , and then populated. The reason for doing it like that is exactly to avoid infinite loops with circular imports.
When you run a module, it is imported under the name __main__. You ca
Here is the sequence of events when you run moda as a script:
Start of load of moda.py as sys.modules['__main__']. At this point, this is just an empty namespace
import modb encountered in moda.py. New empty namespace created for sys.modules['modb'].
import moda encountered in modb.py. New empty namespace created for sys.modules['moda']. Notice that this is not the same object as sys.modules['__main__'] in step 1.
import modb encountered in moda.py. Since sys.modules['modb'] exists, it is bound to that name in moda
Since moda.py is currently being loaded under the name moda, it finishes populating its namespace without running the import guard.
modb.py finishes populating its namespace (from step 2.) and runs print('modb loaded').
__main__ defined in moda.py finishes populating its namespace (from step 1.) and runs the import guard.
Hopefully this helps you visualize what happens. You have three modules, not two, that were loaded, because moda is loaded under two different names, and as two entirely different module objects.
The import guard in __main__ calls __main__.main, which does the following:
Set __main__.x = 42 (moda.x is still None)
__main__.printx prints __main__.x, which is 42
modb.printx calls moda.printx, which prints moda.x, which is None.
__main__.printx prints __main__.x again, which is still 42.
Consider the following:
a.py
foo = 1
b.py
bar = 2
c.py
import a
kik = 3
d.py
import a
import c
def main():
import b
main()
main()
How many times is a.py loaded?
How many times is b.py loaded?
More generally, I would like to know how is Python handling imported files and functions/variables?
Both a and b are loaded once. When you import a module, its content is cached so when you load the same module again, you're not calling upon the original script for the import, done using a "finder":
https://www.python.org/dev/peps/pep-0451/#finder
https://docs.python.org/3/library/importlib.html#importlib.abc.MetaPathFinder
This works across modules so if you had a d.py of which import b, it will bind to the same cache as an import within c.py.
Some interesting builtin modules can help understand what happens during an import:
https://docs.python.org/3/reference/import.html#importsystem
When a module is first imported, Python searches for the module and if found, it creates a module object 1, initializing it.
Notably here the first import, all imports after follow the __import__. Internal caches of finders are stored at sys.meta_path.
https://docs.python.org/3/library/functions.html#import
You can leverage the import system to invalidate those caches for example:
https://docs.python.org/3/library/importlib.html#importlib.import_module
If you are dynamically importing a module that was created since the interpreter began execution (e.g., created a Python source file), you may need to call invalidate_caches() in order for the new module to be noticed by the import system.
The imp (and importlib py3.4+) allows the recompilation of a module after import:
import imp
import a
imp.reload(a)
https://docs.python.org/3/library/importlib.html#importlib.reload
Python module’s code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module’s dictionary by reusing the loader which originally loaded the module.
https://docs.python.org/3/library/imp.html
To illustrate the issue I am having, please consider the following. I have two .py files, one named main.py and the other named mymodule.py. They are both in the same directory.
The contents of main.py:
from mymodule import myfunction
myfunction()
The contents of mymodule.py:
def myfunction():
for number in range(0,10):
print(number)
print("Hi")
I was under the impression that importing a function would only import that function. However, when I run main.py, this is what I get:
Hi
0
1
2
3
4
5
6
7
8
9
Why is print("Hi") being called? It isn't part of the function I imported!
I was under the impression that importing a function would only import that function.
It seems there's an incorrect assumption about what a from-import actually does.
The first time a module is imported, an import statement will execute the entire module, including print calls made at the global scope (docs). This is true regardless of whether the mymodule was first imported by using a statement like import mymodule or by using a statement like from mymodule import myfunction.
Subsequent imports of the same module will re-use an existing module cached in sys.modules, which may be how you arrived at the misunderstanding that the entire module is not executed.
There is a common pattern to avoid global level code being executed by a module import. Often you will find code which is not intended to be executed at import time located inside a conditional, like this:
def myfunction():
for number in range(0,10):
print(number)
if __name__ == "__main__":
print("Hi")
In order to import something from the module Python needs to load this module first. At that moment all the code at module-level is executed.
According to the docs:
A module can contain executable statements as well as function
definitions. These statements are intended to initialize the module.
They are executed only the first time the module name is encountered
in an import statement.
this question seems to be a duplicate of this one.
In short : all the code of a python file is called when importing the module. What is neither a function nor a class is usually put in a main function called here:
if __name__ == "__main__":
# stuff only to run when not called via 'import' here
main()
Please consider closing this thread.
When writing python modules, is there a way to prevent it being imported twice by the client codes? Just like the c/c++ header files do:
#ifndef XXX
#define XXX
...
#endif
Thanks very much!
Python modules aren't imported multiple times. Just running import two times will not reload the module. If you want it to be reloaded, you have to use the reload statement. Here's a demo
foo.py is a module with the single line
print("I am being imported")
And here is a screen transcript of multiple import attempts.
>>> import foo
Hello, I am being imported
>>> import foo # Will not print the statement
>>> reload(foo) # Will print it again
Hello, I am being imported
Imports are cached, and only run once. Additional imports only cost the lookup time in sys.modules.
As specified in other answers, Python generally doesn't reload a module when encountering a second import statement for it. Instead, it returns its cached version from sys.modules without executing any of its code.
However there are several pitfalls worth noting:
Importing the main module as an ordinary module effectively creates two instances of the same module under different names.
This occurs because during program startup the main module is set up with the name __main__. Thus, when importing it as an ordinary module, Python doesn't detect it in sys.modules and imports it again, but with its proper name the second time around.
Consider the file /tmp/a.py with the following content:
# /tmp/a.py
import sys
print "%s executing as %s, recognized as %s in sys.modules" % (__file__, __name__, sys.modules[__name__])
import b
Another file /tmp/b.py has a single import statement for a.py (import a).
Executing /tmp/a.py results in the following output:
root#machine:/tmp$ python a.py
a.py executing as __main__, recognized as <module '__main__' from 'a.py'> in sys.modules
/tmp/a.pyc executing as a, recognized as <module 'a' from '/tmp/a.pyc'> in sys.modules
Therefore, it is best to keep the main module fairly minimal and export most of its functionality to an external module, as advised here.
This answer specifies two more possible scenarios:
Slightly different import statements utilizing different entries in sys.path leading to the same module.
Attempting another import of a module after a previous one failed halfway through.