I've read this post about circular imports in Python. It describes the following scenario and argues that this raises an error when run:
# module1
import module2
def function1():
module2.function2()
def function3():
print('Goodbye, World!')
# module2
import module1
def function2():
print('Hello, World!')
module1.function3()
# __init__.py
import module1
module1.function1()
But when I run this (Python 3.95), it runs perfectly fine. The post is pretty old and it doesn't specify the Python version it uses. Maybe there was some change in latter Pythons that support this?
Here's a simplified sequence of events that happen in the code in Python 3:
__init__.py starts running
An empty __main__ module is added to sys.modules
import module1 starts loading module1.py
An empty module1 module is added to sys.modules
import module2 starts loading module2.py
An empty module2 module is added to sys.modules
module2.function2 is created and added to module2.__dict__
The fact that function2 references names in module1 does not affect the creation of the function object in any way
module2 is fully loaded and execution returns to module1
module1.function1 and module1.function3 are created and added to module1.__dict__
Again, it does not matter what names the functions reference because they are not being called. AttributeError and NameError can be raised at runtime if necessary.
module1 is fully loaded and execution returns to __main__
module1.function runs successfully, since all the names it references are resolvable.
As you can see, there are no circular import issues in this particular sequence of imports because module1 and module2 do not attempt to call each other's functions. The current import system allows both modules to load before the functions are called.
The post you mention is from 2017, and must be using a version of python from before 3.0. A hint is found in the link in the following quote, which links to the python-2.x docs:
This approach doesn't contradict Python syntax, as the Python documentation says: "It is customary but not required to place all import statements at the beginning of a module (or script, for that matter)".
The paragraph after that is a bit misleading by the way:
The Python documentation also says that it is advisable to use import X, instead of other statements, such as from module import *, or from module import a,b,c.
While star imports are certainly discouraged, specific-name imports of the form from module import a,b,c are generally very much encouraged with few exceptions.
I currently have a module I created that has a number of functions.
It's getting quite large so I figured I should make it into a package and split the functions up to make it more manageable.
I'm just testing out how this all works before I do this for real so apologies if it seems a bit tenuous.
I've created a folder called pack_test and in it I have:
__init__.py
foo.py
bar.py
__init__.py contains:
__all__ = ['foo', 'bar']
from . import *
import subprocess
from os import environ
In the console I can write import pack_test as pt and this is fine, no errors.
pt. and two tabs shows me that I can see pt.bar, pt.environ, pt.foo and pt.subprocess in there.
All good so far.
If I want to reference subprocess or environ in foo.py or bar.py how do I do it in there?
If in bar.py I have a function which just does return subprocess.call('ls') it errors saying NameError: name 'subprocess' is not defined. There must be something I'm missing which enables me to reference subprocess from the level above? Presumably, once I can get the syntax from that I can also just call environ in a similar way?
The alternative as I could see it would be to have import subprocess in both foo.py and bar.py but then this seems a bit odd to me to have it appear across multiple files when I could have it the once at a higher level, particularly if I went on to have a large number of files rather than just 2 in this example.
TL;DR:
__init__.py :
import foo
import bar
__all__ = ["foo", "bar"]
foo.py:
import subprocess
from os import environ
# your code here
bar.py
import subprocess
from os import environ
# your code here
There must be something I'm missing which enables me to reference subprocess from the level above?
Nope, this is the expected behaviour.
import loads a module (if it isn't already), caches it in sys.modules (idem), and bind the imported names in the current namespace. Each Python module has (or "is") it's own namespace (there's no real "global" namespace). IOW, you have to import what you need in each module, ie if foo.py needs subprocess, it must explicitely import it.
This can seem a bit tedious at first but in the long run it really helps wrt/ maintainability - you just have to read the imports at the top of your module (pep 08: always put all imports at the beginning of the module) to know where a name comes from.
Also you should not use star imports (aka wild card imports aka from xxx import *) anywhere else than in your python shell (and even then...) - it's a maintainance time bomb. Not only because you don't know where each name comes from, but also because it's a sure way to rebind an already import name. Imagine that your foo module defines function "func". Somewhere you have "from foo import *; from bar import *", then later in the code a call to func. Now someone edits bar.py and adds a (distinct) "func" function, and suddenly you call fails, because you're not calling the expected "func". Now enjoy debugging this... And real-life examples are usually a bit more complex than this.
So if you fancy your mental sanity, don't be lazy, don't try to be smart either, just do the simple obvious thing: explicitely import the names you're interested in at the top of your modules.
(been here, done that etc)
You could create modules.py containing
import subprocess
import os
Then in foo.py or any of your files just have.
from modules import *
Your import statements in your files are then static and just update modules.py when you want to add an additional module accessible to them all.
Let's say I have file main.py :
import math
import mymodule.py
print(math.ceil(5/3))
and then mymodule.py :
print(math.ceil(10/3))
mymodule.py gives an error that math is not defined, even though its parent module has it imported.
Considering both main.py and mymodule.py need to use the math lib, do I need to import it twice? It just seems non-optimal. What's the most pythonic way to solve this issue?
I know it's a dumb example, but I'm trying to fragment a code I made into several modules for organization, and this issue appeared multiple times in several levels
mymodule.py is parent for main.py since you are importing mymodule within main.
You need to import math within mymodule so that it gets inherited in main.
Then there won't be a need to import within main.
mymodule.py
import math
main.py
import mymodule
print mymodule.math.pow(10,2)
Result:
>>>
100.0
>>>
This is really very basic. If you have something in a separate file, like mymodule.py, then you can import that function in any python file easily in the same directory.
two files:
mymodule.py:
import math
def aFunc():
return math.ceil(10/3)
# We could also just use this file as a standalone
if __name__ == "__main__":
print(aFunc())
main.py:
import mymodule
print(mymodule.aFunc())
You could also specifically call out the function you want to import.
main.py (alternative):
from mymodule import aFunc
print(aFunc())
I have some code spread across multiple files that try to import from each other, as follows:
main.py:
from entity import Ent
entity.py:
from physics import Physics
class Ent:
...
physics.py:
from entity import Ent
class Physics:
...
I then run from main.py and I get the following error:
Traceback (most recent call last):
File "main.py", line 2, in <module>
from entity import Ent
File ".../entity.py", line 5, in <module>
from physics import Physics
File ".../physics.py", line 2, in <module>
from entity import Ent
ImportError: cannot import name Ent
I'm assume the error is due to importing entity twice - once in main.py and later in physics.py - but how can I work around the problem?
See also What happens when using mutual or circular (cyclic) imports in Python? for a general overview of what is allowed and what causes a problem WRT circular imports. See Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down? for technical details on why and how the problem occurs.
You have circular dependent imports. physics.py is imported from entity before class Ent is defined and physics tries to import entity that is already initializing. Remove the dependency to physics from entity module.
While you should definitely avoid circular dependencies, you can defer imports in python.
for example:
import SomeModule
def someFunction(arg):
from some.dependency import DependentClass
this ( at least in some instances ) will circumvent the error.
This is a circular dependency. It can be solved without any structural modifications to the code. The problem occurs because in vector you demand that entity be made available for use immediately, and vice versa. The reason for this problem is that you asking to access the contents of the module before it is ready -- by using from x import y. This is essentially the same as
import x
y = x.y
del x
Python is able to detect circular dependencies and prevent the infinite loop of imports. Essentially all that happens is that an empty placeholder is created for the module (ie. it has no content). Once the circularly dependent modules are compiled it updates the imported module. This is works something like this.
a = module() # import a
# rest of module
a.update_contents(real_a)
For python to be able to work with circular dependencies you must use import x style only.
import x
class cls:
def __init__(self):
self.y = x.y
Since you are no longer referring to the contents of the module at the top level, python can compile the module without actually having to access the contents of the circular dependency. By top level I mean lines that will be executed during compilation as opposed to the contents of functions (eg. y = x.y). Static or class variables accessing the module contents will also cause problems.
In my case, I was working in a Jupyter notebook and this was happening due the import already being cached from when I had defined the class/function inside my working file.
I restarted my Jupyter kernel and the error disappeared.
To make logic clear is very important. This problem appear, because the reference become a dead loop.
If you don't want to change the logic, you can put the some import statement which caused ImportError to the other position of file, for example the end.
a.py
from test.b import b2
def a1():
print('a1')
b2()
b.py
from test.a import a1
def b1():
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
You will get Import Error: ImportError: cannot import name 'a1'
But if we change the position of from test.b import b2 in A like below:
a.py
def a1():
print('a1')
b2()
from test.b import b2
And the we can get what we want:
b1
a1
b2
This is a circular dependency.
we can solve this problem by using import module or class or function where we needed.
if we use this approach, we can fix circular dependency
A.py
from B import b2
def a1():
print('a1')
b2()
B.py
def b1():
from A import a1
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
I just got this error too, for a different reason...
from my_sub_module import my_function
The main script had Windows line endings. my_sub_module had UNIX line endings. Changing them to be the same fixed the problem. They also need to have the same character encoding.
As already mentioned, this is caused by a circular dependency. What has not been mentioned is that when you're using Python typing module and you import a class only to be used to annotate Types, you can use Forward references:
When a type hint contains names that have not been defined yet, that
definition may be expressed as a string literal, to be resolved later.
and remove the dependency (the import), e.g. instead of
from my_module import Tree
def func(arg: Tree):
# code
do:
def func(arg: 'Tree'):
# code
(note the removed import statement)
The problem is clear: circular dependency between names in entity and physics modules.
Regardless of importing the whole module or just a class, the names must be loaded .
Watch this example:
# a.py
import b
def foo():
pass
b.bar()
# b.py
import a
def bar():
pass
a.foo()
This will be compiled into:
# a.py
# import b
# b.py
# import a # ignored, already importing
def bar():
pass
a.foo()
# name a.foo is not defined!!!
# import b done!
def foo():
pass
b.bar()
# done!
With one slight change we can solve this:
# a.py
def foo():
pass
import b
b.bar()
# b.py
def bar():
pass
import a
a.foo()
This will be compiled into:
# a.py
def foo():
pass
# import b
# b.py
def bar():
pass
# import a # ignored, already importing
a.foo()
# import b done!
b.bar()
# done!
Try this solution: rename your working python script
You should not name your current python script with the name of some other module you import, since you will get that error.
Example:
you are working in medicaltorch.py
in that script, you have: from medicaltorch import X where medicaltorch is supposed to be a separate installed module
This will fail with the ImportError since 2 things refer to medicaltorch
So, just rename your working python script in 1.
If you are importing file1.py from file2.py and used this:
if __name__ == '__main__':
# etc
Variables below that in file1.py cannot be imported to file2.py because __name__ does not equal __main__!
If you want to import something from file1.py to file2.py, you need to use this in file1.py:
if __name__ == 'file1':
# etc
In case of doubt, make an assert statement to determine if __name__=='__main__'
Don't see this one here yet - this is incredibly stupid, but make sure you're importing the correct variable/function.
I was getting this error
ImportError: cannot import name IMPLICIT_WAIT
because my variable was actually IMPLICIT_TIMEOUT.
when I changed my import to use the correct name, I no longer got the error 🤦♂️
One way to track import error is step by step trying to run python on each of imported files to track down bad one.
you get something like:
python ./main.py
ImportError: cannot import name A
then you launch:
python ./modules/a.py
ImportError: cannot import name B
then you launch:
python ./modules/b.py
ImportError: cannot import name C (some NON-Existing module or some other error)
Also not directly relevant to the OP, but failing to restart a PyCharm Python console, after adding a new object to a module, is also a great way to get a very confusing ImportError: Cannot import name ...
The confusing part is that PyCharm will autocomplete the import in the console, but the import then fails.
Not specifically for this asker, but this same error will show if the class name in your import doesn't match the definition in the file you're importing from.
In my case, simply missed filename:
from A.B.C import func_a (x)
from A.B.C.D import func_a (O)
where D is file.
I met this error too, but my case is less common, and it does throw this error too.
My case is that I encounter this error in jupyter notebook; I write from M import c where M is a python file and c is a class in M.py, the reason for the error is because c is just created a few minutes ago, but my jupyter notebook has been running for a long time, so I just need to restart the jupyter notebook and let it reload M.py.
When writing python modules, is there a way to prevent it being imported twice by the client codes? Just like the c/c++ header files do:
#ifndef XXX
#define XXX
...
#endif
Thanks very much!
Python modules aren't imported multiple times. Just running import two times will not reload the module. If you want it to be reloaded, you have to use the reload statement. Here's a demo
foo.py is a module with the single line
print("I am being imported")
And here is a screen transcript of multiple import attempts.
>>> import foo
Hello, I am being imported
>>> import foo # Will not print the statement
>>> reload(foo) # Will print it again
Hello, I am being imported
Imports are cached, and only run once. Additional imports only cost the lookup time in sys.modules.
As specified in other answers, Python generally doesn't reload a module when encountering a second import statement for it. Instead, it returns its cached version from sys.modules without executing any of its code.
However there are several pitfalls worth noting:
Importing the main module as an ordinary module effectively creates two instances of the same module under different names.
This occurs because during program startup the main module is set up with the name __main__. Thus, when importing it as an ordinary module, Python doesn't detect it in sys.modules and imports it again, but with its proper name the second time around.
Consider the file /tmp/a.py with the following content:
# /tmp/a.py
import sys
print "%s executing as %s, recognized as %s in sys.modules" % (__file__, __name__, sys.modules[__name__])
import b
Another file /tmp/b.py has a single import statement for a.py (import a).
Executing /tmp/a.py results in the following output:
root#machine:/tmp$ python a.py
a.py executing as __main__, recognized as <module '__main__' from 'a.py'> in sys.modules
/tmp/a.pyc executing as a, recognized as <module 'a' from '/tmp/a.pyc'> in sys.modules
Therefore, it is best to keep the main module fairly minimal and export most of its functionality to an external module, as advised here.
This answer specifies two more possible scenarios:
Slightly different import statements utilizing different entries in sys.path leading to the same module.
Attempting another import of a module after a previous one failed halfway through.