I get an ImportError because of wrong path when doing multiple imports in Python. For example with these files:
folder1
first.py
folder2
second.py
folder3
third.py
first.py imports the class in second.py
second.py imports the class in third.py
There's no problem with python ./folder2/second.py but python first.py gives me an ImportError
Traceback (most recent call last):
File "first.py", line 1, in <module>
from folder2.second import SecondClass
File "home/test/folder2/second.py", line 1, in <module>
from folder3.third import ThirdClass
ImportError: No module named 'folder3'
It seems first.py executes the import folder3.third.ThirdClass of second.py when it tries to import SecondClass from second.py, and because folder3 isn't in its path it raises an error.
If I change the import path in second.py from from folder3.third import ThirdClass to from folder2.folder3.third import ThirdClass, first.py works, but obviously second.py doesn't work anymore.
Is there a way to solve this?
Edit: Adding
import sys
sys.path.append("./folder1")
in first.py solves the problem.
It would have worked as is with Python 2, in Python 3 you need to use relative imports withing the package, on other word your import line the second.py should read:
from .folder3.third import ThirdClass
Now a slight complication. I am not sure where your package boundary is, if your first.py is still part of it or is the main module (meant to be passed directly to the python interpreter). Relative imports are based on module name (__main__ for the main one), main module must use absolute imports.
This has two implications for you. What do import statements in first.py look like. And also another form your import in second.py could look like, because based on where you package start, apart from the above give example, you could also say (first.py is your main module):
from folder2.folder3.third import ThirdClass
or (folder1 and first.py are already part of the package and imported from a main module):
from folder1.folder2.folder3.third import ThirdClass
All the fun details are as always in the corresponding PEP-328.
Related
Setup
test/
main.py
pkg/
a.py
__init__.py
main.py contains:
import pkg
pkg.a
__init__.py contains:
from . import a
main.py can be run without errors.
Question
Changing the content of __init__.py to
import a
gives the following error when running main.py:
Traceback (most recent call last):
File "C:/Users/me/PycharmProjects/test/main.py", line 1, in <module>
import pkg
File "C:\Users\me\PycharmProjects\test\pkg\__init__.py", line 1, in <module>
import a
ModuleNotFoundError: No module named 'a'
Interestingly, __init__.py can be executed directly with python __init__.py without errors.
What's going on?
When you run a python script, it's parent folder is added to sys.path
run main.py: sys.path[0] = '../test'
run init.py: sys.path[0] = '../test/pkg'
Your case: You try to "absolute-like" import a in __init__.py but the parent folder of a.py - which is '../test/pkg' - is not in the sys.path when you run main.py. This is why you get an error. However, your absolute import is incomplete as it should always start at the top level folder, e.g.
from test.pkg import a
Final answer to your question: You don't have to use relative imports!
See: PEP-8: Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path).
And keep in mind that relative imports don't work in a top-level-script when __name__ = "__main__", but from imported modules only.
You can learn more about absolute and relative imports here:
Absolute vs. explicit relative import of Python module
https://realpython.com/absolute-vs-relative-python-imports/
I suppose you are using Pycharm? Then that's one of the confusion cause.
For example, let's say your directory looks like this
project1
p1.py
test/
__init__.py
main.py
pkg/
a.py
__init__.py
If you run (F10) the main.py your default working directory will be project1/test, which does not contain the a.py so import a will not find anything.
But if you run (F10) the pkg/__init__.py your working directory will be project1/test/pkg which has the a.py, and it works like what you tested.
So in these situation, if you use from . import a it will look for the directory that file is, project1/test/pkg in this case, which will always work regardless your working directory.
I have a very simple test Python 3 project with the following file structure:
test/a.py
test/b.py
test/__init__.py
Everywhere I read, people say that in a.py I should import b.py using an absolute path:
from test.b import *
However, when I try I get the following error:
Traceback (most recent call last):
File "a.py", line 1, in <module>
from test.b import *
ModuleNotFoundError: No module named 'test.b'
I understand that I can import b.py using from b import *, however this is not what people recommend. They all recommend from test.b import *. But I can't get even this simple example to work.
As Martijn said in the comment, it depends on how you call a.py.
If you call it directly from within the directory by typing python a.py you will get the error above.
However, if you call it like that: python -m test.a while being one directory above the test directory, your import will work just fine.
The common directory structure is like this:
test/a.py
test/b.py
test/__init__.py
run.py
The main code should be put into run.py. When you want to import a.py in run.py, just write from test.a import * or something like that. And if you need to import b.py in a.py, do as you have been told from test.b import *. Then, run run.py would get the correct result.
Python imports. Again...
I have this file structure:
[test]
start.py (from scripts import main)
[scripts]
__init__.py (empty)
main.py (from . import install)
install.py (from scripts import main # or # from . import main)
I get import error:
vic#wic:~/projects/test$ python3 start.py
Traceback (most recent call last):
File "start.py", line 2, in <module>
from scripts import main
File "/home/vic/projects/test/scripts/main.py", line 1, in <module>
from . import install
File "/home/vic/projects/test/scripts/install.py", line 1, in <module>
from scripts import main
ImportError: cannot import name main
vic#wic:~/projects/test$
I don't understand: first time from scripts import main worked (by "worked" I mean it didn't fail with ImportError), and second time the same code gives ImportError: cannot import name main
What is going on?
UPDATE:
My question is not about circular imports. I am confused by the fact that exactly the same code from scripts import main first time works ok and then second time fails.
I guess there is some internal import mechanism which i don't understand.
First time a statement imports a module, second time exactly the same code tries to import a name from a module. How this works?
You created a circular import. You can't do that. Avoid importing main from install.
What is happening is that a module, while being imported, is incomplete until the whole top level has been executed. If during that execution it imports another module that (directly or indirectly) tries to import the original module again, this will fail. The original module was not yet done importing yet.
In other words, you are creating a circular graph:
start -> scripts.main -> scripts.install
^ |
| |
----------------
You need to re-arrange your code to not need to import from main from within your scripts package.
See What are the “best practices” for using import in a module? for some tips on how to handle this.
main imports install and install imports main:
from scripts import main in start.py leads to
from . import install in main.py leads to
from scripts import main in install.py -- circular import
To solve the circular import either combine main and install into a single module or create the 3rd module that both modules can import.
This should work:
start.py:
from scripts import main
scripts/main.py:
import install
scripts/install.py:
import main
This is a patch I use to override the default behavior:
import sys
import builtins
import importlib
def _import(name, globals=None, locals=None, fromlist=None, level=0, __import__=__import__):
"""A hack to to make names of imported modules to be available in the parent package before
they are fully imported. If a module is present in sys.modules event if it's not fully
imported, it should not be a problem.
I.e. ``from package import module1, module2`` will create variables ``package.module1`` and
``package.module2`` at start of the import, not when it's finished.
This helps in some situations with circular import.
"""
module = __import__(name, globals, locals, fromlist, level)
for attr in fromlist or ():
sub_module = sys.modules.get(module.__name__ + '.' + attr)
if sub_module: # is this a module?
# if subpackage is already being imported, even if not finished,
# inject its name into the parent package
setattr(module, attr, sub_module)
return module
builtins.__import__ = _import
Here is a project to test this: https://github.com/warvariuc/python_import_improver
I have the following directory structure:
test1/
test1/a.py
test1/test2/b.py
b.py needs to import a class in a.py. So I can add the following line to b.py before importing a.
sys.path.append(os.path.dirname(sys.argv[0]) + "/..")
This works and I can invoke b.py from any directory and it is able to import a.
But this fails when I write a script in another directory to invoke this file using execfile().
I tried relative imports but I get a "Attempted Relative Import in Non-Package error"
from ..a import someclass as cls
I have __init__.py in both test1, test2
Does someone have an idea how to make it work?
Is PYTHONPATH the way to go?
The problem is that execfile will evaluate the file you are calling as pure python code. Every relative import statement inside of b.py (and any package module imported by it) will have to remain true to your calling script.
One solution is to not use any relative import paths within the package. Make sure test1 package is on your PYTHONPATH as well.
b.py
from test1 import a
With test1 in your PYTHONPATH, the import of a should success in your execfile
>>> import sys
>>> sys.path.append('/path/to/parent/of_test1')
>>> execfile('/path/to/parent/of_test1/test1/test2/b.py')
This is my folder structure:
src/
__init__py
Lowlevel/
__init__.py
ModuleToCheck.Py
Test/
__init__.py
ModuleToCheck_test.py
(__init__.py are empty files)
Now I want to import ModuleToCheck.py in ModuleToCheck_test.py
How am I able to do this without appending anything to sys.path?
Update:
from ..Lowlevel import ModuleToCheck leads to:
src$ python Test/ModuleToCheck_test.py
Traceback (most recent call last):
File "Test/ModuleToCheck_test.py", line 6, in <module>
from ..Lowlevel import ModuleToCheck
ValueError: Attempted relative import in non-package
The following is from http://docs.python.org/tutorial/modules.html#intra-package-references
Note that both explicit and implicit
relative imports are based on the name
of the current module. Since the name
of the main module is always
"__main__", modules intended for use
as the main module of a Python
application should always use absolute
imports.
You're running your module ModuleToCheck_test.py as the main module, hence the exception.
One solution is to create a test.py module in your src directory containing the following:
import Test.ModuleToCheck_test
You can then run that module using python test.py