I am having the age old problem of Module not found while importing files from different folder, kindly help me.
My project directory has the following things:
knowledge_generators --> __init__.py
knowledge_generator.py
absorb.py
In __init__.py I have the following content:
from knowledge_generator import *
And absorb.py has:
from knowledge_generators import *
On running absorb.py I get the following error:
File "D:/some/path/project/absorb.py", line 2, in <module>
from knowledge_generators import *
File "D:\some\path\project\knowledge_generators\__init__.py", line 1, in <module>
from knowledge_generator import *
ModuleNotFoundError: No module named 'knowledge_generator'
Also, on running __init__.py everything's working fine(i.e no ModuleNotFoundError). Kindly help me decipher the problem.
I suspect you need to use a relative import:
In __init__.py:
from .knowledge_generator import *
Working solution:
Just add your project root directory to environment variable: PYTHONPATH.
so for the below project structure, just add Rootdir path(For e.g: add E:\Projects\Rootdir) in PYTHONPATH.
Rootdir
└── pkg2
├── b.py
├── c.py
└── pkg2
├── b.py
├── c.py
Related
I'm working on a python 3.8.5 project which has this folder structure:
project/
├── __init__.py
|── foo.py
|── bar.py
├── utils/
│ ├── __init__py
│ └── configurator.py
└── server/
├── __init__.py
├── models.py
In server/models.py I need to use a class Configurator declared inside utils/configurator.py:
# server/models.py
from utils.configurator import Configurator
# some code here ...
Unfortunately I get this error: ModuleNotFoundError: No module named 'utils'. I dont't understand how it cannot find the utils module since I've properly set the init.py file inside that folder.
I've also tried with relative import:
# server/models.py
from ..utils.configurator import Configurator
# some code here ...
this time getting the following error: ImportError: attempted relative import with no known parent package
Finally, I've attempted with an absolute import, with no results again:
# server/models.py
from projects.utils.configurator import Configurator
# some code here ...
The error this time is: ModuleNotFoundError: No module named 'projects'
Based on this previous question it seems correct to me, I can't understand what I'm messing with.
My code editor is Visual Studio Code and when I type from utils. ... it gives the right suggestion, although I don't think this piece of information could be of any utility.
My project structure is the following:
.
└── project name
├── project name
│ ├── __init__.py
│ ├── module.py
│
├── PACKAGE_A
│ ├── __init__.py
│ ├── PACKAGE_A.py
│ ├── module_a.py
│
In PACKAGE_A.py
from module_a import Some_Class
a = Some_Class()
class Another_Class:
# class code here
In module.py
"""
Notes
-----
https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
"""
# Standard library imports
import os, sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
# Local application imports
from PACKAGE_A.PACKAGE_A import Another_Class
from PACKAGE_A.module_a import some_function
While module_a.py and PACKAGE_A.py run without problem, running module.py fails with:
Traceback (most recent call last):
File "path\to\project name\project name\module.py", line 12, in <module>
from PACKAGE_A.PACKAGE_A import Another_Class
File "path\to\project name\PACKAGE_A\PACKAGE_A.py", line 1, in <module>
from module_a import Some_Class
ModuleNotFoundError: No module named 'module_a'
What am I doing wrong here?
You need to change your import statement in PACKAGE_A.py from:
from module_a import Some_Class
to:
from PACKAGE_A.module_a import Some_Class
The reason is that you are adding the path\to\project name\ to sys.path, but you have no module_a.py in path\to\project name\, and path\to\project name\PACKAGE_A (where module_a.py resides) is not in sys.path.
As for why you succeed in running everything in PACKAGE_A, it's because Python adds the directory containing the script you are running to the list (as explained by gaFF).
I would recommend you read a bit more about python imports, if the doc seems too cluttered, you can check this link.
This is a personal preference, but I find it simpler to add the root directory of the project to the PYTHONPATH environment variable and then running all the scripts from that directory's level and changing the import statements accordingly. In your example, the root directory would be path\to\project name\.
import search for your packages in specific places, listed in sys.path. See the doc for the full details.
The current directory is always appended to this list, that's why you succeed to run everything inside PACKAGE_A. But from project name, there is no way to know where to find PACKAGE_A.
Solutions include:
use relative import
always run from the root directory (and start all your imports from the root directory)
add the root directory to the environment variable PYTHONPATH (same)
use a tool which sets PYTHONPATH when you enter your virtual environment (same)
...
and depends on your project and your needs.
I am looking for a way to import modules so that I can run a script both from the sub-folder project/v0 and from the root folder project
My file structure in python 3.6 (which is why there are no init files)
project
├── api.py
├── v0
│ ├── SearchEngine.py => contains SearchEngine class
│ └── SearchEngineBE.py
My SearchEngineBE.py module contains
from SearchEngine import SearchEngine
My api.py module contains
from v0.SearchEngineBE import SearchEngineBE
step1: When from project/v0 I run python3 SearcheEngineBE.py my module is correctly imported and everything goes well.
step2: However, when from project I run python3 run api.py I get the error:
Traceback (most recent call last):
File "api.py", line 3, in <module>
from v0.SearchEngineBE import SearchEngineBE
File "/xxx/project/v0/SearchEngineBE.py", line 3, in <module>
from SearchEngine import SearchEngine
ModuleNotFoundError: No module named 'SearchEngine'
How can I fix this so that both step 1 and step 2 would work ?
Non-relative imports are searched by the interpreter in the current directory (and any additional search paths).
You could use relative imports in your SearchEngineBE.py file to let the interpreter know you want the relative module, and not a module off the import path:
# SearchEngineBE.py
from .SearchEngine import SearchEngine
The . lets the interpreter know that you are referencing a module relative to the current module. You'll need at least a blank __init__.py file in the same directory as SearchEngine.py for relative imports to work though:
The __init__.py files are required to make Python treat directories
containing the file as packages.
https://docs.python.org/3/tutorial/modules.html
See this guide for some more discussion: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html#absolute-vs-relative-import
Edit:
Without a root package, this won't work. See this post for an alternative approach: Importing modules from a neighbouring folder in Python
Add __init__.py to make it a package.
(it could be empty file : __init__.py)
project
├── __init__.py
├── api.py
├── v0
│ ├── SearchEngine.py => contains SearchEngine class
│ └── SearchEngineBE.py
Edit 1:
try :
from project.v0.SearchEngine import SearchEngine
I have a project which I want to structure like this:
myproject
├── api
│ ├── __init__.py
│ └── api.py
├── backend
│ ├── __init__.py
│ └── backend.py
├── models
│ ├── __init__.py
│ └── some_model.py
└── __init__.py
Now, I want to import the module some_model.py in both api.py and backend.py. How do I properly do this?
I tried:
from models import some_model
but that fails with ModuleNotFoundError: No module named 'models'.
I also tried:
from ..models import some_model
which gave me ValueError: attempted relative import beyond top-level package.
What am I doing wrong here? How can I import a file from a different directory, which is not a subdirectory?
Firstly, this import statement:
from models import some_model
should be namespaced:
# in myproject/backend/backend.py or myproject/api/api.py
from myproject.models import some_model
Then you will need to get the directory which contains myproject, let's call this /path/to/parent, into the sys.path list. You can do this temporarily by setting an environment variable:
export PYTHONPATH=/path/to/parent
Or, preferably, you can do it by writing a setup.py file and installing your package. Follow the PyPA packaging guide. After you have written your setup.py file, from within the same directory, execute this to setup the correct entries in sys.path:
pip install --editable .
Unfortunately, Python will only find your file if your file is in the systems path. But fear not! There is a way around this!
Using python's sys module, we can add a directory to the path just while Python is running, and once Python stops running, it will remove it from the path.
You can do this by:
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import [file]
It is important to import sys and set the directory path before you import the file however.
Good luck!
Jordan.
I would lay out two approaches:
Simply import some_model via absolute importing:
from myproject.models import some_model
Note that the myproject should be treated as an module (i.e. having __init__.py)
Or
You can add the previous path to the sys.path which I use in such parallel level modules:
import sys
sys.path.append('../')
from models import some_model
My file directory tree looks like this:
my_repo
├── experiments
│ ├── foo.py
│ └── __init__.py
└── tests
├── baz.py
└── __init__.py
Inside baz.py, I try
from experiments.foo import FooExperiment
but I get
*** ImportError: No module named experiments.foo
When I open python from the terminal (Mac OSX 10.9) and run
from experiments.foo import FooExperiment
the class is imported properly. What is going on? Please help.
In both situations the sys.path is exactly the same, except when I'm in baz.py the current path (to baz.py) is included. And yes /path/to/my_repo is in my sys.path as well.
EDIT: my issue was with conflicting egg files, so reinstalling did the trick (below). Accepting #Austin Marshall's answer though because it's a viable solution to the general case of this problem.
pip uninstall my_repo
python setup.py develop --user
experiments is not in PYTHONPATH, nor is it installed using the standard setuptools technique. I'm able to replicate your problem, which is resolved by putting my_repo in PYTHONPATH:
Austins-MacBook-Pro-2:my_repo amarshall$ tree .
.
├── experiments
│ ├── __init__.py
│ └── foo.py
└── tests
├── __init__.py
└── baz.py
2 directories, 4 files
Austins-MacBook-Pro-2:my_repo amarshall$ PATH=$PATH:`pwd`/experiments python tests/baz.py
Traceback (most recent call last):
File "tests/baz.py", line 1, in <module>
from experiments.foo import FooExperiment
ImportError: No module named experiments.foo
Austins-MacBook-Pro-2:my_repo amarshall$ PYTHONPATH=$PYTHONPATH:`pwd` python tests/baz.py
Where there's no output, or error in the last line when PYTHONPATH is specified, rather than PATH
I usually use the following to append path to sys.path:
sys.path.append(os.path.join(os.path.dirname(__file__), "../myfolder"))
With
Import sys, os