I have (ended up with) my packages being setup in the following way. There are two packages with the same name, but one of them is inside package_a.
├── package_a/
│ ├── __init__.py
│ └── package_aa/
│ ├── __init__.py
│ └── module_a.py
│ └── module_c.py
└── package_aa/
├── __init__.py
└── module_b.py
How do I import both module_a.py and module_b.py in my code? The following
from package_a.package_aa import module_a; from package_aa import module_b;
fails with the error message
ImportError: cannot import name 'module_b' from 'package_aa' (.\package_a\package_aa\__init__.py)
All __init__.py files contain a single line
import os, sys; sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
Renaming is unfortunately not an option as I am not the owner of these packages.
EDIT : Swapping works. Thanks. I have a follow up question : Suppose in module_c.py, modula_a.py is imported like this from package_aa import module_a. Then the following fail with similar error, but it is not as apparent (to the user) as in the original question
from package_aa import module_b; from package_a import module_c
How can the owner of package_a modify it so that its users don't run into errors like this? I am starting to think, the way __init__.py is written might not be right. Although, I have seen answers elsewhere which suggests adding sys.path.append or sys.path.insert in this fashion.
My guess would be to swap them like this:
from package_aa import module_b
from package_a.package_aa import module_a
This way, package_aa is not referenced in the path.
Related
├── README.md
├── src/
│ └── ptmodeling
│ ├── __init__.py
│ ├── driver.py
│ ├── features.py
│ └── modeling.py
└── tests/
When I run python3 driver.py from within the ptmodeling directory I get
ModuleNotFoundError: No module named 'ptmodeling' . I tried running from project root, adding everything to PYTHONPATH and nothing works with the driver.py below
driver.py
import logging
import os
import sys
sys.path.append(os.getcwd())
print(sys.path)
from ptmodeling.modeling import ModelEngine
from ptmodeling.features import FeatureExtractor
But this works
driver.py
import logging
import os
import sys
sys.path.append(os.getcwd())
print(sys.path)
from modeling import Model
from features import Feature
I don't see any problem here? Your question itself has the answer. And to your doubt, your code where you get an error won't work because python is looking into the ptmodeling folder inside the current directory. This means you're surfing through src/ptmodeling/ptmodeling which doesn't exist.
I have a project that I run from
root_dir> python ./src/main_file.py. When I want to test it, I use root_dir> python -m pytest and it will search for any files of the format test*.py or *test.py.
I have a file structure similar to the following:
root_dir
├── src
│ ├── __init__.py
│ ├── main_file.py
│ ├── file1.py
│ ├── file2.py
└── tests
├── __init__.py
└── file1_test.py
What I'm wanting to do is create a test file to test file1.py, but I'm getting a ModuleNotFoundError for what I believe is the following reason and I'm not fully sure what the proper way to address it is.
In file1_test.py:
./file1_test.py/
import src.file1 as file1
def test_file1():
<do stuff>
The problem occurs inside file1.py, where:
./file1.py/
import file2.py
<blah blah blah>
pytest fails with ModuleNotFoundError: No module named 'file2'. If I change file1.py to:
./file1.py/ (EDITED)
import src.file2.py
<blah blah blah>
then pytest will run successfully. However, if my main_file.py imports file1.py from within the same module (src),
./main_file.py/
import file1
<blah blah blah>
Then python ./src/main_file.py will fail with ModuleNotFoundError: No module named 'src.file2' due to that change. Changing file1 back to the original will work for main_file.py, but not pytest, and I'm stuck in a loop.
I can't figure out what I should be doing differently to get the normal import structure of main_file.py and pytest to cooperate. Should I be changing the directory? Should main_file.py be outside of src? I think I would just run into a similar issue where if file1 is importing file2, the import path a file outside of src expects and one inside of src will expect are always going to conflict.
There are three ways to do what you want.
Prefered way
root_dir
├── my_module
│ ├── __init__.py
│ ├── main_file.py
│ ├── file1.py
│ ├── file2.py
└── tests
├── __init__.py
└── file1_test.py
All files in my_module shoud import like
from my_module.file1 import some_function
Then you don't need to do any hacks
Keepin src and assuming statements like
from src.file1 import some_function
are not what you want you need to modify your tests like
import os
import sys
sys.path.insert(0, os.path.abspath('src')
Not modifying src, not modifying tests: Run tests with
PYTHONPATH=path/to/src pytest
root
├── rootfile.py
├── folder
│ └── __init__.py
│ └── file.py
│ └── subfolder
│ └── __init__.py
│ └── subfile.py
The folder structure is as above.
assume file has function foo(), subfile subfoo().
rootfile imports foo() from file.py by
from folder.file import foo
file imports subfoo() from subfile.py by
from subfolder.subfile import subfoo
and reports ModuleNotFoundError: No module named 'subfolder'.
How should I address this error? I have attempted sys.path.append in the file.py but it did not work.
Interestingly,
from folder.subfolder.subfile import subfoo
works from rootfile, but this is not what I need.
I appreciate your time and guidance in advance.
Most likely your "path to look for imports" contains root of your project, but not folder/subfolder etc.
Using relative import should help:
from .subfolder.subfile import subfoo
Python 3.8
MacOS
I am trying to import different modules inside a package and I am getting ModuleNotFoundError or ImportError when trying to use relative imports.
I have a folder called "practice_dir" with the following structure:
(base) or ƒ(~/practice_dir) >> tree
.
├── __init__.py
├── dir_1
│ ├── __init__.py
│ ├── file_1.py
│ └── subdir
│ ├── __init__.py
│ └── subfile.py
├── dir_2
│ ├── __init__.py
│ └── file_2.py
└── parent_file.py
It was mentioned online that once I have an init file on every folder containing modules than the absolute imports would start from the parent package
And relative imports would be using . or .. notation
Here's some of the examples I tried
In file_1:
from dir_2 import file_2
ModuleNotFoundError
from .dir_2 import file_2
ImportError: attempted relative import with no known parent package
It's the same if I do the same thing in dir_2, and dir_1/subfolder
The init.py files are empty right now, I tried to include imports to the files but that didn't work.
I've been looking at some questions posted online and on youtube, there seems to be various ways and one guy even mentioned that you don't need the init files if you have python 3.3+
Been battling hard with this, any help would be very much appreciated!
You can add the file to the python path at runtime:
import sys
sys.path.insert(0, 'path/to/file')
import file
Yes, I know this is a recurrent question but I still couldn't find a convincing answer. I even read at https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html but could not find out how to solve the problem:
I'm running python 3.6 project that includes jupyter (ipython) notebooks. I want the notebook to import a custom local helpers.py package that I will probably use also later in other sources.
The project structure is similar to:
my_project/
│
├── my_project/
│ ├── notebooks/
│ └── a_notebook.ipynb
│ ├── __init__.py # suppose to make package `my_project` importable
│ └── helpers.py
│
├── tests/
│ └── helpers_tests.py
│
├── .gitignore
├── LICENSE
├── README.md
├── requirements.txt
└── setup.py
When importing helpers in the notebook I get the error:
----> 4 import helpers
ModuleNotFoundError: No module named 'helpers'
I also tried from my_project import helpers and I get the same error ModuleNotFoundError: No module named 'my_project'
I finally (and temporarily) used the usual trick:
import sys
sys.path.append('..')
import helpers
But it looks awful and I'm still looking for a better solution
One can tell python where to look for modules via sys.path. I have a project structure like this:
project/
│
├── src/
│ └── my_module/
│ ├── __init__.py
│ └── helpers.py
├── notebooks/
│ └── a_notebook.ipynb
...
I was able to load the module like so:
import sys
sys.path.append('../src/')
from my_module import helpers
One should be able load the module from wherever they have it.
Here I could find several solutions. Some of them are similar to the ones answered before:
https://mg.readthedocs.io/importing-local-python-modules-from-jupyter-notebooks/index.html
If you move the notebooks directory out one level, and then explicitly import your module from the package, that should do it. So your directory would look like this:
my_project/
│
├── my_project/
│ ├── __init__.py
│ └── helpers.py
├── notebooks/
│ └── a_notebook.ipynb
...
and then your import statement within the notebook would be:
from my_project import helpers.
Try the following line:
from my_project.helpers import what_you_need
This line should also work:
import my_project.helpers
I think you need a __init__.py module in the notebooks/ directory. I haven't really used Jupyter notebooks before so I could be wrong. You may also need to try changing your import statement to:
import .. helpers
to indicate that the import statement is for a local package that is located in the parent directory of the Jupyter notebook.
This worked for me.
import sys
MODULE_FULL_PATH = '/<home>/<username>/my_project/my_project'
sys.path.insert(1, MODULE_FULL_PATH)
from my_module import helpers
If you are on a Unix/Linux system another elegant solution may be creating a "soft link" to the module file helpers.py that you would like to use. Change to the notebooks directory and create the link to the module file this way:
cd notebooks; ln -fs ../my_project/helpers.py .
This "soft link" is essentially a pointer (a shortcut) to the original target file. With the link in place you will be import your module file as usual:
import helpers