Develop Reference

Trouble importing a module that imports a module

I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.

The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file

Python Module names are contradicting depending on where you run the code from

This is my folder structure -
From api.py I import IncidentHandler as below -
import data.IncidentHandler
data.IncidentHandler works fine HOWEVER - inside of the IncidentHandler.py I have the below present:
import BrowserHandler
When I run the code from api.py I get this error -
Traceback (most recent call last):
File "api.py", line 9, in <module>
import data.IncidentHandler
File "/User/**mask**/**mask**/**mask**/**mask**/tpptickethandler/src/data/IncidentHandler.py", line 1, in <module>
import BrowserHandler
ModuleNotFoundError: No module named 'BrowserHandler'
When I run the code directly from IncidentHandler.py it is working as expected.
I realised that the solution was in the modular name so I have changed as follows -
import BrowserHandler
to
import data.BrowserHandler
This now works from api.py but NOT from IncidentHandler.py
New Error from IncidentHandler.py
Traceback (most recent call last):
File "IncidentHandler.py", line 1, in <module>
import data.BrowserHandler
ModuleNotFoundError: No module named 'data'
I can see that I may be working against the typical Python principles however facing this issue was quite a unique challenge and I have not found anything on the internet thus far. Could someone please assist me with a solution and/or how to tackle this better next time.

You are trying to organize the modules under different folders, which are packages. For python to recognize a folder as a package, you need to have an __init__.py file under it, it doesn't matter if this file is empty or not, that will let python correctly recognize the folder as a package
https://docs.python.org/3/tutorial/modules.html#packages
example from the official site
sound/ Top-level package
__init__.py Initialize the sound package
formats/ Subpackage for file format conversions
__init__.py
wavread.py
wavwrite.py
aiffread.py
aiffwrite.py
auread.py
auwrite.py
...
effects/ Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
...
filters/ Subpackage for filters
__init__.py
equalizer.py
vocoder.py
karaoke.py
...

Having Trouble with Import Statement in Python

My current projects are structured like so.
python/
__init__.py
project_1/
__init__.py
program_1.py
project_2/
__init__.py
program_2.py
project_3/
__init__.py
program_3.py
I wanted to import a class from project_3/program_3.py, called INFO, from both project_1/program_1.py and project_2/program_2.py. In both my program_1.py and program_2.py, I've tried the following import statements that didn't work.
Edit - I typically "cd" into project_1 and run my program_1.py from there.
from project_3.program_3 import INFO
Traceback (most recent call last):
File "./program_1.py", line 43, in <module>
from project_3.program_3 import INFO
ImportError: No module named 'project_3'
from python.project_3.program_3 import INFO
Traceback (most recent call last):
File "./program_1.py", line 43, in <module>
from python.project_3.program_3 import INFO
ImportError: No module named 'python'
The only way for me to import class INFO into program_1.py and program_2.py is by copying program_3.py in both program_1 and program_2's folder. My import in program_1 and program_2's statement would become:
python/
__init__.py
project_1/
__init__.py
program_1.py
program_3.py
project_2/
__init__.py
program_2.py
program_3.py
project_3/
__init__.py
program_3.py
from program_3 import INFO
This is quite inconvenience because now I have to update program_3.py in all 3 locations. Assuming my the way I structured my project folder is retarded:
What's the correct/best way to structure my folders so that I could call program_3.py from program_1/2.py?
If there's nothing wrong with my current structure, how do I correctly import the INFO class into program_1/2.py?
I have read the python documentation but it didn't say anything regarding importing classes from one subfolder to another. I also looked at another post from Stack Overflow as well but it didn't have the same structure as mine.
Update 1 - It's important to note that I'm merely using project_1/2 as folder to organized my projects. I originally had init.py in all my folders because I have no clue what I'm doing. The only module/package that I wanted to import in my project_1/program_1.py is from project_3/program_3.py. Example:
python/
project_1/
program_1.py
project_2/
program_2.py
project_3/
__init__.py
program_3.py

When using submodules, you need to use relative imports. In the example provided, importing program_3 from program_1 should be :
from ..project_3 import program_3
and then you can use the INFO class: program_3.INFO.
This works if your executable file is outside the script, that is the file which uses the module python in the example should be outside the python module.
If parts of the python package are executed as a script, the -m option should be passed to the interpreter:
python3 -m python.project_1.program_1
I hope this helps.

The previous solution provides me the same error.
I found a fix by using sys:
import sys
sys.path.append('../')
from project_3 import program_3
...
This way, you setup 'python' folder as reference (in your case it's the parent folder but you can select the path you want)
I hope it helps,

Python cannot import name <class>

I've been wrestling most of the night trying to solve an import error.
This is a common issue, but no previous question quite answers my issue.
I am using PyDev (an Eclipse plugin), and the library Kivy (a Python library)
I have a file structure set up like this:
<code>
__init__.py
main.py
engine.py
main_menu_widget.py
"code" is held within the eclipse folder "MyProject" but it's not a package so I didn't include it.
The files look like this:
main.py
# main.py
from code.engine import Engine
class MotionApp(App):
# Ommited
engine.py
# engine.py
from code.main_menu_widget import MainMenuWidget
class Engine():
# Ommited
main_menu_widget.py
# main_menu_widget.py
from code.engine import Engine
class MainMenuWidget(Screen):
pass
The error I recieve, in full detail, is:
Traceback (most recent call last):
File "C:\MyProject\code\main.py", line 8, in <module>
from code.engine import Engine
File "C:\MyProject\code\engine.py", line 6, in <module>
from code.main_menu_widget import MainMenuWidget
File "C:\MyProject\code\main_menu_widget.py", line 3, in <module>
from code.engine import Engine
Any idea what I did wrong here? I just renamed my entire folder structure because I screwed up this module structure so bad, but I think i'm close to how it should look....

There seems to be a circular import.
from engine.py you are importing main_menu_widget while from main_menu_widgetyou are importing engine.
That is clearly a circular import which is not allowed by python.

it's in the same folder, use a relative package name (it's a good practice to do so anyway):
from .engine import Engine

Your code directory is a package. Ensure that the directory above it, i.e C:\MyProject judging from your error messages, is in your PYTHONPATH.
Open the context menu by selecting your project and clicking your mouse's right button, then select Properties. Select PyDev - PYTHONPATH and from there the Source folders tab.
Check that the directory mentioned above is present; if it isn't press Add source folder, select it from the dialogue and press OK.

Import from different directories in python

This is my folder structure:
src/
__init__py
Lowlevel/
__init__.py
ModuleToCheck.Py
Test/
__init__.py
ModuleToCheck_test.py
(__init__.py are empty files)
Now I want to import ModuleToCheck.py in ModuleToCheck_test.py
How am I able to do this without appending anything to sys.path?
Update:
from ..Lowlevel import ModuleToCheck leads to:
src$ python Test/ModuleToCheck_test.py
Traceback (most recent call last):
File "Test/ModuleToCheck_test.py", line 6, in <module>
from ..Lowlevel import ModuleToCheck
ValueError: Attempted relative import in non-package

The following is from http://docs.python.org/tutorial/modules.html#intra-package-references
Note that both explicit and implicit
relative imports are based on the name
of the current module. Since the name
of the main module is always
"__main__", modules intended for use
as the main module of a Python
application should always use absolute
imports.
You're running your module ModuleToCheck_test.py as the main module, hence the exception.
One solution is to create a test.py module in your src directory containing the following:
import Test.ModuleToCheck_test
You can then run that module using python test.py