Develop Reference

Trouble importing a module that imports a module

I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.

The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file

__init__.py does not find modules in same directory [duplicate]

This question already has an answer here:
ImportError on python 3, worked fine on python 2.7
(1 answer)
Closed 5 years ago.
I need assistance on how to organize source in a python package - I've already followed several tutorials on the web (especially this one) on how to do so, but it does not work as explained and how I imagined it.
I want to create a python package named binaryio. It should offer two classes named BinaryReader and BinaryWriter which I want users to be able to import with
from binaryio import BinaryReader
from binaryio import BinaryWriter
Thus I have created my repository and package directory structure as follows:
binaryio (repository root)
binaryio (package root)
__init__.py (s. below)
binaryreader.py (contains the BinaryReader class)
binarywriter.py (contains the BinaryWriter class)
setup.py (contains the setuptools.setup call)
.gitignore, README.md, LICENSE, ...
As you can see, the classes are in separate files as I'm used to this (coming from a C# background). I'm not sure if this is a good idea due to modules being the "unit" in Python - but otherwise cramping all classes into one huge file did not seem logical to me.
__init__.py looks as follows to import those classes, making (as I understood it) the from binaryio import BinaryReader imports possible for users later on:
from binaryreader import BinaryReader
from binarywriter import BinaryWriter
However, when I install the package locally (which seems to work fine) and try to import binaryio, I get the following error:
>>> import binaryio
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Projects\Git\binaryio\binaryio\__init__.py", line 1, in <module>
from binaryreader import BinaryReader
ModuleNotFoundError: No module named 'binaryreader'
Apparently, something is wrong with my __init__.py file. I don't understand this, as a binaryreader.py file aka module exists in the same folder as you can see above. Funnily enough, my IDE (PyCharm, having set the package root as source folder) does not complain about the statements in it and can resolve all references.
What am I doing wrong here? According to the tutorial linked above, putting a file into xyz.py with a class named Abc and then write from xyz import Abc into __init__.py should work, but apparently it doesn't for me.

Your code would work for Python 2.x, but not 3.x because of different relative imports syntax: without dot, Python 2.x would look for modules in module root and current package, and Python 3.x will look only in module root.
Import statements you want to use are these:
from binaryio.binaryreader import BinaryReader
from binaryio.binarywriter import BinaryWriter
Works in both Python 2.x and 3.x without "futures"

I think you need to add a dot in your import statements :
from .binaryreader import BinaryReader
from .binarywriter import BinaryWriter

Python 3.5 vs Python 2.7: Modules importing submodules

I have been googling this for the past hours and can't find an equivalent question anywhere. Also the documentation for 2.7 and 3.5 seem identical, so I don't think this behavior is documented.
Here is my directory structure:
project
-- project.py
-- api
-- __init__.py
-- subapi
-- __init__.py
contents of project/project.py: import api
contents of project/api/__init__.py: import subapi
If I execute python project.py (using python 2.7) from inside the projects folder, it returns without an error. If I do the same with python 3 (python3 project.py), then it crashes with
Traceback (most recent call last):
File "project.py", line 1, in <module>
import api
File "/home/me/Documents/project/api/__init__.py", line 1, in <module>
import subapi
ImportError: No module named 'subapi'
If I rewrite the import statement to use paths relative to the projects directory (import api.subapi), then it works with python 2 as well as 3. It's not a satisfying solution though because that requires me to reference parent modules from within sub modules which kind of defeats the idea of modularity..
Does anyone have an idea what I can do to get the behavior of python2 back? The module search algorithm should prioritize searching in the local directory of the file using the import statement. It should also prioritize these files above built in modules by the way. Try importing a module 'test'..
--EDIT--
I was asked by stackoverflow to differentiate my question from another called "How to do relative imports". I think this question is different because I am asking specifically about differences between two versions. Using relative imports is the solution, not the question.

Use an explicit relative import:
from . import subapi

relative imports doesn't work, looks like can't find the module

I have the following situation, where a is a directory:
a/
__init__.py
first.py
second.py
__init__.py
print('i\'m starting the directory')
__all__ = ['second', 'first']
first.py
print('hi, i\'m the first')
from . import *
second.py
print('hi, i\'m the second')
So when I run from the interactive prompt:
>>> import a.first
i'm starting the directory
hi, i'm the first
hi, i'm the second
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/antox/Scrivania/a/first.py", line 2, in <module>
from . import *
AttributeError: 'module' object has no attribute 'first'
Why doesn't it find the first.py module? I mean I expected to get no error; during the run of the import I thought that it could see that first.py was already loaded so there was no error, it just skipped to the next listed in __all__.

This appears to be a long-standing bug in the Python import machinery. The issue is that the a module is not added to its package's global namespace until after it has completely loaded. This breaks from package import module statements, if module is only partly loaded. You're still able to use import package.module though, which has for a long time had special support from the import system.
As jonrsharpe commented, the Python developers have been aware of this issue since 2004. Since the issue only comes up when there are circular imports involving modules in packages, and circular imports are usually considered bad design, they did not put a very high priority on fixing it.
However, there has been recent progress! A partial fix was added a few months ago, for Python 3.5, which has just had its first alpha release (the full release is scheduled for September). The fix still doesn't actually add module to package while it's being loaded, but instead adds an extra check to from package import module statements so that they work anyway in circular import situations that previously raised an ImportError.
This doesn't fix the from package import * case however. The code for wildcard imports apparently still wants all of the names in package.__all__ to actually be present in the module itself. It doesn't appear to check the sys.modules dictionary to check for modules that are still in the processes of being loaded.
So, what does this mean for your code? I think there are two important lessons: First, don't use circular imports if you can help it (instead try to factor out some code from the two modules into a third utility module). Second, don't use wildcard imports (if first.py used from . import first, second you'll have no error in Python 3.5).

Python Module Imports - Explicit vs Implicit Relative Imports

Last night, when working on my mac, I set up some module imports in my __init__.py's
from MongoProvider import MongoProvider
from Settings import Settings
etc. I'm unsure of what version of Python is on that machine. I'll edit the question later with that info once I have it.
Today, working on a different machine, which is Windows and using Python 3.3.3, my module imports were breaking. By adding an explicit relative import (adding a leading dot), I was able to fix the issue.
from .MongoProvider import MongoProvider
from .Settings import Settings
The trace I'm receiving is:
Traceback (most recent call last):
File "app.py", line 5, in <module> from modules.route_handlers import Route_Handlers
File "C:\Users\willb\bearded-dubstep\modules\route_handlers\Route_Handlers.py", line 9, in <module> from modules.backend_providers import Settings
File "C:\Users\willb\bearded-dubstep\modules\backend_providers\__init__.py", line 1, in <module> from MongoProvider import MongoProvider
ImportError: No module named 'MongoProvider'
My Project Layout is
root
|_modules
|_api_helpers
| __init__.py
| InvalidUsage.py
| response_utils.py
|_backend_providers
| __init__.py
| MongoProvider.py
| Settings.py
|_route_handlers
| __init__.py
| Route_Handlers
| app.py
Any ideas what would cause this? Is there a configuration file I should be looking at?

Well, according to PEP-8 imports section:
Implicit relative imports should never be used and have been removed in Python 3.
As Python 3.3 is the one causing you trouble, making you explicit import relatives modules, I assume this explains the situation. It's probably that on Mac you have Python 2.x, that's why it works there.
Looking at your project's file distribution, Settings.py and MongoProvider are indeed relative modules. This means that the removal of implicit relative imports in Python 3 is the one causing you trouble, because the import system is unable to find the module:
ImportError: No module named 'MongoProvider'

It seems that the Python Version on your mac is 2.x while the Python on your windows is 3.x.
I have met the same problem before using module tkinter (which is Tkinter in Python 2.x).
It seems that we need to use from XXX.xxx import xxx to import...●﹏●
I don't know why, maybe it's the designer's adjusting to Python.