Develop Reference

Python - Structuring an application with tests

I want to create the following structure for a simple app, i am having some trouble with imports:
Suggested structure: (Its the "Installable Single Package" from various sites)
/myapp
/helloworld
__init__.py
helloworld.py
utils.py
/tests
__init__.py
test_helloworld.py
test_utils.py
With helloworld.py:
import sys
import utils
def hello_world():
print("Hello world")
if __name__ == '__main__':
utils.util()
and utils.py:
def util():
print("util")
and test_helloworld.py:
import sys
sys.path.insert(0,'..')
from helloworld.helloworld import hello_world
from helloworld.utils import util
hello_world()
When i attempt to run helloworld.py it runs normally.
When i attempt to run test_helloworld.py i get this error:
Traceback (most recent call last):
File "test_helloworld.py", line 8, in <module>
from helloworld.helloworld import hello_world
File "..\helloworld\helloworld.py", line 3, in <module>
import utils
ModuleNotFoundError: No module named 'utils'
I think this is due to some relative path error, the only way i ve found to fix this was to alter the helloworld.py to be like:
import sys
# import utils
def hello_world():
print("Hello world")
if __name__ == '__main__':
import utils
utils.util()
and then they both run. Is there a way to do this import better? Is this a problem that i will not have to address if i construct unittests in "test_helloworld.py" and run it with unittest?

Import Error when trying to make python 2 modul

This doesn't seem to be an issue in python 3 but I'm needing to use python 2.7 for this and get the following issue
DIRECTORY STRUCTURE
module
├── __init__.py
└── submodule
├── __init__.py
└── test.py
# module/__init__.py
from module import submodule
# module/submodule/__init__.py
from module.submodule import test
# module/submodule/test.py
from module import submodule
when I try to import module from somewhere else it results in the following error:
>>> import module
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "module/__init__.py", line 1, in <module>
from module import submodule
File "module/submodule/__init__.py", line 1, in <module>
from module.submodule import test
File "module/submodule/test.py", line 1, in <module>
from module import submodule
ImportError: cannot import name submodule
I'm assuming the issue has something to do with circular imports but i need to use submodule in both module/__init__.py and module/submodule/test.py
any help is appreciated

As you've mentioned circular imports, one way to resolve this is by importing the module locally.
For e.g.
def fun():
from module import submodule

Issues with importing from submodules

I've got a project with the following structure:
\main_model
__init__.py
main.py
\second_module
__init__.py
second.py
\third_module
__init__.py
third.py
second_module and third_module are both Git submodules used as libraries by many other projects.
The first line of main.py is an import statement: from main_model.second_module import second
And the first line of second.py is too: from second_module.third_module import third
Running main.py raises an error with the following Traceback:
Traceback (most recent call last):
File "/main_model/main.py", line 1, in <module>
from main_model.second_module import func_a, func_b
File "/main_model/second_module/second.py", line 1, in <module>
from third_module.third import func_c, func_d
ModuleNotFoundError: No module named 'third_module'
So evidently I would need to amend the import statement in second.py to be from main_model.second_module.third_module import third. I can't do that, though, because second_module is a standalone submodule.
Is there a consistent way of structuring the imports, or modifying PYTHONPATH, that would allow the same import syntax from all levels?

Python nested subpackage doesn't import when running tests

I have a folder structure like this:
api
-- test
-- test_api.py
-- __init__.py
-- api
-- api.py
-- __init__.py
-- sub
-- sub.py
-- __init__.py
sub.py:
Base = 'base'
api.py:
from sub.sub import Base
def stuff_to_test():
pass
test_api.py:
from api.api import stuff_to_test
def test_stuff_to_test():
stuff_to_test()
I am in directory api.
I run pytest:
==================================== ERRORS ====================================
______________________ ERROR collecting tests/test_api.py ______________________
ImportError while importing test module '/<somepath>/api/tests/test_api.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_api.py:1: in <module>
from ..api.api import stuff_to_test
api/__init__.py:1: in <module>
from . import api
api/api.py:1: in <module>
from sub.sub import Base
E ImportError: No module named 'sub'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.08 seconds ============================
Same happens if I run the python interpreter and import stuff from test_api.py:
>>> from tests.test_api import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<somepath>/api/tests/test_api.py", line 1, in <module>
from api.api import stuff_to_test
File "/<somepath>/api/api/__init__.py", line 1, in <module>
from . import api
File "/<somepath>/api/api/api.py", line 1, in <module>
from sub.sub import Base
ImportError: No module named 'sub'
My first idea was to make the import in api.py relative:
from .sub.sub import Base
This way tests run fine.
But if I run python api/api.py I get this error:
Traceback (most recent call last):
File "api/api.py", line 1, in <module>
from .sub.sub import Base
SystemError: Parent module '' not loaded, cannot perform relative import
How can I have it so tests run and application runs?

I solved it by adding the following to test.__init__.py
project_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
api_path= os.path.join(project_path, 'api')
sys.path.append(api_path)

In python there are two ways to import a module, with a relative path or with an absolute path.
When you write from sub.sub import Base you are doing an absolute path import, for a relative path import write from .sub.sub import Base.
An absolute path import go look in the PYTHONPATH to find the starting point of your import, so you should write from api.sub.sub import Base.
For more information: Absolute vs. explicit relative import of Python module

Why can `__init__.py` import a submodule but a test file with `from module import *` can't?

I have the following folder structure.
project/
sandbox.py
module/
__init__.py
klass.py
__init__.py has the following:
from klass import Klass
if __name__ == '__main__':
'do stuff'
This works fine. sandbox.py looks like this:
from module import *
if __name__ == '__main__":
'do similar stuff'
On the command line, while in the project directory:
$ python module/__init__.py
# things work correctly
$ python sandbox.py
Traceback (most recent call last):
File "sandbox.py", line 1, in <module>
from module import *
File "/Uses/chuck/.../project/module/__init__.py, line 1 in <module>
from klass import Klass
ImportError: No module named 'klass'
I think this might have to do with relative imports, and tried changing from klass import Klass to from .klass import Klass, but that didn't work either.
My goal is, from files outside the module, to be able to use from module import * or from module import Klass. What do I need to do?
In case it matters, this is with Python 3.5.2 under macOS within a virtual environment.

Use absolute path and __main__.py file for this:
from module.klass import Klass
'do stuff'
In __init__.py:
from .klass import Klass
Now you can execute it as follows:
$ python sandbox.py
do similar stuff
$ python -m module
do somthing