Develop Reference

Importing files from sibling directory

I need to run unit tests for my Flask app. In my test config file, I need to import the flask create_app function from a sibling directory to initialize the test app. I cannot figure it out without getting import errors.
I have tried putting __init__.py on virtually every folder without success. I have read that editing the sys path is not recommended so I would like a solution without.
Folder Structure
root/
----__init__.py
----server/
--------__init__.py
--------database.py
----tests/
--------__init__.py
--------config.py
config.py
from server import create_app
from server.database import db
Raises this error:
Traceback (most recent call last):
File "tests/config.py", line 2, in <module>
from server import create_app
ModuleNotFoundError: No module named 'server'
and:
from ..server import create_app
from ..server.database import db
Raises this error:
Traceback (most recent call last):
File "tests/config.py", line 2, in <module>
from ..server import create_app
ValueError: attempted relative import beyond top-level package
Can someone explain why it doesn't work and how python imports work in general? I have never been able to figure them out.

For an import statement to work correctly, names must be resolved in sys.path somehow. Creating a correct package structure and installing the package is usually the best way to get the names visible in sys.path.
Remove root/__init__.py and tests/__init__.py
Correct your import statements:
from ..server import create_app # no
from server import create_app # yes
Add root/setup.py with contents as described in setuptools basic guide.
Create/activate virtualenv (usually in root/.venv/ subdir, but doesn't really matter where)
From the project root, i.e. the directory containing setup.py, install your package:
pip install --editable .

How to import submodules from relative path?

I have multiple modules in my project and specify some execution point. But when I try to import files from submodules, it doesn't work.
So, how to specify submodules to execute from selected execution file?
project
--bin
---- executeFile
--modules
---- __init__.py
----fileA.py
in executeFile, I try:
from ..modules.fileA import *
but get the error:
Traceback (most recent call last):
File "./bin/muexecute", line 10, in <module>
from ..modules.os import *
SystemError: Parent module '' not loaded, cannot perform relative import

I found solution.
The problem was in my opinion about using init.py.
I placed in executable scripts path to including and it works fine
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))

So you are having trouble defining your relative path, correct? Try the following:
from sys import path
path.append('C:\\realative_path')
from function_file import required_function
Hope that helps.

All modules you want to import should in in your PYTHONPATH. Therefore there is no hierarchy.
In your case It seems to me that an __init__.py is missing from your project's main folder (with all the models included), so executefile doesn't know about your modules.

File path relative to where it was imported

I have:
project
|--__init__.py
|--...
+--package1
| |--__init__.py
| |--...
+--dbs
|--...
Where project is a collection of packages to be imported and used by the user. The names are just an example.
Now, some files under project/package1/ need to access some files in project/dbs/, but depending on where the user placed the project folder and from where he or she imported it, path/to/project/dbs/file is not the same. Because of this issue and for other purposes I thought I should have a variable projectroot defined as the root of the project, relative to where it was imported from. For instance:
If the user has ~/Desktop/project and imported it from ~/ then projectroot would be Desktop/project, and path/to/project/dbs/file would be "%s/dbs/file" % (projectroot). Is this really needed? If so, what is the best way of doing it?
I tried using this in project/pathtfinder.py and used project/__init__.py as
import pathfinder
projectroot=pathfinder.module_path()
import package1
But when I run import project I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "project/__init__.py", line 3, in <module>
import package1
File "project/package1/__init__.py", line 5, in <module>
db = sqlite3.connect("%s/dbs/main.db" % projectroot)
NameError: name 'projectroot' is not defined
And if use print projectroot just after defining it I get Desktop/project, as expected.

You need to add the path of project to the PYTHONPATH

This is a pattern I use for my Flask apps, but the same should be ideal for you:
Assume the following is in a file under helloworld/app.py
import sys
import os
# Add this app to the path so imports work as expected
APP_PATH = os.path.join(os.path.dirname(__file__), '../')
sys.path.append(APP_PATH)
from helloworld.utils import a_utility
The convention explains itself, but it allows for the imports to occur from the "root" of the project, regardless of where the file is placed. You'll want to place that APP_PATH code perhaps in your init.py, or similar.

When automatically importing modules from a subfolder, their imports fail

I've read through a couple of similar questions, notably this one about imp.load_module which seems to be close to what I want, but I can't understand why I'm still getting ImportErrors. Here is my folder hierarchy:
program\
__init__.py
main.py
thirdparty\
__init__.py
css\
__init__.py
css.py
utils\
__init__.py
http.py
In main.py I have the following code. This is intended to search the thirdparty\ directory and load each module it finds. Each module is in its own separate directory.
import os
import imp
for root, dirs, files in os.walk("thirdparty"):
for source in (s for s in files if s.endswith(".py")):
name = os.path.splitext(os.path.basename(source))[0]
m = imp.load_module(name, *imp.find_module(name, [root]))
The problem is that css.py happens to use its own subfolder that it loads stuff off of, utils. It has a line in it that says:
from utils import http
And that is where it fails. I get this error when I run main.py.
Traceback (most recent call last):
File "main.py", line 7, in <module>
m = imp.load_module(name, *imp.find_module(name, [root]))
File "thirdparty/css/css.py", line 1, in <module>
from utils import http
ImportError: No module named utils
I'm stumped. css.py is self contained in its own folder, and when I run css.py separately it imports utils just fine. What is causing this?

Maybe you can solve this by changing the import to:
from .utils import http
Or by adding the folder you import to the Python Path:
sys.path.append(os.path.join(root, source))
When you import modules in thirdparty, the place Python looks for modules is still the main directory. The initial import works, because you give the right path to imp.find_module, but after that Python has no idea where to look for the modules.

Import from different directories in python

This is my folder structure:
src/
__init__py
Lowlevel/
__init__.py
ModuleToCheck.Py
Test/
__init__.py
ModuleToCheck_test.py
(__init__.py are empty files)
Now I want to import ModuleToCheck.py in ModuleToCheck_test.py
How am I able to do this without appending anything to sys.path?
Update:
from ..Lowlevel import ModuleToCheck leads to:
src$ python Test/ModuleToCheck_test.py
Traceback (most recent call last):
File "Test/ModuleToCheck_test.py", line 6, in <module>
from ..Lowlevel import ModuleToCheck
ValueError: Attempted relative import in non-package

The following is from http://docs.python.org/tutorial/modules.html#intra-package-references
Note that both explicit and implicit
relative imports are based on the name
of the current module. Since the name
of the main module is always
"__main__", modules intended for use
as the main module of a Python
application should always use absolute
imports.
You're running your module ModuleToCheck_test.py as the main module, hence the exception.
One solution is to create a test.py module in your src directory containing the following:
import Test.ModuleToCheck_test
You can then run that module using python test.py