Develop Reference

How to get absolute path of root directory from anywhere within the directory in python

Let's say I have the following directory
model_folder
|
|
------- model_modules
| |
| ---- __init__.py
| |
| ---- foo.py
| |
| ---- bar.py
|
|
------- research
| |
| ----- training.ipynb
| |
| ----- eda.ipynb
|
|
------- main.py
and I want to import model_modules into a script in research
I can do that with the following
import sys
sys.path.append('/absolute/path/model_folder')
from model_modules.foo import Foo
from model_modules.bar import Bar
However, let's say I don't explicitly know the absolute path of the root, or perhaps just don't want to hardcode it as it may change locations. How could I get the absolute path of module_folder from anywhere in the directory so I could do something like this?
import sys
sys.path.append(root)
from model_modules.foo import Foo
from model_modules.bar import Bar
I referred to this question in which one of the answers recommends adding the following to the root directory, like so:
utils.py
from pathlib import Path
def get_project_root() -> Path:
return Path(__file__).parent.parent
model_folder
|
|
------- model_modules
| |
| ---- __init__.py
| |
| ---- foo.py
| |
| ---- bar.py
|
|
|
------- src
| |
| ---- utils.py
|
|
|
|
|
------- research
| |
| ----- training.ipynb
| |
| ----- eda.ipynb
|
|
------- main.py
But then when I try to import this into a script in a subdirectory, like training.ipynb, I get an error
from src.utils import get_project_root
root = get_project_root
ModuleNotFoundError: No module named 'src'
So my question is, how can I get the absolute path to the root directory from anywhere within the directory in python?

sys.path[0] contain your root directory (the directory where the program is located). You can use that to add your sub-directories.
import sys
sys.path.append( sys.path[0] + "/model_modules")
import foo
and for cases where foo.py may exist elsewhere:
import sys
sys.path.insert( 1, sys.path[0] + "/model_modules") # put near front of list
import foo

exposing all classes and functions visible from the main module

I'm a little confused about the import in a python project.
I used this as a model to create my project:
https://docs.python-guide.org/writing/structure/
at the moment, working in spyder, I set my working directory to MyProject/
MyProject
|
|
--- mymodule
| |
| |--- myclass1.py (contains def MyClass1 )
| |
| |--- myclass2.py (contains def MyClass2 )
|
|
|--- tests
| |
| |-- test_MyClass1.py (contains def TestMyClass1(unittest.TestCase)
| |
| |
| |-- test_MyClass2.py (contains def TestMyClass2(unittest.TestCase)
then I run test_MyClass1.py
the test_MyClass1.py references the MyClass1 this way:
from mymodule.myclass1 import MyClass1
and in the myclass1.py, I reference the MyClass2 this way:
from mymodule.myclass2 import MyClass2
I read about the __init__.py and the namespace packages, the more I read the more confused I get...
Basically I do not want to do :
mymodule.myfile.myclass
but rather:
import mymodule as mm
mm.MyClass1
or again:
from mymodule import *
a = MyClass1()
Still, I want one file by class.

You can add the import of MyClass1 and MyClass2 in mymodule/__init__.py.
Basically you will have the following files:
mymodule/
__init__.py
myclass1.py
myclass2.py
tests/
test_myclass1.py
test_myclass2.py
where:
mymodule/__init__.py contains the following lines:
from mymodule.myclass1 import MyClass1
from mymodule.myclass2 import MyClass2
mymodule/myclass1.py contains MyClass1 definition
mymodule/myclass2.py contains MyClass2 definition
Then in tests/test_myclass1.py you can import MyClass1 thanks to:
from mymodule import MyClass1
a = MyClass1()
or
import mymodule as mm
a = mm.MyClass1()
You can do the same for MyClass2

Importing classes - Python

Good morning I am stuck with the following problem. Precisely, I have the following setup:
Project_Name
|
|--> __init__.py
|
|--> Tool1
| |
| |--> Object1.py
| |
| |--> __init__.py
|
|--> Tool2
|
|--> Object2.py
|
|--> __init__.py
where Project_name, Tool1 and Tool2 are folders. Object2 contains a class named 'House'. How can I use the class 'House' in Object1? I tried the following:
from Tool2.Object2 import House
but I receive error message 'No module named 'Tool2'.
What am I doing wrong? All init.py files are empty, should I change that?

1
If you are using VS code, the easiest fix is to start your script being executed like the following.
import sys
sys.path.append('/PATH/TO/Project_Name')
import Tool1.Object1
...
2
Or, you can add the environment variable PYTHONPATH to settings.json (can be found ctrl + shift + P then type settings.json)
"terminal.integrated.env.linux": {
"PYTHONPATH": "/PATH/TO/Project_Name/"
}
In this way, you can just
import Tool1.Object1
sys.path and PYTHONPATH will do the same for you.

You need to export PYTHONPATH to your Project_Name so that the interpreter knows the specific folder(s) to look up.
export PYTHONPATH=path/to/your/project
For example:
object1.py
from tool2.object2 import House
house = House()
house.print_message("Hello World!!!")
object2.py
class House(object):
def __init__(self):
pass
def print_message(self, text):
print(text)
Outputs before and after exporting PYTHONPATH
$ python tool1/object1.py
Traceback (most recent call last):
File "tool1/object1.py", line 2, in <module>
from tool2.object2 import House
ImportError: No module named tool2.object2
$ pwd
/Users/.../StackOverflow
$ export PYTHONPATH=/Users/.../StackOverflow
$ python tool1/object1.py
Hello World!!!
$ python object1.py
Hello World!!!

Python Submodule is not getting imported

This is the file path for my Pydev project in Eclipse:
project
|
+----tests
| |
| +----subtests
| | |
| | +----__init__.py
| | |
| | +----test1.py
| |
| +----__init__.py
| |
| +----test2.py
|
+----mods
|
+----__init__.py
|
+----submods1
|
+----__init__.py
|
+----submods2
|
+----__init__.py
|
+----a.py
|
+----b.py
|
...
|
+----z.py
test1 and test2 are exactly the same, all of the init files only have comments in them. The tests are getting the modules from the mods directory and those modules dependencies. When I run test1, all of the modules are found, but test2 always unable to find the same module (let's call it "z.py") in submods2. But somehow it's able to find the rest of the modules. It's not that it's unable to import something in z.py, it just cannot find the file at all.
test2:
>>> from mods.submod1.submod2 import z
exec exp in global_vars, local_vars
File "<console>", line 1, in <module>
ImportError: cannot import name z
>>> from mods.submod1 import submod2
>>> hasattr(submod2, 'z')
False
The only difference in the sys.path during the two tests are the directories that tests are located in, project/tests/subtests for test1 and project/tests for test2.
I cannot figure out why test2 is unable to import z.py but test1 can and test2 can import the rest of the modules.

To help diagnose the issue, do:
from mods.submod1 import submod2
print(submod2)
My guess is that it's not the module you're expecting.
What Python version are you using?

I think I found my solution to this. In my Run Configurations for test2, the Working directory in the Arguments tab had a custom path ${workspace_loc:project/tests/}, I switched it to the default path ${project_loc:/selected project name} and that seems to be fixing the issue. While I don't understand how this fixed the problem, the result is good enough for me.

List modules in namespace package

I'm trying to get Python to list all modules in a namespace package.
I have the following file structure:
cwd
|--a
| `--ns
| |--__init__.py
| `--amod.py
|--b
| `--ns
| |--__init__.py
| `--bmod.py
`--c
`--ns
|--__init__.py
`--cmod.py
Each __init__.py defines it's package as a namespace package by having the following line:
__import__('pkg_resources').declare_namespace(__name__)
The amod module contains a class A, bmod contains another class B and cmod contains the C class.
When having a clean environment the following happens:
>>> import inspect, sys, glob
>>> sys.path += glob.glob('*')
>>> import ns
>>> inspect.getmembers(ns, inspect.ismodule)
[]
As you can see, the modules are not listed.
Now when I import the modules manually and then call the inspect again:
>>> inspect.getmembers(ns, inspect.ismodule)
[('amod', <module 'ns.amod' from 'a/ns/amod.pyc'>), ('bmod', <module 'ns.bmod' from 'b/ns/bmod.pyc'>), ('cmod', <module 'ns.cmod' from 'c/ns/cmod.pyc'>)]
Now I'd like the inspect call to work without importing the modules manually, so how can I achieve this?