I have:
project
|--__init__.py
|--...
+--package1
| |--__init__.py
| |--...
+--dbs
|--...
Where project is a collection of packages to be imported and used by the user. The names are just an example.
Now, some files under project/package1/ need to access some files in project/dbs/, but depending on where the user placed the project folder and from where he or she imported it, path/to/project/dbs/file is not the same. Because of this issue and for other purposes I thought I should have a variable projectroot defined as the root of the project, relative to where it was imported from. For instance:
If the user has ~/Desktop/project and imported it from ~/ then projectroot would be Desktop/project, and path/to/project/dbs/file would be "%s/dbs/file" % (projectroot). Is this really needed? If so, what is the best way of doing it?
I tried using this in project/pathtfinder.py and used project/__init__.py as
import pathfinder
projectroot=pathfinder.module_path()
import package1
But when I run import project I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "project/__init__.py", line 3, in <module>
import package1
File "project/package1/__init__.py", line 5, in <module>
db = sqlite3.connect("%s/dbs/main.db" % projectroot)
NameError: name 'projectroot' is not defined
And if use print projectroot just after defining it I get Desktop/project, as expected.
You need to add the path of project to the PYTHONPATH
This is a pattern I use for my Flask apps, but the same should be ideal for you:
Assume the following is in a file under helloworld/app.py
import sys
import os
# Add this app to the path so imports work as expected
APP_PATH = os.path.join(os.path.dirname(__file__), '../')
sys.path.append(APP_PATH)
from helloworld.utils import a_utility
The convention explains itself, but it allows for the imports to occur from the "root" of the project, regardless of where the file is placed. You'll want to place that APP_PATH code perhaps in your init.py, or similar.
Related
Given the following project structure:
test/
data/
__init__.py
a/
data/
__init__.py
main.py
__init__.py
In test/data/__init__.py
from pathlib import Path
DATA_DIR = Path(__file__).parent
In main.py
from data import DATA_DIR
if __name__ == "__main__":
print(DATA_DIR)
When running from the terminal, it works fine. When running from PyCharm, it gives the following error:
Traceback (most recent call last):
File "/Users/<USER>/code/test/a/main.py", line 1, in <module>
from data import DATA_DIR
ImportError: cannot import name 'DATA_DIR' from 'data'
(/Users/<USER>/code/test/a/data/__init__.py)
Has anyone found a way to get around this?
Solution
In PyCharms Choose target to run configuration, change from Script path to Module name and it should work as expected.
test isn't in the installed modules. It needs to be a location that sys.path knows about.
https://docs.python.org/3/library/sys_path_init.html#sys-path-init might give you the information you need to add test to your sys.path.
After you add test to the installed modules you might also have to remove test/a since otherwise from data import ... will have a conflict between the two locations. Both of them have the same name and with both directories accessible only the first one in the list will be seen. The order of the values in sys.path matters.
I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.
The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file
I am currently having an issue with importing files from other directories in my python project.
My current file structure is
Project
- Backend
- Config
+ __init__.py
+ databaseConfig.py
- DataAccess
+ __init__.py
+ sqlConns.py
- __init__.py
- api.py
- main.py
- setup.py
What I am trying to do is import /Config/databaseConfig.py file into /DataAccess/sqlConns.py file. I get the following error when trying to run the sqlConns.py file
PS C:\source\repos\aaStats\aaStats> py .\Backend\DataAccess\sqlConns.py
Traceback (most recent call last):
File "C:\source\repos\aaStats\aaStats\Backend\DataAccess\sqlConns.py", line 2, in <module>
import Config.databaseConfig
ModuleNotFoundError: No module named 'Config'
I have also tried using relative imports, but I am met with another error.
PS C:\source\repos\aaStats\aaStats> py .\Backend\DataAccess\sqlConns.py
Traceback (most recent call last):
File "C:\source\repos\aaStats\aaStats\Backend\DataAccess\sqlConns.py", line 2, in <module>
from ..Config import databaseConfig as dbcfg
ImportError: attempted relative import with no known parent package
Config/databaseConfig.py contains database configuration parameters that I want to reference is various places in my project. It isn't a huge deal if I had to move this single file in order to get it to be referenced properly, but I will want to use structures like this for files later on in my project.
Here are some details about my files:
/Config/__init__.py
from . import databaseConfig
/DataAccess/__init__.py
from . import sqlConns
Backend/__init__.py
from . import DataAccess
from . import Config
Backend/setup.py
from setuptools import setup, find_packages
setup(
name='aaStatsApi',
version='0.1.0',
packages= ['DataAccess','Config'],
install_requires=[
'fastapi==0.63.0',
'uvicorn==0.13.4',
'requests==2.25.1',
'pyodbc==4.0.30',
]
)
Check out this post.
The fact that you can't perform relative imports so easily is by design, for better or for worse. The ideal way is have your main script in the root (Backend) directory and do all your calls from there. The function that has __name__ == __main__ is your calling function. If you do not directly call Calls.py or Configs.py from a console, but are calling them from another main function within your root directory, you should be able to place the following into Conns.py:
# FILE: DataAcess\sqlConns.py
from Config.dataBaseConfig import * # or whatever you need to import
Again, the key is to ensure that your starting point in from your root project directory.
NOT RECOMMENDED:
For risk of getting downvoted, and I do not recommend this, but you could append the relative path to your calling class:
import sys, os
sys.path.append(os.path.abspath("../Config"))
from sqlConns import * # or whatever
sys.path.pop() # clear sys.path
I have multiple modules in my project and specify some execution point. But when I try to import files from submodules, it doesn't work.
So, how to specify submodules to execute from selected execution file?
project
--bin
---- executeFile
--modules
---- __init__.py
----fileA.py
in executeFile, I try:
from ..modules.fileA import *
but get the error:
Traceback (most recent call last):
File "./bin/muexecute", line 10, in <module>
from ..modules.os import *
SystemError: Parent module '' not loaded, cannot perform relative import
I found solution.
The problem was in my opinion about using init.py.
I placed in executable scripts path to including and it works fine
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
So you are having trouble defining your relative path, correct? Try the following:
from sys import path
path.append('C:\\realative_path')
from function_file import required_function
Hope that helps.
All modules you want to import should in in your PYTHONPATH. Therefore there is no hierarchy.
In your case It seems to me that an __init__.py is missing from your project's main folder (with all the models included), so executefile doesn't know about your modules.
I've read through a couple of similar questions, notably this one about imp.load_module which seems to be close to what I want, but I can't understand why I'm still getting ImportErrors. Here is my folder hierarchy:
program\
__init__.py
main.py
thirdparty\
__init__.py
css\
__init__.py
css.py
utils\
__init__.py
http.py
In main.py I have the following code. This is intended to search the thirdparty\ directory and load each module it finds. Each module is in its own separate directory.
import os
import imp
for root, dirs, files in os.walk("thirdparty"):
for source in (s for s in files if s.endswith(".py")):
name = os.path.splitext(os.path.basename(source))[0]
m = imp.load_module(name, *imp.find_module(name, [root]))
The problem is that css.py happens to use its own subfolder that it loads stuff off of, utils. It has a line in it that says:
from utils import http
And that is where it fails. I get this error when I run main.py.
Traceback (most recent call last):
File "main.py", line 7, in <module>
m = imp.load_module(name, *imp.find_module(name, [root]))
File "thirdparty/css/css.py", line 1, in <module>
from utils import http
ImportError: No module named utils
I'm stumped. css.py is self contained in its own folder, and when I run css.py separately it imports utils just fine. What is causing this?
Maybe you can solve this by changing the import to:
from .utils import http
Or by adding the folder you import to the Python Path:
sys.path.append(os.path.join(root, source))
When you import modules in thirdparty, the place Python looks for modules is still the main directory. The initial import works, because you give the right path to imp.find_module, but after that Python has no idea where to look for the modules.