Hey all -- I am pulling my hair out with relative imports in Python. I've read the documentation 30 times and numerous posts here on SO and other forums -- still doesn't seem to work.
My directory structure currently looks like this
src/
__init__.py
main.py
components/
__init__.py
expander.py
language_id.py
utilities/
__init__.py
functions.py
I want expander.py and language_id.py to have access to the functions module. I run python main.py which accesses the modules just fine with from components.expander import * and components.language_id import *.
However, the code inside expander and language_id to access the functions module:
from ..utilities.functions import *
I receive this error:
ValueError: Attempted relative import beyond toplevel package
I have gone over it a bunch of times and it seems to follow the documentation. Anyone have any ideas of what's going wrong here?
Nevermind, I solved it:
src/
main.py
mod/
__init__.py
components/
__init__.py
expander.py
language_id.py
utilities/
__init__.py
functions.py
main.py then refers to the subpackages as:
from mod.components.expander import *
from mod.utilities.functions import *
expander.py and language_id.py have access to functions.py with:
from ..utilities.functions import *
But the interesting thing is that I had a text file inside the components directory that expander.py uses. However, at runtime it couldn't locate the file even though it was in the same directory. I moved the text file to the same directory as main.py and it worked. Seems counter-intuitive.
Related
Project level:
├─project_root
├───application
│ └─── target.py
│ └─── utils.py
├───tests
├ └─── test_*.py
When I run py -m unittest discoveron the CLI from the project_root directory it discovers all tests. However, the tests cannot run as they cannot find the utils.py (the target.py relies on this).
Anyone know how to fix this?
This is a scope issue, you need to use relative imports, as test_*.py is located within a folder that doesn't contain utils.py in its scope.
So, two options, copy utils.py into the directory or use relative imports to find and import the library.
Will update answer with example of relative import when I can find one - its been a while since I've had to use them.
EDIT - Found one.
Haven't tested it, but the generic format for relative import of this nature (using the scope you showed above would be:)
from .application import utils
The dot out the front of "application" : ".application" tells the python interpreter to go one directory back, and locate a module called "application" (that other folder) and then to import the file from there.
I have a directory structure like this.
Chatbot/
utils/
abc.py
projects/
proj1/
utils/
__init__.py
data_process.py
components/
class1.py
I have two utils folders in my structure, one at the top level and one inside my projects folder.
Now I want to import data_process.py file inside class1.py. So I tried like this
from utils.data_process import DataProcess
but it is referencing the top-level utils folder and even VSCode is not recognizing it. I tried creating __init__.py file inside the utils folder but still did not work.
I tried with empty __init__.py and then placing this content
from . import data_process
__all__ = ['data_proces']
then
from .data_process import DataPreprocess
__all__ = ['DataPreprocess']
then I tried
from ..utils.data_process import DataProcess
VSCode is recognizing this but it is not working and throws the error
ValueError: attempted relative import beyond top-level package
Even I tried changing the name utils to some other name but still the same issue
How can I solve this?
A python module is defined by a special file called __init__.py and this file should be present in all the subdirectories. So, your file structure would be:
Chatbot/
__init__.py
utils/
__init__.py
abc.py
projects/
__init__.py
proj1/
__init__.py
utils/
__init__.py
data_process.py
components/
__init__.py
class1.py
class2.py
Now, you can do a relative import like:
Use . for importing something within the same directory. Example:
# file Chatbot/projects/proj1/components/class2.py
from .class1 import *
Similarly, use .. for two-level, and ... for three-level, and so on!
For instance:
# file Chatbot/projects/proj1/components/class2.py
from ..utils.data_process import * # import from data_process.py
# there will be four "." earlier I made a counting mistake
from ....utils.abc import * # import something from abc.py
Coding Convention
When writing a python module/package, you might want to follow PEP8 Convention.
Moreover, I believe you are trying to do different projects using your package Chatbot, is that correct? Then, it is a good practice that you set PYTHONPATH for Chatbot and do all the projects, and imports seperately.
EDIT: Uploading Dummy File
I'm able to work on this project seamlessly even with using two utils directories. Project Structure:
main file and its output:
# file main.py
from chatbot.utils.abc import hello as a1
from chatbot.projects.proj1.components.class1 import super_hello
from chatbot.projects.proj1.utils.data_process import hello as b1
print(a1(), b1())
print(super_hello())
Similarly, if you have chatbot under PYTHONPATH you can call the project from anywhere from your device.
Code details and files are added into my github account for details.
sys.path is where Python searches to find modules and packages and It does it in order.
So you can put ...../proj1/ at the beginning of the list, when python start searching it will find the utils folder in that path first !
import sys
sys.path.insert(0, r'...../proj1/')
But this cause another problem, python always find utils in that folder first, that's not what you want.
solution number 1
Absolute import:
from Chatbot.projects.proj1.utils.data_process import DataProcess
solution number 2
use relative imports which is mentioned by #Mr.Hobo.
There are lots of similar questions, but none of them seem to help me with this.
I have the following file structure:
rest_api
__init__.py
utils
__init__.py
config.py
foo
__init__.py
foo.py
I'd like to import the Config class from config.py, into foo.py. I attempt to run:
pipenv run python rest_api/foo/foo.py
I have tried numerous things, including adding the parent folder to PATH and using relative imports, but the import line always fails:
from rest_api.utils.config import Config
I get the following error:
ModuleNotFoundError: No module named 'rest_api'
I should mention that the __init__.py files are all empty. Not sure if they are neccessary, since I have Python 3.8.2.
My file stucture is :
top\
my_package\
__init__.py
functions.py
scripts\
test.py
main.py
I would like to import the content of functions.py in test.py.
In main.py, I can import functions.py with from my_package.functions import ....
I was expecting to be able to import functions.py in test.py with something like from ..my_package.functions import ... but it raises the following error :
SystemError: Parent module '' not loaded, cannot perform relative import
top directory shoudln't be a package because I want to be able to run main.py without being running a script in a package.
What is the proper/pythonic way import functions.py from test.py ?
I could add my_package to the PYTHONPATH, but my code would be less portable. I'm using Python 3.5
This is happening because as far as python is concerned my_package and scripts directories are not related to each other.
To solve your problem you can add __init__.py to your top directory and try using
from ..function import *
I know a lot of questions have been posted about the intra-package importing. I want to know whether the below is the way for Python 2.7 too.
Source/
anomalyCheck/
__init__.py
DLthput.py
ULPowerStats.py
ULThput.py
config/
__init__.py
configure.py
parserTools/
__init__.py
logParser.py
utilities/
__init__.py
plotLogResults.py
__init__.py
lteDebugger.py
---- lteDebugger.py----
import parserTools.logParser
import anomalyCheck.DLthput
import utilities.plotLogResults
import configuration.TDDFDDconfiguration
import anomalyCheck.ULthput
### Work done after here ####
------DLThput.py------
from ..utilities.plotLogResults import *
from ..parserTools.logParser import *
### Work done after here ####
------ULThput.py-------
from ..parserTools.logParser import *
from ..utilities.plotLogResults import *
Error :
Upon running the lteDebugger.py file, the error is
ValueError: Attempted relative import beyond toplevel package
File "C:\Users\manojtut\Desktop\Project_LTE_SVN\Source\lteDebugger.py", line 2, in
import anomalyChecker.DLthput
File "C:\Users\manojtut\Desktop\Project_LTE_SVN\Source\anomalyChecker\DLthput.py", line 1, in
I've read almost all available docs and Guido's guide for intra-package importing. Also, I guess I've everything exactly in the right place. Am I missing something here? Please point out. Thanks a lot in advance. :) :)
Edit 1: The Problem mentioned is solved by Amber's answer. So, lteDebugger.py is working fine by importing all other modules. Now, another problem is that, I am unable to solve is that... when I want to compile/interpret(whatever u want to call) the DLThput.py/ULthput.py , it is showing the same error as above ... ValueError: Attempted relative import beyond toplevel package. Do I have any solution other adding paths to sys hacks? I really don't want to do that unless it's the only thing to do.
So, how can I dodge this?
You're running lteDebugger.py, which means that any "packages" must be at least one level lower in the directory tree - they need to be contained inside a folder for Python to recognize them as packages rather than modules (and thus for relative imports to work).
anomalyCheck is recognized as a package, but its parent directory is not (because that's where lteDebugger.py is), and thus you aren't allowed to use relative imports to go up to that parent directory.
One way you could fix this is by moving everything except lteDebugger.py into a subdirectory, e.g.:
Source/
debugger/
anomalyCheck/
__init__.py
DLthput.py
ULPowerStats.py
ULThput.py
config/
__init__.py
configure.py
parserTools/
__init__.py
logParser.py
utilities/
__init__.py
plotLogResults.py
__init__.py
lteDebugger.py
and then lteDebugger.py would do things like import debugger.anomalyCheck.DLthput.py.