I have a project which I want to structure like this:
myproject
├── api
│ ├── __init__.py
│ └── api.py
├── backend
│ ├── __init__.py
│ └── backend.py
├── models
│ ├── __init__.py
│ └── some_model.py
└── __init__.py
Now, I want to import the module some_model.py in both api.py and backend.py. How do I properly do this?
I tried:
from models import some_model
but that fails with ModuleNotFoundError: No module named 'models'.
I also tried:
from ..models import some_model
which gave me ValueError: attempted relative import beyond top-level package.
What am I doing wrong here? How can I import a file from a different directory, which is not a subdirectory?
Firstly, this import statement:
from models import some_model
should be namespaced:
# in myproject/backend/backend.py or myproject/api/api.py
from myproject.models import some_model
Then you will need to get the directory which contains myproject, let's call this /path/to/parent, into the sys.path list. You can do this temporarily by setting an environment variable:
export PYTHONPATH=/path/to/parent
Or, preferably, you can do it by writing a setup.py file and installing your package. Follow the PyPA packaging guide. After you have written your setup.py file, from within the same directory, execute this to setup the correct entries in sys.path:
pip install --editable .
Unfortunately, Python will only find your file if your file is in the systems path. But fear not! There is a way around this!
Using python's sys module, we can add a directory to the path just while Python is running, and once Python stops running, it will remove it from the path.
You can do this by:
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import [file]
It is important to import sys and set the directory path before you import the file however.
Good luck!
Jordan.
I would lay out two approaches:
Simply import some_model via absolute importing:
from myproject.models import some_model
Note that the myproject should be treated as an module (i.e. having __init__.py)
Or
You can add the previous path to the sys.path which I use in such parallel level modules:
import sys
sys.path.append('../')
from models import some_model
Related
I am trying to import a util package one directory up from where my code is, but I get an ImportError which I don't understand.
I have a number of different variations on the import syntax in Python, none of which are working.
There are a number of similar questions on Stack Overflow, but none have helped me understand or fix this issue.
Of the top of my head, I have tried the following variations:
import util
import ..util
from .. import util
from ..util import parser
from AdventOfCode2022 import util
from ..AdventOfCode2022 import util
from ...AdventOfCode2022 import util
Most of these I guessed wouldn't work, but I tried them anyway to be sure.
Error message:
ImportError: attempted relative import with no known parent package
Directory structure:
.
├── day03
│ ├── input.txt
│ ├── part1.py
│ ├── part2.py
│ └── test_input.txt
└── util
├── __init__.py
└── parser.py
I just want to import my util package from any "day0*/" directory - not sure why Python makes it so hard!
Two options:
Add the full path to ./util/ to your PYTHONPATH environment variable.
For example on Bash, your ~/.bashrc might have export PYTHONPATH="${PYTHONPATH}:/Users/foobar/projects/advent-of-code/util/".
Add sys.path.append('/path/to/application/app/folder') before the import.
The other solutions don't work because:
day03 and the parent directory are not modules with their own __init__.py. Lines like from ..util import parser only work if everything involved is a module.
You are presumably running the code from within ./day03/.
View this as 'I have a bunch of independent Python projects (day01, day02 etc) that all want to share a common piece of code I have living in a different project (util) that lives somewhere else on my computer.'
I have the following folder structure
parent Folder
├── __init__.py
├── app_folder
│ └── file_1.py
│ └── __init__.py
└── Model_Folder
└── model.py
└── __init__.py.py
I have a class_object in model.py and I want to use it in file_1.py
I have setup Model_Folder/init.py as follows:
from .model import class_object
I have setup Parent\ Folder/init.py as follows:
from Model_folder import class_object
now when I call the class_object from file_1.py I use this code:
from .. import class_object
but i receive this error:
ImportError: attempted relative import with no known parent package
Its worth mentioning that I have created a test file under the parent folder and called the class_object and it worked fine with no errors but I cant do the same for file_1.py. And the IDE does seem to have read it successfully without flagging any errors.
Moreover, I tried the following as well with the same error
from ..Model_Folder.model import class_object
from ..Model_Folder import class_object
so how do I import class_object in file_1? and what am I doing wrong?
I've had similar issues and I've created an experimental new import library ultraimport that allows to do file system based imports to solve your issue.
In your file_1.py you would then write:
import ultraimport
class_object = ultraimport('__dir__/../Model_Folder/model.py', 'class_object')
This will always work, no matter how you run your code or what is your current working directory. You would also not need the __init__.py files but you could use them of course if you wanted to.
My project structure is the following:
.
└── project name
├── project name
│ ├── __init__.py
│ ├── module.py
│
├── PACKAGE_A
│ ├── __init__.py
│ ├── PACKAGE_A.py
│ ├── module_a.py
│
In PACKAGE_A.py
from module_a import Some_Class
a = Some_Class()
class Another_Class:
# class code here
In module.py
"""
Notes
-----
https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
"""
# Standard library imports
import os, sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
# Local application imports
from PACKAGE_A.PACKAGE_A import Another_Class
from PACKAGE_A.module_a import some_function
While module_a.py and PACKAGE_A.py run without problem, running module.py fails with:
Traceback (most recent call last):
File "path\to\project name\project name\module.py", line 12, in <module>
from PACKAGE_A.PACKAGE_A import Another_Class
File "path\to\project name\PACKAGE_A\PACKAGE_A.py", line 1, in <module>
from module_a import Some_Class
ModuleNotFoundError: No module named 'module_a'
What am I doing wrong here?
You need to change your import statement in PACKAGE_A.py from:
from module_a import Some_Class
to:
from PACKAGE_A.module_a import Some_Class
The reason is that you are adding the path\to\project name\ to sys.path, but you have no module_a.py in path\to\project name\, and path\to\project name\PACKAGE_A (where module_a.py resides) is not in sys.path.
As for why you succeed in running everything in PACKAGE_A, it's because Python adds the directory containing the script you are running to the list (as explained by gaFF).
I would recommend you read a bit more about python imports, if the doc seems too cluttered, you can check this link.
This is a personal preference, but I find it simpler to add the root directory of the project to the PYTHONPATH environment variable and then running all the scripts from that directory's level and changing the import statements accordingly. In your example, the root directory would be path\to\project name\.
import search for your packages in specific places, listed in sys.path. See the doc for the full details.
The current directory is always appended to this list, that's why you succeed to run everything inside PACKAGE_A. But from project name, there is no way to know where to find PACKAGE_A.
Solutions include:
use relative import
always run from the root directory (and start all your imports from the root directory)
add the root directory to the environment variable PYTHONPATH (same)
use a tool which sets PYTHONPATH when you enter your virtual environment (same)
...
and depends on your project and your needs.
I am looking for a way to import modules so that I can run a script both from the sub-folder project/v0 and from the root folder project
My file structure in python 3.6 (which is why there are no init files)
project
├── api.py
├── v0
│ ├── SearchEngine.py => contains SearchEngine class
│ └── SearchEngineBE.py
My SearchEngineBE.py module contains
from SearchEngine import SearchEngine
My api.py module contains
from v0.SearchEngineBE import SearchEngineBE
step1: When from project/v0 I run python3 SearcheEngineBE.py my module is correctly imported and everything goes well.
step2: However, when from project I run python3 run api.py I get the error:
Traceback (most recent call last):
File "api.py", line 3, in <module>
from v0.SearchEngineBE import SearchEngineBE
File "/xxx/project/v0/SearchEngineBE.py", line 3, in <module>
from SearchEngine import SearchEngine
ModuleNotFoundError: No module named 'SearchEngine'
How can I fix this so that both step 1 and step 2 would work ?
Non-relative imports are searched by the interpreter in the current directory (and any additional search paths).
You could use relative imports in your SearchEngineBE.py file to let the interpreter know you want the relative module, and not a module off the import path:
# SearchEngineBE.py
from .SearchEngine import SearchEngine
The . lets the interpreter know that you are referencing a module relative to the current module. You'll need at least a blank __init__.py file in the same directory as SearchEngine.py for relative imports to work though:
The __init__.py files are required to make Python treat directories
containing the file as packages.
https://docs.python.org/3/tutorial/modules.html
See this guide for some more discussion: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html#absolute-vs-relative-import
Edit:
Without a root package, this won't work. See this post for an alternative approach: Importing modules from a neighbouring folder in Python
Add __init__.py to make it a package.
(it could be empty file : __init__.py)
project
├── __init__.py
├── api.py
├── v0
│ ├── SearchEngine.py => contains SearchEngine class
│ └── SearchEngineBE.py
Edit 1:
try :
from project.v0.SearchEngine import SearchEngine
I've attempted a few different techniques trying to do something that to me seems doable but I guess I am missing some gotchas about python (using 2.7 but would like this to work also for 3.* if possible).
I am not sure about terminology like package or module, but to me the following seems quite a "simple" doable scenario.
This is the directory structure:
.
├── job
│ └── the_script.py
└── modules
├── __init__.py
└── print_module.py
The content of the_script.py:
# this does not work
import importlib
print_module = importlib.import_module('.print_module', '..modules')
# this also does not work
from ..modules import print_module
print_module.do_stuff()
The content of print_module:
def do_stuff():
print("This should be in stdout")
I would like to run all this "relative paths" stuff as:
/job$ python2 the_script.py
But the importlib.import_module gives various errors:
if I just use 1 input parameter ..modules.print_module, then I get: TypeError("relative imports require the 'package' argument")
if I use 2 input parameters (as in the example above), then I get: ValueError: Empty module name
On the other hand using the from ..modules syntax I get: ValueError: Attempted relative import in non-package.
I think the __init__.py empty file should be enough to qualify that code as "packages" (or modules? not sure about the terminology), but it seems there's something I am missing about how to manage relative paths.
I read that in the past people was hacking this using the path and other functions from import os and import sys, but according to the official docs (python 2.7 and 3.*) this should not be needed anymore.
What am I doing wrong and how could I achieve the result of printing the content modules/print_module.do_stuff calling it from a script in the "relative directory" job/?
If you follow the structure of this guide here: http://docs.python-guide.org/en/latest/writing/structure/#test-suite (highly recommend reading it all, it is very helpful) you will see this:
To give the individual tests import context, create a tests/context.py file:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import sample
Then, within the individual test modules, import the module like so:
from .context import sample
This will always work as expected, regardless of installation method.
Translated in your case this means:
root_folder
├── job
│ ├── context.py <- create this file
│ └── the_script.py
└── modules
├── __init__.py
└── print_module.py
In the context.py file write the lines shown above, but import modules instead of import samples
Finally in your the_script.py: from .context import module and you will be set to go!
Good luck :)
I found a solution using sys and os.
The script the_script.py should be:
import sys
import os
lib_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '../modules'))
sys.path.append(lib_path)
# commenting out the following shows the `modules` directory in the path
# print(sys.path)
import print_module
print_module.do_stuff()
Then I can run it via command line no matter where I am in the path e.g.:
/job$ python2 the_script.py
<...>/job$ python2 <...>/job/the_script.py
If you are not sure about terminology go to very nice tutorials:
http://docs.python-guide.org/en/latest/writing/structure/#modules
and
http://docs.python-guide.org/en/latest/writing/structure/#packages
But for your structure:
.
├── job
│ └── the_script.py
└── modules
├── __init__.py
└── print_module.py
just say in the the_script.py:
import sys
sys.append('..')
import modules.print_module
This will add parent directory to PYTHONPATH, and python will see directory 'parallel' to job directory and it will work.
I think that at the most basic level it is sufficent to know that:
package is any directory with __init__.py file
module is a file with .py, but when you are importing module you omit extension.