Develop Reference

relative reference in python

am beginner in python and working on multi folder structure and struggling to refer the files using relative path. tried in different ways to refer the files in other folders but ended up in error "attempted relative import with no known parent package". Can somebody explain how the relative paths will work in python?
here is the example what am trying to achieve..
folder structure
Main
source
Function1
inputs
1.py
2.py
uts
test_1.py
code_1.py
code_2.py
Now inside test_1.py am trying to refer code_1 using relative path like
from ..code_1 import Code
from .code_1 import Code
but, ended up in error..
some where I found that we need to have init.py in each folder to use relative path and I created empty init.py in each folder but, no use..
I may get the answer but more importantly need to know how this relative path works in python.
Thanks in advance

I've also had issues with relative imports in Python and thus I've created an experimental, new import library: ultraimport
It gives you more control over your imports and lets you do file system based imports.
You could then write in your test_1.py:
import ultraimport
Code = ultraimport('__dir__/code_1.py', 'Code')
This would import code_1.py from the same directory as test_1.py.
This will always work, no matter how you run your code.

ImportError: attempted relative import with no known parent package STILL NO SOLUTION

I have looked at I think 5 different answers to this problem, yet none of them have worked for me yet. For reference, I've looked through all of these posts:
Relative imports for the billionth time
Attempted relative import with no known parent package
"Attempted relative import with no known parent package"
From what I've gathered, there are two solutions to this problem:
Move the .py file you're trying to import functions from into the same directory as the script you're trying to run (this works, but it is not a good solution, I should be able to import from a parent directory without this error)
Create a __init__.py file in the directory of the .py file you're trying to import from, and use import package_name to it. (I have tried this, but same issue)
Here is my project's structure:
I'm trying to run the test.py script, which (attempts) to import the function add_technical_indicators from the add_technical_indicators.py file. My import statement looks like this:
from ..utils.add_technical_indicators import add_technical_indicators
Looking at the folder structure again, I have to go UP one directory, then into the utils folder to bring in the add_technical_indicators .py file, and finally the function add_technical_indicators.
Here's what I have tried so far:
from ..utils.add_technical_indicators import add_technical_indicators
from .utils.add_technical_indicators import add_technical_indicators
from utils.add_technical_indicators import add_technical_indicators (this doesn't work of course because add_technical_indicators is not in the same folder as the script being run)
Created an __init__.py file in the utils folder that reads import add_technical_indicators
Created an __init__.py file in the misc folder that reads import test
None of it works. I need a concise and actionable answer as to why this is still not working. I'm running Python 3.7.9, Windows 10, and VS code in case that matters.
I have looked through previous, repeat answers but none of them have worked for me, so although this IS a duplicate question, please do not close it until I have a solution because linking to the already "answered" questions didn't help me.

short solution
In test.py import as from utils.add_technical_indicators import add_technical_indicators
and run as python -m misc.test (not test.py) in parent directory(in STOCK_PEAKS_ADN_THROUGHS_2)
explained
python cannot import files from upper directory of your working directory unless you add them to PATH, so from ..utils.add_technical_indicators import add_technical_indicators won't work.
python finds files starting from your working directory, so from utils.add_technical_indicators import add_technical_indicators in test.py can find utils/add_technical_indicators.py if you run script at parent directory.
python -m misc.test will run misc/test.py in parent directory. Note that python misc/test.py will run script in misc directory and will give you same error.

You did the right thing here: from ..utils.add_technical_indicators import add_technical_indicators
The main issue must be in the imports of utils/add_technical_indicators.py file
Did you try importing stuffs inside add_technical_indicators.py relative to itself (if so, those imports only work when add_technical_indicators.py is run instead of test.py)

In contrast to what minolee has said Python actually can import files from upper directories. Python has a built-in SourceFileLoader that can load Python code from any file in the file system, also from any upper directory of your working directory.
It requires some boilerplate code, so I have decided to wrap this approach into an experimental import library: ultraimport
ultraimport can do file system based imports. In your test.py you could then write:
import ultraimport
add_technical_indicators = ultraimport('__dir__/../utils/add_technical_indicators.py')
This will always work, no matter what is your current working directory and no matter what is your current sys.path. You don't need to create __init__.py files and it also works if you run the code as a script or as a module.
One caveat when importing files like this is if the imported code contains further relative imports. ultraimport has a built-in preprocessor to automatically rewrite subsequent relative imports so they continue to work.

Unable to import class even though I already have __init__.py files

I'm trying to import a class in a different directory to another file, but can't seem to get it to work. I know this question has been asked a lot and I have looked through multiple stackoverflow solutions and at https://docs.python.org/3/tutorial/modules.html#packages
1: Importing files from different folder
2: import python file in another directory failed
I want to try to just use the method containing just __init__.py file instead of doing an import sys
My directory structure is as follows:
django_vue/
__init__.py
devices/
__init__.py
models.py
lib/
__init__.py
my_file.py
I'm trying to import the class Device from /django_vue/devices/models.py to /django_vue/lib/my_file.py by:
from devices.models import Device
However when I do that I still get the error:
from devices.models import Device
ModuleNotFoundError: No module named 'devices'
I'm not sure what I'm dong wrong since I already have the __init__ file in both directories. Any help is appreciated. Also I'm running python 3.6.

This is the folder structure I'm working with.
.
└── django_vue
├── devices
│   └── models.py
└── lib
└── file.py
When you run
$ python file.py
python has no way of knowing what's outside the directory.
python can't go back and then into devices/ just like that.
The easiest way to solve this would be to add the folder devices/ to sys.path. When python imports a module, it searches for the module from sys.path. Adding the path to devices/ would make it available for imports.
Here are my files.
# models.py
Device = 'device'
# file.py
import sys
sys.path.append('..') # adds the parent dir (which is django-vue/) to path
# django-vue dir has devices/ so now this is available for imports
# importing this works now
from devices.models import Device
print(Device)
Output
django_vue/lib$ python3 file.py
device

Think about it your are inside my_file.py and import something called devices.
How can python know where the name devices has come from.
It won't search your entire Drive for that module/package
Relative Import
use a relative import instead. write from ..devices.models import Device. This is telling python to go up one directory to the parent directory and that's where it will find the devices package. Your lib module should now work as a module
If you however run the my_file.py package directly (as in python C:/django_vue/lib/my_file.py)
You will still get an error. Not the same error; the new error will be something like
ImportError: attempted relative import with no known parent package
This is happening because you are actually running my_file.py
If you think about it why would you want to run my_file.py by itself when it is clearly a part of a package. Maybe you are just testing to see if the import works when you use your package. The problem with this is that it makes it seem like your packages relative imports don't work even though this actually works.
Create a main.py in django_vue and write from lib import my_file. This will run your my_file.py and you will notice there is no error.
What's happening here
Have you heard of __package__?
if you put print(str(__package__)) in your my_file.py and run my_file.py directly you will see that it prints None.
However if you run main.py (that you just created) you will see that when It reaches my_file.py, __package__ will actually be defined to something.
Ahhh... you see now it all makes sense; The error you originally got said something about no known parent package. If __package__ is undefined that means there is no relative parent package because the file was obviously run directly instead of as part of a package.
Consider Absolute imports
you also might want to consider using absolute imports because if you are working on the package you might change it directory structure while developing. so you need to keep changing the import references on the affected files.
Although you can find IDE's with python extensions that automatically to this as you change your directory. I believe VS Code does this automatically.

Replace the __init__ files with __main__.

Importing correctly with pytest

I just got set up to use pytest with Python 2.6. It has worked well so far with the exception of handling "import" statements: I can't seem to get pytest to respond to imports in the same way that my program does.
My directory structure is as follows:
src/
main.py
util.py
test/
test_util.py
geom/
vector.py
region.py
test/
test_vector.py
test_region.py
To run, I call python main.py from src/.
In main.py, I import both vector and region with
from geom.region import Region
from geom.vector import Vector
In vector.py, I import region with
from geom.region import Region
These all work fine when I run the code in a standard run. However, when I call "py.test" from src/, it consistently exits with import errors.
Some Problems and My Solution Attempts
My first problem was that, when running "test/test_foo.py", py.test could not "import foo.py" directly. I solved this by using the "imp" tool. In "test_util.py":
import imp
util = imp.load_source("util", "util.py")
This works great for many files. It also seems to imply that when pytest is running "path/test/test_foo.py" to test "path/foo.py", it is based in the directory "path".
However, this fails for "test_vector.py". Pytest can find and import the vector module, but it cannot locate any of vector's imports. The following imports (from "vector.py") both fail when using pytest:
from geom.region import *
from region import *
These both give errors of the form
ImportError: No module named [geom.region / region]
I don't know what to do next to solve this problem; my understanding of imports in Python is limited.
What is the proper way to handle imports when using pytest?
Edit: Extremely Hacky Solution
In vector.py, I changed the import statement from
from geom.region import Region
to simply
from region import Region
This makes the import relative to the directory of "vector.py".
Next, in "test/test_vector.py", I add the directory of "vector.py" to the path as follows:
import sys, os
sys.path.append(os.path.realpath(os.path.dirname(__file__)+"/.."))
This enables Python to find "../region.py" from "geom/test/test_vector.py".
This works, but it seems extremely problematic because I am adding a ton of new directories to the path. What I'm looking for is either
1) An import strategy that is compatible with pytest, or
2) An option in pytest that makes it compatible with my import strategy
So I am leaving this question open for answers of these kinds.

The issue here is that Pytest walks the filesystem to discover files that contain tests, but then needs to generate a module name that will cause import to load that file. (Remember, files are not modules.)
Pytest comes up with this test package name by finding the first directory at or above the level of the file that does not include an __init__.py file and declaring that the "basedir" for the module tree containing a module generated from this file. It then adds the basedir to sys.path and imports using the module name that will find that file relative to the basedir.
There are some implications of this of which you should beware:
The basepath may not match your intended basepath in which case the module will have a name that doesn't match what you would normally use. E.g., what you think of as geom.test.test_vector will actually be named just test_vector during the Pytest run because it found no __init__.py in src/geom/test/ and so added that directory to sys.path.
You may run into module naming collisions if two files in different directories have the same name. For example, lacking __init__.py files anywhere, adding geom/test/test_util.py will conflict with test/test_util.py because both are loaded as import test_util.py, with both test/ and geom/test/ in the path.
The system you're using here, without explicit __init__.py modules, is having Python create implicit namespace packages for your directories. (A package is a module with submodules.) Ideally we'd configure Pytest with a path from which it would also generate this, but it doesn't seem to know how to do that.
The easiest solution here is simply to add empty __init__.py files to all of the subdirectories under src/; this will cause Pytest to import everything using package/module names that start with directory names under src/.
The question How do I Pytest a project using PEP 420 namespace packages? discusses other solutions to this.

import looks in the following directories to find a module:
The home directory of the program. This is the directory of your root script. When you are running pytest your home directory is where it is installed (/usr/local/bin probably). No matter that you are running it from your src directory because the location of your pytest determines your home directory. That is the reason why it doesn't find the modules.
PYTHONPATH. This is an environment variable. You can set it from the command line of your operating system. In Linux/Unix systems you can do this by executing: 'export PYTHONPATH=/your/custom/path' If you wanted Python to find your modules from the test directory you should include the src path in this variable.
The standard libraries directory. This is the directory where all your libraries are installed.
There is a less common option using a pth file.
sys.path is the result of combining the home directory, PYTHONPATH and the standard libraries directory. What you are doing, modifying sys.path is correct. It is something I do regularly. You could try using PYTHONPATH if you don't like messing with sys.path

If you include an __init__.py file inside your tests directory, then when the program is looking to set a home directory it will walk 'upwards' until it finds one that does not contain an init file. In this case src/.
From here you can import by saying :
from geom.region import *
you must also make sure that you have an init file in any other subdirectories, such as the other nested test directory

I was wondering what to do about this problem too. After reading this post, and playing around a bit, I figured out an elegant solution. I created a file called "test_setup.py" and put the following code in it:
import sys, os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
I put this file in the top-level directory (such as src). When pytest is run from the top-level directory, it will run all test files including this one since the file is prefixed with "test". There are no tests in the file, but it is still run since it begins with "test".
The code will append the current directory name of the test_setup.py file to the system path within the test environment. This will be done only once, so there are not a bunch of things added to the path.
Then, from within any test function, you can import modules relative to that top-level folder (such as import geom.region) and it knows where to find it since the src directory was added to the path.
If you want to run a single test file (such as test_util.py) instead of all the files, you would use:
pytest test_setup.py test\test_util.py
This runs both the test_setup and test_util code so that the test_setup code can still be used.

Are so late to answer that question but usining python 3.9 or 3.10 u just need to add __init__.py folder in tests folders.
When u add this file python interprets this folders as a module.
Wold be like this
src/
main.py
util.py
test/
__init__.py
test_util.py
geom/
vector.py
region.py
test/
__init__.py
test_vector.py
test_region.py
so u just run pytest.
Sorry my poor english

Not the best solution, but maybe the fastest one:
cd path/python_folder
python -m pytest python_file.py

How do I make relative importing work in Python using Eclipse with PyDev?

So I am working on a Python project that was here before me in an SVN repo. When I first pulled it, the structure was a bit odd due to the fact that it was similar to:
Proj\
src\
tags\
trunk\
And then everything is inside src\ are the python module files except src\ turns out to just be a logical folder with no overall package inside. There isn't a __init__.py in the project anywhere. So I want to restructure it at least so I can use relative imports through my project. I also want to set it up so it looks more like this.
Proj\
src\
model\
controller\
view\
test\
tags\
trunk\
However, I tried setting this up and no matter what I seem to do, it cannot resolve the relative import the moment I have to traverse packages. I placed a __init__.py file in each level package including one inside the src\ folder with all of them having __all__ defined. However, when I try to make a unit test in my test\ package and do an import saying:
from ..model.foo import Foo
to attempt to import the Foo class from module foo.py located inside of the model package, it doesn't resolve. Just in case it was a problem specifically with unit tests, I also tried this with a module in the controller package that was dependent on a class in the model package and vice versa. None of them worked. How do I resolve this?

Have you added the root folder to your system path?
import sys
sys.path.append(<place the Proj dir here>)
then you could import as follows:
from src.model.somefile import Something
If you don't know the absolute path for Proj, you can always use combinations such as
os.path.dirname(os.getcwd())