Develop Reference

Import package from parent folder [duplicate]

This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)

From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.

You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.

How to properly use python import when building an exportable library

Say I have a project structure as such:
/project_name
setup.cfg
pyproject.toml
/src
/package1
__init__.py
/package2
__init__.py
module.py
I want to import and use module.py in package1/__init__.py and I can do so with the two following imports (relative or absolute):
from ..package2 import module
import src.package2.module
This works ok for developing the library and testing it, but my issue arises when the end user installs and uses the package downstream. Inside my setup.cfg file I have:
...
[options]
package_dir =
= src
packages = find:
...
[options.packages.find]
where = src
I've been using py -m build to build the .tar.gz and -py3-none-any.whl files and uploading them to a private repository. Users can then install it using:
pip install --extra-index-url http://mysite/private/repo project_name --trusted-host mysite
The install goes ok, but when when they go to try and use package1 with import package1 that is where things get messy as they get one of these errors:
"ValueError: attempted relative import beyond top-level package"
This happens when I use from ..package2 import module as described above
"ModuleNotFoundError: No module named 'src'"
This happens when I use import src.package2.module as described above
Update
I found that if I restructure my project as such I get it working for the end user:
/project_name
setup.cfg
pyproject.toml
/src
/project_name
__init__.py
/package1
__init__.py
/package2
__init__.py
module.py
The end user can use the modules using from project_name import package1 which I was hoping to keep it down to just import package1 but whatever that is fine. What I really have a problem with is that in package1 I HAVE to use a relative import:
from ..package2 import module
The reason this bothers me is because I've read in several places that it is better to use absolute imports as the end user can modify the path and screw things up, but if I do it this way:
import src.project_name.package2.module
It breaks the end user running the code because they don't see src!!! So I guess the heart of my question is: how does python determine what the root package is? Why do I have to use src in the import statement (which DOESN'T have a __init__.py) instead of just project_name.package2.module. I've looked all over and can't seem to answer this. Very confused.

The way you setup your build/install, everything would be installed following the tree hierarchy below src (not including src). And you can do:
import project_name
import project_name.package1
Note that in this setup you cannot do import package1. It seems like in your development environment you have modified PYTHONPATH so you can do import package1, and this is what causes your issues. In this case python treats package1 as the top-level package, and you cannot do a relative import that goes above the top-level. It seems you have also set your development environment so you can do import src.project_name.package1, which is inconsistent with the actual install location.
You cannot do relative imports between two separate packages. They need to be within a larger package, like your project_name, as you found out. If you want relative imports, you definitely need this top-level package.
Bottom-line, if in your development environment you set your PYTHONPATH to search in the src directory, you can do import project_name.package1 and either a relative import
from ..package2 import module
or absolute import (note I do not include src)
import project_name.package2.module

Import a module from a sub package not working

My file structure is
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
in test.py I am trying to import CompareText
from project.features import CompareText
I get an error of:
ModuleNotFoundError: No module named 'features'`
I checked the documentation and I think my import statement is correct. How can I fix it?

Add an __init__ file in test. Your project directory should look like this:
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
__init__.py
test.py
Then in project/tests/test.py the following import statement will work:
from ..features import CompareText
Oh, and this will still raise an error if you try to run it directly. In the question you said you tried to import it like this:
from project.features import CompareText
This will only work if the parent directory of project is in Python's module search path. So, if you want to run the tests directly then modify the module search path as needed (See: sys.path).

Your import statement is supposed to look like this :
(But make sure your working directory is the same directory as your project folder is located during execution)
from project.features import CompareText
This is supposed to work if your current path while executing the script has the project folder
If you execute it while inside project folder you can use:
from .features import CompareText
Hope this helps!

I assume you are running test.py as a script. test.py needs to find the project package and two ways to do that are to make your project installable or to hack sys.path.
Installable
First, change your directory structure a bit so that project is a subdirectory of some anonymous directory you happen to be using for development. If you are checking this stuff into source control, it needs to be written so that it can be checked out anywhere. Move tests down one directory.
mydevdir/
setup.py
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
How write a setup.py. This can get quite complicated. You can read Building and Distributing Packages with Setuptools and lookup other resources on the net, but a minimalist setup.py is
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='project',
version='0.1',
description='This is project: project',
packages=find_packages(),
)
Now, while in mydevdir do python setup.py develop. Or you can actually produce an install package and put it in a virtual env for test.
Hack sys.path
It may be easier to hack paths in test.py. Note that this will need to be undone if you make project installable later. Just add to the top of test.py
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).absolute().parents[2]))
This puts the parent directory in the python path and now project will be found on import. It runs the risk that a .py file in the same directory as project can mask an installed module. If you have a local csv.py and you import csv, you'd get the local.

Module not found in same directory as running script when installed as a package

I have installed a custom package I made called pytable using python setup.py develop in the package folder. The package hierarchy looks like this:
pytable/
__pycache__/
MANIFEST.in
pytable/
__init__.py
__pycache__/
pytable.py (main module)
tableError.py (class with custom errors)
testScript.py (imports pytable.py)
README.md
setup.py
The tableError.py module is imported into the pytable.py module and works fine when running testScript.py without having pytable installed with python, but when I install it with the above command and run import pytable from pytable anywhere else it errors.
ModuleNotFoundError: No module named 'tableError'

The problem was after the package was installed Python did not know where to look to find depending modules. Adding...
import os
import sys
dir_path = os.path.dirname(os.path.realpath(__file__))
sys.path.append(dir_path)
...to the top of pytable.py tells Python to look in the same directory where it is installed for tableError.py.

Need help understanding this issue regarding relative vs absolute imports in Python

I was getting an error when I wanted to install a certain Python package in a specific folder (using pip install -t), and then import a module from that package.
I posted the issue on the package's Github and it turns out I can't install a package in a certain folder; that it isn't supported for the package to be installed that way.
Even without google collab, I can reproduce the import error when trying to install sklearn in some specific folder with pip install -t sklearnFolder then importing it as sklearnFolder.sklearn.manifold.
That is not a supported way of installing / using scikit-lean. One reason why it wouldn't work is that a few modules in scikit-learn use absolute imports (e.g. from sklearn import something) which will fail with such setup.
You should either install it with pip install or if you want to have it in some specific folder, clone the repo to that folder, then run pip install -e , in both cases it will be imported as sklearn.
From https://github.com/scikit-learn/scikit-learn/issues/11656
I don't quite get the explanation.
I thought something like
from folderName.package import module
is the same as
from package import module
Because they're both absolute imports. As in, they both completely specify the path of the imported module. So there is something off about my understanding but I don't know what it is.

In an import you do not specify the folderName to prefix a package. If the package is installed or in the python path, then you just use the package name to import.
# Assume the below structure is under a directory (folder) called /app/home/packages.
reservation/ This is your top-level pacakge
__init__.py Initialize the package
hotels/ Subpackage for hotel reservations
__init__.py
slots.py
bid.py
demand.py
...
restaurents/ Another Subpackage under hotels
__init__.py
cuisine.py
hours.py
tableslots.py
...
rewards/ Subpackage for rewards
__init__.py
points.py
discounts.py
membersonly.py
...
As the package is under /app/home/packages, then the following import is NOT valid as you prefix the folder name.
from packages.reservation import hotels
The correct way to import is from the actual package which has the package initialization __init__.py. If you see in the example, reservation folder has __init__.py.
from reservation import hotels
If you want to import the sub module under the hotels, then you will prefix with the package:
from reservation.hotels import restaurents
Alternatively you can import directly the sub module but you will have to prefix with the package when you use it:
import reservation.hotels.restaurents