Develop Reference

Import package from parent folder [duplicate]

This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)

From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.

You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.

Structuring Python project to allow scripts to import local packages, referencing relative to the project root directory

I have a python project set up where my scripts are stored in one folder, my packages and data in other folders, and I am trying to organize the best structure and procedures for making referencing between these items more robust:
project_dir/
data/
raw/
source_1.csv
source_2.csv
processed/
tidydata.csv
results.csv
src/
scripts/
clean_raw_data.py
calc_results.py
packages/
import_tools
tool_a.py
tool_b.py
calc_tools
Makefile
My desire is to be able to robustly reference my packages through imports (./src/packages) and my data (./data)
with file read and write operations from any of my scripts in the ./src/scripts folder.
My current setup involves doing things like this:
To import packages (this seems like bad practice to call functions in order to import other functions):
# clean_raw_data.py
import sys
from pathlib import Path
sys.path.append(str(Path(__file__).parent.parent))
import packages.import_tools as imptool
To read and write files:
import pandas as pd
df = pd.read_csv('../../data/raw/source_1.csv')
# operations
df.to_csv('../../data/processed/tidydata.csv')
Ideally I would prefer that everything were available referenced from the project folder project_dir
in any file or script in my structure, such that I could do things like:
import src.packages.import_tools as imptool
df = pd.read_csv(f'{ROOT_DIR}/data/raw/source_1.csv')
In some way or another. I presume there is a best practice guideline for configuring things to behave
in a similar way but haven't seen any good recommendations. What would be the best approach
for handling this?

In Python the mechanisms to reference data files and source code are completely different. While you always have to specify the full path to your data file when you want to open it, Python will use the sys.path to autonomously search for modules that you want to import. However, "hacking" the sys.path manually in all your script files is bad practice. Instead, use pip to install your project in editable mode:
pip install --editable path/to/project_dir
but make sure there is a minimal setup.py in project_dir with the following contents
from setuptools import setup
setup(name='myproject')
pip will put the symlink myproject.egg-info into your site-packages folder that you can verify that via
pip show myproject
This allows you to import your packages using what is called absolute imports by always starting from within your project_dir
from src.packages.import_tools import tool_a
(Note that your import packages.import_tools as imptool didn't work anyway since import_tools is a package and not a module.)
The next things you could add to your project_dir are a README.MD, a requirements.txt and a test folder for your unit tests. And keep in mind that the distinction between scripts and packages is somewhat artificial, because all Python files are basically modules that can be imported.

importing a package from within another package in python

Please assume the following project structure:
/project
/packages
/files
__init__.py
fileChecker.py
/hasher
__init__.py
fileHash.py
mainProject.py
/test
I would like to get access to the module fileChecker.py from within the module fileHash.py.
This is some kind of global package.
One way is to append paths to sys.path.
[Is this PYTHONPATH by the way?]
What would be a solution when distributing the project?
Same as above? --> But then there could be paths to modules with the
same name in PYTHONPATH?
Is setuptools doing all the work?
How can I achieve it in a nice and clean way?
Thanks alot.
Update:
Also see my answer below
--> when calling fileHash.py (including an import like from files import fileChecker) directly from within its package directory, the project's path needs to be added to sys.path (described below).
Test cases situated within /test (see structure above) also need the path added to sys.path, when called from within /test.

Thanks mguijarr.
I found a solution here on stackoverflow:
source:
How to fix "Attempted relative import in non-package" even with __init__.py
when I am in the project folder /project, I can call the module like this:
python -m packages.files.fileHash (no .py here, because it is a package)
This is wokring well.
In this case PYTHONPATH is known and the import can look like this:
from packages.files import fileChecker
If it is not called directly, but from within the package directory in my case /packages/hasher --> setting the PYTHONPATH is needed:
if __package__ is None:
import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
from packages.files import fileChecker
else:
from packages.files import fileChecker
The important thing for me here is, that the path to include is the PROJECT path.
The code snippet above(the last one) already includes the case describes both cases (called as package and directly).
Thanks alot for your help.
Update:
Just to make my answer more complete
Python adds the current path to the PYTHONPATH automatically when doing
python fileHash.py
Another option, in addition to the one above, is to set the PYTHONPATH when running the program like this
PYTHONPATH=/path/to/project python fileHash.py
I gained some experience, I would like to share:
I don't run modules from within their directories anymore.
Starting the app, running tests or sphinx or pylint or whatever is all done from the project directory.
This ensures that the project directory is contained in the python path and all packages, modules are found without doing additional stuff on imports.
The only place I still set the python path to the project folder using sys.path is in my setup.py in order to make codeship work.
Still, in my opinion this is somehow not an easy matter and I find myself reflecting the PYTHONPATH often enough :)

I finally found another solution, quite intuitive, no sys path or anything needed: https://www.tutorialsteacher.com/python/python-package
And then do what they explain in ''Install a Package Globally''.
In your case, put the "setup.py" file in the packages directory and add the two packages:
from setuptools import setup
setup(name='mypackage',
version='0.1',
description='Testing installation of Package',
url='#',
author='malhar',
author_email='mlathkar#gmail.com',
license='MIT',
packages=['files', 'hasher'], ## here the names
zip_safe=False)

py2app not including pythonpath

I'm trying to build an app with py2app. I can build the app but when I run it I get ImportError in the console, the import error is that there is No module named PythonApp which is the folder all my source is in, the location of which is in my PYTHONPATH.
I have been importing local files like from PythonApp import file instead of just import file to help avoid namespace issues.
I have tried to build it with the following flag python setup.py py2app --use-pythonpath but this hasn't appeared to make any difference.
Should I just change the import statements throughout to import file?
How can I make py2app realise my PYTHONPATH?

If I understood your problem well, you have the following layout:
PythonApp/
__init__.py
file.py
foo.py
And you try to import the module file from foo.py by from PythonApp import file. If this the only reason you want to set PYTHONPATH, there is a simpler solution: use relative imports:
from . import file
You can use from .. import file in a sub-package of PythonApp, and so on. This way you can avoid name collisions with standard modules.
If you need to hack the import path for some other reasons, you can also set the sys.path variable in the startup script (probably py2app has some options for that, too). Keep in mind though that if you add external directories into the import path, it will be harder to distribute the app bundle.
Also, a more trivial explanation for the ImportError is that py2app did not copy your package into the app bundle. Make sure you have all your packages listed in the packages parameter of setup().

Import python package from local directory into interpreter

I'm developing/testing a package in my local directory. I want to import it in the interpreter (v2.5), but sys.path does not include the current directory. Right now I type in sys.path.insert(0,'.'). Is there a better way?
Also,
from . import mypackage
fails with this error:
ValueError: Attempted relative import in non-package

You can use relative imports only from in a module that was in turn imported as part of a package -- your script or interactive interpreter wasn't, so of course from . import (which means "import from the same package I got imported from") doesn't work. import mypackage will be fine once you ensure the parent directory of mypackage is in sys.path (how you managed to get your current directory away from sys.path I don't know -- do you have something strange in site.py, or...?)
To get your current directory back into sys.path there is in fact no better way than putting it there.

Keep it simple:
try:
from . import mymodule # "myapp" case
except:
import mymodule # "__main__" case

See the documentation for sys.path:
http://docs.python.org/library/sys.html#sys.path
To quote:
If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.
So, there's no need to monkey with sys.path if you're starting the python interpreter from the directory containing your module.
Also, to import your package, just do:
import mypackage
Since the directory containing the package is already in sys.path, it should work fine.

If you want to run an unmodified python script so it imports libraries from a specific local directory you can set the PYTHONPATH environment variable - e.g. in bash:
export PYTHONPATH=/home/user/my_libs
python myscript.py
If you just want it to import from the current working directory use the . notation:
export PYTHONPATH=.
python myscript.py

Inside a package if there is setup.py, then better to install it
pip install -e .

A simple way to make it work is to run your script from the parent directory using python's -m flag, e.g. python -m packagename.scriptname. Obviously in this situation you need an __init__.py file to turn your directory into a package.

Using sys.path should include current directory already.
Try:
import .
or:
from . import sth
however it may be not a good practice, so why not just use:
import mypackage

A bit late to the party, but this is what worked for me:
>>> import sys
>>> sys.path.insert(0, '')
Apparently, if there is an empty string, Python knows that it should look in the current directory. I did not have the empty string in sys.path, which caused this error.

Speaking for python3.. I wanted to use an improved version of a library that's installed in my environment. There are some extra print statements it makes to show that it and not the original lib is being used.
I placed the lib's folder next to the python script. Ran the script.. it ran with the local lib with the modifications.
Removed the folder and ran it again - this time it ran with the installed lib.
So, solution was simple : place the lib's folder (with same name as in your import statement) in your project folder. That does the job, at least at my end.
This is on a standard Linux Mint 20.04 system, with a python 3.8 virutal environment activated (so "(py3.8)" appears in my terminal when I'm in the virtual env)

You can import package_name if the package is a module: this needs you have init.py under the package and things that you want to use are needed to import in the init.py
Or if you want to import class under the package, you can use from package_name import class_name