Develop Reference

Import package from parent folder [duplicate]

This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)

From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.

You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.

pip installing a package with the same namespace as a local package

I am using Python 3.6.5, installed via miniconda. My issue is arising from the fact that I'm pip installing a package that has the same namespace as a local package. After pip installing this package, I can no longer import from the local package. I receive a ModuleNotFoundError error. The namespaces need to stay this way, if possible.
Here is my directory structure:
/root
stuff
- __init__.py
- my_stuff.py
app.py
init.py
__import__('pkg_resources').declare_namespace(__name__)
app.py
from stuff.my_stuff import my_fun
This works fine until I pip install the package with the same namespace, "stuff". After pip installing the package, the import statement, from stuff.my_stuff import my_fun throws the following error: ModuleNotFoundError: No module named 'stuff.my_stuff'. I kind of understand why. When importing modules in Python, it will look for built-in modules first, then sys.path, PYTHONPATH etc...
Here's the part thats really confusing me. If I create another arbitrary local module, like some_stuff, as shown below:
/root
stuff
- __init__.py
- my_stuff.py
some_stuff
- __init__.py
- more_stuff.py
app.py
and if I then run:
app.py
from some_stuff.more_stuff import more_fun
from stuff.my_stuff import my_fun
Everything works as expected. i.e. if I import some_stuff.more_stuff before stuff.my_stuff, everything works. But not vice versa. Solely importing stuff.my_stuff causes the ModuleNotFoundError.
app.py
# The code above works, but this causes the error
from stuff.my_stuff import my_fun
What is causing this behaviour? How can I solve this issue of locally referencing a package with the same namespace as one that was pip installed?
Edit:
I continued experimenting and noticed that when I remove all __init__.py files, everything works as expected. I came across this post: Since Python 3.3, a folder without an __init__.py can be considered part of an implicit namespace package. I'm still confused about the behaviour mentioned above though.

This SO question should answer your question
I am still putting the original answer here for convenience.
It's not possible to change "import path" (installed name) by specifying arguments to pip. You can, however, make some changes to the package after installing:
use pip install -e git+http://some_url#egg=some-name that way even if both packages have the same import path, they will be saved under different directories (using some-name provided after #egg=). After this you can go to the source directories of packages (usually venv/src/some-name) and rename some folders to change import paths
fork the repository, make changes, then install the package from that repository. Or you can publish your package on PyPI using different name and install it by that name
use pip download to put one of the packages in your project, then rename folders as you like

Importing from sibling directories (python 3)

I can't figure out how to import modules from sibling directories in Python 3 using absolute imports.
modify the sys.path.
turn the directory into a pip installable package via __init__.py and setup.py.
For option 1. I figured out how to import modules from sibling directories by modifying the sys.path, but this method seems a little hackey to me. Also, I've read that it is not preferred. Why? Is there something inherently wrong or dangerous about modifying the sys.path?
For option 2. What exactly do I need to do make my package pip installable? I've alreay created my __init__.py file, but it seems that I need to create and configure a setup.py script to prepare my package for distribution? I'm still in the development mode, so is this really the best/pythonic method? If it is, then do I just type python setup.py install into my terminal after creating the setup.py?
Edit: I'm now trying to figure this out using absolute imports as python 3 does support relative imports.

From what I've read, Python 3 does not support relative imports
It does.
To import myproject/foo/__init__.py from myproject/bar/baz.py, you can use this:
from .. import foo
Or if you want to import an object/module in foo:
from ..foo import object
This requires myproject to be a package, so myproject/__init__.py has to exist.

py2app not including pythonpath

I'm trying to build an app with py2app. I can build the app but when I run it I get ImportError in the console, the import error is that there is No module named PythonApp which is the folder all my source is in, the location of which is in my PYTHONPATH.
I have been importing local files like from PythonApp import file instead of just import file to help avoid namespace issues.
I have tried to build it with the following flag python setup.py py2app --use-pythonpath but this hasn't appeared to make any difference.
Should I just change the import statements throughout to import file?
How can I make py2app realise my PYTHONPATH?

If I understood your problem well, you have the following layout:
PythonApp/
__init__.py
file.py
foo.py
And you try to import the module file from foo.py by from PythonApp import file. If this the only reason you want to set PYTHONPATH, there is a simpler solution: use relative imports:
from . import file
You can use from .. import file in a sub-package of PythonApp, and so on. This way you can avoid name collisions with standard modules.
If you need to hack the import path for some other reasons, you can also set the sys.path variable in the startup script (probably py2app has some options for that, too). Keep in mind though that if you add external directories into the import path, it will be harder to distribute the app bundle.
Also, a more trivial explanation for the ImportError is that py2app did not copy your package into the app bundle. Make sure you have all your packages listed in the packages parameter of setup().

Tests and python package structure

I have some problems in structuring my python project. Currently it is a bunch of files in the same folder. I have tried to structure it like
proj/
__init__.py
foo.py
...
bar/
__init__.py
foobar.py
...
tests/
foo_test.py
foobar_test.py
...
The problem is that I'm not able, from inner directories, to import modules from the outer directories. This is particularly annoying with tests.
I have read PEP 328 about relative imports and PEP 366 about relative imports from the main module. But both these methods require the base package to be in my PYTHONPATH. Indeed I obtain the following error
ValueError: Attempted relative import in non-package.
So I added the following boilerplate code on top of the test files
import os, sys
sys.path.append(os.path.join(os.getcwd(), os.path.pardir))
Still I get the same error. What is the correct way to
structure a package, complete with tests, and
add the base directory to the path to allow imports?
EDIT As requested in the comment, I add an example import that fails (in the file foo_test.py)
import os, sys
sys.path.append(os.path.join(os.getcwd(), os.path.pardir))
from ..foo import Foo

When you use the -m switch to run code, the current directory is added to sys.path. So the easiest way to run your tests is from the parent directory of proj, using the command:
python -m proj.tests.foo_test
To make that work, you will need to include an __init__.py file in your tests directory so that the tests are correctly recognised as part of the package.

I like to import modules using the full proj.NAME package prefix whenever possible. This is the approach the Google Python styleguide recommends.
One option to allow you to keep your package structure, use full package paths, and still move forward with development would be to use a virtualenv and put your project in develop mode. Your project's setup.py will need to use setuptools instead of distutils, to get the develop command.
This will let you avoid the sys.path.append stuff above:
% virtualenv ~/virt
% . ~/virt/bin/activate
(virt)~% cd ~/myproject
(virt)~/myproject% python setup.py develop
(virt)~/myproject% python tests/foo_test.py
Where foo_test.py uses:
from proj.foo import Foo
Now when you run python from within your virtualenv your PYTHONPATH will point to all of the packages in your project. You can create a shorter shell alias to enter your virtualenv without having to type . ~/virt/bin/activate every time.