I have the following project architecture (simplified):
root/
main.py
mypackage/
__init__.py
module1.py
module2.py
ci_scripts/
script1.py
script2.py
doc/
doc
where the root is nothing more than a folder.
How do I import module1 and module2 in script1.py for example?
I tried to:
Add the relative path inside script1.py: from ..mypackage import module1
The absolute path
Use of sys package to append the path of mypackage
To give more context, I was expecting that the following code inside of script1.py would work:
from ..mypackage import module1
module1.func1()
but I get:
ImportError: attempted relative import with no known parent package
And when I try to use the absolute path:
import mypackage.module1
module1.func1()
I get the following error:
ModuleNotFoundError: No module named 'mypackage'
Your absolute import probably does not work because your root folder is not set to be mypackage. You can see here on how to do that: python: Change the scripts working directory to the script's own directory
Alternatively, you can use relative imports. You are correctly importing with from ..mypackage import module1 - however, you cannot execute a script directly with a relative import. You need to import the script into some other module which you are then executing. See explanation here: Relative imports in Python 3
I've had similar problems in the past and thus I've created a new import library: ultraimport
It gives you more control over your imports and lets you do file system based relative and absolute imports that do always work.
In your script1.py you could then write:
import ultraimport
module1 = ultraimport("__dir__/../mypackage/module1.py")
This will always work, no matter how you run your program and independent of all external factors.
Related
Setup
test/
main.py
pkg/
a.py
__init__.py
main.py contains:
import pkg
pkg.a
__init__.py contains:
from . import a
main.py can be run without errors.
Question
Changing the content of __init__.py to
import a
gives the following error when running main.py:
Traceback (most recent call last):
File "C:/Users/me/PycharmProjects/test/main.py", line 1, in <module>
import pkg
File "C:\Users\me\PycharmProjects\test\pkg\__init__.py", line 1, in <module>
import a
ModuleNotFoundError: No module named 'a'
Interestingly, __init__.py can be executed directly with python __init__.py without errors.
What's going on?
When you run a python script, it's parent folder is added to sys.path
run main.py: sys.path[0] = '../test'
run init.py: sys.path[0] = '../test/pkg'
Your case: You try to "absolute-like" import a in __init__.py but the parent folder of a.py - which is '../test/pkg' - is not in the sys.path when you run main.py. This is why you get an error. However, your absolute import is incomplete as it should always start at the top level folder, e.g.
from test.pkg import a
Final answer to your question: You don't have to use relative imports!
See: PEP-8: Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path).
And keep in mind that relative imports don't work in a top-level-script when __name__ = "__main__", but from imported modules only.
You can learn more about absolute and relative imports here:
Absolute vs. explicit relative import of Python module
https://realpython.com/absolute-vs-relative-python-imports/
I suppose you are using Pycharm? Then that's one of the confusion cause.
For example, let's say your directory looks like this
project1
p1.py
test/
__init__.py
main.py
pkg/
a.py
__init__.py
If you run (F10) the main.py your default working directory will be project1/test, which does not contain the a.py so import a will not find anything.
But if you run (F10) the pkg/__init__.py your working directory will be project1/test/pkg which has the a.py, and it works like what you tested.
So in these situation, if you use from . import a it will look for the directory that file is, project1/test/pkg in this case, which will always work regardless your working directory.
My file stucture is :
top\
my_package\
__init__.py
functions.py
scripts\
test.py
main.py
I would like to import the content of functions.py in test.py.
In main.py, I can import functions.py with from my_package.functions import ....
I was expecting to be able to import functions.py in test.py with something like from ..my_package.functions import ... but it raises the following error :
SystemError: Parent module '' not loaded, cannot perform relative import
top directory shoudln't be a package because I want to be able to run main.py without being running a script in a package.
What is the proper/pythonic way import functions.py from test.py ?
I could add my_package to the PYTHONPATH, but my code would be less portable. I'm using Python 3.5
This is happening because as far as python is concerned my_package and scripts directories are not related to each other.
To solve your problem you can add __init__.py to your top directory and try using
from ..function import *
I can't make this work..
My structure is:
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
thetest.py:
from ..src.Process.thefile.py import sth
Running: pytest ./tests/thetest.py from program_name gives :
ValueError: attempted relative import beyond top-level package
I tried also other approaches but i am receiving various errors.
But I would expect for the above to work.
ValueError: Attempted relative import in non-package
States that you're trying to use relative import in the module, which are to be used for packages i.e. to make it a package add __init__.py and call the thetest.py from some file outside the package.
Directly running thetest.py from interpreter won't work.
Relative imports require that the module which uses them is being
imported itself either as package module.
Suggestion 1:
The current tests directory has a __init__.py file but that doesn't allow you to run it as a module (via shell) - to make your current (relative) import work, you need to import it in an external (to package) file/module - let's create a main.py (can name it anything you like):
main.py
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
src/Process/thefile.py:
s = 'Hello world'
tests/thetest.py:
from ..src.Process.thefile import s
print s
main.py:
from program_name.tests.thetest import s
Executing main.py:
[nahmed#localhost ~]$ python main.py
Hello world
Suggestion 2:
Execute the file just above root dir i.e. one level up the program_name/ , in the following fashion:
[nahmed#localhost ~]$ python -m program_name.tests.thetest
Hell World
P.S. relative imports are for packages, not modules.
Just solved a similar problem with a lot of googling.
Here's two solutions without changing the existing file structor:
1
The way to import module from parent folder from ..src.Process.thefile.py import sth is called "relative import".
It's only supported when launching as a package from the top-level package. In your case, that is launching command line from the directory which contains program_name/ and type (for win environment)
python -m program_name.tests.thetest
or simply (useful for many pytest files):
python -m pytest
2
Otherwise -- when trying to run a script alone or from a non top-level package --
you could manually add directory to the PYTHONPATH at run time.
import sys
from os import path
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
from src.Process.thefile import s
Try the first one first see if it's compatiable with the pytest framework. Otherwise the second one should always solve the problem.
Reference (How to fix "Attempted relative import in non-package" even with __init__.py)
When importing a file, Python only searches the current directory, the directory that the entry-point script is running from.
you can use sys.path to include different locations
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import thefile
I have several python modules in the project and I put them in different folders, for example,
pythonProject\folderA\modulex.py
pythonProject\folderB\moduley.py
pythonProject\commonModule\module1.py
I have __init__.py in each folder.
In this situation, how can I import module1 into modulex?
Use relatively import
# in modulex
from ..commonModule import module1
Whenever you have python packages (those folders that contain __init__.py files), you can import the modules like below
modulex.py
----------
from pythonproject.commonModule import module1
Try this, If the pythonproject is not defined by the tool, then you could use the relative addressing like below
from ..commonModule import module1
The best if all modules are in the same directory. In case any of them in different possible use of os.chdir(path). With os.chdir(path) method (https://docs.python.org/3.2/library/os.html) possible change working directory in your program.
import os
import modulex
#assume working directory is "pythonProject\folderA\"
os.chdir(r'pythonProject\commonModule\')
#now working directory is "pythonProject\commonModule\"
import module1
If I have a small python project split into a main directory with 2 subdirs:
src/
run.py
subdir1/
__init__.py
module1.py
subdir2/
__init__.py
module2.py
In order to include module2 in module1 would I have to use the full include path: from subdir1.subdir2.module2 import Class2 (relative to the python file that is being run), or could I use from subdir2.module2 import Class2? (relative to the file that statement is in)
src/
run.py
subdir1/
__init__.py
module1.py
subdir2/
__init__.py
module2.py
How is it now best to include module2 in module1, I have tried relative to where the program is run, i.e. from subdir2.module2 import Class2 but this gives the error "No module named module2" I cant help thinking this is not the most portable way of doing things anyhow, would it not be better with a path relative to the file where the statement exists, especcially in the 1st example.
I'm assuming it wont mess things up if I call the code from another active directory, or will it? python dir/run.py
Thanks!
There are two ways to import module:
absolute import
relative import
Absolute import makes python to look for desired module in directories stored in sys.path
Relative import address to module relative to current module. You can use relative import inside python packages only.
for first case You can use relative import because subdir1 is a package
for the second one You can't use relative import because subdir1 and subdir2 are not in package.
according to pep-8:
Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path):
import mypkg.sibling
from mypkg import sibling
from mypkg.sibling import example
However, explicit relative imports are an acceptable alternative to absolute imports, especially when dealing with complex package layouts where using absolute imports would be unnecessarily verbose:
from . import sibling
from .sibling import example
Standard library code should avoid complex package layouts and always use absolute imports.
Implicit relative imports should never be used and have been removed in Python 3.