Related
I want to import a function from another file in the same directory.
Usually, one of the following works:
from .mymodule import myfunction
from mymodule import myfunction
...but the other one gives me one of these errors:
ImportError: attempted relative import with no known parent package
ModuleNotFoundError: No module named 'mymodule'
SystemError: Parent module '' not loaded, cannot perform relative import
Why is this?
unfortunately, this module needs to be inside the package, and it also
needs to be runnable as a script, sometimes. Any idea how I could
achieve that?
It's quite common to have a layout like this...
main.py
mypackage/
__init__.py
mymodule.py
myothermodule.py
...with a mymodule.py like this...
#!/usr/bin/env python3
# Exported function
def as_int(a):
return int(a)
# Test function for module
def _test():
assert as_int('1') == 1
if __name__ == '__main__':
_test()
...a myothermodule.py like this...
#!/usr/bin/env python3
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
...and a main.py like this...
#!/usr/bin/env python3
from mypackage.myothermodule import add
def main():
print(add('1', '1'))
if __name__ == '__main__':
main()
...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...
from .mymodule import as_int
The way you're supposed to run it is...
python3 -m mypackage.myothermodule
...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.
The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...
from mymodule import as_int
...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...
from mypackage.mymodule import as_int
...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
from mypackage.mymodule import as_int
It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...
I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.
Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.
Explanation
From PEP 328
Relative imports use a module's __name__ attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to '__main__')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
At some point PEP 338 conflicted with PEP 328:
... relative imports rely on __name__ to determine the current
module's position in the package hierarchy. In a main module, the
value of __name__ is always '__main__', so explicit relative imports
will always fail (as they only work for a module inside a package)
and to address the issue, PEP 366 introduced the top level variable __package__:
By adding a new module level attribute, this PEP allows relative
imports to work automatically if the module is executed using the -m
switch. A small amount of boilerplate in the module itself will allow
the relative imports to work when the file is executed by name. [...] When it [the attribute] is present, relative imports will be based on this attribute
rather than the module __name__ attribute. [...] When the main module is specified by its filename, then the __package__ attribute will be set to None. [...] When the import system encounters an explicit relative import in a
module without __package__ set (or with it set to None), it will
calculate and store the correct value (__name__.rpartition('.')[0]
for normal modules and __name__ for package initialisation modules)
(emphasis mine)
If the __name__ is '__main__', __name__.rpartition('.')[0] returns empty string. This is why there's empty string literal in the error description:
SystemError: Parent module '' not loaded, cannot perform relative import
The relevant part of the CPython's PyImport_ImportModuleLevelObject function:
if (PyDict_GetItem(interp->modules, package) == NULL) {
PyErr_Format(PyExc_SystemError,
"Parent module %R not loaded, cannot perform relative "
"import", package);
goto error;
}
CPython raises this exception if it was unable to find package (the name of the package) in interp->modules (accessible as sys.modules). Since sys.modules is "a dictionary that maps module names to modules which have already been loaded", it's now clear that the parent module must be explicitly absolute-imported before performing relative import.
Note: The patch from the issue 18018 has added another if block, which will be executed before the code above:
if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
PyErr_SetString(PyExc_ImportError,
"attempted relative import with no known parent package");
goto error;
} /* else if (PyDict_GetItem(interp->modules, package) == NULL) {
...
*/
If package (same as above) is empty string, the error message will be
ImportError: attempted relative import with no known parent package
However, you will only see this in Python 3.6 or newer.
Solution #1: Run your script using -m
Consider a directory (which is a Python package):
.
├── package
│ ├── __init__.py
│ ├── module.py
│ └── standalone.py
All of the files in package begin with the same 2 lines of code:
from pathlib import Path
print('Running' if __name__ == '__main__' else 'Importing', Path(__file__).resolve())
I'm including these two lines only to make the order of operations obvious. We can ignore them completely, since they don't affect the execution.
__init__.py and module.py contain only those two lines (i.e., they are effectively empty).
standalone.py additionally attempts to import module.py via relative import:
from . import module # explicit relative import
We're well aware that /path/to/python/interpreter package/standalone.py will fail. However, we can run the module with the -m command line option that will "search sys.path for the named module and execute its contents as the __main__ module":
vaultah#base:~$ python3 -i -m package.standalone
Importing /home/vaultah/package/__init__.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/module.py
>>> __file__
'/home/vaultah/package/standalone.py'
>>> __package__
'package'
>>> # The __package__ has been correctly set and module.py has been imported.
... # What's inside sys.modules?
... import sys
>>> sys.modules['__main__']
<module 'package.standalone' from '/home/vaultah/package/standalone.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
-m does all the importing stuff for you and automatically sets __package__, but you can do that yourself in the
Solution #2: Set __package__ manually
Please treat it as a proof of concept rather than an actual solution. It isn't well-suited for use in real-world code.
PEP 366 has a workaround to this problem, however, it's incomplete, because setting __package__ alone is not enough. You're going to need to import at least N preceding packages in the module hierarchy, where N is the number of parent directories (relative to the directory of the script) that will be searched for the module being imported.
Thus,
Add the parent directory of the Nth predecessor of the current module to sys.path
Remove the current file's directory from sys.path
Import the parent module of the current module using its fully-qualified name
Set __package__ to the fully-qualified name from 2
Perform the relative import
I'll borrow files from the Solution #1 and add some more subpackages:
package
├── __init__.py
├── module.py
└── subpackage
├── __init__.py
└── subsubpackage
├── __init__.py
└── standalone.py
This time standalone.py will import module.py from the package package using the following relative import
from ... import module # N = 3
We'll need to precede that line with the boilerplate code, to make it work.
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[3]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import package.subpackage.subsubpackage
__package__ = 'package.subpackage.subsubpackage'
from ... import module # N = 3
It allows us to execute standalone.py by filename:
vaultah#base:~$ python3 package/subpackage/subsubpackage/standalone.py
Running /home/vaultah/package/subpackage/subsubpackage/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/subpackage/__init__.py
Importing /home/vaultah/package/subpackage/subsubpackage/__init__.py
Importing /home/vaultah/package/module.py
A more general solution wrapped in a function can be found here. Example usage:
if __name__ == '__main__' and __package__ is None:
import_parents(level=3) # N = 3
from ... import module
from ...module.submodule import thing
Solution #3: Use absolute imports and setuptools
The steps are -
Replace explicit relative imports with equivalent absolute imports
Install package to make it importable
For instance, the directory structure may be as follows
.
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
│ └── setup.py
where setup.py is
from setuptools import setup, find_packages
setup(
name = 'your_package_name',
packages = find_packages(),
)
The rest of the files were borrowed from the Solution #1.
Installation will allow you to import the package regardless of your working directory (assuming there'll be no naming issues).
We can modify standalone.py to use this advantage (step 1):
from package import module # absolute import
Change your working directory to project and run /path/to/python/interpreter setup.py install --user (--user installs the package in your site-packages directory) (step 2):
vaultah#base:~$ cd project
vaultah#base:~/project$ python3 setup.py install --user
Let's verify that it's now possible to run standalone.py as a script:
vaultah#base:~/project$ python3 -i package/standalone.py
Running /home/vaultah/project/package/standalone.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
Note: If you decide to go down this route, you'd be better off using virtual environments to install packages in isolation.
Solution #4: Use absolute imports and some boilerplate code
Frankly, the installation is not necessary - you could add some boilerplate code to your script to make absolute imports work.
I'm going to borrow files from Solution #1 and change standalone.py:
Add the parent directory of package to sys.path before attempting to import anything from package using absolute imports:
import sys
from pathlib import Path # if you haven't already done so
file = Path(__file__).resolve()
parent, root = file.parent, file.parents[1]
sys.path.append(str(root))
# Additionally remove the current file's directory from sys.path
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
Replace the relative import by the absolute import:
from package import module # absolute import
standalone.py runs without problems:
vaultah#base:~$ python3 -i package/standalone.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
I feel that I should warn you: try not to do this, especially if your project has a complex structure.
As a side note, PEP 8 recommends the use of absolute imports, but states that in some scenarios explicit relative imports are acceptable:
Absolute imports are recommended, as they are usually more readable
and tend to be better behaved (or at least give better error
messages). [...] However, explicit relative imports are an acceptable
alternative to absolute imports, especially when dealing with complex
package layouts where using absolute imports would be unnecessarily
verbose.
Put this inside your package's __init__.py file:
# For relative imports to work in Python 3.6
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Assuming your package is like this:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module1.py
│ │ └── module2.py
│ └── setup.py
Now use regular imports in you package, like:
# in module2.py
from module1 import class1
This works in both python 2 and 3.
I ran into this issue. A hack workaround is importing via an if/else block like follows:
#!/usr/bin/env python3
#myothermodule
if __name__ == '__main__':
from mymodule import as_int
else:
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
SystemError: Parent module '' not loaded, cannot perform relative import
This means you are running a module inside the package as a script. Mixing scripts inside packages is tricky and should be avoided if at all possible. Use a wrapper script that imports the package and runs your scripty function instead.
If your top-level directory is called foo, which is on your PYTHONPATH module search path, and you have a package bar there (it is a directory you'd expect an __init__.py file in), scripts should not be placed inside bar, but should live on in foo at best.
Note that scripts differ from modules here in that they are used as a filename argument to the python command, either by using python <filename> or via a #! (shebang) line. It is loaded directly as the __main__ module (this is why if __name__ == "__main__": works in scripts), and there is no package context to build on for relative imports.
Your options
If you can, package your project with setuptools (or poetry or flit, which can help simplify packaging), and create console script entrypoints; installing your project with pip then creates scripts that know how to import your package properly. You can install your package locally with pip install -e ., so it can still be edited in-place.
Otherwise, never, ever, use python path/to/packagename/file.py, always use python path/to/script.py and script.py can use from packagename import ....
As a fallback, you could use the -m command-line switch to tell Python to import a module and use that as the __main__ file instead. This does not work with a shebang line, as there is no script file any more, however.
If you use python -m foo.bar and foo/bar.py is found in a sys.path directory, that is then imported and executed as __main__ with the right package context. If bar is also a package, inside foo/, it must have a __main__.py file (so foo/bar/__main__.py as the path from the sys.path directory).
In extreme circumstances, add the metadata Python uses to resolve relative imports by setting __package__ directly; the file foo/bar/spam.py, importable as foo.bar.spam, is given the global __package__ = "foo.bar". It is just another global, like __file__ and __name__, set by Python when imported.
On sys.path
The above all requires that your package can be imported, which means it needs to be found in one of the directories (or zipfiles) listed in sys.path. There are several options here too:
The directory where path/to/script.py was found (so path/to) is automatically added to sys.path. Executing python path/to/foo.py adds path/to to sys.path.
If you packaged your project (with setuptools, poetry, flit or another Python packaging tool), and installed it, the package has been added to the right place already.
As a last resort, add the right directory to sys.path yourself. If the package can be located relatively to the script file, use the __file__ variable in the script global namespace (e.g. using the pathlib.Path object, HERE = Path(__file__).resolve().parent is a reference to the directory the file lives in, as absolute path).
For PyCharm users:
I also was getting ImportError: attempted relative import with no known parent package because I was adding the . notation to silence a PyCharm parsing error. PyCharm innaccurately reports not being able to find:
lib.thing import function
If you change it to:
.lib.thing import function
it silences the error but then you get the aforementioned ImportError: attempted relative import with no known parent package. Just ignore PyCharm's parser. It's wrong and the code runs fine despite what it says.
To obviate this problem, I devised a solution with the repackage package, which has worked for me for some time. It adds the upper directory to the lib path:
import repackage
repackage.up()
from mypackage.mymodule import myfunction
Repackage can make relative imports that work in a wide range of cases, using an intelligent strategy (inspecting the call stack).
TL;DR: to #Aya's answer, updated with pathlib library, and working for Jupyter notebooks where __file__ is not defined:
You want to import my_function defined under ../my_Folder_where_the_package_lives/my_package.py
respect to where you are writing the code.
Then do:
import os
import sys
import pathlib
PACKAGE_PARENT = pathlib.Path(__file__).parent
#PACKAGE_PARENT = pathlib.Path.cwd().parent # if on jupyter notebook
SCRIPT_DIR = PACKAGE_PARENT / "my_Folder_where_the_package_lives"
sys.path.append(str(SCRIPT_DIR))
from my_package import my_function
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
TL;DR
You can only relatively import modules inside another module in the same package.
Concept Clarify
We see a lot of example code in books/docs/articles, they show us how to relatively import a module, but when we do so, it fails.
The reason is, put it in a simple sentence, we did not run the code as the python module mechanism expects, even though the code is written totally right. It's like some kind of runtime thing.
Module loading is depended on how you run the code. That is the source of confusion.
What is a module?
A module is a python file when and only when it is being imported by another file. Given a file mod.py, is it a module? Yes and No, if you run python mod.py, it is not a module, because it is not imported.
What is a package?
A package is a folder that includes Python module(s).
BTW, __init__.py is not necessary from python 3.3, if you don't need any package initialization or auto-load submodules. You don't need to place a blank __init__.py in a directory.
That proves a package is just a folder as long as there are files being imported.
Real Answer
Now, this description becomes clearer.
You can only relatively import modules inside another module in the same package.
Given a directory:
. CWD
|-- happy_maker.py # content: print('Sends Happy')
`-- me.py # content: from . import happy_maker
Run python me.py, we got attempted relative import with no known parent package
me.py is run directly, it is not a module, and we can't use relative import in it.
Solution 1
Use import happy_maker instead of from . import happy_maker
Solution 2
Switch our working directory to the parent folder.
. CWD
|-- happy
| |-- happy_maker.py
`-- me.py
Run python -m happy.me.
When we are in the directory that includes happy, happy is a package, me.py, happy_maker.py are modules, we can use relative import now, and we still want to run me.py, so we use -m which means run the module as a script.
Python Idiom
. CWD
|-- happy
| |-- happy_maker.py # content: print('Sends Happy')
| `-- me.py # content: from . import happy_maker
`-- main.py # content: import happy.me
This structure is the python idiom. main is our script, best practice in Python. Finally, we got there.
Siblings or Grandparents
Another common need:
.
|-- happy
| |-- happy_maker.py
| `-- me.py
`-- sad
`-- sad_maker.py
We want to import sad_maker in me.py, How to do that?
First, we need to make happy and sad in the same package, so we have to go up a directory level. And then from ..sad import sad_maker in the me.py.
That is all.
My boilerplate to make a module with relative imports in a package runnable standalone.
package/module.py
## Standalone boilerplate before relative imports
if __package__ is None:
DIR = Path(__file__).resolve().parent
sys.path.insert(0, str(DIR.parent))
__package__ = DIR.name
from . import variable_in__init__py
from . import other_module_in_package
...
Now you can use your module in any fashion:
Run module as usual: python -m package.module
Use it as a module: python -c 'from package import module'
Run it standalone: python package/module.py
or with shebang (#!/bin/env python) just: package/module.py
NB! Using sys.path.append instead of sys.path.insert will give you a hard to trace error if your module has the same name as your package. E.g. my_script/my_script.py
Of course if you have relative imports from higher levels in your package hierarchy, than this is not enough, but for most cases, it's just okay.
I needed to run python3 from the main project directory to make it work.
For example, if the project has the following structure:
project_demo/
├── main.py
├── some_package/
│ ├── __init__.py
│ └── project_configs.py
└── test/
└── test_project_configs.py
Solution
I would run python3 inside folder project_demo/ and then perform a
from some_package import project_configs
I was getting this ImportError: attempted relative import with no known parent package
In my program I was using the file from current path for importing its function.
from .filename import function
Then I modified the current path (Dot) with package name. Which resolved my issue.
from package_name.filename import function
I hope the above answer helps you.
Importing from same directory
Firstly, you can import from the same directory.
Here is the file structure...
Folder
|
├─ Scripts
| ├─ module123.py
|
├─ main.py
├─ script123.py
Here is main.py
from . import script123
from Scripts import module123
As you can see, importing from . imports from current directory.
Note: if running using anything but IDLE, make sure that your terminal is navigated to the same directory as the main.py file before running.
Also, importing from a local folder also works.
Importing from parent directory
As seen in my GitHub gist here, there is the following method.
Take the following file tree...
ParentDirectory
├─ Folder
| |
| ├─ Scripts
| | ├─ module123.py
| |
| ├─ main.py
| ├─ script123.py
|
├─ parentModule.py
Then, just add this code to the top of your main.py file.
import inspect
import os
import sys
current_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parent_dir = os.path.dirname(current_dir)
sys.path.insert(0, parent_dir)
from ParentDirectory import Stuff
I tried all of the above to no avail, only to realize I mistakenly had a - in my package name.
In short, don't have - in the directory where __init__.py is. I've never felt elated after finding out such inanity.
if both packages are in your import path (sys.path), and the module/class you want is in example/example.py, then to access the class without relative import try:
from example.example import fkt
If none of the above worked for you, you can specify the module explicitly.
Directory:
├── Project
│ ├── Dir
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
Solution:
#in standalone.py
from Project.Dir.module import ...
module - the module to be imported
Here is a three-liner for those who disagree with Guido:
import sys
from pathlib import Path
sys.path.append(str(Path(sys.argv[0]).absolute().parent.parent))
Hope it helps.
I think the best solution is to create a package for your module:
Here is more info on how to do it.
Once you have a package you don't need to worry about relative import, you can just do absolute imports.
I encounter this a lot when I am working with Django, since a lot of functionality is performed from the manage.py script but I also want to have some of my modules runnable directly as scripts as well (ideally you would make them manage.py directives but we're not there yet).
This is a mock up of what such a project might look like;
├── dj_app
│ ├── models.py
│ ├── ops
│ │ ├── bar.py
│ │ └── foo.py
│ ├── script.py
│ ├── tests.py
│ ├── utils.py
│ └── views.py
└── manage.py
The important parts here being manage.py, dj_app/script.py, and dj_app/tests.py. We also have submodules dj_app/ops/bar.py and dj_app/ops/foo.py which contain more items we want to use throughout the project.
The source of the issue commonly comes from wanting your dj_app/script.py script methods to have test cases in dj_app/tests.py which get invoked when you run manage.py test.
This is how I set up the project and its imports;
# dj_app/ops/foo.py
# Foo operation methods and classes
foo_val = "foo123"
.
# dj_app/ops/bar.py
# Bar operations methods and classes
bar_val = "bar123"
.
# dj_app/script.py
# script to run app methods from CLI
# if run directly from command line
if __name__ == '__main__':
from ops.bar import bar_val
from ops.foo import foo_val
# otherwise
else:
from .ops.bar import bar_val
from .ops.foo import foo_val
def script_method1():
print("this is script_method1")
print("bar_val: {}".format(bar_val))
print("foo_val: {}".format(foo_val))
if __name__ == '__main__':
print("running from the script")
script_method1()
.
# dj_app/tests.py
# test cases for the app
# do not run this directly from CLI or the imports will break
from .script import script_method1
from .ops.bar import bar_val
from .ops.foo import foo_val
def main():
print("Running the test case")
print("testing script method")
script_method1()
if __name__ == '__main__':
print("running tests from command line")
main()
.
# manage.py
# just run the test cases for this example
import dj_app.tests
dj_app.tests.main()
.
Running the test cases from manage.py;
$ python3 manage.py
Running the test case
testing script method
this is script_method1
bar_val: bar123
foo_val: foo123
Running the script on its own;
$ python3 dj_app/script.py
running from the script
this is script_method1
bar_val: bar123
foo_val: foo123
Note that you get an error if you try to run the test.py directly however, so don't do that;
$ python3 dj_app/tests.py
Traceback (most recent call last):
File "dj_app/tests.py", line 5, in <module>
from .script import script_method1
ModuleNotFoundError: No module named '__main__.script'; '__main__' is not a package
If I run into more complicated situations for imports, I usually end up implementing something like this to hack through it;
import os
import sys
THIS_DIR = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, THIS_DIR)
from script import script_method1
sys.path.pop(0)
This my project structure
├── folder
| |
│ ├── moduleA.py
| | |
| | └--function1()
| | └~~ uses function2()
| |
│ └── moduleB.py
| |
| └--function2()
|
└── main.py
└~~ uses function1()
Here my moduleA imports moduleB and main imports moduleA
I added the snippet below in moduleA to import moduleB
try:
from .moduleB import function2
except:
from moduleB import function2
Now I can execute both main.py as well as moduleA.py individually
Is this a solution ?
The below solution is tested on Python3
├── classes
| |
| ├──__init__.py
| |
│ ├── userclass.py
| | |
| | └--viewDetails()
| |
| |
│ └── groupclass.py
| |
| └--viewGroupDetails()
|
└── start.py
└~~ uses function1()
Now, in order to use viewDetails of userclass or viewGroupDetails of groupclass define that in _ init _.py of classess directory first.
Ex: In _ init _.py
from .userclasss import viewDetails
from .groupclass import viewGroupDetails
Step2: Now, in start.py we can directly import viewDetails
Ex: In start.py
from classes import viewDetails
from classes import viewGroupDetails
I ran into a similar problem when trying to write a python file that can be loaded either as a module or an executable script.
Setup
/path/to/project/
├── __init__.py
└── main.py
└── mylib/
├── list_util.py
└── args_util.py
with:
main.py:
#!/usr/bin/env python3
import sys
import mylib.args_util
if __name__ == '__main__':
print(f'{mylib.args_util.parseargs(sys.argv[1:])=}')
mylib/list_util.py:
def to_int_list(args):
return [int(x) for x in args]
mylib/args_util.py:
#!/usr/bin/env python3
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
Traceback (most recent call last):
File "/path/to/project/mylib/args_util.py", line 10, in <module>
from . import list_util as lu
ImportError: attempted relative import with no known parent package
Solution
I settled for a Bash/Python polyglot solution. The Bash version of the program just calls python3 -m mylib.args_util then exits.
The Python version ignores the Bash code because it's contained in the docstring.
The Bash version ignores the Python code because it uses exec to stop parsing/running lines.
mylib/args_util.py:
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 -m mylib.args_util "$#"
'''
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
parseargs(sys.argv[1:])=6
Explanation
Line 1: #!/bin/bash; this is the "shebang" line; it tells the interactive shell how run this script.
Python: ignored (comment)
Bash: ignored (comment)
Line 2: # -*- Mode: python -*- optional; this is called the "mode-line"; it tells Emacs to use Python syntax highlighting instead of guessing that the language is Bash when reading the file.
Python: ignored (comment)
Bash: ignored (comment)
Line 3: ''''true
Python: views this as an unassigned docstring starting with 'true\n
Bash: views this as three strings (of which the first two are empty strings) that expand to true (i.e. '' + '' + 'true' = 'true'); it then runs true (which does nothing) and continues to the next line
Line 4: exec /usr/bin/env python3 -m mylib.args_util "$#"
Python: still views this as part of the docstring from line 3.
Bash: runs python3 -m mylib.args_util then exits (it doesn't parse anything beyond this line)
Line 5: '''
Python: views this as the end of the docstring from line 3.
Bash: doesn't parse this line
Caveats
This doesn't work on Windows:
Workaround: Use WSL or a Batch wrapper script to call python -m mylib.args_util.
This only works if the current working directory is set to /path/to/project/.
Workaround: Set PYTHONPATH when calling /usr/bin/env
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 \
PYTHONPATH="$(cd "$(dirname "$0")/.." ; pwd)" \
-m mylib.args_util "$#"
'''
I've created a new, experimental import library for Python: ultraimport
It gives the programmer more control over imports and makes them unambiguous. Also it gives better error messages when an import fails.
It allows you to do relative, file-system based imports that always work, no matter how you run your code and no matter what is your current working directory. It does not matter if you run a script or module. You also don't have to change sys.path which might have other side effects.
You would then change
from .mymodule import myfunction
to
import ultraimport
myfunction = ultraimport('__dir__/mymodule.py', 'myfunction')
This way the import will always work, even if you run the code as script.
One issue when importing scripts like this is that subsequent relative imports might fail. ultraimport has a builtin preprocessor to automatically rewrite relative imports.
I had a similar problem: I needed a Linux service and cgi plugin which use common constants to cooperate. The 'natural' way to do this is to place them in the init.py of the package, but I cannot start the cgi plugin with the -m parameter.
My final solution was similar to Solution #2 above:
import sys
import pathlib as p
import importlib
pp = p.Path(sys.argv[0])
pack = pp.resolve().parent
pkg = importlib.import_module('__init__', package=str(pack))
The disadvantage is that you must prefix the constants (or common functions) with pkg:
print(pkg.Glob)
TLDR; Append Script path to the System Path by adding following in the entry point of your python script.
import os.path
import sys
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
Thats it now you can run your project in PyCharma as well as from Terminal!!
Moving the file from which you are importing to an outside directory helps.
This is extra useful when your main file makes any other files in its own directory.
Ex:
Before:
Project
|---dir1
|-------main.py
|-------module1.py
After:
Project
|---module1.py
|---dir1
|-------main.py
I was getting the same error and my project structure was like
->project
->vendors
->vendors.py
->main.py
I was trying to call like this
from .vendors.Amazon import Amazom_Purchase
Here it was throwing an error so I fixed it simply by removing the first . from the statement
from vendors.Amazon import Amazom_Purchase
Hope this helps.
It's good to note that sometimes the cache causes of all it - I've tried different things after re-arranging classes into new directories and relative import started to work after I removed the __pycache__
If the following import:
from . import something
doesn't work for you it is because this is python-packaging import and will not work with your regular implementation, and here is an example to show how to use it:
Folder structure:
.
└── funniest
├── funniest
│ ├── __init__.py
│ └── text.py
├── main.py
└── setup.py
inside __init__.py add:
def available_module():
return "hello world"
text.py add:
from . import available_module
inside setup.py add
from setuptools import setup
setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus#example.com',
license='MIT',
packages=['funniest'],
zip_safe=False)
Now, this is the most important part you need to install your package:
pip install .
Anywhere else in our system using the same Python, we can do this now:
>> import funnies.text as fun
>> fun.available_module()
This should output 'hello world'
you can test this in main.py (this will not require any installation of the Package)
Here is main.py as well
import funniest.text as fun
print(fun.available_module())
I want to import a function from another file in the same directory.
Usually, one of the following works:
from .mymodule import myfunction
from mymodule import myfunction
...but the other one gives me one of these errors:
ImportError: attempted relative import with no known parent package
ModuleNotFoundError: No module named 'mymodule'
SystemError: Parent module '' not loaded, cannot perform relative import
Why is this?
unfortunately, this module needs to be inside the package, and it also
needs to be runnable as a script, sometimes. Any idea how I could
achieve that?
It's quite common to have a layout like this...
main.py
mypackage/
__init__.py
mymodule.py
myothermodule.py
...with a mymodule.py like this...
#!/usr/bin/env python3
# Exported function
def as_int(a):
return int(a)
# Test function for module
def _test():
assert as_int('1') == 1
if __name__ == '__main__':
_test()
...a myothermodule.py like this...
#!/usr/bin/env python3
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
...and a main.py like this...
#!/usr/bin/env python3
from mypackage.myothermodule import add
def main():
print(add('1', '1'))
if __name__ == '__main__':
main()
...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...
from .mymodule import as_int
The way you're supposed to run it is...
python3 -m mypackage.myothermodule
...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.
The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...
from mymodule import as_int
...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...
from mypackage.mymodule import as_int
...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
from mypackage.mymodule import as_int
It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...
I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.
Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.
Explanation
From PEP 328
Relative imports use a module's __name__ attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to '__main__')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
At some point PEP 338 conflicted with PEP 328:
... relative imports rely on __name__ to determine the current
module's position in the package hierarchy. In a main module, the
value of __name__ is always '__main__', so explicit relative imports
will always fail (as they only work for a module inside a package)
and to address the issue, PEP 366 introduced the top level variable __package__:
By adding a new module level attribute, this PEP allows relative
imports to work automatically if the module is executed using the -m
switch. A small amount of boilerplate in the module itself will allow
the relative imports to work when the file is executed by name. [...] When it [the attribute] is present, relative imports will be based on this attribute
rather than the module __name__ attribute. [...] When the main module is specified by its filename, then the __package__ attribute will be set to None. [...] When the import system encounters an explicit relative import in a
module without __package__ set (or with it set to None), it will
calculate and store the correct value (__name__.rpartition('.')[0]
for normal modules and __name__ for package initialisation modules)
(emphasis mine)
If the __name__ is '__main__', __name__.rpartition('.')[0] returns empty string. This is why there's empty string literal in the error description:
SystemError: Parent module '' not loaded, cannot perform relative import
The relevant part of the CPython's PyImport_ImportModuleLevelObject function:
if (PyDict_GetItem(interp->modules, package) == NULL) {
PyErr_Format(PyExc_SystemError,
"Parent module %R not loaded, cannot perform relative "
"import", package);
goto error;
}
CPython raises this exception if it was unable to find package (the name of the package) in interp->modules (accessible as sys.modules). Since sys.modules is "a dictionary that maps module names to modules which have already been loaded", it's now clear that the parent module must be explicitly absolute-imported before performing relative import.
Note: The patch from the issue 18018 has added another if block, which will be executed before the code above:
if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
PyErr_SetString(PyExc_ImportError,
"attempted relative import with no known parent package");
goto error;
} /* else if (PyDict_GetItem(interp->modules, package) == NULL) {
...
*/
If package (same as above) is empty string, the error message will be
ImportError: attempted relative import with no known parent package
However, you will only see this in Python 3.6 or newer.
Solution #1: Run your script using -m
Consider a directory (which is a Python package):
.
├── package
│ ├── __init__.py
│ ├── module.py
│ └── standalone.py
All of the files in package begin with the same 2 lines of code:
from pathlib import Path
print('Running' if __name__ == '__main__' else 'Importing', Path(__file__).resolve())
I'm including these two lines only to make the order of operations obvious. We can ignore them completely, since they don't affect the execution.
__init__.py and module.py contain only those two lines (i.e., they are effectively empty).
standalone.py additionally attempts to import module.py via relative import:
from . import module # explicit relative import
We're well aware that /path/to/python/interpreter package/standalone.py will fail. However, we can run the module with the -m command line option that will "search sys.path for the named module and execute its contents as the __main__ module":
vaultah#base:~$ python3 -i -m package.standalone
Importing /home/vaultah/package/__init__.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/module.py
>>> __file__
'/home/vaultah/package/standalone.py'
>>> __package__
'package'
>>> # The __package__ has been correctly set and module.py has been imported.
... # What's inside sys.modules?
... import sys
>>> sys.modules['__main__']
<module 'package.standalone' from '/home/vaultah/package/standalone.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
-m does all the importing stuff for you and automatically sets __package__, but you can do that yourself in the
Solution #2: Set __package__ manually
Please treat it as a proof of concept rather than an actual solution. It isn't well-suited for use in real-world code.
PEP 366 has a workaround to this problem, however, it's incomplete, because setting __package__ alone is not enough. You're going to need to import at least N preceding packages in the module hierarchy, where N is the number of parent directories (relative to the directory of the script) that will be searched for the module being imported.
Thus,
Add the parent directory of the Nth predecessor of the current module to sys.path
Remove the current file's directory from sys.path
Import the parent module of the current module using its fully-qualified name
Set __package__ to the fully-qualified name from 2
Perform the relative import
I'll borrow files from the Solution #1 and add some more subpackages:
package
├── __init__.py
├── module.py
└── subpackage
├── __init__.py
└── subsubpackage
├── __init__.py
└── standalone.py
This time standalone.py will import module.py from the package package using the following relative import
from ... import module # N = 3
We'll need to precede that line with the boilerplate code, to make it work.
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[3]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import package.subpackage.subsubpackage
__package__ = 'package.subpackage.subsubpackage'
from ... import module # N = 3
It allows us to execute standalone.py by filename:
vaultah#base:~$ python3 package/subpackage/subsubpackage/standalone.py
Running /home/vaultah/package/subpackage/subsubpackage/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/subpackage/__init__.py
Importing /home/vaultah/package/subpackage/subsubpackage/__init__.py
Importing /home/vaultah/package/module.py
A more general solution wrapped in a function can be found here. Example usage:
if __name__ == '__main__' and __package__ is None:
import_parents(level=3) # N = 3
from ... import module
from ...module.submodule import thing
Solution #3: Use absolute imports and setuptools
The steps are -
Replace explicit relative imports with equivalent absolute imports
Install package to make it importable
For instance, the directory structure may be as follows
.
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
│ └── setup.py
where setup.py is
from setuptools import setup, find_packages
setup(
name = 'your_package_name',
packages = find_packages(),
)
The rest of the files were borrowed from the Solution #1.
Installation will allow you to import the package regardless of your working directory (assuming there'll be no naming issues).
We can modify standalone.py to use this advantage (step 1):
from package import module # absolute import
Change your working directory to project and run /path/to/python/interpreter setup.py install --user (--user installs the package in your site-packages directory) (step 2):
vaultah#base:~$ cd project
vaultah#base:~/project$ python3 setup.py install --user
Let's verify that it's now possible to run standalone.py as a script:
vaultah#base:~/project$ python3 -i package/standalone.py
Running /home/vaultah/project/package/standalone.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
Note: If you decide to go down this route, you'd be better off using virtual environments to install packages in isolation.
Solution #4: Use absolute imports and some boilerplate code
Frankly, the installation is not necessary - you could add some boilerplate code to your script to make absolute imports work.
I'm going to borrow files from Solution #1 and change standalone.py:
Add the parent directory of package to sys.path before attempting to import anything from package using absolute imports:
import sys
from pathlib import Path # if you haven't already done so
file = Path(__file__).resolve()
parent, root = file.parent, file.parents[1]
sys.path.append(str(root))
# Additionally remove the current file's directory from sys.path
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
Replace the relative import by the absolute import:
from package import module # absolute import
standalone.py runs without problems:
vaultah#base:~$ python3 -i package/standalone.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
I feel that I should warn you: try not to do this, especially if your project has a complex structure.
As a side note, PEP 8 recommends the use of absolute imports, but states that in some scenarios explicit relative imports are acceptable:
Absolute imports are recommended, as they are usually more readable
and tend to be better behaved (or at least give better error
messages). [...] However, explicit relative imports are an acceptable
alternative to absolute imports, especially when dealing with complex
package layouts where using absolute imports would be unnecessarily
verbose.
Put this inside your package's __init__.py file:
# For relative imports to work in Python 3.6
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Assuming your package is like this:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module1.py
│ │ └── module2.py
│ └── setup.py
Now use regular imports in you package, like:
# in module2.py
from module1 import class1
This works in both python 2 and 3.
I ran into this issue. A hack workaround is importing via an if/else block like follows:
#!/usr/bin/env python3
#myothermodule
if __name__ == '__main__':
from mymodule import as_int
else:
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
SystemError: Parent module '' not loaded, cannot perform relative import
This means you are running a module inside the package as a script. Mixing scripts inside packages is tricky and should be avoided if at all possible. Use a wrapper script that imports the package and runs your scripty function instead.
If your top-level directory is called foo, which is on your PYTHONPATH module search path, and you have a package bar there (it is a directory you'd expect an __init__.py file in), scripts should not be placed inside bar, but should live on in foo at best.
Note that scripts differ from modules here in that they are used as a filename argument to the python command, either by using python <filename> or via a #! (shebang) line. It is loaded directly as the __main__ module (this is why if __name__ == "__main__": works in scripts), and there is no package context to build on for relative imports.
Your options
If you can, package your project with setuptools (or poetry or flit, which can help simplify packaging), and create console script entrypoints; installing your project with pip then creates scripts that know how to import your package properly. You can install your package locally with pip install -e ., so it can still be edited in-place.
Otherwise, never, ever, use python path/to/packagename/file.py, always use python path/to/script.py and script.py can use from packagename import ....
As a fallback, you could use the -m command-line switch to tell Python to import a module and use that as the __main__ file instead. This does not work with a shebang line, as there is no script file any more, however.
If you use python -m foo.bar and foo/bar.py is found in a sys.path directory, that is then imported and executed as __main__ with the right package context. If bar is also a package, inside foo/, it must have a __main__.py file (so foo/bar/__main__.py as the path from the sys.path directory).
In extreme circumstances, add the metadata Python uses to resolve relative imports by setting __package__ directly; the file foo/bar/spam.py, importable as foo.bar.spam, is given the global __package__ = "foo.bar". It is just another global, like __file__ and __name__, set by Python when imported.
On sys.path
The above all requires that your package can be imported, which means it needs to be found in one of the directories (or zipfiles) listed in sys.path. There are several options here too:
The directory where path/to/script.py was found (so path/to) is automatically added to sys.path. Executing python path/to/foo.py adds path/to to sys.path.
If you packaged your project (with setuptools, poetry, flit or another Python packaging tool), and installed it, the package has been added to the right place already.
As a last resort, add the right directory to sys.path yourself. If the package can be located relatively to the script file, use the __file__ variable in the script global namespace (e.g. using the pathlib.Path object, HERE = Path(__file__).resolve().parent is a reference to the directory the file lives in, as absolute path).
For PyCharm users:
I also was getting ImportError: attempted relative import with no known parent package because I was adding the . notation to silence a PyCharm parsing error. PyCharm innaccurately reports not being able to find:
lib.thing import function
If you change it to:
.lib.thing import function
it silences the error but then you get the aforementioned ImportError: attempted relative import with no known parent package. Just ignore PyCharm's parser. It's wrong and the code runs fine despite what it says.
To obviate this problem, I devised a solution with the repackage package, which has worked for me for some time. It adds the upper directory to the lib path:
import repackage
repackage.up()
from mypackage.mymodule import myfunction
Repackage can make relative imports that work in a wide range of cases, using an intelligent strategy (inspecting the call stack).
TL;DR: to #Aya's answer, updated with pathlib library, and working for Jupyter notebooks where __file__ is not defined:
You want to import my_function defined under ../my_Folder_where_the_package_lives/my_package.py
respect to where you are writing the code.
Then do:
import os
import sys
import pathlib
PACKAGE_PARENT = pathlib.Path(__file__).parent
#PACKAGE_PARENT = pathlib.Path.cwd().parent # if on jupyter notebook
SCRIPT_DIR = PACKAGE_PARENT / "my_Folder_where_the_package_lives"
sys.path.append(str(SCRIPT_DIR))
from my_package import my_function
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
TL;DR
You can only relatively import modules inside another module in the same package.
Concept Clarify
We see a lot of example code in books/docs/articles, they show us how to relatively import a module, but when we do so, it fails.
The reason is, put it in a simple sentence, we did not run the code as the python module mechanism expects, even though the code is written totally right. It's like some kind of runtime thing.
Module loading is depended on how you run the code. That is the source of confusion.
What is a module?
A module is a python file when and only when it is being imported by another file. Given a file mod.py, is it a module? Yes and No, if you run python mod.py, it is not a module, because it is not imported.
What is a package?
A package is a folder that includes Python module(s).
BTW, __init__.py is not necessary from python 3.3, if you don't need any package initialization or auto-load submodules. You don't need to place a blank __init__.py in a directory.
That proves a package is just a folder as long as there are files being imported.
Real Answer
Now, this description becomes clearer.
You can only relatively import modules inside another module in the same package.
Given a directory:
. CWD
|-- happy_maker.py # content: print('Sends Happy')
`-- me.py # content: from . import happy_maker
Run python me.py, we got attempted relative import with no known parent package
me.py is run directly, it is not a module, and we can't use relative import in it.
Solution 1
Use import happy_maker instead of from . import happy_maker
Solution 2
Switch our working directory to the parent folder.
. CWD
|-- happy
| |-- happy_maker.py
`-- me.py
Run python -m happy.me.
When we are in the directory that includes happy, happy is a package, me.py, happy_maker.py are modules, we can use relative import now, and we still want to run me.py, so we use -m which means run the module as a script.
Python Idiom
. CWD
|-- happy
| |-- happy_maker.py # content: print('Sends Happy')
| `-- me.py # content: from . import happy_maker
`-- main.py # content: import happy.me
This structure is the python idiom. main is our script, best practice in Python. Finally, we got there.
Siblings or Grandparents
Another common need:
.
|-- happy
| |-- happy_maker.py
| `-- me.py
`-- sad
`-- sad_maker.py
We want to import sad_maker in me.py, How to do that?
First, we need to make happy and sad in the same package, so we have to go up a directory level. And then from ..sad import sad_maker in the me.py.
That is all.
My boilerplate to make a module with relative imports in a package runnable standalone.
package/module.py
## Standalone boilerplate before relative imports
if __package__ is None:
DIR = Path(__file__).resolve().parent
sys.path.insert(0, str(DIR.parent))
__package__ = DIR.name
from . import variable_in__init__py
from . import other_module_in_package
...
Now you can use your module in any fashion:
Run module as usual: python -m package.module
Use it as a module: python -c 'from package import module'
Run it standalone: python package/module.py
or with shebang (#!/bin/env python) just: package/module.py
NB! Using sys.path.append instead of sys.path.insert will give you a hard to trace error if your module has the same name as your package. E.g. my_script/my_script.py
Of course if you have relative imports from higher levels in your package hierarchy, than this is not enough, but for most cases, it's just okay.
I needed to run python3 from the main project directory to make it work.
For example, if the project has the following structure:
project_demo/
├── main.py
├── some_package/
│ ├── __init__.py
│ └── project_configs.py
└── test/
└── test_project_configs.py
Solution
I would run python3 inside folder project_demo/ and then perform a
from some_package import project_configs
I was getting this ImportError: attempted relative import with no known parent package
In my program I was using the file from current path for importing its function.
from .filename import function
Then I modified the current path (Dot) with package name. Which resolved my issue.
from package_name.filename import function
I hope the above answer helps you.
Importing from same directory
Firstly, you can import from the same directory.
Here is the file structure...
Folder
|
├─ Scripts
| ├─ module123.py
|
├─ main.py
├─ script123.py
Here is main.py
from . import script123
from Scripts import module123
As you can see, importing from . imports from current directory.
Note: if running using anything but IDLE, make sure that your terminal is navigated to the same directory as the main.py file before running.
Also, importing from a local folder also works.
Importing from parent directory
As seen in my GitHub gist here, there is the following method.
Take the following file tree...
ParentDirectory
├─ Folder
| |
| ├─ Scripts
| | ├─ module123.py
| |
| ├─ main.py
| ├─ script123.py
|
├─ parentModule.py
Then, just add this code to the top of your main.py file.
import inspect
import os
import sys
current_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parent_dir = os.path.dirname(current_dir)
sys.path.insert(0, parent_dir)
from ParentDirectory import Stuff
I tried all of the above to no avail, only to realize I mistakenly had a - in my package name.
In short, don't have - in the directory where __init__.py is. I've never felt elated after finding out such inanity.
if both packages are in your import path (sys.path), and the module/class you want is in example/example.py, then to access the class without relative import try:
from example.example import fkt
If none of the above worked for you, you can specify the module explicitly.
Directory:
├── Project
│ ├── Dir
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
Solution:
#in standalone.py
from Project.Dir.module import ...
module - the module to be imported
Here is a three-liner for those who disagree with Guido:
import sys
from pathlib import Path
sys.path.append(str(Path(sys.argv[0]).absolute().parent.parent))
Hope it helps.
I think the best solution is to create a package for your module:
Here is more info on how to do it.
Once you have a package you don't need to worry about relative import, you can just do absolute imports.
I encounter this a lot when I am working with Django, since a lot of functionality is performed from the manage.py script but I also want to have some of my modules runnable directly as scripts as well (ideally you would make them manage.py directives but we're not there yet).
This is a mock up of what such a project might look like;
├── dj_app
│ ├── models.py
│ ├── ops
│ │ ├── bar.py
│ │ └── foo.py
│ ├── script.py
│ ├── tests.py
│ ├── utils.py
│ └── views.py
└── manage.py
The important parts here being manage.py, dj_app/script.py, and dj_app/tests.py. We also have submodules dj_app/ops/bar.py and dj_app/ops/foo.py which contain more items we want to use throughout the project.
The source of the issue commonly comes from wanting your dj_app/script.py script methods to have test cases in dj_app/tests.py which get invoked when you run manage.py test.
This is how I set up the project and its imports;
# dj_app/ops/foo.py
# Foo operation methods and classes
foo_val = "foo123"
.
# dj_app/ops/bar.py
# Bar operations methods and classes
bar_val = "bar123"
.
# dj_app/script.py
# script to run app methods from CLI
# if run directly from command line
if __name__ == '__main__':
from ops.bar import bar_val
from ops.foo import foo_val
# otherwise
else:
from .ops.bar import bar_val
from .ops.foo import foo_val
def script_method1():
print("this is script_method1")
print("bar_val: {}".format(bar_val))
print("foo_val: {}".format(foo_val))
if __name__ == '__main__':
print("running from the script")
script_method1()
.
# dj_app/tests.py
# test cases for the app
# do not run this directly from CLI or the imports will break
from .script import script_method1
from .ops.bar import bar_val
from .ops.foo import foo_val
def main():
print("Running the test case")
print("testing script method")
script_method1()
if __name__ == '__main__':
print("running tests from command line")
main()
.
# manage.py
# just run the test cases for this example
import dj_app.tests
dj_app.tests.main()
.
Running the test cases from manage.py;
$ python3 manage.py
Running the test case
testing script method
this is script_method1
bar_val: bar123
foo_val: foo123
Running the script on its own;
$ python3 dj_app/script.py
running from the script
this is script_method1
bar_val: bar123
foo_val: foo123
Note that you get an error if you try to run the test.py directly however, so don't do that;
$ python3 dj_app/tests.py
Traceback (most recent call last):
File "dj_app/tests.py", line 5, in <module>
from .script import script_method1
ModuleNotFoundError: No module named '__main__.script'; '__main__' is not a package
If I run into more complicated situations for imports, I usually end up implementing something like this to hack through it;
import os
import sys
THIS_DIR = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, THIS_DIR)
from script import script_method1
sys.path.pop(0)
This my project structure
├── folder
| |
│ ├── moduleA.py
| | |
| | └--function1()
| | └~~ uses function2()
| |
│ └── moduleB.py
| |
| └--function2()
|
└── main.py
└~~ uses function1()
Here my moduleA imports moduleB and main imports moduleA
I added the snippet below in moduleA to import moduleB
try:
from .moduleB import function2
except:
from moduleB import function2
Now I can execute both main.py as well as moduleA.py individually
Is this a solution ?
The below solution is tested on Python3
├── classes
| |
| ├──__init__.py
| |
│ ├── userclass.py
| | |
| | └--viewDetails()
| |
| |
│ └── groupclass.py
| |
| └--viewGroupDetails()
|
└── start.py
└~~ uses function1()
Now, in order to use viewDetails of userclass or viewGroupDetails of groupclass define that in _ init _.py of classess directory first.
Ex: In _ init _.py
from .userclasss import viewDetails
from .groupclass import viewGroupDetails
Step2: Now, in start.py we can directly import viewDetails
Ex: In start.py
from classes import viewDetails
from classes import viewGroupDetails
I ran into a similar problem when trying to write a python file that can be loaded either as a module or an executable script.
Setup
/path/to/project/
├── __init__.py
└── main.py
└── mylib/
├── list_util.py
└── args_util.py
with:
main.py:
#!/usr/bin/env python3
import sys
import mylib.args_util
if __name__ == '__main__':
print(f'{mylib.args_util.parseargs(sys.argv[1:])=}')
mylib/list_util.py:
def to_int_list(args):
return [int(x) for x in args]
mylib/args_util.py:
#!/usr/bin/env python3
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
Traceback (most recent call last):
File "/path/to/project/mylib/args_util.py", line 10, in <module>
from . import list_util as lu
ImportError: attempted relative import with no known parent package
Solution
I settled for a Bash/Python polyglot solution. The Bash version of the program just calls python3 -m mylib.args_util then exits.
The Python version ignores the Bash code because it's contained in the docstring.
The Bash version ignores the Python code because it uses exec to stop parsing/running lines.
mylib/args_util.py:
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 -m mylib.args_util "$#"
'''
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
parseargs(sys.argv[1:])=6
Explanation
Line 1: #!/bin/bash; this is the "shebang" line; it tells the interactive shell how run this script.
Python: ignored (comment)
Bash: ignored (comment)
Line 2: # -*- Mode: python -*- optional; this is called the "mode-line"; it tells Emacs to use Python syntax highlighting instead of guessing that the language is Bash when reading the file.
Python: ignored (comment)
Bash: ignored (comment)
Line 3: ''''true
Python: views this as an unassigned docstring starting with 'true\n
Bash: views this as three strings (of which the first two are empty strings) that expand to true (i.e. '' + '' + 'true' = 'true'); it then runs true (which does nothing) and continues to the next line
Line 4: exec /usr/bin/env python3 -m mylib.args_util "$#"
Python: still views this as part of the docstring from line 3.
Bash: runs python3 -m mylib.args_util then exits (it doesn't parse anything beyond this line)
Line 5: '''
Python: views this as the end of the docstring from line 3.
Bash: doesn't parse this line
Caveats
This doesn't work on Windows:
Workaround: Use WSL or a Batch wrapper script to call python -m mylib.args_util.
This only works if the current working directory is set to /path/to/project/.
Workaround: Set PYTHONPATH when calling /usr/bin/env
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 \
PYTHONPATH="$(cd "$(dirname "$0")/.." ; pwd)" \
-m mylib.args_util "$#"
'''
I've created a new, experimental import library for Python: ultraimport
It gives the programmer more control over imports and makes them unambiguous. Also it gives better error messages when an import fails.
It allows you to do relative, file-system based imports that always work, no matter how you run your code and no matter what is your current working directory. It does not matter if you run a script or module. You also don't have to change sys.path which might have other side effects.
You would then change
from .mymodule import myfunction
to
import ultraimport
myfunction = ultraimport('__dir__/mymodule.py', 'myfunction')
This way the import will always work, even if you run the code as script.
One issue when importing scripts like this is that subsequent relative imports might fail. ultraimport has a builtin preprocessor to automatically rewrite relative imports.
I had a similar problem: I needed a Linux service and cgi plugin which use common constants to cooperate. The 'natural' way to do this is to place them in the init.py of the package, but I cannot start the cgi plugin with the -m parameter.
My final solution was similar to Solution #2 above:
import sys
import pathlib as p
import importlib
pp = p.Path(sys.argv[0])
pack = pp.resolve().parent
pkg = importlib.import_module('__init__', package=str(pack))
The disadvantage is that you must prefix the constants (or common functions) with pkg:
print(pkg.Glob)
TLDR; Append Script path to the System Path by adding following in the entry point of your python script.
import os.path
import sys
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
Thats it now you can run your project in PyCharma as well as from Terminal!!
Moving the file from which you are importing to an outside directory helps.
This is extra useful when your main file makes any other files in its own directory.
Ex:
Before:
Project
|---dir1
|-------main.py
|-------module1.py
After:
Project
|---module1.py
|---dir1
|-------main.py
I was getting the same error and my project structure was like
->project
->vendors
->vendors.py
->main.py
I was trying to call like this
from .vendors.Amazon import Amazom_Purchase
Here it was throwing an error so I fixed it simply by removing the first . from the statement
from vendors.Amazon import Amazom_Purchase
Hope this helps.
It's good to note that sometimes the cache causes of all it - I've tried different things after re-arranging classes into new directories and relative import started to work after I removed the __pycache__
If the following import:
from . import something
doesn't work for you it is because this is python-packaging import and will not work with your regular implementation, and here is an example to show how to use it:
Folder structure:
.
└── funniest
├── funniest
│ ├── __init__.py
│ └── text.py
├── main.py
└── setup.py
inside __init__.py add:
def available_module():
return "hello world"
text.py add:
from . import available_module
inside setup.py add
from setuptools import setup
setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus#example.com',
license='MIT',
packages=['funniest'],
zip_safe=False)
Now, this is the most important part you need to install your package:
pip install .
Anywhere else in our system using the same Python, we can do this now:
>> import funnies.text as fun
>> fun.available_module()
This should output 'hello world'
you can test this in main.py (this will not require any installation of the Package)
Here is main.py as well
import funniest.text as fun
print(fun.available_module())
I want to import a function from another file in the same directory.
Usually, one of the following works:
from .mymodule import myfunction
from mymodule import myfunction
...but the other one gives me one of these errors:
ImportError: attempted relative import with no known parent package
ModuleNotFoundError: No module named 'mymodule'
SystemError: Parent module '' not loaded, cannot perform relative import
Why is this?
unfortunately, this module needs to be inside the package, and it also
needs to be runnable as a script, sometimes. Any idea how I could
achieve that?
It's quite common to have a layout like this...
main.py
mypackage/
__init__.py
mymodule.py
myothermodule.py
...with a mymodule.py like this...
#!/usr/bin/env python3
# Exported function
def as_int(a):
return int(a)
# Test function for module
def _test():
assert as_int('1') == 1
if __name__ == '__main__':
_test()
...a myothermodule.py like this...
#!/usr/bin/env python3
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
...and a main.py like this...
#!/usr/bin/env python3
from mypackage.myothermodule import add
def main():
print(add('1', '1'))
if __name__ == '__main__':
main()
...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...
from .mymodule import as_int
The way you're supposed to run it is...
python3 -m mypackage.myothermodule
...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.
The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...
from mymodule import as_int
...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...
from mypackage.mymodule import as_int
...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
from mypackage.mymodule import as_int
It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...
I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.
Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.
Explanation
From PEP 328
Relative imports use a module's __name__ attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to '__main__')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
At some point PEP 338 conflicted with PEP 328:
... relative imports rely on __name__ to determine the current
module's position in the package hierarchy. In a main module, the
value of __name__ is always '__main__', so explicit relative imports
will always fail (as they only work for a module inside a package)
and to address the issue, PEP 366 introduced the top level variable __package__:
By adding a new module level attribute, this PEP allows relative
imports to work automatically if the module is executed using the -m
switch. A small amount of boilerplate in the module itself will allow
the relative imports to work when the file is executed by name. [...] When it [the attribute] is present, relative imports will be based on this attribute
rather than the module __name__ attribute. [...] When the main module is specified by its filename, then the __package__ attribute will be set to None. [...] When the import system encounters an explicit relative import in a
module without __package__ set (or with it set to None), it will
calculate and store the correct value (__name__.rpartition('.')[0]
for normal modules and __name__ for package initialisation modules)
(emphasis mine)
If the __name__ is '__main__', __name__.rpartition('.')[0] returns empty string. This is why there's empty string literal in the error description:
SystemError: Parent module '' not loaded, cannot perform relative import
The relevant part of the CPython's PyImport_ImportModuleLevelObject function:
if (PyDict_GetItem(interp->modules, package) == NULL) {
PyErr_Format(PyExc_SystemError,
"Parent module %R not loaded, cannot perform relative "
"import", package);
goto error;
}
CPython raises this exception if it was unable to find package (the name of the package) in interp->modules (accessible as sys.modules). Since sys.modules is "a dictionary that maps module names to modules which have already been loaded", it's now clear that the parent module must be explicitly absolute-imported before performing relative import.
Note: The patch from the issue 18018 has added another if block, which will be executed before the code above:
if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
PyErr_SetString(PyExc_ImportError,
"attempted relative import with no known parent package");
goto error;
} /* else if (PyDict_GetItem(interp->modules, package) == NULL) {
...
*/
If package (same as above) is empty string, the error message will be
ImportError: attempted relative import with no known parent package
However, you will only see this in Python 3.6 or newer.
Solution #1: Run your script using -m
Consider a directory (which is a Python package):
.
├── package
│ ├── __init__.py
│ ├── module.py
│ └── standalone.py
All of the files in package begin with the same 2 lines of code:
from pathlib import Path
print('Running' if __name__ == '__main__' else 'Importing', Path(__file__).resolve())
I'm including these two lines only to make the order of operations obvious. We can ignore them completely, since they don't affect the execution.
__init__.py and module.py contain only those two lines (i.e., they are effectively empty).
standalone.py additionally attempts to import module.py via relative import:
from . import module # explicit relative import
We're well aware that /path/to/python/interpreter package/standalone.py will fail. However, we can run the module with the -m command line option that will "search sys.path for the named module and execute its contents as the __main__ module":
vaultah#base:~$ python3 -i -m package.standalone
Importing /home/vaultah/package/__init__.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/module.py
>>> __file__
'/home/vaultah/package/standalone.py'
>>> __package__
'package'
>>> # The __package__ has been correctly set and module.py has been imported.
... # What's inside sys.modules?
... import sys
>>> sys.modules['__main__']
<module 'package.standalone' from '/home/vaultah/package/standalone.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
-m does all the importing stuff for you and automatically sets __package__, but you can do that yourself in the
Solution #2: Set __package__ manually
Please treat it as a proof of concept rather than an actual solution. It isn't well-suited for use in real-world code.
PEP 366 has a workaround to this problem, however, it's incomplete, because setting __package__ alone is not enough. You're going to need to import at least N preceding packages in the module hierarchy, where N is the number of parent directories (relative to the directory of the script) that will be searched for the module being imported.
Thus,
Add the parent directory of the Nth predecessor of the current module to sys.path
Remove the current file's directory from sys.path
Import the parent module of the current module using its fully-qualified name
Set __package__ to the fully-qualified name from 2
Perform the relative import
I'll borrow files from the Solution #1 and add some more subpackages:
package
├── __init__.py
├── module.py
└── subpackage
├── __init__.py
└── subsubpackage
├── __init__.py
└── standalone.py
This time standalone.py will import module.py from the package package using the following relative import
from ... import module # N = 3
We'll need to precede that line with the boilerplate code, to make it work.
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[3]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import package.subpackage.subsubpackage
__package__ = 'package.subpackage.subsubpackage'
from ... import module # N = 3
It allows us to execute standalone.py by filename:
vaultah#base:~$ python3 package/subpackage/subsubpackage/standalone.py
Running /home/vaultah/package/subpackage/subsubpackage/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/subpackage/__init__.py
Importing /home/vaultah/package/subpackage/subsubpackage/__init__.py
Importing /home/vaultah/package/module.py
A more general solution wrapped in a function can be found here. Example usage:
if __name__ == '__main__' and __package__ is None:
import_parents(level=3) # N = 3
from ... import module
from ...module.submodule import thing
Solution #3: Use absolute imports and setuptools
The steps are -
Replace explicit relative imports with equivalent absolute imports
Install package to make it importable
For instance, the directory structure may be as follows
.
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
│ └── setup.py
where setup.py is
from setuptools import setup, find_packages
setup(
name = 'your_package_name',
packages = find_packages(),
)
The rest of the files were borrowed from the Solution #1.
Installation will allow you to import the package regardless of your working directory (assuming there'll be no naming issues).
We can modify standalone.py to use this advantage (step 1):
from package import module # absolute import
Change your working directory to project and run /path/to/python/interpreter setup.py install --user (--user installs the package in your site-packages directory) (step 2):
vaultah#base:~$ cd project
vaultah#base:~/project$ python3 setup.py install --user
Let's verify that it's now possible to run standalone.py as a script:
vaultah#base:~/project$ python3 -i package/standalone.py
Running /home/vaultah/project/package/standalone.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
Note: If you decide to go down this route, you'd be better off using virtual environments to install packages in isolation.
Solution #4: Use absolute imports and some boilerplate code
Frankly, the installation is not necessary - you could add some boilerplate code to your script to make absolute imports work.
I'm going to borrow files from Solution #1 and change standalone.py:
Add the parent directory of package to sys.path before attempting to import anything from package using absolute imports:
import sys
from pathlib import Path # if you haven't already done so
file = Path(__file__).resolve()
parent, root = file.parent, file.parents[1]
sys.path.append(str(root))
# Additionally remove the current file's directory from sys.path
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
Replace the relative import by the absolute import:
from package import module # absolute import
standalone.py runs without problems:
vaultah#base:~$ python3 -i package/standalone.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
I feel that I should warn you: try not to do this, especially if your project has a complex structure.
As a side note, PEP 8 recommends the use of absolute imports, but states that in some scenarios explicit relative imports are acceptable:
Absolute imports are recommended, as they are usually more readable
and tend to be better behaved (or at least give better error
messages). [...] However, explicit relative imports are an acceptable
alternative to absolute imports, especially when dealing with complex
package layouts where using absolute imports would be unnecessarily
verbose.
Put this inside your package's __init__.py file:
# For relative imports to work in Python 3.6
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Assuming your package is like this:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module1.py
│ │ └── module2.py
│ └── setup.py
Now use regular imports in you package, like:
# in module2.py
from module1 import class1
This works in both python 2 and 3.
I ran into this issue. A hack workaround is importing via an if/else block like follows:
#!/usr/bin/env python3
#myothermodule
if __name__ == '__main__':
from mymodule import as_int
else:
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
SystemError: Parent module '' not loaded, cannot perform relative import
This means you are running a module inside the package as a script. Mixing scripts inside packages is tricky and should be avoided if at all possible. Use a wrapper script that imports the package and runs your scripty function instead.
If your top-level directory is called foo, which is on your PYTHONPATH module search path, and you have a package bar there (it is a directory you'd expect an __init__.py file in), scripts should not be placed inside bar, but should live on in foo at best.
Note that scripts differ from modules here in that they are used as a filename argument to the python command, either by using python <filename> or via a #! (shebang) line. It is loaded directly as the __main__ module (this is why if __name__ == "__main__": works in scripts), and there is no package context to build on for relative imports.
Your options
If you can, package your project with setuptools (or poetry or flit, which can help simplify packaging), and create console script entrypoints; installing your project with pip then creates scripts that know how to import your package properly. You can install your package locally with pip install -e ., so it can still be edited in-place.
Otherwise, never, ever, use python path/to/packagename/file.py, always use python path/to/script.py and script.py can use from packagename import ....
As a fallback, you could use the -m command-line switch to tell Python to import a module and use that as the __main__ file instead. This does not work with a shebang line, as there is no script file any more, however.
If you use python -m foo.bar and foo/bar.py is found in a sys.path directory, that is then imported and executed as __main__ with the right package context. If bar is also a package, inside foo/, it must have a __main__.py file (so foo/bar/__main__.py as the path from the sys.path directory).
In extreme circumstances, add the metadata Python uses to resolve relative imports by setting __package__ directly; the file foo/bar/spam.py, importable as foo.bar.spam, is given the global __package__ = "foo.bar". It is just another global, like __file__ and __name__, set by Python when imported.
On sys.path
The above all requires that your package can be imported, which means it needs to be found in one of the directories (or zipfiles) listed in sys.path. There are several options here too:
The directory where path/to/script.py was found (so path/to) is automatically added to sys.path. Executing python path/to/foo.py adds path/to to sys.path.
If you packaged your project (with setuptools, poetry, flit or another Python packaging tool), and installed it, the package has been added to the right place already.
As a last resort, add the right directory to sys.path yourself. If the package can be located relatively to the script file, use the __file__ variable in the script global namespace (e.g. using the pathlib.Path object, HERE = Path(__file__).resolve().parent is a reference to the directory the file lives in, as absolute path).
For PyCharm users:
I also was getting ImportError: attempted relative import with no known parent package because I was adding the . notation to silence a PyCharm parsing error. PyCharm innaccurately reports not being able to find:
lib.thing import function
If you change it to:
.lib.thing import function
it silences the error but then you get the aforementioned ImportError: attempted relative import with no known parent package. Just ignore PyCharm's parser. It's wrong and the code runs fine despite what it says.
To obviate this problem, I devised a solution with the repackage package, which has worked for me for some time. It adds the upper directory to the lib path:
import repackage
repackage.up()
from mypackage.mymodule import myfunction
Repackage can make relative imports that work in a wide range of cases, using an intelligent strategy (inspecting the call stack).
TL;DR: to #Aya's answer, updated with pathlib library, and working for Jupyter notebooks where __file__ is not defined:
You want to import my_function defined under ../my_Folder_where_the_package_lives/my_package.py
respect to where you are writing the code.
Then do:
import os
import sys
import pathlib
PACKAGE_PARENT = pathlib.Path(__file__).parent
#PACKAGE_PARENT = pathlib.Path.cwd().parent # if on jupyter notebook
SCRIPT_DIR = PACKAGE_PARENT / "my_Folder_where_the_package_lives"
sys.path.append(str(SCRIPT_DIR))
from my_package import my_function
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
TL;DR
You can only relatively import modules inside another module in the same package.
Concept Clarify
We see a lot of example code in books/docs/articles, they show us how to relatively import a module, but when we do so, it fails.
The reason is, put it in a simple sentence, we did not run the code as the python module mechanism expects, even though the code is written totally right. It's like some kind of runtime thing.
Module loading is depended on how you run the code. That is the source of confusion.
What is a module?
A module is a python file when and only when it is being imported by another file. Given a file mod.py, is it a module? Yes and No, if you run python mod.py, it is not a module, because it is not imported.
What is a package?
A package is a folder that includes Python module(s).
BTW, __init__.py is not necessary from python 3.3, if you don't need any package initialization or auto-load submodules. You don't need to place a blank __init__.py in a directory.
That proves a package is just a folder as long as there are files being imported.
Real Answer
Now, this description becomes clearer.
You can only relatively import modules inside another module in the same package.
Given a directory:
. CWD
|-- happy_maker.py # content: print('Sends Happy')
`-- me.py # content: from . import happy_maker
Run python me.py, we got attempted relative import with no known parent package
me.py is run directly, it is not a module, and we can't use relative import in it.
Solution 1
Use import happy_maker instead of from . import happy_maker
Solution 2
Switch our working directory to the parent folder.
. CWD
|-- happy
| |-- happy_maker.py
`-- me.py
Run python -m happy.me.
When we are in the directory that includes happy, happy is a package, me.py, happy_maker.py are modules, we can use relative import now, and we still want to run me.py, so we use -m which means run the module as a script.
Python Idiom
. CWD
|-- happy
| |-- happy_maker.py # content: print('Sends Happy')
| `-- me.py # content: from . import happy_maker
`-- main.py # content: import happy.me
This structure is the python idiom. main is our script, best practice in Python. Finally, we got there.
Siblings or Grandparents
Another common need:
.
|-- happy
| |-- happy_maker.py
| `-- me.py
`-- sad
`-- sad_maker.py
We want to import sad_maker in me.py, How to do that?
First, we need to make happy and sad in the same package, so we have to go up a directory level. And then from ..sad import sad_maker in the me.py.
That is all.
My boilerplate to make a module with relative imports in a package runnable standalone.
package/module.py
## Standalone boilerplate before relative imports
if __package__ is None:
DIR = Path(__file__).resolve().parent
sys.path.insert(0, str(DIR.parent))
__package__ = DIR.name
from . import variable_in__init__py
from . import other_module_in_package
...
Now you can use your module in any fashion:
Run module as usual: python -m package.module
Use it as a module: python -c 'from package import module'
Run it standalone: python package/module.py
or with shebang (#!/bin/env python) just: package/module.py
NB! Using sys.path.append instead of sys.path.insert will give you a hard to trace error if your module has the same name as your package. E.g. my_script/my_script.py
Of course if you have relative imports from higher levels in your package hierarchy, than this is not enough, but for most cases, it's just okay.
I needed to run python3 from the main project directory to make it work.
For example, if the project has the following structure:
project_demo/
├── main.py
├── some_package/
│ ├── __init__.py
│ └── project_configs.py
└── test/
└── test_project_configs.py
Solution
I would run python3 inside folder project_demo/ and then perform a
from some_package import project_configs
I was getting this ImportError: attempted relative import with no known parent package
In my program I was using the file from current path for importing its function.
from .filename import function
Then I modified the current path (Dot) with package name. Which resolved my issue.
from package_name.filename import function
I hope the above answer helps you.
Importing from same directory
Firstly, you can import from the same directory.
Here is the file structure...
Folder
|
├─ Scripts
| ├─ module123.py
|
├─ main.py
├─ script123.py
Here is main.py
from . import script123
from Scripts import module123
As you can see, importing from . imports from current directory.
Note: if running using anything but IDLE, make sure that your terminal is navigated to the same directory as the main.py file before running.
Also, importing from a local folder also works.
Importing from parent directory
As seen in my GitHub gist here, there is the following method.
Take the following file tree...
ParentDirectory
├─ Folder
| |
| ├─ Scripts
| | ├─ module123.py
| |
| ├─ main.py
| ├─ script123.py
|
├─ parentModule.py
Then, just add this code to the top of your main.py file.
import inspect
import os
import sys
current_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parent_dir = os.path.dirname(current_dir)
sys.path.insert(0, parent_dir)
from ParentDirectory import Stuff
I tried all of the above to no avail, only to realize I mistakenly had a - in my package name.
In short, don't have - in the directory where __init__.py is. I've never felt elated after finding out such inanity.
if both packages are in your import path (sys.path), and the module/class you want is in example/example.py, then to access the class without relative import try:
from example.example import fkt
If none of the above worked for you, you can specify the module explicitly.
Directory:
├── Project
│ ├── Dir
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
Solution:
#in standalone.py
from Project.Dir.module import ...
module - the module to be imported
Here is a three-liner for those who disagree with Guido:
import sys
from pathlib import Path
sys.path.append(str(Path(sys.argv[0]).absolute().parent.parent))
Hope it helps.
I think the best solution is to create a package for your module:
Here is more info on how to do it.
Once you have a package you don't need to worry about relative import, you can just do absolute imports.
I encounter this a lot when I am working with Django, since a lot of functionality is performed from the manage.py script but I also want to have some of my modules runnable directly as scripts as well (ideally you would make them manage.py directives but we're not there yet).
This is a mock up of what such a project might look like;
├── dj_app
│ ├── models.py
│ ├── ops
│ │ ├── bar.py
│ │ └── foo.py
│ ├── script.py
│ ├── tests.py
│ ├── utils.py
│ └── views.py
└── manage.py
The important parts here being manage.py, dj_app/script.py, and dj_app/tests.py. We also have submodules dj_app/ops/bar.py and dj_app/ops/foo.py which contain more items we want to use throughout the project.
The source of the issue commonly comes from wanting your dj_app/script.py script methods to have test cases in dj_app/tests.py which get invoked when you run manage.py test.
This is how I set up the project and its imports;
# dj_app/ops/foo.py
# Foo operation methods and classes
foo_val = "foo123"
.
# dj_app/ops/bar.py
# Bar operations methods and classes
bar_val = "bar123"
.
# dj_app/script.py
# script to run app methods from CLI
# if run directly from command line
if __name__ == '__main__':
from ops.bar import bar_val
from ops.foo import foo_val
# otherwise
else:
from .ops.bar import bar_val
from .ops.foo import foo_val
def script_method1():
print("this is script_method1")
print("bar_val: {}".format(bar_val))
print("foo_val: {}".format(foo_val))
if __name__ == '__main__':
print("running from the script")
script_method1()
.
# dj_app/tests.py
# test cases for the app
# do not run this directly from CLI or the imports will break
from .script import script_method1
from .ops.bar import bar_val
from .ops.foo import foo_val
def main():
print("Running the test case")
print("testing script method")
script_method1()
if __name__ == '__main__':
print("running tests from command line")
main()
.
# manage.py
# just run the test cases for this example
import dj_app.tests
dj_app.tests.main()
.
Running the test cases from manage.py;
$ python3 manage.py
Running the test case
testing script method
this is script_method1
bar_val: bar123
foo_val: foo123
Running the script on its own;
$ python3 dj_app/script.py
running from the script
this is script_method1
bar_val: bar123
foo_val: foo123
Note that you get an error if you try to run the test.py directly however, so don't do that;
$ python3 dj_app/tests.py
Traceback (most recent call last):
File "dj_app/tests.py", line 5, in <module>
from .script import script_method1
ModuleNotFoundError: No module named '__main__.script'; '__main__' is not a package
If I run into more complicated situations for imports, I usually end up implementing something like this to hack through it;
import os
import sys
THIS_DIR = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, THIS_DIR)
from script import script_method1
sys.path.pop(0)
This my project structure
├── folder
| |
│ ├── moduleA.py
| | |
| | └--function1()
| | └~~ uses function2()
| |
│ └── moduleB.py
| |
| └--function2()
|
└── main.py
└~~ uses function1()
Here my moduleA imports moduleB and main imports moduleA
I added the snippet below in moduleA to import moduleB
try:
from .moduleB import function2
except:
from moduleB import function2
Now I can execute both main.py as well as moduleA.py individually
Is this a solution ?
The below solution is tested on Python3
├── classes
| |
| ├──__init__.py
| |
│ ├── userclass.py
| | |
| | └--viewDetails()
| |
| |
│ └── groupclass.py
| |
| └--viewGroupDetails()
|
└── start.py
└~~ uses function1()
Now, in order to use viewDetails of userclass or viewGroupDetails of groupclass define that in _ init _.py of classess directory first.
Ex: In _ init _.py
from .userclasss import viewDetails
from .groupclass import viewGroupDetails
Step2: Now, in start.py we can directly import viewDetails
Ex: In start.py
from classes import viewDetails
from classes import viewGroupDetails
I ran into a similar problem when trying to write a python file that can be loaded either as a module or an executable script.
Setup
/path/to/project/
├── __init__.py
└── main.py
└── mylib/
├── list_util.py
└── args_util.py
with:
main.py:
#!/usr/bin/env python3
import sys
import mylib.args_util
if __name__ == '__main__':
print(f'{mylib.args_util.parseargs(sys.argv[1:])=}')
mylib/list_util.py:
def to_int_list(args):
return [int(x) for x in args]
mylib/args_util.py:
#!/usr/bin/env python3
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
Traceback (most recent call last):
File "/path/to/project/mylib/args_util.py", line 10, in <module>
from . import list_util as lu
ImportError: attempted relative import with no known parent package
Solution
I settled for a Bash/Python polyglot solution. The Bash version of the program just calls python3 -m mylib.args_util then exits.
The Python version ignores the Bash code because it's contained in the docstring.
The Bash version ignores the Python code because it uses exec to stop parsing/running lines.
mylib/args_util.py:
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 -m mylib.args_util "$#"
'''
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
parseargs(sys.argv[1:])=6
Explanation
Line 1: #!/bin/bash; this is the "shebang" line; it tells the interactive shell how run this script.
Python: ignored (comment)
Bash: ignored (comment)
Line 2: # -*- Mode: python -*- optional; this is called the "mode-line"; it tells Emacs to use Python syntax highlighting instead of guessing that the language is Bash when reading the file.
Python: ignored (comment)
Bash: ignored (comment)
Line 3: ''''true
Python: views this as an unassigned docstring starting with 'true\n
Bash: views this as three strings (of which the first two are empty strings) that expand to true (i.e. '' + '' + 'true' = 'true'); it then runs true (which does nothing) and continues to the next line
Line 4: exec /usr/bin/env python3 -m mylib.args_util "$#"
Python: still views this as part of the docstring from line 3.
Bash: runs python3 -m mylib.args_util then exits (it doesn't parse anything beyond this line)
Line 5: '''
Python: views this as the end of the docstring from line 3.
Bash: doesn't parse this line
Caveats
This doesn't work on Windows:
Workaround: Use WSL or a Batch wrapper script to call python -m mylib.args_util.
This only works if the current working directory is set to /path/to/project/.
Workaround: Set PYTHONPATH when calling /usr/bin/env
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 \
PYTHONPATH="$(cd "$(dirname "$0")/.." ; pwd)" \
-m mylib.args_util "$#"
'''
I've created a new, experimental import library for Python: ultraimport
It gives the programmer more control over imports and makes them unambiguous. Also it gives better error messages when an import fails.
It allows you to do relative, file-system based imports that always work, no matter how you run your code and no matter what is your current working directory. It does not matter if you run a script or module. You also don't have to change sys.path which might have other side effects.
You would then change
from .mymodule import myfunction
to
import ultraimport
myfunction = ultraimport('__dir__/mymodule.py', 'myfunction')
This way the import will always work, even if you run the code as script.
One issue when importing scripts like this is that subsequent relative imports might fail. ultraimport has a builtin preprocessor to automatically rewrite relative imports.
I had a similar problem: I needed a Linux service and cgi plugin which use common constants to cooperate. The 'natural' way to do this is to place them in the init.py of the package, but I cannot start the cgi plugin with the -m parameter.
My final solution was similar to Solution #2 above:
import sys
import pathlib as p
import importlib
pp = p.Path(sys.argv[0])
pack = pp.resolve().parent
pkg = importlib.import_module('__init__', package=str(pack))
The disadvantage is that you must prefix the constants (or common functions) with pkg:
print(pkg.Glob)
TLDR; Append Script path to the System Path by adding following in the entry point of your python script.
import os.path
import sys
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
Thats it now you can run your project in PyCharma as well as from Terminal!!
Moving the file from which you are importing to an outside directory helps.
This is extra useful when your main file makes any other files in its own directory.
Ex:
Before:
Project
|---dir1
|-------main.py
|-------module1.py
After:
Project
|---module1.py
|---dir1
|-------main.py
I was getting the same error and my project structure was like
->project
->vendors
->vendors.py
->main.py
I was trying to call like this
from .vendors.Amazon import Amazom_Purchase
Here it was throwing an error so I fixed it simply by removing the first . from the statement
from vendors.Amazon import Amazom_Purchase
Hope this helps.
It's good to note that sometimes the cache causes of all it - I've tried different things after re-arranging classes into new directories and relative import started to work after I removed the __pycache__
If the following import:
from . import something
doesn't work for you it is because this is python-packaging import and will not work with your regular implementation, and here is an example to show how to use it:
Folder structure:
.
└── funniest
├── funniest
│ ├── __init__.py
│ └── text.py
├── main.py
└── setup.py
inside __init__.py add:
def available_module():
return "hello world"
text.py add:
from . import available_module
inside setup.py add
from setuptools import setup
setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus#example.com',
license='MIT',
packages=['funniest'],
zip_safe=False)
Now, this is the most important part you need to install your package:
pip install .
Anywhere else in our system using the same Python, we can do this now:
>> import funnies.text as fun
>> fun.available_module()
This should output 'hello world'
you can test this in main.py (this will not require any installation of the Package)
Here is main.py as well
import funniest.text as fun
print(fun.available_module())
I want to import a function from another file in the same directory.
Usually, one of the following works:
from .mymodule import myfunction
from mymodule import myfunction
...but the other one gives me one of these errors:
ImportError: attempted relative import with no known parent package
ModuleNotFoundError: No module named 'mymodule'
SystemError: Parent module '' not loaded, cannot perform relative import
Why is this?
unfortunately, this module needs to be inside the package, and it also
needs to be runnable as a script, sometimes. Any idea how I could
achieve that?
It's quite common to have a layout like this...
main.py
mypackage/
__init__.py
mymodule.py
myothermodule.py
...with a mymodule.py like this...
#!/usr/bin/env python3
# Exported function
def as_int(a):
return int(a)
# Test function for module
def _test():
assert as_int('1') == 1
if __name__ == '__main__':
_test()
...a myothermodule.py like this...
#!/usr/bin/env python3
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
...and a main.py like this...
#!/usr/bin/env python3
from mypackage.myothermodule import add
def main():
print(add('1', '1'))
if __name__ == '__main__':
main()
...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...
from .mymodule import as_int
The way you're supposed to run it is...
python3 -m mypackage.myothermodule
...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.
The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...
from mymodule import as_int
...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...
from mypackage.mymodule import as_int
...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
from mypackage.mymodule import as_int
It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...
I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.
Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.
Explanation
From PEP 328
Relative imports use a module's __name__ attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to '__main__')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
At some point PEP 338 conflicted with PEP 328:
... relative imports rely on __name__ to determine the current
module's position in the package hierarchy. In a main module, the
value of __name__ is always '__main__', so explicit relative imports
will always fail (as they only work for a module inside a package)
and to address the issue, PEP 366 introduced the top level variable __package__:
By adding a new module level attribute, this PEP allows relative
imports to work automatically if the module is executed using the -m
switch. A small amount of boilerplate in the module itself will allow
the relative imports to work when the file is executed by name. [...] When it [the attribute] is present, relative imports will be based on this attribute
rather than the module __name__ attribute. [...] When the main module is specified by its filename, then the __package__ attribute will be set to None. [...] When the import system encounters an explicit relative import in a
module without __package__ set (or with it set to None), it will
calculate and store the correct value (__name__.rpartition('.')[0]
for normal modules and __name__ for package initialisation modules)
(emphasis mine)
If the __name__ is '__main__', __name__.rpartition('.')[0] returns empty string. This is why there's empty string literal in the error description:
SystemError: Parent module '' not loaded, cannot perform relative import
The relevant part of the CPython's PyImport_ImportModuleLevelObject function:
if (PyDict_GetItem(interp->modules, package) == NULL) {
PyErr_Format(PyExc_SystemError,
"Parent module %R not loaded, cannot perform relative "
"import", package);
goto error;
}
CPython raises this exception if it was unable to find package (the name of the package) in interp->modules (accessible as sys.modules). Since sys.modules is "a dictionary that maps module names to modules which have already been loaded", it's now clear that the parent module must be explicitly absolute-imported before performing relative import.
Note: The patch from the issue 18018 has added another if block, which will be executed before the code above:
if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
PyErr_SetString(PyExc_ImportError,
"attempted relative import with no known parent package");
goto error;
} /* else if (PyDict_GetItem(interp->modules, package) == NULL) {
...
*/
If package (same as above) is empty string, the error message will be
ImportError: attempted relative import with no known parent package
However, you will only see this in Python 3.6 or newer.
Solution #1: Run your script using -m
Consider a directory (which is a Python package):
.
├── package
│ ├── __init__.py
│ ├── module.py
│ └── standalone.py
All of the files in package begin with the same 2 lines of code:
from pathlib import Path
print('Running' if __name__ == '__main__' else 'Importing', Path(__file__).resolve())
I'm including these two lines only to make the order of operations obvious. We can ignore them completely, since they don't affect the execution.
__init__.py and module.py contain only those two lines (i.e., they are effectively empty).
standalone.py additionally attempts to import module.py via relative import:
from . import module # explicit relative import
We're well aware that /path/to/python/interpreter package/standalone.py will fail. However, we can run the module with the -m command line option that will "search sys.path for the named module and execute its contents as the __main__ module":
vaultah#base:~$ python3 -i -m package.standalone
Importing /home/vaultah/package/__init__.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/module.py
>>> __file__
'/home/vaultah/package/standalone.py'
>>> __package__
'package'
>>> # The __package__ has been correctly set and module.py has been imported.
... # What's inside sys.modules?
... import sys
>>> sys.modules['__main__']
<module 'package.standalone' from '/home/vaultah/package/standalone.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
-m does all the importing stuff for you and automatically sets __package__, but you can do that yourself in the
Solution #2: Set __package__ manually
Please treat it as a proof of concept rather than an actual solution. It isn't well-suited for use in real-world code.
PEP 366 has a workaround to this problem, however, it's incomplete, because setting __package__ alone is not enough. You're going to need to import at least N preceding packages in the module hierarchy, where N is the number of parent directories (relative to the directory of the script) that will be searched for the module being imported.
Thus,
Add the parent directory of the Nth predecessor of the current module to sys.path
Remove the current file's directory from sys.path
Import the parent module of the current module using its fully-qualified name
Set __package__ to the fully-qualified name from 2
Perform the relative import
I'll borrow files from the Solution #1 and add some more subpackages:
package
├── __init__.py
├── module.py
└── subpackage
├── __init__.py
└── subsubpackage
├── __init__.py
└── standalone.py
This time standalone.py will import module.py from the package package using the following relative import
from ... import module # N = 3
We'll need to precede that line with the boilerplate code, to make it work.
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[3]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import package.subpackage.subsubpackage
__package__ = 'package.subpackage.subsubpackage'
from ... import module # N = 3
It allows us to execute standalone.py by filename:
vaultah#base:~$ python3 package/subpackage/subsubpackage/standalone.py
Running /home/vaultah/package/subpackage/subsubpackage/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/subpackage/__init__.py
Importing /home/vaultah/package/subpackage/subsubpackage/__init__.py
Importing /home/vaultah/package/module.py
A more general solution wrapped in a function can be found here. Example usage:
if __name__ == '__main__' and __package__ is None:
import_parents(level=3) # N = 3
from ... import module
from ...module.submodule import thing
Solution #3: Use absolute imports and setuptools
The steps are -
Replace explicit relative imports with equivalent absolute imports
Install package to make it importable
For instance, the directory structure may be as follows
.
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
│ └── setup.py
where setup.py is
from setuptools import setup, find_packages
setup(
name = 'your_package_name',
packages = find_packages(),
)
The rest of the files were borrowed from the Solution #1.
Installation will allow you to import the package regardless of your working directory (assuming there'll be no naming issues).
We can modify standalone.py to use this advantage (step 1):
from package import module # absolute import
Change your working directory to project and run /path/to/python/interpreter setup.py install --user (--user installs the package in your site-packages directory) (step 2):
vaultah#base:~$ cd project
vaultah#base:~/project$ python3 setup.py install --user
Let's verify that it's now possible to run standalone.py as a script:
vaultah#base:~/project$ python3 -i package/standalone.py
Running /home/vaultah/project/package/standalone.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
Note: If you decide to go down this route, you'd be better off using virtual environments to install packages in isolation.
Solution #4: Use absolute imports and some boilerplate code
Frankly, the installation is not necessary - you could add some boilerplate code to your script to make absolute imports work.
I'm going to borrow files from Solution #1 and change standalone.py:
Add the parent directory of package to sys.path before attempting to import anything from package using absolute imports:
import sys
from pathlib import Path # if you haven't already done so
file = Path(__file__).resolve()
parent, root = file.parent, file.parents[1]
sys.path.append(str(root))
# Additionally remove the current file's directory from sys.path
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
Replace the relative import by the absolute import:
from package import module # absolute import
standalone.py runs without problems:
vaultah#base:~$ python3 -i package/standalone.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
I feel that I should warn you: try not to do this, especially if your project has a complex structure.
As a side note, PEP 8 recommends the use of absolute imports, but states that in some scenarios explicit relative imports are acceptable:
Absolute imports are recommended, as they are usually more readable
and tend to be better behaved (or at least give better error
messages). [...] However, explicit relative imports are an acceptable
alternative to absolute imports, especially when dealing with complex
package layouts where using absolute imports would be unnecessarily
verbose.
Put this inside your package's __init__.py file:
# For relative imports to work in Python 3.6
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Assuming your package is like this:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module1.py
│ │ └── module2.py
│ └── setup.py
Now use regular imports in you package, like:
# in module2.py
from module1 import class1
This works in both python 2 and 3.
I ran into this issue. A hack workaround is importing via an if/else block like follows:
#!/usr/bin/env python3
#myothermodule
if __name__ == '__main__':
from mymodule import as_int
else:
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
SystemError: Parent module '' not loaded, cannot perform relative import
This means you are running a module inside the package as a script. Mixing scripts inside packages is tricky and should be avoided if at all possible. Use a wrapper script that imports the package and runs your scripty function instead.
If your top-level directory is called foo, which is on your PYTHONPATH module search path, and you have a package bar there (it is a directory you'd expect an __init__.py file in), scripts should not be placed inside bar, but should live on in foo at best.
Note that scripts differ from modules here in that they are used as a filename argument to the python command, either by using python <filename> or via a #! (shebang) line. It is loaded directly as the __main__ module (this is why if __name__ == "__main__": works in scripts), and there is no package context to build on for relative imports.
Your options
If you can, package your project with setuptools (or poetry or flit, which can help simplify packaging), and create console script entrypoints; installing your project with pip then creates scripts that know how to import your package properly. You can install your package locally with pip install -e ., so it can still be edited in-place.
Otherwise, never, ever, use python path/to/packagename/file.py, always use python path/to/script.py and script.py can use from packagename import ....
As a fallback, you could use the -m command-line switch to tell Python to import a module and use that as the __main__ file instead. This does not work with a shebang line, as there is no script file any more, however.
If you use python -m foo.bar and foo/bar.py is found in a sys.path directory, that is then imported and executed as __main__ with the right package context. If bar is also a package, inside foo/, it must have a __main__.py file (so foo/bar/__main__.py as the path from the sys.path directory).
In extreme circumstances, add the metadata Python uses to resolve relative imports by setting __package__ directly; the file foo/bar/spam.py, importable as foo.bar.spam, is given the global __package__ = "foo.bar". It is just another global, like __file__ and __name__, set by Python when imported.
On sys.path
The above all requires that your package can be imported, which means it needs to be found in one of the directories (or zipfiles) listed in sys.path. There are several options here too:
The directory where path/to/script.py was found (so path/to) is automatically added to sys.path. Executing python path/to/foo.py adds path/to to sys.path.
If you packaged your project (with setuptools, poetry, flit or another Python packaging tool), and installed it, the package has been added to the right place already.
As a last resort, add the right directory to sys.path yourself. If the package can be located relatively to the script file, use the __file__ variable in the script global namespace (e.g. using the pathlib.Path object, HERE = Path(__file__).resolve().parent is a reference to the directory the file lives in, as absolute path).
For PyCharm users:
I also was getting ImportError: attempted relative import with no known parent package because I was adding the . notation to silence a PyCharm parsing error. PyCharm innaccurately reports not being able to find:
lib.thing import function
If you change it to:
.lib.thing import function
it silences the error but then you get the aforementioned ImportError: attempted relative import with no known parent package. Just ignore PyCharm's parser. It's wrong and the code runs fine despite what it says.
To obviate this problem, I devised a solution with the repackage package, which has worked for me for some time. It adds the upper directory to the lib path:
import repackage
repackage.up()
from mypackage.mymodule import myfunction
Repackage can make relative imports that work in a wide range of cases, using an intelligent strategy (inspecting the call stack).
TL;DR: to #Aya's answer, updated with pathlib library, and working for Jupyter notebooks where __file__ is not defined:
You want to import my_function defined under ../my_Folder_where_the_package_lives/my_package.py
respect to where you are writing the code.
Then do:
import os
import sys
import pathlib
PACKAGE_PARENT = pathlib.Path(__file__).parent
#PACKAGE_PARENT = pathlib.Path.cwd().parent # if on jupyter notebook
SCRIPT_DIR = PACKAGE_PARENT / "my_Folder_where_the_package_lives"
sys.path.append(str(SCRIPT_DIR))
from my_package import my_function
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
TL;DR
You can only relatively import modules inside another module in the same package.
Concept Clarify
We see a lot of example code in books/docs/articles, they show us how to relatively import a module, but when we do so, it fails.
The reason is, put it in a simple sentence, we did not run the code as the python module mechanism expects, even though the code is written totally right. It's like some kind of runtime thing.
Module loading is depended on how you run the code. That is the source of confusion.
What is a module?
A module is a python file when and only when it is being imported by another file. Given a file mod.py, is it a module? Yes and No, if you run python mod.py, it is not a module, because it is not imported.
What is a package?
A package is a folder that includes Python module(s).
BTW, __init__.py is not necessary from python 3.3, if you don't need any package initialization or auto-load submodules. You don't need to place a blank __init__.py in a directory.
That proves a package is just a folder as long as there are files being imported.
Real Answer
Now, this description becomes clearer.
You can only relatively import modules inside another module in the same package.
Given a directory:
. CWD
|-- happy_maker.py # content: print('Sends Happy')
`-- me.py # content: from . import happy_maker
Run python me.py, we got attempted relative import with no known parent package
me.py is run directly, it is not a module, and we can't use relative import in it.
Solution 1
Use import happy_maker instead of from . import happy_maker
Solution 2
Switch our working directory to the parent folder.
. CWD
|-- happy
| |-- happy_maker.py
`-- me.py
Run python -m happy.me.
When we are in the directory that includes happy, happy is a package, me.py, happy_maker.py are modules, we can use relative import now, and we still want to run me.py, so we use -m which means run the module as a script.
Python Idiom
. CWD
|-- happy
| |-- happy_maker.py # content: print('Sends Happy')
| `-- me.py # content: from . import happy_maker
`-- main.py # content: import happy.me
This structure is the python idiom. main is our script, best practice in Python. Finally, we got there.
Siblings or Grandparents
Another common need:
.
|-- happy
| |-- happy_maker.py
| `-- me.py
`-- sad
`-- sad_maker.py
We want to import sad_maker in me.py, How to do that?
First, we need to make happy and sad in the same package, so we have to go up a directory level. And then from ..sad import sad_maker in the me.py.
That is all.
My boilerplate to make a module with relative imports in a package runnable standalone.
package/module.py
## Standalone boilerplate before relative imports
if __package__ is None:
DIR = Path(__file__).resolve().parent
sys.path.insert(0, str(DIR.parent))
__package__ = DIR.name
from . import variable_in__init__py
from . import other_module_in_package
...
Now you can use your module in any fashion:
Run module as usual: python -m package.module
Use it as a module: python -c 'from package import module'
Run it standalone: python package/module.py
or with shebang (#!/bin/env python) just: package/module.py
NB! Using sys.path.append instead of sys.path.insert will give you a hard to trace error if your module has the same name as your package. E.g. my_script/my_script.py
Of course if you have relative imports from higher levels in your package hierarchy, than this is not enough, but for most cases, it's just okay.
I needed to run python3 from the main project directory to make it work.
For example, if the project has the following structure:
project_demo/
├── main.py
├── some_package/
│ ├── __init__.py
│ └── project_configs.py
└── test/
└── test_project_configs.py
Solution
I would run python3 inside folder project_demo/ and then perform a
from some_package import project_configs
I was getting this ImportError: attempted relative import with no known parent package
In my program I was using the file from current path for importing its function.
from .filename import function
Then I modified the current path (Dot) with package name. Which resolved my issue.
from package_name.filename import function
I hope the above answer helps you.
Importing from same directory
Firstly, you can import from the same directory.
Here is the file structure...
Folder
|
├─ Scripts
| ├─ module123.py
|
├─ main.py
├─ script123.py
Here is main.py
from . import script123
from Scripts import module123
As you can see, importing from . imports from current directory.
Note: if running using anything but IDLE, make sure that your terminal is navigated to the same directory as the main.py file before running.
Also, importing from a local folder also works.
Importing from parent directory
As seen in my GitHub gist here, there is the following method.
Take the following file tree...
ParentDirectory
├─ Folder
| |
| ├─ Scripts
| | ├─ module123.py
| |
| ├─ main.py
| ├─ script123.py
|
├─ parentModule.py
Then, just add this code to the top of your main.py file.
import inspect
import os
import sys
current_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parent_dir = os.path.dirname(current_dir)
sys.path.insert(0, parent_dir)
from ParentDirectory import Stuff
I tried all of the above to no avail, only to realize I mistakenly had a - in my package name.
In short, don't have - in the directory where __init__.py is. I've never felt elated after finding out such inanity.
if both packages are in your import path (sys.path), and the module/class you want is in example/example.py, then to access the class without relative import try:
from example.example import fkt
If none of the above worked for you, you can specify the module explicitly.
Directory:
├── Project
│ ├── Dir
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
Solution:
#in standalone.py
from Project.Dir.module import ...
module - the module to be imported
Here is a three-liner for those who disagree with Guido:
import sys
from pathlib import Path
sys.path.append(str(Path(sys.argv[0]).absolute().parent.parent))
Hope it helps.
I think the best solution is to create a package for your module:
Here is more info on how to do it.
Once you have a package you don't need to worry about relative import, you can just do absolute imports.
I encounter this a lot when I am working with Django, since a lot of functionality is performed from the manage.py script but I also want to have some of my modules runnable directly as scripts as well (ideally you would make them manage.py directives but we're not there yet).
This is a mock up of what such a project might look like;
├── dj_app
│ ├── models.py
│ ├── ops
│ │ ├── bar.py
│ │ └── foo.py
│ ├── script.py
│ ├── tests.py
│ ├── utils.py
│ └── views.py
└── manage.py
The important parts here being manage.py, dj_app/script.py, and dj_app/tests.py. We also have submodules dj_app/ops/bar.py and dj_app/ops/foo.py which contain more items we want to use throughout the project.
The source of the issue commonly comes from wanting your dj_app/script.py script methods to have test cases in dj_app/tests.py which get invoked when you run manage.py test.
This is how I set up the project and its imports;
# dj_app/ops/foo.py
# Foo operation methods and classes
foo_val = "foo123"
.
# dj_app/ops/bar.py
# Bar operations methods and classes
bar_val = "bar123"
.
# dj_app/script.py
# script to run app methods from CLI
# if run directly from command line
if __name__ == '__main__':
from ops.bar import bar_val
from ops.foo import foo_val
# otherwise
else:
from .ops.bar import bar_val
from .ops.foo import foo_val
def script_method1():
print("this is script_method1")
print("bar_val: {}".format(bar_val))
print("foo_val: {}".format(foo_val))
if __name__ == '__main__':
print("running from the script")
script_method1()
.
# dj_app/tests.py
# test cases for the app
# do not run this directly from CLI or the imports will break
from .script import script_method1
from .ops.bar import bar_val
from .ops.foo import foo_val
def main():
print("Running the test case")
print("testing script method")
script_method1()
if __name__ == '__main__':
print("running tests from command line")
main()
.
# manage.py
# just run the test cases for this example
import dj_app.tests
dj_app.tests.main()
.
Running the test cases from manage.py;
$ python3 manage.py
Running the test case
testing script method
this is script_method1
bar_val: bar123
foo_val: foo123
Running the script on its own;
$ python3 dj_app/script.py
running from the script
this is script_method1
bar_val: bar123
foo_val: foo123
Note that you get an error if you try to run the test.py directly however, so don't do that;
$ python3 dj_app/tests.py
Traceback (most recent call last):
File "dj_app/tests.py", line 5, in <module>
from .script import script_method1
ModuleNotFoundError: No module named '__main__.script'; '__main__' is not a package
If I run into more complicated situations for imports, I usually end up implementing something like this to hack through it;
import os
import sys
THIS_DIR = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, THIS_DIR)
from script import script_method1
sys.path.pop(0)
This my project structure
├── folder
| |
│ ├── moduleA.py
| | |
| | └--function1()
| | └~~ uses function2()
| |
│ └── moduleB.py
| |
| └--function2()
|
└── main.py
└~~ uses function1()
Here my moduleA imports moduleB and main imports moduleA
I added the snippet below in moduleA to import moduleB
try:
from .moduleB import function2
except:
from moduleB import function2
Now I can execute both main.py as well as moduleA.py individually
Is this a solution ?
The below solution is tested on Python3
├── classes
| |
| ├──__init__.py
| |
│ ├── userclass.py
| | |
| | └--viewDetails()
| |
| |
│ └── groupclass.py
| |
| └--viewGroupDetails()
|
└── start.py
└~~ uses function1()
Now, in order to use viewDetails of userclass or viewGroupDetails of groupclass define that in _ init _.py of classess directory first.
Ex: In _ init _.py
from .userclasss import viewDetails
from .groupclass import viewGroupDetails
Step2: Now, in start.py we can directly import viewDetails
Ex: In start.py
from classes import viewDetails
from classes import viewGroupDetails
I ran into a similar problem when trying to write a python file that can be loaded either as a module or an executable script.
Setup
/path/to/project/
├── __init__.py
└── main.py
└── mylib/
├── list_util.py
└── args_util.py
with:
main.py:
#!/usr/bin/env python3
import sys
import mylib.args_util
if __name__ == '__main__':
print(f'{mylib.args_util.parseargs(sys.argv[1:])=}')
mylib/list_util.py:
def to_int_list(args):
return [int(x) for x in args]
mylib/args_util.py:
#!/usr/bin/env python3
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
Traceback (most recent call last):
File "/path/to/project/mylib/args_util.py", line 10, in <module>
from . import list_util as lu
ImportError: attempted relative import with no known parent package
Solution
I settled for a Bash/Python polyglot solution. The Bash version of the program just calls python3 -m mylib.args_util then exits.
The Python version ignores the Bash code because it's contained in the docstring.
The Bash version ignores the Python code because it uses exec to stop parsing/running lines.
mylib/args_util.py:
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 -m mylib.args_util "$#"
'''
import sys
from . import list_util as lu
def parseargs(args):
return sum(lu.to_int_list(args))
if __name__ == '__main__':
print(f'{parseargs(sys.argv[1:])=}')
Output
$ ./main.py 1 2 3
mylib.args_util.parseargs(sys.argv[1:])=6
$ mylib/args_util.py 1 2 3
parseargs(sys.argv[1:])=6
Explanation
Line 1: #!/bin/bash; this is the "shebang" line; it tells the interactive shell how run this script.
Python: ignored (comment)
Bash: ignored (comment)
Line 2: # -*- Mode: python -*- optional; this is called the "mode-line"; it tells Emacs to use Python syntax highlighting instead of guessing that the language is Bash when reading the file.
Python: ignored (comment)
Bash: ignored (comment)
Line 3: ''''true
Python: views this as an unassigned docstring starting with 'true\n
Bash: views this as three strings (of which the first two are empty strings) that expand to true (i.e. '' + '' + 'true' = 'true'); it then runs true (which does nothing) and continues to the next line
Line 4: exec /usr/bin/env python3 -m mylib.args_util "$#"
Python: still views this as part of the docstring from line 3.
Bash: runs python3 -m mylib.args_util then exits (it doesn't parse anything beyond this line)
Line 5: '''
Python: views this as the end of the docstring from line 3.
Bash: doesn't parse this line
Caveats
This doesn't work on Windows:
Workaround: Use WSL or a Batch wrapper script to call python -m mylib.args_util.
This only works if the current working directory is set to /path/to/project/.
Workaround: Set PYTHONPATH when calling /usr/bin/env
#!/bin/bash
# -*- Mode: python -*-
''''true
exec /usr/bin/env python3 \
PYTHONPATH="$(cd "$(dirname "$0")/.." ; pwd)" \
-m mylib.args_util "$#"
'''
I've created a new, experimental import library for Python: ultraimport
It gives the programmer more control over imports and makes them unambiguous. Also it gives better error messages when an import fails.
It allows you to do relative, file-system based imports that always work, no matter how you run your code and no matter what is your current working directory. It does not matter if you run a script or module. You also don't have to change sys.path which might have other side effects.
You would then change
from .mymodule import myfunction
to
import ultraimport
myfunction = ultraimport('__dir__/mymodule.py', 'myfunction')
This way the import will always work, even if you run the code as script.
One issue when importing scripts like this is that subsequent relative imports might fail. ultraimport has a builtin preprocessor to automatically rewrite relative imports.
I had a similar problem: I needed a Linux service and cgi plugin which use common constants to cooperate. The 'natural' way to do this is to place them in the init.py of the package, but I cannot start the cgi plugin with the -m parameter.
My final solution was similar to Solution #2 above:
import sys
import pathlib as p
import importlib
pp = p.Path(sys.argv[0])
pack = pp.resolve().parent
pkg = importlib.import_module('__init__', package=str(pack))
The disadvantage is that you must prefix the constants (or common functions) with pkg:
print(pkg.Glob)
TLDR; Append Script path to the System Path by adding following in the entry point of your python script.
import os.path
import sys
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
Thats it now you can run your project in PyCharma as well as from Terminal!!
Moving the file from which you are importing to an outside directory helps.
This is extra useful when your main file makes any other files in its own directory.
Ex:
Before:
Project
|---dir1
|-------main.py
|-------module1.py
After:
Project
|---module1.py
|---dir1
|-------main.py
I was getting the same error and my project structure was like
->project
->vendors
->vendors.py
->main.py
I was trying to call like this
from .vendors.Amazon import Amazom_Purchase
Here it was throwing an error so I fixed it simply by removing the first . from the statement
from vendors.Amazon import Amazom_Purchase
Hope this helps.
It's good to note that sometimes the cache causes of all it - I've tried different things after re-arranging classes into new directories and relative import started to work after I removed the __pycache__
If the following import:
from . import something
doesn't work for you it is because this is python-packaging import and will not work with your regular implementation, and here is an example to show how to use it:
Folder structure:
.
└── funniest
├── funniest
│ ├── __init__.py
│ └── text.py
├── main.py
└── setup.py
inside __init__.py add:
def available_module():
return "hello world"
text.py add:
from . import available_module
inside setup.py add
from setuptools import setup
setup(name='funniest',
version='0.1',
description='The funniest joke in the world',
url='http://github.com/storborg/funniest',
author='Flying Circus',
author_email='flyingcircus#example.com',
license='MIT',
packages=['funniest'],
zip_safe=False)
Now, this is the most important part you need to install your package:
pip install .
Anywhere else in our system using the same Python, we can do this now:
>> import funnies.text as fun
>> fun.available_module()
This should output 'hello world'
you can test this in main.py (this will not require any installation of the Package)
Here is main.py as well
import funniest.text as fun
print(fun.available_module())
I have this folder structure:
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
How can I import a function from file.py, from within some_file.py? I tried:
from application.app.folder.file import func_name
but it doesn't work.
Note: This answer was intended for a very specific question. For most programmers coming here from a search engine, this is not the answer you are looking for. Typically you would structure your files into packages (see other answers) instead of modifying the search path.
By default, you can't. When importing a file, Python only searches the directory that the entry-point script is running from and sys.path which includes locations such as the package installation directory (it's actually a little more complex than this, but this covers most cases).
However, you can add to the Python path at runtime:
# some_file.py
import sys
# caution: path[0] is reserved for script path (or '' in REPL)
sys.path.insert(1, '/path/to/application/app/folder')
import file
Nothing wrong with:
from application.app.folder.file import func_name
Just make sure folder also contains an __init__.py, this allows it to be included as a package. Not sure why the other answers talk about PYTHONPATH.
When modules are in parallel locations, as in the question:
application/app2/some_folder/some_file.py
application/app2/another_folder/another_file.py
This shorthand makes one module visible to the other:
import sys
sys.path.append('../')
First import sys in name-file.py
import sys
Second append the folder path in name-file.py
sys.path.insert(0, '/the/folder/path/name-package/')
Third Make a blank file called __ init __.py in your subdirectory (this tells Python it is a package)
name-file.py
name-package
__ init __.py
name-module.py
Fourth import the module inside the folder in name-file.py
from name-package import name-module
I think an ad-hoc way would be to use the environment variable PYTHONPATH as described in the documentation: Python2, Python3
# Linux & OSX
export PYTHONPATH=$HOME/dirWithScripts/:$PYTHONPATH
# Windows
set PYTHONPATH=C:\path\to\dirWithScripts\;%PYTHONPATH%
Your problem is that Python is looking in the Python directory for this file and not finding it. You must specify that you are talking about the directory that you are in and not the Python one.
To do this you change this:
from application.app.folder.file import func_name
to this:
from .application.app.folder.file import func_name
By adding the dot you are saying look in this folder for the application folder instead of looking in the Python directory.
Try Python's relative imports:
from ...app.folder.file import func_name
Every leading dot is another higher level in the hierarchy beginning with the current directory.
Problems? If this isn't working for you then you probably are getting bit by the many gotcha's relative imports has.
Read answers and comments for more details:
How to fix "Attempted relative import in non-package" even with __init__.py
Hint: have __init__.py at every directory level. You might need python -m application.app2.some_folder.some_file (leaving off .py) which you run from the top level directory or have that top level directory in your PYTHONPATH. Phew!
The answers here are lacking in clarity, this is tested on Python 3.6
With this folder structure:
main.py
|
---- myfolder/myfile.py
Where myfile.py has the content:
def myfunc():
print('hello')
The import statement in main.py is:
from myfolder.myfile import myfunc
myfunc()
and this will print hello.
In Python 3.4 and later, you can import from a source file directly (link to documentation). This is not the simplest solution, but I'm including this answer for completeness.
Here is an example. First, the file to be imported, named foo.py:
def announce():
print("Imported!")
The code that imports the file above, inspired heavily by the example in the documentation:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
foo = module_from_file("foo", "/path/to/foo.py")
if __name__ == "__main__":
print(foo)
print(dir(foo))
foo.announce()
The output:
<module 'foo' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Note that the variable name, the module name, and the filename need not match. This code still works:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
baz = module_from_file("bar", "/path/to/foo.py")
if __name__ == "__main__":
print(baz)
print(dir(baz))
baz.announce()
The output:
<module 'bar' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Programmatically importing modules was introduced in Python 3.1 and gives you more control over how modules are imported. Refer to the documentation for more information.
From what I know, add an __init__.py file directly in the folder of the functions you want to import will do the job.
Using sys.path.append with an absolute path is not ideal when moving the application to other environments. Using a relative path won't always work because the current working directory depends on how the script was invoked.
Since the application folder structure is fixed, we can use os.path to get the full path of the module we wish to import. For example, if this is the structure:
/home/me/application/app2/some_folder/vanilla.py
/home/me/application/app2/another_folder/mango.py
And let's say that you want to import the mango module. You could do the following in vanilla.py:
import sys, os.path
mango_dir = (os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+ '/another_folder/')
sys.path.append(mango_dir)
import mango
Of course, you don't need the mango_dir variable.
To understand how this works look at this interactive session example:
>>> import os
>>> mydir = '/home/me/application/app2/some_folder'
>>> newdir = os.path.abspath(os.path.join(mydir, '..'))
>>> newdir
'/home/me/application/app2'
>>> newdir = os.path.abspath(os.path.join(mydir, '..')) + '/another_folder'
>>>
>>> newdir
'/home/me/application/app2/another_folder'
>>>
And check the os.path documentation.
Also worth noting that dealing with multiple folders is made easier when using packages, as one can use dotted module names.
I was faced with the same challenge, especially when importing multiple files, this is how I managed to overcome it.
import os, sys
from os.path import dirname, join, abspath
sys.path.insert(0, abspath(join(dirname(__file__), '..')))
from root_folder import file_name
Considering application as the root directory for your python project, create an empty __init__.py file in application, app and folder folders. Then in your some_file.py make changes as follows to get the definition of func_name:
import sys
sys.path.insert(0, r'/from/root/directory/application')
from application.app.folder.file import func_name ## You can also use '*' wildcard to import all the functions in file.py file.
func_name()
Worked for me in python3 on linux
import sys
sys.path.append(pathToFolderContainingScripts)
from scriptName import functionName #scriptName without .py extension
The best practice for creating a package can be running and accessing the other modules from a module like main_module.py at highest level directory.
This structure demonstrates you can use and access sub package, parent package, or same level packages and modules by using a top level directory file main_module.py.
Create and run these files and folders for testing:
package/
|
|----- __init__.py (Empty file)
|------- main_module.py (Contains: import subpackage_1.module_1)
|------- module_0.py (Contains: print('module_0 at parent directory, is imported'))
|
|
|------- subpackage_1/
| |
| |----- __init__.py (Empty file)
| |----- module_1.py (Contains: print('importing other modules from module_1...')
| | import module_0
| | import subpackage_2.module_2
| | import subpackage_1.sub_subpackage_3.module_3)
| |----- photo.png
| |
| |
| |----- sub_subpackage_3/
| |
| |----- __init__.py (Empty file)
| |----- module_3.py (Contains: print('module_3 at sub directory, is imported'))
|
|------- subpackage_2/
| |
| |----- __init__.py (Empty file)
| |----- module_2.py (Contains: print('module_2 at same level directory, is imported'))
Now run main_module.py
the output is
>>>'importing other modules from module_1...'
'module_0 at parent directory, is imported'
'module_2 at same level directory, is imported'
'module_3 at sub directory, is imported'
Opening pictures and files note:
In a package structure if you want to access a photo, use absolute directory from highest level directory.
let's Suppose you are running main_module.py and you want to open photo.png inside module_1.py.
what module_1.py must contain is:
Correct:
image_path = 'subpackage_1/photo.png'
cv2.imread(image_path)
Wrong:
image_path = 'photo.png'
cv2.imread(image_path)
although module_1.py and photo.png are at same directory.
├───root
│ ├───dir_a
│ │ ├───file_a.py
│ │ └───file_xx.py
│ ├───dir_b
│ │ ├───file_b.py
│ │ └───file_yy.py
│ ├───dir_c
│ └───dir_n
You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:
# file_b.py
import sys
sys.path.insert(0, '..')
from dir_a.file_a import func_name
This works for me on windows
# some_file.py on mainApp/app2
import sys
sys.path.insert(0, sys.path[0]+'\\app2')
import some_file
In my case I had a class to import. My file looked like this:
# /opt/path/to/code/log_helper.py
class LogHelper:
# stuff here
In my main file I included the code via:
import sys
sys.path.append("/opt/path/to/code/")
from log_helper import LogHelper
I bumped into the same question several times, so I would like to share my solution.
Python Version: 3.X
The following solution is for someone who develops your application in Python version 3.X because Python 2 is not supported since Jan/1/2020.
Project Structure
In python 3, you don't need __init__.py in your project subdirectory due to the Implicit Namespace Packages. See Is init.py not required for packages in Python 3.3+
Project
├── main.py
├── .gitignore
|
├── a
| └── file_a.py
|
└── b
└── file_b.py
Problem Statement
In file_b.py, I would like to import a class A in file_a.py under the folder a.
Solutions
#1 A quick but dirty way
Without installing the package like you are currently developing a new project
Using the try catch to check if the errors. Code example:
import sys
try:
# The insertion index should be 1 because index 0 is this file
sys.path.insert(1, '/absolute/path/to/folder/a') # the type of path is string
# because the system path already have the absolute path to folder a
# so it can recognize file_a.py while searching
from file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
#2 Install your package
Once you installed your application (in this post, the tutorial of installation is not included)
You can simply
try:
from __future__ import absolute_import
# now it can reach class A of file_a.py in folder a
# by relative import
from ..a.file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
Happy coding!
I'm quite special : I use Python with Windows !
I just complete information : for both Windows and Linux, both relative and absolute path work into sys.path (I need relative paths because I use my scripts on the several PCs and under different main directories).
And when using Windows both \ and / can be used as separator for file names and of course you must double \ into Python strings,
some valid examples :
sys.path.append('c:\\tools\\mydir')
sys.path.append('..\\mytools')
sys.path.append('c:/tools/mydir')
sys.path.append('../mytools')
(note : I think that / is more convenient than \, event if it is less 'Windows-native' because it is Linux-compatible and simpler to write and copy to Windows explorer)
If the purpose of loading a module from a specific path is to assist you during the development of a custom module, you can create a symbolic link in the same folder of the test script that points to the root of the custom module. This module reference will take precedence over any other modules installed of the same name for any script run in that folder.
I tested this on Linux but it should work in any modern OS that supports symbolic links.
One advantage to this approach is that you can you can point to a module that's sitting in your own local SVC branch working copy which can greatly simplify the development cycle time and reduce failure modes of managing different versions of the module.
I was working on project a that I wanted users to install via pip install a with the following file list:
.
├── setup.py
├── MANIFEST.in
└── a
├── __init__.py
├── a.py
└── b
├── __init__.py
└── b.py
setup.py
from setuptools import setup
setup (
name='a',
version='0.0.1',
packages=['a'],
package_data={
'a': ['b/*'],
},
)
MANIFEST.in
recursive-include b *.*
a/init.py
from __future__ import absolute_import
from a.a import cats
import a.b
a/a.py
cats = 0
a/b/init.py
from __future__ import absolute_import
from a.b.b import dogs
a/b/b.py
dogs = 1
I installed the module by running the following from the directory with MANIFEST.in:
python setup.py install
Then, from a totally different location on my filesystem /moustache/armwrestle I was able to run:
import a
dir(a)
Which confirmed that a.cats indeed equalled 0 and a.b.dogs indeed equalled 1, as intended.
Instead of just doing an import ..., do this :
from <MySubFolder> import <MyFile>
MyFile is inside the MySubFolder.
In case anyone still looking for a solution. This worked for me.
Python adds the folder containing the script you launch to the PYTHONPATH, so if you run
python application/app2/some_folder/some_file.py
Only the folder application/app2/some_folder is added to the path (not the base dir that you're executing the command in). Instead, run your file as a module and add a __init__.py in your some_folder directory.
python -m application.app2.some_folder.some_file
This will add the base dir to the python path, and then classes will be accessible via a non-relative import.
The code below imports the Python script given by it's path, no matter where it is located, in a Python version-safe way:
def import_module_by_path(path):
name = os.path.splitext(os.path.basename(path))[0]
if sys.version_info[0] == 2:
# Python 2
import imp
return imp.load_source(name, path)
elif sys.version_info[:2] <= (3, 4):
# Python 3, version <= 3.4
from importlib.machinery import SourceFileLoader
return SourceFileLoader(name, path).load_module()
else:
# Python 3, after 3.4
import importlib.util
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
I found this in the codebase of psutils, at line 1042 in psutils.test.__init__.py (most recent commit as of 09.10.2020).
Usage example:
script = "/home/username/Documents/some_script.py"
some_module = import_module_by_path(script)
print(some_module.foo())
Important caveat: The module will be treated as top-level; any relative imports from parent packages in it will fail.
Wow, I did not expect to spend so much time on this. The following worked for me:
OS: Windows 10
Python: v3.10.0
Note: Since I am Python v3.10.0, I am not using __init__.py files, which did not work for me anyway.
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
WY Hsu's 1st solution worked for me. I have reposted it with an absolute file reference for clarity:
import sys
sys.path.insert(1, 'C:\\Users\\<Your Username>\\application')
import app2.some_folder.some_file
some_file.hello_world()
Alternative Solution: However, this also worked for me:
import sys
sys.path.append( '.' )
import app2.some_folder.some_file
some_file.hello_world()
Although, I do not understand why it works. I thought the dot is a reference to the current directory. However, when printing out the paths to the current folder, the current directory is already listed at the top:
for path in sys.path:
print(path)
Hopefully, someone can provide clarity as to why this works in the comments. Nevertheless, I also hope it helps someone.
This problem may be due Pycharm
I had the same problem while using Pycharm. I had this project structure
skylake\
backend\
apps\
example.py
configuration\
settings.py
frontend\
...some_stuff
and code from configuration import settings in example.py raised import error
the problem was that when I opened Pycharm, it considered that skylake is root path and ran this code
sys.path.extend(['D:\\projects\\skylake', 'D:/projects/skylake'])
To fix this I just marked backend directory as source root
And it's fixed my problem
My solution for people. who have all the necessary __init__.py in the package, but import still doesn't work.
import sys
import os
sys.path.insert(0, os.getcwd())
import application.app.folder.file as file
You can use importlib to import modules where you want to import a module from a folder using a string like so:
import importlib
scriptName = 'Snake'
script = importlib.import_module('Scripts\\.%s' % scriptName)
This example has a main.py which is the above code then a folder called Scripts and then you can call whatever you need from this folder by changing the scriptName variable. You can then use script to reference to this module. such as if I have a function called Hello() in the Snake module you can run this function by doing so:
script.Hello()
I have tested this in Python 3.6
I usually create a symlink to the module I want to import. The symlink makes sure Python interpreter can locate the module inside the current directory (the script you are importing the other module into); later on when your work is over, you can remove the symlink. Also, you should ignore symlinks in .gitignore, so that, you wouldn't accidentally commit symlinked modules to your repo. This approach lets you even successfully work with modules that are located parallel to the script you are executing.
ln -s ~/path/to/original/module/my_module ~/symlink/inside/the/destination/directory/my_module