I have a Python project structure like this:
└── my_project
├── __init__.py (empty)
├── options.py
└── main.py
inside main.py I import options like so:
from my_project.options import some_function
However (and this is the weird part - this only happens on one computer (Python 3.9.1)) when running main.py I get the following error:
Traceback (most recent call last):
File ".../my_project/main.py", line 1, in <module>
from my_project.options import some_function
ModuleNotFoundError: No module named 'my_project'
This used to work before, but has now for some reason broken. Do you have any idea as to why it might not wanna work, and how I can fix it? It also works just fine when I install this program with pip - just not when ran locally with my Python interpreter
Check for the PYTHONPATH environment variable, which is the default path to search for python modules.
Must contain the root path of your project. You can find more info here
Related
I have a project hierarchy like below, when I run python src/bot/main I got no error. While if I run python -m src.bot.main I got error. Why?
This is my file hierarchy:
MyProject
└── src
├── __init__.py
├── bot
│ ├── __init__.py
│ ├── main.py
│ └── sib1.py
└── mod
├── __init__.py
└── module1.py
This is the content of main.py:
import sys
if __name__ == "__main__":
# sys.path will print the parent folder.
print(sys.path, end="\n\n")
# my problem is on this line.
import sib1
sib1.test()
The error:
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/me/Desktop/test_py/src/bot/main.py", line 16, in <module>
import sib1
ModuleNotFoundError: No module named 'sib1'
Some conclusion I've made so far:
Since the output of sys.path in both cases include /Users/me/Desktop/MyProject, the reason should not related to scope?
The output of sys.path of both python -m src.bot.main and python src/bot/main:
(test) ✔ me Desktop/test_py % python -m src.bot.main
['/Users/me/Desktop/test_py', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python39.zip', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/lib-dynload', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/site-packages']
I will try my best to clarify each of my confusions in a Q&A form, and organize #Brain's comments and during the way:
Q1: when I run python src/bot/main I got no error.
The sys.path will include the current directory containing the file main.py, i.e. the interpreter will see the file MyProject/src/bot:
import sib1
is (logically) equivalent to:
import /path/to/MyProject/src/bot/sib1.py
Hence, no error.
BUT this is called IMPLICIT-RELATIVE-IMPORT, which should be used only on the main script. See Example.
Q2: While if I run python -m src.bot.main I got error. Why?
Now it's time to quote #Brain's valuable (first) comment:
Using python -m src.bot.main tells Python that src is a top-level package. Everything below src in the directory structure will be considered submodules/subpackages of src. The proper name for sib1 under that organization is src.bot.sib1. No top-level module named sib1 exists as far as Python is concerned.
(emphasis by me)
So:
The -m option will provide some context information(details, I recommend reading the first two high-upvoted answers entirely) for the file you're going to run.
Now every absolute (import-)path that does not begin with src. will be regarded as built-in or third-party libraries installed in your virtual environment. Since I didn't install any package called sib1, you got the error. (Ignore about the inclusion of the current directory in the sys.path. I guess this is for compatibility reason. Remember that we DONT want IMPLICIT-RELATIVE-IMPORT in python3)
You can use relative path BUT you should not go out of the src (since src is the top-level package in the execution) by prepending too many .'s.
For example this will work:
from . import sib1
from ..mod import module1 # The `..` is equivalent to the MyProject/src.
module1.hello()
sib1.test()
Finally, don't test your package by inserting many if __name__ == '__main__'. Do so by professional tools:
If you only need to run submodules for testing purpose, you could consider using more robust testing tools like doctest (lightweight) or unittest (heavyweight).
In python I have a folder with three files
- __init__.py
- module.py
- test_module.py
in which the module module.py is imported inside the file test_module.py as follows:
from . import module
Of course, when I just run test_module.py I get an error
> python test_module.py
Traceback (most recent call last):
File "test_module.py", line 4, in <module>
from . import module
ImportError: attempted relative import with no known parent package
But as I set the PYTHONPATH to the absolute path to where I am working in
export PYTHONPATH=`pwd`
I expect the import to work (as I did set the PYTHONPATH). But to my surprise I get the same error!
So can I fix the relative import error without any code change?
Since the directory you describe (let's call it thatdirectory) is a package, as marked by an __init__ file, you can "fix" that by cding a directory higher up, then running
python -m thatdirectory.test_module
since running modules with -m fixes up sys.path in a way that makes this particular configuration work.
(In general, any code that messes with sys.path manually, or requires changes to PYTHONPATH to work, is broken in my eyes...)
I created a project that runs fine locally, but fails when run from PyPi deploy/install, with ModuleNotFound. The link contains a very small sample, and run / deploy instructions. My original thought was regarding inheritance, but it seems more basic.
Traceback (most recent call last):
File "/Users/val/python/vscode/inheritance/venv/bin/inheritance-run", line 5, in <module>
from inheritance.run import start
ModuleNotFoundError: No module named 'inheritance'
Your setup.py has:
packages=find_packages(),
This will find modules (directories with __init__.py files) in the same directory as your setup.py file, but you don't have any. Instead, you have a base.py file.
If you want to be able to import inheritance, you should move this into an __init__.py file in new directory named inheritance, so you have something like:
.
├── inheritance
│ └── __init__.py
└── setup.py
Then setuptools will find this module and include it as an importable package in your project.
FYI, It's usually preferable to use a src-based layout instead, however. See https://github.com/pypa/sampleproject/ for a full example.
I've been reading tons of questions related to this matter but none of the has help me so far. I'm currently using the Python click library to execute scripts as commands.
The current command that I'm trying to execute is placed inside a Python Package which has a __main__.py file, like the parent dir has. The current project structure is the following one.
/myproject
/foo_one
__init__.py
foo_one.py
/foo_two
__init__.py
foo_two.py
/foo_three
__init__.py
foo_three.py
/foo_four
__init__.py
foo_four.py
/foo_five
__init__.py
foo_five.py
/foo_six
__init__.py
foo_six.py
__init__.py
__main__.py
foo_seven.py
Whenever I try to run the __main__.py script located in the project folder, the following error comes up.
ModuleNotFoundError: No module named '__main__.foo_two'; '__main__' is not a package
However, if I try to execute that same script from a folder above with the -m option like this python3 myproject -m, the following is shown up.
ImportError: attempted relative import with no known parent package
The __main__.py has 2 imports like this... The __init__.py is empty.
from .foo_two.foo_two import AClass, AnotherClass, OtherClass
from .foo_three.foo_three import AnotherClassMore
UPDATE: Correcting the syntax error in a previous command, while calling python -m myproject gives me a ModuleNotFoundError because of a module that isn't my responsibility, which is basically a library that is used in the project.
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't just add your export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
The correct syntax would be
python -m myproject
This should execute __main__ in the top-level package.
You need to have __init__.py in each sub folder with python code to tell the interpreter to treat the folder as a module
/myproject
/foo_one
__init__.py # add this
foo_one.py
/foo_two
__init__.py # add this
foo_two.py
/foo_three
__init__.py # add this
foo_three.py
/foo_four
__init__.py # add this
foo_four.py
/foo_five
__init__.py # add this
foo_five.py
/foo_six
__init__.py # add this
foo_six.py
__init__.py
__main__.py
foo_seven.py
the __init__.py is telling the interpreter to treat sub folders as python modules / packages and you should be able to import
The __init__.py file can be empty but needs to be present in the sub folders to be able to import that module / package
My current projects are structured like so.
python/
__init__.py
project_1/
__init__.py
program_1.py
project_2/
__init__.py
program_2.py
project_3/
__init__.py
program_3.py
I wanted to import a class from project_3/program_3.py, called INFO, from both project_1/program_1.py and project_2/program_2.py. In both my program_1.py and program_2.py, I've tried the following import statements that didn't work.
Edit - I typically "cd" into project_1 and run my program_1.py from there.
from project_3.program_3 import INFO
Traceback (most recent call last):
File "./program_1.py", line 43, in <module>
from project_3.program_3 import INFO
ImportError: No module named 'project_3'
from python.project_3.program_3 import INFO
Traceback (most recent call last):
File "./program_1.py", line 43, in <module>
from python.project_3.program_3 import INFO
ImportError: No module named 'python'
The only way for me to import class INFO into program_1.py and program_2.py is by copying program_3.py in both program_1 and program_2's folder. My import in program_1 and program_2's statement would become:
python/
__init__.py
project_1/
__init__.py
program_1.py
program_3.py
project_2/
__init__.py
program_2.py
program_3.py
project_3/
__init__.py
program_3.py
from program_3 import INFO
This is quite inconvenience because now I have to update program_3.py in all 3 locations. Assuming my the way I structured my project folder is retarded:
What's the correct/best way to structure my folders so that I could call program_3.py from program_1/2.py?
If there's nothing wrong with my current structure, how do I correctly import the INFO class into program_1/2.py?
I have read the python documentation but it didn't say anything regarding importing classes from one subfolder to another. I also looked at another post from Stack Overflow as well but it didn't have the same structure as mine.
Update 1 - It's important to note that I'm merely using project_1/2 as folder to organized my projects. I originally had init.py in all my folders because I have no clue what I'm doing. The only module/package that I wanted to import in my project_1/program_1.py is from project_3/program_3.py. Example:
python/
project_1/
program_1.py
project_2/
program_2.py
project_3/
__init__.py
program_3.py
When using submodules, you need to use relative imports. In the example provided, importing program_3 from program_1 should be :
from ..project_3 import program_3
and then you can use the INFO class: program_3.INFO.
This works if your executable file is outside the script, that is the file which uses the module python in the example should be outside the python module.
If parts of the python package are executed as a script, the -m option should be passed to the interpreter:
python3 -m python.project_1.program_1
I hope this helps.
The previous solution provides me the same error.
I found a fix by using sys:
import sys
sys.path.append('../')
from project_3 import program_3
...
This way, you setup 'python' folder as reference (in your case it's the parent folder but you can select the path you want)
I hope it helps,