This question already has an answer here:
ImportError on python 3, worked fine on python 2.7
(1 answer)
Closed 5 years ago.
I need assistance on how to organize source in a python package - I've already followed several tutorials on the web (especially this one) on how to do so, but it does not work as explained and how I imagined it.
I want to create a python package named binaryio. It should offer two classes named BinaryReader and BinaryWriter which I want users to be able to import with
from binaryio import BinaryReader
from binaryio import BinaryWriter
Thus I have created my repository and package directory structure as follows:
binaryio (repository root)
binaryio (package root)
__init__.py (s. below)
binaryreader.py (contains the BinaryReader class)
binarywriter.py (contains the BinaryWriter class)
setup.py (contains the setuptools.setup call)
.gitignore, README.md, LICENSE, ...
As you can see, the classes are in separate files as I'm used to this (coming from a C# background). I'm not sure if this is a good idea due to modules being the "unit" in Python - but otherwise cramping all classes into one huge file did not seem logical to me.
__init__.py looks as follows to import those classes, making (as I understood it) the from binaryio import BinaryReader imports possible for users later on:
from binaryreader import BinaryReader
from binarywriter import BinaryWriter
However, when I install the package locally (which seems to work fine) and try to import binaryio, I get the following error:
>>> import binaryio
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Projects\Git\binaryio\binaryio\__init__.py", line 1, in <module>
from binaryreader import BinaryReader
ModuleNotFoundError: No module named 'binaryreader'
Apparently, something is wrong with my __init__.py file. I don't understand this, as a binaryreader.py file aka module exists in the same folder as you can see above. Funnily enough, my IDE (PyCharm, having set the package root as source folder) does not complain about the statements in it and can resolve all references.
What am I doing wrong here? According to the tutorial linked above, putting a file into xyz.py with a class named Abc and then write from xyz import Abc into __init__.py should work, but apparently it doesn't for me.
Your code would work for Python 2.x, but not 3.x because of different relative imports syntax: without dot, Python 2.x would look for modules in module root and current package, and Python 3.x will look only in module root.
Import statements you want to use are these:
from binaryio.binaryreader import BinaryReader
from binaryio.binarywriter import BinaryWriter
Works in both Python 2.x and 3.x without "futures"
I think you need to add a dot in your import statements :
from .binaryreader import BinaryReader
from .binarywriter import BinaryWriter
Related
I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.
The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file
I have been creating an application that will run just like Microsoft Notepad. My program comprises of the following file and folder:
libs (Folder)
scripts (Folder)
test.py
The test.py is the file in which I am trying the following code:
from .libs import tkinter
from .libs import pyglet
from .libs import threading
test_window = Tk()
test_window.mainloop()
In the folder libs I have added the modules that are required in the application.
Example: Tkinter, Pyglet and Threading
When I am trying to import them into test.py using the code above and then running the program. I see the following Traceback:
Traceback (most recent call last):
File "c:\Users\Bhavyadeep\Desktop\NuclearPad\test.py", line 1, in <module>
from .libs import tkinter
ImportError: attempted relative import with no known parent package
I don't really know how I solve this problem. Is it because I added the module in the folder where Import in not available? Or is it because I did something wrong in this module folder copying-pasting to a folder not at Python's PATH?
from .libs import tkinter
Means "import tkinter from the package called lib which is located in the same directory as file (this script)"
so your first line should say something like
test_window = tkinter.Tk()
unless you use either of the following:
from .libs.tkinter import *
from .libs.tkinter import Tk
The error is likely due to you using relative import syntax from within the script you are trying to execute. Remove the leading dots in order to execute test.py, otherwise append its directory to sys.path or use site.addsitedir(path_to_directory_containing_test_py) and import it as follows import test.
With that said, you should not call the module's file test.py because there is already something with that name in python's default Lib directory or builtins. If you insist, there should be a hacky way to get it done without changing the name through importlib.
As for why this error occurs, I think it's because the interpreter is designed to execute scripts but not modules/packages whose contents are placed in .pyc files in __pycache__s when imported.
In order to import from scripts or libs, they must be packages. In order to turn them into packages you must add __init__.py files to them.
Lastly, It's a pretty terrible idea to place third-party libraries in your source like this. At the very least you should have installed them properly on your machine and then copied their sources from site-packages/the equivalent for venvs, which will fail if they use any executables and/or have any external dependencies
Documentation for distributing python packages
I have trouble importing package.
My file structure is like this:
filelib/
__init__.py
converters/
__init__.py
cmp2locus.py
modelmaker/
__init__.py
command_file.py
In module command_file.py I have a class named CommandFile which i want to call in the cmp2locus.py module.
I have tried the following in cmp2locus.py module:
import filelib.modelmaker.command_file
import modelmaker.command_file
from filelib.modelmaker.command_file import CommandFile
All these options return ImportError: No modules named ...
Appreciate any hint on solving this. I do not understand why this import does not work.
To perform these imports you have 3 options, I'll list them in the order I'd prefer. (For all of these options I will be assuming python 3)
Relative imports
Your file structure looks like a proper package file structure so this should work however anyone else trying this option should note that it requires you to be in a package; this won't work for some random script.
You'll also need to run the script doing the importing from outside the package, for example by importing it and running it from there rather than just running the cmp2locus.py script directly
Then you'll need to change your imports to be relative by using ..
So:
import filelib.modelmaker.command_file
becomes
from ..modelmaker import command_file
The .. refers to the parent folder (like the hidden file in file systems).
Also note you have to use the from import syntax because names starting with .. aren't valid identifiers in python. However you can of course import it as whatever you'd like using from import as.
See also the PEP
Absolute imports
If you place your package in site-packages (the directories returned by site.getsitepackages()) you will be able to use the format of imports that you were trying to use in the question. Note that this requires any users of your package to install it there too so this isn't ideal (although they probably would, relying on it is bad).
Modifying the python path
As Meera answered you can also directly modify the python path by using sys.
I dislike this option personally as it feels very 'hacky' but I've been told it can be useful as it gives you precise control of what you can import.
To import from another folder, you have to append that path of the folder to sys.path:
import sys
sys.path.append('path/filelib/modelmaker')
import command_file
I have been googling this for the past hours and can't find an equivalent question anywhere. Also the documentation for 2.7 and 3.5 seem identical, so I don't think this behavior is documented.
Here is my directory structure:
project
-- project.py
-- api
-- __init__.py
-- subapi
-- __init__.py
contents of project/project.py: import api
contents of project/api/__init__.py: import subapi
If I execute python project.py (using python 2.7) from inside the projects folder, it returns without an error. If I do the same with python 3 (python3 project.py), then it crashes with
Traceback (most recent call last):
File "project.py", line 1, in <module>
import api
File "/home/me/Documents/project/api/__init__.py", line 1, in <module>
import subapi
ImportError: No module named 'subapi'
If I rewrite the import statement to use paths relative to the projects directory (import api.subapi), then it works with python 2 as well as 3. It's not a satisfying solution though because that requires me to reference parent modules from within sub modules which kind of defeats the idea of modularity..
Does anyone have an idea what I can do to get the behavior of python2 back? The module search algorithm should prioritize searching in the local directory of the file using the import statement. It should also prioritize these files above built in modules by the way. Try importing a module 'test'..
--EDIT--
I was asked by stackoverflow to differentiate my question from another called "How to do relative imports". I think this question is different because I am asking specifically about differences between two versions. Using relative imports is the solution, not the question.
Use an explicit relative import:
from . import subapi
I have been googling this for the past hours and can't find an equivalent question anywhere. Also the documentation for 2.7 and 3.5 seem identical, so I don't think this behavior is documented.
Here is my directory structure:
project
-- project.py
-- api
-- __init__.py
-- subapi
-- __init__.py
contents of project/project.py: import api
contents of project/api/__init__.py: import subapi
If I execute python project.py (using python 2.7) from inside the projects folder, it returns without an error. If I do the same with python 3 (python3 project.py), then it crashes with
Traceback (most recent call last):
File "project.py", line 1, in <module>
import api
File "/home/me/Documents/project/api/__init__.py", line 1, in <module>
import subapi
ImportError: No module named 'subapi'
If I rewrite the import statement to use paths relative to the projects directory (import api.subapi), then it works with python 2 as well as 3. It's not a satisfying solution though because that requires me to reference parent modules from within sub modules which kind of defeats the idea of modularity..
Does anyone have an idea what I can do to get the behavior of python2 back? The module search algorithm should prioritize searching in the local directory of the file using the import statement. It should also prioritize these files above built in modules by the way. Try importing a module 'test'..
--EDIT--
I was asked by stackoverflow to differentiate my question from another called "How to do relative imports". I think this question is different because I am asking specifically about differences between two versions. Using relative imports is the solution, not the question.
Use an explicit relative import:
from . import subapi