The following is my directory structure.
ankur
├── ankur1
│ ├── __init__.py
│ └── util.py
├── ankur2
│ └── main.py
└── __init__.py
In main.py, I am importing the following.
import ankur.ankur1.util
When I execute the code in Windows, it works perfectly fine. But in Linux, I get the following error.
ImportError: No module named ankur.ankur1.util
I also read the official python doc on Modules and Packages.
Your package structure is OK. Your import statement is OK. The only thing missing is for the package to be visible in sys.path, a list of locations where import statements can be resolved.
Usually we do this by "installing" the package locally with pip, which copies your code into site-packages†. This directory is one of the entries in sys.path, so when your code is installed in site-packages, the import statements can now be resolved as usual.
However, to install your code you'll need an installer (setup.py script) or a build system (pyproject.toml file) defined for the package. Your project doesn't appear to have any installer or build system, so you'll need to create one (see the Python Packaging User Guide for details about that) and then install the package with pip. If you don't want to learn Python packaging just yet, you'll need to find another way around.
It is possible to modify sys.path directly in main.py, which is subsequently enabling the statement import ankur.ankur1.util to be resolved. This is hacky and I recommend against that. It would add the restriction that executing main.py is the only entry point to the rest of the package, and so any other code wanting to import ankur will first need to know the path to main.py on the filesystem. That's a messy approach and should be avoided.
Another way around is to use the environment - there is an environment variable PYTHONPATH which can be used to augment the default search path for module files. In your shell:
export PYTHONPATH=/path/to/parent # linux/macOS
SET PYTHONPATH=C:/path/to/parent # Windows
Where parent is the directory containing ankur subdirectory.
† The exact location of site-packages depends on your OS/platform, but you can check with import sysconfig; sysconfig.get_paths()["purelib"]
Related
I'm trying to run a simple project in python but I have always the same problem and it drives me crazy every single time.
My project structure looks like the following:
└── myproject
├── mypackage
├── __init__.py
├── a.py
├── b.py
With vscode I open up the myproject folder. When I click on run in the a.py file which looks like
# a.py
from mypackage import b
.
.
.
I always get ModuleNotFoundError: No module named 'mypackage'. What is confusing to me is that vscode itself sees it correctly and shows the text colored (means it detects the module correctly).
Edit:
I want to use myproject as a package so just import b won't help.
The solution I found is actually relatively easy. Just type python3 -m mypackage.a into the command line in the top myproject directory. Worked for me.
I know this has nothing to do with vscode but I was just confused why vscode seems to accept the import but the python interpreter didn't.
Since you've probably didn't install myproject (or mypackage, if it is top level Python package), Python cannot find it on the path. You have two options:
Install the package.
Use relative import import b. However, see here for implications.
I recommend to avoid approaches like adding mypackage path to system path (see here). Maybe this documentation/tutorial can help you to understand better, what happened and why the import you used is not working.
However, I do not know, how VS Code searches for files in project, so I cannot tell you, why it marks it like this...
according to what you mentioned, both file 'a.py' and 'b.py' are in the same directory and if both files are in the same director, you don't have to actually mention the name of the directory in which they are present.
To import the file 'b.py' in 'a.py', you can just simply type the code shown below to import 'b.py':
import b
This should work.
I want to import business code from my test folder.
I installed my code using python setup.py install, which copy the code into the anaconda site-package folder. But I want to test my code in my dev. directory, as I don't want to constantly install my code to test it after any small change.
I am using Anaconda and Spyder IDE.
Here is my dir structure:
dev
└── myproject
├── setup.py
├── myproject
│ ├── __init__.py
│ └── myproject.py
└── test
├── __init__.py
└── test_importprojecthere.py
I took it from here: Running unittest with typical test directory structure
For now, I'm simply trying to import a function.
# dev: projectfile.py
def hello():
print('Hello world!')
Here is where I call it from.
# dev: test_importprojecthere.py
from myproject.projectfile import hello # Use installed package.
hello()
more information:
Python relative paths for unit tests
Import a module from a relative path Not ideal!
relative path not working even with __init__.py
When you are creating packages, you do not want to install them in your site-packages.
Do not use
python setup.py install
Instead, use
python setup.py develop or...
pip install -e .
Those commands install your setup using symlinks. Instead of creating a copy, it links to your actual project. So when you modify it, it's being modified instantly everywhere.
Thanks to that, you will not have to update/install or do anything to run your unit tests.
-- "But now, every time I modify my package, everything using it might break. I don't want to break my production code!"
Do not worry! This is the reason why we use virtual environments. When you create a package, make sure you create a virtual environment, that you might name, by example "myproject_dev".
In the dev. env. use python setup.py develop so that you can develop on it.
In the prod. env. use python setup.py install, so that you use a copy, that will not suddenly change. You won't modify it, because in prod, you consume your package, you don't develop on it! Keeps things simple an safe.
First of all, there are a bunch of solutions on stackoverflow regarding this but from the ones I tried none of them is working. I am working on a remote machine (linux). I am prototyping within the dir-2/module_2.py file using an ipython interpreter. Also I am trying to avoid using absolute paths as the absolute path in this remote machine is long and ugly, and I want my code to run on other machines upon download.
My directory structure is as follows:
/project-dir/
-/dir-1/
-/__ init__.py
-/module_1.py
-/dir-2/
-/__ init__.py
-/module_2.py
-/module_3.py
Now I want to import module_1 from module_2. However the solution mentioned in this stackoverflow post: link of using
sys.path.append('../..')
import module_2
Does not work. I get the error: ModuleNotFoundError: No module named 'module_1'
Moreover, within the ipython interpreter things like import .module_3 within module_2 throws error:
import .module_3
^ SyntaxError: invalid syntax
Isn't the dot operator supposed to work within the same directory as well. Overall I am quite confused by the importing mechanism. Any help with the initial problem is greatly appreciated! Thanks a lot!
Why it didn't work?
If you run the module1.py file and you want to import module2 then you need something like
sys.path.append("../dir-2")
If you use sys.path.append("../..") then the folder you added to the path is the folder containing project-dirand there is notmodule2.py` file inside it.
The syntax import .module_3 is for relative imports. if you tried to execute module2.py and it contains import .module_3 it does not work because you are using module2.py as a script. To use relative imports you need to treat both module2.py and module_3.py as modules. That is, some other file imports module2 and module2 import something from module3 using this syntax.
Suggestion on how you can proceed
One possible solution that solves both problems is property organizing the project and (optionally, ut a good idea) packaging your library (that is, make your code "installable"). Then, once your library is installed (in the virtual environment you are working) you don't need hacky sys.path solutions. You will be able to import your library from any folder.
Furthermore, don't treat your modules as scripts (don't run your modules). Use a separate python file as your "executable" (or entry point) and import everything you need from there. With this, relative imports in your module*.py files will work correctly and you don't get confused.
A possible directory structure could be
/project-dir/
- apps/
- main.py
- yourlib/
-/__ init__.py
-/dir-1/
-/__ init__.py
-/module_1.py
-/dir-2/
-/__ init__.py
-/module_2.py
-/module_3.py
Notice that the the yourlib folder as well as subfolders contain an __init__.py file. With this structure, you only run main.py (the name does not need to be main.py).
Case 1: You don't want to package your library
If you don't want to package your library, then you can add sys.path.append("../") in main.py to add "the project-dir/ folder to the path. With that your yourlib library will be "importable" in main.py. You can do something like from yourlib import module_2 and it will work correctly (and module_2 can use relative imports). Alternatively, you can also directly put main.py in the project-dir/ folder and you don't need to change sys.path at all, since project-dir/ will be the "working directory" in that case.
Note that you can also have a tests folder inside project-dir and to run a test file you can do the same as you did to run main.py.
Case 2: You want to package your library
The previous solution already solves your problems, but going the extra mile adds some benefits, such as dependency management and no need to change sys.path no matter where you are. There are several options to package your library and I will show one option using poetry due to its simplicity.
After installing poetry, you can run the command below in a terminal to create a new project
poetry new mylib
This creates the following folder structure
mylib/
- README.rst
- mylib/
- __init__.py
- pyproject.toml
- tests
You can then add the apps folder if you want, as well as subfolders inside mylib/ (each with a __init__.py file).
The pyproject.toml file specifies the dependencies and project metadata. You can edit it by hand and/or use poetry to add new dependencies, such as
poetry add pandas
poetry add --dev mypy
to add pandas as a dependency and mypy as a development dependency, for instance. After that, you can run
poetry build
to create a virtual environment and install your library in it. You can activate the virtual environment with poetry shell and you will be able to import your library from anywhere. Note that you can change your library files without the need to run poetry build again.
At last, if you want to publish your library in PyPi for everyone to see you can use
poetry publish --username your_pypi_username --password _passowrd_
TL; DR
Use an organized project structure with a clear place for the scripts you execute. Particularly, it is better if the script you execute is outside the folder with your modules. Also, don't run a module as a script (otherwise you can't use relative imports).
My file structure is
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
in test.py I am trying to import CompareText
from project.features import CompareText
I get an error of:
ModuleNotFoundError: No module named 'features'`
I checked the documentation and I think my import statement is correct. How can I fix it?
Add an __init__ file in test. Your project directory should look like this:
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
__init__.py
test.py
Then in project/tests/test.py the following import statement will work:
from ..features import CompareText
Oh, and this will still raise an error if you try to run it directly. In the question you said you tried to import it like this:
from project.features import CompareText
This will only work if the parent directory of project is in Python's module search path. So, if you want to run the tests directly then modify the module search path as needed (See: sys.path).
Your import statement is supposed to look like this :
(But make sure your working directory is the same directory as your project folder is located during execution)
from project.features import CompareText
This is supposed to work if your current path while executing the script has the project folder
If you execute it while inside project folder you can use:
from .features import CompareText
Hope this helps!
I assume you are running test.py as a script. test.py needs to find the project package and two ways to do that are to make your project installable or to hack sys.path.
Installable
First, change your directory structure a bit so that project is a subdirectory of some anonymous directory you happen to be using for development. If you are checking this stuff into source control, it needs to be written so that it can be checked out anywhere. Move tests down one directory.
mydevdir/
setup.py
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
How write a setup.py. This can get quite complicated. You can read Building and Distributing Packages with Setuptools and lookup other resources on the net, but a minimalist setup.py is
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='project',
version='0.1',
description='This is project: project',
packages=find_packages(),
)
Now, while in mydevdir do python setup.py develop. Or you can actually produce an install package and put it in a virtual env for test.
Hack sys.path
It may be easier to hack paths in test.py. Note that this will need to be undone if you make project installable later. Just add to the top of test.py
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).absolute().parents[2]))
This puts the parent directory in the python path and now project will be found on import. It runs the risk that a .py file in the same directory as project can mask an installed module. If you have a local csv.py and you import csv, you'd get the local.
I'm facing a very strange problem. I've got three files, the first contains the base class, from which the classes in the other two files inherit from.
The strange thing is, everything worked fine yesterday, but one of the files doesn't work anymore today. I haven't touched the imports in the meantime.
.
└── orangecontrib
├──__init__.py
└── prototypes
├──__init__.py
└── widgets
├──__init__.py
├── owpythagorastree.py
├── owclassificationpythagorastree.py
└── owregressionpythagorastree.py
So the classification and regression classes need to inherit from the base class, and the imports are done in the exact same way:
owclassificationpythagorastree.py
...
from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree
...
owregressionpythagorastree.py
...
from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree
...
Yet when I try to run the two scripts from the command line (with python owregressionpythagorastree.py) the regression widget works fine, but the classification widget produces the following error:
Traceback (most recent call last): File
"owclassificationpythagorastree.py", line 6, in
from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree ImportError: No module named
'orangecontrib.prototypes'
This has happened several times in other projects as well, but it eventually sorts itself out. But it bothers me that I don't know what is causing this.
I did try running this both from my regular machine and a python virtualenv, where I have the module installed (I did this with pip install -e . in the base directory).
I see no apparent reason for this behaviour and it is bothering me a lot, so any help or insight as to why this is happening and how to fix it would be appreciated.
EDIT
As requested, I ran import sys; print(sys.path) at the top of both scripts, and after running it through diff, they are both completely identical. Nevertheless, I am posting the results here.
['/home/pavlin/dev/orange3-prototypes/orangecontrib/prototypes/widgets',
'/home/pavlin/dev/orange3',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages/setuptools_git-1.1-py3.5.egg',
'/home/pavlin/dev/orange-bio',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages/pyqtgraph-0.9.10-py3.5.egg',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages/requests-2.9.1-py3.5.egg',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages/slumber-0.7.1-py3.5.egg',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages/Genesis_PyAPI-1.2.0-py3.5.egg',
'/usr/lib/python3.5/site-packages/qt_graph_helpers-0.1.3-py3.5-linux-x86_64.egg',
'/home/pavlin/dev/orange3-prototypes',
'/usr/lib/python3.5/site-packages',
'/home/pavlin/dev/orange3env/lib/python35.zip',
'/home/pavlin/dev/orange3env/lib/python3.5',
'/home/pavlin/dev/orange3env/lib/python3.5/plat-linux',
'/home/pavlin/dev/orange3env/lib/python3.5/lib-dynload',
'/usr/lib64/python3.5', '/usr/lib/python3.5',
'/usr/lib/python3.5/plat-linux',
'/home/pavlin/dev/orange3env/lib/python3.5/site-packages',
'/usr/lib/python3.5/site-packages/setuptools-18.7.1-py3.5.egg',
'/home/pavlin/.local/lib/python3.5/site-packages']
It looks like your problem is a not-complete understanding of how python finds modules.
For an absolute import (i.e. one where you specify the name of the first module such as import mymodule, and don't use a period to do a relative import from the package to which the code belongs, such as from . import mymodule), the order of lookups is:
The current directory.
Any directories in PYTHONPATH in your environment.
Any installed system paths, which may be set in various ways.
To see your complete set of paths to import from, use:
import sys
print(sys.path)
Also, remember that a directory is only importable if it has an __init__.py file – you didn't say whether that was the case or not with your code.
Therefore, provided you have the following layout:
.
└── orangecontrib
├── __init__.py
└── prototypes
├── __init__.py
└── widgets
├── __init__.py
├── owpythagorastree.py
├── owclassificationpythagorastree.py
└── owregressionpythagorastree.py
I would expect that if you run python orangecontrib/prototypes/widgets/owclassificationpythagorastree.py from the base directory ., both should work just fine, without needing to modify your PYTHONPATH, since python always looks in the current directory for absolutely imported modules first.
If you are running from the widgets directory, I would expect it not to work, unless you had first added the base directory to your PYTHONPATH.
As a general hint though, except for very small things, you should avoid mixing module code with script code, for exactly these reasons! Make separate python scripts that are designed to be run from a command line, and separate python module code that is designed to be imported. Avoid making modules that can also be run from the command line as scripts.
Just a thought. Have you checked the permissions? Maybe somehow the permissions got messed up and they are giving you the problems.