I made a very simple python package by following the packaging guide:
tester
├── A.py
├── B.py
└── __init__.py
The contents of A.py are simply:
import B
__init__.py and B.py are empty files.
I uploaded the package to testpypi and installed it in a virtual environment. I started first by checking if it works by making a simple test.py file:
from tester import A
Then I try to run that file:
(scratch-venv) [ccn#sy337b4 scratch-box]$ python test.py
Traceback (most recent call last):
File "/home/ccn/scratch-box/test.py", line 1, in <module>
from tester import A
File "/home/ccn/scratch-box/scratch-venv/lib/python3.9/site-packages/tester/A.py", line 1, in <module>
import B
ModuleNotFoundError: No module named 'B'
I'm confused because when I was developing the package running python A.py never caused an error, but now when I'm importing A it doesn't work any more.
(within the venv, aka: client using package) I was able to fix this by changing the contents of A.py to :
from tester import B
But that's not a solution because when I go back to developing the package I get this error when running python A.py
Traceback (most recent call last):
File "/home/ccn/tester_package/src/tester/A.py", line 1, in <module>
from tester import B
ModuleNotFoundError: No module named 'tester'
Finally for full context I will post the structure of the package. I am developing in src/tester
.
├── build
│ ├── bdist.linux-x86_64
│ └── lib
│ └── tester
│ ├── A.py
│ ├── B.py
│ └── __init__.py
├── dist
│ ├── example_pkg_2_cuppajoeman-0.0.1-py3-none-any.whl
│ └── example-pkg-2-cuppajoeman-0.0.1.tar.gz
├── LICENSE
├── pyproject.toml
├── README.md
├── setup.cfg
└── src
├── example_pkg_2_cuppajoeman.egg-info
│ ├── dependency_links.txt
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ └── top_level.txt
└── tester
├── A.py
├── B.py
├── __init__.py
└── __pycache__
└── B.cpython-39.pyc
9 directories, 17 files
If I remember correctly, in tester/a.py you should have either one of those:
from . import B
from tester import B
import tester.B
Related
Currently I'm working on a python application to import another three self-defined packages in the same directory. Everything went smoothly on my Linux machine. However, while trying to deploy on another windows11 machine, the relative import mechanism seems to become malfunctional. The second package cannot be import successfully. The detail information list below.
directory structure
.
├── COGNet_lite
│ ├── __init__.py
│ └── src
│ ├ ...
│ └── COGNet.py
├── MICRON_lite
│ ├── __init__.py
│ └── src
│ ├── ...
│ └── MICRON.py
├── PCCNet
│ ├── __init__.py
│ └── src
│ ├── ...
│ └── SafeDrug.py
└── tmp.py
./tmp.py
from PCCNet import (
seed,
sample_desc_d,
sample,
data_voc_convert,
icd9_desc,
atc3ToDrugs,
SafeDrug_main,
SafeDrug_parser
)
from COGNet_lite import COGNet_main
from MICRON_lite import MICRON_main
./PCCNet/__init__.py
from .src.select_pat import sample_desc_d, sample, data_voc_convert, seed
from .src.util import icd9_desc, atc3ToDrugs
from .src.SafeDrug import main as SafeDrug_main, parser as SafeDrug_parser
./COGNet_lite/__init__.py
from .src.COGNet import main as COGNet_main
error message on windows11 with python3.7.13
(python37_env) W:\data\fix_tmp\medrec>python tmp.py
Traceback (most recent call last):
File "tmp.py", line 11, in <module>
from COGNet_lite import COGNet_main
File "W:\data\fix_tmp\medrec\COGNet_lite\__init__.py", line 1, in <module>
from .src.COGNet import main as COGNet_main
ModuleNotFoundError: No module named 'COGNet_lite.src.COGNet'
(python37_env) W:\data\fix_tmp\medrec>python --version
Python 3.7.13
My python project is structured in the so called src-layout.
hyperorg
├── README.md
├── setup.cfg
├── setup.py
├── src
│ └── hyperorg
│ ├── content.py
│ ├── exporter.py
│ ├── __init__.py
│ ├── __main__.py
│ └── reader.py
└── tests
├── helper.py
├── __init__.py
├── test_content.py
├── test_exporter.py
├── test_hyperorg.py
└── test_reader.py
I am not able to get coverage running for it. How can I do this?
What I tried so far
Run coverage in the projects root folder just gives me Nothing to run.
Or coverage run ./src/hyperorg gives me
Traceback (most recent call last):
File "/home/user/tab-cloud/hyperorg/src/hyperorg/__main__.py", line 6, in <module>
from .content import Content
ImportError: attempted relative import with no known parent package
One of the ways of doing that is entering the src folder by cd src and then running coverage from there as following:
coverage run -m unittest discover -s ../tests
You can also leverage rcfile (default .coveragerc) to specify configurations and change HTML report path to be created outside the src folder as well.
I am trying to better understand importing modules. I read about how to do this from here https://stackoverflow.com/a/14132912/14179793 and I can get it to work using solution 1. There is an additional variable that I need to figure out though.
This is a dummy project I am testing with:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
└── src
└── src_lib
└── src_lib.py
With this setup I can do:
python -m b_package.test_script
this is lib a function
This is src_lib_function.
test_script.py:
from a_package.lib_a import lib_a_function
from src.src_lib.src_lib import src_lib_function
if __name__ == '__main__':
lib_a_function()
src_lib_function()
pass
The goal is to make b_package/test_script.py executable without using python test_script ie ./test_script
However, adding the shebang at the top #!/usr/bin/env python causes an import error:
$ ./b_package/test_script.py
Traceback (most recent call last):
File "./b_package/test_script.py", line 2, in <module>
from a_package.lib_a import lib_a_function
ModuleNotFoundError: No module named 'a_package'
I assume it is because python is not loading it as a module based off the above mentioned question but I am not sure how to resolve this.
I ended up using setuptools as suggested by kosciej16 to achieve the desired results.
New project structure:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
├── pyproject.toml
├── setup.cfg
└── src
├── __init__.py
└── src_lib
├── __init__.py
└── src_lib.py
setup.cfg:
[metadata]
name = cli_test
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
test_script = b_package.test_script:main
This allows the user to clone the repo and run pip install . from the top level then they can execute the script by just typing test_script
I have a python project and want to write a command line launcher. My project works fine when I run tests or launch from my IDE (pycharm). However when I try to launch via the command line I get a ModuleNotFoundError: No module named 'python' error.
Here is my directory structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── create_jenkins_jobs.sh
└── python
├── __init__.py
├── create_credentials.py
├── create_jenkins_jobs_cli.py
├── credential_builder.py
├── jenkins_multibranch_job_builder.py
├── templates
│ ├── job.xml
│ ├── multibranch_job_template.xml
│ ├── ssh_cred_template.xml
│ └── user_pw_cred_template.xml
└── tests
├── __init__.py
├── integration
│ ├── __init__.py
│ ├── build_credential_test.py
│ ├── build_job_test.py
│ ├── conftest.py
│ ├── test_id_rsa
│ └── test_id_rsa.pub
└── unit
The shell script create_jenkins_jobs.sh contains:
#!/bin/bash
python3 python/create_jenkins_jobs_cli.py "${#}"
the file create_jenkins_jobs_cli.py contains the following code:
from typing import Set
from shared.common import preflight_env_check
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
def configure_jenkins_jobs(git_repos: Set[str]):
jenkins_job_builder = JenkinsMultibranchJobBuilder(jenkins_server_url=JENKINS_BASE_SERVER_URL,
jenkins_folder=JENKINS_ROOT_JOB_FOLDER,
user=JENKINS_USER, token=JENKINS_TOKEN,
git_cred_id=GIT_SSH_CRED_ID,
git_repos=git_repos)
jenkins_job_builder.configure_jobs()
configure_jenkins_jobs(GIT_REPOS)
When I launch create_jenkins_jobs.sh or create_jenkins_jobs_cli.py I always get
ModuleNotFoundError: No module named 'python'
$>./create_jenkins_jobs.sh
/Users/pswenson/dev/cc-cicd-automation/.venv/bin/python3: Error while finding module specification for 'python/create_jenkins_jobs_cli.py' (ModuleNotFoundError: No module named 'python/create_jenkins_jobs_cli')
I've tried all sorts of different techniques to get this to work with PYTHONPATH, working directories, etc...
It seems there is something basic about how python modules work that I don't understand.
Does changing the line:
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
to
from jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
Sort you?
Directory structure:
.
├── classification
│ ├── lstm_repres.py
│ ├── lstm_test.py
│ ├── lstm_train.py
│ ├── lstm_utils.py
│ ├── lstm_utils.pyc
│ └── svm_run.py
├── utils
│ ├── data_splits.py
│ ├── evaluation.py
│ ├── load_data.py
│ ├── NLTKPreprocessor.py
│ ├── resources.py
│ ├── TfIdf.py
│ └── vector_utils.py
at lstm_train.py:
from classification.lstm_utils import *
when i try to run with both python2 lstm_train.py or python3 lstm_train.py, i get:
Traceback (most recent call last):
File "classification/lstm_train.py", line 7, in <module>
from classification.lstm_utils import *
ModuleNotFoundError: No module named 'classification'
The same happens at the line
from utils.data_splits import get_train, get_val
What is the issue here? Why does the format from <dir>.<script> import * doesn't seem to be working here?
EDIT
As suggested by #deathangel908, I've added __init__.py empty files to both directories, but the error persists.
In order to mark directory as a package you want to import, it needs to have least empty __init__.py within that directory. So add __init__.py files to classification and utils directories.
In order to import absolute path package you need to run python interpreter from the root package as a module: python3 -m "classification.lstm_train"