I have a python project and want to write a command line launcher. My project works fine when I run tests or launch from my IDE (pycharm). However when I try to launch via the command line I get a ModuleNotFoundError: No module named 'python' error.
Here is my directory structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── create_jenkins_jobs.sh
└── python
├── __init__.py
├── create_credentials.py
├── create_jenkins_jobs_cli.py
├── credential_builder.py
├── jenkins_multibranch_job_builder.py
├── templates
│ ├── job.xml
│ ├── multibranch_job_template.xml
│ ├── ssh_cred_template.xml
│ └── user_pw_cred_template.xml
└── tests
├── __init__.py
├── integration
│ ├── __init__.py
│ ├── build_credential_test.py
│ ├── build_job_test.py
│ ├── conftest.py
│ ├── test_id_rsa
│ └── test_id_rsa.pub
└── unit
The shell script create_jenkins_jobs.sh contains:
#!/bin/bash
python3 python/create_jenkins_jobs_cli.py "${#}"
the file create_jenkins_jobs_cli.py contains the following code:
from typing import Set
from shared.common import preflight_env_check
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
def configure_jenkins_jobs(git_repos: Set[str]):
jenkins_job_builder = JenkinsMultibranchJobBuilder(jenkins_server_url=JENKINS_BASE_SERVER_URL,
jenkins_folder=JENKINS_ROOT_JOB_FOLDER,
user=JENKINS_USER, token=JENKINS_TOKEN,
git_cred_id=GIT_SSH_CRED_ID,
git_repos=git_repos)
jenkins_job_builder.configure_jobs()
configure_jenkins_jobs(GIT_REPOS)
When I launch create_jenkins_jobs.sh or create_jenkins_jobs_cli.py I always get
ModuleNotFoundError: No module named 'python'
$>./create_jenkins_jobs.sh
/Users/pswenson/dev/cc-cicd-automation/.venv/bin/python3: Error while finding module specification for 'python/create_jenkins_jobs_cli.py' (ModuleNotFoundError: No module named 'python/create_jenkins_jobs_cli')
I've tried all sorts of different techniques to get this to work with PYTHONPATH, working directories, etc...
It seems there is something basic about how python modules work that I don't understand.
Does changing the line:
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
to
from jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
Sort you?
Related
I am trying to better understand importing modules. I read about how to do this from here https://stackoverflow.com/a/14132912/14179793 and I can get it to work using solution 1. There is an additional variable that I need to figure out though.
This is a dummy project I am testing with:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
└── src
└── src_lib
└── src_lib.py
With this setup I can do:
python -m b_package.test_script
this is lib a function
This is src_lib_function.
test_script.py:
from a_package.lib_a import lib_a_function
from src.src_lib.src_lib import src_lib_function
if __name__ == '__main__':
lib_a_function()
src_lib_function()
pass
The goal is to make b_package/test_script.py executable without using python test_script ie ./test_script
However, adding the shebang at the top #!/usr/bin/env python causes an import error:
$ ./b_package/test_script.py
Traceback (most recent call last):
File "./b_package/test_script.py", line 2, in <module>
from a_package.lib_a import lib_a_function
ModuleNotFoundError: No module named 'a_package'
I assume it is because python is not loading it as a module based off the above mentioned question but I am not sure how to resolve this.
I ended up using setuptools as suggested by kosciej16 to achieve the desired results.
New project structure:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
├── pyproject.toml
├── setup.cfg
└── src
├── __init__.py
└── src_lib
├── __init__.py
└── src_lib.py
setup.cfg:
[metadata]
name = cli_test
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
test_script = b_package.test_script:main
This allows the user to clone the repo and run pip install . from the top level then they can execute the script by just typing test_script
Directory structure:
.
├── classification
│ ├── lstm_repres.py
│ ├── lstm_test.py
│ ├── lstm_train.py
│ ├── lstm_utils.py
│ ├── lstm_utils.pyc
│ └── svm_run.py
├── utils
│ ├── data_splits.py
│ ├── evaluation.py
│ ├── load_data.py
│ ├── NLTKPreprocessor.py
│ ├── resources.py
│ ├── TfIdf.py
│ └── vector_utils.py
at lstm_train.py:
from classification.lstm_utils import *
when i try to run with both python2 lstm_train.py or python3 lstm_train.py, i get:
Traceback (most recent call last):
File "classification/lstm_train.py", line 7, in <module>
from classification.lstm_utils import *
ModuleNotFoundError: No module named 'classification'
The same happens at the line
from utils.data_splits import get_train, get_val
What is the issue here? Why does the format from <dir>.<script> import * doesn't seem to be working here?
EDIT
As suggested by #deathangel908, I've added __init__.py empty files to both directories, but the error persists.
In order to mark directory as a package you want to import, it needs to have least empty __init__.py within that directory. So add __init__.py files to classification and utils directories.
In order to import absolute path package you need to run python interpreter from the root package as a module: python3 -m "classification.lstm_train"
I installed a package in my anaconda environment by entering the following line:
pip3 install -e git+https://github.com/gauravmm/jupyter-testing.git#egg=jupyter-testing
I keep getting ModuleNotFoundError: No module named 'testing' on line: from testing.testing import test. I have no idea why this is happening, and believe it has something to do with the way my directory structure is set up.
My directory tree looks like this:
├── hw1_get_started.ipynb
├── requirements.txt
└── src
└── jupyter-testing
├── jupyter_testing.egg-info
│ ├── dependency_links.txt
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ └── top_level.txt
├── LICENSE
├── README.md
├── setup.py
└── testing
├── __init__.py
└── testing.py
I am trying to use this module : https://github.com/gauravmm/jupyter-testing.git#egg=jupyter-testing to do some testing in an online class.
I appreciate any help and explanation as to what I am doing wrong! :)
I have a project with directory structure like below
.
├── Pipfile
├── Pipfile.lock
├── module
│ ├── __init__.py
│ ├── helpers
│ │ ├── __init__.py
│ │ ├── __pycache__
│ │ │ └── __init__.cpython-36.pyc
│ │ ├── dynamo.py
│ │ └── logger.py
│ └── test.py
Relavant code
logger.py
import click
import sys
from tabulate import tabulate
def formatter(string, *rest):
return string.format(*rest)
def info(*rest):
"""Write text in blue color
"""
click.echo(click.style('☵ ' + formatter(*rest), fg='blue'))
test.py
import helpers
helpers.logger.info('Trying')
When I try to run using the command
python3 module/test.py
I get this error
Traceback (most recent call last):
File "module/test.py", line 4, in <module>
helpers.logger.info('Trying')
AttributeError: module 'helpers' has no attribute 'logger'
I have tried restructuring the code. Putting the helpers directory outside, in level with module directory. But still it didn't work, which it should not have, from what I have read. I tried researching a bit about __init__.py and python module system. The more I read, the more confusing it gets. But from whatever I learned, I created another sample project. With the following structure,
.
└── test
├── __init__.py
├── helpers
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-36.pyc
│ │ └── quote.cpython-36.pyc
│ └── quote.py
├── index.py
├── logger
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-36.pyc
│ │ └── info.cpython-36.pyc
│ └── info.py
Code is same as first project.
Here when I do,
python3 test/index.py
It works as expected. The only difference between the two projects:
In the first project, I used pipenv to install deps and create virtual environment.
Using your initial layout (with loggers as a submodule of the helpers package), you'd need to explicitely import loggers in helpers/__init__.py to expose it as an attribute of the helpers package:
# helpers/__init__.py
from . import logger
logger is module and not attribute and helpers.logger evalutes logger as attribute. Actually you should do as follow:
from helpers import logger
print(logger.info('Trying'))
I'm trying to import the module my_app/common/model/Model.py into the file /my_app/common/database/db_update.py. My PYTHONPATH points to app/common/. However, when I try to do so, I get the error message:
File "/home/me/my_app/common/database/db_update.py", line 26, in <module>
import model.Model
ImportError: No module named Model
A simplified representation of my file structure is as follows:
common/
├── customlogger
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── logger.py
│ └── logger.pyc
├── database
│ ├── DBSetup.py
│ ├── DBSetup.pyc
│ ├── db_update.py
│ ├── db_update.pyc
│ ├── __init__.py
│ └── __init__.pyc
├── __init__.py
├── __init__.pyc
└── model
├── __init__.py
├── __init__.pyc
├── Model.py
└── Model.pyc
The head of the db_update.py file contains the following imports (amongst others) in this order:
import customlogger.logger
import database.DBSetup
import model.Model
Some information that may be useful:
I have tried importing other modules from the model/ directory. These are also not recognised and result in the same error (with the respective module's name instead).
I am running a bash script to do this, from a directory adjacent to common/ - call it my_app/src/.
All my __init__.py files are empty.
I'm running Ubuntu 14.04.
It seems to me that python recognises the model/ directory (else, the error message would read ImportError: No module named model.Model), but seems to think that it doesn't contains any modules despite the presence of the __init__.py file. Is this the case? And if so, why does this happen exclusively in the model/ directory and not the database/ nor the customlogger/ directories? Finally, how can I fix this?
Many thanks in advance.