Develop Reference

ModuleNotFoundError: No module named 'python'

I have a python project and want to write a command line launcher. My project works fine when I run tests or launch from my IDE (pycharm). However when I try to launch via the command line I get a ModuleNotFoundError: No module named 'python' error.
Here is my directory structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── create_jenkins_jobs.sh
└── python
├── __init__.py
├── create_credentials.py
├── create_jenkins_jobs_cli.py
├── credential_builder.py
├── jenkins_multibranch_job_builder.py
├── templates
│   ├── job.xml
│   ├── multibranch_job_template.xml
│   ├── ssh_cred_template.xml
│   └── user_pw_cred_template.xml
└── tests
├── __init__.py
├── integration
│   ├── __init__.py
│   ├── build_credential_test.py
│   ├── build_job_test.py
│   ├── conftest.py
│   ├── test_id_rsa
│   └── test_id_rsa.pub
└── unit
The shell script create_jenkins_jobs.sh contains:
#!/bin/bash
python3 python/create_jenkins_jobs_cli.py "${#}"
the file create_jenkins_jobs_cli.py contains the following code:
from typing import Set
from shared.common import preflight_env_check
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
def configure_jenkins_jobs(git_repos: Set[str]):
jenkins_job_builder = JenkinsMultibranchJobBuilder(jenkins_server_url=JENKINS_BASE_SERVER_URL,
jenkins_folder=JENKINS_ROOT_JOB_FOLDER,
user=JENKINS_USER, token=JENKINS_TOKEN,
git_cred_id=GIT_SSH_CRED_ID,
git_repos=git_repos)
jenkins_job_builder.configure_jobs()
configure_jenkins_jobs(GIT_REPOS)
When I launch create_jenkins_jobs.sh or create_jenkins_jobs_cli.py I always get
ModuleNotFoundError: No module named 'python'
$>./create_jenkins_jobs.sh
/Users/pswenson/dev/cc-cicd-automation/.venv/bin/python3: Error while finding module specification for 'python/create_jenkins_jobs_cli.py' (ModuleNotFoundError: No module named 'python/create_jenkins_jobs_cli')
I've tried all sorts of different techniques to get this to work with PYTHONPATH, working directories, etc...
It seems there is something basic about how python modules work that I don't understand.

Does changing the line:
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
to
from jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
Sort you?

Cant import module from script in same/other directories

Directory structure:
.
├── classification
│   ├── lstm_repres.py
│   ├── lstm_test.py
│   ├── lstm_train.py
│   ├── lstm_utils.py
│   ├── lstm_utils.pyc
│   └── svm_run.py
├── utils
│   ├── data_splits.py
│   ├── evaluation.py
│   ├── load_data.py
│   ├── NLTKPreprocessor.py
│   ├── resources.py
│   ├── TfIdf.py
│   └── vector_utils.py
at lstm_train.py:
from classification.lstm_utils import *
when i try to run with both python2 lstm_train.py or python3 lstm_train.py, i get:
Traceback (most recent call last):
File "classification/lstm_train.py", line 7, in <module>
from classification.lstm_utils import *
ModuleNotFoundError: No module named 'classification'
The same happens at the line
from utils.data_splits import get_train, get_val
What is the issue here? Why does the format from <dir>.<script> import * doesn't seem to be working here?
EDIT
As suggested by #deathangel908, I've added __init__.py empty files to both directories, but the error persists.

In order to mark directory as a package you want to import, it needs to have least empty __init__.py within that directory. So add __init__.py files to classification and utils directories.
In order to import absolute path package you need to run python interpreter from the root package as a module: python3 -m "classification.lstm_train"

ModuleNotFoundError: No module named '<Directory name>'

When I call python __main__.py
I receive error:
from facebook.fb_api import FacebookApi
ModuleNotFoundError: No module named 'facebook'
# __main__.py
from facebook.fb_api import FacebookApi
if __name__ == "__main__":
api = FacebookApi()
api.start()
Project structure
facebook/
├── cache.py
├── configs.py
├── fb_api.py
├── __init__.py
├── __main__.py
├── parser
│   ├── cfg.py
│   ├── example.json
│   ├── __init__.py
│   ├── models.py
│   ├── run_parser.py
│   └── utils.py
├── __pycache__
│   └── fb_api.cpython-36.pyc
├── request_handler.py
└── services
├── case_service.py
└── __init__.py

Your fb_api.py is in the same directory as __main__.py so no facebook folder exists for your application. Remove it from your import and it should work

Because __main__.py is in the same directory. So just import fb_api. To use Facebook.fb_api then move main.py to Facebook directory.

You need to run your code from a higher level in your directory structure. Rather than running from within the facebook folder, run from its parent folder with python -m facebook. The -m flag tells Python you're running module by its Python name, which in this case is the facebook package. A package being run that way has its __main__.py file run as the main script, which is exactly what you want.

Try local import
# __main__.py
from .fb_api import FacebookApi
if __name__ == "__main__":
api = FacebookApi()
api.start()

When package has same name as .py file inside the package, import X imports the module and not the package itself. How to avoid?

So the package Im constructing looks something like this:
Blur/
├── blur
│   ├── __init__.py
│   ├── blur.py
│   ├── funcs
│   │   ├── __init__.py
│   │   ├── face_funcs.py
│   │   └── funcs.py
│   ├── tests
│   │   ├── __init__.py
│   │   └── test_blur.py
│   └── utils
│   ├── __init__.py
│   └── timer.py
└── setup.py
Doing import blur imports blur.py module, and not the whole package itself. If I change blur.py's name, then do the import, i get the whole package. Any way to get the whole package without changing blur.py's name?

If you specify the file path to the "Blur" folder in the import statement, that should force python to import the package, and not the .py file. I believe by default python looks for files ending in .py when importing.

AFAIK it is not possible as a built-in Python mechanism, so changing the name would be your best bet.
There are, however, workarounds available, but in my opinion such solutions are rather ugly.
Here is an example:
import importlib
import os
import sys
def custom_import(repo_path, directory, package=False):
absolute_path = repo_path if package else os.path.join(repo_path, directory)
if os.path.exists(absolute_path):
sys.path.insert(0, absolute_path)
if package:
imported = importlib.import_module(f'{directory}')
else:
imported = importlib.import_module(f'{directory}.{directory}')
sys.path.pop(0)
return imported
raise ImportError(' No such package/module found')
Example use:
package = custom_import('D:\PycharmProjects\python-tools', 'roomba', package=True)
module = custom_import('D:\PycharmProjects\python-tools', 'roomba')
module.main()

Python: AttributeError module x has no attribute y

I have a project with directory structure like below
.
├── Pipfile
├── Pipfile.lock
├── module
│   ├── __init__.py
│   ├── helpers
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   └── __init__.cpython-36.pyc
│   │   ├── dynamo.py
│   │   └── logger.py
│   └── test.py
Relavant code
logger.py
import click
import sys
from tabulate import tabulate
def formatter(string, *rest):
return string.format(*rest)
def info(*rest):
"""Write text in blue color
"""
click.echo(click.style('☵ ' + formatter(*rest), fg='blue'))
test.py
import helpers
helpers.logger.info('Trying')
When I try to run using the command
python3 module/test.py
I get this error
Traceback (most recent call last):
File "module/test.py", line 4, in <module>
helpers.logger.info('Trying')
AttributeError: module 'helpers' has no attribute 'logger'
I have tried restructuring the code. Putting the helpers directory outside, in level with module directory. But still it didn't work, which it should not have, from what I have read. I tried researching a bit about __init__.py and python module system. The more I read, the more confusing it gets. But from whatever I learned, I created another sample project. With the following structure,
.
└── test
├── __init__.py
├── helpers
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── quote.cpython-36.pyc
│   └── quote.py
├── index.py
├── logger
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── info.cpython-36.pyc
│   └── info.py
Code is same as first project.
Here when I do,
python3 test/index.py
It works as expected. The only difference between the two projects:
In the first project, I used pipenv to install deps and create virtual environment.

Using your initial layout (with loggers as a submodule of the helpers package), you'd need to explicitely import loggers in helpers/__init__.py to expose it as an attribute of the helpers package:
# helpers/__init__.py
from . import logger

logger is module and not attribute and helpers.logger evalutes logger as attribute. Actually you should do as follow:
from helpers import logger
print(logger.info('Trying'))