Develop Reference

How to run coverage for a Python project with src-layout?

My python project is structured in the so called src-layout.
hyperorg
├── README.md
├── setup.cfg
├── setup.py
├── src
│   └── hyperorg
│   ├── content.py
│   ├── exporter.py
│   ├── __init__.py
│   ├── __main__.py
│   └── reader.py
└── tests
├── helper.py
├── __init__.py
├── test_content.py
├── test_exporter.py
├── test_hyperorg.py
└── test_reader.py
I am not able to get coverage running for it. How can I do this?
What I tried so far
Run coverage in the projects root folder just gives me Nothing to run.
Or coverage run ./src/hyperorg gives me
Traceback (most recent call last):
File "/home/user/tab-cloud/hyperorg/src/hyperorg/__main__.py", line 6, in <module>
from .content import Content
ImportError: attempted relative import with no known parent package

One of the ways of doing that is entering the src folder by cd src and then running coverage from there as following:
coverage run -m unittest discover -s ../tests
You can also leverage rcfile (default .coveragerc) to specify configurations and change HTML report path to be created outside the src folder as well.

Python executable script ModuleNotFound

I am trying to better understand importing modules. I read about how to do this from here https://stackoverflow.com/a/14132912/14179793 and I can get it to work using solution 1. There is an additional variable that I need to figure out though.
This is a dummy project I am testing with:
.
├── a_package
│   ├── __init__.py
│   └── lib_a.py
├── b_package
│   ├── __init__.py
│   └── test_script.py
├── main.py
└── src
└── src_lib
└── src_lib.py
With this setup I can do:
python -m b_package.test_script
this is lib a function
This is src_lib_function.
test_script.py:
from a_package.lib_a import lib_a_function
from src.src_lib.src_lib import src_lib_function
if __name__ == '__main__':
lib_a_function()
src_lib_function()
pass
The goal is to make b_package/test_script.py executable without using python test_script ie ./test_script
However, adding the shebang at the top #!/usr/bin/env python causes an import error:
$ ./b_package/test_script.py
Traceback (most recent call last):
File "./b_package/test_script.py", line 2, in <module>
from a_package.lib_a import lib_a_function
ModuleNotFoundError: No module named 'a_package'
I assume it is because python is not loading it as a module based off the above mentioned question but I am not sure how to resolve this.

I ended up using setuptools as suggested by kosciej16 to achieve the desired results.
New project structure:
.
├── a_package
│   ├── __init__.py
│   └── lib_a.py
├── b_package
│   ├── __init__.py
│   └── test_script.py
├── main.py
├── pyproject.toml
├── setup.cfg
└── src
├── __init__.py
└── src_lib
├── __init__.py
└── src_lib.py
setup.cfg:
[metadata]
name = cli_test
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
test_script = b_package.test_script:main
This allows the user to clone the repo and run pip install . from the top level then they can execute the script by just typing test_script

ModuleNotFoundError: No module named 'main' but path seems correct

The structure of my_dir is
├── README.md
├── main
│   ├── functions
│   │   ├── __pycache__
│   │   ├── my_function.py
│   ├── pipeline.py
│   ├── options
│   │   └── pipeline_options.py
│   └── transforms
│   ├── __pycache__
│   ├── my_transform.py
├── poetry.lock
├── pyproject.toml
└── tests
On pipeline.py:
from main.functions.my_function import MyFunction
On my_function.py:
import apache_beam as beam
class MyFunction(beam.DoFn):
...
I have read similar questions here, including this one which is the solution I have currently. Also read this on Python imports.
When I run pipeline.py, I get
Traceback (most recent call last):
File "pipeline.py", line 7, in <module>
from main.functions.my_function import MyFunction
ModuleNotFoundError: No module named 'main'
Additionally, I'm using VSCode and if I run the file via "Python: Run Python File in Terminal" I get back the error. However, if I run via the debugger the paths are all found and the error is not thrown, which I found odd. Also, VSCode isn't throwing any path warnings.
I know there are similar questions but I haven't been able to figure out what is wrong here and have spent quite some time on this already. Any help/pointer is much appreciated.
Additional info
I'm using python 3.8.1
Using poetry run python main/pipeline.py to run the code
Running from command from my_dir

ModuleNotFoundError with package installed from github

I installed a package in my anaconda environment by entering the following line:
pip3 install -e git+https://github.com/gauravmm/jupyter-testing.git#egg=jupyter-testing
I keep getting ModuleNotFoundError: No module named 'testing' on line: from testing.testing import test. I have no idea why this is happening, and believe it has something to do with the way my directory structure is set up.
My directory tree looks like this:
├── hw1_get_started.ipynb
├── requirements.txt
└── src
└── jupyter-testing
├── jupyter_testing.egg-info
│   ├── dependency_links.txt
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   └── top_level.txt
├── LICENSE
├── README.md
├── setup.py
└── testing
├── __init__.py
└── testing.py
I am trying to use this module : https://github.com/gauravmm/jupyter-testing.git#egg=jupyter-testing to do some testing in an online class.
I appreciate any help and explanation as to what I am doing wrong! :)

How to run my python program with -m (full path)?

I have a structure similar to this:
/home/user/nginx_http/
├── nginx_http
│   ├── common
│   │   ├── autodiscovery.py
│   │   ├── __init__.py
│   ├── __init__.py
│   ├── __main__.py
│   ├── nginx_http.py
├── README.rst
└── setup.py
When I'm in the /home/user/nginx_http folder and I run this:
python -m nginx_http
it works fine. But soon as I try to run it with full path it gives me an error;
python -m /home/user/nginx_http/nginx_http
/bin/python: Import by filename is not supported.; '/home/user/nginx_http/nginx_http' is a package and cannot be directly executed
I want to run my python program as a cronjob which would require the full path. The two options I wanted clarification on:
Should I consider moving my python program into python's module folder?
/usr/lib/python2.7/site-packages
Or should I set PYTHONPATH env to the location of /home/user/nginx_http. Will that break anything as far as other modules are concerned?
Thanks.