Develop Reference

pytest coverage report missing code in parallel gitlab pipelines' test execution

I am trying to set up test coverage and generate reports for sonarqube, however, there are some inconsistencies with my results.
I have several steps defined in my gitlab-ci.yml file to run tests in parallel, that generate a single coverage report:
.run_warehouse_tests: &run_warehouse_tests |
echo "Running tests for Warehouse"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-warehouse.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.run_logistics_tests: &run_logistics_tests |
echo "Running tests for Logistics"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-logistics.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.run_packaging_tests: &run_packaging_tests |
echo "Running tests for Packaging"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-packaging.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.
.
.
test_warehouse:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_warehouse_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-warehouse.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
test_logistics:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_logistics_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-logistics.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
test_packaging:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_packaging_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-packaging.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
I also have a .coveragerc file:
[run]
parallel = true
However, at the moment the report that gets generated misses some code. For some of the files the coverage is shown only for imports and function definitions, but not the functions' code itself.
Does anyone have experience with this and has an idea of why some code is not covered? I am thinking this might be because of the parallel execution in gitlab pipelines and maybe some tests are finishing before others? Although I'm using --cov-append to make sure that only one report is generated...
I have tried various things, like using coverage instead of pytest, trying to configure tox.ini, setting various flags in the pytest command, but so far no success.

Cobertura coverage report not showing in MR's diff

I have recently added the cobertura coverage report to my repository, but it still does not show the coverage in an MR's diff.
Here is the job of my .gitlab-ci.yml that generates the coverage report:
coverage-report:
stage: coverage
script:
- tox -e coverage-report
coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
artifacts:
name: "coverage"
paths:
- public/coverage
expire_in: 1 week
reports:
cobertura: public/coverage/coverage.xml
expose_as: "coverage"
And here is my tox.ini:
[tox]
envlist =
coverage-report
minversion = 3.4
[testenv:coverage-report]
basepython = python2.7-32
skip_install = True
deps =
coverage
commands =
coverage run -m pytest -s -vv -x --junitxml=public/test-report.xml tests/
coverage report
coverage html
coverage xml
I am pretty sure everything goes well with the report because not only does its XML exist under public/coverage (which I can see through the published artifacts), but the coverage % summary also shows up in the job and MR. But the coverage still does not show up in the MR's diff. I also tried opening the Network tab of my browser and look for the merge_requests/26/coverage_reports.json HTTP request, and that is coming up empty (more specifically, the response is {"files":{}}), which I do not think is supposed to be happening.
I am using Python 2.7-32 and Coverage.py to get the report. My GitLab is self-hosted with version 14.9.5-ee. Here is a link to download my coverage.xml. It is not the complete coverage, but it shows 2 files which show up in the MR's diff but have no coverage information.

Give coverage_report in this form.
artifacts:
expire_in: 1 week
paths:
- public/coverage
reports:
coverage_report:
coverage_format: cobertura
path: public/coverage/coverage.xml
Also at times it can take some time for the coverage to be reflected in the diff. Gitlab runs it a background process when it's completed it should show. Sometimes takes a while for me.

Pytest + Coverage generates wrong coverage

I have a source code structure that looks like this
src
main.py
test
default_case
test_default.py
Then if I run coverage against the test folder
coverage run --source test -m py.test
It gives me
$ poetry run coverage report
Name Stmts Miss Cover
--------------------------------------------------
test/default/test_default.py 35 1 97%
--------------------------------------------------
TOTAL 35 1 97%
If I run it against the source (src) folder
$ poetry run coverage report
Name Stmts Miss Cover
---------------------------------
src/main.py 105 105 0%
---------------------------------
TOTAL 105 105 0%
I get no coverage.
In my test, I run main.py as a child process.
Then I switched to run main() function inside main.py and the coverage result turned out to be the same.
Where am I wrong?

After a little digging, combining the Coverage official tutorial and this github thread, the problem was solved.
cd /path/to/project_root
poetry run coverage run --omit '*virtualenvs*' -m pytest
poetry run coverage report -m
Note that I use poetry to manage my dependencies, so you should remove poetry run if you don't use poetry.

How can I use pytest-cov to both generate a coverage report and also print to terminal?

Background
I'm new to using pytest and pytest-cov having switched over from unittest + coverage.py
I first set up my automated tests to run in this way:
python3 -m pytest --cov=myapplication
which gave me output like this to the terminal:
----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name Stmts Miss Cover
-----------------------------------------------
myapplication/__init__.py 0 0 100%
myapplication/file.py 30 30 0%
myapplication/another_file.py 20 6 70%
[...]
-----------------------------------------------
TOTAL 1195 464 61%
Then i wanted to generate an xml report so i changed the command:
python3 -m pytest --cov-report xml:coverage.xml --cov=myapplication
Problem
The problem i'm having is that after adding --cov-report xml:coverage.xml i no longer get any output to the terminal
Looking at the documentation for pytest-cov i find this:
These three report options output to files without showing anything on the terminal:
[goes on to show xml, html and annotation reporting options]
Question
How can i both generate a report and also print to terminal in the same test run? (Is this even possible?)
(I could run the test suite two times, but if i can i'd like to do everything at once)
I am using these versions:
Python 3.8.5
pytest 6.2.2 (the latest version as of writing this)
pytest-cov 2.11.1 (-"-)

You can do this by specifying another --cov-report argument with one of the terminal output formats. You can have --cov-report term or --cov-report term-missing. For example:
python3 -m pytest --cov-report term --cov-report xml:coverage.xml --cov=myapplication
See the pytest-cov docs you linked to for how term and term-missing work.

nose get list of test without running them

Is there a way to get a list of all the tests currently recognized by nose, without running them?
According to the doc
--collect-only Enable collect-only: Collect and output test names only, don't run any tests. [COLLECT_ONLY]
However when I do nosetests --collect-only I get:
Ran 101 tests in 0.642s
OK
How can I get names those tests?

Try with the -v option:
nosetests -v --collect-only
If you're trying to debug how nose actually finds your tests, go for -vv