Develop Reference

pytest coverage report missing code in parallel gitlab pipelines' test execution

I am trying to set up test coverage and generate reports for sonarqube, however, there are some inconsistencies with my results.
I have several steps defined in my gitlab-ci.yml file to run tests in parallel, that generate a single coverage report:
.run_warehouse_tests: &run_warehouse_tests |
echo "Running tests for Warehouse"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-warehouse.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.run_logistics_tests: &run_logistics_tests |
echo "Running tests for Logistics"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-logistics.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.run_packaging_tests: &run_packaging_tests |
echo "Running tests for Packaging"
python -m pytest --durations=0 ./src/unittest -m eagle --junitxml="junit-test-result-packaging.xml" --cov-config=.coveragerc --cov-report xml --cov=./src/main --cov-append
coverage lcov
.
.
.
test_warehouse:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_warehouse_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-warehouse.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
test_logistics:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_logistics_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-logistics.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
test_packaging:
stage: test
tags:
- dind
script:
- *set_common_env_vars
- *setup_venv
- *run_packaging_tests
artifacts:
when: always
paths:
- infrastructure/junit-test-result-packaging.xml
- infrastructure/coverage.xml
- infrastructure/coverage.lcov
except:
- tags
I also have a .coveragerc file:
[run]
parallel = true
However, at the moment the report that gets generated misses some code. For some of the files the coverage is shown only for imports and function definitions, but not the functions' code itself.
Does anyone have experience with this and has an idea of why some code is not covered? I am thinking this might be because of the parallel execution in gitlab pipelines and maybe some tests are finishing before others? Although I'm using --cov-append to make sure that only one report is generated...
I have tried various things, like using coverage instead of pytest, trying to configure tox.ini, setting various flags in the pytest command, but so far no success.

Publish Python project code coverage with pytest in Azure Pipelines

I am trying to pblish code coverage results on the pipeline run summary page. This is my pipeline.yaml file:
- bash: |
pip install .[test]
pip install pytest pytest-azurepipelines pytest-cov
pytest --junitxml=junit.xml --cov=./src_dir --cov-report=xml --cov-report=html tests
displayName: Test
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)/**/htmlcov'
The coverage report keeps showing 0% always
How to get the correct code coverage results?
Thanks!

I was able to get the correct code coverage using the following in my Azure pipeline "Test" stage:
echo "
[run]
source =
$(Build.Repository.Name)" > .coveragerc
coverage run --context=mytest -m pytest -v -rA --junitxml=junit.xml --rootdir=tests
coverage report -m --context=mytest

How can I use pytest-cov to both generate a coverage report and also print to terminal?

Background
I'm new to using pytest and pytest-cov having switched over from unittest + coverage.py
I first set up my automated tests to run in this way:
python3 -m pytest --cov=myapplication
which gave me output like this to the terminal:
----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name Stmts Miss Cover
-----------------------------------------------
myapplication/__init__.py 0 0 100%
myapplication/file.py 30 30 0%
myapplication/another_file.py 20 6 70%
[...]
-----------------------------------------------
TOTAL 1195 464 61%
Then i wanted to generate an xml report so i changed the command:
python3 -m pytest --cov-report xml:coverage.xml --cov=myapplication
Problem
The problem i'm having is that after adding --cov-report xml:coverage.xml i no longer get any output to the terminal
Looking at the documentation for pytest-cov i find this:
These three report options output to files without showing anything on the terminal:
[goes on to show xml, html and annotation reporting options]
Question
How can i both generate a report and also print to terminal in the same test run? (Is this even possible?)
(I could run the test suite two times, but if i can i'd like to do everything at once)
I am using these versions:
Python 3.8.5
pytest 6.2.2 (the latest version as of writing this)
pytest-cov 2.11.1 (-"-)

You can do this by specifying another --cov-report argument with one of the terminal output formats. You can have --cov-report term or --cov-report term-missing. For example:
python3 -m pytest --cov-report term --cov-report xml:coverage.xml --cov=myapplication
See the pytest-cov docs you linked to for how term and term-missing work.

Jenkins Build not failing though coverage is below 80%

Trying to fail Jenkins build even if one of the python test files has less than 80% coverage.
Towards it, in Jenkins I'm using nosetests to run test coverage on 2 python test files. It prints results as below. Though one of them has 78% coverage, the build passes. I would want the build to fail in this case
I have added the Cobertura plugin with post build options as, Fail builds if no reports, Fail unhealthy builds, Fail unstable builds. Also set threshold as 80,0,0 for Methods, Packages, Conditionals , Classes & Files.
I tried to run so that total goes below 80 but it still fails.
+ nosetests --with-xunit --with-coverage --cover-erase --cover-package=.
Name Stmts Miss **Cover**
test_sample_script.py 5 0 **100%**
test_sample_script1_80.py 9 2 **78%**
TOTAL 14 2 **86%**
Ran 2 tests in 0.110s
OK
+ python3 -m coverage xml
[Cobertura] Publishing Cobertura coverage report...
[Cobertura] Publishing Cobertura coverage results...
[Cobertura] Cobertura coverage report found.
Finished: SUCCESS

As mentioned here : https://github.com/jenkinsci/cobertura-plugin/issues/111#issuecomment-580886792
With lineCoverageTargets: '90.0, 80.1, 50':
Report health as 100% if line coverage > 90%
Report health as 0% if line coverage < 80.1%
Mark build as unstable if line coverage < 50%

Coverage reports not being processed by codecov anymore

I had setup codecov with gitlab pipelines a while back and was able to see coverage reports in codecov. Since the initial setup the reports stopped processing after a few commits, and I have not been able to figure out what I'm doing wrong to get the reports processing again.
In gitlab pipelines I use tox and pip install codecov:
test:
stage: test
script:
- pip install circuitpython-build-tools Sphinx sphinx-rtd-theme tox codecov
- tox
- codecov -t $CODECOV_TOKEN
artifacts:
paths:
- htmlcov/
In tox I run coverage:
[testenv:coverage]
deps = -rrequirements.txt
-rtest-requirements.txt
commands = coverage run -m unittest discover tests/
coverage html
In codecov I can see where the upload attempts to process, but it fails without much description:
There was an error processing coverage reports.
I've referenced the python tutorials, but can't see what I'm getting wrong.
https://github.com/codecov/codecov-python
https://github.com/codecov/example-python

Looks like maybe this was something on the codecov.io side. I didn't change anything, but came into coverage reports parsing and the badge working again this morning.