I have a source code structure that looks like this
src
main.py
test
default_case
test_default.py
Then if I run coverage against the test folder
coverage run --source test -m py.test
It gives me
$ poetry run coverage report
Name Stmts Miss Cover
--------------------------------------------------
test/default/test_default.py 35 1 97%
--------------------------------------------------
TOTAL 35 1 97%
If I run it against the source (src) folder
$ poetry run coverage report
Name Stmts Miss Cover
---------------------------------
src/main.py 105 105 0%
---------------------------------
TOTAL 105 105 0%
I get no coverage.
In my test, I run main.py as a child process.
Then I switched to run main() function inside main.py and the coverage result turned out to be the same.
Where am I wrong?
After a little digging, combining the Coverage official tutorial and this github thread, the problem was solved.
cd /path/to/project_root
poetry run coverage run --omit '*virtualenvs*' -m pytest
poetry run coverage report -m
Note that I use poetry to manage my dependencies, so you should remove poetry run if you don't use poetry.
Related
I have recently added the cobertura coverage report to my repository, but it still does not show the coverage in an MR's diff.
Here is the job of my .gitlab-ci.yml that generates the coverage report:
coverage-report:
stage: coverage
script:
- tox -e coverage-report
coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
artifacts:
name: "coverage"
paths:
- public/coverage
expire_in: 1 week
reports:
cobertura: public/coverage/coverage.xml
expose_as: "coverage"
And here is my tox.ini:
[tox]
envlist =
coverage-report
minversion = 3.4
[testenv:coverage-report]
basepython = python2.7-32
skip_install = True
deps =
coverage
commands =
coverage run -m pytest -s -vv -x --junitxml=public/test-report.xml tests/
coverage report
coverage html
coverage xml
I am pretty sure everything goes well with the report because not only does its XML exist under public/coverage (which I can see through the published artifacts), but the coverage % summary also shows up in the job and MR. But the coverage still does not show up in the MR's diff. I also tried opening the Network tab of my browser and look for the merge_requests/26/coverage_reports.json HTTP request, and that is coming up empty (more specifically, the response is {"files":{}}), which I do not think is supposed to be happening.
I am using Python 2.7-32 and Coverage.py to get the report. My GitLab is self-hosted with version 14.9.5-ee. Here is a link to download my coverage.xml. It is not the complete coverage, but it shows 2 files which show up in the MR's diff but have no coverage information.
Give coverage_report in this form.
artifacts:
expire_in: 1 week
paths:
- public/coverage
reports:
coverage_report:
coverage_format: cobertura
path: public/coverage/coverage.xml
Also at times it can take some time for the coverage to be reflected in the diff. Gitlab runs it a background process when it's completed it should show. Sometimes takes a while for me.
I am trying to pblish code coverage results on the pipeline run summary page. This is my pipeline.yaml file:
- bash: |
pip install .[test]
pip install pytest pytest-azurepipelines pytest-cov
pytest --junitxml=junit.xml --cov=./src_dir --cov-report=xml --cov-report=html tests
displayName: Test
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)/**/htmlcov'
The coverage report keeps showing 0% always
How to get the correct code coverage results?
Thanks!
I was able to get the correct code coverage using the following in my Azure pipeline "Test" stage:
echo "
[run]
source =
$(Build.Repository.Name)" > .coveragerc
coverage run --context=mytest -m pytest -v -rA --junitxml=junit.xml --rootdir=tests
coverage report -m --context=mytest
Background
I'm new to using pytest and pytest-cov having switched over from unittest + coverage.py
I first set up my automated tests to run in this way:
python3 -m pytest --cov=myapplication
which gave me output like this to the terminal:
----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name Stmts Miss Cover
-----------------------------------------------
myapplication/__init__.py 0 0 100%
myapplication/file.py 30 30 0%
myapplication/another_file.py 20 6 70%
[...]
-----------------------------------------------
TOTAL 1195 464 61%
Then i wanted to generate an xml report so i changed the command:
python3 -m pytest --cov-report xml:coverage.xml --cov=myapplication
Problem
The problem i'm having is that after adding --cov-report xml:coverage.xml i no longer get any output to the terminal
Looking at the documentation for pytest-cov i find this:
These three report options output to files without showing anything on the terminal:
[goes on to show xml, html and annotation reporting options]
Question
How can i both generate a report and also print to terminal in the same test run? (Is this even possible?)
(I could run the test suite two times, but if i can i'd like to do everything at once)
I am using these versions:
Python 3.8.5
pytest 6.2.2 (the latest version as of writing this)
pytest-cov 2.11.1 (-"-)
You can do this by specifying another --cov-report argument with one of the terminal output formats. You can have --cov-report term or --cov-report term-missing. For example:
python3 -m pytest --cov-report term --cov-report xml:coverage.xml --cov=myapplication
See the pytest-cov docs you linked to for how term and term-missing work.
I have come up with this issue, where by following the instructions found in the coverage.py page I end up getting the report on the test files themselves instead of the application files. I am referring to a Flask application, using unittest as a testing framework, and coverage.py for coverage reports.
The project's folder tree can be represented as follows:
+ root
+
++++ Server
+
++++ server.py
++++ Models
+
++++ other_files
+
++++ Tests
+
++++ test_file.py
If I use the command:
coverage run -m unittest discover && coverage report
I get:
Name Stmts Miss Cover
---------------------------------------------
Models\__init__.py 0 0 100%
test_1.py 76 16 79%
test_2.py 67 3 96%
---------------------------------------------
TOTAL 143 19 87%
Where the output I get doesn't refer to the ApplicationFile(s).py.
If instead I use:
coverage run --source ./../ApplicationFile.py -m unittest discover && coverage report
I get:
Coverage.py warning: Module ./../ApplicationFile.py was never imported. (module-not-imported)
Coverage.py warning: No data was collected. (no-data-collected)
No data to report.
Does someone have an idea of where the issue could be?
I think --source is expecting a directory rather than a file. Try instead coverage run --source ./../ -m unittest discover && coverage report
Trying to fail Jenkins build even if one of the python test files has less than 80% coverage.
Towards it, in Jenkins I'm using nosetests to run test coverage on 2 python test files. It prints results as below. Though one of them has 78% coverage, the build passes. I would want the build to fail in this case
I have added the Cobertura plugin with post build options as, Fail builds if no reports, Fail unhealthy builds, Fail unstable builds. Also set threshold as 80,0,0 for Methods, Packages, Conditionals , Classes & Files.
I tried to run so that total goes below 80 but it still fails.
+ nosetests --with-xunit --with-coverage --cover-erase --cover-package=.
Name Stmts Miss **Cover**
test_sample_script.py 5 0 **100%**
test_sample_script1_80.py 9 2 **78%**
TOTAL 14 2 **86%**
Ran 2 tests in 0.110s
OK
+ python3 -m coverage xml
[Cobertura] Publishing Cobertura coverage report...
[Cobertura] Publishing Cobertura coverage results...
[Cobertura] Cobertura coverage report found.
Finished: SUCCESS
As mentioned here : https://github.com/jenkinsci/cobertura-plugin/issues/111#issuecomment-580886792
With lineCoverageTargets: '90.0, 80.1, 50':
Report health as 100% if line coverage > 90%
Report health as 0% if line coverage < 80.1%
Mark build as unstable if line coverage < 50%