My project has the following structure:
my_project
|
setup.py
tox.ini
src
|
core_functions.py
client.py
server.py
sql_database.py
tests.py
The tests.py file contains the tests for all the function in files client.py, server.py, sql_database.py, core_functions.py
Next, I created the tox.ini file which I want to use for automation of py.tests and coverage tests. The structure of the tox.ini file is as follows:
[tox]
envlist = py27,cov
[testenv]
commands = py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
[testenv:py27]
commands = coverage erase
coverage run --source=my_project -m pytest
coverage report -m
deps = pytest
When I run in the command line tox the py.tests succeed, but the coverage tests fail with the following error:
============== test session starts ===================
platform darwin -- Python 2.7.13, pytest-3.1.1, py-1.4.33, pluggy-0.4.0
rootdir: /Users/xyz/Documents/ProjectGit/MyProject/my_project, inifile:
plugins: cov-2.5.1
collecting 10334 items / 167 errorsERROR: InvocationError:
'/Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/py27/bin/coverage run --source=my_project -m pytest'
cov inst-nodeps: /Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/dist/my_project-0.0.1.zip
cov installed: asn1crypto==0.22.0,attrs==17.2.0,Automat==0.6.0,bitarray==0.8.1,boto3==1.4.4,
botocore==1.5.59,cffi==1.10.0,constantly==15.1.0,coverage==4.4.1,cryptography==1.9,
cycler==0.10.0,docutils==0.13.1,enum34==1.1.6,Fabric==1.13.2,functools32==3.2.3.post2,
future==0.16.0,futures==3.1.1,idna==2.5,incremental==17.5.0,ipaddress==1.0.18,jmespath==0.9.3,
my_project==0.0.1,matplotlib==2.0.2,msgpack-python==0.4.8,numpy==1.12.1,paramiko==2.1.2,
Paver==1.2.4,petlib==0.0.41,py==1.4.33,pyasn1==0.2.3,pybloom==1.1,pycparser==2.17,
pyparsing==2.2.0,pytest==3.1.1,pytest-cov==2.5.1,python-dateutil==2.6.0,pytz==2017.2,
s3transfer==0.1.10,scapy==2.3.3,scipy==0.19.0,six==1.10.0,sphinxmix==0.0.6,
subprocess32==3.2.7,Twisted==17.1.0,zope.interface==4.4.1
cov runtests: PYTHONHASHSEED='3903710496'
cov runtests: commands[0] | py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
How can I fix my tox.ini file to make it work?
The problem is the difference between your two invocations.
In [testenv] you're essentially calling:
py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
But the equivalent with coverage you're just running:
py.test
py.test with no arguments will recurse from the current directory and attempt to discover tests. This generally isn't what you want (you don't care about testing the stdlib, and all the dependencies you've added to a virtualenv, etc.)
If you add your arguments from your other invocation to your coverage line it will act as you want:
coverage run --source=my_project -m pytest -sv --doctest-modules my_project/__init__.py my_project/tests.py
If this happens after some modification, you may remove the .lock file and let tox create it again.
Related
script: |
python3 -m coverage erase
python3 -m coverage run -m pytest tests/ -v --junitxml=junit/test-results.xml --capture=${{ parameters.pytest_capture }}
TEST_RESULT=$? # Create coverage report even if a test case failed
python3 -m coverage html
python3 -m coverage xml
python3 -m coverage report --show-missing --fail-under=${{ parameters.covFailUnder }} && exit $TEST_RESULT
Here in python3 -m coverage run -m pytest tests/ -v --junitxml=junit/test-results.xml --capture=${{ parameters.pytest_capture }}
tests are running from test folder, i need to make sure to check for both test and unittestcase folder.
so how do i write the script for reading both tests and **unittests **
I am struck in these, please provide the solution
Here is an example of how to
def test():
result = your_function()
if result == expected_result:
print("Passed")
else:
print("Fail")
Also you may need to know maybe this was answered previously: Whats the efficient way to test if-elif-else conditions in python
To run tests from both tests/ and unittests/ folders, you can modify the script as follows:
python3 -m coverage erase
python3 -m coverage run -m pytest tests/ unittests/ -v --junitxml=junit/test-results.xml --capture=${{ parameters.pytest_capture }}
TEST_RESULT=$? # Create coverage report even if a test case failed
python3 -m coverage html
python3 -m coverage xml
python3 -m coverage report --show-missing --fail-under=${{ parameters.covFailUnder }} && exit $TEST_RESULT
Here, the argument tests/ unittests/ specifies that pytest should search for tests in both tests/ and unittests/ folders.
When using pytest and pytest-cov with tox. My tox.ini is
[tox]
envlist = py3
[testenv]
deps =
pytest
pytest-codeblocks
pytest-cov
commands =
pytest {tty:--color=yes} {posargs} --cov=mypackagename
The output will contain the full tox path of the files, e.g.:
---------- coverage: platform linux, python 3.11.1-final-0 -----------
Name Stmts Miss Cover
------------------------------------------------------------------------------------------------------------
.tox/py3/lib/python3.11/site-packages/mypackagename/__init__.py 1 0 100%
------------------------------------------------------------------------------------------------------------
TOTAL 678 168 75%
I'm not interested in the prefix .tox/py3/lib/python3.11/site-packages/. Is there any way to remove it from the output?
mypackagename/__init__.py 1 0 100%
This is expected. tox runs your tests on the installed project, not local code. Coveragepy can remap from installed to local via https://coverage.readthedocs.io/en/7.0.1/config.html#paths. See tox itself https://github.com/tox-dev/tox/blob/main/pyproject.toml#L98
I have a python project that uses poetry and tox. It has source code, tests and scripts (juptext notebooks). I can't import the dev dependencies in the scripts, but I can in the tests.
When I came across this problem, I created the following minimal example. At first, it didn't work, then I fiddled around with it, and now it's working. So I stripped the project that has the actual problem down so it's indistinguishable other than the project name, location, virtual env, and .git directory, but that's still not working.
UPDATE deleting all build artifacts and the virtualenv for the minimal example makes it stop working again
UPDATE adding the line scripts: poetry install to the tox commands fixed only the minimal example
The source code, tests and scripts are in the following layout
foo
+--foo
| +--__init__.py
|
+--tests
| +--__init__.py
| +--test_foo.py
|
+--scripts
| +--foo_script.py
|
+--pyproject.toml
+--tox.ini
The files are either empty or as follows:
foo_script.py
import requests
test_foo.py
import requests
import pytest
def test():
assert True
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
[tool.poetry.dev-dependencies]
pytest = "^4.6"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
skipsdist = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
commands =
test: pytest
scripts: bash -c 'python3 scripts/*.py'
When I run tox, I get
test run-test-pre: PYTHONHASHSEED='4126239415'
test run-test: commands[0] | pytest
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
cachedir: .tox/test/.pytest_cache
rootdir: /home/#######/foo
collected 1 item
tests/test_foo.py . [100%]
============================== 1 passed in 0.09s ===============================
scripts run-test-pre: PYTHONHASHSEED='4126239415'
scripts run-test: commands[0] | bash -c 'python3 scripts/*.py'
Traceback (most recent call last):
File "scripts/foo_script.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
ERROR: InvocationError for command /bin/bash -c 'python3 scripts/*.py' (exited with code 1)
___________________________________ summary ____________________________________
test: commands succeeded
ERROR: scripts: commands failed
I believe something like the following should work:
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
#
pytest = { version = "^4.6", optional = true }
[tool.poetry.extras]
test = ["pytest"]
# [tool.poetry.dev-dependencies]
# use 'test' extra instead
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
extras =
test
commands =
test: pytest
scripts: bash -c 'for f in scripts/*.py; do python "$f"; done'
Assuming you have installed poetry and tox and pytest are dependencies in your pyproject.yml (notice poetry run, see https://python-poetry.org/docs/cli/#run):
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
optionally you can make the install happen on running the tests by changing the last bit to (but then you will need to have tox installed outside of poetry, which could cause you issues down the line)
commands=
poetry install
poetry run pytest
Also depending on your root folder and where the tests are you can configure the path for tox to change directory to by adding
changedir = tests
In which case the whole file would look like this if you are in directory foo executing tox:
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
changedir = tests
This is an easy fix. First, run rm -rf .tox to delete the .tox directory, probably hasn't installed the files you wanted.
Here is an example of my tox.ini.
[tox]
isolated_build = true
skipsdist = true
envlist = py39
[testenv]
deps = -rrequirements-dev.txt
whitelist_externals =
poetry
skip_install = true
commands =
python -m pytest tests/
As you can see in the tox.ini file I have a separate requirements for dev. This can be generated from poetry via the commands below.
poetry export --format=requirements.txt --without-hashes --with dev --output=requirements.txt
poetry export --format=requirements.txt --without-hashes --output=requirements-dev.txt
Don't ask me why but the deps in [testenv] are concatenated out, as you can see with deps = -rrequirements-dev.txt, would also change your command to python -m pytest /tests
Ciao.
First I need to install the dependencies with poetry install. Then append poetry run to the beginning of the command to enable the dependencies. Also running python scripts like that will just run the first, passing the name of the others as args to the first program. Instead use for f in scripts/*.py; do python "$f"; done (see here)
All together
poetry install
poetry run bash -c 'for f in scripts/*.py; do python "$f"; done'
Is it possible to do the following using a single tox virtual environment?
[tox]
envlist = test, pylint, flake8, mypy
skipsdist = true
[testenv:lint]
deps = pylint
commands = pylint .
[testenv:flake8]
deps = flake8
commands = flake8 .
[testenv:mypy]
commands = mypy . --strict
[testenv:test]
deps = pytest
commands = pytest
As I am only testing on my python version (py3.7), I don't want tox to have to create 4 environments (.tox/test, .tox/pylint,.tox/flake8, .tox/mypy) when they could all be run on a single environment.
I also want to see what failed individually individually, thus don't want to do:
[tox]
skipsdist = true
[testenv]
commands = pylint .
flake8 .
mypy . --strict
pytest
as the output would be like this:
_____________ summary ___________
ERROR: python: commands failed
and not like this:
____________________summary _________________
ERROR: test: commands failed
ERROR: lint: commands failed
ERROR: mypy: commands failed
test: commands succeeded
Use generative names and factor-specific commands (search the tox configuration doc page for those terms for more info) to implement all of your desired environments as one so they share the same envdir:
[testenv:{lint,flake8,mypy,test}]
envdir = {toxworkdir}/.work_env
deps = pylint, flake8, pytest
commands =
lint: pylint .
flake8: flake8 .
mypy: mypy . --strict
test: pytest
In tox 4+, you may use the plugin tox-ignore-env-name-mismatch and set runner = ignore_env_name_mismatch in the testenv to allow testenvs with different names to share the same virtualenv.
I'm learning to write tests with tox. How do I test only one function with tox? For example if I want to test only test_simple_backup_generation from tests/test_backup_cmd.py of django-backup extension
If you define a parameter {posargs} in your tox.ini you can pass in arguments during execution. In the case of py.test, where
py.test -k test_simple_backup_generation
would only test one function:
[tox]
envlist = py27,py35
[testenv]
deps=pytest
commands=
pip install -e .[tests,docs]
py.test {posargs}
and run like
tox -- -k test_simple_backup_generation