How do you test a single file in pytest? I could only find ignore options and no "test this file only" option in the docs.
Preferably this would work on the command line instead of setup.cfg, as I would like to run different file tests in the ide. The entire suite takes too long.
simply run pytest with the path to the file
something like
pytest tests/test_file.py
Use the :: syntax to run a specific test in the test file:
pytest test_mod.py::test_func
Here test_func can be a test method or a class (e.g.: pytest test_mod.py::TestClass).
For more ways and details, see "Specifying which tests to run" in the docs.
This is pretty simple:
$ pytest -v /path/to/test_file.py
The -v flag is to increase verbosity. If you want to run a specific test within that file:
$ pytest -v /path/to/test_file.py::test_name
If you want to run test which names follow a patter you can use:
$ pytest -v -k "pattern_one or pattern_two" /path/to/test_file.py
You also have the option of marking tests, so you can use the -m flag to run a subset of marked tests.
test_file.py
def test_number_one():
"""Docstring"""
assert 1 == 1
#pytest.mark.run_these_please
def test_number_two():
"""Docstring"""
assert [1] == [1]
To run test marked with run_these_please:
$ pytest -v -m run_these_please /path/to/test_file.py
This worked for me:
python -m pytest -k some_test_file.py
This works for individual test functions too:
python -m pytest -k test_about_something
Related
Suppose the pytest config file pyproject.toml is in the project directory:
[tool.pytest.ini_options]
addopts = "-rap --capture=tee-sys"
I usually use the options in this file when running the command pytest. And the options are automatically read from the file.
However, sometimes I want to run pytest with -rA only:
pytest -rA # it becomes: pytest -rA --capture=tee-sys
I have to run it with: pytest -rA --capture=fd to override the option --capture=tee-sys in the config file.
Is there any way to force pytest to read options from commandline only?
What I've tried:
In #Andrei 's answer of this question: Is there an option for pytest to ignore the setup.cfg file?
Use -c /dev/null will prevent pytest from finding the config file. I also find that set -c any-non-exist-path is ok.
However, I'm afraid if -c /dev/null has side effects. In pytest docs (https://docs.pytest.org/en/latest/reference/customize.html#finding-the-rootdir), it says If -c is passed in the command-line, use that as configuration file, and its directory as rootdir. I don't know what will happen if rootdir is set as a non exist dir.
My file structure is below,
Logic
--packageA
----file1
----file2
--packageB
----file3
----file4
--tests
----testfile1
----testfile2
----testfile3
--.coveragerc
I'd like to include packageA only for tests, and run testfile1 and testfile2 to measure the coverage.
testfile3 is for packageB.
So I wrote my .coveragerc file,
[run]
branch = True
include = /packageA/*
omit = *tests*
and, when I run coverage with command coverage run --rcfile=../.coveragerc -m pytest in directory tests, it tries running testfile3 as well.
How can I run testfile1 and testfile2 with .coveragerc configuration?
You can use the [run] command_line option to set the command line to use when you run coverage run. But coverage isn't trying to be a general runner. You might want a shell script, a Makefile, or a tox.ini file instead.
Are you typing the coverage run --rcfile=../.coveragerc -m pytest command by hand?
tl;dr:
I'm setting up CI for a project of mine, hosted on github, using tox and travis-ci. At the end of the build, I run converalls to push the coverage reports to coveralls.io. I would like to make this command 'conditional' - for execution only when the tests are run on travis; not when they are run on my local machine. Is there a way to make this happen?
The details:
The package I'm trying to test is a python package. I'm using / planning to use the following 'infrastructure' to set up the tests :
The tests themselves are of the py.test variety.
The CI scripting, so to speak, is from tox. This lets me run the tests locally, which is rather important to me. I don't want to have to push to github every time I need a test run. I also use numpy and matplotlib in my package, so running an inane number of test cycles on travis-ci seems overly wasteful to me. As such, ditching tox and simply using .travis.yml alone is not an option.
The CI server is travis-ci
The relevant test scripts look something like this :
.travis.yml
language: python
python: 2.7
env:
- TOX_ENV=py27
install:
- pip install tox
script:
- tox -e $TOX_ENV
tox.ini
[tox]
envlist = py27
[testenv]
passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH
deps =
pytest
coverage
pytest-cov
coveralls
commands =
py.test --cov={envsitepackagesdir}/mypackage --cov-report=term --basetemp={envtmpdir}
coveralls
This file lets me run the tests locally. However, due to the final coveralls call, the test fails in principle, with :
py27 runtests: commands[1] | coveralls
You have to provide either repo_token in .coveralls.yml, or launch via Travis
ERROR: InvocationError: ...coveralls'
This is an expected error. The passenv bit sends along the necessary information from travis to be able to write to coveralls, and without travis there to provide this information, the command should fail. I don't want this to push the results to coveralls.io, either. I'd like to have coveralls run only if the test is occuring on travis-ci. Is there any way in which I can have this command run conditionally, or set up a build configuration which achieves the same effect?
I've already tried moving the coveralls portion into .travis.yml, but when that is executed coveralls seems to be unable to locate the appropriate .coverage file to send over. I made various attempts in this direction, none of which resulted in a successful submission to coveralls.io except the combination listed above. The following was what I would have hoped would work, given that when I run tox locally I do end up with a .coverage file where I'd expect it - in the root folder of my source tree.
No submission to coveralls.io
language: python
python: 2.7
env:
- TOX_ENV=py27
install:
- pip install tox
- pip install python-coveralls
script:
- tox -e $TOX_ENV
after_success:
- coveralls
An alternative solution would be to prefix the coveralls command with a dash (-) to tell tox to ignore its exit code as explained in the documentation. This way even failures from coveralls will be ignored and tox will consider the test execution as successful when executed locally.
Using the example configuration above, it would be as follows:
[tox]
envlist = py27
[testenv]
passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH
deps =
pytest
coverage
pytest-cov
coveralls
commands =
py.test --cov={envsitepackagesdir}/mypackage --cov-report=term --basetemp={envtmpdir}
- coveralls
I have a similar setup with Travis, tox and coveralls. My idea was to only execute coveralls if the TRAVIS environment variable is set. However, it seems this is not so easy to do as tox has trouble parsing commands with quotes and ampersands. Additionally, this confused Travis me a lot.
Eventually I wrote a simple python script run_coveralls.py:
#!/bin/env/python
import os
from subprocess import call
if __name__ == '__main__':
if 'TRAVIS' in os.environ:
rc = call('coveralls')
raise SystemExit(rc)
In tox.ini, replace your coveralls command with python {toxinidir}/run_coveralls.py.
I am using a environmental variable to run additional commands.
tox.ini
commands =
coverage run runtests.py
{env:POST_COMMAND:python --version}
.travis.yml
language: python
python:
- "3.6"
install: pip install tox-travis
script: tox
env:
- POST_COMMAND=codecov -e TOX_ENV
Now in my local setup, it print the python version. When run from Travis it runs codecov.
Alternative solution if you use a Makefile and dont want a new py file:
define COVERALL_PYSCRIPT
import os
from subprocess import call
if __name__ == '__main__':
if 'TRAVIS' in os.environ:
rc = call('coveralls')
raise SystemExit(rc)
print("Not in Travis CI, skipping coveralls")
endef
export COVERALL_PYSCRIPT
coveralls: ## runs coveralls if TRAVIS in env
#python -c "$$COVERALL_PYSCRIPT"
In tox.ini add make coveralls to commands
Is there a way to get a list of all the tests currently recognized by nose, without running them?
According to the doc
--collect-only Enable collect-only: Collect and output test names only, don't run any tests. [COLLECT_ONLY]
However when I do nosetests --collect-only I get:
Ran 101 tests in 0.642s
OK
How can I get names those tests?
Try with the -v option:
nosetests -v --collect-only
If you're trying to debug how nose actually finds your tests, go for -vv
The python nosetest framework has some command line options to include, exclude and match regex for tests which can be included/excluded and matched respectively.
However they don't seem to be working correctly.
[kiran#my_redhat test]$ nosetests -w cases/ -s -v -m='_size'
----------------------------------------------------------------------
Ran 0 tests in 0.001s
OK
[kiran#my_redhat test]$ grep '_size' cases/test_case_4.py
def test_fn_size_sha(self):
is there some thing wrong with regex matching semantics of nose framework?
Nosetests' -m argument is used to match directories, filenames, classes, and functions. (See the nose docs explanation of this parameter) In your case, the filename of your test file (test_case_4.py) does not match the -m match expression (_size), so is never opened.
You may notice that if you force nose to open your test file, it will run only the specified test:
nosetests -sv -m='_size' cases/test_case_4.py
In general, when I want to match specific tests or subsets of tests I use the --attrib plugin, which is available in the default nose install. You may also want to try excluding tests that match some pattern.
Try removing '=' when specifying the regexp:
$ nosetests -w cases/ -s -v -m '_size'
or keep '=' and spell out --match:
$ nosetests -w cases/ -s -v --match='_size'
Nose is likely using Python's re.match, or something equivalent, which requires a match at the beginning of the string. _size doesn't match because the function name test_fn_size_sha doesn't start with the regex _size.
Try using a regex that matches from the beginning:
nosetests -w cases/ -s -v -m='\w+_size'
This works for me .
nosetests --collect-only test_mytest\test_category --exclude=test_.*Pin
Here I have excluded all test that has a word "Pin" in the test case name.
Note: All test case names start with test_ in my case.