Develop Reference

How to use pytest-custom_exit_code plugin

Need help!
I have a job on Gitlab ci, that runs tests and reruns failed ones. If there are no failed tests, job fails with exit code 5, that means that there are no tests for running. I found out that there is plugin "pytest-custom_exit_code", but I don't know how to correctly use it.
I need just to add command 'pytest --suppress-no-test-exit-code' to my runner.sh?
It looks like this now:
#!/bin/sh
/usr/local/bin/pytest -m test
/usr/local/bin/pytest -m --last-failed --last-failed-no-failures none test

Assumption here is that plugin is installed first using
pip install pytest-custom_exit_code
command like option pytest --suppress-no-test-exit-code should work after that.
If configuration file like .pytest.ini is used , following lines should be added in it
[pytest]
addopts = --suppress-no-test-exit-code

print the failed test output right after and not only at the end pytest

I have multiple tests that run as part of a GitLab pipline, the issue is that the tests run became very long and when a test in the middle fails one need to wait to the end of the run to see the output of the failed test.
I'm using the following:
pytest -v -m -rxs --stepwise
Is there a way to print the failed test traceback right after it fails without doing the same for the passed tests

How to test single file under pytest

How do you test a single file in pytest? I could only find ignore options and no "test this file only" option in the docs.
Preferably this would work on the command line instead of setup.cfg, as I would like to run different file tests in the ide. The entire suite takes too long.

simply run pytest with the path to the file
something like
pytest tests/test_file.py
Use the :: syntax to run a specific test in the test file:
pytest test_mod.py::test_func
Here test_func can be a test method or a class (e.g.: pytest test_mod.py::TestClass).
For more ways and details, see "Specifying which tests to run" in the docs.

This is pretty simple:
$ pytest -v /path/to/test_file.py
The -v flag is to increase verbosity. If you want to run a specific test within that file:
$ pytest -v /path/to/test_file.py::test_name
If you want to run test which names follow a patter you can use:
$ pytest -v -k "pattern_one or pattern_two" /path/to/test_file.py
You also have the option of marking tests, so you can use the -m flag to run a subset of marked tests.
test_file.py
def test_number_one():
"""Docstring"""
assert 1 == 1
#pytest.mark.run_these_please
def test_number_two():
"""Docstring"""
assert [1] == [1]
To run test marked with run_these_please:
$ pytest -v -m run_these_please /path/to/test_file.py

This worked for me:
python -m pytest -k some_test_file.py
This works for individual test functions too:
python -m pytest -k test_about_something

nosetests --cover-html does not generate html docs

I have installed nose for python 2.6 and it works fine but I was trying use the --cover-html option to generate a html report. I typed the following command from the command line:
nosetests --cover-html
It ran the tests but did not generate the html.
Am I missing something ?

It's old news but the order of the options is important:
nosetests --with-coverage --cover-erase --cover-html-dir=C:/temp/res --cover-html test.py

You also need the --with-coverage option to enable the coverage plugin in the first place.

How do I see stdout when running Django tests?

When I run tests with ./manage.py test, whatever I send to the standard output through print doesn't show. When tests fail, I see an "stdout" block per failed test, so I guess Django traps it (but doesn't show it when tests pass).

Checked TEST_RUNNER in settings.py, it's using a project-specific runner that calls out to Nose. Nose has the -s option to stop it from capturing stdout, but if I run:
./manage.py test -s
manage.py captures it first and throws a "no such option" error. The help for manage.py doesn't mention this, but I found that if I run:
./manage.py test -- -s
it ignores the -s and lets me capture it on the custom runner's side, passing it to Nose without a problem.

Yeah, this issue is caused by NoseTestSuiteRunner. Adding -- -s is tricky and not the best solution.
Try to add the following lines in the settings.py:
NOSE_ARGS = ['--nocapture',
'--nologcapture',]
This solved my problems.

There are several levels of verbosity available which affects the amount of details we see:
You can try:
python manage.py test -v 2
Other levels available are:
-v 0 : Least amount of details
-v 1: Default
-v 2 : More details e.g. print statements included.

Using current versions of all the relevant packages (Django==1.11.2, django-nose==1.4.5 and nose==1.3.7) it is sufficient to add the --nocapture flag when running your tests. Thus a simple
./manage.py test --nocapture
will suffice.
Granted of course that you have
TEST_RUNNER = "django_nose.NoseTestSuiteRunner"
in your settings.py

You probably have some intermediate test runner, such as Nose, intercepting and storing stdout. Try either running the Django tests directly, or write to stderr instead.