Develop Reference

Python unittest output only on test failures

Is there a way to write (or run) a set of Python unittest tests so that there is no output except when there is a test failure?
For example, if tests/mytests.py contains some unittest tests, then running python3 test/mytests.py will only output to stdout (or stderr) if there is a test failure.

Yes there is. I was able to combine the techniques in the answers to these two questions to get it to work:
Python, unittest: Can one make the TestRunner completely quiet?
argparse and unittest python
You can uncomment the test_should_fail() test to verify what happens when a test fails.
# mymodule.py
def myfunc():
return True
import unittest
import os
class TestMyFunc(unittest.TestCase):
def test_myfunc(self):
self.assertEqual(myfunc(), True)
# def test_should_fail(self):
# self.assertEqual(True, False)
if __name__ == '__main__':
alltests = unittest.TestLoader().loadTestsFromTestCase(TestMyFunc)
runner = unittest.TextTestRunner(stream=open(os.devnull, 'w'))
result = runner.run(alltests)
if len(result.failures) or len(result.errors):
print('There were failures or errors.')

Skip testsuite if fixture fails in pytest

I am having a testsuite which is a class and each testmethod is a testcase. I am also having a fixture for each method (testcase) which does some basic things. I am having a strange requirement where I want to skip all the testcases in the test suite if testfixture fails.
below is the sample suite I am having
class TestFeature:
#pytest.fixture(scope="function", autouse=True)
def _wrapper(self):
print("Some precomditions")
yield
print("some post conditions")
def test_first_testcase(self):
print("First testcas")
def test_second_testcase(self):
print("second testcas")
def test_third_testcase(self):
print("Third testcas")
Now I want if my fixture fails, I want to abort this test suite. Suppose My first testcase failed. Fixture for next test case failed. I want to abort the suite and do not want to execute third testcase.
How to do this in pytest

pytest allows you to do it by using the x flag:
pytest -x
You can also use maxfail flag if you wish to customize the number of failures before quitting:
--maxfail=NUMBER
In case you need to exit the test suite from within your code, you can use the exit method:
import pytest
pytest.exit()

py.test reporting IndexError/KeyError as a Failure instead of Error

import unittest
import logging
FORMAT = '%(asctime)-15s %(message)s'
logging.basicConfig(format=FORMAT)
log = logging.getLogger(__name__)
log.root.setLevel(logging.DEBUG)
class FixturesTest(unittest.TestCase):
def setUp(self):
log.info('starting setup')
self.fixture = range(1, 10)
log.warn('Hello world')
def tearDown(self):
log.info('starting teardown')
del self.fixture
log.info('Goodbye world')
def test_pass(self):
log.info('in test pass')
self.assertEqual(self.fixture, range(1, 10))
log.error('Test world')
def test_fail(self):
log.info('in test fail')
log.error("let's fail a test")
assert(1==0)
def test_error(self):
log.info('in test error')
log.info("let's error a test")
d = []
d[1]
if __name__ == '__main__':
unittest.main()
running the above code using py.test, it reports the test_error test method as a test failure instead of test error. whereas running it using unittest shows the test_error as an error instead of failure, could you share any thoughts on this?
Update : Same code, nose also consider it as an error instead of failure for the test_error, is this an issue in py.test ??
Update : If you are not sure what I refer by this, if you run py.test test_filename.py, you get 1 pass, 2 failures, if you run python -m unittest test_filename.py or nosetest test_filename.py, it's 1 pass, 1 failure, 1 error.

pytest doesn't differentiate from test failures (due to an AssertionError) and test errors (any other exception) like unittest does. This is not a bug, but by design.
There's been some discussion in this thread to add support for this type of separation and it resulted in the pytest-finer-verdicts plugin.

testing click python applications

click is a python package for creating nice commandline interfaces for your applications. I have been playing with click a bit and today pushed this simple roman numeral converter on github.
What I want to do now is to test my click application. I am reading through the documentation, but don't know how to run the tests.
Does anyone have experience with testing click applications?

Putting the code below in test_greet.py:
import click
from click.testing import CliRunner
def test_greet():
#click.command()
#click.argument('name')
def greet(name):
click.echo('Hello %s' % name)
runner = CliRunner()
result = runner.invoke(greet, ['Sam'])
assert result.output == 'Hello Sam\n'
if __name__ == '__main__':
test_greet()
If simply called with python test_greet.py the tests pass and nothing is shown. When used in a testing framework, it performs as expected. For example nosetests test_greet.py returns
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK

pytest has the handlers for the asserts.
To run tests against an existing script, it must be 'imported'.
import click
from click.testing import CliRunner
from click_app import configure, cli
def test_command_configure():
runner = CliRunner()
result = runner.invoke(cli, ["configure"])
assert result.exit_code == 0
assert result.output == 'configure'

Trying to implement python TestSuite

I have two test cases (two different files) that I want to run together in a Test Suite. I can get the tests to run just by running python "normally" but when I select to run a python-unit test it says 0 tests run. Right now I'm just trying to get at least one test to run correectly.
import usertest
import configtest # first test
import unittest # second test
testSuite = unittest.TestSuite()
testResult = unittest.TestResult()
confTest = configtest.ConfigTestCase()
testSuite.addTest(configtest.suite())
test = testSuite.run(testResult)
print testResult.testsRun # prints 1 if run "normally"
Here's an example of my test case set up
class ConfigTestCase(unittest.TestCase):
def setUp(self):
##set up code
def runTest(self):
#runs test
def suite():
"""
Gather all the tests from this module in a test suite.
"""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(ConfigTestCase))
return test_suite
if __name__ == "__main__":
#So you can run tests from this module individually.
unittest.main()
What do I have to do to get this work correctly?

you want to use a testsuit. So you need not call unittest.main().
Use of testsuit should be like this:
#import usertest
#import configtest # first test
import unittest # second test
class ConfigTestCase(unittest.TestCase):
def setUp(self):
print 'stp'
##set up code
def runTest(self):
#runs test
print 'stp'
def suite():
"""
Gather all the tests from this module in a test suite.
"""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(ConfigTestCase))
return test_suite
mySuit=suite()
runner=unittest.TextTestRunner()
runner.run(mySuit)

All of the code to create a loader and suite is unnecessary. You should write your tests so that they are runnable via test discovery using your favorite test runner. That just means naming your methods in a standard way, putting them in an importable place (or passing a folder containing them to the runner), and inheriting from unittest.TestCase. After you've done that, you can use python -m unittest discover at the simplest, or a nicer third party runner to discover and then run your tests.

If you are trying to manually collect TestCases, this is useful: unittest.loader.findTestCases():
# Given a module, M, with tests:
mySuite = unittest.loader.findTestCases(M)
runner = unittest.TextTestRunner()
runner.run(mySuit)

I am assuming you are referring to running python-unit test against the module that consolidates the two test. It will work if you create test case for that module ie. subclassing unittest.TestCase and having a simple test that starts with the word 'test'.
e.g.
class testall(unittest.TestCase):
def test_all(self):
testSuite = unittest.TestSuite()
testResult = unittest.TestResult()
confTest = configtest.ConfigTestCase()
testSuite.addTest(configtest.suite())
test = testSuite.run(testResult)
print testResult.testsRun # prints 1 if run "normally"
if __name__ == "__main__":
unittest.main()