Is there a way to write (or run) a set of Python unittest tests so that there is no output except when there is a test failure?
For example, if tests/mytests.py contains some unittest tests, then running python3 test/mytests.py will only output to stdout (or stderr) if there is a test failure.
Yes there is. I was able to combine the techniques in the answers to these two questions to get it to work:
Python, unittest: Can one make the TestRunner completely quiet?
argparse and unittest python
You can uncomment the test_should_fail() test to verify what happens when a test fails.
# mymodule.py
def myfunc():
return True
import unittest
import os
class TestMyFunc(unittest.TestCase):
def test_myfunc(self):
self.assertEqual(myfunc(), True)
# def test_should_fail(self):
# self.assertEqual(True, False)
if __name__ == '__main__':
alltests = unittest.TestLoader().loadTestsFromTestCase(TestMyFunc)
runner = unittest.TextTestRunner(stream=open(os.devnull, 'w'))
result = runner.run(alltests)
if len(result.failures) or len(result.errors):
print('There were failures or errors.')
Related
I have test cases in unittest framework. Have integrated them with teamcity.
Teamcity gives success even if my testcase fail. I Think, it is so since my unittest process exit with code 0 which is success for teamcity. I have integrated report, which show test failure. What needs to be done, so that teamcity also show failure if test fails.
code:
import unittest
class Login(unittest.TestCase):
def test_002_login_failure(self):
assert 1==2
You should prefer to use the unittest assertions so for example
self.assertEqual(1, 2)
Also you can use the teamcity-messages module to emit test messages the TeamCity will handle appropriately. Then you could change your main (if you have one) to something like the following
import teamcity
from teamcity.unittestpy import TeamcityTestRunner
if __name__ == '__main__':
if teamcity.is_running_under_teamcity():
runner = TeamcityTestRunner()
else:
runner = unittest.TextTestRunner()
unittest.main(testRunner=runner)
I am writing a set of test cases say Test1, Test2 in a test module.
Is there a way to skip Test1 or selectively execute only Test2 in that module using the command nose.main()?
My module contains,
test_module.py,
class Test1:
setUp(self):
print('setup')
tearDown(self):
print('teardown')
test(self):
print('test1')
class Test2:
setUp(self):
print('setup')
tearDown(self):
print('teardown')
test(self):
print('test2')
I run it from a different python file using,
if __name__ == '__main__':
nose.main('test_module')
The notion of skipping test and not running a test are different in the context of nose: skipped tests will be reported as skipped at the end of the test result. If you want to skip the test you would have to monkey patch your test module with decorators or do some other dark magic.
But if you want to just not run a test, you can do it the same way you would do it from the command line: using --exclude option. It takes a regular expression of the test you do not want to run. Something like this:
import sys
import nose
def test_number_one():
pass
def test_number_two():
pass
if __name__ == '__main__':
module_name = sys.modules[__name__].__file__
nose.main(argv=[sys.argv[0],
module_name,
'--exclude=two',
'-v'
])
Running the test will give you:
$ python stackoverflow.py
stackoverflow.test_number_one ... ok
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
In python 2.6, I've been reading through the unittest documentation. But I still haven't found this answer.
What function does pyton -m unittest execute?
For example, how would I modify this code such that just executing python -m unittest would detect it and run the test?
import random
import unittest
class TestSequenceFunctions(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def test_shuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
def test_choice(self):
element = random.choice(self.seq)
self.assertTrue(element in self.seq)
def test_sample(self):
self.assertRaises(ValueError, random.sample, self.seq, 20)
for element in random.sample(self.seq, 5):
self.assertTrue(element in self.seq)
if __name__ == '__main__':
unittest.main()
EDIT:
Note this is just a example, I'm actually trying to get it to detect and run multiple tests as a suite, here is my starting point - but python -m unittest doesn't detect it nor does python -m unittest discovery work with it. I have to call python discovery.py to execute it.
discovery.py:
import os
import unittest
def makeSuite():
"""Function stores all the modules to be tested"""
modules_to_test = []
test_dir = os.listdir('.')
for test in test_dir:
if test.startswith('test') and test.endswith('.py'):
modules_to_test.append(test.rstrip('.py'))
all_tests = unittest.TestSuite()
for module in map(__import__, modules_to_test):
module.testvars = ["variables you want to pass through"]
all_tests.addTest(unittest.findTestCases(module))
return all_tests
if __name__ == '__main__':
unittest.main(defaultTest='makeSuite')
python -m something executes the module something as a script. i.e. from python --help:
-m mod : run library module as a script (terminates option list)
The unittest module can be run as a script -- And the arguments that you pass to it determine which files it tests. The commandline interface is also documented in the language reference.
I cannot figure out how to run single unit tests from within a python script and collect the results.
Scenario: I have a battery of tests that check various methods producing various statistical distributions of different objects. The tests sometimes fail, as they should given that I am basically checking for particular kinds of randomness. I would like to run the tests repeatedly from a script or even from the interpreter and collect results for further analysis.
Suppose I have a module myTest.py with:
class myTest(unittest.TestCase):
def setup(self):
...building objects, etc....
def testTest1(self):
..........
def testTest2(self):
..........
Basically I need to:
run the setup method
run testTest1 (say), 100 times
collect the failures
return the failures
The closest I got to was (using code from a similar question):
from unittest import TextTestRunner, TestSuite
runner = TextTestRunner(verbosity = 2)
tests = ['testTest1']
suite = unittest.TestSuite(map(myTest, tests))
runner.run(suite)
But this does not work, because:
runner.run(suite) does not run the setup method
and
I cannot catch the exception it throws when testTest1 fails
you simply need to add the test that you want to run multiple times to the suite.
Here is a complete code example. You can also see this code running in an interactive Python console to prove that it does actually work.
import unittest
import random
class NullWriter(object):
def write(*_, **__):
pass
def flush(*_, **__):
pass
SETUP_COUNTER = 0
class MyTestCase(unittest.TestCase):
def setUp(self):
global SETUP_COUNTER
SETUP_COUNTER += 1
def test_one(self):
self.assertTrue(random.random() > 0.3)
def test_two(self):
# We just want to make sure this isn't run
self.assertTrue(False, "This should not have been run")
def suite():
tests = []
for _ in range(100):
tests.append('test_one')
return unittest.TestSuite(map(MyTestCase, tests))
results = unittest.TextTestRunner(stream=NullWriter()).run(suite())
print dir(results)
print 'setUp was run', SETUP_COUNTER, 'times'
I have two test cases (two different files) that I want to run together in a Test Suite. I can get the tests to run just by running python "normally" but when I select to run a python-unit test it says 0 tests run. Right now I'm just trying to get at least one test to run correectly.
import usertest
import configtest # first test
import unittest # second test
testSuite = unittest.TestSuite()
testResult = unittest.TestResult()
confTest = configtest.ConfigTestCase()
testSuite.addTest(configtest.suite())
test = testSuite.run(testResult)
print testResult.testsRun # prints 1 if run "normally"
Here's an example of my test case set up
class ConfigTestCase(unittest.TestCase):
def setUp(self):
##set up code
def runTest(self):
#runs test
def suite():
"""
Gather all the tests from this module in a test suite.
"""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(ConfigTestCase))
return test_suite
if __name__ == "__main__":
#So you can run tests from this module individually.
unittest.main()
What do I have to do to get this work correctly?
you want to use a testsuit. So you need not call unittest.main().
Use of testsuit should be like this:
#import usertest
#import configtest # first test
import unittest # second test
class ConfigTestCase(unittest.TestCase):
def setUp(self):
print 'stp'
##set up code
def runTest(self):
#runs test
print 'stp'
def suite():
"""
Gather all the tests from this module in a test suite.
"""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(ConfigTestCase))
return test_suite
mySuit=suite()
runner=unittest.TextTestRunner()
runner.run(mySuit)
All of the code to create a loader and suite is unnecessary. You should write your tests so that they are runnable via test discovery using your favorite test runner. That just means naming your methods in a standard way, putting them in an importable place (or passing a folder containing them to the runner), and inheriting from unittest.TestCase. After you've done that, you can use python -m unittest discover at the simplest, or a nicer third party runner to discover and then run your tests.
If you are trying to manually collect TestCases, this is useful: unittest.loader.findTestCases():
# Given a module, M, with tests:
mySuite = unittest.loader.findTestCases(M)
runner = unittest.TextTestRunner()
runner.run(mySuit)
I am assuming you are referring to running python-unit test against the module that consolidates the two test. It will work if you create test case for that module ie. subclassing unittest.TestCase and having a simple test that starts with the word 'test'.
e.g.
class testall(unittest.TestCase):
def test_all(self):
testSuite = unittest.TestSuite()
testResult = unittest.TestResult()
confTest = configtest.ConfigTestCase()
testSuite.addTest(configtest.suite())
test = testSuite.run(testResult)
print testResult.testsRun # prints 1 if run "normally"
if __name__ == "__main__":
unittest.main()