I would like to get the test results from my unit tests and then log them. Having some trouble figuring out the best way to do it. Ideally I think I would like to get them from the tearDown method, and then log them there, so that each test is logging it's result as it finishes, but I can't seem to get it to work.
Here is some example code that you can run:
import unittest
class sample_tests(unittest.TestCase):
def test_it(self):
self.assertTrue(1==2)
def tearDown(self):
print("Get test results and log them here")
print(unittest.TestResult())
if __name__=='__main__':
#unittest.main()
suite = unittest.TestSuite()
suite.addTest(sample_tests("test_it"))
runner = unittest.TextTestRunner()
result = runner.run(suite)
print(result.failures)
When you run this you will get the following output:
Get test results and log them here
<unittest.result.TestResult run=0 errors=0 failures=0>
F
======================================================================
FAIL: test_it (__main__.sample_tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\sample.py", line 6, in test_it
self.assertTrue(1==2)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.005s
FAILED (failures=1)
[(<__main__.sample_tests testMethod=test_it>, 'Traceback (most recent call last):\n File ".\\sample.py", line 6, in test_it\n self.assertTrue(1==2)\nAssertionError: False is not true\n')]
PS C:\Users\cn187366\Documents\Python_Test\ETL_Test_Framework>
As you can see, the tear down method is not returning the expected results and I think it is because I'm not referencing the test runner which contains the TestResults object.
EDIT
I've found a solution here:
Getting Python's unittest results in a tearDown() method
Here is the actual code that does what I wanted:
def tearDown(self):
print("Get test results and log them here")
if hasattr(self,'_outcome'):
result = self.defaultTestResult()
self._feedErrorsToResult(result,self._outcome.errors)
print(result)
You can stream to a file with stream=file-name.log. For more detail, check the unittest.TextTestRunner class.
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(YourTestClass)
with open('test_result.out', 'w') as f:
unittest.TextTestRunner(stream=f, verbosity=2).run(suite)
This should work!
import xmlrunner
with open('test-reports/result.xml', 'wb') as output:
unittest.main(testRunner=xmlrunner.XMLTestRunner(output=output),
failfast=False, buffer=False, catchbreak=False)
Alternative:
def tearDown(self)
super(sample_tests, self).tearDown()
with open('result.txt', 'w+') as output:
test_failed = self._outcomes.errors
output.write(test_failed)
Related
I am learning unittest and am trying to work on the following two .py scripts but when i run on terminal it shows "ran 0 tests". What am i doing wrong?
sanity.py
def firstname(name):
return name.title()
and then the second
sanitycheck.py
import unittest
import sanity
class TestingCap(unittest.TestCase):
def firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
Thank you!
By default, unittest assumes that tests in a unittest.TestCase are methods whose names begin with "test_"
Change your test method name to "test_firstone":
import unittest
import sanity
class TestingCap(unittest.TestCase):
def test_firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
python sanitycheck.py
F
======================================================================
FAIL: test_firstone (__main__.TestingCap)
----------------------------------------------------------------------
Traceback (most recent call last):
File "sanitycheck.py", line 9, in test_firstone
self.assertEqual(result,'apple')
AssertionError: 'Apple' != 'apple'
- Apple
? ^
+ apple
? ^
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
You may change the behavior of unittest if you like. Check out the documentation: https://docs.python.org/3/library/unittest.html
You should name the file with test.
example: test_sanity, sanity_test, testsanity.
Your function names should begin with test then an underscore like:
def test_firstone(self):
...
I have different unit_test modules which are included by unit test codes as you'll see:
import unittest
class TestTest(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def test_type(self):
self.assertTrue(False)
if __name__ == "__main__":
unittest.main()
I need to get the result of each module after running it and write it in the database. So I expect 3 different outcomes after running each module:
PASSED
NOTPASSED
WORKING
But the problem is after running the test module I get such a output.
F
======================================================================
FAIL: test_test (main.TestTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/~/mytest.py", line 12, in test_test
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
I don't know if there is something in unittest to give me a straight value as I expect?
I also tried this:
TestTest = unittest.TestLoader().loadTestsFromTestCase(TestTest)
suit = unittest.TestSuite([TestTest])
result = unittest.TextTestRunner(verbosity=2).run(suit)
print(result)
But it gives me this:
test_test (main.TestTest) ... FAIL
======================================================================
FAIL: test_test (__main__.TestTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/~/mytest.py", line 12, in test_test
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
Basically if you need in your test results more information just add the keyword msg documenting the test, for example:
self.assertTrue(False, msg = 'Host connection not stablished.')
How to log the debug or info comments to the generated log file from python unit test
import unittest
import logging
class Arming(unittest.TestCase):
def testCase1(self):
logging.info('I told you so')
actual = 3
expected = 3
self.assertEqual(actual,expected)
def testCase2(self):
actual = 3
expected = 4
testcase = "Test Case 2"
self.assertEqual(actual,expected)
if __name__ == '__main__':
log_file = 'Arming_Command.log'
f = open(log_file, "w")
runner = unittest.TextTestRunner(f,verbosity=2)
unittest.main(testRunner=runner)
f.close()
Getting log file
testCase1 (__main__.Arming) ... ok
testCase2 (__main__.Arming) ... FAIL
======================================================================
FAIL: testCase2 (__main__.Arming)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:/Users/hariom.singh/PycharmProjects/Connect_Pversion/venv/Python_Logging.py", line 15, in testCase2
self.assertEqual(actual,expected)
AssertionError: 3 != 4
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
logging.info('I told you so')
is missed in the log file
You failed to configure the logging module to use the specified file. You can use basicConfig for that. Code could become:
if __name__ == '__main__':
log_file = 'Arming_Command.log'
f = open(log_file, "w")
logging.basicConfig(stream=f, level=logging.INFO) # use the opened file for logging
runner = unittest.TextTestRunner(f,verbosity=2)
unittest.main(testRunner=runner)
f.close()
I am trying to create automatically test cases and run them through nosetest.
The test runs fine when run with commands:
python –m unittest test_auto1
python –m unittest test_auto1.TestAuto.test_two
However, if I try to run a test using nosetest, it fails in certain conditions:
1) nosetests test_auto1 - Fails with error
2) nosetests test_auto1:TestAuto – Works Fine
3) nosetests test_auto1:TestAuto.test_one – Fails with error
Here is the test code:
import unittest
def generator(test_class, a, b):
def test(self):
self.assertEqual(a, b)
return test
def add_test_methods(test_class):
#First element of list is variable "a", then variable "b", then name of test case that will be used as suffix.
test_list = [[2,3, 'one'], [5,5, 'two'], [0,0, 'three']]
for case in test_list:
test = generator(test_class, case[0], case[1])
setattr(test_class, "test_%s" % case[2], test)
class TestAuto(unittest.TestCase):
def setUp(self):
print 'Setup'
pass
def tearDown(self):
print 'TearDown'
pass
add_test_methods(TestAuto)
if __name__ == '__main__':
unittest.main(verbosity=1)
The error while running single test is:
======================================================================
ERROR: Failure: ValueError (no such test method in <class 'test_auto2.TestAuto'>: test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\nose-1.3.1-py2.7.egg\nose\loader.py", line 516, in makeTest
return self._makeTest(obj, parent)
File "C:\Python27\lib\site-packages\nose-1.3.1-py2.7.egg\nose\loader.py", line 570, in _makeTest
return parent(obj.__name__)
File "C:\Python27\lib\unittest\case.py", line 189, in __init__
(self.__class__, methodName))
ValueError: no such test method in <class 'test_auto2.TestAuto'>: test
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
The only issue that I see is that most likely add_test_methods is interpreted as test. When I mark it as nottest the same code above runs as expected:
from nose.tools import nottest
#nottest
def add_test_methods(test_class):
....
And now running it:
nosetests -v
test_one (auto.TestAuto) ... FAIL
test_three (auto.TestAuto) ... ok
test_two (auto.TestAuto) ... ok
======================================================================
FAIL: test_one (auto.TestAuto)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/opikalo/src/nose/auto.py", line 7, in test
self.assertEqual(a, b)
AssertionError: 2 != 3
-------------------- >> begin captured stdout << ---------------------
Setup
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)
I wanted to call setUpClass and tearDownClass so that setup and teardown would be performed only once for each test. However, it keeps failing for me when I call tearDownClass. I only want to record 1 test result, either PASS if both tests passed or FAIL if both tests failed. If I call only setup and tearDown then all works fine:
Calling setUpClass and tearDownClass:
#!/usr/bin/python
import datetime
import itertools
import logging
import os
import sys
import time
import unittest
LOGFILE = 'logfile.txt'
class MyTest(unittest.TestCase):
global testResult
testResult = None
#classmethod
def setUpClass(self):
## test result for DB Entry:
self.dbresult_dict = {
'SCRIPT' : 'MyTest.py',
'RESULT' : testResult,
}
def test1(self):
expected_number = 10
actual_number = 10
self.assertEqual(expected_number, actual_number)
def test2(self):
expected = True
actual = True
self.assertEqual(expected, actual)
def run(self, result=None):
self.testResult = result
unittest.TestCase.run(self, result)
#classmethod
def tearDownClass(self):
ok = self.testResult.wasSuccessful()
errors = self.testResult.errors
failures = self.testResult.failures
if ok:
self.dbresult_dict['RESULT'] = 'Pass'
else:
logging.info(' %d errors and %d failures',
len(errors), len(failures))
self.dbresult_dict['RESULT'] = 'Fail'
if __name__ == '__main__':
logger = logging.getLogger()
logger.addHandler(logging.FileHandler(LOGFILE, mode='a'))
stderr_file = open(LOGFILE, 'a')
runner = unittest.TextTestRunner(verbosity=2, stream=stderr_file, descriptions=True)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
sys.exit()
unittest.main(module=itersuite, exit=True)
stderr_file.close()
Error:
test1 (__main__.MyTest) ... ok
test2 (__main__.MyTest) ... ok
ERROR
===================================================================
ERROR: tearDownClass (__main__.MyTest)
-------------------------------------------------------------------
Traceback (most recent call last):
File "testTearDownClass.py", line 47, in tearDownClass
ok = self.testResult.wasSuccessful()
AttributeError: type object 'MyTest' has no attribute 'testResult'
----------------------------------------------------------------------
Ran 2 tests in 0.006s
FAILED (errors=1)
like #Marcin already pointed out, you're using the Unittest-Framework in a way it isn't intended.
To see if the tests are successful you check the given values with the expected, like you already did: assertEqual(given, expected). Unittest will then collect a summary of failed ones. you don't have to do this manually.
If you want to check that two tests need to be together successful or fail together, these should be combined in ONE Test, maybe as a additionally one, if the individual Tests need to be checked as well. This is nothing you want to save and load afterwards. The tests itself should be as stateless as possible.
When you say you want to run the SetUp and TearDown 'once per test', do you mean once per test-method or per test-run? This is different if you have more than one test-method inside your class:
setUp() Will be called before each test-method
tearDown() Will be called after each test-method
setUpClass() Will be called once per class (before the first test-method of this class)
tearDownClass() Will be called once per class (after the last test-method of this class)
Here's the official documentation
Here's a related answer
Change tearDownClass(self) to tearDownClass(cls) and setUpClass(self) to setUpClass(cls).