Develop Reference

Why is terminal providing me with zero results for this .py script?

I am learning unittest and am trying to work on the following two .py scripts but when i run on terminal it shows "ran 0 tests". What am i doing wrong?
sanity.py
def firstname(name):
return name.title()
and then the second
sanitycheck.py
import unittest
import sanity
class TestingCap(unittest.TestCase):
def firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
Thank you!

By default, unittest assumes that tests in a unittest.TestCase are methods whose names begin with "test_"
Change your test method name to "test_firstone":
import unittest
import sanity
class TestingCap(unittest.TestCase):
def test_firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
python sanitycheck.py
F
======================================================================
FAIL: test_firstone (__main__.TestingCap)
----------------------------------------------------------------------
Traceback (most recent call last):
File "sanitycheck.py", line 9, in test_firstone
self.assertEqual(result,'apple')
AssertionError: 'Apple' != 'apple'
- Apple
? ^
+ apple
? ^
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
You may change the behavior of unittest if you like. Check out the documentation: https://docs.python.org/3/library/unittest.html

You should name the file with test.
example: test_sanity, sanity_test, testsanity.

Your function names should begin with test then an underscore like:
def test_firstone(self):
...

How to retrieve the TestResult object from a python unittest?

I would like to get the test results from my unit tests and then log them. Having some trouble figuring out the best way to do it. Ideally I think I would like to get them from the tearDown method, and then log them there, so that each test is logging it's result as it finishes, but I can't seem to get it to work.
Here is some example code that you can run:
import unittest
class sample_tests(unittest.TestCase):
def test_it(self):
self.assertTrue(1==2)
def tearDown(self):
print("Get test results and log them here")
print(unittest.TestResult())
if __name__=='__main__':
#unittest.main()
suite = unittest.TestSuite()
suite.addTest(sample_tests("test_it"))
runner = unittest.TextTestRunner()
result = runner.run(suite)
print(result.failures)
When you run this you will get the following output:
Get test results and log them here
<unittest.result.TestResult run=0 errors=0 failures=0>
F
======================================================================
FAIL: test_it (__main__.sample_tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\sample.py", line 6, in test_it
self.assertTrue(1==2)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.005s
FAILED (failures=1)
[(<__main__.sample_tests testMethod=test_it>, 'Traceback (most recent call last):\n File ".\\sample.py", line 6, in test_it\n self.assertTrue(1==2)\nAssertionError: False is not true\n')]
PS C:\Users\cn187366\Documents\Python_Test\ETL_Test_Framework>
As you can see, the tear down method is not returning the expected results and I think it is because I'm not referencing the test runner which contains the TestResults object.
EDIT
I've found a solution here:
Getting Python's unittest results in a tearDown() method
Here is the actual code that does what I wanted:
def tearDown(self):
print("Get test results and log them here")
if hasattr(self,'_outcome'):
result = self.defaultTestResult()
self._feedErrorsToResult(result,self._outcome.errors)
print(result)

You can stream to a file with stream=file-name.log. For more detail, check the unittest.TextTestRunner class.
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(YourTestClass)
with open('test_result.out', 'w') as f:
unittest.TextTestRunner(stream=f, verbosity=2).run(suite)

This should work!
import xmlrunner
with open('test-reports/result.xml', 'wb') as output:
unittest.main(testRunner=xmlrunner.XMLTestRunner(output=output),
failfast=False, buffer=False, catchbreak=False)
Alternative:
def tearDown(self)
super(sample_tests, self).tearDown()
with open('result.txt', 'w+') as output:
test_failed = self._outcomes.errors
output.write(test_failed)

How do you show an error message when a test does not throw an expected exception?

I am new to Python. I wanted to test if my code threw an exception. I got the code from How do you test that a Python function throws an exception?
import mymod
import unittest
class MyTestCase(unittest.TestCase):
def test1(self):
self.assertRaises(SomeCoolException, mymod.myfunc, compulsory_argument)
Now, I also want to display a message if the exception is not thrown. How do I do that? The Python documentation does not mention it clearly. I added the message after "compulsory_argument" and it failed.
I tried the first answer with modifications and got an exception. What is my mistake here?
import unittest
def sayHelloTo(name):
print("Hello " + name)
class MyTestCase(unittest.TestCase):
def test1(self):
person = "John"
with self.assertRaises(Exception, "My insightful message"):
sayHelloTo(person)
Error:
Error
Traceback (most recent call last):
File "C:\tests\tester.py", line 9, in test1
with self.assertRaises(Exception, "My insightful message"):
AttributeError: __exit__

As of Python 3.3, assertRaises can be used as a context manager with a message:
import unittest
def sayHelloTo(name):
print("Hello " + name)
class MyTestCase(unittest.TestCase):
def test1(self):
person = "John"
with self.assertRaises(Exception, msg="My insightful message"):
sayHelloTo(person)
if __name__ == "__main__":
unittest.main()
It results in
Hello John
F
======================================================================
FAIL: test1 (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "r.py", line 10, in test1
sayHelloTo(person)
AssertionError: Exception not raised : My insightful message
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)

Now, I also want to display a message if the exception is not thrown. How do I do that ?
The general philosophy of unittest is for the tests to be silent when they succeed and only become verbose when they fail. Accordingly, the API provides a "msg" keyword argument for the unsuccessful case but does not offer an alternative for the successful case.
That said, another part of the philosophy works in your favor. In general, test cases internally raise an exception when a testcase fails. That means that if you want to display a message when there is a success, you just add another statement after the test:
with self.assertRaises(TypeError, msg='Oh no, I did not get a TypeError')
somecode()
logging.info('Yippee, we got a TypeError!') # Runs after a successful test

Creating automatically test cases

I am trying to create automatically test cases and run them through nosetest.
The test runs fine when run with commands:
python –m unittest test_auto1
python –m unittest test_auto1.TestAuto.test_two
However, if I try to run a test using nosetest, it fails in certain conditions:
1) nosetests test_auto1 - Fails with error
2) nosetests test_auto1:TestAuto – Works Fine
3) nosetests test_auto1:TestAuto.test_one – Fails with error
Here is the test code:
import unittest
def generator(test_class, a, b):
def test(self):
self.assertEqual(a, b)
return test
def add_test_methods(test_class):
#First element of list is variable "a", then variable "b", then name of test case that will be used as suffix.
test_list = [[2,3, 'one'], [5,5, 'two'], [0,0, 'three']]
for case in test_list:
test = generator(test_class, case[0], case[1])
setattr(test_class, "test_%s" % case[2], test)
class TestAuto(unittest.TestCase):
def setUp(self):
print 'Setup'
pass
def tearDown(self):
print 'TearDown'
pass
add_test_methods(TestAuto)
if __name__ == '__main__':
unittest.main(verbosity=1)
The error while running single test is:
======================================================================
ERROR: Failure: ValueError (no such test method in <class 'test_auto2.TestAuto'>: test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\nose-1.3.1-py2.7.egg\nose\loader.py", line 516, in makeTest
return self._makeTest(obj, parent)
File "C:\Python27\lib\site-packages\nose-1.3.1-py2.7.egg\nose\loader.py", line 570, in _makeTest
return parent(obj.__name__)
File "C:\Python27\lib\unittest\case.py", line 189, in __init__
(self.__class__, methodName))
ValueError: no such test method in <class 'test_auto2.TestAuto'>: test
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)

The only issue that I see is that most likely add_test_methods is interpreted as test. When I mark it as nottest the same code above runs as expected:
from nose.tools import nottest
#nottest
def add_test_methods(test_class):
....
And now running it:
nosetests -v
test_one (auto.TestAuto) ... FAIL
test_three (auto.TestAuto) ... ok
test_two (auto.TestAuto) ... ok
======================================================================
FAIL: test_one (auto.TestAuto)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/opikalo/src/nose/auto.py", line 7, in test
self.assertEqual(a, b)
AssertionError: 2 != 3
-------------------- >> begin captured stdout << ---------------------
Setup
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)

How do I signal test error (not failure) from python unittest

I have a test case with a helper method assertContains(super, sub). The sub arguments are a hard-coded part of the test cases. In case they're malformed, I would like my test case to abort with an error.
How do I do that? I have tried
def assertContains(super, sub):
if isinstance(super, foo): ...
elif isinstance(super, bar): ...
else: assert False, repr(sub)
However, this turns the test into a failure rather than an error.
I could raise some other exception (e.g. ValueError), but I want to explicitly state that I'm declaring the test case to be in error. I could do things like ErrorInTest = ValueError and then raise ErrorInTest(repr(sub)), but it feels kinda' icky. I feel there should be a batteries-included way of doing this, but reading the friendly manual didn't suggest anything to me.

There is an assertRaises() for aspects in class TestCase in which you want to ensure an error is raised by the to-be-tested code.
If you want to raise an error and abort testing that unit at this point (and continue with the next unit test), just raise an uncaught exception; the unit test module will catch it:
raise NotImplementedError("malformed sub: %r" % (sub,))
I don't think that there is any other API aspect available besides raising errors directly to state that a unit test case results in an error.
class PassingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
class FailingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(False)
class ErrorTest(unittest.TestCase):
def runTest(self):
raise NotImplementedError("error")
class PassingTest2(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
results in:
EF..
======================================================================
ERROR: runTest (__main__.ErrorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 15, in runTest
raise NotImplementedError("error")
NotImplementedError: error
======================================================================
FAIL: runTest (__main__.FailingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 11, in runTest
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 4 tests in 0.002s
FAILED (failures=1, errors=1)