Develop Reference

Why is terminal providing me with zero results for this .py script?

I am learning unittest and am trying to work on the following two .py scripts but when i run on terminal it shows "ran 0 tests". What am i doing wrong?
sanity.py
def firstname(name):
return name.title()
and then the second
sanitycheck.py
import unittest
import sanity
class TestingCap(unittest.TestCase):
def firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
Thank you!

By default, unittest assumes that tests in a unittest.TestCase are methods whose names begin with "test_"
Change your test method name to "test_firstone":
import unittest
import sanity
class TestingCap(unittest.TestCase):
def test_firstone(self):
word = 'apple'
result = sanity.firstname(word)
self.assertEqual(result,'apple')
if __name__ == '__main__':
unittest.main()
python sanitycheck.py
F
======================================================================
FAIL: test_firstone (__main__.TestingCap)
----------------------------------------------------------------------
Traceback (most recent call last):
File "sanitycheck.py", line 9, in test_firstone
self.assertEqual(result,'apple')
AssertionError: 'Apple' != 'apple'
- Apple
? ^
+ apple
? ^
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
You may change the behavior of unittest if you like. Check out the documentation: https://docs.python.org/3/library/unittest.html

You should name the file with test.
example: test_sanity, sanity_test, testsanity.

Your function names should begin with test then an underscore like:
def test_firstone(self):
...

How to get a single outcome from each python unit-test module

I have different unit_test modules which are included by unit test codes as you'll see:
import unittest
class TestTest(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def test_type(self):
self.assertTrue(False)
if __name__ == "__main__":
unittest.main()
I need to get the result of each module after running it and write it in the database. So I expect 3 different outcomes after running each module:
PASSED
NOTPASSED
WORKING
But the problem is after running the test module I get such a output.
F
======================================================================
FAIL: test_test (main.TestTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/~/mytest.py", line 12, in test_test
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
I don't know if there is something in unittest to give me a straight value as I expect?
I also tried this:
TestTest = unittest.TestLoader().loadTestsFromTestCase(TestTest)
suit = unittest.TestSuite([TestTest])
result = unittest.TextTestRunner(verbosity=2).run(suit)
print(result)
But it gives me this:
test_test (main.TestTest) ... FAIL
======================================================================
FAIL: test_test (__main__.TestTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/~/mytest.py", line 12, in test_test
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)

Basically if you need in your test results more information just add the keyword msg documenting the test, for example:
self.assertTrue(False, msg = 'Host connection not stablished.')

How to retrieve the TestResult object from a python unittest?

I would like to get the test results from my unit tests and then log them. Having some trouble figuring out the best way to do it. Ideally I think I would like to get them from the tearDown method, and then log them there, so that each test is logging it's result as it finishes, but I can't seem to get it to work.
Here is some example code that you can run:
import unittest
class sample_tests(unittest.TestCase):
def test_it(self):
self.assertTrue(1==2)
def tearDown(self):
print("Get test results and log them here")
print(unittest.TestResult())
if __name__=='__main__':
#unittest.main()
suite = unittest.TestSuite()
suite.addTest(sample_tests("test_it"))
runner = unittest.TextTestRunner()
result = runner.run(suite)
print(result.failures)
When you run this you will get the following output:
Get test results and log them here
<unittest.result.TestResult run=0 errors=0 failures=0>
F
======================================================================
FAIL: test_it (__main__.sample_tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\sample.py", line 6, in test_it
self.assertTrue(1==2)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.005s
FAILED (failures=1)
[(<__main__.sample_tests testMethod=test_it>, 'Traceback (most recent call last):\n File ".\\sample.py", line 6, in test_it\n self.assertTrue(1==2)\nAssertionError: False is not true\n')]
PS C:\Users\cn187366\Documents\Python_Test\ETL_Test_Framework>
As you can see, the tear down method is not returning the expected results and I think it is because I'm not referencing the test runner which contains the TestResults object.
EDIT
I've found a solution here:
Getting Python's unittest results in a tearDown() method
Here is the actual code that does what I wanted:
def tearDown(self):
print("Get test results and log them here")
if hasattr(self,'_outcome'):
result = self.defaultTestResult()
self._feedErrorsToResult(result,self._outcome.errors)
print(result)

You can stream to a file with stream=file-name.log. For more detail, check the unittest.TextTestRunner class.
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(YourTestClass)
with open('test_result.out', 'w') as f:
unittest.TextTestRunner(stream=f, verbosity=2).run(suite)

This should work!
import xmlrunner
with open('test-reports/result.xml', 'wb') as output:
unittest.main(testRunner=xmlrunner.XMLTestRunner(output=output),
failfast=False, buffer=False, catchbreak=False)
Alternative:
def tearDown(self)
super(sample_tests, self).tearDown()
with open('result.txt', 'w+') as output:
test_failed = self._outcomes.errors
output.write(test_failed)

How do I signal test error (not failure) from python unittest

I have a test case with a helper method assertContains(super, sub). The sub arguments are a hard-coded part of the test cases. In case they're malformed, I would like my test case to abort with an error.
How do I do that? I have tried
def assertContains(super, sub):
if isinstance(super, foo): ...
elif isinstance(super, bar): ...
else: assert False, repr(sub)
However, this turns the test into a failure rather than an error.
I could raise some other exception (e.g. ValueError), but I want to explicitly state that I'm declaring the test case to be in error. I could do things like ErrorInTest = ValueError and then raise ErrorInTest(repr(sub)), but it feels kinda' icky. I feel there should be a batteries-included way of doing this, but reading the friendly manual didn't suggest anything to me.

There is an assertRaises() for aspects in class TestCase in which you want to ensure an error is raised by the to-be-tested code.
If you want to raise an error and abort testing that unit at this point (and continue with the next unit test), just raise an uncaught exception; the unit test module will catch it:
raise NotImplementedError("malformed sub: %r" % (sub,))
I don't think that there is any other API aspect available besides raising errors directly to state that a unit test case results in an error.
class PassingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
class FailingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(False)
class ErrorTest(unittest.TestCase):
def runTest(self):
raise NotImplementedError("error")
class PassingTest2(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
results in:
EF..
======================================================================
ERROR: runTest (__main__.ErrorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 15, in runTest
raise NotImplementedError("error")
NotImplementedError: error
======================================================================
FAIL: runTest (__main__.FailingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 11, in runTest
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 4 tests in 0.002s
FAILED (failures=1, errors=1)

Nose: generator for TestCase-based classes

I want to create a generator for variations of a TestCase-derived class.
What I tried is this:
import unittest
def create_class(param):
class Test(unittest.TestCase):
def setUp(self):
pass
def test_fail(self):
assert False
return Test
def test_basic():
for i in range(5):
yield create_class(i)
What I get is this:
======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.3/site-packages/nose/case.py", line 268, in setUp
try_run(self.test, names)
File "/usr/lib/python3.3/site-packages/nose/util.py", line 478, in try_run
return func()
TypeError: setUp() missing 1 required positional argument: 'self'
Yielding instances instead of classes (yield create_class(i)()) leaves me with this error:
======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/usr/lib/python3.3/unittest/case.py", line 492, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.3/unittest/case.py", line 423, in run
testMethod = getattr(self, self._testMethodName)
AttributeError: 'Test' object has no attribute 'runTest'
Any ideas?

When instantiating a TestCase you should pass the method name of the test:
yield create_class(i)('test_fail')
Otherwise the name defaults to runTest(and thus the last error you got).
Also note that there is a strange interaction between test generators and TestCase. With the following code:
import unittest
def create_class(param):
class Test(unittest.TestCase):
def setUp(self):
pass
def test_fail(self):
print('executed')
assert False
print('after assert')
return Test
def test_basic():
for i in range(5):
yield create_class(i)('test_fail')
I obtain this output:
$ nosetests -s
executed
.executed
.executed
.executed
.executed
.
----------------------------------------------------------------------
Ran 5 tests in 0.004s
OK
As you can see the test does not fail, even though the assert works. This is probably due to the fact that TestCase handles the AssertionError but nose does not expect this to be handled and thus it cannot see that the test failed.
This can be seen from the documentation of TestCase.run:
Run the test, collecting the result into the test result object passed as result. If result is omitted or None, a temporary result
object is created (by calling the defaultTestResult() method) and
used. The result object is not returned to run()‘s caller.
The same effect may be had by simply calling the TestCase instance.
So, nose doesn't see that the objected yielded by the generator is a TestCase which should be handled in a special manner, it simply expects a callable. The TestCase is run, but the result is put into a temporary object that is lost, and this eats all test failures that happen inside the tests. Hence yielding TestCasees simply doesn't work.

I have run the codes you provides. I received no error. The version I use is python2.7. System is ubuntu12.10. Maybe you need to check with python2.7.