Here is a sample script that checks for a precondition in the very first test case and my intention is to abort the script if the precondition is not met.
#!/usr/bin/python
import unittest
import sys
class TestMyScript(unittest.TestCase):
def test_000_prerequisite(self):
a = 0
if not a:
sys.exit()
return
def test_001_test1(self):
print "Inside test 1"
return
def test_002_test2(self):
print "Inside test 2"
return
if __name__ == "__main__":
unittest.main()
However, the sys.exit() only exits from the individual test case of the suite. It doesn't exit the whole script.
I understand that unittest treats each test case individually which is why any exceptions caused by any testcase are handled by the test runner and it proceeds to the next test case.
But I want the script to kill itself. How do I do that?
Here is the output of my script:
./temp.py
EInside test 1
.Inside test 2
.
======================================================================
ERROR: test_000_prerequisite (__main__.TestMyScript)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./temp.py", line 9, in test_000_prerequisite
sys.exit()
SystemExit
----------------------------------------------------------------------
Ran 3 tests in 0.000s
FAILED (errors=1)
My guess is that I have to mess around with TestRunner and kill the script if a test case returns some signal. But I am not sure how to really achieve it.
Here is the answer:
Stop testsuite if a testcase find an error
Here is the change I need to make when calling unittest.main(). The failfast keyword argument stops the script after the first failure.
if __name__ == "__main__":
unittest.main(failfast=True)
p.s. failfast keyword argument is only available for python 2.7+
p.p.s. you can also use failfast on unittest.TextTestRunner()
Related
I was trying to dynamically create test functions when I notice a weird behavior that I do not understand. I have reduced the code to this for simple understanding. I am running it on python 3.6.7
Code:
import unittest
class MyTestCase(unittest.TestCase):
#classmethod
def setUpClass(cls):
print('SET UP')
def test_x(self):
print('hi')
x = MyTestCase # <-This causes the test function to run twice, Why???
if __name__ == "__main__":
print('test start')
unittest.main()
print('test end') # <- This does not get printed
Output:
test start
SET UP
hi
.hi
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
As you can see, the test function test_x is being run twice and the last print statement is not executed
So my questions are:
why does the test function run twice?
why is the print statement print('test end') not executed?
1. why does the test function run twice?
Because the unittest code finds two objects in the module's namespace that are unittest.TestCase classes, MyTestCase and x. It doesn't check that these are actually the same object. (Well, it has some awareness of this, in that it doesn't treat x as a new class to be tested, so it doesn't call setUpClass again; see "Class and Module Fixtures".)
2. why is the print statement print('test end') not executed?
By default, unittest.main() calls sys.exit() after running the tests. You can disable this by adding the argument exit=False:
if __name__ == "__main__":
print('test start')
unittest.main(exit=False)
print('test end')
I am trying to run some unittest tests via a Python Invoke library, but my poor knowledge of Python prevents me from doing so.
This is the sample code I have:
my_tests.py
import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
def main():
unittest.main()
if __name__ == '__main__':
main()
tasks.py
from invoke import task
#task
def tests(ctx):
main()
#task
def other_task(ctx):
print("This is fine")
def main():
import my_tests
import unittest
unittest.main(module='my_tests')
if __name__ == '__main__':
main()
And this is what I get:
C:\ittle_projects\invoke_unittest>python my_tests.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.002s
OK
C:\ittle_projects\invoke_unittest>python tasks.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
C:\ittle_projects\invoke_unittest>inv tests
E
======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
AttributeError: module 'my_tests' has no attribute 'tests'
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
The tests run fine from my_tests.py and from tasks.py, but when I use invoke stuff breaks.
How can I make it work or where should I look next?
The issue you are running into is that unittest.main() uses the command line arguments your program is called with to determine which tests to run. Since your program is being executed as inv tests, the first argument to your program is tests, so unittest is attempting to run tests for a module name tests which does not exist.
You can get around this by popping the last argument (tests) from the system arguments list:
import sys
from invoke import task
#task
def tests(ctx):
# Pop "tests" off the end of the system arguments
sys.argv.pop()
main()
#task
def other_task(ctx):
print("This is fine")
def main():
import my_tests
import unittest
unittest.main(module='my_tests')
if __name__ == '__main__':
main()
I have test cases in unittest framework. Have integrated them with teamcity.
Teamcity gives success even if my testcase fail. I Think, it is so since my unittest process exit with code 0 which is success for teamcity. I have integrated report, which show test failure. What needs to be done, so that teamcity also show failure if test fails.
code:
import unittest
class Login(unittest.TestCase):
def test_002_login_failure(self):
assert 1==2
You should prefer to use the unittest assertions so for example
self.assertEqual(1, 2)
Also you can use the teamcity-messages module to emit test messages the TeamCity will handle appropriately. Then you could change your main (if you have one) to something like the following
import teamcity
from teamcity.unittestpy import TeamcityTestRunner
if __name__ == '__main__':
if teamcity.is_running_under_teamcity():
runner = TeamcityTestRunner()
else:
runner = unittest.TextTestRunner()
unittest.main(testRunner=runner)
I am using the command:
nosetests test.py
When this is run only the first line of the description gets printed.
I want the whole description along with the test name. How do i do that?
test.py file
import unittests
class TestClass(unittest.TestCase):
def test_1(self):
"""this is a long description //
continues on second line which does not get printed """
some code;
self.assertTrue(True)
def test_2(self):
"""this is another or different long description //
continues on second line which does not get printed """
some code;
self.assertTrue(True)
if __name__ == '__main__':
unittest.main()
Unittest is documented as only showing the first line of the test method's docstring. But you could override the default implementation of shortDescription method to customise that behaviour:
import unittest
class TestClass(unittest.TestCase):
def shortDescription(self):
return self._testMethodDoc
def test_1(self):
"""this is a long description //
continues on second line """
self.assertTrue(True)
def test_2(self):
"""this is another or different long description //
continues on second line which also gets printed :) """
self.assertTrue(True)
if __name__ == '__main__':
unittest.main(verbosity=2)
Demo:
$ nosetests -v example.py
this is a long description //
continues on second line ... ok
this is another or different long description //
continues on second line which also gets printed :) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Someone wrote a nose plugin to fix this exact annoyance, maybe you'd be interested to use that. Here it is: https://bitbucket.org/natw/nose-description-fixer-plugin/
I have a Python class that uses the logging module to provide some debug output:
File someclass.py:
import logging
class SomeClass:
def do_stuff(self):
# do some things
logging.debug("I just did some stuff")
# do some more stuff
return True
I do unit testing on this class with the unittest module
File test_someclass.py
import unittest
from someclass import SomeClass
class SomeClassTests(unittest.TestCase):
def test_do_stuff(self):
obj = SomeClass()
self.assertFalse(obj.do_stuff())
def main():
unittest.main()
if __name__ == '__main__':
main()
What I want to do is show the debug messages while I am running the unit tests. I tried to set the verbosity to debug from the unit test module:
import logging
# ....
def main():
unittest.main()
logging.basicConfig(level=logging.DEBUG)
This didn't work. What would be the way to achieve this? Even better would be enabling DEBUG verbosity for only one test.
UPDATE:
Apparently it works when running it from the Python shell, but not in PyDev (it probably uses a different test runner).
If you want to output debug messages on failures only, using nose test runner would be the easiest way to go since nose captures stdout and print it out on failures. It works out of the box:
$ nosetests test.py
F
======================================================================
FAIL: test_stuff (test.SomeClass)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/../../test.py", line 7, in test_stuff
self.assertFalse(True)
AssertionError: True is not false
-------------------- >> begin captured stdout << ---------------------
I just did some stuff
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
where test.py contains:
from unittest import TestCase
class SomeClass(TestCase):
def test_stuff(self):
print "I just did some stuff"
self.assertFalse(True)
call unittest.main() from your main().
def main():
logging.basicConfig(level=logging.DEBUG)
unittest.main()
My output shows:
DEBUG:root:I just did some stuff
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
The basic example on unittests docs at: https://docs.python.org/2/library/unittest.html shows the simple way of calling and running a unit test from main.