I have a Python class that uses the logging module to provide some debug output:
File someclass.py:
import logging
class SomeClass:
def do_stuff(self):
# do some things
logging.debug("I just did some stuff")
# do some more stuff
return True
I do unit testing on this class with the unittest module
File test_someclass.py
import unittest
from someclass import SomeClass
class SomeClassTests(unittest.TestCase):
def test_do_stuff(self):
obj = SomeClass()
self.assertFalse(obj.do_stuff())
def main():
unittest.main()
if __name__ == '__main__':
main()
What I want to do is show the debug messages while I am running the unit tests. I tried to set the verbosity to debug from the unit test module:
import logging
# ....
def main():
unittest.main()
logging.basicConfig(level=logging.DEBUG)
This didn't work. What would be the way to achieve this? Even better would be enabling DEBUG verbosity for only one test.
UPDATE:
Apparently it works when running it from the Python shell, but not in PyDev (it probably uses a different test runner).
If you want to output debug messages on failures only, using nose test runner would be the easiest way to go since nose captures stdout and print it out on failures. It works out of the box:
$ nosetests test.py
F
======================================================================
FAIL: test_stuff (test.SomeClass)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/../../test.py", line 7, in test_stuff
self.assertFalse(True)
AssertionError: True is not false
-------------------- >> begin captured stdout << ---------------------
I just did some stuff
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
where test.py contains:
from unittest import TestCase
class SomeClass(TestCase):
def test_stuff(self):
print "I just did some stuff"
self.assertFalse(True)
call unittest.main() from your main().
def main():
logging.basicConfig(level=logging.DEBUG)
unittest.main()
My output shows:
DEBUG:root:I just did some stuff
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
The basic example on unittests docs at: https://docs.python.org/2/library/unittest.html shows the simple way of calling and running a unit test from main.
Related
I have read several posts saying that if you call your unittests with unittest.main() that they should exit with a failure code if they fail. I call my unittests with the command: python -m unittest discover -v. I am using Python 3.6.6. An example unittest could look like this:
from server import app
import unittest
class ServerTestCase(unittest.TestCase):
"""
Unittesting for the server application.
"""
def setUp(self):
"""
Create a test client
"""
self.app = app.test_client()
self.app.testing = True
def tearDown(self):
pass
def test_root_endpoint(self):
"""
Testing the root endpoint
"""
result = self.app.get('/')
self.assertEqual(result.status_code, 200)
def test_health_endpoint(self):
"""
Testing the health endpoint
"""
result = self.app.get('/health')
assert b'UP' in result.data
if __name__ == '__main__':
unittest.main()
Even if one of my tests fails I get this when checking the exit code:
$ echo $?
0
What am I doing wrong?
Did you name the unittest files something like test_*.py? Because that's what discover is looking for. Otherwise no matter your tests, the result will be:
Ran 0 tests in 0.000s
OK
(BTW you don't have to if __name__ ... unittest.main() when using -m unittest discover)
I am trying to run some unittest tests via a Python Invoke library, but my poor knowledge of Python prevents me from doing so.
This is the sample code I have:
my_tests.py
import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
def main():
unittest.main()
if __name__ == '__main__':
main()
tasks.py
from invoke import task
#task
def tests(ctx):
main()
#task
def other_task(ctx):
print("This is fine")
def main():
import my_tests
import unittest
unittest.main(module='my_tests')
if __name__ == '__main__':
main()
And this is what I get:
C:\ittle_projects\invoke_unittest>python my_tests.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.002s
OK
C:\ittle_projects\invoke_unittest>python tasks.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
C:\ittle_projects\invoke_unittest>inv tests
E
======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
AttributeError: module 'my_tests' has no attribute 'tests'
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
The tests run fine from my_tests.py and from tasks.py, but when I use invoke stuff breaks.
How can I make it work or where should I look next?
The issue you are running into is that unittest.main() uses the command line arguments your program is called with to determine which tests to run. Since your program is being executed as inv tests, the first argument to your program is tests, so unittest is attempting to run tests for a module name tests which does not exist.
You can get around this by popping the last argument (tests) from the system arguments list:
import sys
from invoke import task
#task
def tests(ctx):
# Pop "tests" off the end of the system arguments
sys.argv.pop()
main()
#task
def other_task(ctx):
print("This is fine")
def main():
import my_tests
import unittest
unittest.main(module='my_tests')
if __name__ == '__main__':
main()
import unittest
import logging
FORMAT = '%(asctime)-15s %(message)s'
logging.basicConfig(format=FORMAT)
log = logging.getLogger(__name__)
log.root.setLevel(logging.DEBUG)
class FixturesTest(unittest.TestCase):
def setUp(self):
log.info('starting setup')
self.fixture = range(1, 10)
log.warn('Hello world')
def tearDown(self):
log.info('starting teardown')
del self.fixture
log.info('Goodbye world')
def test_pass(self):
log.info('in test pass')
self.assertEqual(self.fixture, range(1, 10))
log.error('Test world')
def test_fail(self):
log.info('in test fail')
log.error("let's fail a test")
assert(1==0)
def test_error(self):
log.info('in test error')
log.info("let's error a test")
d = []
d[1]
if __name__ == '__main__':
unittest.main()
running the above code using py.test, it reports the test_error test method as a test failure instead of test error. whereas running it using unittest shows the test_error as an error instead of failure, could you share any thoughts on this?
Update : Same code, nose also consider it as an error instead of failure for the test_error, is this an issue in py.test ??
Update : If you are not sure what I refer by this, if you run py.test test_filename.py, you get 1 pass, 2 failures, if you run python -m unittest test_filename.py or nosetest test_filename.py, it's 1 pass, 1 failure, 1 error.
pytest doesn't differentiate from test failures (due to an AssertionError) and test errors (any other exception) like unittest does. This is not a bug, but by design.
There's been some discussion in this thread to add support for this type of separation and it resulted in the pytest-finer-verdicts plugin.
Here is a sample script that checks for a precondition in the very first test case and my intention is to abort the script if the precondition is not met.
#!/usr/bin/python
import unittest
import sys
class TestMyScript(unittest.TestCase):
def test_000_prerequisite(self):
a = 0
if not a:
sys.exit()
return
def test_001_test1(self):
print "Inside test 1"
return
def test_002_test2(self):
print "Inside test 2"
return
if __name__ == "__main__":
unittest.main()
However, the sys.exit() only exits from the individual test case of the suite. It doesn't exit the whole script.
I understand that unittest treats each test case individually which is why any exceptions caused by any testcase are handled by the test runner and it proceeds to the next test case.
But I want the script to kill itself. How do I do that?
Here is the output of my script:
./temp.py
EInside test 1
.Inside test 2
.
======================================================================
ERROR: test_000_prerequisite (__main__.TestMyScript)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./temp.py", line 9, in test_000_prerequisite
sys.exit()
SystemExit
----------------------------------------------------------------------
Ran 3 tests in 0.000s
FAILED (errors=1)
My guess is that I have to mess around with TestRunner and kill the script if a test case returns some signal. But I am not sure how to really achieve it.
Here is the answer:
Stop testsuite if a testcase find an error
Here is the change I need to make when calling unittest.main(). The failfast keyword argument stops the script after the first failure.
if __name__ == "__main__":
unittest.main(failfast=True)
p.s. failfast keyword argument is only available for python 2.7+
p.p.s. you can also use failfast on unittest.TextTestRunner()
I am writing a set of test cases say Test1, Test2 in a test module.
Is there a way to skip Test1 or selectively execute only Test2 in that module using the command nose.main()?
My module contains,
test_module.py,
class Test1:
setUp(self):
print('setup')
tearDown(self):
print('teardown')
test(self):
print('test1')
class Test2:
setUp(self):
print('setup')
tearDown(self):
print('teardown')
test(self):
print('test2')
I run it from a different python file using,
if __name__ == '__main__':
nose.main('test_module')
The notion of skipping test and not running a test are different in the context of nose: skipped tests will be reported as skipped at the end of the test result. If you want to skip the test you would have to monkey patch your test module with decorators or do some other dark magic.
But if you want to just not run a test, you can do it the same way you would do it from the command line: using --exclude option. It takes a regular expression of the test you do not want to run. Something like this:
import sys
import nose
def test_number_one():
pass
def test_number_two():
pass
if __name__ == '__main__':
module_name = sys.modules[__name__].__file__
nose.main(argv=[sys.argv[0],
module_name,
'--exclude=two',
'-v'
])
Running the test will give you:
$ python stackoverflow.py
stackoverflow.test_number_one ... ok
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK