Develop Reference

Python unittest output only on test failures

Is there a way to write (or run) a set of Python unittest tests so that there is no output except when there is a test failure?
For example, if tests/mytests.py contains some unittest tests, then running python3 test/mytests.py will only output to stdout (or stderr) if there is a test failure.

Yes there is. I was able to combine the techniques in the answers to these two questions to get it to work:
Python, unittest: Can one make the TestRunner completely quiet?
argparse and unittest python
You can uncomment the test_should_fail() test to verify what happens when a test fails.
# mymodule.py
def myfunc():
return True
import unittest
import os
class TestMyFunc(unittest.TestCase):
def test_myfunc(self):
self.assertEqual(myfunc(), True)
# def test_should_fail(self):
# self.assertEqual(True, False)
if __name__ == '__main__':
alltests = unittest.TestLoader().loadTestsFromTestCase(TestMyFunc)
runner = unittest.TextTestRunner(stream=open(os.devnull, 'w'))
result = runner.run(alltests)
if len(result.failures) or len(result.errors):
print('There were failures or errors.')

Running python unittests from TestSuite in PyCharm will not order the tests alphabetically, but running the same file in terminal will order them

I am working on writing unit tests, but they needed to run as I added them to the TestSuite. So my problem is when I run the testrunner.py from the PyCharm I get what I need, but if I run the same file from the terminal the tests will be sorted alphabetically, and that is bad for me.
For example: I have 2 tests, test_dummy_01.py and test_dummy_01 in the "tests/" folder of my Framework. Also I created the testrunner.py file where I add the TestDummy02 class first and after the TestDummy01 class. So when I run the the testrunner.py from PyCharm the test_dummy_02.py test will run first and the test_dummy_01.py by second which is good for me. Now I need the same behavior when I run the same file in the terminal, but it will always run the test_dummy_01.py file first. So please if anyone can help me?
Folder hierarchy of my files:
Project/
tests/
test_dummy_01.py
test_dummy_02.py
testrunner.py
The test_dummy_01.py test: - the test_dummy_02.py file is same as this
class TestDummy01(BaseTestCase):
def setUp(self):
super(TestDummy, self).setUp()
def test_dummy_01(self):
driver = self.driver
login = LoginPage(driver)
login.enter_username("Something")
def tearDown(self):
super(TestDummy, self).tearDown()
The testrunner.py file where I added the order
import sys
import unittest
from tests.test_dummy_01 import TestDummy01
from tests.test_dummy_02 import TestDummy02
def suite():
suite_x = unittest.TestSuite()
suite_x.addTest(unittest.makeSuite(TestDummy02)) # In PyCharm this test is runned first, but in Teminal this will run by second
suite_x.addTest(unittest.makeSuite(TestDummy01))
return suite_x
def run():
result = unittest.TextTestRunner(verbosity=2).run(suite())
if not result.wasSuccessful():
sys.exit(1)
if __name__ == '__main__':
run()
PyCharm terminal

Tests from other files are run with unittest.main()

I am using spyder(3.3.4) to run some python(3.7) unit tests. I have two different scripts that use unittest and unittest.main() open in spyder: test_1.py and test_2.py. Both are saved in the same folder.
If I run test_1.py before ever running test_2.py, I get the expected results; only the tests in test_1.py are run.
If I then run test_2.py, the tests in test_2.py are run, but then the tests in test_1.py are run also. If I restart the Ipython kernel in between running test_1.py and running test_2.py, then only test_2.py runs, as expected. Weirdest of all, if I close test_1.py after running it, the output of test_1.py is still printed when running test_2.py.
Why is this happening?
I am guessing this has do to do with how the __name__ variable is saved in the IPython console or how unittest.main() searches for tests?
Here's the code for my two test scripts:
test_1.py:
import unittest
class TestStuff(unittest.TestCase):
def test_1(self):
print('test 1')
pass
def test_2(self):
print('test 2')
pass
if __name__ == '__main__':
unittest.main()
and test_2.py:
import unittest
class TestOtherStuff(unittest.TestCase):
def test_this(self):
print('this')
pass
def test_that(self):
print('that')
pass
if __name__ == '__main__':
unittest.main()
Thanks!

What function does python -m unittest execute?

In python 2.6, I've been reading through the unittest documentation. But I still haven't found this answer.
What function does pyton -m unittest execute?
For example, how would I modify this code such that just executing python -m unittest would detect it and run the test?
import random
import unittest
class TestSequenceFunctions(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def test_shuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
def test_choice(self):
element = random.choice(self.seq)
self.assertTrue(element in self.seq)
def test_sample(self):
self.assertRaises(ValueError, random.sample, self.seq, 20)
for element in random.sample(self.seq, 5):
self.assertTrue(element in self.seq)
if __name__ == '__main__':
unittest.main()
EDIT:
Note this is just a example, I'm actually trying to get it to detect and run multiple tests as a suite, here is my starting point - but python -m unittest doesn't detect it nor does python -m unittest discovery work with it. I have to call python discovery.py to execute it.
discovery.py:
import os
import unittest
def makeSuite():
"""Function stores all the modules to be tested"""
modules_to_test = []
test_dir = os.listdir('.')
for test in test_dir:
if test.startswith('test') and test.endswith('.py'):
modules_to_test.append(test.rstrip('.py'))
all_tests = unittest.TestSuite()
for module in map(__import__, modules_to_test):
module.testvars = ["variables you want to pass through"]
all_tests.addTest(unittest.findTestCases(module))
return all_tests
if __name__ == '__main__':
unittest.main(defaultTest='makeSuite')

python -m something executes the module something as a script. i.e. from python --help:
-m mod : run library module as a script (terminates option list)
The unittest module can be run as a script -- And the arguments that you pass to it determine which files it tests. The commandline interface is also documented in the language reference.

Problem's running unittest test suite OO

I have a test suite to perform smoke tests. I have all my script stored in various classes but when I try and run the test suite I can't seem to get it working if it is in a class. The code is below: (a class to call the tests)
from alltests import SmokeTests
class CallTests(SmokeTests):
def integration(self):
self.suite()
if __name__ == '__main__':
run = CallTests()
run.integration()
And the test suite:
class SmokeTests():
def suite(self): #Function stores all the modules to be tested
modules_to_test = ('external_sanity', 'internal_sanity')
alltests = unittest.TestSuite()
for module in map(__import__, modules_to_test):
alltests.addTest(unittest.findTestCases(module))
return alltests
if __name__ == '__main__':
unittest.main(defaultTest='suite')
This output's an error:
Attribute Error: 'module' object has no attribute 'suite'
So I can see how to call a normal function defined but I'm finding it difficult calling in the suite. In one of the tests the suite is set up like so:
class InternalSanityTestSuite(unittest.TestSuite):
# Tests to be tested by test suite
def makeInternalSanityTestSuite():
suite = unittest.TestSuite()
suite.addTest(TestInternalSanity("BasicInternalSanity"))
suite.addTest(TestInternalSanity("VerifyInternalSanityTestFail"))
return suite
def suite():
return unittest.makeSuite(TestInternalSanity)
If I have someSuite() inside the class SmokeTests python cannot find the attribute suite but if I remove the class it work's. I run this as a script and call in variables into the tests. I do not want to have to run the tests by os.system('python tests.py'). I was hoping to call the tests through the class I have like any other function
Can anyone help me with getting this running?
Thanks for any help in advance.

I know this is not the answer, but I'd suggest using library that can use test discovery, like nose or unittest capability from Python 2.7+.
Possibility to do
nosetests module.submodule
or
nosetests module.submodule:TestCase.test_method
is priceless :)

This can't work:
class SmokeTests():
def suite(self): #Function stores all the modules to be tested
modules_to_test = ('external_sanity', 'internal_sanity')
alltests = unittest.TestSuite()
for module in map(__import__, modules_to_test):
alltests.addTest(unittest.findTestCases(module))
return alltests
if __name__ == '__main__':
unittest.main(defaultTest='suite')
This output's an error: Attribute Error: 'module' object has no attribute 'suite'.
Your suite the value of the SmokeTests().suite() method. Note a variable named suite, since you have no such variable.
It's easier to use a simple function for your suite.
def someSuite():
modules_to_test
...
return alltests
if __name__ == "__main__":
unittest.main( defaultTest= someSuite() )
Something like that would be closer to correct.