I'm trying to run a TestCase in python 3.3.2 that has several test methods in it:
class ttt(unittest.TestCase):
def setUp(self):
...
def tearDown(self):
...
def test_test1(self):
...
def test_test2(self):
...
if __name__ == "__main__":
instance = ttt()
instance.run()
The documentation states the following:
Each instance of TestCase will run a single base method: the method
named methodName. However, the standard implementation of the default
methodName, runTest(), will run every method starting with test as an
individual test, and count successes and failures accordingly.
Therefore, in most uses of TestCase, you will neither change the
methodName nor reimplement the default runTest() method.
However, when I run the code I get the following:
'ttt' object has no attribute 'runTest'
I want to ask: Is this a bug? And if it's not why is there no runTest method? Am I doing something wrong?
When the unit test framework runs test cases, it creates an instance of the test class for each test.
I.e. to simulate what the unit test framework does you need to do:
if __name__ == "__main__":
for testname in ["test_test1", "test_test2"]:
instance = ttt(testname)
instance.run()
The correct way to run unit tests in a module is:
if __name__ == "__main__":
unittest.main()
... but I assume you know this already.
Regarding runTest: unittest.TestCase.__init__ signature and docstring is:
def __init__(self, methodName='runTest'):
"""Create an instance of the class that will use the named test
method when executed. Raises a ValueError if the instance does
not have a method with the specified name.
"""
Meaning that if you don't specify a test name in the constructor, the default is runTest.
Related
Like many people, I'm having issues with mock patching and getting the path right. Specifically, my code references another class in the same file and I'm having trouble patching that reference.
I have the following python file, package/engine/dataflows/flow.py:
class Flow:
def run(self, type):
if type == 'A':
method1()
elif type == 'B':
method2()
else:
backfill = Backfill()
backfill.run()
class Backfill(Flow):
def run(self):
...
And a test file package/tests/engine/dataflows/test_Flow.py
import unittest
from unittest.mock import Mock, patch
from engine.dataflows.flow import Flow
class MockFlow(Flow):
...
class TestFlowRun(unittest.TestCase):
def setUp(self):
self.flow = MockFlow()
def test_run_type_c(self):
with patch('engine.dataflows.flow.Backfill') as mock_backfill:
self.flow.run(type='C')
assert mock_backfill.run.call_count == 1
The patch works in that it doesn't throw an error when run with pytest, but the assertion is failing. I assume that is because the local reference to the Backfill class has essentially already been imported when MockFlow was initialized, but I have been unable to come up with a patching path that handles this.
The contents of flow.py include the Flow base class and a couple of child classes that implement different data flow patterns. They're co-located in the same file for ease of understanding and common dependencies.
The problem is that you are checking the run() function of a class, not an instantiation of that class. The mocked Backfill class will return an instance of the class via its constructor/init. That is the mock / object you will want to check.
with patch('engine.dataflows.flow.Backfill') as mock_backfill:
mocked_backfill_instance = mock_backfill.return_value
self.flow.run(type='C')
assert mocked_backfill_instance.run.call_count == 1
I'm trying to unit test a class which is derived from a base_class in an external module. In my dev/test environment I have not access to this external module, which means I have to somehow mock this base_class.
My test-code resides in a different file from the code I'm trying to test.
The problem can be summarized as follows:
my_class.py
import external_module
class MyClass(external_module.ExternalClass):
def test_method(self):
return "successful"
test_my_class.py
import sys
import unittest
from unittest.mock import MagicMock
sys.modules['external_module'] = MagicMock()
from my_class import MyClass
class TestMyClass(unittest.TestCase):
def test_first(self):
my_class = MyClass()
result = my_class.test_method()
self.assertEqual(result, "successful")
if __name__ == '__main__':
unittest.main()
Results
When running test_my_class.py the result are the following.
AssertionError: <MagicMock name='mock.ExternalClass.test_method()' id='140272215184664'> != 'successful'
Clearly since the external_module is mocked, even MyClass becomes an instance of a mock-object.
Similar posts
The problem is similar to as described in Python mock: mocking base class for inheritance, but has the difference that the base_class is from an external module.
Even How to mock a base class with python mock library show som similarities to my problem, though the solutions can not be directly applied.
Tries and failures
To get the import
import external_module
to work in my_class.py
sys.modules['external_module'] = MagicMock()
need to be set in test_my_class.py.
Though, this leads to that external_module.* becomes a Mock-instance.
You could create a helper module mocked_external_module, which can be imported from your tests and also contains a class base_class. Then you do the following in your test code:
import mocked_external_module
sys.modules['external_module'] = mocked_external_module
Plus, every method of your base_class that you need to mock you can create as a Mock or MagicMock.
I want to run a set of tests under different conditions and therefore share these tests between two different TestCase-derived classes.
One creates its own standalone session and the other attaches to an existing session and executes the same tests in there.
I guess I'm kind of abusing the unittest framework when testing an API with it but it doesn't feel like it's too far from its original purpose. Am I good so far?
I hacked a few things together and got it kind of running. But the way it's done, doesn't feel right and I'm afraid will cause problems sooner or later.
These are the problems I have with my solution:
When simply running the thing with PyCharm without limiting the tests, it attempts to run not only the intended StandaloneSessionTests and ExistingSessionTests but also GroupOfTests which is only the collection and has no session, i.e. execution context.
I can make it not run GroupOfTests by not deriving that one from TestCase but then PyCharm complains that it doesn't know about the assert...() functions. Rightly so, because GroupOfTest only gets indirect access to these functions at runtime when a derived class also inherits from TestCase. Coming from a C++ background, this feels like black magic and I don't think I should be doing this.
I tried passing the session creation classes to the constructor of GroupOfTests like this: __init__(self, session_class). But this causes problems when the unittest framework attempts to instantiate the tests: It doesn't know what to do with the additional __init__ parameter.
I learned about #classmethod, which seems to be a way to get around the "only one constructor" limitation of Python but I couldn't figure out a way to get it running.
I'm looking for a solution that lets me state something as straightforward as this:
suite = unittest.TestSuite()
suite.addTest(GroupOfTests(UseExistingSession))
suite.addTest(GroupOfTests(CreateStandaloneSession))
...
This is what I got so far:
#!/usr/bin/python3
import unittest
def existing_session():
return "existingsession"
def create_session():
return "123"
def close_session(session_id):
print("close session %s" % session_id)
return True
def do_thing(session_id):
return len(session_id)
class GroupOfTests(unittest.TestCase): # GroupOfTests gets executed, which makes no sense.
#class GroupOfTests: # use of assertGreaterThan() causes pycharm warning
session_id = None
def test_stuff(self):
the_thing = do_thing(self.session_id)
self.assertGreater(the_thing, 2)
# Original code contains many other tests, which must not be duplicated
class UseExistingSession(unittest.TestCase):
session_id = None
def setUp(self):
self.session_id = existing_session()
def tearDown(self):
pass # Nothing to do
class CreateStandaloneSession(unittest.TestCase):
session_id = None
def setUp(self):
self.session_id = create_session()
def tearDown(self):
close_session(self.session_id)
# unittest framework runs inherited test_stuff()
class StandaloneSessionTests(CreateStandaloneSession, GroupOfTests):
pass
# unittest framework runs inherited test_stuff()
class ExistingSessionTests(UseExistingSession, GroupOfTests):
pass
def main():
suite = unittest.TestSuite()
suite.addTest(StandaloneSessionTests)
suite.addTest(ExistingSessionTests)
runner = unittest.TextTestRunner()
runner.run(suite())
if __name__ == '__main__':
main()
I'm not sure if using pytest is an option for you but if so, here is an example which might do what you want.
import pytest
class Session:
def __init__(self, session_id=None):
self.id = session_id
existing_session = Session(999)
new_session = Session(111)
#pytest.fixture(params=[existing_session, new_session])
def session_fixture(request):
return request.param
class TestGroup:
def test_stuff(self, session_fixture):
print('(1) Test with session: {}'.format(session_fixture.id))
assert True
def test_more_stuff(self, session_fixture):
print('(2) Test with session: {}'.format(session_fixture.id))
assert True
Output:
$ pytest -v -s hmm.py
======================================================= test session starts ========================================================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /home/lettuce/Dropbox/Python/Python_3/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/lettuce/Dropbox/Python/Python_3, inifile:
collected 4 items
hmm.py::TestGroup::test_stuff[session_fixture0] (1) Test with session: 999
PASSED
hmm.py::TestGroup::test_stuff[session_fixture1] (1) Test with session: 111
PASSED
hmm.py::TestGroup::test_more_stuff[session_fixture0] (2) Test with session: 999
PASSED
hmm.py::TestGroup::test_more_stuff[session_fixture1] (2) Test with session: 111
PASSED
===================================================== 4 passed in 0.01 seconds =====================================================
If you are actually going to use pytest you will probably want follow the conventions for Python test discovery rather than using hmm.py as a filename though!
You can get pycharm to ignore these not existing function by creating abstract methods with raise NotImplementetError:
class GroupOfTests:
session_id = None
def test_stuff(self):
the_thing = do_thing(self.session_id)
self.assertGreater(the_thing, 2)
def assertGreater(self, a, b): # Pycharm treats these like abstract methods from the ABC module
raise NotImplementetError
This will let python believe this is an abstract class and will make pycharm raise errors if a subclass doesn't define these functions.
I have 92 tests and I want to make sure that no silent errors occurred during the calls.
Unfortunately error handing in OpenGL is not quite good. I want to test if glGetError() returns other then GL_NO_ERROR It is enough if I test it once per TestCase. It would be better if I could add an assert after every test methods. (I don't want to add it manually in 92 methods)
I made an example snippet that shows a solution that is not acceptable since the assert is done in the tearDownClass(cls) method and tearDownClass should not do any testing logic.
How can I add an extra assert after my tests?
The lines with comments show what I wan't to achieve.
import struct
import unittest
import ModernGL
class TestCase(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.ctx = ModernGL.create_standalone_context()
#classmethod
def tearDownClass(cls):
error = cls.ctx.error # Store error in a variable
cls.ctx.release() # Then release the context
cls.assertEqual(error, 'GL_NO_ERROR') # Check if there were errors before the release
def test_1(self):
...
def test_2(self):
...
def test_3(self):
...
if __name__ == '__main__':
unittest.main()
NOTE:
cls.ctx.error is a property (glGetError() as a string) and the possible values are:
"GL_NO_ERROR"
"GL_INVALID_ENUM"
"GL_INVALID_VALUE"
"GL_INVALID_OPERATION"
"GL_INVALID_FRAMEBUFFER_OPERATION"
"GL_OUT_OF_MEMORY"
"GL_STACK_UNDERFLOW"
"GL_STACK_OVERFLOW"
"GL_UNKNOWN_ERROR"
You could do the test in the tearDown (as opposed to tearDownClass) method, as this is a regular instance method:
class TestCase(unittest.TestCase):
def setUp(self):
self.ctx = ModernGL.create_standalone_context()
def tearDown(self):
error = self.ctx.error # Store error in a variable
self.ctx.release() # Then release the context
self.assertEqual(error, 'GL_NO_ERROR') # Check if there were errors before the release
How do I run multiple Classes in a single test suite in Python using unit testing?
If you want to run all of the tests from a specific list of test classes, rather than all of the tests from all of the test classes in a module, you can use a TestLoader's loadTestsFromTestCase method to get a TestSuite of tests for each class, and then create a single combined TestSuite from a list containing all of those suites that you can use with run:
import unittest
# Some tests
class TestClassA(unittest.TestCase):
def testOne(self):
# test code
pass
class TestClassB(unittest.TestCase):
def testOne(self):
# test code
pass
class TestClassC(unittest.TestCase):
def testOne(self):
# test code
pass
def run_some_tests():
# Run only the tests in the specified classes
test_classes_to_run = [TestClassA, TestClassC]
loader = unittest.TestLoader()
suites_list = []
for test_class in test_classes_to_run:
suite = loader.loadTestsFromTestCase(test_class)
suites_list.append(suite)
big_suite = unittest.TestSuite(suites_list)
runner = unittest.TextTestRunner()
results = runner.run(big_suite)
# ...
if __name__ == '__main__':
run_some_tests()
I'm a bit unsure at what you're asking here, but if you want to know how to test multiple classes in the same suite, usually you just create multiple testclasses in the same python file and run them together:
import unittest
class TestSomeClass(unittest.TestCase):
def testStuff(self):
# your testcode here
pass
class TestSomeOtherClass(unittest.TestCase):
def testOtherStuff(self):
# testcode of second class here
pass
if __name__ == '__main__':
unittest.main()
And run with for example:
python mytestsuite.py
Better examples can be found in the official documention.
If on the other hand you want to run multiple test files, as detailed in "How to organize python test in a way that I can run all tests in a single command?", then the other answer is probably better.
The unittest.TestLoader.loadTestsFromModule() method will discover and load all classes in the specified module. So you can just do this:
import unittest
import sys
class T1(unittest.TestCase):
def test_A(self):
pass
def test_B(self):
pass
class T2(unittest.TestCase):
def test_A(self):
pass
def test_B(self):
pass
if __name__ == "__main__":
suite = unittest.TestLoader().loadTestsFromModule( sys.modules[__name__] )
unittest.TextTestRunner(verbosity=3).run( suite )
Normally you would do in the following way (which adds only one class per suite):
# Add tests.
alltests = unittest.TestSuite()
alltests.addTest(unittest.makeSuite(Test1))
alltests.addTest(unittest.makeSuite(Test2))
If you'd like to have multiple classes per suite, you can use add these tests in the following way:
for name in testnames:
suite.addTest(tc_class(name, cargs=args))
Here is same example to run all classes per separate suite you can define your own make_suite method:
# Credits: http://codereview.stackexchange.com/a/88662/15346
def make_suite(tc_class):
testloader = unittest.TestLoader()
testnames = testloader.getTestCaseNames(tc_class)
suite = unittest.TestSuite()
for name in testnames:
suite.addTest(tc_class(name, cargs=args))
return suite
# Add all tests.
alltests = unittest.TestSuite()
for name, obj in inspect.getmembers(sys.modules[__name__]):
if inspect.isclass(obj) and name.startswith("FooTest"):
alltests.addTest(make_suite(obj))
result = unittest.TextTestRunner(verbosity=2).run(alltests)
If above doesn't suite, you can convert above example into method which could accept multiple classes.
I've found nose to be a good tool for this. It discovers all unit tests in a directory structure and executes them.