I'm fairly new to testing in Python and I've come upon some weird behavior which I can't understand.
I have mocked class MockedClass and I'm using it in two test classes like this (both classes are in the same file):
First class:
class TestClass1(unittest.TestCase):
def setUp(self):
self.mocked_object_1 = MockedClass()
Second class:
class TestClass2(unittest.TestCase):
def setUp(self):
self.mocked_object_2 = MockedClass()
Mocked objects are passed as arguments to tested functions. What happens now is that when these two tests are run separately (in different files), all tests in those classes are successful, but when they are both run in the same file, all test cases in TestClass2 fail.
I managed to solve this (somehow) by declaring global temp_mocked_object and doing this:
class TestClass1(unittest.TestCase):
def setUp(self):
temp_mocked_object = self.mocked_object_1
self.mocked_object_1 = MockedClass()
def tearDown(self):
self.mocked_object_1 = temp_mocked_object
But I don't see why would this problem even occur, much less do I understand why this solution worked. What confuses me the most is that second test fails despite different mocked object names.
Mocked class is structured this way:
class MockedClass(someScript.OriginalClass):
def __init__(self):
# some unimportant objects are initialized here
I'm working in older Python version (2.7), but I think that doesn't matter.
So I would greatly appreciate some kind of explanation about what's going on here, since I couldn't find the answer elsewhere.
Related
I have a class and I need to test it
class ToBeTested():
def my_method():
pass
I have designed some unittests in python.
import unittest
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
# prepare some data
# run a method
# collect results
def test_one_assumption_about_results(self):
#self.assertEqual(...)
def test_another_assumption_about_results(self):
#self.assertEqual(...)
if __name__ == '__main__':
unittest.main()
I have designed this code looking at the examples.
Now I do not understand how to interface it the tesing class MyFirstUnitTest with the classe to be tested ToBeTested?
By inheritance?
class MyFirstUnitTest(unittest.TestCase,ToBeTested):
######
By creating a object of class ToBeTested as static member of
MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
mytestobj=ToBeTested()
def setUp(self):
By creating a object of class ToBeTested within each of the test
cases of MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
###
def test_one():
mytestobj=ToBeTested()
def test_two():
mytestobj=ToBeTested()
please suggest alternatives and why you would prefer any of them
There is a drawback to using inheritance and static member instead of creating a new object for every test: If tests change the state of the object, then the tests influence each other with their side effects. This gets even worse if the order of the test method calls are not guaranteed and changes (say, you rename a test and suddenly a different test case fails - these things have happened before!)
Additionally, having your test case inherit from the SUT (subject under test) means you only get to test one constructor.
I don't see any advantage to having the testcase inherit from the SUT, especially in python where there are no private members. If for some reason inheritance is necessary for the test (which sometimes IS the case), then having a seperate TestAdapter class that inherits and is instantiated by the test is the best practice.
There are other questions with answers which go more in depth:
https://softwareengineering.stackexchange.com/questions/366425/making-a-test-fixture-inherit-from-the-sut-class
https://softwareengineering.stackexchange.com/questions/154144/how-to-define-implementation-details
You can create an instance of your class(es) in your setup if it will not have to be reinstantiated.
And if you have to create a new instance in every test then just create a new instance in every test.
I've searched for hours. Can't find anyone even trying to do this. Hmmm.
I believe I have to override a single method within a class instance. I do not mean patch(return_value=). I need to make the method in question do something involving self.
I'll try to break it down. Liberally paraphrasing and including one of the many things I tried, which doesn't work ...
class SetupClass(object):
def set_some_stuff(self):
data_list = functon_cannot_be_run_on_test_platform()
self.something = data_list[0]
self.something_else = data_list[1]
class UUT(object):
self.config = SetupClass()
assert self.config.something == 'foo'
class UnitTests(TestCase):
#patch('SetupClass')
def test_UUT(self, mock1):
def WedgeClass(SetupClass):
def set_some_stuff(self):
self.something = 'foo'
pass # I'm a Python newbie, in too deep
wedge_class = WedgeClass()
mock1.return_value = wedge_class # doesn't work. context errors
uut = UUT() # <-- would crash here, because assert above
Assume that I cannot make changes to UUT or SetupClass.
Testing cannot even get off the ground because the assertion will fail, due to, SetupClass.functon_cannot_be_run_on_test_platform(). Note that simply mocking SetupClass.functon_cannot_be_run_on_test_platform will not solve the problem, because reasons.
ATM, I figure the only way to get around this mess is to somehow override SetupClass.set_some_stuff. I cannot simply mock the entire class, because UUT relies heavily on its other functionality as well. I need everything to work as is, except this one method and I need that method to be able to access, self in the same context as originally intend.
I tried various things involving subclassing and mock.return_value etc. I'd rather not recall the pain that caused. :p
My kingdom for test-driven code in the first place! This code contains a convolution of co-dependencies. :-/
Apart from the multiple errors in the example code I wrote up off the top of my head at the cafe ...
The problem was (mostly) that I was using return_value instead of side_effect to 'replace' the patched class with my own subclass.
My need, re-stated perhaps more clearly now, is to override a single method, set_some_stuff, in a class within the unit under test (UUT) but without mocking the class.
The method issues several 'self.foo = bar' statements, which I want to change for testing purposes. Thus, mocking the method's (unused) return value is not enough ... and patch.object seems to lose class context, "'self' unknown" or the like.
Finally, here is the working code, doing what I need ...
import unittest
import mock
class SetupClass(object):
def set_some_stuff(self):
data_list = ['data not available', 'on test_platform'] # CRASH!
self.something = data_list[0]
self.something_else = data_list[1]
class UUT:
def __init__(self):
self.config = SetupClass()
self.config.set_some_stuff()
assert self.config.something == 'foo' # <-- used to crash before here ...
self.got_here = True # ... but now the override method from
# WedgeClass is being used! :=)
"""
Subclass the original class, overriding just the method in question. Then set
the subclass as the side_effect of the patched original, effectively replacing it.
"""
class WedgeClass(SetupClass):
def set_some_stuff(self):
self.something = 'foo'
pass # I'm a Python newbie, in too deep
class UnitTests(unittest.TestCase):
#mock.patch(__module__+'.SetupClass')
def test_UUT(self, mock1):
wedge_class = WedgeClass()
mock1.side_effect = WedgeClass # <--- Ureka! 'side_effect' not 'return_value' was the ley!
uut = UUT()
self.assertTrue(uut.got_here)
I'm working on a module using sockets with hundreds of test cases. Which is nice. Except now I need to test all of the cases with and without socket.setdefaulttimeout( 60 )... Please don't tell me cut and paste all the tests and set/remove a default timeout in setup/teardown.
Honestly, I get that having each test case laid out on it's own is good practice, but i also don't like to repeat myself. This is really just testing in a different context not different tests.
i see that unittest supports module level setup/teardown fixtures, but it isn't obvious to me how to convert my one test module into testing itself twice with two different setups.
any help would be much appreciated.
you could do something like this:
class TestCommon(unittest.TestCase):
def method_one(self):
# code for your first test
pass
def method_two(self):
# code for your second test
pass
class TestWithSetupA(TestCommon):
def SetUp(self):
# setup for context A
do_setup_a_stuff()
def test_method_one(self):
self.method_one()
def test_method_two(self):
self.method_two()
class TestWithSetupB(TestCommon):
def SetUp(self):
# setup for context B
do_setup_b_stuff()
def test_method_one(self):
self.method_one()
def test_method_two(self):
self.method_two()
The other answers on this question are valid in as much as they make it possible to actually perform the tests under multiple environments, but in playing around with the options I think I like a more self contained approach. I'm using suites and results to organize and display results of tests. In order to run one tests with two environments rather than two tests I took this approach - create a TestSuite subclass.
class FixtureSuite(unittest.TestSuite):
def run(self, result, debug=False):
socket.setdefaulttimeout(30)
super().run(result, debug)
socket.setdefaulttimeout(None)
...
suite1 = unittest.TestSuite(testCases)
suite2 = FixtureSuite(testCases)
fullSuite = unittest.TestSuite([suite1,suite2])
unittest.TextTestRunner(verbosity=2).run(fullSuite)
I would do it like this:
Make all of your tests derive from your own TestCase class, let's call it SynapticTestCase.
In SynapticTestCase.setUp(), examine an environment variable to determine whether to set the socket timeout or not.
Run your entire test suite twice, once with the environment variable set one way, then again with it set the other way.
Write a small shell script to invoke the test suite both ways.
If your code does not call socket.setdefaulttimeout then you can run tests the following way:
import socket
socket.setdeaulttimeout(60)
old_setdefaulttimeout, socket.setdefaulttimeout = socket.setdefaulttimeout, None
unittest.main()
socket.setdefaulttimeout = old_setdefaulttimeout
It is a hack, but it can work
You could also inherit and rerun the original suite, but overwrite the whole setUp or a part of it:
class TestOriginal(TestCommon):
def SetUp(self):
# common setUp here
self.current_setUp()
def current_setUp(self):
# your first setUp
pass
def test_one(self):
# your test
pass
def test_two(self):
# another test
pass
class TestWithNewSetup(TestOriginal):
def current_setUp(self):
# overwrite your first current_setUp
I have a group of test cases that all should have exactly the same test done, along the lines of "Does method x return the name of an existing file?"
I thought that the best way to do it would be a base class deriving from TestCase that they all share, and simply add the test to that class. Unfortunately, the testing framework still tries to run the test for the base class, where it doesn't make sense.
class SharedTest(TestCase):
def x(self):
...do test...
class OneTestCase(SharedTest):
...my tests are performed, and 'SharedTest.x()'...
I tried to hack in a check to simply skip the test if it's called on an object of the base class rather than a derived class like this:
class SharedTest(TestCase):
def x(self):
if type(self) != type(SharedTest()):
...do test...
else:
pass
but got this error:
ValueError: no such test method in <class 'tests.SharedTest'>: runTest
First, I'd like any elegant suggestions for doing this. Second, though I don't really want to use the type() hack, I would like to understand why it's not working.
You could use a mixin by taking advantage that the test runner only runs tests inheriting from unittest.TestCase (which Django's TestCase inherits from.) For example:
class SharedTestMixin(object):
# This class will not be executed by the test runner (it inherits from object, not unittest.TestCase.
# If it did, assertEquals would fail , as it is not a method that exists in `object`
def test_common(self):
self.assertEquals(1, 1)
class TestOne(TestCase, SharedTestMixin):
def test_something(self):
pass
# test_common is also run
class TestTwo(TestCase, SharedTestMixin):
def test_another_thing(self):
pass
# test_common is also run
For more information on why this works do a search for python method resolution order and multiple inheritance.
I faced a similar problem. I couldn't prevent the test method in the base class being executed but I ensured that it did not exercise any actual code. I did this by checking for an attribute and returning immediately if it was set. This attribute was only set for the Base class and hence the tests ran everywhere else but the base class.
class SharedTest(TestCase):
def setUp(self):
self.do_not_run = True
def test_foo(self):
if getattr(self, 'do_not_run', False):
return
# Rest of the test body.
class OneTestCase(SharedTest):
def setUp(self):
super(OneTestCase, self).setUp()
self.do_not_run = False
This is a bit of a hack. There is probably a better way to do this but I am not sure how.
Update
As sdolan says a mixin is the right way. Why didn't I see that before?
Update 2
(After reading comments) It would be nice if (1) the superclass method could avoid the hackish if getattr(self, 'do_not_run', False): check; (2) if the number of tests were counted accurately.
There is a possible way to do this. Django picks up and executes all test classes in tests, be it tests.py or a package with that name. If the test superclass is declared outside the tests module then this won't happen. It can still be inherited by test classes. For instance SharedTest can be located in app.utils and then used by the test cases. This would be a cleaner version of the above solution.
# module app.utils.test
class SharedTest(TestCase):
def test_foo(self):
# Rest of the test body.
# module app.tests
from app.utils import test
class OneTestCase(test.SharedTest):
...
I have a Python class and its methods are partially covered by unit tests like this:
class MyClass:
def __init__(self):
self.instancevar1 = None # some methods change this value
self.instancevar2 = None # some methods change this value
...
def method1(self, input_data):
...
<self.instancevar1 and self.instancevar2 are used here>
<return True or False>
...
class TestMyClass(unittest.TestCase):
def test_method1(self):
mc = MyClass()
input_data = 'foo'
self.assertEqual(mc.method1(input_data), True)
The tests are passing for simple cases like this.
I also have other classes that use instances of MyClass like this:
class MyOtherClass:
def __init__(self):
self.mc = MyClass()
Now, in some method of MyOtherClass, I have a situation like this:
class MyOtherClass:
...
def some_method(self):
...
result = self.mc.method1('foo') # result is expected to be True, but it is False
I think I understand the reason why this might happen. In the test case, I instantiate MyClass and call the test. In MyOtherClass, self.mc is referenced multiple times, misc methods of it are called and they probably change the internal state of the object (some values of instance variables), thus producing a different return value.
What are the suggested best practices for debugging such issues?
The fact that I have some tests for MyClass is good, but doesn't seem to help much here. Does this example contain some design flaws which cause my problems?
As Rudi said, there's either a bug in MyClass.method1() that causes it to return False when it shouldn't, or there's a bug in other parts of the code that puts mc in a state you didn't expect.
You can address both problems with additional unit tests:
If there's a bug in MyClass.method1() when mc is in a particular state, you should add more unit tests to verify that mc.method1() returns the correct value for each state that mc is in.
Also, add unit tests for MyOtherClass and other relevant parts of the code to ensure that self.mc is always in the state you're expecting, regardless of what operations you perform on the MyClass instance.
It's hard to come up with more specific advice than this since your example is fairly abstract, but I think you should get an idea about where to start looking and how to improve your tests to handle these issues better.
At first you need to find out if the MyClass object is in a correct state when some_method calls the defect method. If the state is correct, there is a bug in MyClass.method1, if it is not you need to search for the wrong state transition.
If there was a wrong state transition, you can add code to MyClass to detect the fault, and raise an exception or assert, when the wrong state is entered.