I am using setUp to create a new object from a class I created. It is my understanding that this function will be executed before each test in the test case, and that this should result in new objects being created for each test. It seems that this is not what is happening; at least not in my tests.
Here is my class:
class Entity:
def __init__(self, entities = []):
self.entities = entities
def add(self, entity: str):
if entity not in self.entities:
self.entities.append(entity)
And here are the corresponding tests:
import unittest
from entity import Entity
class EntityTestCase(unittest.TestCase):
def setUp(self):
self.entity = Entity()
print("Starting Entites = {}".format(self.entity.entities))
def testA(self):
self.entity.add("Foo")
self.assertEqual(self.entity.entities, ["Foo"])
def testB(self):
self.entity.add("Bar")
self.assertEqual(self.entity.entities, ["Bar"])
if __name__ == '__main__':
unittest.main()
I would expect that testA and testB would start with new Entity objects. That is, I would expect Entity.entities to be a brand new list for each test.
I am running my tests with python -m unittest discover -v which results in the following output:
$ python -m unittest discover -v
testA (test_entity.EntityTestCase) ... Starting Entites = []
ok
testB (test_entity.EntityTestCase) ... Starting Entites = ['Foo']
FAIL
======================================================================
FAIL: testB (test_entity.EntityTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/julioguzman/Sites/Foobar/test_entity.py", line 15, in testB
self.assertEqual(self.entity.entities, ["Bar"])
AssertionError: Lists differ: ['Foo', 'Bar'] != ['Bar']
First differing element 0:
'Foo'
'Bar'
First list contains 1 additional elements.
First extra element 1:
'Bar'
- ['Foo', 'Bar']
+ ['Bar']
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
As you can see, testB starts with the data from testA. This is not the desired behavior, although it may be the intended one.
How do I ensure that my objects are "clean" for each test? Why is this happening?
You have
class Entity:
def __init__(self, entities = []):
self.entities = entities
Where the default argument is instantiated once and thus resulting in this common issue
"Least Astonishment" and the Mutable Default Argument
Simply modify the __init__ to use a surrogate value and assign a brand new list if so:
class Entity:
def __init__(self, entities=NotImplemented):
if entities is NotImplemented:
self.entities = []
else:
self.entities = entities
As an aside, this is one reason why unittests are written: they also test whether the class was created correctly with the expected behavior. It was not an issue with your test when the setUp clearly showed that it created a brand new object - the fault lies with the implementation where it did something unexpected.
Related
I'm writing a simple example to help me understand how mocking works in unittest. I have a module with two functions:
# model.animals.py
def get_animals(animal_type):
db = connect_to_db()
result = db.query_all_data()
return list(filter(lambda x: x['animal_type'] == animal_type, result))
def connect_to_db():
pass # That would normally return a DB connection instance
I want to test the get_animals() function which uses a DB connection to retrieve information about all animals and then filters returned data based on animal type. Since I don't want to set up the whole database, I just want to mock the connect_to_db() function which returns a DB connection instance.
This is my test class:
# test_mock.py
from unittest import TestCase, main
from unittest.mock import Mock, patch
from model.animals import get_animals
class GetDataTest(TestCase):
#patch('model.animals.connect_to_db')
def test_get_animals(self, mock_db: Mock):
mock_db.return_value.query_all_data.return_value = [
{
'animal_type': 'meerkat',
'age': 5
},
{
'animal_type': 'meerkat',
'age': 11
},
{
'animal_type': 'cow',
'age': 3
}
]
result = get_animals('meerkat') # Run the function under test
mock_db.assert_called_once() # OK
mock_db.query_all_data.assert_called_once() # AssertionError
self.assertEqual(len(result), 2) # OK
self.assertEqual(result[0]['age'], 5) # OK
if __name__ == "__main__":
main()
As part of the test I wanted to not only check the filtering of animals based on their type but also whether all the methods inside get_animals() are called.
The test generally works as expected but I get an error when checking whether the query_all_data() function has been called:
AssertionError: Expected 'query_all_data' to have been called once. Called 0 times.
When I add spec=True to my patch I get another error:
AttributeError: Mock object has no attribute 'query_all_data'
Clearly, the function query_all_data is not visible inside the mock even though I set its return value in the test with mock_db.return_value.query_all_data.return_value = ....
What am I missing?
The reason that mock_db.query_all_data.assert_called_once() failed is that it should be mock_db.return_value.query_all_data.assert_called_once().
I have created a helper library to help me generate asserts for mocks so that I won't stumble on such issues as often.
To use it do: pip install mock-generator
Then, in your test place these lines after result = get_animals('meerkat'):
from mock_autogen import generate_asserts
generate_asserts(mock_db)
When you run the test, it would generate the asserts for you (printed to the console and copied to the clipboard):
assert 1 == mock_db.call_count
mock_db.assert_called_once_with()
mock_db.return_value.query_all_data.assert_called_once_with()
You can then edit the generated asserts and use whichever fits your test.
I am learning Python unit testing using unittest module.
I stumbled accross a strange behavior.
Consider this code :
import unittest
class Foo:
pass
class FooTest(unittest.TestCase):
def test_non_existent_property(self):
foo = Foo()
self.assertTrue(0, len(foo.class_name))
def test_assigning_name(self):
foo = Foo()
foo.class_name = 'bar'
self.assertEqual('bar', foo.class_name)
unittest.main()
The tests results are :
ERROR: test_non_existent_property (__main__.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<pyshell#28>", line 4, in test_non_existent_property
AttributeError: 'Foo' object has no attribute 'class_name'
----------------------------------------------------------------------
Ran 2 tests in 0.037s
FAILED (errors=1)
The first test fails as expected.
But the second test passes, and this puzzles me.
Shouldn't it fail too ? Why doesn't it fail ?
You can add attributes after the class definition.
Class Foo has no attribute 'class_name' as Class attribute. But you created 'class_name' in your test method as Instance attribute:
foo.class_name = 'bar'
All class instances don't have this attribute, only one used in the test has it.
Thats why your second test passed.
https://docs.python.org/3/tutorial/classes.html#class-and-instance-variables
https://www.tutorialsteacher.com/articles/class-attributes-vs-instance-attributes-in-python
I wanted to call setUpClass and tearDownClass so that setup and teardown would be performed only once for each test. However, it keeps failing for me when I call tearDownClass. I only want to record 1 test result, either PASS if both tests passed or FAIL if both tests failed. If I call only setup and tearDown then all works fine:
Calling setUpClass and tearDownClass:
#!/usr/bin/python
import datetime
import itertools
import logging
import os
import sys
import time
import unittest
LOGFILE = 'logfile.txt'
class MyTest(unittest.TestCase):
global testResult
testResult = None
#classmethod
def setUpClass(self):
## test result for DB Entry:
self.dbresult_dict = {
'SCRIPT' : 'MyTest.py',
'RESULT' : testResult,
}
def test1(self):
expected_number = 10
actual_number = 10
self.assertEqual(expected_number, actual_number)
def test2(self):
expected = True
actual = True
self.assertEqual(expected, actual)
def run(self, result=None):
self.testResult = result
unittest.TestCase.run(self, result)
#classmethod
def tearDownClass(self):
ok = self.testResult.wasSuccessful()
errors = self.testResult.errors
failures = self.testResult.failures
if ok:
self.dbresult_dict['RESULT'] = 'Pass'
else:
logging.info(' %d errors and %d failures',
len(errors), len(failures))
self.dbresult_dict['RESULT'] = 'Fail'
if __name__ == '__main__':
logger = logging.getLogger()
logger.addHandler(logging.FileHandler(LOGFILE, mode='a'))
stderr_file = open(LOGFILE, 'a')
runner = unittest.TextTestRunner(verbosity=2, stream=stderr_file, descriptions=True)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
sys.exit()
unittest.main(module=itersuite, exit=True)
stderr_file.close()
Error:
test1 (__main__.MyTest) ... ok
test2 (__main__.MyTest) ... ok
ERROR
===================================================================
ERROR: tearDownClass (__main__.MyTest)
-------------------------------------------------------------------
Traceback (most recent call last):
File "testTearDownClass.py", line 47, in tearDownClass
ok = self.testResult.wasSuccessful()
AttributeError: type object 'MyTest' has no attribute 'testResult'
----------------------------------------------------------------------
Ran 2 tests in 0.006s
FAILED (errors=1)
like #Marcin already pointed out, you're using the Unittest-Framework in a way it isn't intended.
To see if the tests are successful you check the given values with the expected, like you already did: assertEqual(given, expected). Unittest will then collect a summary of failed ones. you don't have to do this manually.
If you want to check that two tests need to be together successful or fail together, these should be combined in ONE Test, maybe as a additionally one, if the individual Tests need to be checked as well. This is nothing you want to save and load afterwards. The tests itself should be as stateless as possible.
When you say you want to run the SetUp and TearDown 'once per test', do you mean once per test-method or per test-run? This is different if you have more than one test-method inside your class:
setUp() Will be called before each test-method
tearDown() Will be called after each test-method
setUpClass() Will be called once per class (before the first test-method of this class)
tearDownClass() Will be called once per class (after the last test-method of this class)
Here's the official documentation
Here's a related answer
Change tearDownClass(self) to tearDownClass(cls) and setUpClass(self) to setUpClass(cls).
When using the new discover feature in Python 2.7, I'm getting a weird error. I have some unit tests that require a bit of extra setup and some member data from a file. I'm trying to add my setup test cases to the current test suite that is passed to load_tests(). But because the test suite tests already contains the standard tests (including the TestCase objects in the current module), the proper setup for the automatically added testcase is not done and I get an AttributeError.
In the code below, load_tests() is used to create one test case for each line of data in a csv file. The file has three lines, but for some reason a fourth testcase is being created.
#!/usr/bin/python
import unittest
class Foo(unittest.TestCase):
def setup(self,bar):
print "Foo.setup()"
self.bar = bar
def runTest(self):
print self.bar
def load_tests(loader, tests, pattern):
f = open('data.csv') # data.csv contains three lines: "a\nb\nc"
for line in f:
tc = Foo()
tc.setup(line)
tests.addTest(tc)
return tests
unittest.main()
When I execute this code, the output shows that 4 tests were executed and one of them failed. The data file only contains three lines, and Foo.setup() was only called three times. So load_tests() created the three test cases as designed.
Foo.setup()
Foo.setup()
Foo.setup()
Ea
.b
.c
.
======================================================================
ERROR: runTest (__main__.Foo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./foo.py", line 11, in runTest
print self.bar
AttributeError: 'Foo' object has no attribute 'bar'
----------------------------------------------------------------------
Ran 4 tests in 0.002s
Is there a way to remove the TestCase that was automatically loaded in the suite? I cannot create a new empty TestSuite because I need all the other tests that are already there. I just want to add these tests to the suite.
Edit: clarified my question and code examples. I was a bit ambiguous before.
There are a couple of ways you can do this.
If you are always going to instantiate Foo, you could set up bar in the __init__ method of the class.
class Foo(unittest.TestCase):
def __init__(self, bar, *args, **kwargs):
super(Foo, self).__init__(*args, **kwargs)
self.bar = bar
def runTest(self):
print self.bar
f = Foo("some string")
I haven't used the load_tests pattern directly before -- I usually just rely on nose's auto test discovery, but all of these would work for your setup. If you later wanted to use TestSuites and unittest/nose's test autodiscovery, and wanted to use classes, you could use a class factory:
def make_foo_case(the_bar):
class Foo(unittest.TestCase):
bar = the_bar
def runTest(self):
print self.bar
return Foo
f = (make_testcase("some string"))()
or use type to subclass foo and set up the element in the members, (essentially the same as the previous) e.g.:
class Foo2(object):
def runTest(self):
print self.bar
f = (type("Tester2", (Foo2,unittest.TestCase), {"bar":"some string"}))()
Apparently I misunderstand load_tests(). According to the Python code in /usr/lib/python2.7/unittest/loader.py, the tests are loaded from the module as normal. Then, if load_tests() exist, it is called as well.
def loadTestsFromModule(self, module, use_load_tests=True):
"""Return a suite of all tests cases contained in the given module"""
tests = []
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
load_tests = getattr(module, 'load_tests', None)
tests = self.suiteClass(tests)
if use_load_tests and load_tests is not None:
try:
return load_tests(self, tests, None)
except Exception, e:
return _make_failed_load_tests(module.__name__, e,
self.suiteClass)
return tests
So I guess my solution is to accept the extra test case that is created, check for it, and just pass it. Much thanks to Jeff for trying to help me out.
I've got test classes that inherit from unittest.TestCase. I load the classes multiple times like so:
tests = [TestClass1, TestClass2]
for test in tests:
for var in array:
# somehow indicate that this test should have the value of 'var'
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(test))
Thing is, I want to pass the value of 'var' to each test, but I cannot use class variables because they are shared between every instance of the class, and I don't have access to the code that actually does the instantiation of the objects. What is the best way of accomplishing this?
I think that even if you don't have access to the class that implement the test cases, you can subclass them and overload the setUp method.
I think you're going about this the wrong way. Rather than doing what you are trying there why dont you just do, say you have in class:
from my_tests.variables import my_array
class TestClass1(unittest.TestCase):
def setUp():
....initializations...
def tearDown():
....clean up after...
def my_test_that_should_use_value_from_array(self):
for value in my_array:
test_stuff(value)
UPDATE:
Since you need to:
feed some variable value to MyTestCase
run MyTestCase using this value
change value
If MyTestCase still running - use updated value.
Consider this:
keep values map in the file (.csv/.txt/.xml/etc.)
read values map from file in the setUp()
find value for your MyTestCase from values map using TestCase.id() method (as shown in the example below).
use it in the test cases.
unittest has handy id() method, which returns test case name in filename.testclassname.methodname format.
So you can use it like this:
import unittest
my_variables_map = {
'test_01': 'foo',
'test_02': 'bar',
}
class MyTest(unittest.TestCase):
def setUp(self):
test_method_name = self.id() # filename.testclassname.methodname
test_method_name = test_method_name.split('.')[-1] # method name
self.variable_value = my_variables_map.get(test_method_name)
self.error_message = 'No values found for "%s" method.' % test_method_name
def test_01(self):
self.assertTrue(self.variable_value is not None, self.error_message)
def test_02(self):
self.assertTrue(self.variable_value is not None, self.error_message)
def test_03(self):
self.assertTrue(self.variable_value is not None, self.error_message)
if __name__ == '__main__':
unittest.main()
This gives you:
$ python /tmp/ut.py
..F
======================================================================
FAIL: test_03 (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/ut.py", line 25, in test_03
self.assertTrue(self.variable_value is not None, self.error_message)
AssertionError: No values found for "test_03" method.
----------------------------------------------------------------------
Ran 3 tests in 0.000s
FAILED (failures=1)
$
I found the Data-Driven Tests (DDT - not the pesticide) package helpful for this.
http://ddt.readthedocs.org/en/latest/example.html