When using the new discover feature in Python 2.7, I'm getting a weird error. I have some unit tests that require a bit of extra setup and some member data from a file. I'm trying to add my setup test cases to the current test suite that is passed to load_tests(). But because the test suite tests already contains the standard tests (including the TestCase objects in the current module), the proper setup for the automatically added testcase is not done and I get an AttributeError.
In the code below, load_tests() is used to create one test case for each line of data in a csv file. The file has three lines, but for some reason a fourth testcase is being created.
#!/usr/bin/python
import unittest
class Foo(unittest.TestCase):
def setup(self,bar):
print "Foo.setup()"
self.bar = bar
def runTest(self):
print self.bar
def load_tests(loader, tests, pattern):
f = open('data.csv') # data.csv contains three lines: "a\nb\nc"
for line in f:
tc = Foo()
tc.setup(line)
tests.addTest(tc)
return tests
unittest.main()
When I execute this code, the output shows that 4 tests were executed and one of them failed. The data file only contains three lines, and Foo.setup() was only called three times. So load_tests() created the three test cases as designed.
Foo.setup()
Foo.setup()
Foo.setup()
Ea
.b
.c
.
======================================================================
ERROR: runTest (__main__.Foo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./foo.py", line 11, in runTest
print self.bar
AttributeError: 'Foo' object has no attribute 'bar'
----------------------------------------------------------------------
Ran 4 tests in 0.002s
Is there a way to remove the TestCase that was automatically loaded in the suite? I cannot create a new empty TestSuite because I need all the other tests that are already there. I just want to add these tests to the suite.
Edit: clarified my question and code examples. I was a bit ambiguous before.
There are a couple of ways you can do this.
If you are always going to instantiate Foo, you could set up bar in the __init__ method of the class.
class Foo(unittest.TestCase):
def __init__(self, bar, *args, **kwargs):
super(Foo, self).__init__(*args, **kwargs)
self.bar = bar
def runTest(self):
print self.bar
f = Foo("some string")
I haven't used the load_tests pattern directly before -- I usually just rely on nose's auto test discovery, but all of these would work for your setup. If you later wanted to use TestSuites and unittest/nose's test autodiscovery, and wanted to use classes, you could use a class factory:
def make_foo_case(the_bar):
class Foo(unittest.TestCase):
bar = the_bar
def runTest(self):
print self.bar
return Foo
f = (make_testcase("some string"))()
or use type to subclass foo and set up the element in the members, (essentially the same as the previous) e.g.:
class Foo2(object):
def runTest(self):
print self.bar
f = (type("Tester2", (Foo2,unittest.TestCase), {"bar":"some string"}))()
Apparently I misunderstand load_tests(). According to the Python code in /usr/lib/python2.7/unittest/loader.py, the tests are loaded from the module as normal. Then, if load_tests() exist, it is called as well.
def loadTestsFromModule(self, module, use_load_tests=True):
"""Return a suite of all tests cases contained in the given module"""
tests = []
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
load_tests = getattr(module, 'load_tests', None)
tests = self.suiteClass(tests)
if use_load_tests and load_tests is not None:
try:
return load_tests(self, tests, None)
except Exception, e:
return _make_failed_load_tests(module.__name__, e,
self.suiteClass)
return tests
So I guess my solution is to accept the extra test case that is created, check for it, and just pass it. Much thanks to Jeff for trying to help me out.
Related
Not wanting to test manually my code, I'm trying to write a test which mocks/patches one of my dependencies (PyInquirer is a pretty neat package which handles the CLI for me - question dicts in, answer dicts out).
However, being very new to Python, I'm having difficulties with mocking that dependency. Here's the code I'm testing:
from PyInquirer import prompt
class Foo:
def bar(self):
# this line is asking the user for inpit, and that's what I want to mock.
a = prompt({'name': 'q',
'type': 'input',
'message': 'well, foo'})
print("f is", f)
return a
And this is the test:
import unittest
from unittest.mock import patch
from Foo import Foo
class TestFoo(unittest.TestCase):
#patch('PyInquirer.prompt', return_value=24)
def test_bar(self):
f = Foo()
a = f.bar()
assert a == 24
if __name__ == '__main__':
unittest.main()
(the real code is obviously more complicated, but this is the essence of the problem). Which manifests itself as:
Error
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 605, in run
testMethod()
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/mock.py", line 1179, in patched
return func(*args, **keywargs)
TypeError: test_bar() takes 1 positional argument but 2 were given
I'm quite confused.
If I omit the patch decorator, the invocation fails with an expected assertion error - the dictionary produced by prompt isn't equal to 24. But if I do provide the decorator, I get the argument mismatch above. And indeed the last line in the stacktrace does show the function "func", which I presume is what the decorator was applied to, is invoked with two arguments... but... why? Can't be the essence of a problem? That only functions with arity of two can be thus patched =)
Your class uses the name Foo.prompt (because of how you import it), so that's what you need to patch.
class TestFoo(unittest.TestCase):
#patch('Foo.prompt', return_value=24)
def test_bar(self, mock_prompt):
f = Foo()
a = f.bar()
assert a == 24
You also need to add a parameter to test_bar to receive the patched object, whether or not you plan to use it. If you don't want to do that,
you can move the call to patch inside the method, using it with a with statement.
class TestFoo(unittest.TestCase):
def test_bar(self):
with patch('Foo.prompt', return_value=24):
f = Foo()
a = f.bar()
assert a == 24
I am using setUp to create a new object from a class I created. It is my understanding that this function will be executed before each test in the test case, and that this should result in new objects being created for each test. It seems that this is not what is happening; at least not in my tests.
Here is my class:
class Entity:
def __init__(self, entities = []):
self.entities = entities
def add(self, entity: str):
if entity not in self.entities:
self.entities.append(entity)
And here are the corresponding tests:
import unittest
from entity import Entity
class EntityTestCase(unittest.TestCase):
def setUp(self):
self.entity = Entity()
print("Starting Entites = {}".format(self.entity.entities))
def testA(self):
self.entity.add("Foo")
self.assertEqual(self.entity.entities, ["Foo"])
def testB(self):
self.entity.add("Bar")
self.assertEqual(self.entity.entities, ["Bar"])
if __name__ == '__main__':
unittest.main()
I would expect that testA and testB would start with new Entity objects. That is, I would expect Entity.entities to be a brand new list for each test.
I am running my tests with python -m unittest discover -v which results in the following output:
$ python -m unittest discover -v
testA (test_entity.EntityTestCase) ... Starting Entites = []
ok
testB (test_entity.EntityTestCase) ... Starting Entites = ['Foo']
FAIL
======================================================================
FAIL: testB (test_entity.EntityTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/julioguzman/Sites/Foobar/test_entity.py", line 15, in testB
self.assertEqual(self.entity.entities, ["Bar"])
AssertionError: Lists differ: ['Foo', 'Bar'] != ['Bar']
First differing element 0:
'Foo'
'Bar'
First list contains 1 additional elements.
First extra element 1:
'Bar'
- ['Foo', 'Bar']
+ ['Bar']
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
As you can see, testB starts with the data from testA. This is not the desired behavior, although it may be the intended one.
How do I ensure that my objects are "clean" for each test? Why is this happening?
You have
class Entity:
def __init__(self, entities = []):
self.entities = entities
Where the default argument is instantiated once and thus resulting in this common issue
"Least Astonishment" and the Mutable Default Argument
Simply modify the __init__ to use a surrogate value and assign a brand new list if so:
class Entity:
def __init__(self, entities=NotImplemented):
if entities is NotImplemented:
self.entities = []
else:
self.entities = entities
As an aside, this is one reason why unittests are written: they also test whether the class was created correctly with the expected behavior. It was not an issue with your test when the setUp clearly showed that it created a brand new object - the fault lies with the implementation where it did something unexpected.
I wanted to call setUpClass and tearDownClass so that setup and teardown would be performed only once for each test. However, it keeps failing for me when I call tearDownClass. I only want to record 1 test result, either PASS if both tests passed or FAIL if both tests failed. If I call only setup and tearDown then all works fine:
Calling setUpClass and tearDownClass:
#!/usr/bin/python
import datetime
import itertools
import logging
import os
import sys
import time
import unittest
LOGFILE = 'logfile.txt'
class MyTest(unittest.TestCase):
global testResult
testResult = None
#classmethod
def setUpClass(self):
## test result for DB Entry:
self.dbresult_dict = {
'SCRIPT' : 'MyTest.py',
'RESULT' : testResult,
}
def test1(self):
expected_number = 10
actual_number = 10
self.assertEqual(expected_number, actual_number)
def test2(self):
expected = True
actual = True
self.assertEqual(expected, actual)
def run(self, result=None):
self.testResult = result
unittest.TestCase.run(self, result)
#classmethod
def tearDownClass(self):
ok = self.testResult.wasSuccessful()
errors = self.testResult.errors
failures = self.testResult.failures
if ok:
self.dbresult_dict['RESULT'] = 'Pass'
else:
logging.info(' %d errors and %d failures',
len(errors), len(failures))
self.dbresult_dict['RESULT'] = 'Fail'
if __name__ == '__main__':
logger = logging.getLogger()
logger.addHandler(logging.FileHandler(LOGFILE, mode='a'))
stderr_file = open(LOGFILE, 'a')
runner = unittest.TextTestRunner(verbosity=2, stream=stderr_file, descriptions=True)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
sys.exit()
unittest.main(module=itersuite, exit=True)
stderr_file.close()
Error:
test1 (__main__.MyTest) ... ok
test2 (__main__.MyTest) ... ok
ERROR
===================================================================
ERROR: tearDownClass (__main__.MyTest)
-------------------------------------------------------------------
Traceback (most recent call last):
File "testTearDownClass.py", line 47, in tearDownClass
ok = self.testResult.wasSuccessful()
AttributeError: type object 'MyTest' has no attribute 'testResult'
----------------------------------------------------------------------
Ran 2 tests in 0.006s
FAILED (errors=1)
like #Marcin already pointed out, you're using the Unittest-Framework in a way it isn't intended.
To see if the tests are successful you check the given values with the expected, like you already did: assertEqual(given, expected). Unittest will then collect a summary of failed ones. you don't have to do this manually.
If you want to check that two tests need to be together successful or fail together, these should be combined in ONE Test, maybe as a additionally one, if the individual Tests need to be checked as well. This is nothing you want to save and load afterwards. The tests itself should be as stateless as possible.
When you say you want to run the SetUp and TearDown 'once per test', do you mean once per test-method or per test-run? This is different if you have more than one test-method inside your class:
setUp() Will be called before each test-method
tearDown() Will be called after each test-method
setUpClass() Will be called once per class (before the first test-method of this class)
tearDownClass() Will be called once per class (after the last test-method of this class)
Here's the official documentation
Here's a related answer
Change tearDownClass(self) to tearDownClass(cls) and setUpClass(self) to setUpClass(cls).
I have a tests file in one of my Pyramid projects. It has one suite with six tests in it:
...
from .scripts import populate_test_data
class FunctionalTests(unittest.TestCase):
def setUp(self):
settings = appconfig('config:testing.ini',
'main',
relative_to='../..')
app = main({}, **settings)
self.testapp = TestApp(app)
self.config = testing.setUp()
engine = engine_from_config(settings)
DBSession.configure(bind=engine)
populate_test_data(engine)
def tearDown(self):
DBSession.remove()
tearDown()
def test_index(self):
...
def test_login_form(self):
...
def test_read_recipe(self):
...
def test_tag(self):
...
def test_dish(self):
...
def test_dashboard_forbidden(self):
...
Now, when I run nosetests templates.py (where templates.py is the mentioned file) I get the following output:
......E
======================================================================
ERROR: templates.populate_test_data
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yentsun/env/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/yentsun/env/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/util.py", line 622, in newfunc
return func(*arg, **kw)
TypeError: populate_test_data() takes exactly 1 argument (0 given)
----------------------------------------------------------------------
Ran 7 tests in 1.985s
FAILED (errors=1)
When I run the tests with test suite specified nosetests templates.py:FunctionalTests, the output is, as expected, ok:
......
----------------------------------------------------------------------
Ran 6 tests in 1.980s
OK
Why do I have different output and why is an extra (7th) test run?
UPDATE. Its a bit frustrating, but I when removed the word test from the name populate_test_data (it became populate_dummy_data), everything worked fine.
The problem is solved for now, but maybe somebody knows what went wrong here - why a function from setUp had been tested?
Finding and running tests
nose, by default, follows a few simple rules for test discovery.
If it looks like a test, it’s a test. Names of directories, modules, classes and functions are compared against the testMatch regular
expression, and those that match are considered tests. Any class that
is a unittest.TestCase subclass is also collected, so long as it is
inside of a module that looks like a test.
(from nose 1.3.0 documentation)
In the nose's code, the regexp is defined as r'(?:^|[\b_\.%s-])[Tt]est' % os.sep, and if you inpect nose/selector.py, method Selector.matches(self, name) you'll see that the code uses re.search, which looks for a match anywhere in the string, not only at the beginning, as re.match does.
A small test:
>>> import re
>>> import os
>>> testMatch = r'(?:^|[\b_\.%s-])[Tt]est' % os.sep
>>> re.match(testMatch, 'populate_test_data')
>>> re.search(testMatch, 'populate_test_data')
<_sre.SRE_Match object at 0x7f3512569238>
So populate_test_data indeed "looks like a test" by nose's standards.
I've got test classes that inherit from unittest.TestCase. I load the classes multiple times like so:
tests = [TestClass1, TestClass2]
for test in tests:
for var in array:
# somehow indicate that this test should have the value of 'var'
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(test))
Thing is, I want to pass the value of 'var' to each test, but I cannot use class variables because they are shared between every instance of the class, and I don't have access to the code that actually does the instantiation of the objects. What is the best way of accomplishing this?
I think that even if you don't have access to the class that implement the test cases, you can subclass them and overload the setUp method.
I think you're going about this the wrong way. Rather than doing what you are trying there why dont you just do, say you have in class:
from my_tests.variables import my_array
class TestClass1(unittest.TestCase):
def setUp():
....initializations...
def tearDown():
....clean up after...
def my_test_that_should_use_value_from_array(self):
for value in my_array:
test_stuff(value)
UPDATE:
Since you need to:
feed some variable value to MyTestCase
run MyTestCase using this value
change value
If MyTestCase still running - use updated value.
Consider this:
keep values map in the file (.csv/.txt/.xml/etc.)
read values map from file in the setUp()
find value for your MyTestCase from values map using TestCase.id() method (as shown in the example below).
use it in the test cases.
unittest has handy id() method, which returns test case name in filename.testclassname.methodname format.
So you can use it like this:
import unittest
my_variables_map = {
'test_01': 'foo',
'test_02': 'bar',
}
class MyTest(unittest.TestCase):
def setUp(self):
test_method_name = self.id() # filename.testclassname.methodname
test_method_name = test_method_name.split('.')[-1] # method name
self.variable_value = my_variables_map.get(test_method_name)
self.error_message = 'No values found for "%s" method.' % test_method_name
def test_01(self):
self.assertTrue(self.variable_value is not None, self.error_message)
def test_02(self):
self.assertTrue(self.variable_value is not None, self.error_message)
def test_03(self):
self.assertTrue(self.variable_value is not None, self.error_message)
if __name__ == '__main__':
unittest.main()
This gives you:
$ python /tmp/ut.py
..F
======================================================================
FAIL: test_03 (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/ut.py", line 25, in test_03
self.assertTrue(self.variable_value is not None, self.error_message)
AssertionError: No values found for "test_03" method.
----------------------------------------------------------------------
Ran 3 tests in 0.000s
FAILED (failures=1)
$
I found the Data-Driven Tests (DDT - not the pesticide) package helpful for this.
http://ddt.readthedocs.org/en/latest/example.html