Develop Reference

Mocking an entire class

Long story short, I'm perfectly able to mock class method, when it's just that method that's replaced by mock object, but I'm unable to mock that method when I'm trying to replace the whole class by the mock object
The #mock.patch.object successfully mocks the scan method but #mock.patch fails to do so. I've followed the example at
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch
but apparently I'm doing something wrong.
I'm mocking the lexicon module in the same namespace in both cases (it's imported by import lexicon in the sentence_parser) but the mock_lexicon is lexicon.lexicon check fails
#!python
import sys;
sys.path.append('D:\python\lexicon');
import lexicon;
import sentence_parser;
import unittest2 as unittest;
import mock;
class ParserTestCases(unittest.TestCase) :
def setUp(self) :
self.Parser = sentence_parser.Parser();
#mock.patch('lexicon.lexicon')
def test_categorizedWordsAreAssigned_v1(self, mock_lexicon) :
print "mock is lexicon:";
print mock_lexicon is lexicon.lexicon + "\n";
instance = mock_lexicon.return_value;
instance.scan.return_value = "anything";
self.Parser.categorize_words_in_sentence("sentence");
instance.scan.assert_called_once_with("sentence");
#mock.patch.object(lexicon.lexicon, 'scan')
def test_categorizedWordsAreAssigned_v2(self, mock_scan) :
mock_scan.return_value = "anything";
self.Parser.categorize_words_in_sentence("sentence");
mock_scan.assert_called_once_with("sentence");
if (__name__ == '__main__') :
unittest.main()
Output :
mock is lexicon:
False
======================================================================
FAIL: test_categorizedWordsAreAssigned_v1 (__main__.ParserTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\python\get_img\getImage_env\lib\site-packages\mock\mock.py", line 1305, in patched
return func(*args, **keywargs)
File "./test_sentence_parser.py", line 26, in test_categorizedWordsAreAssigned_v1
instance.scan.assert_called_once_with("sentence");
File "D:\python\get_img\getImage_env\lib\site-packages\mock\mock.py", line 947, in assert_called_once_with
raise AssertionError(msg)
AssertionError: Expected 'scan' to be called once. Called 0 times.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
FAILED (failures=1)
EDIT :
To clarify, the Parser is defined as follows
#!python
import sys;
sys.path.append('D:\python\lexicon');
import lexicon;
class Parser(object) :
my_lexicon = lexicon.lexicon()
def __init__(self) :
self.categorized_words = ['test'];
def categorize_words_in_sentence(self, sentence) :
self.categorized_words = self.my_lexicon.scan(sentence);
if (__name__ == '__main__') :
instance = Parser();
instance.categorize_words_in_sentence("bear");
print instance.categorized_words;

What is real relevant here is how categorize_words_in_sentence Parser's method use lexicon. But first of all we should remove the noise:
print mock_lexicon is lexicon.lexicon + "\n"
Is what can lead us to the wrong direction: try to replace it by
self.assertIs(mock_lexicon, lexicon.lexicon)
and you will understand that you are printing False because mock_lexicon is not lexicon.lexicon + "\n" but just lexicon.lexicon.
Now I cannot tell you why the first test doesn't work because the answer is in categorize_words_in_sentence method or more probably in sentence_parser module where I can guess you can have something like
from lexicon import lexicon
In both case take a look to Where to Patch documentation that can enlighten you on what can be the cause and what you really need to patch in your case.
The second version works just because you are patching the object and not the reference (that should be different).
Finally the more concise and general version can be:
#mock.patch('lexicon.lexicon.scan', return_value="anything")
def test_categorizedWordsAreAssigned_v3(self, mock_scan) :
self.Parser.categorize_words_in_sentence("sentence")
mock_scan.assert_called_once_with("sentence")
One more thing: remove unittest2 at least you're not using python 2.4 and you are interested on backported unittest features.
[EDIT]
Now I can stop to guess and point to you why the first version doesn't work and will never work:
class Parser(object) :
my_lexicon = lexicon.lexicon()
Parser.my_lexicon attribute is evaluated at the load time. That means when you import sentence_parser a lexicon is created and the reference associated to Parser.my_lexicon. When you patch lexicon.lexicon you leave this reference untouched and your parser object still use the original reference created when is imported.
What you can do is to patch the reference in Parser class by
#patch("sentence_parser.Parser.my_lexicon")
You can use create_autospect if you want give to your mock the same lexicon's signature.
#patch("sentence_parser.Parser.my_lexicon", create_autospec("lexicon.lexicon", instance=True))

How to use Python Unittest TearDownClass with TestResult.wasSuccessful()

I wanted to call setUpClass and tearDownClass so that setup and teardown would be performed only once for each test. However, it keeps failing for me when I call tearDownClass. I only want to record 1 test result, either PASS if both tests passed or FAIL if both tests failed. If I call only setup and tearDown then all works fine:
Calling setUpClass and tearDownClass:
#!/usr/bin/python
import datetime
import itertools
import logging
import os
import sys
import time
import unittest
LOGFILE = 'logfile.txt'
class MyTest(unittest.TestCase):
global testResult
testResult = None
#classmethod
def setUpClass(self):
## test result for DB Entry:
self.dbresult_dict = {
'SCRIPT' : 'MyTest.py',
'RESULT' : testResult,
}
def test1(self):
expected_number = 10
actual_number = 10
self.assertEqual(expected_number, actual_number)
def test2(self):
expected = True
actual = True
self.assertEqual(expected, actual)
def run(self, result=None):
self.testResult = result
unittest.TestCase.run(self, result)
#classmethod
def tearDownClass(self):
ok = self.testResult.wasSuccessful()
errors = self.testResult.errors
failures = self.testResult.failures
if ok:
self.dbresult_dict['RESULT'] = 'Pass'
else:
logging.info(' %d errors and %d failures',
len(errors), len(failures))
self.dbresult_dict['RESULT'] = 'Fail'
if __name__ == '__main__':
logger = logging.getLogger()
logger.addHandler(logging.FileHandler(LOGFILE, mode='a'))
stderr_file = open(LOGFILE, 'a')
runner = unittest.TextTestRunner(verbosity=2, stream=stderr_file, descriptions=True)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
sys.exit()
unittest.main(module=itersuite, exit=True)
stderr_file.close()
Error:
test1 (__main__.MyTest) ... ok
test2 (__main__.MyTest) ... ok
ERROR
===================================================================
ERROR: tearDownClass (__main__.MyTest)
-------------------------------------------------------------------
Traceback (most recent call last):
File "testTearDownClass.py", line 47, in tearDownClass
ok = self.testResult.wasSuccessful()
AttributeError: type object 'MyTest' has no attribute 'testResult'
----------------------------------------------------------------------
Ran 2 tests in 0.006s
FAILED (errors=1)

like #Marcin already pointed out, you're using the Unittest-Framework in a way it isn't intended.
To see if the tests are successful you check the given values with the expected, like you already did: assertEqual(given, expected). Unittest will then collect a summary of failed ones. you don't have to do this manually.
If you want to check that two tests need to be together successful or fail together, these should be combined in ONE Test, maybe as a additionally one, if the individual Tests need to be checked as well. This is nothing you want to save and load afterwards. The tests itself should be as stateless as possible.
When you say you want to run the SetUp and TearDown 'once per test', do you mean once per test-method or per test-run? This is different if you have more than one test-method inside your class:
setUp() Will be called before each test-method
tearDown() Will be called after each test-method
setUpClass() Will be called once per class (before the first test-method of this class)
tearDownClass() Will be called once per class (after the last test-method of this class)
Here's the official documentation
Here's a related answer

Change tearDownClass(self) to tearDownClass(cls) and setUpClass(self) to setUpClass(cls).

Writing a pytest function for checking the output on console (stdout)

This link gives a description how to use pytest for capturing console outputs.
I tried on this following simple code, but I get error
import sys
import pytest
def f(name):
print "hello "+ name
def test_add(capsys):
f("Tom")
out,err=capsys.readouterr()
assert out=="hello Tom"
test_add(sys.stdout)
Output:
python test_pytest.py
hello Tom
Traceback (most recent call last):
File "test_pytest.py", line 12, in <module>
test_add(sys.stdout)
File "test_pytest.py", line 8, in test_add
out,err=capsys.readouterr()
AttributeError: 'file' object has no attribute 'readouterr'
what is wrong and what fix needed? thank you
EDIT:
As per the comment, I changed capfd, but I still get the same error
import sys
import pytest
def f(name):
print "hello "+ name
def test_add(capfd):
f("Tom")
out,err=capfd.readouterr()
assert out=="hello Tom"
test_add(sys.stdout)

Use the capfd fixture.
Example:
def test_foo(capfd):
foo() # Writes "Hello World!" to stdout
out, err = capfd.readouterr()
assert out == "Hello World!"
See: http://pytest.org/en/latest/fixture.html for more details
And see: py.test --fixtures for a list of builtin fixtures.
Your example has a few problems. Here is a corrected version:
def f(name):
print "hello {}".format(name)
def test_f(capfd):
f("Tom")
out, err = capfd.readouterr()
assert out == "hello Tom\n"
Note:
Do not use sys.stdout -- Use the capfd fixture as-is as provided by pytest.
Run the test with: py.test foo.py
Test Run Output:
$ py.test foo.py
====================================================================== test session starts ======================================================================
platform linux2 -- Python 2.7.5 -- pytest-2.4.2
plugins: flakes, cache, pep8, cov
collected 1 items
foo.py .
=================================================================== 1 passed in 0.01 seconds ====================================================================
Also Note:
You do not need to run your Test Function(s) in your test modules. py.test (The CLI tool and Test Runner) does this for you.
py.test does mainly three things:
Collect your tests
Run your tests
Display statistics and possibly errors
By default py.test looks for (configurable iirc) test_foo.py test modules and test_foo() test functions in your test modules.

The problem is with your explicit call of your test function at the very end of your first code snippet block:
test_add(sys.stdout)
You should not do this; it is pytest's job to call your test functions.
When it does, it will recognize the name capsys (or capfd, for that matter)
and automatically provide a suitable pytest-internal object for you as a call argument.
(The example given in the pytest documentation is quite complete as it is.)
That object will provide the required readouterr() function.
sys.stdout does not have that function, which is why your program fails.

Weird behavior when calling load_tests()

When using the new discover feature in Python 2.7, I'm getting a weird error. I have some unit tests that require a bit of extra setup and some member data from a file. I'm trying to add my setup test cases to the current test suite that is passed to load_tests(). But because the test suite tests already contains the standard tests (including the TestCase objects in the current module), the proper setup for the automatically added testcase is not done and I get an AttributeError.
In the code below, load_tests() is used to create one test case for each line of data in a csv file. The file has three lines, but for some reason a fourth testcase is being created.
#!/usr/bin/python
import unittest
class Foo(unittest.TestCase):
def setup(self,bar):
print "Foo.setup()"
self.bar = bar
def runTest(self):
print self.bar
def load_tests(loader, tests, pattern):
f = open('data.csv') # data.csv contains three lines: "a\nb\nc"
for line in f:
tc = Foo()
tc.setup(line)
tests.addTest(tc)
return tests
unittest.main()
When I execute this code, the output shows that 4 tests were executed and one of them failed. The data file only contains three lines, and Foo.setup() was only called three times. So load_tests() created the three test cases as designed.
Foo.setup()
Foo.setup()
Foo.setup()
Ea
.b
.c
.
======================================================================
ERROR: runTest (__main__.Foo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./foo.py", line 11, in runTest
print self.bar
AttributeError: 'Foo' object has no attribute 'bar'
----------------------------------------------------------------------
Ran 4 tests in 0.002s
Is there a way to remove the TestCase that was automatically loaded in the suite? I cannot create a new empty TestSuite because I need all the other tests that are already there. I just want to add these tests to the suite.
Edit: clarified my question and code examples. I was a bit ambiguous before.

There are a couple of ways you can do this.
If you are always going to instantiate Foo, you could set up bar in the __init__ method of the class.
class Foo(unittest.TestCase):
def __init__(self, bar, *args, **kwargs):
super(Foo, self).__init__(*args, **kwargs)
self.bar = bar
def runTest(self):
print self.bar
f = Foo("some string")
I haven't used the load_tests pattern directly before -- I usually just rely on nose's auto test discovery, but all of these would work for your setup. If you later wanted to use TestSuites and unittest/nose's test autodiscovery, and wanted to use classes, you could use a class factory:
def make_foo_case(the_bar):
class Foo(unittest.TestCase):
bar = the_bar
def runTest(self):
print self.bar
return Foo
f = (make_testcase("some string"))()
or use type to subclass foo and set up the element in the members, (essentially the same as the previous) e.g.:
class Foo2(object):
def runTest(self):
print self.bar
f = (type("Tester2", (Foo2,unittest.TestCase), {"bar":"some string"}))()

Apparently I misunderstand load_tests(). According to the Python code in /usr/lib/python2.7/unittest/loader.py, the tests are loaded from the module as normal. Then, if load_tests() exist, it is called as well.
def loadTestsFromModule(self, module, use_load_tests=True):
"""Return a suite of all tests cases contained in the given module"""
tests = []
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
load_tests = getattr(module, 'load_tests', None)
tests = self.suiteClass(tests)
if use_load_tests and load_tests is not None:
try:
return load_tests(self, tests, None)
except Exception, e:
return _make_failed_load_tests(module.__name__, e,
self.suiteClass)
return tests
So I guess my solution is to accept the extra test case that is created, check for it, and just pass it. Much thanks to Jeff for trying to help me out.

nose plugin for expected-failures

Is there an existing plugin which could be used like:
#nose.plugins.expectedfailure
def not_done_yet():
a = Thingamajig().fancynewthing()
assert a == "example"
If the test fails, it would appear like a skipped test:
$ nosetests
...S..
..but if it unexpected passes, it would appear similarly to a failure, maybe like:
=================================
UNEXPECTED PASS: not_done_yet
---------------------------------
-- >> begin captured stdout << --
Things and etc
...
Kind of like SkipTest, but not implemented as an exception which prevents the test from running.
Only thing I can find is this ticket about supporting the unittest2 expectedFailure decorator (although I'd rather not use unittest2, even if nose supported it)

I don't know about a nose plugin, but you could easily write your own decorator to do that. Here's a simple implementation:
import functools
import nose
def expected_failure(test):
#functools.wraps(test)
def inner(*args, **kwargs):
try:
test(*args, **kwargs)
except Exception:
raise nose.SkipTest
else:
raise AssertionError('Failure expected')
return inner
If I run these tests:
#expected_failure
def test_not_implemented():
assert False
#expected_failure
def test_unexpected_success():
assert True
I get the following output from nose:
tests.test.test_not_implemented ... SKIP
tests.test.test_unexpected_success ... FAIL
======================================================================
FAIL: tests.test.test_unexpected_success
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\nose-1.1.2-py3.2.egg\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "G:\Projects\Programming\dt-tools\new-sanbi\tests\test.py", line 16, in inner
raise AssertionError('Failure expected')
AssertionError: Failure expected
----------------------------------------------------------------------
Ran 2 tests in 0.016s
FAILED (failures=1)

Forgive me if I've misunderstood, but isn't the behavior you desire provided by core python's unittest library with the expectedFailure decorator, which is—by extension—compatible with nose?
For an example of use see the docs and a post about its implementation.

You can do it with one of two ways:
nose.tools.raises decorator
from nose.tools import raises
#raises(TypeError)
def test_raises_type_error():
raise TypeError("This test passes")
nose.tools.assert_raises
from nose.tools import assert_raises
def test_raises_type_error():
with assert_raises(TypeError):
raise TypeError("This test passes")
Tests will fail if exception is not raised.
Yup, I know, asked 3 years ago :)