Is there an existing plugin which could be used like:
#nose.plugins.expectedfailure
def not_done_yet():
a = Thingamajig().fancynewthing()
assert a == "example"
If the test fails, it would appear like a skipped test:
$ nosetests
...S..
..but if it unexpected passes, it would appear similarly to a failure, maybe like:
=================================
UNEXPECTED PASS: not_done_yet
---------------------------------
-- >> begin captured stdout << --
Things and etc
...
Kind of like SkipTest, but not implemented as an exception which prevents the test from running.
Only thing I can find is this ticket about supporting the unittest2 expectedFailure decorator (although I'd rather not use unittest2, even if nose supported it)
I don't know about a nose plugin, but you could easily write your own decorator to do that. Here's a simple implementation:
import functools
import nose
def expected_failure(test):
#functools.wraps(test)
def inner(*args, **kwargs):
try:
test(*args, **kwargs)
except Exception:
raise nose.SkipTest
else:
raise AssertionError('Failure expected')
return inner
If I run these tests:
#expected_failure
def test_not_implemented():
assert False
#expected_failure
def test_unexpected_success():
assert True
I get the following output from nose:
tests.test.test_not_implemented ... SKIP
tests.test.test_unexpected_success ... FAIL
======================================================================
FAIL: tests.test.test_unexpected_success
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\nose-1.1.2-py3.2.egg\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "G:\Projects\Programming\dt-tools\new-sanbi\tests\test.py", line 16, in inner
raise AssertionError('Failure expected')
AssertionError: Failure expected
----------------------------------------------------------------------
Ran 2 tests in 0.016s
FAILED (failures=1)
Forgive me if I've misunderstood, but isn't the behavior you desire provided by core python's unittest library with the expectedFailure decorator, which is—by extension—compatible with nose?
For an example of use see the docs and a post about its implementation.
You can do it with one of two ways:
nose.tools.raises decorator
from nose.tools import raises
#raises(TypeError)
def test_raises_type_error():
raise TypeError("This test passes")
nose.tools.assert_raises
from nose.tools import assert_raises
def test_raises_type_error():
with assert_raises(TypeError):
raise TypeError("This test passes")
Tests will fail if exception is not raised.
Yup, I know, asked 3 years ago :)
Related
I am new to Python. I wanted to test if my code threw an exception. I got the code from How do you test that a Python function throws an exception?
import mymod
import unittest
class MyTestCase(unittest.TestCase):
def test1(self):
self.assertRaises(SomeCoolException, mymod.myfunc, compulsory_argument)
Now, I also want to display a message if the exception is not thrown. How do I do that? The Python documentation does not mention it clearly. I added the message after "compulsory_argument" and it failed.
I tried the first answer with modifications and got an exception. What is my mistake here?
import unittest
def sayHelloTo(name):
print("Hello " + name)
class MyTestCase(unittest.TestCase):
def test1(self):
person = "John"
with self.assertRaises(Exception, "My insightful message"):
sayHelloTo(person)
Error:
Error
Traceback (most recent call last):
File "C:\tests\tester.py", line 9, in test1
with self.assertRaises(Exception, "My insightful message"):
AttributeError: __exit__
As of Python 3.3, assertRaises can be used as a context manager with a message:
import unittest
def sayHelloTo(name):
print("Hello " + name)
class MyTestCase(unittest.TestCase):
def test1(self):
person = "John"
with self.assertRaises(Exception, msg="My insightful message"):
sayHelloTo(person)
if __name__ == "__main__":
unittest.main()
It results in
Hello John
F
======================================================================
FAIL: test1 (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "r.py", line 10, in test1
sayHelloTo(person)
AssertionError: Exception not raised : My insightful message
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
Now, I also want to display a message if the exception is not thrown. How do I do that ?
The general philosophy of unittest is for the tests to be silent when they succeed and only become verbose when they fail. Accordingly, the API provides a "msg" keyword argument for the unsuccessful case but does not offer an alternative for the successful case.
That said, another part of the philosophy works in your favor. In general, test cases internally raise an exception when a testcase fails. That means that if you want to display a message when there is a success, you just add another statement after the test:
with self.assertRaises(TypeError, msg='Oh no, I did not get a TypeError')
somecode()
logging.info('Yippee, we got a TypeError!') # Runs after a successful test
(Python 3.4.0)
I got this strange error, which took me a while to debug:
user.py
class User:
def __init__(self, name):
self.name = name
def new_user(name):
user = User(name)
test.py
import unittest
from unittest.mock import Mock, patch
from user import new_user
#patch('user.User')
class TestUser(unittest.TestCase):
#unittest.skip
def test_new_user(self, mockUser):
new_user('Frank')
mockUser.assert_called_once_with('Frank')
unittest.main()
Running it will crash:
» python test.py
E
======================================================================
ERROR: test_new_user (__main__.TestUser)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/unittest/mock.py", line 1125, in patched
return func(*args, **keywargs)
TypeError: decorator() takes 1 positional argument but 2 were given
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
Removing the skip will let it run normally. It seems patch and skip do not stack well. Is this correct, or am I doing something stupid?
unittest.skip requires a string argument of its own, the reason for skipping the test.
#unittest.skip("Not yet ready to test")
def test_new_user(self, mockUser):
new_user('Frank')
mockUser.assert_called_once_with('Frank')
The interaction you are seeing comes from the skip decorator consuming the method itself as the reason argument (def skip(reason):), which results in test_new_user being bound to a one-argument function defined inside the decorator, not the two-argument function you define in the test case.
Note that if you left your call to skip in place and commented out the patch instead, your test would still pass, despite test_new_user seemingly not receiving its mockUser argument.
unittest.skip itself is technically not a decorator; it is a function which returns a decorator, which is then applied to test_new_user. Using regular function-call syntax, your code does
def test_new_user(self, mockUser):
...
test_new_user = unittest.skip(test_new_user)
when what you need is
test_new_user = unittest.skip("my reason")(test_new_user)
Your test_new_user is being bound to the decorator itself, not the decorated method.
I have a test case with a helper method assertContains(super, sub). The sub arguments are a hard-coded part of the test cases. In case they're malformed, I would like my test case to abort with an error.
How do I do that? I have tried
def assertContains(super, sub):
if isinstance(super, foo): ...
elif isinstance(super, bar): ...
else: assert False, repr(sub)
However, this turns the test into a failure rather than an error.
I could raise some other exception (e.g. ValueError), but I want to explicitly state that I'm declaring the test case to be in error. I could do things like ErrorInTest = ValueError and then raise ErrorInTest(repr(sub)), but it feels kinda' icky. I feel there should be a batteries-included way of doing this, but reading the friendly manual didn't suggest anything to me.
There is an assertRaises() for aspects in class TestCase in which you want to ensure an error is raised by the to-be-tested code.
If you want to raise an error and abort testing that unit at this point (and continue with the next unit test), just raise an uncaught exception; the unit test module will catch it:
raise NotImplementedError("malformed sub: %r" % (sub,))
I don't think that there is any other API aspect available besides raising errors directly to state that a unit test case results in an error.
class PassingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
class FailingTest(unittest.TestCase):
def runTest(self):
self.assertTrue(False)
class ErrorTest(unittest.TestCase):
def runTest(self):
raise NotImplementedError("error")
class PassingTest2(unittest.TestCase):
def runTest(self):
self.assertTrue(True)
results in:
EF..
======================================================================
ERROR: runTest (__main__.ErrorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 15, in runTest
raise NotImplementedError("error")
NotImplementedError: error
======================================================================
FAIL: runTest (__main__.FailingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./t.py", line 11, in runTest
self.assertTrue(False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 4 tests in 0.002s
FAILED (failures=1, errors=1)
I have a tests file in one of my Pyramid projects. It has one suite with six tests in it:
...
from .scripts import populate_test_data
class FunctionalTests(unittest.TestCase):
def setUp(self):
settings = appconfig('config:testing.ini',
'main',
relative_to='../..')
app = main({}, **settings)
self.testapp = TestApp(app)
self.config = testing.setUp()
engine = engine_from_config(settings)
DBSession.configure(bind=engine)
populate_test_data(engine)
def tearDown(self):
DBSession.remove()
tearDown()
def test_index(self):
...
def test_login_form(self):
...
def test_read_recipe(self):
...
def test_tag(self):
...
def test_dish(self):
...
def test_dashboard_forbidden(self):
...
Now, when I run nosetests templates.py (where templates.py is the mentioned file) I get the following output:
......E
======================================================================
ERROR: templates.populate_test_data
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yentsun/env/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/yentsun/env/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/util.py", line 622, in newfunc
return func(*arg, **kw)
TypeError: populate_test_data() takes exactly 1 argument (0 given)
----------------------------------------------------------------------
Ran 7 tests in 1.985s
FAILED (errors=1)
When I run the tests with test suite specified nosetests templates.py:FunctionalTests, the output is, as expected, ok:
......
----------------------------------------------------------------------
Ran 6 tests in 1.980s
OK
Why do I have different output and why is an extra (7th) test run?
UPDATE. Its a bit frustrating, but I when removed the word test from the name populate_test_data (it became populate_dummy_data), everything worked fine.
The problem is solved for now, but maybe somebody knows what went wrong here - why a function from setUp had been tested?
Finding and running tests
nose, by default, follows a few simple rules for test discovery.
If it looks like a test, it’s a test. Names of directories, modules, classes and functions are compared against the testMatch regular
expression, and those that match are considered tests. Any class that
is a unittest.TestCase subclass is also collected, so long as it is
inside of a module that looks like a test.
(from nose 1.3.0 documentation)
In the nose's code, the regexp is defined as r'(?:^|[\b_\.%s-])[Tt]est' % os.sep, and if you inpect nose/selector.py, method Selector.matches(self, name) you'll see that the code uses re.search, which looks for a match anywhere in the string, not only at the beginning, as re.match does.
A small test:
>>> import re
>>> import os
>>> testMatch = r'(?:^|[\b_\.%s-])[Tt]est' % os.sep
>>> re.match(testMatch, 'populate_test_data')
>>> re.search(testMatch, 'populate_test_data')
<_sre.SRE_Match object at 0x7f3512569238>
So populate_test_data indeed "looks like a test" by nose's standards.
I want to create a generator for variations of a TestCase-derived class.
What I tried is this:
import unittest
def create_class(param):
class Test(unittest.TestCase):
def setUp(self):
pass
def test_fail(self):
assert False
return Test
def test_basic():
for i in range(5):
yield create_class(i)
What I get is this:
======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.3/site-packages/nose/case.py", line 268, in setUp
try_run(self.test, names)
File "/usr/lib/python3.3/site-packages/nose/util.py", line 478, in try_run
return func()
TypeError: setUp() missing 1 required positional argument: 'self'
Yielding instances instead of classes (yield create_class(i)()) leaves me with this error:
======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/usr/lib/python3.3/unittest/case.py", line 492, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.3/unittest/case.py", line 423, in run
testMethod = getattr(self, self._testMethodName)
AttributeError: 'Test' object has no attribute 'runTest'
Any ideas?
When instantiating a TestCase you should pass the method name of the test:
yield create_class(i)('test_fail')
Otherwise the name defaults to runTest(and thus the last error you got).
Also note that there is a strange interaction between test generators and TestCase. With the following code:
import unittest
def create_class(param):
class Test(unittest.TestCase):
def setUp(self):
pass
def test_fail(self):
print('executed')
assert False
print('after assert')
return Test
def test_basic():
for i in range(5):
yield create_class(i)('test_fail')
I obtain this output:
$ nosetests -s
executed
.executed
.executed
.executed
.executed
.
----------------------------------------------------------------------
Ran 5 tests in 0.004s
OK
As you can see the test does not fail, even though the assert works. This is probably due to the fact that TestCase handles the AssertionError but nose does not expect this to be handled and thus it cannot see that the test failed.
This can be seen from the documentation of TestCase.run:
Run the test, collecting the result into the test result object passed as result. If result is omitted or None, a temporary result
object is created (by calling the defaultTestResult() method) and
used. The result object is not returned to run()‘s caller.
The same effect may be had by simply calling the TestCase instance.
So, nose doesn't see that the objected yielded by the generator is a TestCase which should be handled in a special manner, it simply expects a callable. The TestCase is run, but the result is put into a temporary object that is lost, and this eats all test failures that happen inside the tests. Hence yielding TestCasees simply doesn't work.
I have run the codes you provides. I received no error. The version I use is python2.7. System is ubuntu12.10. Maybe you need to check with python2.7.