I am having a testsuite which is a class and each testmethod is a testcase. I am also having a fixture for each method (testcase) which does some basic things. I am having a strange requirement where I want to skip all the testcases in the test suite if testfixture fails.
below is the sample suite I am having
class TestFeature:
#pytest.fixture(scope="function", autouse=True)
def _wrapper(self):
print("Some precomditions")
yield
print("some post conditions")
def test_first_testcase(self):
print("First testcas")
def test_second_testcase(self):
print("second testcas")
def test_third_testcase(self):
print("Third testcas")
Now I want if my fixture fails, I want to abort this test suite. Suppose My first testcase failed. Fixture for next test case failed. I want to abort the suite and do not want to execute third testcase.
How to do this in pytest
pytest allows you to do it by using the x flag:
pytest -x
You can also use maxfail flag if you wish to customize the number of failures before quitting:
--maxfail=NUMBER
In case you need to exit the test suite from within your code, you can use the exit method:
import pytest
pytest.exit()
Related
I am trying to figure out how to write a pytest plugin that can be used to fail a test after it has been run (for anyone who wants more context, this is related to astropy/pytest-openfiles#28). Let's consider the following simple test file:
class TestClass:
def setup_method(self, method):
print("In setup_method")
def teardown_method(self, method):
print("In teardown_method")
def test_simple(self):
print("In test")
I can now define a conftest.py file that contains:
def pytest_runtest_teardown(item, nextitem):
print("In pytest_runtest_teardown")
In this hook, I can carry out checks - for example in the case I'm interested in, we are checking for unclosed file handles. However, the issue is that this hook gets run after setup_method and after the test itself (test_simple) but before teardown_method:
% pytest test.py -vs
...
collected 1 item
test.py::TestClass::test_simple In setup_method
In test
PASSEDIn pytest_runtest_teardown
In teardown_method
I have considered instead using:
def pytest_runtest_makereport(item, call):
if call.when != 'teardown':
return
print("In pytest_runtest_makereport")
which does get executed after teardown_method but at that point if I raise an exception pytest will output an INTERNALERROR and will still consider the test as successful.
Is there any way to fail/error a test after teardown_method has been called?
After test_simple is over Pytest consider it done, everything from now own will be outside the test scope. You could add #pytest.fixture annotation to a function, this will provide you with pytest setup and teardown functionality. The test itself will be considered ass passed, but not the all suite. TestClass will be marked as failure with the exception raised from teardown_method
class TestClass:
#pytest.fixture
def run_for_test(self):
self.setup_method()
yield
self.teardown_method()
def setup_method(self, method):
print("In setup_method")
def teardown_method(self, method):
print("In teardown_method")
# just for the example
if not method:
raise AssertionError
def test_simple(self):
print("In test")
Output
In setup_method
.In test
In teardown_method
> raise AssertionError
E AssertionError
I have the following as conftest.py -->
def pytest_addoption(parser):
parser.addoption("--browser")
parser.addoption("--osType", help="Type of operating system")
parser.addoption("--hostURL", action="store", help="prod, stage, or dev")
#pytest.fixture(scope="session")
def browser(request):
return request.config.getoption("--browser")
#pytest.fixture(scope="session")
def osType(request):
return request.config.getoption("--osType")
#pytest.fixture(autouse=True)
def hostURL(request):
return request.config.getoption("--hostURL")
I would like to use the --hostURL flag to pass in value such as prod, stage or dev.
Here's how my test_TheMainTest.py looks -->
import unitest
import pytest
class CheckStatusCodeTest(unittest.TestCase, LoginPage, CustomSeleniumDriver):
def test_CheckStatusCodeOfPages(self, hostURL):
self.login(hostURL)
When I run the above test using pytest -q -s --hostURL prod I get the following error -->
TypeError: test_CheckStatusCodeOfCRPages() missing 1 required positional argument: 'hostURL'
Quoting the docs:
Note
unittest.TestCase methods cannot directly receive fixture arguments as implementing that is likely to inflict on the ability to run general unittest.TestCase test suites.
However, you can still pass regular fixture values to unittest-style tests using autouse fixtures:
class CheckStatusCodeTest(unittest.TestCase):
#pytest.fixture(autouse=True)
def _pass_fixture_value(self, hostURL):
self._hostURL = hostURL
def test_CheckStatusCodeOfPages(self):
assert self._hostURL
You can also check out this answer of mine tackling the same issue for more examples.
Another possibility is to implement an autouse fixture that modifies the test class explicitly. This is useful if you have lots of test classes that should have the identical setup:
#pytest.fixture(scope="class")
def set_fixture_value_to_class(request, hostURL):
request.cls._hostURL = hostURL
#pytest.mark.usefixtures('set_fixture_value_to_class')
class CheckStatusCodeTest(unittest.TestCase):
def test_CheckStatusCodeOfPages(self):
assert self._hostURL
#pytest.mark.usefixtures('set_fixture_value_to_class')
class AnotherTest(unittest.TestCase):
def test_spam(self):
assert self._hostURL
In this case, no need to copy the same fixture to each test class. Just mark all relevant test classes and you're good to go.
I have test cases in unittest framework. Have integrated them with teamcity.
Teamcity gives success even if my testcase fail. I Think, it is so since my unittest process exit with code 0 which is success for teamcity. I have integrated report, which show test failure. What needs to be done, so that teamcity also show failure if test fails.
code:
import unittest
class Login(unittest.TestCase):
def test_002_login_failure(self):
assert 1==2
You should prefer to use the unittest assertions so for example
self.assertEqual(1, 2)
Also you can use the teamcity-messages module to emit test messages the TeamCity will handle appropriately. Then you could change your main (if you have one) to something like the following
import teamcity
from teamcity.unittestpy import TeamcityTestRunner
if __name__ == '__main__':
if teamcity.is_running_under_teamcity():
runner = TeamcityTestRunner()
else:
runner = unittest.TextTestRunner()
unittest.main(testRunner=runner)
How to execute setup and teardown functions once for all nosetests tests?
def common_setup():
#time consuming code
pass
def common_teardown():
#tidy up
pass
def test_1():
pass
def test_2():
pass
#desired behavior
common_setup()
test_1()
test_2()
common_teardown()
Note that there exists a similar question with an answer that is not working with python 2.7.9-1, python-unittest2 0.5.1-1 and python-nose 1.3.6-1 after replacing the dots with pass and adding the line import unittest.
Unfortunately my reputation is too low to comment on that.
You can have a module level setup function. According to nose documentation:
Test modules offer module-level setup and teardown; define the method
setup, setup_module, setUp or setUpModule for setup, teardown,
teardown_module, or tearDownModule for teardown.
So, more specifically, for your case:
def setup_module():
print "common_setup"
def teardown_module():
print "common_teardown"
def test_1():
print "test_1"
def test_2():
print "test_2"
Running test gives you:
$ nosetests common_setup_test.py -s -v
common_setup
common_setup_test.test_1 ... test_1
ok
common_setup_test.test_2 ... test_2
ok
common_teardown
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
It does not matter which name you choose, so both setup and setup_module would work the same, but setup_module has more clarity to it.
In a hobby project I intend to use nose for testing, I want to put all tests for specific classes into classes since these tests share setup and other functionality. But I can't seem to get nose to execute the setup methods inside the classes.
Here is an example class that is tested:
class mwe():
def __init__(self):
self.example = ""
def setExample(self, ex):
self.example = ex
The tests work, when I don't use classes:
from nose.tools import ok_
import mwe
exampleList = []
def setUp():
print("setup")
exampleList.append("1")
exampleList.append("2")
exampleList.append("3")
def test_Example():
print("test")
for ex in exampleList:
t = mwe.mwe()
t.setExample(ex)
yield check, t, ex
def check(e, ex):
ok_(e.example == ex)
The output is as expected:
setup
test
...
----------------------------------------------------------------------
Ran 3 tests in 0.004s
OK
When a test class is used, the setup method is not executed and therefore no tests are executed.
from nose.tools import ok_
import mwe
class TestexampleClass(object):
def __init__(self):
print("__init__")
self.exampleList = []
def setup(self):
print("setup class")
self.exampleList.append("1")
self.exampleList.append("2")
self.exampleList.append("3")
def test_ExampleClass(self):
print("test class")
for ex in self.exampleList:
t = mwe.mwe()
t.setExample(ex)
yield self.check, t, ex
def check(self, we, ex):
print("check class")
ok_(we.example == ex)
I'm fairly new to python and a newbie with nose, my question is, why is setup not executed? Where is the error in my code?
__init__
test class
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
I will be glad for any feedback.
When I use the code from this Question on SO the setup method is executed, as I would expect it.
SOLUTION: After a lot of desperation I found the following:
Nose executes the the class level setup method before the execution of the yielded function, not when the test_* methods are called, as I expected and as is the case for other test_* methods. This obviously goes against the nose documentation:
Setup and teardown functions may be used with test generators. However, please note that setup and teardown attributes attached to the generator function will execute only once. To execute fixtures for each yielded test, attach the setup and teardown attributes to the function that is yielded, or yield a callable object instance with setup and teardown attributes.
Looking at the bug reports, I found the bug report on github.
A possible workaround is to use class level fixtures:
#classmethod
def setup_class(cls):
#do stuff
pass
Your test class needs to extend TestCase and the setup method needs to be called setUp
from unittest import TestCase
class TestUtils(TestCase):
def setUp(self):
self.x = 1
def test_something(self):
self.assertEqual(1, self.x)
which outputs
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK