I have some common code which are used in all my PyTest. I want to keep the complete code inside any file and load it during run time. Is it possible using Python ?
Eg:
def teardown_method(self, method):
print "This is tear down method"
def setup_method(self, method):
print "This is setup method"
Apart from simply using import, for the example you showed (setup/teardown), you'd usually use a pytest fixture instead, and put them in a conftest.py to share them between tests.
The nose discovery process finds all modules whose name starts with test, and within them all functions which have test in the name and tries to run them as unit tests. See http://nose.readthedocs.org/en/latest/man.html
I have a function whose name is say, make_test_account, in the file accounts.py. I want to test that function in a test module called test_account. So at the start of that file I do:
from foo.accounts import make_test_account
But now I find that nose treats the function make_test_account as a unit test and tries to run it (which fails because it doesn't pass in any parameters, which are required).
How can I make sure nose ignores that function specifically? I would prefer to do it in a way which means I can invoke nose as nosetests, without any command line arguments.
Tell nose that the function is not a test - use the nottest decorator.
# module foo.accounts
from nose.tools import nottest
#nottest
def make_test_account():
...
Nose has a nottest decorator. However, if you don't want to apply the #nottest decorator in the module you are importing from you can also simply modify the method after the import. It may be cleaner to keep unit test logic close to the unit test itself.
from foo.accounts import make_test_account
# prevent nose test from running this imported method
make_test_account.__test__ = False
You can still use nottest but it has the same effect:
from nose.tools import nottest
from foo.accounts import make_test_account
# prevent nose test from running this imported method
make_test_account = nottest(make_test_account)
I want to setup a test suite wherein I will read a json file in setup_class method and in that json file I will mention which tests should run and which tests should not run. So with this approach I can mention which test cases to run by altering the json file only and not touching the test suite.
But in the setup_class method when I try to do the following:-
class TestCPU:
testme=False
#classmethod
def setup_class(cls):
cls.test_core.testme=True
def test_core(self):
print 'Test CPU Core'
assert 1
Executing below command:-
nosetests -s -a testme
It gives following error:-
File "/home/murtuza/HWTestCert/testCPU.py", line 7, in setup_class
cls.test_core.testme=False
AttributeError: 'instancemethod' object has no attribute 'testme'
So, is it possible to set the attributes of test methods during setup_class?
The way it is defined, testme is a member of the TestCPU class, and the <unbound method TestCPU.test_core> has no idea about this attribute. You can inject nose attribute by using cls.test_core.__dict__['testme']=True. However, the attributes are checked before your setup_class method is called, so even though the attribute will be set, your test will be skipped. But you can certainly decorate your test with attributes on import, like this:
import unittest
class TestCPU(unittest.TestCase):
def test_core(self):
print 'Test CPU Core'
assert 1
TestCPU.test_core.__dict__['testme']=True
You may also want to try --pdb option to nosetests, it will bring out debugger on error so that you can dive in to see what is wrong. It is definitely my second favorite thing in life.
I am sure there are multiple ways to achieve this, but here is one way you can do it.
Inside your test class, create a method that reads in your JSON file and creates a global array for methods to be skipped for testing - skiptests
All you need to do now is use a setup decorator for every test case method in your suite. Within this decorator, check if the current function is in skiptests. If so, call nosetests' custom decorator nose.tools.nottest which is used to skip a test. Otherwise, return the function being tested.
To put it in code:
def setup_test_method1(func):
if func.__name__ not in skiptests:
return func
else:
return nose.tools.nottest(func)
#with_setup(setup_test_method1)
def test_method1():
pass
I have not tested this code, but I think we can invoke a decorator within another decorator. In which case this could work.
I have just starting using unittesting, I want some code to run for every test (which is how the default setUp() seems to work) but I want some code to just run once, right at the beginning of the run, is this possible?
You should define a setUpClass method.
You can use setUpClass which runs once to set up the test class. It runs before the test methods in that class are executed:
#classmethod
def setUpClass(cls):
pass
I'm extending the python 2.7 unittest framework to do some function testing. One of the things I would like to do is to stop all the tests from running inside of a test, and inside of a setUpClass() method. Sometimes if a test fails, the program is so broken it is no longer of any use to keep testing, so I want to stop the tests from running.
I noticed that a TestResult has a shouldStop attribute, and a stop() method, but I'm not sure how to get access to that inside of a test.
Does anyone have any ideas? Is there a better way?
In case you are interested, here is a simple example how you could make a decision yourself about exiting a test suite cleanly with py.test:
# content of test_module.py
import pytest
counter = 0
def setup_function(func):
global counter
counter += 1
if counter >=3:
pytest.exit("decided to stop the test run")
def test_one():
pass
def test_two():
pass
def test_three():
pass
and if you run this you get:
$ pytest test_module.py
============== test session starts =================
platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1
test path 1: test_module.py
test_module.py ..
!!!! Exit: decided to stop the test run !!!!!!!!!!!!
============= 2 passed in 0.08 seconds =============
You can also put the py.test.exit() call inside a test or into a project-specific plugin.
Sidenote: py.test natively supports py.test --maxfail=NUM to implement stopping after NUM failures.
Sidenote2: py.test has only limited support for running tests in the traditional unittest.TestCase style.
Here's another answer I came up with after a while:
First, I added a new exception:
class StopTests(Exception):
"""
Raise this exception in a test to stop the test run.
"""
pass
then I added a new assert to my child test class:
def assertStopTestsIfFalse(self, statement, reason=''):
try:
assert statement
except AssertionError:
result.addFailure(self, sys.exc_info())
and last I overrode the run function to include this right below the testMethod() call:
except StopTests:
result.addFailure(self, sys.exc_info())
result.stop()
I like this better since any test now has the ability to stop all the tests, and there is no cpython-specific code.
Currently, you can only stop the tests at the suite level. Once you are in a TestCase, the stop() method for the TestResult is not used when iterating through the tests.
Somewhat related to your question, if you are using python 2.7, you can use the -f/--failfast flag when calling your test with python -m unittest. This will stop the test at the first failure.
See 25.3.2.1. failfast, catch and buffer command line options
You can also consider using Nose to run your tests and use the -x, --stop flag to stop the test early.
In the test loop of unittest.TestSuite, there is a break condition at the start:
class TestSuite(BaseTestSuite):
def run(self, result, debug=False):
topLevel = False
if getattr(result, '_testRunEntered', False) is False:
result._testRunEntered = topLevel = True
for test in self:
if result.shouldStop:
break
So I am using a custom test suite like this:
class CustomTestSuite(unittest.TestSuite):
""" This variant registers the test result object with all ScriptedTests,
so that a failed Loign test can abort the test suite by setting result.shouldStop
to True
"""
def run(self, result, debug=False):
for test in self._tests:
test.result = result
return super(CustomTestSuite, self).run(result, debug)
with a custom test result class like this:
class CustomTestResult(TextTestResult):
def __init__(self, stream, descriptions, verbosity):
super(CustomTestResult, self).__init__(stream, descriptions, verbosity)
self.verbosity = verbosity
self.shouldStop = False
and my test classes are like:
class ScriptedTest(unittest.TestCase):
def __init__(self, environment, test_cfg, module, test):
super(ScriptedTest, self).__init__()
self.result = None
Under certain conditions, I then abort the test suite; for example, the test suite starts with a login, and if that fails, I do not have to try the rest:
try:
test_case.execute_script(test_command_list)
except AssertionError as e:
if test_case.module == 'session' and test_case.test == 'Login':
test_case.result.shouldStop = True
raise TestFatal('Login failed, aborting test.')
else:
raise sys.exc_info()
Then I use the test suite in the following way:
suite = CustomTestSuite()
self.add_tests(suite)
result = unittest.TextTestRunner(verbosity=self.environment.verbosity, stream=UnitTestLoggerStream(self.logger),
resultclass=CustomTestResult).run(suite)
I'm not sure if there is a better way to do it, but it behaves correctly for my tests.
Though you won't get the usual test reports of the tests run so far, a very easy way to stop the test run from within a TestCase method is simply to raise KeyboardInterrupt inside the method.
You can see how only KeyboardInterrupt is allowed to bubble up inside unittest's test runner by looking at CPython's code here inside testPartExecutor().
The OP was about python 2.7. Skip ahead a decade, and for python 3.1 and above, the way to skip tests in python unittest has had an upgrade, but the documentation could use some clarification (IMHO):
The documentation covers the following:
Skip All tests after first failure: use failfast (only useful if you really don't want to continue any further tests at all, including in other unrelated TestCase classes)
Skip All tests in a TestCase class: decorate class with #unittest.skip(), etc.
Skip a single method within a TestCase: decorate method with #unittest.skip(), etc.
Conditionally skip a method or a class: decorate with #unittest.skipIf() or #unittest.skipUnless() etc.
Conditionally skip a method, but not until something within that method runs: use self.skipTest() inside the method (this will skip that method, and ONLY that method, not subsequent methods)
The documentation does not cover the following (as of this writing):
Skip all tests within a TestCase class if a condition is met inside the setUpClass method: solution from this post raise unittest.SkipTest("skip all tests in this class") (there may be another way, but I'm unaware)
Skip all subsequent test methods in a TestCase class after a condition is met in one of the first tests, but still continue to test other unrelated TestCase classes. For this, I propose the following solution...
This solution assumes that you encounter the "bad state" in the middle of a test method, and which could only be noticed in a test method ONLY (i.e., it is not something that could have been determined in the setUpClass method, for whatever reason). Indeed the setUpClass method is the best location for determining whether to proceed if the initial conditions aren't right, but sometimes (as I've encountered) you just don't know until you run some test method. This solution assumes that test methods are in alphabetical order and that subsequent tests methods that you don't want to run after encountering the "bad" state follow alphabetically.
import unittest
class SkipMethodsConditionally(unittest.TestCase):
#classmethod
def setUpClass(cls):
#this class variable maintains whether or not test methods should continue
cls.should_continue = True
#this class variable represents the state of your system. Replace with function of your own
cls.some_bad_condition = False
def setUp(self) -> None:
"""setUp runs before every single test method in this class"""
if not self.__class__.should_continue:
self.skipTest("no reason to go on.")
def test_1_fail(self):
#Do some work here. Let's assume you encounter a "bad state,"" that could
#only be noticed in this first test method only, (i.e., it's not something that
#can be placed in the setUpClass method, for whatever reason)
self.__class__.some_bad_condition = True
if self.__class__.some_bad_condition:
self.__class__.should_continue = False
self.assertTrue(False,"this test should fail, rendering the rest of the tests irrelevant")
def test_2_pass(self):
self.assertFalse(self.__class__.some_bad_condition,"this test would pass normally if run, but should be skipped, because it would fail")
The above test will yield the following output:
test_1_fail (__main__.SkipMethodsConditionally) ... FAIL
test_2_pass (__main__.SkipMethodsConditionally) ... skipped 'no reason to go on.'
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1, skipped=1)
I looked at the TestCase class and decided to subclass it. The class just overrides run(). I copied the method and starting at line 318 in the original class added this:
# this is CPython specific. Jython and IronPython may do this differently
if testMethod.func_code.co_argcount == 2:
testMethod(result)
else:
testMethod()
It has some CPython specific code in there to tell if the test method can accept another parameter, but since I'm using CPython everywhere, this isn't an issue for me.
Use:
if condition:
return 'pass'