I am writing a test framework using pytest. Is there a way to get testcase object in classes other than testcase. For example utility classes.
I want to print the testcase name and some markers for the test in utility classes. Are these information available in some contextmanager?
You cannot directly access pytest test properties if you are not inside a test fixture or a hook function, as there is no fixed test case class as in unittest. Your best bet is probably to get this information in a fixture and store it globally for access from a utility function:
testinfo={}
#pytest.fixture(autouse=True)
def test_info(request):
global testinfo
testinfo['name'] = request.node.name
testinfo['markers'] = [m.name for m in request.node.iter_markers()]
...
yield # the information is stored at test start...
testinfo = {} # ... and removed on test teardown
def utility_func():
if testinfo:
print(f"Name: {testinfo['name']} Markers: {testinfo['markers']}")
...
Or, the same if you use a test class:
class TestSomething:
def setup_method(self):
self.testinfo = {}
#pytest.fixture(autouse=True)
def info(self, request):
self.testinfo['name'] = request.node.name
self.testinfo['markers'] = [m.name for m in
request.node.iter_markers()]
yield # the information is stored at test start...
self.testinfo = {} # ... and removed on test teardown
def utility_func(self):
if self.testinfo:
print(f"Name: {self.testinfo['name']} Markers:"
f" {self.testinfo['markers']}")
#pytest.mark.test_marker
def test_something(self):
self.utility_func()
assert True
This will show the output:
Name: test_something Markers: ['test_marker']
This will work if you call the utility function during test execution - otherwise no value will be set.
Note however that this will only work reliably if you execute the test synchronously. If using pytest-xdist or similar tools for asynchronous test execution, this may not work due to the testinfo variable being overwitten by another test (though that depends on the implementation - it may work, if the variables are copied during a test run). In that case you can to do the logging directly in the fixture or hook function (which may generally be a better idea, depending on your use case).
For more information about available test node properties you can check the documentation of a request node.
Related
So, I have fixtures defined in conftest.py file with scope="class" as I want to run them before each test class is invoked. The conftest file is placed inside project root directory for it to be visible to every test module.
Now in one of the test modules, I have another setup function which I want to run once for that module only. But the problem is setup_class() method is called before running fixtures defined in conftest.py. Is this expected? I wanted it to be opposite because I want to use something done in the fixtures defined in conftest. How to do that?
Code -
conftest.py:
#pytest.fixture(scope="class")
def fixture1(request):
#set a
#pytest.fixture(scope="class")
def fixture1(request):
test_1.py:
#pytest.mark.usefixtures("fixture_1", "fixture_2")
class Test1():
#need this to run AFTER the fixture_1 & fixture_2
def setup_class():
#setup
#get a set in fixture_1
def test_1()
.....
I know that I could simply define a fixture in the test file instead of setup_class but then I will have to specify it in arguments of every test method in order it to be invoked by pytest. But suggestions are welcome!
I have exactly the same problem. Only now I have realized that the problem might be taht the setup_class is called before the fixture >-/
I think that this question is similar to this one
Pytest - How to pass an argument to setup_class?
And the problem is mixing the unittest and pytest methods.
I kind of did what they suggested - I ommitted the setup_class and created a new fixture within the particular test file,
calling the fixture in the conftest.py.
It works so far.
M.
The problem is that you can use the result of a fixture only in test function (or method) which is run by pytest. Here I can suggest a workaround. But of course I'm not sure if it suites your needs.
The workaround is to call the function from a test method:
conftest.py
#pytest.fixture(scope='class')
def fixture1():
yield 'MYTEXT'
test_1.py
class Test1:
def setup_class(self, some_var):
print(some_var)
def test_one(self, fixture1):
self.setup_class(fixture1)
Fixtures and setup_class are two different paradigms to initialize test functions (and classes). In this case, mixing the two creates a problem: The class-scoped fixtures run when the individual test functions (methods) run. On the other hand, setup_class runs before they do. Hence, it is not possible to access a fixture value (or fixture-modified state) from setup_class.
One of the solutions is to stop using setup_class entirely and stick with a fixtures-only solution which is the preferred way in pytest nowadays (see the note at the beginning).
# conftest.py or the test file:
#pytest.fixture(scope="class")
def fixture_1(request):
print('fixture_1')
# the test file:
class Test1():
#pytest.fixture(scope="class", autouse=True)
def setup(self, fixture_1, request):
print('Test1.setup')
def test_a(self):
print('Test1.test_a')
def test_b(self):
print('Test1.test_b')
Note that the setup fixture depends on fixture_1 and hence can access it.
This test session works properly :
from myapp.models import MyModel
#pytest.fixture(scope='function')
def mymodel():
return G(MyModel)
#pytest.mark.django_db
def test_mymodel_one(mymodel):
assert MyModel.objects.count() > 0
#pytest.mark.django_db
def test_mymodel_two(mymodel):
assert MyModel.objects.count() > 0
and produces this output:
========= test session starts =========
tests/myapp/test_pp.py::test_mymodel_one PASSED
tests/myapp/test_pp.py::test_mymodel_two PASSED
but if I change the scope of my fixture to 'session', test two fails:
========= test session starts =========
tests/myapp/test_pp.py::test_mymodel_one PASSED
tests/myapp/test_pp.py::test_mymodel_two FAILED
========= FAILURES ==============
_________ test_mymodel_two ________
tests/myapp/test_pp.py:85: in test_mymodel_two
assert MyModel.objects.count() > 0
E assert 0 > 0
The object created is correctly returned from fixture (I can access his values) but it isn't stored no more.
How can I use session scope and maintain the storage in my test db?
I tried to replicate the context in my tests package and I've found the same situation you exposed.
First of all, I want to share with you two page of the pytest documentation, where we could find the answer to this question.
In the documentation¹ the organization of the methods is a little bit different, in fact the method which is delegated to create the fixture is inside the conftest.py .
# content of conftest.py
#pytest.fixture(scope="session")
def smtp(...):
# the returned fixture value will be shared for
# all tests needing it
According to your tests settings, you can try moving the mymodel method inside the conftest module. I've tried to move my fixture generator inside the conftest file, but I found several traversal problems due to the required django_db mark, which might conflict with the session scope (I guess?).
I also found another help inside the examples page² of pytest, where is used the session scope widely for different python modules, where is pointed out the impossibility to an internal module test to access to the same session scope defined at the same level of the parents' one.
# content of a/conftest.py
import pytest
class DB:
pass
#pytest.fixture(scope="session")
def db():
return DB()
# content of a/test_db.py
def test_a1(db):
assert 0, db # to show value
# content of a/test_db2.py
def test_a2(db):
assert 0, db # to show value
if
# content of b/test_error.py
def test_root(db): # no db here, will error out
pass
will fail the test, because
The two test modules in the a directory see the same db fixture instance while the one test in the sister-directory b doesn’t see it. We could of course also define a db fixture in that sister directory’s conftest.py file. Note that each fixture is only instantiated if there is a test actually needing it (unless you use “autouse” fixture which are always executed ahead of the first test executing).
In the second example I've noticed that the method for generating the fixture is instantiated for each test and even though the scope is set to session or module, it works like function scope.
Which version of pytest are you using?
Could try moving from the current module to the conftest module your mymodel method?
I want to run additional setup and teardown checks before and after each test in my test suite. I've looked at fixtures but not sure on whether they are the correct approach. I need to run the setup code prior to each test and I need to run the teardown checks after each test.
My use-case is checking for code that doesn't cleanup correctly: it leaves temporary files. In my setup, I will check the files and in the teardown I also check the files. If there are extra files I want the test to fail.
py.test fixtures are a technically adequate method to achieve your purpose.
You just need to define a fixture like that:
#pytest.fixture(autouse=True)
def run_around_tests():
# Code that will run before your test, for example:
files_before = # ... do something to check the existing files
# A test function will be run at this point
yield
# Code that will run after your test, for example:
files_after = # ... do something to check the existing files
assert files_before == files_after
By declaring your fixture with autouse=True, it will be automatically invoked for each test function defined in the same module.
That said, there is one caveat. Asserting at setup/teardown is a controversial practice. I'm under the impression that the py.test main authors do not like it (I do not like it either, so that may colour my own perception), so you might run into some problems or rough edges as you go forward.
You can use a fixture in order to achieve what you want.
import pytest
#pytest.fixture(autouse=True)
def run_before_and_after_tests(tmpdir):
"""Fixture to execute asserts before and after a test is run"""
# Setup: fill with any logic you want
yield # this is where the testing happens
# Teardown : fill with any logic you want
Detailed Explanation
#pytest.fixture(autouse=True), from the docs: "Occasionally, you may want to have fixtures get invoked automatically without declaring a function argument explicitly or a usefixtures decorator." Therefore, this fixture will run every time a test is executed.
# Setup: fill with any logic you want, this logic will be executed before every test is actually run. In your case, you can add your assert statements that will be executed before the actual test.
yield, as indicated in the comment, this is where testing happens
# Teardown : fill with any logic you want, this logic will be executed after every test. This logic is guaranteed to run regardless of what happens during the
tests.
Note: in pytest there is a difference between a failing test and an error while executing a test. A Failure indicates that the test failed in some way.
An Error indicates that you couldn't get to the point of doing a proper test.
Consider the following examples:
Assert fails before test is run -> ERROR
import pytest
#pytest.fixture(autouse=True)
def run_around_tests():
assert False # This will generate an error when running tests
yield
assert True
def test():
assert True
Assert fails after test is run -> ERROR
import pytest
#pytest.fixture(autouse=True)
def run_around_tests():
assert True
yield
assert False
def test():
assert True
Test fails -> FAILED
import pytest
#pytest.fixture(autouse=True)
def run_around_tests():
assert True
yield
assert True
def test():
assert False
Test passes -> PASSED
import pytest
#pytest.fixture(autouse=True)
def run_around_tests():
assert True
yield
assert True
def test():
assert True
Fixtures are exactly what you want.
That's what they are designed for.
Whether you use pytest style fixtures, or setup and teardown (module, class, or method level) xUnit style fixtures, depends on the circumstance and personal taste.
From what you are describing, it seems like you could use pytest autouse fixtures.
Or xUnit style function level setup_function()/teardown_function().
Pytest has you completely covered. So much so that perhaps it's a fire hose of information.
You can use Module level setup/teardown Fixtures of Pytest.
Here's the Link
http://pytest.org/latest/xunit_setup.html
It Works as follows:
def setup_module(module):
""" setup any state specific to the execution of the given module."""
def teardown_module(module):
""" teardown any state that was previously setup with a setup_module
method."""
Test_Class():
def test_01():
#test 1 Code
It will call setup_module before this test and teardown_module after test completes.
You can include this fixture in each test-script to run it for each test.
IF you want to use something that is common to all tests in a directory You can use package/directory level fixtures nose framework
http://pythontesting.net/framework/nose/nose-fixture-reference/#package
In __init__.py file of the package you can include following
def setup_package():
'''Set up your environment for test package'''
def teardown_package():
'''revert the state '''
You may use decorators but programatically, so you don't need to put the decorator in each method.
I'm assuming several things in next code:
The test methods are all named like: "testXXX()"
The decorator is added to the same module where test methods are implemented.
def test1():
print ("Testing hello world")
def test2():
print ("Testing hello world 2")
#This is the decorator
class TestChecker(object):
def __init__(self, testfn, *args, **kwargs):
self.testfn = testfn
def pretest(self):
print ('precheck %s' % str(self.testfn))
def posttest(self):
print ('postcheck %s' % str(self.testfn))
def __call__(self):
self.pretest()
self.testfn()
self.posttest()
for fn in dir() :
if fn.startswith('test'):
locals()[fn] = TestChecker(locals()[fn])
Now if you call the test methods...
test1()
test2()
The output should be something like:
precheck <function test1 at 0x10078cc20>
Testing hello world
postcheck <function test1 at 0x10078cc20>
precheck <function test2 at 0x10078ccb0>
Testing hello world 2
postcheck <function test2 at 0x10078ccb0>
If you have test methods as class methods, the approach is also valid. For instance:
class TestClass(object):
#classmethod
def my_test(cls):
print ("Testing from class method")
for fn in dir(TestClass) :
if not fn.startswith('__'):
setattr(TestClass, fn, TestChecker(getattr(TestClass, fn)))
The call to TestClass.my_test() will print:
precheck <bound method type.my_test of <class '__main__.TestClass'>>
Testing from class method
postcheck <bound method type.my_test of <class '__main__.TestClass'>>
It is an old question but I personally found another way from the docs :
Use the pytest.ini file :
[pytest]
usefixtures = my_setup_and_tear_down
import pytest
#pytest.fixture
def my_setup_and_tear_down():
# SETUP
# Write here the logic that you need for the setUp
yield # this statement will let the tests execute
# TEARDOWN
# Write here the logic that you need after each tests
About the yield statement and how it allows to run the test : HERE
Fixtures by default have scope=function. So, if you just use a definition such as
#pytest.fixture
def fixture_func(self)
It defaults to (scope='function').
So any finalizers in the fixture function will be called after each test.
I have my test class as follows:
class MyTestCase(django.test.TestCase):
def setUp(sefl):
# set up stuff common to ALL the tests
#my_test_decorator('arg1')
#my_test_decorator('arg2')
def test_something_1(self):
# run test
def test_something_2(self):
# run test
#my_test_decorator('arg1')
#my_test_decorator('arg2')
def test_something_3(self):
# run test
...
def test_something_N(self):
# run test
Now, #my_test_decorator is a decorator I made that performs internal changes to setup some changes to the test environment at runtime and undo such changes after the test finished, but I need to do this to a specific set of test cases only, not to ALL of them and I would like to keep the setup general to all the tests and, for the specific tests, maybe do something like this:
def setUp(self):
# set up stuff common to ALL the tests
tests_to_decorate = ['test_something_1', 'test_something_3']
decorator_args = ['arg1', 'arg2']
if self._testMethodNamein in tests_to_decorate:
method = getattr(self, self._testMethodNamein)
for arg in decorator_args:
method = my_test_decorator(arg)(method)
setattr(self, self._testMethodNamein, method)
I mean, without repeating the decorators all over the file, but it seems that the test runner retrieves the set of tests to run even before instantiating the test class and thus is of no use doing this in the __init__ or setUp methods.
It would be nice to have a way to accomplish this without:
having to write my own test runner
needing to split the tests in two or more TestCase subclasses
repeating setUp in different classes
creating a class that hosts the setUp method and have the TestCase subclasses inherit from such class
Is this even possible?
Thanks!! :)
I am unit testing mercurial integration and have a test class which currently creates a repository with a file and a clone of that repository in its setUp method and removes them in its tearDown method.
As you can probably imagine, this gets quite performance heavy very fast, especially if I have to do this for every test individually.
So what I would like to do is create the folders and initialize them for mercurial on loading the class, so each and every unittest in the TestCase class can use these repositories. Then when all the tests are run, I'd like to remove them. The only thing my setUp and tearDown methods then have to take care of is that the two repositories are in the same state between each test.
Basically what I'm looking for is a python equivalent of JUnit's #BeforeClass and #AfterClass annotations.
I've now done it by subclassing the TestSuite class, since the standard loader wraps all the test methods in an instance of the TestCase in which they're defined and puts them together in a TestSuite. I have the TestSuite call the before() and after() methods of the first TestCase. This of course means that you can't initialize any values to your TestCase object, but you probably want to do this in your setUp anyway.
The TestSuite looks like this:
class BeforeAfterSuite(unittest.TestSuite):
def run(self, result):
if len(self._tests) < 1:
return unittest.TestSuite.run(self, result)
first_test = self._tests[0]
if "before" in dir(first_test):
first_test.before()
result = unittest.TestSuite.run(self, result)
if "after" in dir(first_test):
first_test.after()
return result
For some slightly more finegrained control I also created the custom TestLoader which makes sure the BeforeAfterSuite is only used to wrap test-method-TestCase objects in, which looks like this:
class BeforeAfterLoader(unittest.TestLoader):
def loadTestsFromTestCase(self, testCaseClass):
self.suiteClass = BeforeAfterSuite
suite = unittest.TestLoader.loadTestsFromTestCase(self, testCaseClass)
self.suiteClass = unittest.TestLoader.suiteClass
return suite
Probably missing here is a try/except block around the before and after which could fail all the testcases in the suite or something.
from the Python unittest documentation :
setUpClass() :
A class method called before tests in an individual class run. setUpClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def setUpClass(cls):
...
New in version 2.7.
tearDownClass() :
A class method called after tests in an individual class have run. tearDownClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def tearDownClass(cls):
...