I am using pytest in a way in which there are two execution paths , one to configure the test environment (create some buildable folders, such as a c application that needs to be cross compiled and executed later in a distant target), and the second path that will execute the test.
I am having this separation because we need to separate the build and the execution processes so that they can be run separately by different machines in a CI environment.
so in the Configuration phase (controlled using a pytest cli argument), I am simply skipping the tests in a fixture that checks if this is the configuration stage or not, using:
pytest.skip("configuration stage")
What I want to do is to make the test appear as PASSED during the configuration stage, from the fixture context (without going to the test), putting an assert will not suffice as the test function will be executed anyway, and I don't want to pollute all the tests with a check like this:
def test_function(config_fixture):
if config_fixture:
assert True
else:
#run the test
Related
I have a test that sets the ground for a couple of other tests to run, thus, I need the first test to succeed and only then run the other ones. I know I could use pytest-dependencies but that would not allow me to run my tests in parallel. I'm also interested in the option of only having to run one test and have it initialize all the ones that depend on it if it succeeds (instead of running all of them and skipping, them if the test they depend on fails).
I'm having a pytest run that invokes N tests. I want all tests to output into the same log, and I want to clear the log before pytest is triggered, so different runs would not contiminate each other. So far, I tried adding my clear_log() to some fixtures or hooks, but none has achieved what In wanted - or they were called each time (pytest_runtest_makereport for example), or they were not called at all (some function that was decorated with pytest.fixture()
I also tried having a global flag that would indicate if this is the first run, but it didn't work either...
Where can I put the function so it would be invoked only once?
Following decorator allows you to execute your fixture only once (even if there are multiple number of tests are running):
#pytest.fixture(scope="session")
More details here
I am relatively new to pytest at the moment and was curious if there is a way to store the pass/fail results of the test in a variable.
Essentially what I want to do is run my full suite of tests and after the tests are run, send the name of the tests run along with the pass/fail result to a server.
I understand that pytest provides options such as -r that will output the test run with pass or fail after execution, but is there a way to store those into variables or pass those results along?
is there a way to store those into variables or pass those results along?
Pytest can natively output JUnitXML files:
To create result files which can be read by Jenkins or other Continuous integration servers, use this invocation:
pytest --junitxml=path
to create an XML file at path.
There is an available schema for this format and there appear to be several Python libraries that can parse them with varying levels of support. This one looks like a good place to start.
There are also plugins that may be able to help. For example, pytest-json:
pytest-json is a plugin for py.test that generates JSON reports for test results
I need to test functions Initialize/Shutdown with different parameters. Each of these functions can be executed only once during app lifetime. Do I have to create 10 files with only one test function each or can I define 10 tests in one file and mark each function to be run using new instance of python interpreter?
Is this possible with either PyTest or the built-in unittest package?
I made it work with unittest. Created _runner.py (sources below) which runs all unit tests in current directory using test discovery (unittest.TestLoader). It loops through all test suites and checks test case names for "IsolatedTest" words. These will be run using new Python instance by calling subprocess.check_output("python.."). Others are run normally in current process. For example I'm declaring class FooIsolatedTest(unittest.TestCase). In isolated tests as a replacement for unittest.main() using such code: import _runner; _runner.main(os.path.basename(__file__)). You can take a look at sources here.
I have a set of unit tests in python. Some of them open graphical objects using pyqt, some are just standard standalone tests. My goal is to automatically run at least the tests that don't need to open window, because unless it will wait for user input and then fail.
Note that:
I can't remove the graphical tests (constraint from the project)
By default all tests should run, but passing some parameter only non graphical one will run
My test suite is built using unittest.TestLoader().discover
My best guess would be to pass a global parameter to the TestSuite, so that each test could check the value to know if it should skip or not. But after reading unittest documentation I could not find a way to do this.
I'm aware of this question: How To Send Arguments to a UnitTest Without Global Variables, but I would have expected some unittest configuration.
You could use unittest.skipIf(condition, reason) and an environment variable to skip the graphical tests.
Create a decorator like:
graphical_test = unittest.skipIf(
os.environ.get('GRAPHICAL_TESTS', False), 'Non graphical tests only'
)
Then annotate your graphical tests with #graphical_test and run your tests after setting GRAPHICAL_TESTS=1