How can I make it so unit tests in Python (using unittest) are run in the order in which they are specified in the file?
You can change the default sorting behavior by setting a custom comparison function. In unittest.py you can find the class variable unittest.TestLoader.sortTestMethodsUsing which is set to the builtin function cmp by default.
For example you can revert the execution order of your tests with doing this:
import unittest
unittest.TestLoader.sortTestMethodsUsing = lambda _, x, y: cmp(y, x)
Clever Naming.
class Test01_Run_Me_First( unittest.TestCase ):
def test010_do_this( self ):
assertTrue( True )
def test020_do_that( self ):
etc.
Is one way to force a specific order.
As said above, normally tests in test cases should be tested in any (i.e. random) order.
However, if you do want to order the tests in the test case, apparently it is not trivial.
Tests (method names) are retrieved from test cases using dir(MyTest), which returns a sorted list of members. You can use a clever (?) hack to order methods by their line numbers. This will work for one test case:
if __name__ == "__main__":
loader = unittest.TestLoader()
ln = lambda f: getattr(MyTestCase, f).im_func.func_code.co_firstlineno
lncmp = lambda a, b: cmp(ln(a), ln(b))
loader.sortTestMethodsUsing = lncmp
unittest.main(testLoader=loader, verbosity=2)
There are also test runners which do that by themselves – I think py.test does it.
Use proboscis library as I mentioned already (please see short description there).
The default order is alphabetical. You can put test_<int> before every test to specify the order of execution.
Also you can set unittest.TestLoader.sortTestMethodsUsing = None to eliminate the sort.
Check out Unittest Documentation for more details.
I found a solution for it using PyTest ordering plugin provided here.
Try py.test YourModuleName.py -vv in CLI and the test will run in the order they have appeared in your module.
I did the same thing and works fine for me.
Note: You need to install PyTest package and import it.
hacky way (run this file in pycharm or other unit test runner)
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import unittest
def make_suite():
class Test(unittest.TestCase):
def test_32(self):
print "32"
def test_23(self):
print "23"
suite = unittest.TestSuite()
suite.addTest(Test('test_32'))
suite.addTest(Test('test_23'))
return suite
suite = make_suite()
class T(unittest.TestCase):
counter = 0
def __call__(self, *args, **kwargs):
res = suite._tests[T.counter](*args, **kwargs)
T.counter += 1
return res
for t in suite._tests:
name = "{}${}".format(t._testMethodName, t.__class__.__name__)
setattr(T, name, t)
Related
I have one function like this one:
def function(df, path_write):
df['A'] = df['col1'] * df['col2']
write(df, path)
The function is not that simple but the question is, how can i make a unit test if the function do not return any value??
If the function returns the new df it's simple, just make:
assert_frame_equal from the library from pandas.testing import assert_frame_equal and mock the write method.
But without that return, how can i test the df line??
In general, I can think of only two kinds of functions: Those that return a value and those that produce side-effects.
With the second kind, you typically don't want the side-effects to actually happen during testing. For example, if your function writes something to disk or sends some data to some API on the internet, you usually don't want it to actually do that during the test, you only want to ensure that it attempts to do the right thing.
To roll with the example of disk I/O as a side-effect: You would usually have some function that does the actual writing to the filesystem that the function under testing calls. Let's say it is named write. The typical apporach would be to mock that write function in your test. Then you would need to verify that that mocked write was called with the arguments you expected.
Say you have the following code.py for example:
def write(thing: object, path: str) -> None:
print("Some side effect like disk I/O...")
def function(thing: object, file_name: str) -> None:
...
directory = "/tmp/"
write(thing, path=directory + file_name)
To test function, I would suggest the following test.py:
from unittest import TestCase
from unittest.mock import MagicMock, patch
from . import code
class MyTestCase(TestCase):
#patch.object(code, "write")
def test_function(self, mock_write: MagicMock) -> None:
test_thing = object()
test_file_name = "test.txt"
self.assertIsNone(code.function(test_thing, test_file_name))
mock_write.assert_called_once_with(
test_thing,
path="/tmp/" + test_file_name,
)
Check out unittest.mock for more details on mocking with the standard library. I would strongly advise to use the tools there and not do custom monkey-patching. The latter is certainly possible, but always carries the risk that you forget to revert the patched objects back to their original state after every test. That can break the entire rest of your test cases and depending on how you monkey-patched, the source of the resulting errors may become very hard to track down.
Hope this helps.
In the mock for write() you can add assert statements to ensure the form of df is as you would expect. For example:
def _mock_write(df, path):
assert path == '<expected path value>'
assert_frame_equal(df, <expected dataframe>)
So the full test case would be:
def test_function(self, monkeypatch):
# Define mock function
def _mock_write(df, path):
assert path == '<expected path value>'
assert_frame_equal(df, <expected dataframe>)
# Mock write function
monkepyatch.setattr(<MyClass>, 'write', _mock_write)
# Run function to enter mocked write function
function(test_df, test_path_write)
N.B. This is assuming you are using pytest as your test runner which supports the set up and tear down of monkeypatch. Other answers show the usage for the standard unittest framework.
So let's say I have this bit of code:
import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
Now it's a simple enough method, and as you can see we are using an external library(coolObject).
In unit tests, I have to create a mock of this object that roughly replicates it. Let's call this mock object coolMock.
My question is how would I tell the code when to use coolMock or coolObject? I've looked it up online, and a few people have suggested dependency injection, but I'm not sure I understand it correctly.
Thanks in advance!
def doSomething(cool_object=None):
cool_object = cool_object or coolObject()
...
In you test:
def test_do_something(self):
cool_mock = mock.create_autospec(coolObject, ...)
cool_mock.coolOperation.side_effect = ...
doSomthing(cool_object=cool_mock)
...
self.assertEqual(cool_mock.coolOperation.call_count, ...)
As Dan's answer says, one option is to use dependency injection: have the function accept an optional argument, if it's not passed in use the default class, so that a test can pass in a moc.
Another option is to use the mock library (here or here) to replace your coolObject.
Let's say you have a foo.py that looks like
from somewhere.else import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
In your test_foo.py you can do:
import mock
def test_thing():
path = 'foo.coolObject' # The fully-qualified path to the module, class, function, whatever you want to mock.
with mock.patch('foo.coolObject') as m:
doSomething()
# Whatever you want to assert here.
assert m.called
The path you use can include properties on objects, e.g. module1.module2.MyClass.my_class_method. A big gotcha is that you need to mock the object in the module being tested, not where it is defined. In the example above, that means using a path of foo.coolObject and not somwhere.else.coolObject.
So I'm running py.test and trying to use monkeypatch. I understand that monkeypatch's intended purpose is to replace attributes in a module so that they can be tested. And I get that we can substitute in mock functions in order to do this.
Currently I am trying to run essentially the following block of code.
from src.module.submodule import *
def mock_function(parameter = None):
return 0
def test_function_works(monkeypatch):
monkeypatch.setattr("src.module.submodule.function",mock_function ]
assert function(parameter = None) == 0
When the test runs, instead of swapping in mock_function, it just runs function . Could there be a reason why monkeypatch isn't activating
I have got monkey patch running succesfully with other code before. So I don't see why this isn't working.
I haven't used pytest for this stuff, but I know that with the mock library, functions are patched in the namespace where they're called. i.e. from src.module.submodule import * imports src.module.submodule.function into your namespace, but you then patch it in its original namespace, so your local name for the function still accesses the original, unpatched code.
If you change this to
import src.module.submodule
def mock_function(parameter = None):
return 0
def test_function_works(monkeypatch):
monkeypatch.setattr("src.module.submodule.function",mock_function ]
assert src.module.submodule.function(parameter = None) == 0
does it succeed?
Looks like a typo, shouldn't it be
monkeypatch.setattr("src.module.submodule.function",mockIfunction)
i.e. mockIfunction instead of mock_function?
I have a desire to use Nose for an over the wire integration test suite. However, the order of execution of some of these tests is important.
That said, I thought I would toss together a quick plugin to decorate a test with the order I want it executed: https://gist.github.com/Redsz/5736166
def Foo(unittest.TestCase):
#step(number=1)
def test_foo(self):
pass
#step(number=2)
def test_boo(self):
pass
From reviewing the built in plugins I had thought, I could simply override loadTestsFromTestCase and order the tests by the decorated 'step number'.:
def loadTestsFromTestCase(self, cls):
"""
Return tests in this test case class. Ordered by the step definitions.
"""
l = loader.TestLoader()
tmp = l.loadTestsFromTestCase(cls)
test_order = []
for test in tmp._tests:
order = test.test._testMethodName
func = getattr(cls, test.test._testMethodName)
if hasattr(func, 'number'):
order = getattr(func, 'number')
test_order.append((test, order))
test_order.sort(key=lambda tup: tup[1])
tmp._tests = (t[0] for t in test_order)
return tmp
This method is returning the tests in the order I desire, however when the tests are being executed by nose they are not being executed in this order?
Perhaps I need to move this concept of ordering to a different location?
UPDATE: As per the comment I made, the plugin is actually working as expected. I was mistaken to trust the pycharm test reporter. The tests are running as expected. Rather than removing the question I figured I would leave it up.
From the documentation:
[...] nose runs functional tests in the order in which they appear in the module file. TestCase-derived tests and other test classes are run in alphabetical order.
So a simple solution might be to rename the tests in your test case:
class Foo(unittest.TestCase):
def test_01_foo(self):
pass
def test_02_boo(self):
pass
I found a solution for it using PyTest ordering plugin provided here.
Try py.test YourModuleName.py -vv in CLI and the test will run in the order they have appeared in your module (first test_foo and then test_bar)
I did the same thing and works fine for me.
Note: You need to install PyTest package and import it.
I'm writing test system using py.test, and looking for a way to make particular tests execution depending on some other test run results.
For example, we have standard test class:
import pytest
class Test_Smoke:
def test_A(self):
pass
def test_B(self):
pass
def test_C(self):
pass
test_C() should be executed if test_A() and test_B() were passed, else - skipped.
I need a way to do something like this on test or test class level (e.g. Test_Perfo executes, if all of Test_Smoke passed), and I'm unable to find a solution using standard methods (like #pytest.mark.skipif).
Is it possible at all with pytest?
You might want to have a look at pytest-dependency. It is a plugin that allows you to skip some tests if some other test had failed.
Consider also pytest-depends:
def test_build_exists():
assert os.path.exists(BUILD_PATH)
#pytest.mark.depends(on=['test_build_exists'])
def test_build_version():
# this will skip if `test_build_exists` fails...