I'm running into a strange problem that seems to come about from how python unit tests manage their imports and how this relates to the mock package. This is for a django project, using django-nose/nose for unit test running and mock for mocking.
I have a unit test using mock which works perfectly fine when run alone (python manage.py test tests/test_code.py)
inside test_code.py:
from my.app.models.bookstore import create_from_proxy
class MockTestCase( TestCase ):
def setUp( self ):
self.patcher = patch( 'my.app.models.bookstore.BookProxy', autospec=True )
self.mock_proxy = self.patcher.start()
self.proxy_instance = self.mock_proxy.return_value
self.proxy_instance.json = json.loads(BOOK_JSON)
def tearDown( self ):
self.patcher.stop()
def test_mock_works( self ):
book_id = 55
v = create_from_proxy( book_id )
self.assertTrue( self.mock_proxy.called )
... more tests ...
inside bookstore.py:
from my.app.proxies import BookProxy
def create_from_proxy( self, id ):
proxy = BookProxy(id)
...
However when I run this test as part of the entire suite of tests (python manage.py test) then the test fails because the bookstore.py code doesn't get the mock class injected and falls back to the actual code for BookProxy.
So there seems to be something stateful going on when all of the tests are run together, but I can't figure out what's causing the mock injection to fail. Other unit tests that use mock all seem to be cleaning up after theirselves (either using the decorator, the context, or the explicit patch object approach that I'm showing here).
Something like this ever been seen out there before?
Related
For instance, every time a test finds
database.db.session.using_bind("reader")
I want to remove the using_bind("reader")) and just work with
database.db.session
using mocker
Tried to use it like this in conftest.py
#pytest.fixture(scope='function')
def session(mocker):
mocker.patch('store.database.db.session.using_bind', return_value=_db.db.session)
But nothing has worked so far.
Code under test:
from store import database
results = database.db.session.using_bind("reader").query(database.Order.id).join(database.Shop).filter(database.Shop.deleted == False).all(),
and I get
AttributeError: 'scoped_session' object has no attribute 'using_bind' as an error.
Let's start with an MRE where the code under test uses a fake database:
from unittest.mock import Mock, patch
class Session:
def using_bind(self, bind):
raise NotImplementedError(f"Can't bind {bind}")
def query(self):
return "success!"
database = Mock()
database.db.session = Session()
def code_under_test():
return database.db.session.using_bind("reader").query()
def test():
assert code_under_test() == "success!"
Running this test fails with:
E NotImplementedError: Can't bind reader
So we want to mock session.using_bind in code_under_test so that it returns session -- that will make our test pass.
We do that using patch, like so:
#patch("test.database.db.session.using_bind")
def test(mock_bind):
mock_bind.return_value = database.db.session
assert code_under_test() == "success!"
Note that my code is in a file called test.py, so my patch call applies to the test module -- you will need to adjust this to point to the module under test in your own code.
Note also that I need to set up my mock before calling the code under test.
Suppose we have the following Flask view function to test. Specifically, we want to mock create_foo() as it writes to the filesystem.
# proj_root/some_project/views.py
from some_project import APP
from some_project.libraries.foo import create_foo
#APP.route('/run', methods=['POST']
def run():
bar = create_foo()
return 'Running'
Now we want to write a unit test for run() with the call to create_foo() mocked to avoid creating unnecessary files.
# proj_root/tests/test_views.py
from some_project import some_project
#pytest.fixture
def client():
some_project.APP.config['TESTING'] = True
with some_project.APP.test_client() as client:
yield client
def test_run(client, monkeypatch):
monkeypatch.setattr('some_project.libraries.foo.create_foo', lambda: None)
response = client.post('/run')
assert b'Running' in response.data
It seems that this approach should work even with the named create_foo import. The tests all pass, however the original code of create_foo is clearly being executed as a new file in the filesystem is created each time the test suite is run. What am I missing? I suspect it has something to do with the named imports based on some related questions but I'm not sure.
The correct monkeypatch is:
monkeypatch.setattr('some_project.views.create_foo', lambda: None)
The reason for this is pretty well explained here.
I want to run a set of tests under different conditions and therefore share these tests between two different TestCase-derived classes.
One creates its own standalone session and the other attaches to an existing session and executes the same tests in there.
I guess I'm kind of abusing the unittest framework when testing an API with it but it doesn't feel like it's too far from its original purpose. Am I good so far?
I hacked a few things together and got it kind of running. But the way it's done, doesn't feel right and I'm afraid will cause problems sooner or later.
These are the problems I have with my solution:
When simply running the thing with PyCharm without limiting the tests, it attempts to run not only the intended StandaloneSessionTests and ExistingSessionTests but also GroupOfTests which is only the collection and has no session, i.e. execution context.
I can make it not run GroupOfTests by not deriving that one from TestCase but then PyCharm complains that it doesn't know about the assert...() functions. Rightly so, because GroupOfTest only gets indirect access to these functions at runtime when a derived class also inherits from TestCase. Coming from a C++ background, this feels like black magic and I don't think I should be doing this.
I tried passing the session creation classes to the constructor of GroupOfTests like this: __init__(self, session_class). But this causes problems when the unittest framework attempts to instantiate the tests: It doesn't know what to do with the additional __init__ parameter.
I learned about #classmethod, which seems to be a way to get around the "only one constructor" limitation of Python but I couldn't figure out a way to get it running.
I'm looking for a solution that lets me state something as straightforward as this:
suite = unittest.TestSuite()
suite.addTest(GroupOfTests(UseExistingSession))
suite.addTest(GroupOfTests(CreateStandaloneSession))
...
This is what I got so far:
#!/usr/bin/python3
import unittest
def existing_session():
return "existingsession"
def create_session():
return "123"
def close_session(session_id):
print("close session %s" % session_id)
return True
def do_thing(session_id):
return len(session_id)
class GroupOfTests(unittest.TestCase): # GroupOfTests gets executed, which makes no sense.
#class GroupOfTests: # use of assertGreaterThan() causes pycharm warning
session_id = None
def test_stuff(self):
the_thing = do_thing(self.session_id)
self.assertGreater(the_thing, 2)
# Original code contains many other tests, which must not be duplicated
class UseExistingSession(unittest.TestCase):
session_id = None
def setUp(self):
self.session_id = existing_session()
def tearDown(self):
pass # Nothing to do
class CreateStandaloneSession(unittest.TestCase):
session_id = None
def setUp(self):
self.session_id = create_session()
def tearDown(self):
close_session(self.session_id)
# unittest framework runs inherited test_stuff()
class StandaloneSessionTests(CreateStandaloneSession, GroupOfTests):
pass
# unittest framework runs inherited test_stuff()
class ExistingSessionTests(UseExistingSession, GroupOfTests):
pass
def main():
suite = unittest.TestSuite()
suite.addTest(StandaloneSessionTests)
suite.addTest(ExistingSessionTests)
runner = unittest.TextTestRunner()
runner.run(suite())
if __name__ == '__main__':
main()
I'm not sure if using pytest is an option for you but if so, here is an example which might do what you want.
import pytest
class Session:
def __init__(self, session_id=None):
self.id = session_id
existing_session = Session(999)
new_session = Session(111)
#pytest.fixture(params=[existing_session, new_session])
def session_fixture(request):
return request.param
class TestGroup:
def test_stuff(self, session_fixture):
print('(1) Test with session: {}'.format(session_fixture.id))
assert True
def test_more_stuff(self, session_fixture):
print('(2) Test with session: {}'.format(session_fixture.id))
assert True
Output:
$ pytest -v -s hmm.py
======================================================= test session starts ========================================================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /home/lettuce/Dropbox/Python/Python_3/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/lettuce/Dropbox/Python/Python_3, inifile:
collected 4 items
hmm.py::TestGroup::test_stuff[session_fixture0] (1) Test with session: 999
PASSED
hmm.py::TestGroup::test_stuff[session_fixture1] (1) Test with session: 111
PASSED
hmm.py::TestGroup::test_more_stuff[session_fixture0] (2) Test with session: 999
PASSED
hmm.py::TestGroup::test_more_stuff[session_fixture1] (2) Test with session: 111
PASSED
===================================================== 4 passed in 0.01 seconds =====================================================
If you are actually going to use pytest you will probably want follow the conventions for Python test discovery rather than using hmm.py as a filename though!
You can get pycharm to ignore these not existing function by creating abstract methods with raise NotImplementetError:
class GroupOfTests:
session_id = None
def test_stuff(self):
the_thing = do_thing(self.session_id)
self.assertGreater(the_thing, 2)
def assertGreater(self, a, b): # Pycharm treats these like abstract methods from the ABC module
raise NotImplementetError
This will let python believe this is an abstract class and will make pycharm raise errors if a subclass doesn't define these functions.
I am writing some python unittest cases and use setUpClass to start with, however, I like some user interface that can control the saving path for some intermediate result, something like following. It is not working though, but the idea is I have a report_path argument in setUpClass argument list, so in some way, when instantiate this class, the user could input his own path:
class MyTestCase(unittest.TestCase):
#classmethod
def setUpClass(cls, report_path=None):
cls.s = cls.analyserfunction(....)
if report_path is None:
cls.report_path = something I defined by default
else:
cls.report_path = report_path
#classmethod
def analyserfunction(cls, ...):
def test_func(self):
do something for self.s
And I use TestLoader() to run it, something like this
suite = unittest.TestLoader().loadTestsFromTestCase(
test01.MyTestCase)
testRunner = unittest.TextTestRunner(
stream=sys.stderr, descriptions=True, verbosity=2)
testRunner.run(suite)
It works perfectly fine if I use the code above, but if naively do this
suite = unittest.TestLoader().loadTestsFromTestCase(
test01.MyTestCase(report_path = 'some path'))
it will fail, it seems that the test suite can only accept the test case class without any argument, not like the traditional class. Is there any way that can get through this, i.e, load MyTestCase(report_path = 'some path')?
In a hobby project I intend to use nose for testing, I want to put all tests for specific classes into classes since these tests share setup and other functionality. But I can't seem to get nose to execute the setup methods inside the classes.
Here is an example class that is tested:
class mwe():
def __init__(self):
self.example = ""
def setExample(self, ex):
self.example = ex
The tests work, when I don't use classes:
from nose.tools import ok_
import mwe
exampleList = []
def setUp():
print("setup")
exampleList.append("1")
exampleList.append("2")
exampleList.append("3")
def test_Example():
print("test")
for ex in exampleList:
t = mwe.mwe()
t.setExample(ex)
yield check, t, ex
def check(e, ex):
ok_(e.example == ex)
The output is as expected:
setup
test
...
----------------------------------------------------------------------
Ran 3 tests in 0.004s
OK
When a test class is used, the setup method is not executed and therefore no tests are executed.
from nose.tools import ok_
import mwe
class TestexampleClass(object):
def __init__(self):
print("__init__")
self.exampleList = []
def setup(self):
print("setup class")
self.exampleList.append("1")
self.exampleList.append("2")
self.exampleList.append("3")
def test_ExampleClass(self):
print("test class")
for ex in self.exampleList:
t = mwe.mwe()
t.setExample(ex)
yield self.check, t, ex
def check(self, we, ex):
print("check class")
ok_(we.example == ex)
I'm fairly new to python and a newbie with nose, my question is, why is setup not executed? Where is the error in my code?
__init__
test class
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
I will be glad for any feedback.
When I use the code from this Question on SO the setup method is executed, as I would expect it.
SOLUTION: After a lot of desperation I found the following:
Nose executes the the class level setup method before the execution of the yielded function, not when the test_* methods are called, as I expected and as is the case for other test_* methods. This obviously goes against the nose documentation:
Setup and teardown functions may be used with test generators. However, please note that setup and teardown attributes attached to the generator function will execute only once. To execute fixtures for each yielded test, attach the setup and teardown attributes to the function that is yielded, or yield a callable object instance with setup and teardown attributes.
Looking at the bug reports, I found the bug report on github.
A possible workaround is to use class level fixtures:
#classmethod
def setup_class(cls):
#do stuff
pass
Your test class needs to extend TestCase and the setup method needs to be called setUp
from unittest import TestCase
class TestUtils(TestCase):
def setUp(self):
self.x = 1
def test_something(self):
self.assertEqual(1, self.x)
which outputs
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK