I am trying to test using unittest a python 3.6 script that starts with (my_app.py):
import sys
from awsglue.utils import getResolvedOptions
args = getResolvedOptions(sys.argv, ['opt1', 'opt2', 'opt3'])
opt1 = args['opt1']
opt2 = args['opt2']
opt3 = args['opt3']
....
so in my test I did something like:
import unittest
import datetime
from mock import patch
import my_app
class TestMyApp(unittest.TestCase):
#patch('awsglue.utils.getResolvedOptions')
def test_mock_stubs(self, patch_opts):
patch_opts.return_value = {}
....
but the test soon fails at import my_app with:
ModuleNotFoundError: No module named 'awsglue'
as there is not awsglue locally installed. How can I test a module that import a not locally installed library and also mock it in my test?
You'll need to mock the imported module before the import for my_app happens. patch won't work here, because patch imports the module in order to patch it. And in this case, that import itself would cause an error.
To do this the simplest way is to trick python into thinking that awsglue is already imported. You can do that by putting your mock directly in the sys.modules dictionary. After this you can do the my_app import.
import unittest
import sys
from unittest import mock
class TestMyApp(unittest.TestCase):
def test_mock_stubs(self):
# mock the module
mocked_awsglue = mock.MagicMock()
# mock the import by hacking sys.modules
sys.modules['awsglue.utils'] = mocked_awsglue
mocked_awsglue.getResolvedOptions.return_value = {}
# move the import here to make sure that the mocks are setup before it's imported
import my_app
You can move the import hack part to a setup fixture method.
However I would suggest just installing the package. The import hack can be very difficult to maintain.
Related
I have something like this in my source file
# code.py
def some_func():
# doing some connections and stuff
return {'someKey': 'someVal'}
class ClassToTest:
var = some_func()
My test file looks like this... I am trying to mock some_func as I want to avoid creating connections.
# test_code.py
from src.code import ClassToTest
def mock_function():
return {"someOtherKey": "someOtherValue"}
class Test_Code(unittest.TestCase):
#mock.patch('src.code.some_func', new=mock_function)
def test_ClassToTest(self):
self.assertEqual(ClassToTest.var, {"someOtherKey": "someOtherValue"})
But this doesn't work. On the other hand if var is an instant variable mock works fine. I guess this is due to class variables getting initialized during imports. How do I properly mock some_func before var even gets initialized?
When you imported code.py, there is no active patch yet, so when ClassToTest.var was initialized, it used the original some_func. Only then would the patch for src.code.some_func would take effect which obviously is too late now.
Solution 1
What you can do is to patch some_func and then reload code.py so that it re-initializes the ClassToTest including its attribute var. Thus since we already have an active patch by the time we reload code.py, then ClassToTest.var would be set with the patched value.
But we can't do it if both the class and the patched function lives in the same file, so to make it testable move some_func to another file and then just import it.
src/code.py
from src.other import some_func
class ClassToTest:
var = some_func()
src/other.py
def some_func():
# doing some connections and stuff
return {'realKey': 'realValue'}
test_code.py
from importlib import reload
import sys
import unittest
from unittest import mock
from src.code import ClassToTest # This will always refer to the unpatched version
def mock_function():
return {"someOtherKey": "someOtherValue"}
class Test_Code(unittest.TestCase):
def test_real_first(self):
self.assertEqual(ClassToTest.var, {"realKey": "realValue"})
#mock.patch('src.other.some_func', new=mock_function)
def test_mock_then_reload(self):
# Option 1:
# import src
# reload(src.code)
# Option 2
reload(sys.modules['src.code'])
from src.code import ClassToTest # This will be the patched version
self.assertEqual(ClassToTest.var, {"someOtherKey": "someOtherValue"})
def test_real_last(self):
self.assertEqual(ClassToTest.var, {"realKey": "realValue"})
Output
$ pytest -q
... [100%]
3 passed in 0.04s
Solution 2
If you don't want the real some_func to be ever called during test, then just reloading isn't enough. What needs to be done is to never import the file containing ClassToTest nor any file that would import it indirectly. Only import it once an active patch for some_func is already established.
from importlib import reload
import sys
import unittest
from unittest import mock
# from src.code import ClassToTest # Remove this import!!!
def mock_function():
return {"someOtherKey": "someOtherValue"}
class Test_Code(unittest.TestCase):
#mock.patch('src.other.some_func', new=mock_function)
def test_mock_then_reload(self):
from src.code import ClassToTest # Move the import here once the patch has taken effect already
self.assertEqual(ClassToTest.var, {"someOtherKey": "someOtherValue"})
How do I make import pkg fail in moduleA.py? I can patch pkg to fail if something's imported from it, but not otherwise:
# test.py
import os
import moduleA
from unittest.mock import patch
from importlib import reload
#patch.dict('sys.modules', pkg=os)
def test_mock():
reload(moduleA)
# moduleA.py
import pkg # make this fail
from pkg import sum # this does fail
Live example
This is a bit more complicated. You have to make sure that reloading fails - this can be done by adding a class that implements find_spec. Second, you have to remove an already loaded package from sys.modules - otherwise this will be used on reload:
import sys
from importlib import reload
import pytest
import moduleA
class ImportRaiser:
def find_spec(self, fullname, path, target=None):
if fullname == 'pkg':
# we get here if the module is not loaded and not in sys.modules
raise ImportError()
sys.meta_path.insert(0, ImportRaiser())
def test_import_error():
if 'pkg' in sys.modules:
del sys.modules['pkg']
with pytest.raises(ImportError):
reload(moduleA)
I'm trying to test some code using pytest and need to change a function from some module. One of my imports also imports that function, but this is failing when I change the method using monkeypatch. Here is what I have:
util.py
def foo():
raise ConnectionError # simulate an error
return 'bar'
something.py
from proj import util
need_this = util.foo()
print(need_this)
test_this.py
import pytest
#pytest.fixture(autouse=True)
def fix_foo(monkeypatch):
monkeypatch.setattr('proj.something.util.foo', lambda: 'bar')
import proj.something
And this raises ConnectionError. If I change
test_this.py
import pytest
#pytest.fixture(autouse=True)
def fix_foo(monkeypatch):
monkeypatch.setattr('proj.something.util.foo', lambda: 'bar')
def test_test():
import proj.something
Then it imports with the monkeypatch working as expected. I've read though this and tried to model my testing from it, but that isn't working unless I import inside of a test. Why does the monkeypatch not do anything if it is just a normal import in the testing file?
That is because the fixture is applied to the test function not the entire code. autouse=True attribute just says that it should be used in every test
I am writing unit tests for the api, which connects to MongoDB. In my API module it looks like this:
from flask import Flask, jsonify
from MyApp import MongoData
api = Flask(__name__)
DB_CONN = MongoData()
#api.route('/bla', methods=['GET'])
def alive():
return jsonify({'response': true})
I have a problem while importing this module in my unittest. I want to mock collection from MongoData() with special mock class, which uses mongomock. The problem is that I cannot mock DB_CONN while importing in tests:
from MyApp import api
I was trying to do this with mock:
DB_CONN = MockMongoData()
with mock.patch('MyApp.api.DB_CONN', DB_CONN):
from MyApp import api
but it still tries to connect to the database as specified in the config file.
Any advice how to mock DB_CONN from MyApp.api module?
Thanks in advance!
EDIT:
This will work:
import sys
from MyApp import MongoData, MockMongoData
sys.modules['MyApp'].MongoData = MockMongoData
from MyApp import api
But is there a better (more pythonic) way to do this?
Import the module first, then monkeypatch its members:
DB_CONN = MockMongoData()
from MyApp import api
with mock.patch('MyApp.api.DB_CONN', DB_CONN):
api.run()
Many Python modules preserve an internal state without defining classes, e.g. logging maintains several loggers accessible via getLogger().
How do you test such a module? Using the standard unittest tools, I would like the various tests inside a TestCase class to re-import my module-under-test so that each time it loses its context. Can this be done?
import unittest
import sys
class Test(unittest.TestCase):
def tearDown(self):
try:
del sys.modules['logging']
except KeyError:
pass
def test_logging(self):
import logging
logging.foo=1
def test_logging2(self):
import logging
print(logging.foo)
if __name__ == '__main__':
unittest.sys.argv.insert(1,'--verbose')
unittest.main(argv = unittest.sys.argv)
% test.py Test.test_logging passes:
test_logging (__main__.Test) ... ok
but
% test.py Test.test_logging2 does not:
test_logging2 (__main__.Test) ... ERROR
since the internal state of logging has been reset.
This will reimport the module as new for you:
import sys
del sys.modules['my_module']
import my_module