I am using requests to create a requests.Session and set it up inside a function:
def create_web_session(cookie=None):
s = requests.Session()
if cookie is not None:
s.cookies.set("my_cookie_name", cookie)
return s
Sessions can be used as context managers. Can I use the function that returns the session (above) in a with statement?
with create_web_session("my_cookie_value") as s:
s.get(...)
Or would I have to change the function that instead takes a session and sets it up inside the context manager:
with requests.Session() as s:
setup_web_session(s, "my_cookie_value")
s.get(...)
Running the former seemed to work however my unit tests failed which is why I asked this question. I am patching the create_web_session and upon testing the s.get(...) I am asserting that my mock gets called with .get() however it seems to be session_mock.__enter__() that is calling it. Is this expected behavior?
Here is an example:
# Function
def my_function():
s = create_web_session()
s.get("https://google.com")
s.close()
# Test
#patch("foo.bar.create_web_session")
def test_my_function(self, mock_create_web_session):
my_function()
mock_create_web_session.assert_called_once()
mock_create_web_session.return_value.get.assert_called_once()
mock_create_web_session.return_value.close.assert_called_once()
Once I change the function to use context managers:
def my_function():
with create_web_session() as s:
s.get("https://google.com")
The test fails with:
Expected 'get' to have been called once. Called 0 times.
Your create_web_session is fine. The problem in the test is that while requests.Session.__enter__ simply returns back the same session, all methods on a mock return a fresh mock object. We can tell the mock to behave how we want and get a working test case like so:
def test_my_function(self, mock_create_web_session):
session = mock_create_web_session.return_value
session.__enter__.return_value = session
my_function()
mock_create_web_session.assert_called_once()
session.get.assert_called_once()
session.__exit__.assert_called_once()
Note that I assert __exit__ is called, not close, because the mock doesn't know anything about close or real sessions.
Related
I have a function than depends on db connection. This function has a lot of return statements, of this kind:
def do_something_with_data(db: Session, data: DataClass):
db = Session()
if condition1:
db.do_something1()
db.close()
return
if condition2:
db.do_something2()
db.close()
return
if condition3:
db.do_something3()
db.close()
return
...
After executing the function, I need to run db.close(), but because of the structure of the function this entry will have to be duplicated many times for each return as shown above.
So I made a decorator that passes the created session to the function and closes the session at the end of the execution of the function instead.
def db_depends(function_that_depends_on_db):
def inner(*args, **kwargs):
db = Session()
result = function_that_depends_on_db(db, *args, **kwargs)
db.close()
return result
return inner
#db_depends
def do_something_with_data(db: Session, data: DataClass):
if condition1:
db.do_something1()
return
if condition2:
db.do_something2()
return
if condition3:
db.do_something3()
return
...
All works great, but the fact, that user see two required arguments in definition, however there is only one (data) seems kinda dirty.
Is it possible to do the same thing without misleading people who will read the code or IDE hints?
Just have the function accept the Session parameter normally:
def do_something_with_data(db: Session, data: DataClass):
if condition1:
db.do_something1()
return
if condition2:
db.do_something2()
return
if condition3:
db.do_something3()
return
This allows the user to specify a Session explicitly, for example to reuse the same Session to do multiple things.
Yes, that doesn't close the Session. Because that is the responsibility of the calling code, since that's where the Session came from in the first place. After all, if the calling code wants to reuse a Session, then it shouldn't be closed.
If you want a convenience method to open a new, temporary Session for the call, you can easily do that using the existing decorator code:
do_something_in_new_session = db_depends(do_something_with_data)
But if we don't need to apply this logic to multiple functions, then "simple is better than complex" - just write an ordinary wrapper:
def do_something_in_new_session(data: DataClass):
db = Session()
result = do_something_with_data(db, data)
db.close()
return result
Either way, it would be better to write the closing logic using a with block, assuming your library supports it:
def do_something_in_new_session(data: DataClass):
with Session() as db:
return do_something_with_data(db, data)
Among other things, this ensures that .close is called even if an exception is raised in do_something_with_data.
If your DB library doesn't support that (i.e., the Session class isn't defined as a context manager - although that should only be true for very old libraries now), that's easy to fix using contextlib.closing from the standard library:
from contextlib import closing
def do_something_in_new_session(data: DataClass):
with closing(Session()) as db:
return do_something_with_data(db, data)
(And of course, if you don't feel the need to make a wrapper like that, you can easily use such a with block directly at the call site.)
I am writing unit tests. I would like to mock the result of a function called on a mock object.
I have a class called OwnerAnalyzer which accepts an object called client in its constructor. Using this client, I can get owner details.
In my unit test, I want to pass a mock for this client and mock results from its get_owners method.
Here is what I have so far:
def test_get_owner_details(mock_datetime, monkeypatch):
mock_datetime.now.return_value.isoformat.return_value = MOCK_NOW
mock_client = mock.MagicMock()
mock_client.return_value.get_owners.return_value = ListOwnerDetails(
main_owner=OwnerDetails(name='test_owner', type='User'), secondary_owners=[])
owner_analyzer = OwnerAnalyzer(OWNER_NAME, client=mock_client)
owner_analyzer.analyze_owner(OWNER_NAME)
assert classUnderTest.owner_name == 'test_owner'
I don't think the mock value is being returned in the get_owners call because I get something like for main_owner
owner is : <MagicMock name='mock.get_owners().main_owner' id='140420863948896'>.
Thanks to #jonrsharpe for pointing me in the right direction.
I was able to get this working by updating my mock setup to -
mock_client.get_owners.return_value = ListOwnerDetails(
main_owner=OwnerDetails(name='test_owner', type='User'), secondary_owners=[])
I'm trying to mock out urllib.request.urlopen's Read method on Python 3:
Function Code:
try:
with request.urlopen(_webhook_url, json.dumps(_message).encode('utf-8')) as _response:
_response_body = _response.read()
return _response_body
Test Code:
with mock.patch('urllib.request.urlopen') as mock_urlopen:
response_mock = MagicMock()
response_mock.read.return_value = 'ok'
mock_urlopen.return_value = response_mock
with self.stubber:
_response = NotifySlack.lambda_handler(_event)
self.assertEqual('ok', _response)
If I call response_mock.read() I get the 'ok' value returned, however when I assert the return value I get a mock signature:
Expected :ok
Actual :<MagicMock name='urlopen().__enter__().read()' id='2148156925992'>
Any ideas on why the mock isn't returning the value assigned to read()?
To follow #jonrsharpe's comment and the Python: Mocking a context manager thread, to properly mock the context manager in this case, you would need this interestingly looking line:
mock_urlopen.return_value.__enter__.return_value.read.return_value = 'ok'
#^^^^^^context manager to return response^^^^^^^|^^^read method^^^
I am in the process of learning unit testing, however I am struggling to understand how to mock functions for unit testing. I have reviewed many how-to's and examples but the concept is not transferring enough for me to use it on my code. I am hoping getting this to work on a actual code example I have will help.
In this case I am trying to mock isTokenValid.
Here is example code of what I want to mock.
<in library file>
import xmlrpc.client as xmlrpclib
class Library(object):
def function:
#...
AuthURL = 'https://example.com/xmlrpc/Auth'
auth_server = xmlrpclib.ServerProxy(AuthURL)
socket.setdefaulttimeout(20)
try:
if pull == 0:
valid = auth_server.isTokenValid(token)
#...
in my unit test file I have
import library
class Tester(unittest.TestCase):
#patch('library.xmlrpclib.ServerProxy')
def test_xmlrpclib(self, fake_xmlrpclib):
assert 'something'
How would I mock the code listed in 'function'? Token can be any number as a string and valid would be a int(1)
First of all, you can and should mock xmlrpc.client.ServerProxy; your library imports xmlrpc.client as a new name, but it is still the same module object so both xmlrpclib.ServerProxy in your library and xmlrpc.client.ServerProxy lead to the same object.
Next, look at how the object is used, and look for calls, the (..) syntax. Your library uses the server proxy like this:
# a call to create an instance
auth_server = xmlrpclib.ServerProxy(AuthURL)
# on the instance, a call to another method
valid = auth_server.isTokenValid(token)
So there is a chain here, where the mock is called, and the return value is then used to find another attribute that is also called. When mocking, you need to look for that same chain; use the Mock.return_value attribute for this. By default a new mock instance is returned when you call a mock, but you can also set test values.
So to test your code, you'd want to influence what auth_server.isTokenValid(token) returns, and test if your code works correctly. You may also want to assert that the correct URL is passed to the ServerProxy instance.
Create separate tests for different outcomes. Perhaps the token is valid in one case, not valid in another, and you'd want to test both cases:
class Tester(unittest.TestCase):
#patch('xmlrpc.client.ServerProxy')
def test_valid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response for a valid token
mock_auth_server.isTokenValid.return_value = 1
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
#patch('xmlrpc.client.ServerProxy')
def test_invalid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response; now testing for an invalid token instead
mock_auth_server.isTokenValid.return_value = 0
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
There are many mock attributes to use, and you can change your patch decorator usage a little as follows:
class Tester(unittest.TestCase):
def test_xmlrpclib(self):
with patch('library.xmlrpclib.ServerProxy.isTokenValid') as isTokenValid:
self.assertEqual(isTokenValid.call_count, 0)
# your test code calling xmlrpclib
self.assertEqual(isTokenValid.call_count, 1)
token = isTokenValid.call_args[0] # assume this token is valid
self.assertEqual(isTokenValid.return_value, 1)
You can adjust the code above to satisfy your requirements.
I have a test class and a setup function that looks like this:
#pytest.fixture(autouse=True, scope='function')
def setup(self, request):
self.client = MyClass()
first_patcher = patch('myclass.myclass.function_to_patch')
first_mock = first_patcher.start()
first_mock.return_value = 'foo'
value_to_return = getattr(request, 'value_name', None)
second_patcher = patch('myclass.myclass.function_two')
second_mock = second_patcher.start()
second_mock.return_value = value_to_return
#could clean up my mocks here, but don't care right now
I see in the documentation for pytest, that introspection can be done for a module level value:
val = getattr(request.module, 'val_name', None)
But, I want to be able to specify different values to return based on the test I am in. So I am looking for a way to introspect the test_function not the test_module.
http://pytest.org/latest/fixture.html#fixtures-can-introspect-the-requesting-test-context
You can use request.function to get to the test function. Just follow the link on the b wepage you referenced to see what is available on the test request object :)
Maybe the documentation has changed since the time of the accepted answer.
At least for me it was not clear how to
Just follow the link
So I thought I'd update this thread with the link itself:
https://pytest.org/en/6.2.x/reference.html#request
Edit December 2021
Even when the link is correct now I think this statement from the pytest documentation is just not correct:
Fixture functions can accept the request object to introspect the “requesting” test function ...
While I found some examples for getting attributes of the module I did not find a single working example of introspecting the test function that requests the fixture. May be related to collection and runtime order.
What really helped me to get the desired behavior was to use the factory idiom a little further down in the pytest documentation:
https://pytest.org/en/6.2.x/fixture.html#factories-as-fixtures
Set up the fixture factory
#pytest.fixture(scope='function')
def getQueryResult() -> object:
def _impl(_mrId: int = 7622):
return QueryResult(_mrId)
return _impl
Usage
# Concrete value
def test_foo(getQueryResult):
queryResult = getQueryResult(4711)
...
# Default value
def test_bar(getQueryResult):
queryResult = getQueryResult()
...