I've recently started using pytest, and even more recently started using mock for mocking the requests library. I have made a requests.Response object okay, and for a 200 status code it works fine. What I'm trying to do here, is to use raise_for_status() to check for a rate limit exceeded error, and test that it handles the exception with pytest.
I'm using the Mock side_effect option, which seems to fire the exception I'm hoping, but pytest doesn't seem to recognise this as having happened and fails the test.
Any thoughts? I'm sure it's something obvious I'm missing!
The code I have for the class is:
class APIClient:
def get_records(self, url):
try:
r = requests.get(url)
r.raise_for_status()
return r.json()
except requests.HTTPError as e:
print("Handling the exception")
In the test class, I have got:
#pytest.fixture
def http_error_response(rate_limit_json):
mock_response = mock.Mock()
mock_response.json.return_value = rate_limit_json
mock_response.status_code = 429
mock_response.raise_for_status.side_effect = requests.exceptions.HTTPError
return mock_response
class TestRecovery(object):
#mock.patch('requests.get')
def test_throws_exception_for_rate_limit_error\
(self, mock_get, api_query_object, http_error_response):
mock_get.return_value = http_error_response
print(http_error_response.raise_for_status.side_effect)
url = api_query_object.get_next_url()
with pytest.raises(requests.exceptions.HTTPError):
api_query_object.get_records(url)
The output I get is:
with pytest.raises(requests.exceptions.HTTPError):
> api_query_object.get_records(url)
E Failed: DID NOT RAISE
---------------------- Captured stdout call ----------------------
<class 'requests.exceptions.HTTPError'>
Handling the exception
You are instructing pytest to expect an exception that should be raised in APIClient.get_records but inside that method definition you are already capturing the exception and just doing a print.
The exception is actually happening and it proved by seeing the result of your print in the console output.
Instead of that you should either check with the mock that the method raise_for_status was called.
Related
I'm writing some unit tests where I mock out requests to another service. Here's the method I'm testing.
def ping_camera(self, camera):
retry_ping = Retry(
total=3,
status_forcelist=[404, 429, 500, 502, 503, 504],
method_whitelist=["GET"],
backoff_factor=1,
)
adapter = HTTPAdapter(max_retries=retry_ping)
http = requests.Session()
http.mount("http://", adapter)
try:
response = http.get("http://{}".format(camera.ip_address), timeout=3)
response.raise_for_status()
except Exception as e:
camera.camera_status = False
camera.save()
else:
camera.camera_status = True
camera.save()
And here's one of my tests.
#patch.object(Session, "get")
def test_expects_true_camera_status_to_return_false_status(self, mock):
# Arrange
mock.side_effect = Exception
# Act
check_cams = CheckCameras()
check_cams.ping_camera(self.camera)
# Assert
self.assertFalse(self.camera.camera_status)
Note that in order to have an exception raised, I need to add mock.side_effect = Exception. I come from the Laravel world where only the response status needs to be specified and if a bad response is returned, an exception will be raised.
I can even specify a good response, and still trigger an exception like so
mock.return_value.status_code = 200
mock.side_effect = Exception
Is this just a quirk of a Python testing, or can I have exceptions raised based off the response status code alone?
Meaning if I add mock.return_value.status_code = 500 I'll get an exception raised and hit the except portion of my code block.
I want to test exceptions with Pytest. I send an HTTP request and get a response. I want the response to be corrupt so response.json() goes to the except block. Below are the examples.
Send a request, receive a response:
def send_message_json():
# ...
try:
response = cls.send_message(method, url, **kwargs)
if response:
return response.json() # this is what should fail
except simplejson.errors.JSONDecodeError as err:
raise err # this is to be achieved
The unit test should assert that the simplejson.errors.JSONDecodeError should be raised.
#mock.patch.object(Service, 'send_message_json')
def test_send_message_json_exception(mock_send):
svc = Service()
with pytest.raises(simplejson.errors.JSONDecodeError): # this should assert the exceptions was raised
svc.send_message_json("GET", 'http://my.url/')
I fail to activate the exception raise by the pytest.mock.object. What would make .json() fail in the mock?
I figured it out by mocking the response of the actual request.
In the original question there's send_message_json() method.
It calls send_message() method that returns requests.Response. So I mocked the return value of send_message() that is also a mock instead of send_message_json():
#mock.patch.object(Service, 'send_message') # mock send_message instead
def test_send_message_json_exception(mock_send):
svc = Service()
mocked_response = Response()
mocked_response.status_code = 200
mocked_response.data = '<!doctype html>' # so this is obviously not a JSON
mock_send.return_value = mocked_response # apply the mocked response here
with pytest.raises(simplejson.errors.JSONDecodeError):
svc.send_message_json("GET", 'http://my.url/') # this calls send_message() and returns a bad mocked requests.Response
The bad mocked requests.Response fails at mocked_response.json() in send_message_json() and raises simplejson.errors.JSONDecodeError
I have a simple function that sends a get request to an API, and I want to be able to write a Unit test to assert that the scriprt prints an error and then exits if the request returns an exception.
def fetch_members(self):
try:
r = requests.get(api_url, auth=('user', api_key))
except requests.exceptions.HTTPError as error:
print(error)
sys.exit(1)
I've tried the following test but the HTTPError is never raised
import requests
from unittest.mock import patch
#patch('sys.exit')
#patch('sys.stdout', new_callable=StringIO)
#patch('requests.get')
def test_fetch_members_with_bad_request(self, mock_get, mock_stdout,
mock_exit):
response = requests.Response()
response.status_code = 400 # Bad reqeust
mock_get.return_value = response
mock_get.side_effect = requests.exceptions.HTTPError
with self.assertRaises(requests.exceptions.HTTPError):
fetch_members()
mock_exit.assert_called_once_with(1)
self.assertIsNotNone(mock_stdout.getvalue())
How would I go about correctly patching requests.get to always raise some sort of error?
Your function catches and handles the exception:
except requests.exceptions.HTTPError as error:
This means it'll never propagate further, so your assertRaises() fails. Just assert that sys.exit() has been called, that's enough.
There is also no point in setting a return value; the call raises an exception instead. The following should suffice:
#patch('sys.exit')
#patch('sys.stdout', new_callable=StringIO)
#patch('requests.get')
def test_fetch_members_with_bad_request(self, mock_get, mock_stdout,
mock_exit):
mock_get.side_effect = requests.exceptions.HTTPError
fetch_members()
mock_exit.assert_called_once_with(1)
self.assertIsNotNone(mock_stdout.getvalue())
In an attempt to use patch from mock to write unit tests to validate our custom urllib2 error handler that logs the error before raising a custom error, I cannot get the custom error to raise. The mocked exception will always raise after the log line in LoggingErrorHandler executes successfully. i.e. my test code is looking for CustomError, but the function still, surprisingly, raises the urllib2.HTTPError.
Test code:
#raises(CustomError)
def test_404_error_logging(self):
''' not the copy and paste code; enough to give you an idea '''
http_error = urllib2.HTTPError("mock_url", 404, "Not Found", "mock_headers", None)
with patch('function.that.invokes.error.handler', side_effect=http_error):
test_client.get("bad url")
Custom error handler for urllib2 HTTPError handling:
class LoggingErrorHandler(urllib2.HTTPErrorProcessor):
''' not the copy and paste code; enough to give you an idea '''
def http_error_404(self, req, fp, code, msg, headers):
log.debug("Look! An error! {0} {1}".format(code, msg))
raise CustomError(req.get_full_url(), code, msg, body, headers)
CustomError is also our custom exception.
I am having problems with a delete request in Tornado. The request arrives to the server and everything in the handler work great, but it never returns the response to the client.
I have tried returning something, with only the "return" and even without the "return" and the result is always the same.
I am using Python 3.4, Tornado 4.1 and the RestClient of Firefox.
#web.asynchronous
#gen.coroutine
def delete(self, _id):
try:
model = Model()
model.delete(_id)
self.set_status(204)
except Exception as e:
logging.error(e)
self.set_status(500)
return
Tornado documentation (tornado.web.asynchronous):
If this decorator is given, the response is not finished when the method > returns. It is up to the request handler to call self.finish() to finish > the HTTP request.
You need to call tornado.web.RequestHandler.finish method. This will work:
#web.asynchronous
#gen.coroutine
def delete(self, _id):
try:
model = Model()
model.delete(_id)
self.set_status(204)
except Exception as e:
logging.error(e)
self.set_status(500)
self.finish()
return
However, you don't need asynchronous approach in this example. This will work also in the same way:
def delete(self, _id):
try:
model = Model()
model.delete(_id)
self.set_status(204)
except Exception as e:
logging.error(e)
self.set_status(500)
return
Also, if you are using #gen.coroutine decorator, you don't need to use #web.asynchronous decorator. Simply use only #gen.coroutine, it is the correct way and much more elegant.
Lastly, I think you should read this article for understanding asynchronous programming in Tornado.