In the pytest documentation it says that you can customize the output message when an assert fails. I want to customize the assert message when testing a REST API method it returns an invalid status code:
def test_api_call(self, client):
response = client.get(reverse('api:my_api_call'))
assert response.status_code == 200
So I tried to put a piece of code like this in conftest.py
def pytest_assertrepr_compare(op, left, right):
if isinstance(left, rest_framework.response.Response):
return left.json()
But the problem is left is the actual value of response.status_code so it's an int instead of a Response. However the default output messages throws something like:
E assert 400 == 201 E + where 400 = .status_code
Saying that the error 400 comes from an attribute status_code of a object Response.
My point is that there is a kind of introspection of the variable being evaluated. So, how can I customize the assert error message in a comfortable way to get a similar output to example posted above?
you could use Python built-in capability to show custom exception message:
assert response.status_code == 200, f"My custom msg: actual status code {response.status_code}"
Or you can built a helper assert functions:
def assert_status(response, status=200): # you can assert other status codes too
assert response.status_code == status, \
f"Expected {status}. Actual status {response.status_code}. Response text {response.text}"
# here is how you'd use it
def test_api_call(self, client):
response = client.get(reverse('api:my_api_call'))
assert_status(response)
also checkout: https://wiki.python.org/moin/UsingAssertionsEffectively
Related
Flask==1.1.2
Python==3.8
I am building a restAPI that is serving machine learning model. My co-worker who will be sending request to my restAPI and use the outcome to send it to users wants me to send him appropriate error message as well with status_code.
I've did a lot of searching on how to properly handle errors in Python Flask however I am still stuck on what would be a best practice for scalable and maintainable restAPI.
Currently whenever some error occurs I simply return dictionary with message and status code. There are some problems with this method that I want to mitigate:
If error occurs inside a function it has to return dictionary containing error messages to where the function was called and at need to check if it was actually an error, if yes then return error message
Example:
def add_data(x,y):
"""return addition of x,y. They both need to be integers"""
if type(x) != int:
return "x has wrong datatype"
if type(y) != int:
return "y has wrong datatype"
return x+y
#app.route("/predict", methods=["POST"])
def predict():
data = request.get_json()
result = add_data(data["x"], data["y"])
if type(result) == str:
return {"message":"input error", "status":222}
Cannot break code inside a function.
following some references
Custom Python Exceptions with Error Codes and Error Messages
What is best practice for flask error handling?
I've changed my code to following:
class InputError(Exception):
status_code = 400
def __init__(self, message, status_code=None):
Exception.__init__(self)
self.message = message
if status_code is not None:
self.status_code = status_code
def __str__(self):
return repr(self.status_code)
def add_data(x,y):
if type(x) != int:
raise InputError("x has wrong datatype", status_code=222)
if type(y) != int:
raise InputError("y has wrong datatype", status_code=222)
return x+y
This does break the code where error is found however I cannot find out how to return dictionary just like before.
How can I do this and which practice is considered a best practice?
The solution is to use error handlers
https://flask.palletsprojects.com/en/1.1.x/errorhandling/
In your case:
#app.errorhandler(InputError)
def handle_input_error(e):
return {"message": e["message"], "status": e["status"]}
Now whenever you raise InputError somewhere in the code, flask will know to call this method and return this response
If you have more types of errors I would switch to something more general
class MyErrors(Exception):
status_code: int
def __init__(self, message):
super(MyErrors, self).__init__(message)
self.message = message
def response(self):
return {"message": self.message, "status": self.status_code}
class InputError(MyErrors):
status_code = 222
class SomeOtherError(MyErrors):
status_code = 123
#app.errorhandler(MyErrors)
def handle_errors(e):
return e.response()
Hi I am new to unit testing and I am digging into mocks and pytest.I am trying to unit test two rest api requests where the GET request checks if an item doesn't exist in the API and if it does not create it with POST request and to create a folder, otherwise if it exists just to create a folder.
I tried to use Mocker() and I am stuck on AttributeError: Mocker when I try to mock the GET request.
This is the code I am trying to test:
client = requests.session()
# get item by name
response = client.get(
f"https://url.com/rest/item/info?name=test_item",
auth=username, password),
)
if (
response.status_code == 200
and response.json()["status"]
== "Item doesn't exist!"
):
logging.info(f"Creating item")
client.post(
"https://url.com/rest/item/create?name=test_item",
auth=username, password),
)
# directory creation
dir_make = "mkdir -p test_item/temperature"
exec_cmd(dir_make)
elif response.status_code == 200 and response.json()["status"]=="OK":
# directory creation
dir_make = "mkdir -p test_item/temperature"
exec_cmd(dir_make)
And this is the unit test that fails with AttributeError:
def test_existing_item(requests_mock, monkeypatch):
with requests_mock.Mocker() as mock:
mock.get("https://url.com/rest/item/info?name=test_item", text="OK")
resp = requests.get("https://url.com/rest/item/info?name=test_item")
assert resp.text == "OK"
EDIT: Test for item not found and POST mock. It seems like it doesn't add coverage to the else statement. How can be tested if the item exists and only the folder needs to be added in that case?
EDIT 2: Added elif statement instead of else and 2 separate tests, still the one test_existing_items() doesn't cover the elif statement...What am I doing wrong in that case?
def test_existing_item(monkeypatch):
with requests_mock.Mocker() as mock_request:
mock_request.get(requests_mock.ANY, text="success!")
resp = requests.get(
"https://url.com/rest/item/info?name=test_item",
auth=("mock_username", "mock_password"),
)
if resp.status_code == 200 and resp.json()["status"] == "OK":
dir_make = "mkdir -p test_item/temperature"
exec_cmd(dir_make)
encoded_auth = b64encode(b"mock_username:mock_password").decode("ascii")
assert mock_request.last_request.headers["Authorization"] == f"Basic {encoded_auth}"
def test_post_item(monkeypatch):
with requests_mock.Mocker() as mock_request:
mock_request.get(requests_mock.ANY, text="success!")
resp = requests.get(
"https://url.com/rest/item/info?name=test_item",
auth=("mock_username", "mock_password"),
)
if resp.status_code == 200 and resp.json()["status"] == "ERROR":
mock_request.get(requests_mock.ANY, text="success!")
requests.post(
"https://url.com/rest/item/create?name=test_item",
auth=("mock_username", "mock_password"),
)
dir_make = "mkdir -p test_item/temperature"
exec_cmd(dir_make)
encoded_auth = b64encode(b"mock_username:mock_password").decode("ascii")
assert mock_request.last_request.headers["Authorization"] == f"Basic {encoded_auth}"
I am not familiar with unit testing so any help would be appreciated to unit test this code.
import requests
import requests_mock
def test_existing_item(monkeypatch):
with requests_mock.Mocker() as mock:
mock.get("https://url.com/rest/item/info?name=test_item", text="OK")
resp = requests.get("https://url.com/rest/item/info?name=test_item")
assert resp.text == "OK"
Don't pass requests_mock as parameter and pytest should work fine.
EDIT:
As for your edit:
It seems like it doesn't add coverage to the else statement. How can be tested if the item exists and only the folder needs to be added in that case?
That would be because your if condition is always true, so it never accesses the code below the else statement. Your second question is rather unclear to me, but I believe you want to write several tests, one for your if statement and one for your else statement. As a rule of thumb, if you need conditional logic in your test, you have a problem: either everything should go as you want it to every time you run your tests, either you should abort and have the test fail, as you want your code to have the exact same behavior every time you run it - and your tests as well, by extension.
I've recently started using pytest, and even more recently started using mock for mocking the requests library. I have made a requests.Response object okay, and for a 200 status code it works fine. What I'm trying to do here, is to use raise_for_status() to check for a rate limit exceeded error, and test that it handles the exception with pytest.
I'm using the Mock side_effect option, which seems to fire the exception I'm hoping, but pytest doesn't seem to recognise this as having happened and fails the test.
Any thoughts? I'm sure it's something obvious I'm missing!
The code I have for the class is:
class APIClient:
def get_records(self, url):
try:
r = requests.get(url)
r.raise_for_status()
return r.json()
except requests.HTTPError as e:
print("Handling the exception")
In the test class, I have got:
#pytest.fixture
def http_error_response(rate_limit_json):
mock_response = mock.Mock()
mock_response.json.return_value = rate_limit_json
mock_response.status_code = 429
mock_response.raise_for_status.side_effect = requests.exceptions.HTTPError
return mock_response
class TestRecovery(object):
#mock.patch('requests.get')
def test_throws_exception_for_rate_limit_error\
(self, mock_get, api_query_object, http_error_response):
mock_get.return_value = http_error_response
print(http_error_response.raise_for_status.side_effect)
url = api_query_object.get_next_url()
with pytest.raises(requests.exceptions.HTTPError):
api_query_object.get_records(url)
The output I get is:
with pytest.raises(requests.exceptions.HTTPError):
> api_query_object.get_records(url)
E Failed: DID NOT RAISE
---------------------- Captured stdout call ----------------------
<class 'requests.exceptions.HTTPError'>
Handling the exception
You are instructing pytest to expect an exception that should be raised in APIClient.get_records but inside that method definition you are already capturing the exception and just doing a print.
The exception is actually happening and it proved by seeing the result of your print in the console output.
Instead of that you should either check with the mock that the method raise_for_status was called.
This is the method in ReportRunner class in report_runner.py in my Flask-Restful app:
class ReportRunner(object):
def __init__(self):
pass
def setup_routes(self, app):
app.add_url_rule("/run_report", view_func=self.run_report)
def request_report(self, key):
# code #
def key_exists(self, key):
# code #
def run_report(self):
key = request.args.get("key", "")
if self.key_exists(key):
self.request_report(report_type, key)
return jsonify(message = "Success! Your report has been created.")
else:
response = jsonify({"message": "Error => report key not found on server."})
response.status_code = 404
return response
and the nose test calls the URL associated with that route
def setUp(self):
self.setup_flask()
self.controller = Controller()
self.report_runner = ReportRunner()
self.setup_route(self.report_runner)
def test_run_report(self):
rr = Report(key = "daily_report")
rr.save()
self.controller.override(self.report_runner, "request_report")
self.controller.expectAndReturn(self.report_runner.request_report("daily_report"), True )
self.controller.replay()
response = self.client.get("/run_report?key=daily_report")
assert_equals({"message": "Success! Your report has been created."}, response.json)
assert_equals(200, response.status_code)
and the test was failing with the following message:
AttributeError: 'Response' object has no attribute 'json'
but according to the docs it seems that this is how you do it. Do I change the return value from the method, or do I need to structure the test differently?
The test is now passing written like this:
json_response = json.loads(response.data)
assert_equals("Success! Your report has been created.", json_response["message"])
but I'm not clear on the difference between the two approaches.
According to Flask API Response object doesn't have attribute json (it's Request object that has it). So, that's why you get exception. Instead, it has generic method get_data() that returns the string representation of response body.
json_response = json.loads(response.get_data())
assert_equals("Success! Your report has been created.", json_response.get("message", "<no message>"))
So, it's close to what you have except:
get_data() is suggested instead of data as API says: This should not be used and will eventually get deprecated.
reading value from dictionary with get() to not generate exception if key is missing but get correct assert about missing message.
Check this Q&A also.
I'm trying to create a Mock of a library's (Hammock) call to a POST method that has an attribute status_code. Here is my test code:
def test_send_text(self):
Hammock.POST = Mock(status_code=201)
print Hammock.POST.status_code
self.task.payload = fixtures.text_payload
self.task.send_text()
# ········
Hammock.POST.assert_any_call()
When I print Hammock.POST.status_code, I get what I expect -- 201. However, when we move into the code I'm testing:
response = self.twilio_api('Messages.json').POST(data=self.payload)
print '*' * 10
print response
print response.status_code
if response.status_code == 201:
self.logger.info('Text message successfully sent.')
else:
raise NotificationDispatchError('Twilio request failed. {}. {}'.format(response.status_code,
response.content))
Things get weird. response is, indeed, a Mock object. But response.status_code, instead of being 201 like when I try it in the test, is an object: <Mock name='mock().status_code' id='4374061968'>. Why is my mocked attribute working in the test code and not in the code that I'm testing?
The issue is POST().status_code vs POST.status_code. POST.status_code will indeed == 201, but the return object from POST() is a completely new mock object.
What you are looking for is roughly
Hammock.POST = Mock()
Hammock.POST.return_value = Mock(status_code=201)