I have a simple router designed to throw an HTTPException:
#router.get('/404test')
async def test():
raise HTTPException(HTTP_404_NOT_FOUND, "404 test!")
I want to assert that the exception was thrown, as per FastaAPI docs:
def test_test():
response = client.get("/404test")
assert response.status_code == 404
The exception is thrown before the assertion gets evaluated, marking test as failed:
> raise HTTPException(HTTP_404_NOT_FOUND, "404 test!")
E fastapi.exceptions.HTTPException: (404, '404 test!')
What am I missing to properly anticipate HTTPExceptions in my test?
Assuming we have the following route set up in our fastapi app:
#router.get('/404test')
async def test():
raise HTTPException(HTTP_404_NOT_FOUND, "404 test!")
I was able to get a pytest to work with the following code snippet:
from fastapi import HTTPException
def test_test():
with pytest.raises(HTTPException) as err:
client.get("/404test")
assert err.value.status_code == 404
assert err.value.detail == "404 test!"
It seems that the err is the actual HTTPException object, not the json representation. When you catch this error you can then make assertions on that HTTPException object.
Make sure you run the assertions (assert) outside of the with statement block because when the error is raised, it stops all execution within the block after the http call so your test will pass but the assertions will never evaluate.
You can reference the details and the status code and any other attributes on the Exception with err.value.XXX.
May be you can do this using the following sample code.
~/Desktop/fastapi_sample $ cat service.py
from fastapi import FastAPI, HTTPException
app = FastAPI()
#app.get("/wrong")
async def wrong_url():
raise HTTPException(status_code=400, detail="404 test!")
~/Desktop/fastapi_sample $ cat test_service.py
from fastapi.testclient import TestClient
from fastapi_sample.service import app
client = TestClient(app)
def test_read_item_bad_token():
response = client.get("/wrong")
assert response.status_code == 400
assert response.json() == {"detail": "404 test!"}%
~/Desktop/fastapi_sample $ pytest
==================================================================== test session starts ====================================
platform darwin -- Python 3.7.9, pytest-6.1.0, py-1.9.0, pluggy-0.13.1
rootdir: /Users/i869007/Desktop/workspace/SAP/cxai/fastapi_postgres_tutorial
collected 1 item
test_service.py . [100%]
===================================================================== 1 passed in 0.78s ======================================
Related
I am struggling to write test cases that will trigger an Exception within one of my FastAPI routes. I was thinking pytest.Raises would do what I intend, however that by itself doesn't seem to be doing what I thought it would.
Since the TestClient runs the API client pretty much separately, it makes sense that I would have this issue - that being said, I am not sure what the best practice is to ensure a high code coverage in testing.
Here is my test function:
def test_function_exception():
with pytest.raises(Exception):
response = client.post("/")
assert response.status_code == 400
and here is the barebones route that I am hitting:
#router.post("/")
def my_function():
try:
do_something()
except Exception as e:
raise HTTPException(400, "failed to do something")
Is there anyway that I can catch this Exception without making changes to the API route? If changes are needed, what are the changes required to ensure thorough testing?
Following the discussion below the question, I assembled a working example for you. Typically, if you can't logically hit your except block, you can ensure that the try block is raising an Exception by monkey patching the function that is tried, and replace it with something that definitely will raise an exception. In the below example, I will change the function do_something() that is defined in app.py with replace_do_something() that will just raise an Exception when called.
You can put the following files in the same folder (not a module) and try it for yourself:
File app.py:
from fastapi import FastAPI, HTTPException
from fastapi.testclient import TestClient
import pytest
app = FastAPI()
def do_something():
return "world"
#app.get("/myroute")
async def myroute():
try:
text = do_something()
return {"hello": text}
except Exception:
raise HTTPException(400, "something went wrong")
File test_app.py:
import pytest
from fastapi.testclient import TestClient
from app import app
client = TestClient(app)
def replace_do_something():
raise Exception()
return
def test_read_main(monkeypatch: pytest.MonkeyPatch):
response = client.get("/myroute")
assert response.status_code == 200
assert response.json() == {"hello": "world"}
def test_read_main_with_error(monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr("app.do_something", replace_do_something)
# Here we replace any reference to do_something
# with replace_do_something. Note the 'app.' prefix!
response = client.get("/myroute")
assert response.status_code == 400
assert response.json() == {"detail": "something went wrong"}
You can call the test_app.py file with pytest (I also have pytest-cov installed to demonstrate the 100% coverage):
(venv) jarro#MacBook-Pro-van-Jarro fastapi-github-issues % pytest --cov=app SO/pytestwithmock/test_app.py
===================================================== test session starts =====================================================
platform darwin -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/jarro/Development/fastapi-github-issues
plugins: anyio-3.6.1, cov-3.0.0
collected 2 items
SO/pytestwithmock/test_app.py .. [100%]
---------- coverage: platform darwin, python 3.10.5-final-0 ----------
Name Stmts Miss Cover
----------------------------------------------
SO/pytestwithmock/app.py 13 0 100%
----------------------------------------------
TOTAL 13 0 100%
====================================================== 2 passed in 0.19s ======================================================
I'm trying make some api server with FastAPI.
I have one endpoint named /hello on my project, which gives:
{msg : "Hello World"}
with JSON format when 200 status.
However, It gives error msg when request fails.
Quite simple service. However, I want to test both cases, just for my study. So I also made test code with pytest.
Now I want to know: how can I raise HTTPException and test it on purpose?
#main.py (FAST API)
#app.get('/hello')
def read_main():
try:
return {"msg":"Hello World"}
except requests.exceptions.HTTPError as e:
raise HTTPException(status_code=400,detail='error occured')
#test.py
from fastapi.testclient import TestClient
client = TestClient(app)
# This test works
def test_read_main():
response = client.get("/hello")
assert response.json() == {"msg":"Hello World"}
assert response.status_code == 200
def test_errors():
# How can I test except in endpoint "/hello" ?
# The code below never works as I expect
# with pytest.raises(HTTPException) as e:
# raise client.get("/hello").raise_for_status()
# print(e.value)
The problem here is that your logic is way to simplistic to test. As luk2302 said; in the current form, your except block is never called and thus can never be tested. Replacing your logic with something more testable, allows us to force an Exception being thrown.
File: app.py
from fastapi import FastAPI
from fastapi.exceptions import HTTPException
import requests
app = FastAPI()
#We've put this in a seperate function so we can mock this.
def get_value():
return {"msg":"Hello World"}
#app.get('/hello')
def read_main():
try:
return get_value()
except requests.exceptions.HTTPError as e:
raise HTTPException(status_code=400,detail='error occured')
Note that the return value of your endpoint is now actually provided by the get_value() function.
The test.py file would look like this:
from fastapi import HTTPException
import app
from fastapi.testclient import TestClient
import requests
from pytest_mock import MockerFixture
client = TestClient(app.app)
def test_read_main():
response = client.get("/hello")
assert response.json() == {"msg":"Hello World"}
assert response.status_code == 200
def get_value_raise():
raise requests.exceptions.HTTPError()
def test_errors(mocker: MockerFixture):
mocker.patch("app.get_value", get_value_raise)
response = client.get("/hello")
assert response.status_code == 400
assert response.json() == {"detail": "error occured"}
Note that we replace the app.get_value function with a function that will definitely raise the type of exception that you are catching in your application logic. The response of the test client is (however) just an HTTP response, but with statuscode 400 and a detail in the json body. We assert for that.
The result:
(.venv) jarro#MBP-van-Jarro test_http_exception % pytest test.py
=================================================== test session starts ===================================================
platform darwin -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/jarro/Development/fastapi-github-issues/SO/test_http_exception
plugins: anyio-3.6.1, mock-3.8.2
collected 2 items
test.py .. [100%]
==================================================== 2 passed in 0.17s ====================================================
I used pytest, and by extension I used pytest-mocker to mock the get_value function.
I built an API with FastAPI that interacts with DynamoDB.
In the beginning of my journey in Test Driven Development, I have doubts about what to mock.
This is the get method, main.py:
router = FastAPI()
#router.get("/{device_id}")
def get_data(request: Request, device_id: str, query: DataQuery = Depends(DataQuery.depends)):
da_service = DaService()
try:
start_time, end_time = DaService.validate_dates(query.start, query.end)
return 'OK'
except WrongDataFormat as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail='Internal Server Error')
In the test file I started by creating the success test, test_main.py:
from fastapi.testclient import TestClient
from unittest import mock
from utils.exceptions import WrongDataFormat
from endpoints.datalake import router
client = TestClient(router)
def test_success_response():
with mock.patch('endpoints.datalake.DataApiService.get_datalake_data'):
response = client.get('/xxxxx', params = {'start': '1629886483', 'end': '1629886504'})
assert response.status_code == 200
assert isinstance(response.json(), dict)
Now I want to create the test for when the exception WrongDataFormat is returned, but I'm not succeeding... This is what I have right now:
def test_exception_response_():
response = client.get('/xxxxx', params = {'2021-08-28', 'end': '2021-12-25'})
assert response.status_code == 400
How can I mock the function main.validate_dates to return the exception WrongDataFormat and assert it correctly?
If you want to test the status code and message of a response you have to use TestClient(app) where app is the FastAPI application. Converting the exception into the appropriate response is the task of the application, not the router (which is what you're testing with).
client = TestClient(app)
This way you can test the API of your application (which is the most useful surface to test, imho).
I have a very simple fastapi application which i want to test , the code for dummy_api.py is as follows :
import os
from fastapi import FastAPI
app = FastAPI()
#app.get(os.getenv("ENDPOINT", "/get"))
def func():
return {
"message": "Endpoint working !!!"
}
When i want to test this i am using the below file :
from fastapi.testclient import TestClient
import dummy_api
def test_dummy_api():
client = TestClient(dummy_api.app)
response = client.get("/get")
assert response.status_code == 200
def test_dummy_api_with_envar(monkeypatch):
monkeypatch.setenv("ENDPOINT", "dummy")
client = TestClient(dummy_api.app)
response = client.get("/dummy")
assert response.status_code == 200
However i am unable to mock the environment variable part as one of the tests fail with a 404.
pytest -s -v
================================================================= test session starts ==================================================================
platform linux -- Python 3.8.5, pytest-6.2.2, py-1.9.0, pluggy-0.13.1 -- /home/subhayan/anaconda3/envs/fastapi/bin/python
cachedir: .pytest_cache
rootdir: /home/subhayan/Codes/ai4bd/roughdir
collected 2 items
test_dummy_api.py::test_dummy_api PASSED
test_dummy_api.py::test_dummy_api_with_envar FAILED
======================================================================= FAILURES =======================================================================
______________________________________________________________ test_dummy_api_with_envar _______________________________________________________________
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff8c4cf1430>
def test_dummy_api_with_envar(monkeypatch):
monkeypatch.setenv("ENDPOINT", "dummy")
client = TestClient(dummy_api.app)
response = client.get("/dummy")
> assert response.status_code == 200
E assert 404 == 200
E +404
E -200
test_dummy_api.py:15: AssertionError
=============================================================== short test summary info ================================================================
FAILED test_dummy_api.py::test_dummy_api_with_envar - assert 404 == 200
============================================================= 1 failed, 1 passed in 0.19s ==============================================================
Can anyone point out where am i going wrong please !!
You could use parametrized fixtures and the importlib.reload function to test that environment variable is indeed used.
My test directory looks like this:
.
└── tests
├── conftest.py
├── dummy_api.py
└── test_api.py
Here is my conftest.py:
import pytest
from fastapi.testclient import TestClient
from importlib import reload
import dummy_api
#pytest.fixture(params=["/get", "/dummy", "/other"])
def endpoint(request, monkeypatch):
monkeypatch.setenv("ENDPOINT", request.param)
return request.param
#pytest.fixture()
def client(endpoint):
app = reload(dummy_api).app
yield TestClient(app=app)
And here is the test_api.py file:
import os
def test_dummy_api(client):
endpoint = os.environ["ENDPOINT"]
response = client.get(endpoint)
assert response.status_code == 200
assert response.json() == {"message": f"Endpoint {endpoint} working !"}
Test output after running pytest:
collected 3 items
tests/test_api.py::test_dummy_api[/get] PASSED [ 33%]
tests/test_api.py::test_dummy_api[/dummy] PASSED [ 66%]
tests/test_api.py::test_dummy_api[/other] PASSED [100%]
To me it looks like this:
When the test file is loaded by pytest then all things that can be executed in dummy_api will be executed because it is imported.
This means that the decorator of the decorated function (i.e. #app.get(...)) will be executed. At this point in time monkey-patching has not kicked in.
Once the test function kicks in, the environment variable has been set too late.
I have a pytest test case like below:
#pytest.fixture
def app():
app = SampleApp().setup()
yield app
#pytest.fixture
def client(app):
return app.test_client()
def test_get_nonexistent_user(client, app):
with pytest.raises(NotFound) as err:
user = client.get('/users/5d4001f799556f10b7462e20')
assert err.type is NotFound
What I have read from pytest documentation I should be able to catch this exception. What I receive instead is as below:
1 failed in 0.25 seconds
And I see NotFound error in traceback error, so I'm sure code is raising the exception I expect it to.