I am trying to have working tests for an API written with FastAPI.
I start the service in a separate process, run the tests with requests to the service, and check if the results are as expected.
I have extracted the key parts into a minimal working example, in the PD.
Running the MWE with the main file works fine. The tests fail, though.
Why do the tests fail?
How should we test APIs?
PD: the code is in a GIST, but now also here:
README.md
# Minimal example to ask in StackOverflow
Pytest does not allow to start a process with a service and tests requests to it,
at least not in the most straightforward way IMHO.
I may be missing something, or I may be doing something wrong.
Hence, I share this short code in a gist, to ask.
## How to run it
The dependencies are: `fastapi pytest requests uvicorn`.
You may install them with your package / environment manager of choice,
or use `pipenv install` with the provided `Pipfile`.
To run the code in the environment (e.g. `pipenv shell`), run: `python3 mwe.py`.
You should see everything is `OK`.
To run the test, run in the environment: `pytest`.
This does not work for me, the request times out.
mwe.py
import fastapi, multiprocessing, requests, time, uvicorn
app = fastapi.FastAPI()
#app.get('/ok')
def ok():
return 'OK'
class service:
def __enter__(self):
def run_service():
uvicorn.run('mwe:app', host='0.0.0.0', port=8000, reload=True)
self.service = multiprocessing.Process(target=run_service)
self.service.start()
time.sleep(10)
def __exit__(self, *args):
self.service.terminate()
def main():
with service():
return requests.get('http://127.0.0.1:8000/ok').ok
if __name__ == '__main__':
print('🆖🆗'[main()])
Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pytest = "*"
fastapi = "*"
requests = "*"
uvicorn = "*"
[requires]
python_version = "3.10"
python_full_version = "3.10.6"
test_mwe.py
from mwe import main
def test_main():
assert main()
Example for testing from official docs
from fastapi import FastAPI
from fastapi.testclient import TestClient
app = FastAPI()
#app.get("/")
async def read_main():
return {"msg": "Hello World"}
client = TestClient(app)
def test_read_main():
response = client.get("/")
assert response.status_code == 200
assert response.json() == {"msg": "Hello World"}
So, for you example from attached git it can be rewrited to something like this:
def test_get_ok():
response = client.get("/ok")
assert response.status_code == 200
assert response.text() == 'OK'
In a nutshell, it is difficult to describe how to test API correctly, but you can take as an example an excellent template for designing RESTfull API on FastAPI in which there are examples of autotests written just with the Pytest
Related
I'm getting started with Flask and Pytest in order to implemente a rest service with unit test, but i'm having some troouble.
I'll like to make a simple test for my simple endpoint but i keep getting a Working outside of application context. error when running the test.
This is the end point:
from flask import jsonify, request, Blueprint
STATUS_API = Blueprint('status_api', __name__)
def get_blueprint():
"""Return the blueprint for the main app module"""
return STATUS_API
#STATUS_API.route('/status', methods=['GET'])
def get_status():
return jsonify({
'status' : 'alive'
})
And this is how I'm trying to test it (i know it should fail the test):
import pytest
from routes import status_api
def test_get_status():
assert status_api.get_status() == ''
I'm guessing I just cant try the method with out building the whole app. But if that's the case i dont really know how to aproach this problem
The Flask documentation on testing is pretty good.
Instead of importing the view functions, you should create a so called test client, e.g. as a pytest fixture.
For my last Flask app this looked like:
#pytest.fixture
def client():
app = create_app()
app.config['TESTING'] = True
with app.app_context():
with app.test_client() as client:
yield client
(create_app is my app factory)
Then you can easily create tests as follows:
def test_status(client):
rv = client.get('/stats')
assert ...
As mentioned at the beginning, the official documentation is really good.
Have you considered trying an API client/development tool? Insomnia and Postman are popular ones. Using one may be able to resolve this for you.
This is really strange. I have the following simple flask application:
- root
- myapp
- a route with /subscription_endpoint
- tests
- test_az.py
- test_bz.py
test_az.py and test_bz.py look both the same. There is a setup (taken from https://diegoquintanav.github.io/flask-contexts.html) and then one simple test:
import pytest
from myapp import create_app
import json
#pytest.fixture(scope='module')
def app(request):
from myapp import create_app
return create_app('testing')
#pytest.fixture(autouse=True)
def app_context(app):
"""Creates a flask app context"""
with app.app_context():
yield app
#pytest.fixture
def client(app_context):
return app_context.test_client(use_cookies=True)
def test_it(client):
sample_payload = {"test": "test"}
response = client.post("/subscription_endpoint", json=sample_payload)
assert response.status_code == 500
running pytest, will run both files, but test_az.py will succeed, while test_bz.py will fail. The http request will return a 404 error, meaning test_bz cannot find the route in the app.
If I run them individually, then they booth succeed. This is very strange! It seems like the first test is somehow influencing the second test.
I have added actually a third test test_cz.py, which will fail as well. So only the first one will ever run. I feel like this has something todo with those fixtures, but no idea where to look.
Create a conftest.py for fixtures e.g. for client fixture and use the same fixture in both tests?
Now if you're saying that the provided code is the example of a test that is the same in another file, then you are creating 2 fixtures for a client. I would first clean it up and create a 1 conftest.py that contains all the fixtures and then use them in your tests this might help you.
Check out also how to use pytest as described in Flask documentation
I have a fastapi project built by poetry. I want to run the application with a scripts section in pyproject.tom like below:
poetry run start
What is inside double quotes in the section?
[tool.poetry.scripts]
start = ""
I tried to run the following script.
import uvicorn
from fastapi import FastAPI
app = FastAPI()
#app.get("/")
async def root():
return {"message": "Hello World"}
def main():
print("Hello World")
uvicorn.run(app, host="0.0.0.0", port=8000, reload=True, workers=2)
if __name__ == "__main__":
main()
It stops the application and just shows warning like this.
WARNING: You must pass the application as an import string to enable 'reload' or 'workers'.
I found the solution to this problem. See below:
In pyproject.toml
[tool.poetry.scripts]
start = "my_package.main:start"
In your main.py inside my_package folder.
import uvicorn
from fastapi import FastAPI
app = FastAPI()
#app.get("/")
async def root():
return {"message": "Hello World"}
def start():
"""Launched with `poetry run start` at root level"""
uvicorn.run("my_package.main:app", host="0.0.0.0", port=8000, reload=True)
You will need to pass the module path (module:function) to the start script in project.toml:
[tool.poetry.scripts]
start = "app:main"
Now run the command below will call the main function in the app module:
$ poetry run start
Just as the error message says, do
uvicorn.run("app")
Note also using reload and workers is useless and will just use the reloader. These flags are mutually exclusive
WARNING: You must pass the application as an import string to enable
'reload' or 'workers'.
try using the same way to run a basic script i.e file:variable
ERROR: Error loading ASGI app. Import string "app" must be in
format ":".
uvicorn.run("backend.main:app", host="0.0.0.0", port=8000, reload=True, workers=2)
I am running unit test to test an API that works fine when tested with Postman. The API takes in two parameters in the form {"body":"hey","title":"title"} adds these values to the database based on the models I have made. A response is returned in similar format with an extra key of id which is obtained from the database. The thing is that it works fine with Postman. However, when tested using the Pytest, just does not work.
Here is the code in the test file.
import os
import unittest
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
# from flask_backend import app
# from flask_backend.core import db
class BasicTests(unittest.TestCase):
app = Flask(__name__)
db = SQLAlchemy(app)
def setUp(self):
file_path = os.path.abspath(os.getcwd()) + "\database_test.db"
self.app.config['TESTING'] = True
self.app.config['DEBUG'] = False
self.app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + file_path
self.app = self.app.test_client()
self.db.drop_all()
self.db.create_all()
def tearDown(self):
self.db.session.remove()
class TestApi(BasicTests):
def test_add_post(self):
url = 'http://127.0.0.1:5000'
parameters = {'body': 'Body', 'title': 'title'}
response = self.app.post(url+'/api/dbapi/post/', data=parameters)
print(self.app)
self.assertEqual(response.status_code, 200)
if __name__ == '__main__':
unittest.main()
I am thinking that while executing the test a server is not started and hence the 404 error is raised.
The reason I am not importing the app variable from the project itself is because the module is not getting imported. I have asked the question in a different thread. Here is the link to it:
Can not import the files from parent directory even though it has __init__.py file in it
From what I understand, if I can import the app instance that is used in the project itself, I should be good but that isn't working either.
Here is how I solved the problem. So, apparently it was required to get the server running for the API to be accessible and the test to run successfully.
So, I added app.run() at the end of the code and it worked fine. Here is what it looks like
if __name__ == '__main__':
unittest.main()
app.run()
I'm building CRUD REST APIs using peewee ORM and sanic(sanic-crud) as app server. Things are working fine. And I wrote couple of unittest cases for the same.
But, I'm facing problem running unittests. The problem is that unittests starts sanic app server and stalled there. Its not running unittest cases at all. But when I press Ctrl+C manually then the sanic server gets terminated and unittests execution starts. So, it means there should be a way to start sanic server and continue unittests run and terminate server at the end.
Can someone please me the correct way writting unittest cases for sanic app?
I followed official docs too but no luck.
http://sanic.readthedocs.io/en/latest/sanic/testing.html
I tried following
from restapi import app # the execution stalled here i guess
import unittest
import asyncio
import aiohttp
class AutoRestTests(unittest.TestCase):
''' Unit testcases for REST APIs '''
def setUp(self):
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(None)
def test_get_metrics_all(self):
#asyncio.coroutine
def get_all():
res = app.test_client.get('/metrics')
assert res.status == 201
self.loop.run_until_complete(get_all())
from restapi.py
app = Sanic(__name__)
generate_crud(app, [Metrics, ...])
app.run(host='0.0.0.0', port=1337, workers=4, debug=True)
Finally managed to run unittests by moving app.run statement to main block
# tiny app server starts here
app = Sanic(__name__)
generate_crud(app, [Metrics, ...])
if __name__ == '__main__':
app.run(host='0.0.0.0', port=1337, debug=True)
# workers=4, log_config=LOGGING)
and
from restapi import app
import json
import unittest
class AutoRestTests(unittest.TestCase):
''' Unit testcases for REST APIs '''
def test_get_metrics_all(self):
request, response = app.test_client.get('/metrics')
self.assertEqual(response.status, 200)
data = json.loads(response.text)
self.assertEqual(data['metric_name'], 'vCPU')
if __name__ == '__main__':
unittest.main()