In my company we have a Django project that has API endpoints.
We are using Pytest as testing framework and some of our tests executing requests to such endpoints using requests.Session(). For example:
The test perform a GET request to /api/account/data
content of root/dir/tests/test_api.py
def test_some_api(self):
client = requests.Session()
response = client.get('/api/account/data')
assert response.status_code == 200
When performing the request, the backend execute this function:
`
root/models/account/api.py
def user_data(request, format=None):
"""
#api {get} /api/account/data
"""
if request.method == "GET":
return APIResponse(code=200)
We would like to measure the code executed in `user_settings` by the test, but I failed to make it work. We are using:
coverage==4.5.4 # forced to use an old version as we have dependencies conflict
pytest-cov==2.10.1
To measure coverage I run this command from root
pytest -v --cov-report html:html-report --cov=dir --cov-config=dir/.coveragerc
.coveragerc - has only files to exclude
I verified in backend logs that when the test runs, it execute the if block inside user_settings
I have tried adding this code to conftest.py as written in coverage docs, but it still didn't measure user_settings code execution.
#pytest.fixture(scope='session', autouse=True)
def run_cov():
process = coverage.process_startup()
source_path = os.path.join(_home_path, "/models/account")
cov = coverage.Coverage(config_file='root/.coveragerc', source=[source_path])
cov.start()
yield cov # Tests runs here
cov.stop()
cov.save()
cov.html_report(directory='html-report')
I've seen this solution How to generate coverage report for http based integration tests?, but failed to understand how to implement it on my end.
I expected user_settings code to be measured by coverage report.
Related
I made simple Django application which returns {'result': 'OK'} for endpoint '/api/v1/test/ping/'. Now I am trying to test it with pytest.
My test directory
/test
conftest.py
/test_app
test_api.py
My conftest.py
import pytest
from rest_framework.test import APIClient
#pytest.fixture
def api_client():
return APIClient
My test_api.py
import pytest
def test_api1(api_client):
response = api_client().get("/api/v1/test/ping")
assert response.status_code == 200
Test script execution fails:
test_api.py::test_api1 FAILED [100%]
test\test_gml_api\test_api.py:3 (test_api1)
301 != 200
But code works correct if I run server and check it manually! Give me please an advise how to solve this?
Before asking you, I tried to search as best as I could in SO for similar questions. I've read https://docs.pytest.org/en/latest/explanation/fixtures.html page twice. However, I could not solve the problem I am facing with. I am new to Python and Flask. So I thought that I am missing something obvious.
I have a small micky-mouse Flask application. I am trying to write tests for this. I voted for using Pytest because fixtures seemed more flexible than unittest. To my surprise, first test case runs with no problem at all. However the exact same test case fails to run for the second time.
Here is my fixtures.py file:
from config import Config
import pytest
from app import create_app, db
from app.models import User, Role
class TestConfig(Config):
SECRET_KEY = 'Shhhhh 007! Be quiet.!'
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
SQLALCHEMY_TRACK_MODIFICATIONS = False
EXPLAIN_TEMPLATE_LOADING = True
DEBUG = True
API_VERSION = 'temp'
#pytest.fixture
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
#pytest.fixture(autouse=True)
def role(client):
role = Role()
role.name = 'test role'
db.session.add(role)
db.session.commit()
yield role
db.session.delete(role)
db.session.commit()
#pytest.fixture(autouse=True)
def user(role):
user = User()
user.role_id = role.id
user.name = 'test name'
user._normalized_name = 'test name'
user.lastname = 'test lastname'
user._normalized_lastname = 'test lastname'
user.phonenumber = '+11231234567'
user.setPassword('testPassword')
db.session.add(user)
db.session.commit()
yield user
db.session.delete(user)
db.session.commit()
And here is my test_login.py file:
import logging
from tests.fixtures import client, role, user
def test_login_page_Once(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
I run my tests using:
pytest -v -k test_login
And here is the outcome:
tests/functional/test_login.py::test_login_page_Once PASSED [ 50%]
tests/functional/test_login.py::test_login_page_Twice FAILED [100%]
========================================== FAILURES ===========================================
____________________________________ test_login_page_Twice ____________________________________
client = <FlaskClient <Flask 'app'>>
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
> assert '<title>Signin' in decodedRv
E assert '<title>Signin' in '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server.
If you entered the URL manually please check your spelling and try again.</p>\n'
tests\functional\test_login.py:17: AssertionError
------------------------------------ Captured stdout call -------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
=================================== short test summary info ===================================
FAILED tests/functional/test_login.py::test_login_page_Twice - assert '<title>Signin' in '<!D...
========================== 1 failed, 1 passed, 2 deselected in 1.57s ==========================
What am I missing? Any help would be highly appreciated. Thanks very much.
Well, I found a solution to the problem for which I was banging my head against the wall for a while. Though I reached that solution by try and error, it is still beyond me why/how it works. If any experts of pytest/flask would give me an explanation, that would be lovely.
I would also appreciate if someone tells me what the differences are among fixture scopes. ( I have read https://docs.pytest.org/en/latest/reference/reference.html#pytest.fixture already but to me it is not clear.)
Thanks.
The only change I made in the code is that I added
scope='session'
to the client fixture.
#pytest.fixture(scope='session')
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
ADDITION #2:
Further down the road, another test of mine failed with similar symptoms. The difference was this test case living in another file (I encountered previous problems while I was testing login page/behavior. Now I am dealing with registration page/behavior). I could not find my way around this error for a while. (I even considered dropping of pytest and going back to unittest. But I am still here because I am a bit of a stubborn. That's a different story!)
Anyways, I realized that one way of grouping fixtures is to place them in a file named conftest.py. Mine was named fixtures.py. In my head, it was just a name and I thought mine was clearer. Well, I was wrong. conftest.py seems to be a magic file. Once I renamed the fixtures.py to conftest.py test all marched pass in shining green armors.
Adding my two cents here just in case anyone else feels lost in the depths of fixtures/routes/imports forests.
I had a similar issue and it turned out that I was accidentally mutating an object inside a function and that was having side effects outside the function.
First time I ran the fixture it worked fine.
Second time I ran the fixture, the function that defined the fixture was working with different data because of a side effect of the first time I ran the function. This gave an error trying to run the function.
Not sure if that is your problem, but this might help someone facing a similar issue.
This is really strange. I have the following simple flask application:
- root
- myapp
- a route with /subscription_endpoint
- tests
- test_az.py
- test_bz.py
test_az.py and test_bz.py look both the same. There is a setup (taken from https://diegoquintanav.github.io/flask-contexts.html) and then one simple test:
import pytest
from myapp import create_app
import json
#pytest.fixture(scope='module')
def app(request):
from myapp import create_app
return create_app('testing')
#pytest.fixture(autouse=True)
def app_context(app):
"""Creates a flask app context"""
with app.app_context():
yield app
#pytest.fixture
def client(app_context):
return app_context.test_client(use_cookies=True)
def test_it(client):
sample_payload = {"test": "test"}
response = client.post("/subscription_endpoint", json=sample_payload)
assert response.status_code == 500
running pytest, will run both files, but test_az.py will succeed, while test_bz.py will fail. The http request will return a 404 error, meaning test_bz cannot find the route in the app.
If I run them individually, then they booth succeed. This is very strange! It seems like the first test is somehow influencing the second test.
I have added actually a third test test_cz.py, which will fail as well. So only the first one will ever run. I feel like this has something todo with those fixtures, but no idea where to look.
Create a conftest.py for fixtures e.g. for client fixture and use the same fixture in both tests?
Now if you're saying that the provided code is the example of a test that is the same in another file, then you are creating 2 fixtures for a client. I would first clean it up and create a 1 conftest.py that contains all the fixtures and then use them in your tests this might help you.
Check out also how to use pytest as described in Flask documentation
I am new to the CI/CD pipelines and trying to implement the end to end automation of Azure Functions in Python . I couldn't find a way to include the unit test code to run in the Yaml file of CI part.
I am using the yaml file provided in the
https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-azure-devops#python
I have written following file to run the test.
import azure.functions as func
from . import main
class TestFunction(unittest.TestCase):
def test_my_function(self):
# Construct a mock HTTP request.
req = func.HttpRequest(
method='GET',
body=None,
url='/api/http_trigger',
params={'inc': 'hello'})
# Call the function.
resp = main(req)
# Check the output.
self.assertEqual(
resp.get_body(),
b"{'inc': 'hello'}" ,
)
i am using Flask to build a web service and pytest for testing
i am using pytest fixtures to set up and tear down the test resources but i need to test a POST endpoint that will create some records in the database
How do we clean up these records ?
You can use a fixture to do that cleanup.
#pytest.fixture
def cleanup():
yield
# This is executed when the test using the fixture is done
db_cleanup()
def test_records_created(cleanup): # pylint: disable=redefined-outer-name,unused-argument
response = app.test_client().post('/path', json=payload)
assert response.status_code == 200
assert ...