Am working on a django reusable package that am planning to use with multiple project. I've used pytest to build test suite, I've used parametrized helped in pytest to run a single test with multiple configuration.
Yet, I would like to run all my tests using different settings combinations
available_backends = [
'django_profile.auth_backends.drf.RestFramework',
'django_profile.auth_backends.kong.Kong',
]
def pytest_generate_tests(metafunc):
# if 'stringinput' in metafunc.fixturenames:
if 'auth_backend' in metafunc.fixturenames:
metafunc.parametrize(
'auth_backend',
available_backends
)
#pytest.fixture(params=['auth_backend', ])
def auth_backend(request, settings):
settings.DJANGO_PROFILE_AUTH_BACKEND = request.auth_backend
return settings
I experimented with the above approach, but this also means I have to add auth_backend to each test case, I don't believe this is ideal. Any one can recommend a way for me to run all my tests using different setting combinations?
Regards
please did you try:
to use code in conftest.py with scope="session"
use available_backends list directly in params= instead of pytest_generate_tests
https://docs.pytest.org/en/latest/fixture.html#parametrizing-fixtures
if not, could you try using this:
#pytest.fixture(scope='session', params=[
'django_profile.auth_backends.drf.RestFramework',
'django_profile.auth_backends.kong.Kong',
])
def auth_backend(request, settings):
settings.DJANGO_PROFILE_AUTH_BACKEND = request.param
yield settings
you can override settings for each test.
django documentation:
Overriding settings
for example:
from django.test import TestCase
class LoginTestCase(TestCase):
def test_login(self):
# First check for the default behavior
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/accounts/login/?next=/sekrit/')
# Then override the LOGIN_URL setting
with self.settings(LOGIN_URL='/other/login/'):
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/other/login/?next=/sekrit/')
or use for decorator
from django.test import TestCase, override_settings
class LoginTestCase(TestCase):
#override_settings(LOGIN_URL='/other/login/')
def test_login(self):
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/other/login/?next=/sekrit/')
Eventually, I found a way to pass different settings to all tests without having to specify the fixture in each test. Below is a working example
# conftest.py
#pytest.yield_fixture(params=available_backends)
def settings(request, settings):
if request.param not in available_backends:
raise ValueError('auth_backend {} not implemented.'.format(request.param))
wrapper = SettingsWrapper()
settings.DJANGO_PROFILE_AUTH_BACKEND = request.param
yield wrapper
wrapper.finalize()
# setup.cfg
usefixtures = settings
Related
What I am trying to achieve.
I am testing rest API in Django project. I want to create test class with test functions that are related (each following test function relies on previous one) - first fail means all failed. In first test function I crete an object with 'post' request. In next test case I want to check that this object actually exists using 'get' request.
How it works
It looks like Django-pytest cleans the database from all records after each test.
It is mentioned in pytest documention: https://pytest-django.readthedocs.io/en/latest/helpers.html#pytest-mark-django-db-request-database-access
Is there any way to change this?
My code:
My conftest.py:
import pytest
#pytest.fixture(scope='session')
def django_db_setup():
from django.conf import settings
settings.DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'db_name.sqlite3',
}
#pytest.fixture
def object_data():
return {"some keys": "some values"}
#pytest.fixture
def object_name():
return "some name"
My tests.py:
import pytest
from rest_framework.test import APIClient
from rest_framework import status
#pytest.mark.django_db
class TestAPI:
def setup(self):
self.client = APIClient()
def test_create_object(self, object_data, object_name):
post_response = self.client.post("/api/object/", data=object_data, format="json")
assert status.is_success(post_response.status_code)
# make sure that report was created
get_response = self.client.get(f"/api/object/{object_name}/")
assert status.is_success(get_response.status_code)
# object exists here (test passes)
def test_get_object(self, object_name):
get_response = self.client.get(f"/api/object/{object_name}/")
assert status.is_success(get_response.status_code)
# object does not exists here (test failes)
You probably don't want to persist data between test cases. You should strive to have test cases that can run in an arbitrary order. If you want data to be shared between test cases, either initialize it in a fixture and reuse that across test cases or combine your test cases into one to ensure everything in that one test case runs sequentially.
Yes, pytest-django does clear your DB after each test run.
In order to solve this, add a pytest configuration file named pytest.ini on your root directory and add as following
[pytest]
addopts = --reuse-db
As this config file suggests, it reuses the current DB without creating a new DB.
So if you need to create a DB for your test, you will have to specify it on command by pytest --create-db
Your test case structure seems wrong. Test cases should never depend on each other. There can be resources such as Model objects common among test cases (for eg. by initialising them in setUpClass) but a resource created in one test case should not be usable in another test case
Instead of the way you've set your test cases, you could do it as:
To test your POST endpoint:
1.1. Make the POST request, as you have in test_create_object and assert the response info
1.2. Assert that the object actually exists, preferably by checking if it exists in the DB else by making a GET request
To test your GET endpoint:
2.1. Create the object preferably directly in the DB or by making a POST request
2.2. Make a GET request, as you have in test_get_object and assert the response info
I am using pytest in my django project. I used pytest.config previously to get conftest.py values into the setUp function but it is deprecated since version 4.1. In the docs they suggest to use request fixture to get configs but I couldn't find a proper way get configs inside to the TestCase class using request fixture. Following code sample is the working code which uses the pytest.config. Any way to get configs without getting PytestDeprecationWarning will be helpful.
import pytest
from django.conf import settings
from django.test import TestCase
from django.test.utils import override_settings
#override_settings(USE_SUBDOMAIN=False, PRODUCTION_DOMAIN='example.com')
class TestSample(TestCase):
fixtures = ['test_data']
def setUp(self):
url_base = '{scheme}://{domain}/sample'.format(
scheme=pytest.config.option.url_scheme,
domain=settings.PRODUCTION_DOMAIN,
)
I'm using Selenium along with StaticLiveServerTestCase to test a Django app.
The test is as follow:
class e2eTest(StaticLiveServerTestCase):
#classmethod
def setUpClass(cls):
super(IntegrationTest, cls).setUpClass()
cls.data = load_data()
cls.driver = webdriver.PhantomJS("path_to_phantomjs")
cls.common = common(cls.driver, cls.live_server_url + settings.STATIC_URL + 'index.html')
def setUp(self):
for data in self.data:
data.refresh_from_db()
def test_login_1(self):
self.common.login('admin')
def test_login_2(self):
self.common.login('admin')
load_data() is used to populate test database (using apps models).
If I run:
1- python manage.py test login.tests: the tests start with test_login_1. test_login_1 succeed and test_login_2 fail.
2- python manage.py test login.tests --reverse: the tests start with test_login_2. test_login_2 succeed and test_login_1 fail.
I believe it have something to do with data begin erased after each test.
P.S. When using same approach using django.test.TestCase & setUpTestData and Django's test client it works ( sending the data to the login api directly ).
Can I have something like setUpTestData from django.test.TestCase ?
EDIT 1
#luke_aus: load_data() contain methods to populate the test database:
from my_app import User, priceModel
user = User(username='test').set_password('test')
user.save()
priceModel = priceModel(name='test')
priceModel.save()
....
#knbk: I'm using Django 1.9. The test still fail after passing --parallel=1.
Thanks you both for your feedback!
The problem was caused by load_data(). The database will get populated in the first call to load_data() from setUpClass, and the first test will passes. For the second test, the database data will be cleared and not refreshed.
Changing load_data() method to load fixtures solved the problem.
Lets say module a code:
from django.conf import settings
print settings.BASE_URL # prints http://example.com
In tests.py I want to mock the BASE_URL to http://localhost
I have tried the following:
with mock.patch('django.conf.settings.BASE_URL', 'http://localhost'):
pass
with mock.patch('a.settings.BASE_URL', 'http://localhost'):
pass
from a import settings
with mock.patch.object(settings, 'BASE_URL', 'http://localhost'):
pass
import a
with mock.patch.object(a.settings, 'BASE_URL', 'http://localhost'):
pass
None of the above worked.
Try to use context manager settings() built-in django.
with self.settings(BASE_URL='http://localhost'):
# perform your test
https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.SimpleTestCase.settings
You can also use the following decorator on your individual test functions or test class as a whole. For example:
from django.test import override_settings
#override_settings(BASE_URL='http://localhost')
def test_case()
...
Django allows overwriting settings, when running tests through SimpleTestCase.settings() (https://docs.djangoproject.com/en/1.8/topics/testing/tools/#django.test.SimpleTestCase.settings). That works fine when I try to overwrite one of Django's settings.
The app I want to run tests for carries its own settings in an app-specific settings.py with the following structure to allow overwriting the app-specific settings in the project wide settings.py:
from django.conf import settings
APP_SETTING1 = getattr(settings, 'APP_SETTING1', 'foo')
The following dummy code in a test shows the problem:
from django.test import TestCase
from django.conf import settings as django_settings
from my_app import settings
class MyTestCase(TestCase):
def test_something(self):
with self.settings(APP_SETTING1='bar'):
print(django_settings.APP_SETTING1) # bar
print(settings.APP_SETTING1) # foo
Why isn't that working and how can I make it work?
you can use django-zero-settings, it lets you define your default settings, and will update them with user settings provided, it also works fine in tests settings overrides, alongside that it auto import string, lets you define removed settings and pre-check user settings too.
as in your case, you can define app settings like this:
from zero_settings import ZeroSettings
app_settings = ZeroSettings(
key="APP",
defaults={
"SETTING1": "foo"
},
)
then in your tests you can use it like:
from django.test import TestCase
from django.conf import settings as django_settings
from my_app import app_settings
class MyTestCase(TestCase):
def test_something(self):
with self.settings(APP={"SETTING1": "bar"}):
print(django_settings.APP["SETTING1"]) # bar
print(app_settings.SETTING1) # bar
self.assertEqual(django_settings.APP["SETTING1"], app_settings.SETTING1)
because that ZeroSettings uses cache and will cache user settings on the first attempt to get key, you may need to set it off before getting keys, to make sure you get the latest updates:
app_settings = ZeroSettings(
key="APP",
defaults={
"SETTING1": "foo"
},
use_cache=False,
)
or clear the cache manually:
class MyTestCase(TestCase):
def test_something(self):
print(app_settings.SETTING1) # foo
with self.settings(APP={"SETTING1": "bar"}):
app_settings._clear_cache()
print(django_settings.APP["SETTING1"]) # bar
print(app_settings.SETTING1) # bar
self.assertEqual(django_settings.APP["SETTING1"], app_settings.SETTING1)
Ideally you should mock settings using unittest library incorporated into Django. Alternatively, you can do this, but it is better to patch those:
https://docs.djangoproject.com/en/2.1/topics/settings/#custom-default-settings
Actually, there are good practices for handling multiple settings files. The basic rules are:
1) Do not import from settings.py file directly:
# BAD - avoid this
from app.settings import SOME_ENV
Because if in the end, someone will try to use custom settings some parts of the old settings will be applied somewhere.
# GOOD
from django.conf import settings
print(settings.SOME_ENV)
In this case, you could be sure that actual settings were used.
2) Split up settings files for different environments, usually, it accomplished via creating several files:
settings/base.py
DEBUG = False
AWS_STORAGE_PREFIX = 'local'
...
settings/dev.py
from settings.base import *
DEBUG = True
AWS_STORAGE_PREFIX = 'dev'
DATABASE = {# your settings for dev env}
...
It is also quite usual to create a separated file for test runner because it gives you more power of controlling your test env and so on:
python manage.py test --settings=app.settings.testrunner
Test instance has bound settings' context manager, so you can override any variables that you need for testing:
class MyTestCase(TestCase):
def test_something(self):
# Old settings
with self.settings(SOME_ENV_VAR='overridden'):
# Overridden settings
pass
...
You can do it directly in apps using a certain setting
settings.py:
TEST_ENVIRONMENT_SETTING = True # Test setting for my app
apps.py:
from django.conf import settings
class MyAppConfig(AppConfig):
def ready(self):
if getattr(settings, 'TEST_ENVIRONMENT_SETTING', False):
# Do stuff or not do stuff
Or you can directly set this setting in your TestCase using
setattr(settings, 'TEST_ENVIRONMENT_SETTING', True)
I'm struggling with the same problem, and I think the only solution is to change approach and use only django.conf.settings directly.
The reason is that if you use a custom settings, your custom settings is taken too early from settings, before override_settings or TestCase.settings is called.