Develop Reference

Why is Django test cases checking actual DB and raising IntegrityError instead of just running in-memory?

When I run my tests with the DB empty (the actual application DB), everything goes fine. But when the DB has data, Django raises an IntegrityError for basically every test. The stact trace looks like the following (but for every test):
======================================================================
ERROR: test_api_get_detail (core.projects.tests.test_project_api.ProjectConfidentialPrivateAPITests)
Getting confidential object via API (GET)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/ramonkcom/Desktop/management/management-backend/venv/lib/python3.10/site-packages/django/test/testcases.py", line 299, in _setup_and_call
self._post_teardown()
File "/Users/ramonkcom/Desktop/management/management-backend/venv/lib/python3.10/site-packages/django/test/testcases.py", line 1199, in _post_teardown
self._fixture_teardown()
File "/Users/ramonkcom/Desktop/management/management-backend/venv/lib/python3.10/site-packages/django/test/testcases.py", line 1461, in _fixture_teardown
connections[db_name].check_constraints()
File "/Users/ramonkcom/Desktop/management/management-backend/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 383, in check_constraints
raise IntegrityError(
django.db.utils.IntegrityError: The row in table 'custom_fields_customfield' with primary key '1' has an invalid foreign key: custom_fields_customfield.definition_id contains a value '1' that does not have a corresponding value in custom_fields_customfielddefinition.id.
At first I thought this was a problem with my fixtures. But they work just fine to set up the application basic data. The DB seems to be the key issue: if it has data, the tests crash, if it's empty, the tests work.
The most weird part is that the error being raised doesn't match with reality, and by that I mean: the CustomFieldDefinition object with pk=1 exists in the DB. Anyway, it shouldn't matter, since I expect Django to build an in-memory DB for every test. It's just an additional piece of info for the mistery.
What could be making Django check for the actual DB instead of doing all its thing in-memory?
Additional pieces of info:
I'll work to provide some code example (didn't provide it right from the start because there's a big test framework behind it, and I'll have to trim it). But for now, I'be adding infos and things that I get asked and things I suspect might be causing the issue:
I'm on Python 3.10.8 and Django 4.0.5
Most of the test cases are inheriting from django.test.TestCase, but some are inheriting from django.test.TransactionTestCase.
I'm using TransactionTestCase in some cases because I need to create dummy models (and maybe this implementation is part of the problem somehow). Example:
# THE DUMMY CLASS
#localized
class DummyClass(models.Model):
"""Represents a dummy model for testing purpose"""
class Meta:
app_label = 'apps.localizer'
name = models.CharField(max_length=255, blank=True, null=True)
# THE TEST CASE SETUP/TEARDOWN
class LocalizedModelsTests(TransactionTestCase):
def setUp(self):
with connection.schema_editor() as schema_editor:
schema_editor.create_model(DummyClass)
def tearDown(self):
with connection.schema_editor() as schema_editor:
schema_editor.delete_model(DummyClass)
The database settings on settings.py (still on dev, so there's no other setting that could replace this one):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}

Maybe you can use TestCase so the DB is created and use the setUp and tearDown to create the Dummy model like this:
class LocalizedModelsTests(TestCase):
def setUp(self):
# Create the dummy model
with connection.schema_editor() as schema_editor:
schema_editor.create_model(DummyClass)
# Create any objects you need for your tests here
DummyClass.objects.create(name='Test')
def test_dummy_class(self):
# Test the DummyClass model here
def tearDown(self):
DummyClass.objects.all().delete()
# Delete the dummy model
with connection.schema_editor() as schema_editor:
schema_editor.delete_model(DummyClass)
Let me know if this help you.

Django's StaticLiveServerTestCase test pass alone but fail when grouped

I'm using Selenium along with StaticLiveServerTestCase to test a Django app.
The test is as follow:
class e2eTest(StaticLiveServerTestCase):
#classmethod
def setUpClass(cls):
super(IntegrationTest, cls).setUpClass()
cls.data = load_data()
cls.driver = webdriver.PhantomJS("path_to_phantomjs")
cls.common = common(cls.driver, cls.live_server_url + settings.STATIC_URL + 'index.html')
def setUp(self):
for data in self.data:
data.refresh_from_db()
def test_login_1(self):
self.common.login('admin')
def test_login_2(self):
self.common.login('admin')
load_data() is used to populate test database (using apps models).
If I run:
1- python manage.py test login.tests: the tests start with test_login_1. test_login_1 succeed and test_login_2 fail.
2- python manage.py test login.tests --reverse: the tests start with test_login_2. test_login_2 succeed and test_login_1 fail.
I believe it have something to do with data begin erased after each test.
P.S. When using same approach using django.test.TestCase & setUpTestData and Django's test client it works ( sending the data to the login api directly ).
Can I have something like setUpTestData from django.test.TestCase ?
EDIT 1
#luke_aus: load_data() contain methods to populate the test database:
from my_app import User, priceModel
user = User(username='test').set_password('test')
user.save()
priceModel = priceModel(name='test')
priceModel.save()
....
#knbk: I'm using Django 1.9. The test still fail after passing --parallel=1.
Thanks you both for your feedback!

The problem was caused by load_data(). The database will get populated in the first call to load_data() from setUpClass, and the first test will passes. For the second test, the database data will be cleared and not refreshed.
Changing load_data() method to load fixtures solved the problem.

How to avoid duplicating test cases in Django?

I have written my test cases in two separate test files (e.g. test_1 and test_2). In both of test cases that I am testing my models I have code duplications because of similar processes.
For example, I need to login the user and test the credential.
Sample of code:
import test_data
from django.test import TestCase
from UserData.models import MyModel
from django.contrib.auth.models import User
class UserDataMyModelTestCalls(TestCase):
#classmethod
def setUpTestData(cls):
cls.test_user = User.objects.create_user(test_data.test_user_data['user_name'],
test_data.test_user_data['email'],
test_data.test_user_data['password'])
def test_faulty_login_credentials(self):
self.client.login(username=test_data.faulty_user_data['user_name'], password=test_data.faulty_user_data['password'])
response = self.client.get('/userdata/mymodelurl/', {})
self.assertEqual(response.status_code, 403)
I am using a separate file with user credentials to avoid duplications again.
Sample of test_data file:
test_user_data = {'id': u'1',
'user_name': 'tempUsername',
'password': 'tempPassword',
'email': 'tempEmaily#test.com'}
Update: Adding the UserTests class that I want to use as a common class for all my test cases. I am defining and calling the test through the test_1.py like this:
import UserTests
from django.test import TestCase
class UserDataWayPointTestCalls(TestCase):
testCasesObject = UserTests.UserDataTestCalls()
test_user = testCasesObject.setUpTestData()
response = testCasesObject.test_faulty_login_credentials()
My UserDataTestCalls class is defined like this:
import test_data
from django.test import Client
from django.test import TestCase
from django.contrib.auth.models import User
class UserDataTestCalls(TestCase):
def __init__(self):
self.test_user = None
self.faulty_login_response = None
def setUpTestData(self):
self.client = User.objects.create_user(test_data.test_user_data['user_name'],
test_data.test_user_data['email'],
test_data.test_user_data['password'])
self.client = Client()
return self.client
def test_faulty_login_credentials(self):
self.client.login(username=test_data.faulty_user_data['user_name'],
password=test_data.faulty_user_data['password'])
response = self.client.get('/userdata/mymodelurl/', {})
return response
When I execute the code above I get IntegrityError: (1062, "Duplicate entry 'tempUsername' for key 'username'"). Temporarily I modify the username value to proceed and I get the following error AttributeError: 'UserDataTestCalls' object has no attribute '_testMethodName'.
I tried to create a separate class with name e.g. UserDataTestCalls and include the common parts of my test cases such as User.objects.create_user, self.client.login etc...
Unfortunately I end up getting errors that the database although it said Destroying test database for alias 'default'... on the next run I got username duplications e.g. Duplicate entry 'tempUsername' for key 'username' etc...
When I tried to overcome this problem by changing the username for testing purposes then I got another problem 'NoneType' object has no attribute 'login'.
Which it points that the self.client variable is not binded with the test_user that I am creating.
I tried to search online and find documentation on how to overcome my problem but all the documentation are pointing to use separate scripts for your tests individually, which I can understand if you have different test cases. In my case 90% of my test cases are exactly the same.
So I am sure there is a way to create a user in a separate class and create all my test cases in that class too, so I could call them from a separate test file(s) when I need them.
Can someone point me to the correct direction or provide some links with examples/documentation that I could read from?
Thank you in advance for your time and effort.

Try creating a common test class.
class CreateUserTestCase(TestCase):
def setUpTestData(self):
self.user = User.objects.create_user(
test_data.test_user_data['user_name'],
test_data.test_user_data['email'],
test_data.test_user_data['password'],
)
You want to assign the new user to self.user. Don't replace self.client which should be the test client, not the user. You don't need to do self.client = Client(), the Django test case will take care of this for you.
Then subclass the test case and add your tests.
class UserDataTestCalls(CreateUserTestCase):
def test_faulty_login_credentials(self):
self.client.login(
username=test_data.faulty_user_data['user_name'],
password=test_data.faulty_user_data['password'],
)
response = self.client.get('/userdata/mymodelurl/', {})
return response
From your question, I wasn't sure if test_data is different for each class. If so, you'll have to change this slightly.

Django - Unit Tests on localhost creating objects in main database, but not finding them in the test database later in the method

Experiencing some fairly strange behaviour with django unit tests. I have a class inherited from django.test.testcase that verifies a registration form creates a person object and a user object, which it does, but I can subsequently look for and find the objects in the Admin interface after the tests have finished. This is the test:
import unittest
import time
from django.test import TestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from django.contrib.auth.models import User
from apps.persons.models import Person
class NewVisitorTest(TestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
def test_can_sign_up_and_login(self):
# User goes to register url
self.browser.get('http://localhost:8000/register/')
# User can see that she has come to the right page on the website
self.assertIn('Register', self.browser.title)
# User can see that there is a sign up form
signup_form = self.browser.find_element_by_id('id_registration_form')
# User can enter text into the first name field
first_name_input = self.browser.find_element_by_id('id_first_name')
first_name_input.send_keys('Jim')
# User can enter text into the last name field
last_name_input = self.browser.find_element_by_id('id_last_name')
last_name_input.send_keys('bob')
# User can enter text into the username field
username_input = self.browser.find_element_by_id('id_username')
username_input.send_keys('jim_bob')
# User can enter text into the email field
email_input = self.browser.find_element_by_id('id_email')
email_input.send_keys('jim_bob#jimbob.com')
# User can enter text into the first password field
password_input = self.browser.find_element_by_id('id_password')
password_input.send_keys('kittensarecute')
# User can enter text into the second password field
password_1_input = self.browser.find_element_by_id('id_password1')
password_1_input.send_keys('kittensarecute')
# Submit form
signup_form.submit()
time.sleep(20)
persons = Person.objects.all()
print persons
self.assertEqual(len(persons), 1)
users = User.objects.all()
print users
self.assertEqual(len(users), 1)
# Deliberately fail
self.fail('Finish the test!')
if __name__ == '__main__':
unittest.main()
The code fails on assertEqual(len(persons), 1). I can reproduce the same behaviour in the browser and have no issues. Each time I run the test I have to delete the user it creates in the main local database, so I'm assuming it's saving the objects in the database defined in settings['databases'], but then looking for them in the test database later in the method. I see in the terminal that the test database is created at the beginning of each test, so it's definitely there, which seems confusing. Any help would be massively appreciated.
EDIT: It's all on sqlite
File ".../tests/tests.py", line 60, in test_can_sign_up_and_login
self.assertEqual(len(persons), 1)
AssertionError: 0 != 1
Edit 2:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': '/Users/me/github/stage/project/data/db.db', # Or path to database file if using sqlite3.
# The following settings are not used with sqlite3:
'USER': '',
'PASSWORD': '',
'HOST': '', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
'PORT': '', # Set to empty string for default.
}
}
I have a folder with apps in them, so to run these tests and run:
python manage.py test tests

You're right, django creates a test database inside your tests. From the docs:
Tests that require a database (namely, model tests) will not use your
“real” (production) database. Separate, blank databases are created
for the tests.
full documentation
Which is the correct thing to do. Unit test should be isolated from one another in order to avoid a dirty environment (a previous test forgets to remove a record/file/object it created and it affects the result of the other tests).
What you are doing is called Functional Testing. You are testing that your web application form submits, validates, saves, etc all in one test. Since the act of doing this implicates that many pieces of code run, it's not considered unit testing. When you perform these types of tests, your assertions should also be similar:
assert thank you/welcome message is shown
go to the user management page and assert new person is there
check that they can log it
etc
This tests that your application functions correctly.
The difference between the two types of testing is that Functional Testing tests what your application is doing, whereas Unit Testing tests how it's doing it.
This is crucial because you could be saving your data in a number of ways:
As JSON into a file
As XML
In a database locally
In the cloud somewhere
Your functional test doesn't really care how this is done, it just cares that when you go to a specific page, it gets the data. But unit test will drastically change depending on which way you're using.
In your case, you are using both ways of testing. Selenium (opening a page, filling it out, submitting it) is testing what the application does (functional), but your assertions (was the data was saved to the database) is testing how you store the data (unit).
So I suggest that you change your assertions to fit the rest of your test. And move the current ones into your unit tests:
def test_user_is_save(self):
addUser(...)
users = User.objects.all()
self.assertEqual(len(users), 1)
This will test that your addUser method is working correctly (saving to the db), which is what unit test is about. And your current test will test that when you submit the form, the user sees the right thing.

You can use LiveServerTestCase instead and access to your service with
self.live_server_url.
You will be able to get objects from the test database models normally with Entry.objects.get
import requests
from django.test import LiveServerTestCase
class FunctionalTest(LiveServerTestCase):
def test_get_comments(self):
response = requests.get(f"{self.live_server_url}/api/comments/?page=1")
self.assertEqual(response.status_code, 200)

You should use LiveServerTestCase, so that django can run it's own test server and therefore use it's own test db.
https://docs.djangoproject.com/en/1.4/topics/testing/#live-test-server

Custom sqlite database for unit tests for code using peewee ORM

I am trying to implement a many-to-many scenario using peewee python ORM and I'd like some unit tests. Peewee tutorial is great but it assumes that database is defined at module level then all models are using it. My situation is different: I don't have a source code file (a module from python's point of view) with tests which I run explicitly, I am using nose which collects tests from that file and runs them.
How do I use a custom database only for models instantiated in tests (which are being run by nose)? My goal is to use an in-memory database for tests only, to speedup the testing process.

I just pushed a commit today that makes this easier.
The fix is in the form of a context manager which allows you to override the database of a model:
from unittest import TestCase
from playhouse.test_utils import test_database
from peewee import *
from my_app.models import User, Tweet
test_db = SqliteDatabase(':memory:')
class TestUsersTweets(TestCase):
def create_test_data(self):
# ... create a bunch of users and tweets
for i in range(10):
User.create(username='user-%d' % i)
def test_timeline(self):
with test_database(test_db, (User, Tweet)):
# This data will be created in `test_db`
self.create_test_data()
# Perform assertions on test data inside ctx manager.
self.assertEqual(Tweet.timeline('user-0') [...])
# once we exit the context manager, we're back to using the normal database
See the documentation and have a look at the example testcases:
Context manager
Testcases showing how to use

To not include context manager in every test case, overwrite run method.
# imports and db declaration
class TestUsersTweets(TestCase):
def run(self, result=None):
with test_database(test_db, (User, Tweet)):
super(TestUsersTweets, self).run(result)
def test_timeline(self):
self.create_test_data()
self.assertEqual(Tweet.timeline('user-0') [...])

I took the great answers from #coleifer and #avalanchy and took them one step further.
In order to avoid overriding the run method on every TestCase subclass, you can use a base class... and I also like the idea of not having to write down every model class I work with, so I came up with this
import unittest
import inspect
import sys
import peewee
from abc import ABCMeta
from playhouse.test_utils import test_database
from business_logic.models import *
test_db = peewee.SqliteDatabase(':memory:')
class TestCaseWithPeewee(unittest.TestCase):
"""
This abstract class is used to "inject" the test database so that the tests don't use the real sqlite db
"""
__metaclass__ = ABCMeta
def run(self, result=None):
model_classes = [m[1] for m in inspect.getmembers(sys.modules['business_logic.models'], inspect.isclass) if
issubclass(m[1], peewee.Model) and m[1] != peewee.Model]
with test_database(test_db, model_classes):
super(TestCaseWithPeewee, self).run(result)
so, now I can just inherit from TestCaseWithPeewee and don't have to worry about anything else other than the test

Apparently, there's a new approach for the scenario described, where you can bind the models in the setUp() method of your test case:
Example from the official docs:
# tests.py
import unittest
from my_app.models import EventLog, Relationship, Tweet, User
MODELS = [User, Tweet, EventLog, Relationship]
# use an in-memory SQLite for tests.
test_db = SqliteDatabase(':memory:')
class BaseTestCase(unittest.TestCase):
def setUp(self):
# Bind model classes to test db. Since we have a complete list of
# all models, we do not need to recursively bind dependencies.
test_db.bind(MODELS, bind_refs=False, bind_backrefs=False)
test_db.connect()
test_db.create_tables(MODELS)
def tearDown(self):
# Not strictly necessary since SQLite in-memory databases only live
# for the duration of the connection, and in the next step we close
# the connection...but a good practice all the same.
test_db.drop_tables(MODELS)
# Close connection to db.
test_db.close()
# If we wanted, we could re-bind the models to their original
# database here. But for tests this is probably not necessary.

When using test_database I encountered problems with test_db not being initialized:
nose.proxy.Exception: Error, database not properly initialized before opening connection
-------------------- >> begin captured logging << --------------------
peewee: DEBUG: ('SELECT "t1"."id", "t1"."name", "t1"."count" FROM "counter" AS t1', [])
--------------------- >> end captured logging << ---------------------
I eventually fixed this by passing create_tables=True like so:
def test_timeline(self):
with test_database(test_db, (User, Tweet), create_tables=True):
# This data will be created in `test_db`
self.create_test_data()
According to the docs create_tables should default to True but it seems that isn't the case in the latest release of peewee.

For anyone who's using pytest, here's how I did it:
conftest.py
MODELS = [User, Tweet] # Also add get_through_model() for ManyToMany fields
test_db = SqliteDatabase(':memory:')
test_db.bind(MODELS, bind_refs=False, bind_backrefs=False)
test_db.connect()
test_db.create_tables(MODELS)
#pytest.fixture(autouse=True)
def in_mem_db(mocker):
mocked_db = mocker.patch("database.db", autospec=True) # "database.db" is where your app's code imports db from
mocked_db.return_value = test_db
return mocked_db
And voila, all your tests run with an in-memory sqlite database.