Question:
How can I add multiple databases for testing in Django?
When I ran my test suits I got this Error:
AssertionError: Database queries to 'mig' are not allowed in this test. Add 'mig' to path_to_test_suit.MigrationServiceTest.databases to ensure proper test isolation and silence this failure.
Here are my PostgreSQL settings:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db_1_name',
'USER': 'db_1_user',
'PASSWORD': 'db_1_passwd',
'HOST': 'db_1_host',
'PORT': 'db_1_port',
},
'mig': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': db_2_name,
'USER': db_2_user,
'PASSWORD': 'db_2_passwd',
'HOST': 'db_2_host',
'PORT': 'db_2_port',
},
}
I use Django-nose for running test suits and use Django 2.2
You could use the database parameter in your tests.
For example:
class TestYourClass(TestCase):
databases = {'default', 'mig'}
def test_some_method(self):
call_some_method()
For more information please see Multi-database support.
P.S. In earliest Django version than 2.2 please use multi_db = True
class TestYourClass(TestCase):
multi_db = True
You need to grant DJANGO user the priviledge to write.
Then, to make the tests and preserve your data, you have to make a copy of your schema with test_ as prefix and use the keywork --keepdb.
Related
I want to keep my unit test database completely separate from other environments including using different user credentials. This is mostly to prevent anyone from unintentionally running unit tests against the development database and mangling the dev data or wiping it out entirely if the --keepdb option isn't specified. The code below detects the "test" in the sys args and this seems to work but is very clunky. If I'm missing a better way to do this please advise.
I have separate settings files for each environment so this will only be on the development server where the unit tests are run automatically and won't end up on any production servers.
Environment:
Django 1.11
Python 3.4.x
MariaDB
# this works but is clunky
import sys
if 'test' in sys.argv:
DATABASES = { # test db and user
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db_test',
'USER': 'test_user',
'PASSWORD': 'secretpassword',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': { # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # redundant but explicit!
},
}
}
I'd like to do this but unfortunately Django doesn't look at the TEST credentials
# cleaner approach but doesn't work - don't do this!
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # Django uses the test db NAME
'USER':'test_user_ignored', # but ignores the USER and PASSWORD
'PASSWORD':'ignoredpassword',
},
}
}
Would something like this work for your situation?
import sys
if 'test' in sys.argv:
NAME = 'dev_db_test'
USER = 'test_user'
PASSWORD ='secretpassword'
else:
NAME = 'dev_db'
USER = 'dev_db_user'
PASSWORD ='dev_password'
DATABASES ={ # test db and user
'default':
{
'ENGINE': 'django.db.backends.mysql',
'NAME': NAME,
'USER': USER,
'PASSWORD': PASSWORD,
'HOST': 'the-db-host',
'PORT': '3306',
'TEST':
{ # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
print(DATABASES)
AFAIK, you don't need to create a separate test database if you want to run unit tests over it. Here is the documentation link of test database.
It states that:
Tests that require a database (namely, model tests) will not use your “real” (production) database.
Separate, blank databases are created for the tests.
You can follow this link to write your tests and it will not affect your production or development database.
I'm using a database URL string in my settings like:
DATABASES = {
'default': "mysql://root:#localhost:3306/mydb"
}
When I migrate I get this warning:
MySQL Strict Mode is not set for database connection 'default'
Now my question: How can I combine the two things?
I cannot use the "regular" way to set the database settings with a dictionary because my database url comes from an environment variable.
Thx in advance!
You could update your settings afterwards:
DATABASES['default']['OPTIONS'] = {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"}
I think this one will help you.
you can pass the options as a query string in the URL
DATABASES = {
'default': dj_database_url.config(default="mssql://USER:PASSWORD#HOST:PORT/NAME?init_command=SET sql_mode='STRICT_TRANS_TABLES'&charset=utf8mb4", conn_max_age=500)
}
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'HOST': '127.0.0.1',
'PORT': '3306',
'USER': 'root',
'PASSWORD': 'test123',
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
'charset': 'utf8mb4',
}
}
}
Try to give init_command of database options in settings.py
If want know more refer the docs django_mysql.W001: Strict Mode
I originally had a django project with a single app and all models were defined in that app. The project, when initiated only used the default database. It has now become an unwieldy app that I'm trying to break down into smaller apps. Doing so, I want to use different databases for the different apps. I've setup new databases and a router in the settings.py file. However, I'm confused about how to migrate existing tables to the new databases.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'db_name',
'USER': 'db_user_name',
'PASSWORD': 'password',
'HOST': 'hostname',
'PORT': '3306',
},
'db2': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'db_name2',
'USER': 'db_user_name2',
'PASSWORD': 'password2',
'HOST': 'hostname2',
'PORT': '3306',
}
}
I would want an app (e.g. app1) to be moved from default to db2. The router already knows to specify app1 to db2 but running migrations is doing nothing. Any ideas?
#knbk's answer was ultimately correct, except that the solution involved an additional step.
1. python manage.py migrate auth --database=db2
2. python manage.py migrate app1 --database=db2
In my project I am using multiple databases, 3 to be precise.
One default and two reference databases (another host), which are powered and updated by our data team. I am using 4 tables from 2 external databases, which were inspected with ./manage.py inspectdb. They are set to managed=False
Everything works fine, my application works as expected and planned.
Now the fun begins, I need to write tests. I do not want to delete my databases.
I need to read from them, I do not want to create fixtures.
Every time when I try to use pytest or custom django tests, it is trying to delete my databases, recreate, run migrations and stuff.
How can I preserve that ? Either in pytest or django tests ? I've tried to --re-usedb, and couple of another params.
This is my database settings (parts which can be uploaded)
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'default_db',
'USER': 'default_usr',
'PASSWORD': 'default_pass',
'HOST': 'postgres.somehost.com',
'SUPPORTS_TRANSACTIONS': True,
},
'geolocations': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'OPTIONS': {
'options': '-c search_path=some_schema'
},
'NAME': 'geo_name',
'USER': 'geo_user',
'PASSWORD': 'geo_password',
'HOST': 'some-host.domain.com',
'SUPPORTS_TRANSACTIONS': True,
},
'eoc': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'OPTIONS': {
'options': '-c search_path=another_schema'
},
'NAME': 'eco_name',
'USER': 'eoc_user',
'PASSWORD': 'eoc_',
'HOST': 'some-another-host.domain.com',
'SUPPORTS_TRANSACTIONS': True,
}
}
So inside of pytest.ini you should be able to add a '--no-migrations' flag in order to ensure that migrations are not being made. Using a pytest marker pytest documentation should enable you to have access to your database.
The mark you need is:
pytestmark = pytest.mark.django_db
I tried setting
DATABASES = {
'default': {
'NAME': 'db',
'ENGINE': 'mysql.connector.django',
'USER': 'dbuser',
'PASSWORD': 'dbpass',
'OPTIONS': {
'autocommit': True,
'init_command' : 'SET storage_engine=INNODB',
},
}
}
(UPD: updated the above code so ppl won't get confused that I am not using django settings the right way)
in Django settings, but this backend doesn't accept such connection option...
http://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html
is there any other way of doing this?
First, to use MySQL Connector/Python Django backend, you have to set the ENGINE setting to mysql.connector.django:
DATABASE = {
'default': {
..
'ENGINE': 'mysql.connector.django',
..
}
}
Indeed, MySQL Connector/Python does not have the init_command connection argument. I can see value in adding an option for setting the default storage engine in the Django OPTIONS though. If you really want it, I would suggest opening a feature request on http://bugs.mysql.com.
Small note that MySQL 5.6 (and if you start out, you should use this version) has storage engine set to InnoDB by default.
You need to set the entire setting:
DATABASES = {
'default': {
'NAME': "MyDatabaseName,
'ENGINE': 'django.db.backends.mysql',
'USER': "MyUsername",
'PASSWORD': "MyPassword",
'HOST': "MyHostName",
'OPTIONS': {'init_command': 'SET storage_engine=INNODB'},
}
}