Develop Reference

How to work with multiple databases in Django [using Database Routers]

I need to isolate django`s default (auth, contenttypes, sessions) apps database, I read in documentation [Multiple Databses][1] all is clear and well explained, but not working to me :(
settings.py
DATABASES = {
'default_django': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
},
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'OPTIONS': {
'options': '-c search_path=eos,public'
},
'NAME': '****',
'USER': '*****',
'PASSWORD': '****',
'HOST': 'localhost',
'PORT': '5432',
}
}
DATABASE_ROUTES = ['core.dbrouter.AuthRoute', 'core.dbrouter.GisRoute']
and ./core/dbrouter.py
class AuthRouter:
"""
A router to control all database operations on models in the
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
"""
route_app_labels = {
'auth', 'admin', 'contenttypes',
'sessions', 'messages', 'staticfiles'
}
def db_for_read(self, model, **hints):
"""
Attempts to read auth and contenttypes models go to auth_db.
"""
if model._meta.app_label in self.route_app_labels:
return 'default_django'
return None
def db_for_write(self, model, **hints):
"""
Attempts to write auth and contenttypes models go to auth_db.
"""
if model._meta.app_label in self.route_app_labels:
return 'default_django'
return None
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations if a model in the auth or contenttypes apps is
involved.
"""
if (
obj1._meta.app_label in self.route_app_labels or
obj2._meta.app_label in self.route_app_labels
):
return True
return None
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Make sure the auth and contenttypes apps only appear in the
'auth_db' database.
"""
if app_label in self.route_app_labels:
return db == 'default_django'
return None
When I do python3 manage.py migrate it is shows
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
No migrations to apply.
Please, could someone explain?(Why?)
[1]: https://docs.djangoproject.com/en/3.1/topics/db/multi-db/

you forgot an R.
DATABASE_ROUTES = ['core.dbrouter.AuthRoute', 'core.dbrouter.GisRoute']
↓
DATABASE_ROUTERS = ['core.dbrouter.AuthRoute', 'core.dbrouter.GisRoute']

How to migrate multiple databases in Django?

I am doing a project with Django, there are too much data so that I need to form different databases to store them. I made a database router for each app to match the unique database, but when I migrate them, all of the tables were migrate to the same database, here are the codes:
from django.conf import settings
DATABASE_MAPPING = settings.DATABASES_APPS_MAPPING
class DatebaseAppsRouter(object):
"""
该类为连接不同数据库的路由,在setting中通过设置DATABASE_MAPPING来匹配数据库
例:
DATABASE_APPS_MAAPING = {"app1":"db1", "app2":"db2"}
"""
def db_for_read(self, model, **kwargs):
"""将所有读操作指向指定的数据库"""
if model._meta.app_label in DATABASE_MAPPING:
return DATABASE_MAPPING[model._meta.app_label]
return None
def db_for_write(self, model, **kwargs):
"""将所有写操作指向指定的数据库"""
if model._meta.app_label in DATABASE_MAPPING:
return DATABASE_MAPPING[model._meta.app_label]
return None
def allow_relation(self, obj1, obj2, **kwargs):
"""允许使用相同数据库的应用程序之间的任何关系"""
db_obj1 = DATABASE_MAPPING.get(obj1._meta.app_label)
db_obj2 = DATABASE_MAPPING.get(obj2._meta.app_label)
if db_obj1 and db_obj2:
if db_obj1 == db_obj2:
return True
else:
return False
else:
return None
def allow_syncdb(self, db, model):
"""确保这些应用程序只出现在相关的数据库中"""
if db in DATABASE_MAPPING.values():
return DATABASE_MAPPING.get(model._meta.app_label) == db
elif model._meta.app_label in DATABASE_MAPPING:
return False
return None
def allow_migrate(self, db, app_label, model=None, **kwargs):
"""确保身份验证应用程序只出现在login数据库中"""
if db in DATABASE_MAPPING.values():
return DATABASE_MAPPING.get(app_label) == db
elif app_label in DATABASE_MAPPING:
return False
return None
By using the router, I would like to migrate multiple databases, but it doesn't work
System check identified some issues:
WARNINGS:
?: (urls.W001) Your URL pattern '^login/$' uses include with a route ending with a '$'. Remove the dollar from the route to avoid problems including URLs.
?: (urls.W001) Your URL pattern '^main/$' uses include with a route ending with a '$'. Remove the dollar from the route to avoid problems including URLs.
?: (urls.W005) URL namespace 'admin' isn't unique. You may not be able to reverse all URLs in this namespace
Operations to perform:
Apply all migrations: Themepark, admin, auth, contenttypes, ginseng_holding_hotel, holiday_resort, login, nature_scenic_area, sessions, the_scent_spot_trusteeship, tourist_communications, travel_agency
Running migrations:
No migrations to apply.
So how to solve this problem? Thanks!

Replace your app name with 'your_app' and the database you want to run the migration on with 'your_app_respective_db' from the mapping you have. Hope this helps.
python manage.py migrate your_app --database your_app_respective_db

I think the problem is with your urls.py
If you are using django.urls.path() you don't have to add the carrot sign and the dollar sign and do it this way.
urlpatterns = [
path('admin/', admin.site.urls),
path('login/', views.login, name='login'),
path('main/', views.main, name='main'),
]
But if you were using django.urls.url() then you should do this
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^login/', views.login, name='login'),
url(r'^main/', views.main, name='main'),
]
if you don't mind, can you show us your urls.py?

You have to add your different DB details as per your app in settings.py as following:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': config('DATABASE_NAME'),
'USER': config('DATABASE_USER'),
'PASSWORD': config('DATABASE_PASSWORD'),
'HOST': config('DATABASE_HOST'),
'PORT': config('DATABASE_PORT'),
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
'charset': 'utf8mb4',
},
},
'app1_name': {
'ENGINE': 'django.db.backends.mysql',
'NAME': config('APP1_DATABASE_NAME'),
'USER': config('DATABASE_USER'),
'PASSWORD': config('DATABASE_PASSWORD'),
'HOST': config('DATABASE_HOST'),
'PORT': config('DATABASE_PORT'),
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
'charset': 'utf8mb4',
},
},
'app2_name': {
'ENGINE': 'django.db.backends.mysql',
'NAME': config('APP2_DATABASE_NAME'),
'USER': config('DATABASE_USER'),
'PASSWORD': config('DATABASE_PASSWORD'),
'HOST': config('DATABASE_HOST'),
'PORT': config('DATABASE_PORT'),
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
'charset': 'utf8mb4',
},
}
}
default will be your main app. Also when you do migrations and migrate, you need to mention the database name like following.
python manage.py makemigrations app1
python manage.py migrate app1 --database=app1

How to stop Django to create lot of unnecessary tables

I have two databases defined in my setting.py file.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'monitoring',
'USER': 'root',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '',
},
'source' :{
'ENGINE': 'django.db.backends.mysql',
'NAME': 'source_db',
'USER': '*****',
'PASSWORD': '*****',
'HOST': '*****',
'PORT': '****',
}
}
I need to access some tables in source_db which django is not allowing to do so unless I migrate the db. So, once we run command python manage.py migrate --database=source , Django is creating some tables in server db. Since we are not allowed to create tables in server db, is there any way to stop django doing so, or any way to access tables without migrating the db?
This is the list of tables which we don't want to create.
+--------------------------------+
| Tables_in_source_db |
+--------------------------------+
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| dashboard_monitoring_features |
| dashboard_monitoring_modelinfo |
| dashboard_monitoring_product |
| django_admin_log |
| django_content_type |
| django_migrations |
| django_session |
+--------------------------------+

If you want to migrate / create tables in the default database and not source database, then you have to define app and database when you run migrations. Like:
python manage.py migrate dashboard --database=default
This will run migration in the dashboard app and create tables in the default database.
Next thing you want to do is set your source models to non-managed. You do by specifying managed = False in the meta class of the model:
class YourModel(models.Model):
... your fields here ...
class Meta:
managed = False
From the documentation:
If False, no database table creation or deletion operations will be
performed for this model. This is useful if the model represents an
existing table or a database view that has been created by some other
means.

If you don't want particular tables, then remove
'django.contrib.auth`,
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
and dashboard app
from your INSTALLED_APPS

Probably nobody was clear about what the author of the question was trying to do. So let me start with the context.
Context
By default, Django uses a database named default. Let's say I have a CustomModel in a CustomApp for which the table should be created in a new database called source_db.
Thus we have two databases: 1)default and 2)source_db.
We want the default tables like auth_group required by apps like django.contrib.auth to be created only in default. The source_db should have and only have tables from CustomApp. How do we do this?
Solution 1: I recommend this method
First, refer to Multiple Databases, the official documentation to understand the routing basics. Give special attention to allow_migrate(). Let me insist, understanding of this documentation is at most important to understand the rest of this solution.
Let's begin:
In your CustomModel, set managed = True in the meta data:
class CustomModel(models.Model):
# your fields here ...
class Meta:
managed = True
This specifies that the table creation and alteration for this model should be taken care of by Django.
Write your CustomDBRouter for tables in CustomApp. (Refer to Multiple Databases for this). Don't forget to define allow_migrate().
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Make sure the 'CustomApp' only appear in the 'source_db' database.
"""
if app_label == 'CustomApp':
# if app=CustomApp and db=source_db, migrate
return db == 'source_db'
return None # No suggestions, ask the next router in the list
Go to your settings.py and split the list INSTALLED_APPS as follows:
MY_APPS = [ # List of apps defined by you
'customApp'
]
INSTALLED_APPS = [ # Full list of apps
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles'
] + MY_APPS
Now add a new router to project-app(default-app) folder (The folder where you have your settings.py)
This router should look like:
from project.settings import MY_APPS
class DefaultDBRouter(object):
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
If an app is not defined by me, migrate only if database is 'default'
"""
if app_label not in MY_APPS:
return db == 'default'
return False
Now when adding DefaultDBRouter to DATABASE_ROUTERS in settings.py, make sure that this is the last router in the list. This is at most important since the routers are processed in the order listed.
DATABASE_ROUTERS = ['customApp.dbRouter.CustomDBRouter', 'project.dbRouter.DefaultDBRouter']
Finally, you can migrate using the following commands:
python manage.py makemigrations # Make migrations for all apps
python manage.py migrate # Do migrations to 'default'
python manage.py migrate --database=source_db # Do migrations to 'source'
Solution 2: I don't recommend this
Follow step 1 and 2 of Solution 1.
Now, while doing migrations follow these steps:
python manage.py makemigrations # Make migrations for all apps
python manage.py migrate # Do migrations to `default`
python manage.py migrate customApp --database=source_db
# Do migrations to 'source_db' from `customApp` only
Since tables like auth_group do not belong to migrations of customApp, they won't be created in the last command. You can add these 3 commands to a shell script to make your job easy. Note that, as the number of apps and databases increase, this method will look dirty. On the other hand, Solution-1 provides a clean way to do this.

Was looking for another issue but stumbled upon this question. The answer is to use a database router. In the project's root_app create a file (I called mine model.py) and add the following settings:
from <app> import models as app_models
class DatabaseRouter(object):
def db_for_read(self, model, **hints):
""" Route for when Django is reading the model(s). """
if model in (
app_models.<name>,
):
return "project_db"
return "default"
def db_for_write(self, model, **hints):
""" Route for when Django is writing the model(s). """
if model in (
app_models.<name>
):
return "project_db"
return "default"
Then inside your settings.py you should have:
DATABASES = {
"default": {"ENGINE": "django.db.backends.sqlite3", "NAME": ":memory:"},
"project_db": { ... },
}
And the following line:
DATABASE_ROUTERS = ("root_app.models.DatabaseRouter",)
You should be able to generalize the above for your liking.

Simple step-by-step guide to Django multiple databases please

I've been through a few posts and the Django documentation but still can't get multiple databases to work. I'm new to Django and feel quite confused indeed.
In my project, one app should use an existing vo.sqlite3 with lots of data in it. I want to be able to handle this vo.sqlite3 in my vo app, separately from the default database db.sqlite3 (the one that handles users, etc).
Would you anyone be able to point me to a simple step-by-step way to get such setup to work?
Many thanks.

An example, in your settings, set up your databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite1'),
},
'client1': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite2'),
},
}
Then wharever you perform a query, you can select the database.
UserModel.objects.using(db).get(username=username)
db = you database name
You can view the full example here

Not only levi's solution above worked, I even managed to make automated routing work, see my settings.py and routers.py.
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'db.sqlite3',
},
'vodb': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'vo.sqlite3',
}
}
DATABASE_ROUTERS = ['vo.routers.vo_Router',]
DATABASE_APPS_MAPPING = {'vo': 'vodb'}
routers.py
from django.conf import settings
class vo_Router(object):
def db_for_read(self, model, **hints):
return 'vodb'
def db_for_write(self, model, **hints):
return 'vodb'
def allow_relation(self, obj1, obj2, **hints):
db_list = ('default', 'vodb')
if obj1._state.db in db_list and obj2._state.db in db_list:
return True
return None
def allow_migrate(self, db, model):
return True
def allow_syncdb(self, db, model):
if db == 'vodb':
return model._meta.app_label == 'vo'
elif model._meta.app_label == 'vo':
return False
return None
The piece that was missing was the execution of manage.py syncdb --database=vodb. However, this added a lot of unnecessary Django tables to the app database vo.sqlite3, and even created a new table for the object defined in my models.py.
In my case, I wanted just to link existing tables (with data) in vo.sqlite3 to the objects in models.py. Is there a way to do it Django? Thanks.

Different databases for different apps in Django [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I have multiple apps in my Django site: mainsite, blog and tutorials.
Can I use different databases (e.g. PostgreSQL, MySQL) in different Django apps?

Here is what you will have to do to get going.
Update settings for databases that you want to use.
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/var/db/projectdb'
}
'db_app1': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/var/db/app1db'
}
'db_app2': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/var/db/app2db'
}
For each database, implement a database router that will route the queries appropriately. In your case implement in each app. Note this more or less taken from django docs.
dbRouter.py (place in 'app1' folder):
# DB router for app1
class App1DBRouter(object):
"""
A router to control db operations
"""
route_app_labels = {'app1'}
db_name = 'db_app1'
def db_for_read(self, model, **hints):
"""
Attempts to read auth and contenttypes models go to self.db_name.
"""
if model._meta.app_label in self.route_app_labels:
return self.db_name
return None
def db_for_write(self, model, **hints):
"""
Attempts to write auth and contenttypes models go to self.db_name.
"""
if model._meta.app_label in self.route_app_labels:
return self.db_name
return None
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations if a model in the auth or contenttypes apps is
involved.
"""
if (
obj1._meta.app_label in self.route_app_labels or
obj2._meta.app_label in self.route_app_labels
):
return True
return None
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Make sure the auth and contenttypes apps only appear in the
self.db_name database.
"""
if app_label in self.route_app_labels:
return db == self.db_name
return None
Update DATABASE_ROUTERS in settings.py
settings.py
DATABASE_ROUTERS = ['app1.dbRouter.App1DBRouter', 'app2.dbRouter.App2DBRouter']