Develop Reference

django-tenant-schemas wont apply migration to tenant schema, only public

I have a multi-tenant django app using django-tenant-schemas.
There is an SiteConfig app:
settings.py:
TENANT_APPS = (
...
'siteconfig',
...
)
INSTALLED_APPS = (
...
'siteconfig',
...
)
But my latest migration on that app won't apply to my tenants:
$ ./manage.py migrate_schemas --shared
[standard:public] === Running migrate for schema public
[standard:public] Operations to perform:
[standard:public] Apply all migrations: account, admin, ... siteconfig, sites, socialaccount, tenant, utilities
[standard:public] Running migrations:
[standard:public] Applying siteconfig.0007_siteconfig_access_code...
[standard:public] OK
As you can see it is only applying the migration to the public schema, and not my tenants.
If I look at my tenant, it shows the migration there as unapplied:
$ ./manage.py tenant_command showmigrations
Enter Tenant Schema ('?' to list schemas): ?
public - localhost
test - test.localhost
Enter Tenant Schema ('?' to list schemas): test
account
[X] 0001_initial
[X] 0002_email_max_length
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add+
.
.
.
siteconfig
[X] 0001_initial
[X] 0002_auto_20200402_2201
[X] 0003_auto_20200402_2218
[X] 0004_auto_20200402_2233
[X] 0005_auto_20200403_0947
[X] 0006_auto_20200403_1528
[ ] 0007_siteconfig_access_code # <-- DIDN'T APPLY!
Why is it not applying to the tenant test and how can I get it to do that?

You are running
manage.py migrate_schemas --shared
Which migrates only public schema
You should run
manage.py migrate_schemas
According to documentation

django postgresql tests.py RuntimeWarning error

I have a Django app that adds and displays stuff to my postgresql database online at elephantsql.com. My project file setup looks like this:
website/
website/
music/
playlist/
__pycache__/
migrations/
static/
templates/
__init__.py
admin.py
apps.py
models.py
tests.py
urls.py
views.py
.coverage
db.sqlite3
manage.py
My project works right now where when I run the server and go to /playlist/ it displays stuff correctly and connects to my postgresql database fine.
My settings.py DATABASES object looks like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'kxvmghva',
'USER': 'kxvmghva',
'PASSWORD': '--an actual password---',
'HOST': 'raja.db.elephantsql.com',
'PORT': '5432',
}
}
Now I'm trying to write test cases in my playlist/tests.py file, but when I try to run these tests I'm getting errors.
my testing file I'm trying to run /playlist/tests.py
from django.test import TestCase
from .models import plays
from django.utils import timezone
class AnimalTestCase(TestCase):
def setUp(self):
print("setup")
#Animal.objects.create(name="lion", sound="roar")
#Animal.objects.create(name="cat", sound="meow")
def test_animals_can_speak(self):
"""Animals that can speak are correctly identified"""
print("test")
#lion = Animal.objects.get(name="lion")
#cat = Animal.objects.get(name="cat")
#self.assertEqual(lion.speak(), 'The lion says "roar"')
#self.assertEqual(cat.speak(), 'The cat says "meow"')
When I run the command "python manage.py test playlist" I get these errors:
C:\Users\marti\Documents\kexp\website>python manage.py test playlist
Creating test database for alias 'default'...
C:\Python36-32\lib\site-packages\django\db\backends\postgresql\base.py:267: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the default database instead.
RuntimeWarning
Got an error creating the test database: permission denied to create database
Type 'yes' if you would like to try deleting the test database 'test_kxvmghva', or 'no' to cancel:
if I type 'yes' it leads to this error:
Destroying old test database for alias 'default'...
Got an error recreating the test database: database "test_kxvmghva" does not exist
I've been trying to solve this error by searching it online and have tried stuff like giving my user 'kxvmghva' CREATEDB permissions as well as running this line in my elephantsql db:
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO kxvmghva;
But I'm still getting these errors when trying to run the tests.py file for my playlist/ app. This is my first time setting up test cases for a postgresql database in django and I would really appreciate any help or guidance. Thanks.

I suppose the plan you are using for database is free/shared, in case which you do not have rights to create additional databases.
This is not a Django problem but restrictions of the service you are using.

As pointed in the first answer, ElephantSQL free tier (I suppose you're using this one) is shared. In that server, they have several DBs, so you have access to one and can't create another one.
I've tried to create an instance just for tests, but it also fails because "On PostgreSQL, USER will also need read access to the built-in postgres database." (from django docs).
So, my solution, and understand it is not the best of practices (because you'll have a different environment for testing) is replacing the engine for testing:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
},
}
This configuration works because (from the django docs):
When using SQLite, the tests will use an in-memory database by default (i.e., the database will be created in memory, bypassing the filesystem entirely!)
Two notes:
Use GitHub actions or a similar solution to test against PostgreSQL in each commit (this way, you reduce the danger of using different test and prod databases)
The easiest way to have a different setting config is to use separate files. I follow the logic in cookiecutter-django

Django MSSQL server on docker container won't migrate

UPDATE: Solved
I am using macOS, Django 2.0 on a virtualenv, and a Docker Container with the SQL Server 2017 up and running. I'm trying to use this package: https://github.com/michiya/django-pyodbc-azure
This is my DATABASES settings.py setup:
DATABASES = {
'default': {
'ENGINE': 'sql_server.pyodbc',
'NAME': 'TUTORIAS_DATOS',
'HOST': '0.0.0.0',
'PORT': '1402',
'USER': '***',
'PASSWORD': '***',
'OPTIONS': {
'driver': 'ODBC Driver 13 for SQL Server',
'driver_charset': 'iso-8859-1',
}
}
}
The problem is that when I do python manage.py makemigrations the migration is made, but when I try to apply the migration to the database (python manage.py migrate) I get:
Operations to perform:
Apply all migrations: admin, auth, contenttypes, menu, sessions
Running migrations:
No migrations to apply.
Instead of what I should get if I use the sqlite (default Django database), which is:
Operations to perform:
Apply all migrations: admin, auth, contenttypes, menu, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying menu.0001_initial... OK
Applying sessions.0001_initial... OK
I don't know what could be the problem, theoretically if I follow this steps (on Windows) it works just fine.
The ODBC connection apparently is working because if I setup an improper username/password I get an error. My database (of the container hosted on 0.0.0.0:1402) name is indeed TUTORIAS_DATOS

UPDATE: I was using TeamSQL app to "see" the tables on the database and make queries. Little I knew the app didn't work properly.. I've connected to the container and queried via sqlcmd and the tables were there... That's why I wasn't getting any "creation" response from Django.
Issue solved, thanks anyways for the comments.

How to stop Django to create lot of unnecessary tables

I have two databases defined in my setting.py file.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'monitoring',
'USER': 'root',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '',
},
'source' :{
'ENGINE': 'django.db.backends.mysql',
'NAME': 'source_db',
'USER': '*****',
'PASSWORD': '*****',
'HOST': '*****',
'PORT': '****',
}
}
I need to access some tables in source_db which django is not allowing to do so unless I migrate the db. So, once we run command python manage.py migrate --database=source , Django is creating some tables in server db. Since we are not allowed to create tables in server db, is there any way to stop django doing so, or any way to access tables without migrating the db?
This is the list of tables which we don't want to create.
+--------------------------------+
| Tables_in_source_db |
+--------------------------------+
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| dashboard_monitoring_features |
| dashboard_monitoring_modelinfo |
| dashboard_monitoring_product |
| django_admin_log |
| django_content_type |
| django_migrations |
| django_session |
+--------------------------------+

If you want to migrate / create tables in the default database and not source database, then you have to define app and database when you run migrations. Like:
python manage.py migrate dashboard --database=default
This will run migration in the dashboard app and create tables in the default database.
Next thing you want to do is set your source models to non-managed. You do by specifying managed = False in the meta class of the model:
class YourModel(models.Model):
... your fields here ...
class Meta:
managed = False
From the documentation:
If False, no database table creation or deletion operations will be
performed for this model. This is useful if the model represents an
existing table or a database view that has been created by some other
means.

If you don't want particular tables, then remove
'django.contrib.auth`,
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
and dashboard app
from your INSTALLED_APPS

Probably nobody was clear about what the author of the question was trying to do. So let me start with the context.
Context
By default, Django uses a database named default. Let's say I have a CustomModel in a CustomApp for which the table should be created in a new database called source_db.
Thus we have two databases: 1)default and 2)source_db.
We want the default tables like auth_group required by apps like django.contrib.auth to be created only in default. The source_db should have and only have tables from CustomApp. How do we do this?
Solution 1: I recommend this method
First, refer to Multiple Databases, the official documentation to understand the routing basics. Give special attention to allow_migrate(). Let me insist, understanding of this documentation is at most important to understand the rest of this solution.
Let's begin:
In your CustomModel, set managed = True in the meta data:
class CustomModel(models.Model):
# your fields here ...
class Meta:
managed = True
This specifies that the table creation and alteration for this model should be taken care of by Django.
Write your CustomDBRouter for tables in CustomApp. (Refer to Multiple Databases for this). Don't forget to define allow_migrate().
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Make sure the 'CustomApp' only appear in the 'source_db' database.
"""
if app_label == 'CustomApp':
# if app=CustomApp and db=source_db, migrate
return db == 'source_db'
return None # No suggestions, ask the next router in the list
Go to your settings.py and split the list INSTALLED_APPS as follows:
MY_APPS = [ # List of apps defined by you
'customApp'
]
INSTALLED_APPS = [ # Full list of apps
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles'
] + MY_APPS
Now add a new router to project-app(default-app) folder (The folder where you have your settings.py)
This router should look like:
from project.settings import MY_APPS
class DefaultDBRouter(object):
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
If an app is not defined by me, migrate only if database is 'default'
"""
if app_label not in MY_APPS:
return db == 'default'
return False
Now when adding DefaultDBRouter to DATABASE_ROUTERS in settings.py, make sure that this is the last router in the list. This is at most important since the routers are processed in the order listed.
DATABASE_ROUTERS = ['customApp.dbRouter.CustomDBRouter', 'project.dbRouter.DefaultDBRouter']
Finally, you can migrate using the following commands:
python manage.py makemigrations # Make migrations for all apps
python manage.py migrate # Do migrations to 'default'
python manage.py migrate --database=source_db # Do migrations to 'source'
Solution 2: I don't recommend this
Follow step 1 and 2 of Solution 1.
Now, while doing migrations follow these steps:
python manage.py makemigrations # Make migrations for all apps
python manage.py migrate # Do migrations to `default`
python manage.py migrate customApp --database=source_db
# Do migrations to 'source_db' from `customApp` only
Since tables like auth_group do not belong to migrations of customApp, they won't be created in the last command. You can add these 3 commands to a shell script to make your job easy. Note that, as the number of apps and databases increase, this method will look dirty. On the other hand, Solution-1 provides a clean way to do this.

Was looking for another issue but stumbled upon this question. The answer is to use a database router. In the project's root_app create a file (I called mine model.py) and add the following settings:
from <app> import models as app_models
class DatabaseRouter(object):
def db_for_read(self, model, **hints):
""" Route for when Django is reading the model(s). """
if model in (
app_models.<name>,
):
return "project_db"
return "default"
def db_for_write(self, model, **hints):
""" Route for when Django is writing the model(s). """
if model in (
app_models.<name>
):
return "project_db"
return "default"
Then inside your settings.py you should have:
DATABASES = {
"default": {"ENGINE": "django.db.backends.sqlite3", "NAME": ":memory:"},
"project_db": { ... },
}
And the following line:
DATABASE_ROUTERS = ("root_app.models.DatabaseRouter",)
You should be able to generalize the above for your liking.

Restart postgres connection in django

I have very strange situation here.
Problem:
I describe original problem inside this post, but to sum up:
After using the project for a while, makemigrations stop working for me.
(Yes I have set everything ok, setting are ok,.. please see my comments on previous post)
I decided that I can't figure this out, so I would start fresh. (thats why git is for :D )
So, I delete my django project, I create new virtual environment, and I also create new database and new user in postgres.
I update new database parameter inside my config file and try to run initial makemigrations but with no luch. It is still say that no new migrations are available.
I am desperate because I can't work on the project, so any solution would do :D
(virtual_environment) MacBook-Pro-2:MY_PROJECT marko$ python manage.py makemigrations connectors environment=local
I will use local settings.
No changes detected in app 'connectors'
All my migrations
(virtual_environment) MacBook-Pro-2:MY_PROJECT marko$ python manage.py showmigrations environment=local
I will use local settings.
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add
auth
[X] 0001_initial
[X] 0002_alter_permission_name_max_length
[X] 0003_alter_user_email_max_length
[X] 0004_alter_user_username_opts
[X] 0005_alter_user_last_login_null
[X] 0006_require_contenttypes_0002
[X] 0007_alter_validators_add_error_messages
[X] 0008_alter_user_username_max_length
connectors
(no migrations)
contenttypes
[X] 0001_initial
[X] 0002_remove_content_type_name
sessions
[X] 0001_initial
When lunch migration
(virtual_environment) MacBook-Pro-2:MY_PROJECT marko$ python manage.py migrate --fake-initial environment=local
I will use local settings.
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
No migrations to apply.
The funny thing is that if I close postgres database, I still get the same text in terminal. I would guess that I should get some connection error.
database config yaml
database:
port: 5432
host: 'localhost'
user: 'user1'
password: 'user_password_123!'
db: 'db1'
and my settings file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': CONFIG['database']['db'],
'USER': CONFIG['database']['user'],
'PASSWORD': CONFIG['database']['password'],
'HOST': CONFIG['database']['host'],
'PORT': CONFIG['database']['port'],
}
}

After a lot of hours, I found solution, and it was actually my mistake.
I will post the answer just for those who make similar mistake :D
In fact, #Alasdair and #AdamBarnes was right. I wasn't connecting to new database, but not because the database settings was wrong (host, port, username, db,...)
What I have in settings.py was settings for database and database for unit tests.
# Databases start
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': CONFIG['database']['db'],
'USER': CONFIG['database']['user'],
'PASSWORD': CONFIG['database']['password'],
'HOST': CONFIG['database']['host'],
'PORT': CONFIG['database']['port'],
}
}
if 'test' or 'jenkins' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'test_database',
}
The problem was if statement below. For some reason he understand that this was always true. So program migrate sqlite internal test database.
I need to fix this with
if ('test' in sys.argv) or ('jenkins' in sys.argv):
This fix first problem, that when make migrations it leave empty database.
After this fix, migrate was populate database with default Django tables, but not with my tables.
The fix for this was to import my models.py inside admin.py file.
import connectors.models
Thank's a lot, everybody that give me great hints, to find the solution.