I have two databases that my Django application needs access. One is a shared database owned by a separate app with the Django app only having read access. The second is entirely owned by the Django app.
For local development I am ok but I'm not sure how to configure things so that Heroku uses the second database.
Currently I have the shared database promoted to DATABASE_URL and the secondary database is at HEROKU_POSTGRESQL_BLUE_URL.
In my settings I have:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'main_database_name',
'USER': 'username',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}, 'secondary': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'secondary_database_name',
'USER': 'username',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}
}
Please ask any more questions if you need me to clarify.
Thanks!
In summary, my specific problem is: I don't know how to have Heroku use the HEROKU_POSTGRESQL_BLUE_URL as the "secondary" database.
---Edit----
At the bottom of settings.py:
# Configure Django App for Heroku.
import django_heroku
django_heroku.settings(locals())
This is where the connection is made between my app's default database and Heroku's DATABASE_URL. I still haven't solved the issue but after some troubleshooting help in the comments, I believe the answer will be found in there.
Here is my working solution.
I have a local_settings.py file not tracked by version control. This contains the database settings for the developer's postgres instance.
local_settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'main_database_name',
'USER': 'username',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}, 'secondary': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'secondary_database_name',
'USER': 'username',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}
}
Then in settings.py I try to import the local_settings.py file. If it doesn't exist, I use the django_heroku package to configure everything for Heroku. In my case, I need to pass the keyword argument databases=False since I want to do a more custom configuration for the databases.
settings.py (only the relevant parts)
# Database
# https://docs.djangoproject.com/en/2.0/ref/settings/#databases
# These database settings are used for heroku deployments. They are overwritten with local_settings.py in development.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ.get('DATABASE_URL'),
'USER': '',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}, 'secondary': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ.get('SECONDARY_DATABASE_URL'),
'USER': '',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}
}
try:
from local_settings import *
except ImportError as e:
# Configure Django App for Heroku.
import django_heroku
django_heroku.settings(locals(), databases=False)
In order for this to work you would need to set the Heroku env variable SECONDARY_DATABASE_NAME to the url of your database.
To find out the URL of your secondary database run heroku config -a your_app_name. Copy the url and then set a new environment param in heroku by running
heroku config:set SECONDARY_DATABASE_URL=postgres://xxxxxxx -a your_app_Name
There are a lot of choices with how you would handle the ENV variables, I liked this because I can just set it in my staging and production environments and the same settings work for both.
Related
I want to keep my unit test database completely separate from other environments including using different user credentials. This is mostly to prevent anyone from unintentionally running unit tests against the development database and mangling the dev data or wiping it out entirely if the --keepdb option isn't specified. The code below detects the "test" in the sys args and this seems to work but is very clunky. If I'm missing a better way to do this please advise.
I have separate settings files for each environment so this will only be on the development server where the unit tests are run automatically and won't end up on any production servers.
Environment:
Django 1.11
Python 3.4.x
MariaDB
# this works but is clunky
import sys
if 'test' in sys.argv:
DATABASES = { # test db and user
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db_test',
'USER': 'test_user',
'PASSWORD': 'secretpassword',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': { # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # redundant but explicit!
},
}
}
I'd like to do this but unfortunately Django doesn't look at the TEST credentials
# cleaner approach but doesn't work - don't do this!
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # Django uses the test db NAME
'USER':'test_user_ignored', # but ignores the USER and PASSWORD
'PASSWORD':'ignoredpassword',
},
}
}
Would something like this work for your situation?
import sys
if 'test' in sys.argv:
NAME = 'dev_db_test'
USER = 'test_user'
PASSWORD ='secretpassword'
else:
NAME = 'dev_db'
USER = 'dev_db_user'
PASSWORD ='dev_password'
DATABASES ={ # test db and user
'default':
{
'ENGINE': 'django.db.backends.mysql',
'NAME': NAME,
'USER': USER,
'PASSWORD': PASSWORD,
'HOST': 'the-db-host',
'PORT': '3306',
'TEST':
{ # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
print(DATABASES)
AFAIK, you don't need to create a separate test database if you want to run unit tests over it. Here is the documentation link of test database.
It states that:
Tests that require a database (namely, model tests) will not use your “real” (production) database.
Separate, blank databases are created for the tests.
You can follow this link to write your tests and it will not affect your production or development database.
I am using 2 postgres databases in my django app.
'newpostgre': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'new_postgre2',
'USER': 'tester',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '',
},
'newpostgre2': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'new_postgre3',
'USER': 'tester',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '',
},
I have a very simple model
class Check1(models.Model):
title = models.CharField(max_length=100)
I had run
python manage.py migrate --database=newpostgre
python manage.py migrate --database=newpostgre2
when I open my new_postgre2(for newpostgre) database in postgre, I can see my Check1 table there.
But in my new_postgre3(for newpostgre2) database in postgre, no Check1 table is there, only those initial migrations are there.
Why I can't see my table in new_postgre3 when migrations have been successfully made?
Because The same object name can be used in different schemas sometimes conflict. The best practice allow many users to use one database without interfering with each other. Use this change second database user and check again. I think it is work.
'newpostgre': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'new_postgre2',
'USER': 'tester',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '',
},
'newpostgre2': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'new_postgre3',
'USER': 'tester_different',
'PASSWORD': 'password_different',
'HOST': 'localhost',
'PORT': '',
},
python manage.py migrate --database=newpostgre2
I originally had a django project with a single app and all models were defined in that app. The project, when initiated only used the default database. It has now become an unwieldy app that I'm trying to break down into smaller apps. Doing so, I want to use different databases for the different apps. I've setup new databases and a router in the settings.py file. However, I'm confused about how to migrate existing tables to the new databases.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'db_name',
'USER': 'db_user_name',
'PASSWORD': 'password',
'HOST': 'hostname',
'PORT': '3306',
},
'db2': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'db_name2',
'USER': 'db_user_name2',
'PASSWORD': 'password2',
'HOST': 'hostname2',
'PORT': '3306',
}
}
I would want an app (e.g. app1) to be moved from default to db2. The router already knows to specify app1 to db2 but running migrations is doing nothing. Any ideas?
#knbk's answer was ultimately correct, except that the solution involved an additional step.
1. python manage.py migrate auth --database=db2
2. python manage.py migrate app1 --database=db2
I have two settings files for my django project with different database settings.
First:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dbname',
'USER': 'user',
'PASSWORD': 'megahardbreakingpassword',
'HOST': 'localhost',
'PORT': '',
'OPTIONS': {
'init_command': 'SET storage_engine=INNODB',
}
}
}
And second:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'db.sqlite3',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
When I run server with second database settings project works fine, but when I run first settings file with mysql, django returns 404 error for all urls.
I also use django-hosts for implementing subdomains patterns, so maybe something wrong here. My hosts.py code:
from django_hosts import patterns, host
host_patterns = patterns('',
host(r'^$', 'project.urls', name='host'),
)
I use django 1.4.2 and Python 2.7.9
And finally I found answer! According to e4c5's comment I searched problem in my views instead of my settings files. I wrote simple unittest and it gives me some strange tracebeck about my MySQL database such as wrong column name and deprecation warnings about duplication indexes.
I reinstall older version of MySQL server (5.0.70 instead of 5.6.25), run syncdb and migrations again and it completely solved my problems. So God bless TDD!
I'm using Heroku with Django.
The database is configured in settings.py this way, using the dj-database-url module:
DATABASES = {'default': dj_database_url.config()}
How can I do to add additional parameters, for example ATOMIC_REQUESTS? In a "normal" case I would do like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
'ATOMIC_REQUESTS': True,
}
}
DATABASES is just a dictionary: you can add the relevant settings after the initial configuration.
DATABASES = {'default': dj_database_url.config()}
DATABASES['default']['ATOMIC_REQUESTS'] = True