I need to migrate from sqlite3 to postgres. The dumpdata of the whole database doesn't work so I'm doing it one by one. The problem is that when I try to load users it raises error:
ValueError: Problem installing fixture '/.../dbmig/auth.user.json': Could not load auth.User(pk=1): Cannot add "<Group: contractor>": instance is on database "postgres", value is on database "default"
This is not true, because the groups were migrated before users.
I've created a command that I use:
def dumpload(model):
json_filename = model+'.json'
DBMIG_DIR = os.path.join(settings.BASE_DIR, 'dbmig')
json_filepath = os.path.join(DBMIG_DIR,json_filename)
with open(json_filepath, 'w') as f:
call_command('dumpdata', model, stdout=f, database='default')
call_command('loaddata', json_filepath, database='postgres')
class Command(BaseCommand):
def handle(self, *args, **options):
dumpload('auth.group')
dumpload('auth.user')
EDIT
It doesn't make sense, the groups are already in the postgres db. I've added print(Group.objects.using('postgres').values('pk')) between the two commands and I can clearly see that the groups are in the postgres database and they have correct ids.
This is the output:
Installed 2 object(s) from 1 fixture(s)
<QuerySet [{'pk': 1}, {'pk': 2}]>
error...
EDIT - settings.DATABASES
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
},
'postgres': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'OPTIONS': {
},
'NAME': config.get('db', 'name'),
'USER': config.get('db', 'user'),
'PASSWORD': config.get('db', 'password'),
'HOST': config.get('db', 'host'),
'PORT': config.get('db', 'port'),
}
}
Related
I'm trying to perform unit tests on Django with db mocking by mongomock.
My connection to the DB is made using settings.py:
DATABASES = {
'dbname': {
'ENGINE': 'djongo',
'NAME': 'db1',
'ENFORCE_SCHEMA': False,
'CLIENT': {
'host': DB_HOST,
'port': DB_PORT,
'username': DB_USERNAME,
'password': DB_PASSWORD,
'authSource': 'admin',
'authMechanism': 'SCRAM-SHA-1'
}
},
'default': {
'ENGINE': 'djongo',
'NAME': 'users',
'ENFORCE_SCHEMA': False,
'CLIENT': {
'host': DB_HOST,
'port': DB_PORT,
'username': DB_USERNAME,
'password': DB_PASSWORD,
'authSource': 'admin',
'authMechanism': 'SCRAM-SHA-1'
}
}
}
and the test looks like this:
from django.contrib.auth.models import User
class BaseApiTest():
def test_access_login(self):
User.objects.create_user(
username='admin',
password='admin'
)
....
self.assertEqual(AccessAttempt.objects.count(), 1)
Here there is an internal connection of Django to the DB. (when its trying to access django.contrib.auth.models.User)
Any ideas?
The goal is to run the tests without a DB instance.
In other words, how can I overwrite the Django connection (self.databases) to the DB using a mongomock connection...
I have a Django application using decouple and dj_database_url (it is not on Heroku). If I put my connection information in settings.ini (DB_NAME, DB_USER, DB_PASSWORD, DB_HOST, and DB_PORT) and set up my database connection in settings.py like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': config('DB_NAME'),
'USER': config('DB_USER'),
'PASSWORD': config('DB_PASSWORD'),
'HOST': config('DB_HOST'),
'PORT': 'DB_PORT',
}
}
It works fine.
However, if I put this in my settings.ini:
DATABASE_URL=postgres://johndoe:mypassword#123.456.789.000:5000/blog_db
and reference it like this in my settings.py:
DATABASES = {"default": dj_database_url.config(default=config("DATABASE_URL"))}
It doesn't work. I just get a 500 when I try to run my server. I presume there is something wrong with my syntax. Can anyone point out what I'm doing wrong?
Thanks!
I want to keep my unit test database completely separate from other environments including using different user credentials. This is mostly to prevent anyone from unintentionally running unit tests against the development database and mangling the dev data or wiping it out entirely if the --keepdb option isn't specified. The code below detects the "test" in the sys args and this seems to work but is very clunky. If I'm missing a better way to do this please advise.
I have separate settings files for each environment so this will only be on the development server where the unit tests are run automatically and won't end up on any production servers.
Environment:
Django 1.11
Python 3.4.x
MariaDB
# this works but is clunky
import sys
if 'test' in sys.argv:
DATABASES = { # test db and user
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db_test',
'USER': 'test_user',
'PASSWORD': 'secretpassword',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': { # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # redundant but explicit!
},
}
}
I'd like to do this but unfortunately Django doesn't look at the TEST credentials
# cleaner approach but doesn't work - don't do this!
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dev_db',
'USER': 'dev_db_user',
'PASSWORD': 'dev_password',
'HOST': 'the-db-host',
'PORT': '3306',
'TEST': {
'NAME':'dev_db_test', # Django uses the test db NAME
'USER':'test_user_ignored', # but ignores the USER and PASSWORD
'PASSWORD':'ignoredpassword',
},
}
}
Would something like this work for your situation?
import sys
if 'test' in sys.argv:
NAME = 'dev_db_test'
USER = 'test_user'
PASSWORD ='secretpassword'
else:
NAME = 'dev_db'
USER = 'dev_db_user'
PASSWORD ='dev_password'
DATABASES ={ # test db and user
'default':
{
'ENGINE': 'django.db.backends.mysql',
'NAME': NAME,
'USER': USER,
'PASSWORD': PASSWORD,
'HOST': 'the-db-host',
'PORT': '3306',
'TEST':
{ # redundant but explicit!
'NAME':'dev_db_test',
},
}
}
print(DATABASES)
AFAIK, you don't need to create a separate test database if you want to run unit tests over it. Here is the documentation link of test database.
It states that:
Tests that require a database (namely, model tests) will not use your “real” (production) database.
Separate, blank databases are created for the tests.
You can follow this link to write your tests and it will not affect your production or development database.
I know django works with multiple database connections, I need to insert in a database that is not the default, but I get the following error.
ProgrammingError at /api-jde/f59presapi/2279/
(1146, 'Table \'dblocal."oracle"."FPRES"\' doesn\'t exist')
this is the configuration of my settings.py file
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dblocal',
'USER': 'root',
'PASSWORD': '',
'HOST': '',
'PORT': '',
},
'bd2': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'XXXX',
'USER': 'oracle',
'PASSWORD': 'xxxxx',
},
}
the bd in which I need to insert is the db2 (I'm not putting all the data here but if I have them full), in the created model I have the METa configuration with the following:
class Meta:
managed = False
db_table = u'"oracle"."FPRES"'
db = DATABASES['db2']
Perhaps this will help if added to the meta class.
As far as I know you can't specify a database for a specific model. You need to do something like this every time:
YourModel.objects.using('bd2').all()
Hope this helps.
I am trying to install graphite-web on my computer by following the instructions on
https://gist.github.com/surjikal/2777886. When I try to populate the database with manage.py syncdb it does not find a proper database throwing the following error:
> sudo python manage.py syncdb
OperationalError: FATAL: database "/opt/graphite/storage/graphite.db" does not exist
Here you can find my database configuration from local_settings.py:
DATABASES = {
'default': {
'NAME': '/opt/graphite/storage/graphite.db',
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'USER': 'graphite',
'PASSWORD': '<password>',
'HOST': 'localhost',
'PORT': '5678'
}
}
Postgres is running properly:
sudo service --status-all | grep postgres
...
[ + ] postgresql
Why manage.py syncdb cannot create / find the graphite.db file?
If you are using postgresql then database path will not be required. You just need database name. like
DATABASES = {
'default': {
'NAME': 'graphite',
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'USER': 'graphite',
'PASSWORD': '<password>',
'HOST': 'localhost',
'PORT': '5678'
}
}