I have a bootstrap script that performs syncdb and migrate:
import settings
from django.core.management import setup_environ, call_command
setup_environ(settings) # Setting up the env settings
call_command('syncdb', migrate=True, interactive=False) # Sync the database
Pre-Requisites:
django-south for migrations.
Process happening:
initial_data fixture contains data for a model that is created by migrations.
syncdb is executed it creates all the tables except for those apps where migrations exists.
Post syncdb it tries to load initial_data and raises error of db not found because the table for app with migrations were not created by syncdb. [ Problem ]
Then it performs migration which creates the db.
Post migration it automatically loads initial_data successfully this time.
Problem:
How can I get rid of the error, when it tries to load fixture for the table that is not yet created ?
Can I edit the above script in a way so that it loads initial_data only after performing the migration ?
You could disable loading initial data when syncdb:
call_command('syncdb', load_initial_data=False, interactive=False)
call_command('migrate', interactive=False)
From the source code of syncdb.py:
# Stealth option -- 'load_initial_data' is used by the testing setup
# process to disable initial fixture loading.
load_initial_data = options.get('load_initial_data', True)
There are a few ways you can solve this:
Exclude the apps from the initial data dump by just calling the apps you want to populate data for
Could try this library https://github.com/davedash/django-fixture-magic
You could write a custom management command to populate the models you require
You can also use data migration that comes with south http://south.aeracode.org/docs/tutorial/part3.html
Personally I would go with either 1 or 3. With the first point, store the fixtures individually in each app under a fixtures folder. However, this is pain to update if your models change. So writing a custom management command might be the most painless.
Related
I have an application in Django 2.0 and as a database engine I use MySQL. I have a problem because the database was previously created and already has records, my idea is to use this same database for the application I am creating.
Use the command
python manage.py inspectdb > models.py
To create the models.py file which will be cleaned as indicated by the models.py file that was generated.
#This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
# * Rearrange models' order
# * Make sure each model has one field with primary_key=True
# * Make sure each ForeignKey has `on_delete` set to the desired behavior.
# * Remove `managed = False` lines if you wish to allow Django to create, modify, and delete the table
# Feel free to rename the models, but don't rename db_table values or field names.
After this I proceed to execute:
python manage.py migrate
python manage.py makemigrations
python manage.py migrate
But it generates the following error:
(1050, "Table 'XXXXXXX' already exists")
Obviously it tells me that the table already exists, but how do I not generate this error and continue administering the tables from Django.
You need to run --fake-initial or --fake. See more at Django migrations. Be careful because running inspectdb doesn't solve all your problems. You need to fix the things inside models.py manually and migrate again.
One of the things (and the main reason I do not use Django) is it likes to take control of everything. The fact that it controls the database means that if you don't start strictly in Django, you are doing it wrong.
However there is a work around:
https://docs.djangoproject.com/en/2.0/howto/legacy-databases/
As the title suggests, I am interested in how can I create the same model in multiple databases. I have the following scenario: every user has his own database (this is the simplified case) and when the user adds a new module to his app, I want to be able to create the db table for the module that he added.
The idea that I currently have is to create dynamic models, add custom app_label, and than use a Database Router that creates the model in the corresponding database.
Is there an easier way to obtain this?
Thanks in advance!
You should look at the syncdb command - you can call it programmatically like so:
from django.core.management import call_command
call_command('syncdb')
In your case, you'll want to use named arguments for any options you would have passed in at the command line:
call_command('syncdb', noinput=True, database=user_db)
Hope that helps.
Check this out also: https://docs.djangoproject.com/en/dev/ref/django-admin/#running-management-commands-from-your-code
Use the migrate command with different arguments. For instance if your db aliases are primary and secondary
do this
python manage.py migrate --database=primary
python manage.py migrate --databases=secondary
Is there a way to execute a Django fixture just once - when the appropriate table is created? I have some initial data that should be put in the app tables, but once the tables are there, I don't want every ./manage.py syncdb to refresh the data. According to Django docs it seems this can only be done for fixtures in SQL format and not JSON / YAML:
http://docs.djangoproject.com/en/1.3/howto/initial-data/
You're going to want to use the post_syncdb signal, and filter/manually load the fixture via the underlying methods when specific apps or models are created.
The django projet I'm working on has a ton of initial_data fixture data. It seems by default the only way to have data load automatically is to have a file in your app folder called fixtures, and the file needs to be named initial_data.ext (ext being xml or json or yaml or something).
This is really unflexable, I think. I'd rather have a fixtures folder, and then inside that a initial_data folder, and then inside there, one file for each model in that app. Or something to that effect. IS this possible to do in django now? Or maybe some other scheme of better fixture organization.
In my experience, hard-coded fixtures are a pain to write and a pain to maintain. Wherever a model change breaks a fixture, the Django initial load will return a very unfriendly error message and you will end-up adding a bunch a of print's in the Django core in order to find where the problem is coming from.
One of the developer I work with has developed a very good library to overcome this problem, it's called django-dynamic-fixture and we really to love it. Here is how it works:
Suppose you have this models:
class Author(models.Model):
name = models.CharField()
class Book(models.Model):
author = models.ForeingKey(Author, required=True)
title = models.CharField()
In order to create a book instance in your test, all you have to do is
from django_dynamic_fixture import get
from app import Book
class MyTest(TestCase):
def setUp(self):
self.book = get(Book)
The django-dynamic-fixture automatically creates for you any dependencies that are required for the Book model to exist. This is just a simple example but the library can handle very complex model structures.
You can reorganize your initial data fixtures however you want and then write a post_syncdb signal handler which loads them. So it will then be automatically loaded on syncdb, according to the logic defined by you.
See: https://docs.djangoproject.com/en/1.3/ref/signals/#post-syncdb
Yes, you can split fixtures into multiple files with subfolder structures. You can specify fixture files to load and create a script that loads some or all of them. I have done this before so can confirm that it works.
Example: django-admin.py loaddata application/module/model.json
See loaddata documentation for more information.
A hacky way to load an additional initial_data.json or two is to create additional empty apps within your Django project that has nothing but the fixtures folder and the initial_data.json file. If you need the fixture loaded before the other apps' fixtures, you could name it something like aa1. If you need another one you could name it aa2. Your directory structure would look like this:
aa1/
fixtures/
initial_data.json
aa2/
fixtures/
initial_data.json
myrealapp/
fixtures/
initial_data.json
...
You would need to add the apps to INSTALLED_APPS in settings.py.
You can then populate the fixture_data.json files with arbitrary app information as necessary:
(virtualenv) ./manage.py dumpdata --indent=4 auth > aa1/fixtures/initial_data.json
(virtualenv) ./manage.py dumpdata --indent=4 oauth2 > aa2/fixtures/initial_data.json
(virtualenv) ./manage.py dumpdata --indent=4 myrealapp > myrealapp/fixtures/initial_data.json
When you run python manage.py syncdb, each of the fixtures will be loaded in automatically in alphabetical order.
As I mentioned, this is quite hacky, but if you only need a couple extra initial_data.json files and need to be able to control the order they are loaded in, this works.
I want to write tests that can show whether or not the database is in sync with my models.py file. Actually I have already written them, only to find out that django creates a new database each time the tests are run based on the models.py file.
Is there any way I can make the models.py test use the existing database schema? The one that's in mysql/postgresql, and not the one that's in /myapp/models.py ?
I don't care about the data that's in the database, I only care about it's schema i.e. I want my tests to notice if a table in the database has less fields than the schema in my models.py file.
I'm using the unittest framework (actually the django extension to it) if this has any relevance.
thanks
What we did was override the default test_runner so that it wouldn't create a new database to test against. This way, it runs the test against whatever our current local database looks like. But be very careful if you use this method because any changes to data you make in your tests will be permanent. I made sure that all our tests restores any changes back to their original state, and keep our pristine version of our database on the server and backed up.
So to do this you need to copy the run_test method from django.test.simple to a location in your project -- I put mine in myproject/test/test_runner.py
Then make the following changes to that method:
// change
old_name = settings.DATABASE_NAME
from django.db import connection
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
connection.creation.destroy_test_db(old_name, verbosity)
// to:
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
Make sure to do all the necessary imports at the top and then in your settings file set the setting:
TEST_RUNNER = 'myproject.test.test_runner.run_tests'
Now when you run ./manage.py test Django will run the tests against the current state of your database rather than creating a new version based on your current model definitions.
Another thing you can do is create a copy of your database locally, and then do a check in your new run_test() method like this:
if settings.DATABASE_NAME != 'my_test_db':
sys.exit("You cannot run tests using the %s database. Please switch DATABASE_NAME to my_test_db in settings.py" % settings.DATABASE_NAME)
That way there's no danger of running tests against your main database.