The django projet I'm working on has a ton of initial_data fixture data. It seems by default the only way to have data load automatically is to have a file in your app folder called fixtures, and the file needs to be named initial_data.ext (ext being xml or json or yaml or something).
This is really unflexable, I think. I'd rather have a fixtures folder, and then inside that a initial_data folder, and then inside there, one file for each model in that app. Or something to that effect. IS this possible to do in django now? Or maybe some other scheme of better fixture organization.
In my experience, hard-coded fixtures are a pain to write and a pain to maintain. Wherever a model change breaks a fixture, the Django initial load will return a very unfriendly error message and you will end-up adding a bunch a of print's in the Django core in order to find where the problem is coming from.
One of the developer I work with has developed a very good library to overcome this problem, it's called django-dynamic-fixture and we really to love it. Here is how it works:
Suppose you have this models:
class Author(models.Model):
name = models.CharField()
class Book(models.Model):
author = models.ForeingKey(Author, required=True)
title = models.CharField()
In order to create a book instance in your test, all you have to do is
from django_dynamic_fixture import get
from app import Book
class MyTest(TestCase):
def setUp(self):
self.book = get(Book)
The django-dynamic-fixture automatically creates for you any dependencies that are required for the Book model to exist. This is just a simple example but the library can handle very complex model structures.
You can reorganize your initial data fixtures however you want and then write a post_syncdb signal handler which loads them. So it will then be automatically loaded on syncdb, according to the logic defined by you.
See: https://docs.djangoproject.com/en/1.3/ref/signals/#post-syncdb
Yes, you can split fixtures into multiple files with subfolder structures. You can specify fixture files to load and create a script that loads some or all of them. I have done this before so can confirm that it works.
Example: django-admin.py loaddata application/module/model.json
See loaddata documentation for more information.
A hacky way to load an additional initial_data.json or two is to create additional empty apps within your Django project that has nothing but the fixtures folder and the initial_data.json file. If you need the fixture loaded before the other apps' fixtures, you could name it something like aa1. If you need another one you could name it aa2. Your directory structure would look like this:
aa1/
fixtures/
initial_data.json
aa2/
fixtures/
initial_data.json
myrealapp/
fixtures/
initial_data.json
...
You would need to add the apps to INSTALLED_APPS in settings.py.
You can then populate the fixture_data.json files with arbitrary app information as necessary:
(virtualenv) ./manage.py dumpdata --indent=4 auth > aa1/fixtures/initial_data.json
(virtualenv) ./manage.py dumpdata --indent=4 oauth2 > aa2/fixtures/initial_data.json
(virtualenv) ./manage.py dumpdata --indent=4 myrealapp > myrealapp/fixtures/initial_data.json
When you run python manage.py syncdb, each of the fixtures will be loaded in automatically in alphabetical order.
As I mentioned, this is quite hacky, but if you only need a couple extra initial_data.json files and need to be able to control the order they are loaded in, this works.
Related
I have an app that can generate some gibberish model instances on queue. As opposed to having static fixture initial_data, I would like to have something like following during migrations:
if IS_DEBUG and not IS_TEST:
(create_post() for _ in xrange(100))
(create_user() for _ in xrange(100))
I know how to load static fixtures in django, but I would like to have more control over fake data I load into my application.
Just a note - this is not for unit/func/anything tests, but to populate the database to be able to browse the development version of a website and look around.
What is the best way to accomplish this? Custom south migration?
UPDATE: Question that concerns me with south migrations, if I change a model in a migration after that custom migration which populates data - it'll be a bit messy. For example:
0001_initial
0002_generate_fixtures
0003_add_field_to_a_model
So now I'll have to remove 0002_generate_fixtures and create new migration 0004_generate_fixtures. That gets messy very quickly.
Serafeim's and nieka's answers have been very helpful, at the end of the day I decided to go with a custom management command in app_name/management/commands/populatedb.py.
Now, I can simply run python manage.py populatedb to dynamically populate my database.
I've used the factory_boy (https://github.com/rbarrois/factory_boy) with greate success to create custom fixtures (for tests or just populating the database).
Copying from the project's README:
factory_boy is a fixtures replacement based on thoughtbot's
factory_girl.
Its features include:
Straightforward syntax Support for multiple build strategies
(saved/unsaved instances, attribute dicts, stubbed objects) Powerful
helpers for common cases (sequences, sub-factories, reverse
dependencies, circular factories, ...) Multiple factories per class
support, including inheritance Support for various ORMs (currently
Django, Mogo, SQLAlchemy)
I think the best way is to write simple custom script to fill the database. Don't use South for this purpose.
Simply run your custom script when you want to fill database:
fill_db.py:
from django.conf import settings
def fill_db()
if settings.IS_DEBUG and not settings.IS_TEST:
for i in xrange(100):
Post.objects.create()
User.objects.create()
You can also add this script to the django management commands:
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, *args, **options):
fill_db()
I am refactoring a Django application. Specifically, I have an application with a big models.py file and I am trying to split it into a bunch of small files, like
myapp/
models/
__init__.py
somemodels.py
someothers.py
somemore.py
...
and in models/__init__.py I import all models from all other files so that I do not have to change client code.
The problem is that Django now complains about table names. Table for model Foo used to be myapp_foo, but it seems that Django now looks for a table myapp.models_foo. That is, it seems that it uses as prefix the package where the models are defined instead of their application (of course myapp.models is not registered as a Django application).
I know I could manually set the table name for each and every models, but is there a way to avoid this and tell Django that these models are actually part of myapp?
Use Meta.app_label
I have multiple Django apps, and I have some code in one of my models.py that really applies to all of the apps. Is there a way to move it somewhere generic, outside of a specific app folder?
Examples of non app specific code I have:
def update_groups(sender, user=None, ldap_user=None, **kwargs):
...
django_auth_ldap.backend.populate_user.connect(update_groups)
A function to correctly identify users, and connect to the correct signal.
I also have a model proxy of django.contrib.admin.models.LogEntry and a modelAdmin of that model proxy so users in the admin site can view the change history. But these really don't belong in any one app's models.py.
Well, just create a python module somewhere in your project and then refference it in your models. To do this, you need a directory with __init__.py file:
helpers/
__init__.py
functions.py
Put your code into the functions.py and in any other place you will be able to:
from helpers.functions import update_groups
post_save.connect(update_groups)
Name of the module is up to you, ofcourse.
You could create a template app that could be used or extended (class hierarchy) by all your other apps that could use that code.
__init__.py in my project's directory seems to be a good place to put this stuff. It gets run right away when the server starts so it's available to everything else. It seems to work fine so far.
I've noticed that in order for me to define models, I need to do something like:
python manage.py startapp app_name
Is there anyway to avoid this convention and be able to create a models.py directly in the top level site that django-admin.py has created for me? Sometimes I'm building a site that can be put together in literally 15 minutes thanks to Django. I don't see the need for complexity of having an additional app built and a modified settings.py just for a form model or something similar.
Not really. The django admin programs expect app/models.py file names.
I want to write tests that can show whether or not the database is in sync with my models.py file. Actually I have already written them, only to find out that django creates a new database each time the tests are run based on the models.py file.
Is there any way I can make the models.py test use the existing database schema? The one that's in mysql/postgresql, and not the one that's in /myapp/models.py ?
I don't care about the data that's in the database, I only care about it's schema i.e. I want my tests to notice if a table in the database has less fields than the schema in my models.py file.
I'm using the unittest framework (actually the django extension to it) if this has any relevance.
thanks
What we did was override the default test_runner so that it wouldn't create a new database to test against. This way, it runs the test against whatever our current local database looks like. But be very careful if you use this method because any changes to data you make in your tests will be permanent. I made sure that all our tests restores any changes back to their original state, and keep our pristine version of our database on the server and backed up.
So to do this you need to copy the run_test method from django.test.simple to a location in your project -- I put mine in myproject/test/test_runner.py
Then make the following changes to that method:
// change
old_name = settings.DATABASE_NAME
from django.db import connection
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
connection.creation.destroy_test_db(old_name, verbosity)
// to:
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
Make sure to do all the necessary imports at the top and then in your settings file set the setting:
TEST_RUNNER = 'myproject.test.test_runner.run_tests'
Now when you run ./manage.py test Django will run the tests against the current state of your database rather than creating a new version based on your current model definitions.
Another thing you can do is create a copy of your database locally, and then do a check in your new run_test() method like this:
if settings.DATABASE_NAME != 'my_test_db':
sys.exit("You cannot run tests using the %s database. Please switch DATABASE_NAME to my_test_db in settings.py" % settings.DATABASE_NAME)
That way there's no danger of running tests against your main database.