Is it possible to run a migration from Python code? I don't want to (actually I cannot) use the terminal command:
venv/bin/python src/manage.py migrate myapp
I need to run it from Python code as part of my business logic on a dynamically created database.
This is what I have so far:
db_name = uuid.uuid4()
from settings.local import DATABASES
new_database = {}
new_database['ENGINE'] = 'django.db.backends.mysql'
new_database['NAME'] = db_name
new_database['USER'] = DATABASES["default"]["USER"]
new_database['PASSWORD'] = DATABASES["default"]["PASSWORD"]
new_database['HOST'] = DATABASES["default"]["HOST"]
new_database['PORT'] = DATABASES["default"]["PORT"]
import settings
database_id = str(uuid.uuid4())
settings.DATABASES[database_id] = new_database
from django.core.management import call_command
call_command('migrate', 'catalogue', database=database_id)
But I get:
KeyError: '28a4eb10-91e4-4de8-8a74-15d72f8245ef'
Yes, you can. Assuming it is called from within Django app:
from django.core.management import call_command
call_command('migrate', 'myapp')
I believe the issue you are having is that you are configuring the new database in runtime. So, the configuration you change in runtime is over the process/thread you are hitting with that code. The settings dict is immutable and you can not change it in runtime (see django docs).
I am not sure, but very probably the setting dict used by call_command is a copy from the original dict loaded from the settings configuration and that is why you get the key error (there is no new config in there).
By the way, is published that doing changes over settings configuration in runtime is not recommended by the django community.
I'm trying to use a different set of configs for testing my django app (like, using in-memory sqlite3 instead of mysql) and for this purpose I'm trying to use another settings module. I run my tests this way:
python manage.py test --settings=test_settings
But Django seems to ignore my test_settings module. Am I doing something wrong?
Is test_settings a file or a directory?
Here's how I load different settings for tests. In settings.py, at the bottom:
# if manage.py test was called, use test settings
if 'test' in sys.argv:
try:
from test_settings import *
except ImportError:
pass
Super bonus! If you want to use sqlite3 for tests, you should activate integrity constraints, so you get the same foreign key exceptions as with mysql (lost of lot of time with this one). In some file of your project:
from django.db.backends.signals import connection_created
def activate_foreign_keys(sender, connection, **kwargs):
"""Enable integrity constraint with sqlite."""
if connection.vendor == 'sqlite':
cursor = connection.cursor()
cursor.execute('PRAGMA foreign_keys = ON;')
connection_created.connect(activate_foreign_keys)
I just bumped into this and confirm that on Django 1.6.1 --settings option is ignored.
I filed a ticket on the Django bug tracker:
https://code.djangoproject.com/ticket/21635#ticket
I have two questions:
How do I delete a table in Django?
How do I remove all the data in the table?
This is my code, which is not successful:
Reporter.objects.delete()
Inside a manager:
def delete_everything(self):
Reporter.objects.all().delete()
def drop_table(self):
cursor = connection.cursor()
table_name = self.model._meta.db_table
sql = "DROP TABLE %s;" % (table_name, )
cursor.execute(sql)
As per the latest documentation, the correct method to call would be:
Reporter.objects.all().delete()
If you want to remove all the data from all your tables, you might want to try the command python manage.py flush. This will delete all of the data in your tables, but the tables themselves will still exist.
See more here: https://docs.djangoproject.com/en/1.8/ref/django-admin/
Using shell,
1) For Deleting the table:
python manage.py dbshell
>> DROP TABLE {app_name}_{model_name}
2) For removing all data from table:
python manage.py shell
>> from {app_name}.models import {model_name}
>> {model_name}.objects.all().delete()
Django 1.11 delete all objects from a database table -
Entry.objects.all().delete() ## Entry being Model Name.
Refer the Official Django documentation here as quoted below -
https://docs.djangoproject.com/en/1.11/topics/db/queries/#deleting-objects
Note that delete() is the only QuerySet method that is not exposed on a Manager itself. This is a safety mechanism to prevent you from accidentally requesting Entry.objects.delete(), and deleting all the entries. If you do want to delete all the objects, then you have to explicitly request a complete query set:
I myself tried the code snippet seen below within my somefilename.py
# for deleting model objects
from django.db import connection
def del_model_4(self):
with connection.schema_editor() as schema_editor:
schema_editor.delete_model(model_4)
and within my views.py i have a view that simply renders a html page ...
def data_del_4(request):
obj = calc_2() ##
obj.del_model_4()
return render(request, 'dc_dash/data_del_4.html') ##
it ended deleting all entries from - model == model_4 , but now i get to see a Error screen within Admin console when i try to asceratin that all objects of model_4 have been deleted ...
ProgrammingError at /admin/dc_dash/model_4/
relation "dc_dash_model_4" does not exist
LINE 1: SELECT COUNT(*) AS "__count" FROM "dc_dash_model_4"
Do consider that - if we do not go to the ADMIN Console and try and see objects of the model - which have been already deleted - the Django app works just as intended.
django admin screencapture
Use this syntax to delete the rows also to redirect to the homepage (To avoid page load errors) :
def delete_all(self):
Reporter.objects.all().delete()
return HttpResponseRedirect('/')
You can use the Django-Truncate library to delete all data of a table without destroying the table structure.
Example:
First, install django-turncate using your terminal/command line:
pip install django-truncate
Add "django_truncate" to your INSTALLED_APPS in the settings.py file:
INSTALLED_APPS = [
...
'django_truncate',
]
Use this command in your terminal to delete all data of the table from the app.
python manage.py truncate --apps app_name --models table_name
There are a couple of ways:
To delete it directly:
SomeModel.objects.filter(id=id).delete()
To delete it from an instance:
instance1 = SomeModel.objects.get(id=id)
instance1.delete()
// don't use same name
Actually, I un-register the model (the table data that I want to delete) from the admin.py. Then I migrate.
python manage.py makemigrations
python manage.py migrate
python runserver
Then I register the model in the admin.py and do migration again. :) Now, the table is empty. This might not be a professional answer, but it helped me.
I'd like to have these lines of code executed on server startup (both development and production):
from django.core import management
management.call_command('syncdb', interactive=False)
Putting it in settings.py doesn't work, as it requires the settings to be loaded already.
Putting them in a view and accessing that view externally doesn't work either, as there are some middlewares that use the database and those will fail and not let me access the view.
Putting them in a middleware would work, but that would get called each time my app is accessed. An possible solution might be to create a middleware that does all the job and then removes itself from MIDDLEWARE_CLASSES so it's not called anymore. Can I do that without too much monkey-patching?
Write middleware that does this in __init__ and afterwards raise django.core.exceptions.MiddlewareNotUsed from the __init__, django will remove it for all requests :). __init__ is called at startup by the way, not at the first request, so it won't block your first user.
There is talk about adding a startup signal, but that won't be available soon (a major problem for example is when this signal should be sent)
Related Ticket: https://code.djangoproject.com/ticket/13024
Update: Django 1.7 includes support for this. (Documentation, as linked by the ticket)
In Django 1.7+ if you want to run a startup code and,
1. Avoid running it in migrate, makemigrations, shell sessions, ...
2. Avoid running it twice or more
A solution would be:
file: myapp/apps.py
from django.apps import AppConfig
def startup():
# startup code goes here
class MyAppConfig(AppConfig):
name = 'myapp'
verbose_name = "My Application"
def ready(self):
import os
if os.environ.get('RUN_MAIN'):
startup()
file: myapp/__init__.py
default_app_config = 'myapp.apps.MyAppConfig'
This post is using suggestions from #Pykler and #bdoering
If you were using Apache/mod_wsgi for both, use the WSGI script file described in:
http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
Add what you need after language translations are activated.
Thus:
import sys
sys.path.insert(0, '/usr/local/django/mysite')
import settings
import django.core.management
django.core.management.setup_environ(settings)
utility = django.core.management.ManagementUtility()
command = utility.fetch_command('runserver')
command.validate()
import django.conf
import django.utils
django.utils.translation.activate(django.conf.settings.LANGUAGE_CODE)
# Your line here.
django.core.management.call_command('syncdb', interactive=False)
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
You can create a custom command and write your code in the handle function. details here https://docs.djangoproject.com/en/dev/howto/custom-management-commands/
Then you can create a startup script that runs the django server then executes your new custom command.
If you are using mod_wsgi you can put it in the wsgi start app
Here is how I work around the missing startup signal for Django:
https://github.com/lsaffre/djangosite/blob/master/djangosite/models.py
The code that is being called there is specific to my djangosite project, but the trick to get it called by writing a special app (based on an idea by Ross McFarland) should work for other environments.
Luc
I have a Django app that requires a settings attribute in the form of:
RELATED_MODELS = ('appname1.modelname1.attribute1',
'appname1.modelname2.attribute2',
'appname2.modelname3.attribute3', ...)
Then hooks their post_save signal to update some other fixed model depending on the attributeN defined.
I would like to test this behaviour and tests should work even if this app is the only one in the project (except for its own dependencies, no other wrapper app need to be installed). How can I create and attach/register/activate mock models just for the test database? (or is it possible at all?)
Solutions that allow me to use test fixtures would be great.
You can put your tests in a tests/ subdirectory of the app (rather than a tests.py file), and include a tests/models.py with the test-only models.
Then provide a test-running script (example) that includes your tests/ "app" in INSTALLED_APPS. (This doesn't work when running app tests from a real project, which won't have the tests app in INSTALLED_APPS, but I rarely find it useful to run reusable app tests from a project, and Django 1.6+ doesn't by default.)
(NOTE: The alternative dynamic method described below only works in Django 1.1+ if your test case subclasses TransactionTestCase - which slows down your tests significantly - and no longer works at all in Django 1.7+. It's left here only for historical interest; don't use it.)
At the beginning of your tests (i.e. in a setUp method, or at the beginning of a set of doctests), you can dynamically add "myapp.tests" to the INSTALLED_APPS setting, and then do this:
from django.core.management import call_command
from django.db.models import loading
loading.cache.loaded = False
call_command('syncdb', verbosity=0)
Then at the end of your tests, you should clean up by restoring the old version of INSTALLED_APPS and clearing the app cache again.
This class encapsulates the pattern so it doesn't clutter up your test code quite as much.
#paluh's answer requires adding unwanted code to a non-test file and in my experience, #carl's solution does not work with django.test.TestCase which is needed to use fixtures. If you want to use django.test.TestCase, you need to make sure you call syncdb before the fixtures get loaded. This requires overriding the _pre_setup method (putting the code in the setUp method is not sufficient). I use my own version of TestCase that lets me add apps with test models. It is defined as follows:
from django.conf import settings
from django.core.management import call_command
from django.db.models import loading
from django import test
class TestCase(test.TestCase):
apps = ()
def _pre_setup(self):
# Add the models to the db.
self._original_installed_apps = list(settings.INSTALLED_APPS)
for app in self.apps:
settings.INSTALLED_APPS.append(app)
loading.cache.loaded = False
call_command('syncdb', interactive=False, verbosity=0)
# Call the original method that does the fixtures etc.
super(TestCase, self)._pre_setup()
def _post_teardown(self):
# Call the original method.
super(TestCase, self)._post_teardown()
# Restore the settings.
settings.INSTALLED_APPS = self._original_installed_apps
loading.cache.loaded = False
I shared my solution that I use in my projects. Maybe it helps someone.
pip install django-fake-model
Two simple steps to create fake model:
1) Define model in any file (I usualy define model in test file near a test case)
from django_fake_model import models as f
class MyFakeModel(f.FakeModel):
name = models.CharField(max_length=100)
2) Add decorator #MyFakeModel.fake_me to your TestCase or to test function.
class MyTest(TestCase):
#MyFakeModel.fake_me
def test_create_model(self):
MyFakeModel.objects.create(name='123')
model = MyFakeModel.objects.get(name='123')
self.assertEqual(model.name, '123')
This decorator creates table in your database before each test and remove the table after test.
Also you may create/delete table manually: MyFakeModel.create_table() / MyFakeModel.delete_table()
I've figured out a way for test-only models for django 1.7+.
The basic idea is, make your tests an app, and add your tests to INSTALLED_APPS.
Here's an example:
$ ls common
__init__.py admin.py apps.py fixtures models.py pagination.py tests validators.py views.py
$ ls common/tests
__init__.py apps.py models.py serializers.py test_filter.py test_pagination.py test_validators.py views.py
And I have different settings for different purposes(ref: splitting up the settings file), namely:
settings/default.py: base settings file
settings/production.py: for production
settings/development.py: for development
settings/testing.py: for testing.
And in settings/testing.py, you can modify INSTALLED_APPS:
settings/testing.py:
from default import *
DEBUG = True
INSTALLED_APPS += ['common', 'common.tests']
And make sure that you have set a proper label for your tests app, namely,
common/tests/apps.py
from django.apps import AppConfig
class CommonTestsConfig(AppConfig):
name = 'common.tests'
label = 'common_tests'
common/tests/__init__.py, set up proper AppConfig(ref: Django Applications).
default_app_config = 'common.tests.apps.CommonTestsConfig'
Then, generate db migration by
python manage.py makemigrations --settings=<your_project_name>.settings.testing tests
Finally, you can run your test with param --settings=<your_project_name>.settings.testing.
If you use py.test, you can even drop a pytest.ini file along with django's manage.py.
py.test
[pytest]
DJANGO_SETTINGS_MODULE=kungfu.settings.testing
Quoting from a related answer:
If you want models defined for testing only then you should check out
Django ticket #7835 in particular comment #24 part of which
is given below:
Apparently you can simply define models directly in your tests.py.
Syncdb never imports tests.py, so those models won't get synced to the
normal db, but they will get synced to the test database, and can be
used in tests.
This solution works only for earlier versions of django (before 1.7). You can check your version easily:
import django
django.VERSION < (1, 7)
Original response:
It's quite strange but form me works very simple pattern:
add tests.py to app which you are going to test,
in this file just define testing models,
below put your testing code (doctest or TestCase definition),
Below I've put some code which defines Article model which is needed only for tests (it exists in someapp/tests.py and I can test it just with: ./manage.py test someapp ):
class Article(models.Model):
title = models.CharField(max_length=128)
description = models.TextField()
document = DocumentTextField(template=lambda i: i.description)
def __unicode__(self):
return self.title
__test__ = {"doctest": """
#smuggling model for tests
>>> from .tests import Article
#testing data
>>> by_two = Article.objects.create(title="divisible by two", description="two four six eight")
>>> by_three = Article.objects.create(title="divisible by three", description="three six nine")
>>> by_four = Article.objects.create(title="divisible by four", description="four four eight")
>>> Article.objects.all().search(document='four')
[<Article: divisible by two>, <Article: divisible by four>]
>>> Article.objects.all().search(document='three')
[<Article: divisible by three>]
"""}
Unit tests also working with such model definition.
I chose a slightly different, albeit more coupled, approach to dynamically creating models just for testing.
I keep all my tests in a tests subdirectory that lives in my files app. The models.py file in the tests subdirectory contains my test-only models. The coupled part comes in here, where I need to add the following to my settings.py file:
# check if we are testing right now
TESTING = 'test' in sys.argv
if TESTING:
# add test packages that have models
INSTALLED_APPS += ['files.tests',]
I also set db_table in my test model, because otherwise Django would have created the table with the name tests_<model_name>, which may have caused a conflict with other test models in another app. Here's my my test model:
class Recipe(models.Model):
'''Test-only model to test out thumbnail registration.'''
dish_image = models.ImageField(upload_to='recipes/')
class Meta:
db_table = 'files_tests_recipe'
Here's the pattern that I'm using to do this.
I've written this method that I use on a subclassed version of TestCase. It goes as follows:
#classmethod
def create_models_from_app(cls, app_name):
"""
Manually create Models (used only for testing) from the specified string app name.
Models are loaded from the module "<app_name>.models"
"""
from django.db import connection, DatabaseError
from django.db.models.loading import load_app
app = load_app(app_name)
from django.core.management import sql
from django.core.management.color import no_style
sql = sql.sql_create(app, no_style(), connection)
cursor = connection.cursor()
for statement in sql:
try:
cursor.execute(statement)
except DatabaseError, excn:
logger.debug(excn.message)
pass
Then, I create a special test-specific models.py file in something like myapp/tests/models.py that's not included in INSTALLED_APPS.
In my setUp method, I call create_models_from_app('myapp.tests') and it creates the proper tables.
The only "gotcha" with this approach is that you don't really want to create the models ever time setUp runs, which is why I catch DatabaseError. I guess the call to this method could go at the top of the test file and that would work a little better.
Combining your answers, specially #slacy's, I did this:
class TestCase(test.TestCase):
initiated = False
#classmethod
def setUpClass(cls, *args, **kwargs):
if not TestCase.initiated:
TestCase.create_models_from_app('myapp.tests')
TestCase.initiated = True
super(TestCase, cls).setUpClass(*args, **kwargs)
#classmethod
def create_models_from_app(cls, app_name):
"""
Manually create Models (used only for testing) from the specified string app name.
Models are loaded from the module "<app_name>.models"
"""
from django.db import connection, DatabaseError
from django.db.models.loading import load_app
app = load_app(app_name)
from django.core.management import sql
from django.core.management.color import no_style
sql = sql.sql_create(app, no_style(), connection)
cursor = connection.cursor()
for statement in sql:
try:
cursor.execute(statement)
except DatabaseError, excn:
logger.debug(excn.message)
With this, you don't try to create db tables more than once, and you don't need to change your INSTALLED_APPS.
If you are writing a reusable django-app, create a minimal test-dedicated app for it!
$ django-admin.py startproject test_myapp_project
$ django-admin.py startapp test_myapp
add both myapp and test_myapp to the INSTALLED_APPS, create your models there and it's good to go!
I have gone through all these answers as well as django ticket 7835, and I finally went for a totally different approach.
I wanted my app (somehow extending queryset.values() ) to be able to be tested in isolation; also, my package does include some models and I wanted a clean distinction between test models and package ones.
That's when I realized it was easier to add a very small django project in the package!
This also allows a much cleaner separation of code IMHO:
In there you can cleanly and without any hack define your models, and you know they will be created when you run your tests from in there!
If you are not writing an independent, reusable app you can still go this way: create a test_myapp app, and add it to your INSTALLED_APPS only in a separate settings_test_myapp.py!
Someone already mentioned Django ticket #7835, but there appears to be a more recent reply that looks much more promising for more recent versions of Django. Specifically #42, which proposes a different TestRunner:
from importlib.util import find_spec
import unittest
from django.apps import apps
from django.conf import settings
from django.test.runner import DiscoverRunner
class TestLoader(unittest.TestLoader):
""" Loader that reports all successful loads to a runner """
def __init__(self, *args, runner, **kwargs):
self.runner = runner
super().__init__(*args, **kwargs)
def loadTestsFromModule(self, module, pattern=None):
suite = super().loadTestsFromModule(module, pattern)
if suite.countTestCases():
self.runner.register_test_module(module)
return suite
class RunnerWithTestModels(DiscoverRunner):
""" Test Runner that will add any test packages with a 'models' module to INSTALLED_APPS.
Allows test only models to be defined within any package that contains tests.
All test models should be set with app_label = 'tests'
"""
def __init__(self, *args, **kwargs):
self.test_packages = set()
self.test_loader = TestLoader(runner=self)
super().__init__(*args, **kwargs)
def register_test_module(self, module):
self.test_packages.add(module.__package__)
def setup_databases(self, **kwargs):
# Look for test models
test_apps = set()
for package in self.test_packages:
if find_spec('.models', package):
test_apps.add(package)
# Add test apps with models to INSTALLED_APPS that aren't already there
new_installed = settings.INSTALLED_APPS + tuple(ta for ta in test_apps if ta not in settings.INSTALLED_APPS)
apps.set_installed_apps(new_installed)
return super().setup_databases(**kwargs)