Develop Reference

Can't apply migrations in Django SQLlite memory database

I'm writing some tests, and I want to be able to run django server with my test settings (that's why I'm using in-memory database).
It seems to be working, no errors reported when running. But migrations are not applied - I can't perform any action on the database, because model tables do not exist.
When I run python manage.py migrate, all my migrations get applied (I see these Applying migrations... OK messages), but it has no effect. When I run python manage.py showmigrations, none of the migrations are applied (I see [ ] 0001_initial etc., without the X).
When I go to django shell, I can't perform any action, because table does not exist. Any idea what might be the reason? It works fine with normal, postgres database.
My settings:
DEBUG = True
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': ':memory:',
'TEST_NAME': ':memory:',
},
}
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
'LOCATION': ''
}
}

Django Error 500 related to staticfiles when DEBUG = False

Here is my settings.py:
import os
import django_heroku
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SECRET_KEY = 'secret_key'
DEBUG = False
ALLOWED_HOSTS = ['*']
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'corsheaders',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'myapp',
'rest_framework',
'django_celery_beat'
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'corsheaders.middleware.CorsMiddleware', # CORS support [keep load position]
'django.middleware.common.CommonMiddleware',
#'django.middleware.cache.UpdateCacheMiddleware',
#'django.middleware.common.CommonMiddleware',
#'django.middleware.cache.FetchFromCacheMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'corsheaders.middleware.CorsPostCsrfMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
CORS_ORIGIN_WHITELIST = [
"http://localhost:8080",
"http://127.0.0.1:8080",
"http://localhost:80"
]
CSRF_TRUSTED_ORIGINS = [
'http://localhost:8080',
]
ROOT_URLCONF = 'django_backend.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'django_backend.wsgi.application'
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'name',
'USER': 'user',
'PASSWORD': "pw",
'HOST': 'localhost',
'PORT': '5432'
}
}
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
CELERY_BROKER_URL = 'redis://url:port'
CELERY_RESULT_BACKEND = 'redis://url:port'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = TIME_ZONE
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'console':{
'level': 'DEBUG',
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
DEBUG_PROPAGATE_EXCEPTIONS = True
STATICFILES_STORAGE = 'whitenoise.storage.CompressedStaticFilesStorage'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'
django_heroku.settings(locals())
On my local server, I do not receive any errors. However in production, I get asset errors on any page that loads a static asset. For example trying to access the Django admin panel:
ValueError: Missing staticfiles manifest
entry for 'admin/css/base.css'
I have tried running python manage.py collectstatic which does not result in an error. It says 152 files collected in /staticfiles (confirmed they exist). Why is Django not able to find these static files? Another SO thread said to try setting ALLOWED_HOSTS to *, which I have as well. Other solutions such as trying the default static files settings configuration don't seem to work either.

Django does not serve static files when the server is running in production (so with DEBUG = False).
As is specified in the documentation on Configuring static files:
In addition to these configuration steps, you’ll also need to
actually serve the static files.
During development, if you use django.contrib.staticfiles, this will be done automatically by runserver when DEBUG is set to
True (see django.contrib.staticfiles.views.serve()).
This method is grossly inefficient and probably insecure, so it is
unsuitable for production.
See Deploying static files for proper strategies to serve static files
in production environments.
If you thus want to run your server in production, you will need to set up the nginx/apache/... configuration to serve static files. collectstatic is used to collect the static files over the static directories, such that you can easily configure your server.
The documentation on serving static files in production has a section on how to configure an Apache server to serve static files. In most cases, one does not run Django and serves static files on the same server. Usually there are two servers that each handle one of the cases, or one uses a CDN or S3 storage.
As an alterantive, you can use the whitenoise package, it manages to improve certain aspects Django's static files will not, although it still might not be the best option to serve the static files through the same server as the Django server:
Isn’t serving static files from Python horribly inefficient?
The short answer to this is that if you care about performance and
efficiency then you should be using WhiteNoise behind a CDN like
CloudFront. If you’re doing that then, because of the caching headers
WhiteNoise sends, the vast majority of static requests will be served
directly by the CDN without touching your application, so it really
doesn’t make much difference how efficient WhiteNoise is.
That said, WhiteNoise is pretty efficient. Because it only has to
serve a fixed set of files it does all the work of finding files and
determining the correct headers upfront on initialization. Requests
can then be served with little more than a dictionary lookup to find
the appropriate response. Also, when used with gunicorn (and most
other WSGI servers) the actual business of pushing the file down the
network interface is handled by the kernel’s very efficient sendfile
syscall, not by Python.

django-rosetta error: You can't use the CacheRosettaStorage

I'm using django-rosetta app, it works on development without a CACHES setting, but on prod I've this setting as follow:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
The problem is that under prod it raises me
django.core.exceptions.ImproperlyConfigured:
You can't use the CacheRosettaStorage if your cache isn't correctly set up,
please double check your Django DATABASES setting and that the cache server is responding
The database setting is simple as
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}

So, as the exception message sais:
double check your Django DATABASES setting and that the cache server is responding
I did it, even my memchached was working correctly I decided reinstall it, and, as magic art, it worked!
Before that I changed my CACHES
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
},
'rosetta': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': '/var/tmp/django_cache',
}
}
Django rosetta will use a key rosetta if a cache with this name exists, or default if not. With the FileBasedCache it wasn't launch any error, so I realized the problem was with MemcachedCache. But, after reinstall it, it worked.

How to make two django projects share the same database

I need to make two separate Django projects share the same database. In project_1 I have models creating objects that I need to use in project_2 (mostly images).
The tree structure of project_1_2 is:
project_1/
manage.py
settings.py
project_1_app1/
...
...
project_2/
manage.py
settings.py
project_2_app1/
...
...
Which is the best approach?
EDIT: I'm using sqlite3 in my development environment.
I'd like to keep my two django projects as stand-alone projects (so that both can be upgraded safely from their respective repositories).
# in project_1/settings.py
import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
..
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(PROJECT_ROOT, 'development.db'),
},
}
...
# in project_2/settings.py
import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
..
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(PROJECT_ROOT, 'development.db'),
},
}
...
In this way, each project has its own development.db (the one that I need to be shared):
project_1/development.db
project_2/development.db
but I guess I need to do something more to make it shared (and unique).
The best for me would be to keep the development.db at project_1/ path and thus set the project_2/settings.py DATABASES to point to project_1/development.db.

You can simply define the same database in DATABASES in your settings.py. So, if your database is PostgreSQL, you could do something like this:
# in project_1/settings.py
DATABASES = {
'default': {
'NAME': 'common_db',
'ENGINE': 'django.db.backends.postgresql',
'USER': 'project_1_user',
'PASSWORD': 'strong_password_1'
},
}
# in project_2/settings.py
DATABASES = {
'default': {
'NAME': 'common_db',
'ENGINE': 'django.db.backends.postgresql',
'USER': 'project_2_user',
'PASSWORD': 'strong_password_2'
},
}
Note that both database users (project_1_user and project_2_user) should have the appropriate privileges on the database you wish to use. Or you could instead use the same user for both projects.
If you want to have more than just one database per project, you should take a look at the docs for multiple databases.
On another matter, since you share data, I guess you share functionalities as well. So for example, if project_1_app1 and project_2_app1 do same (or similar) things, it seems they could instead be a single reusable app.
Edit
Since you use sqlite3, you should ensure the path you use is the same. So, assuming that project_1 and project_2 are siblings, like so:
projects
project_1
settings.py
...
project_2
settings.py
...
you should try this:
# project_1/settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(PROJECT_ROOT, 'development.db'),
},
}
# project_2/settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(
os.path.dirname(os.path.dirname(PROJECT_ROOT)),
'project_1',
'development.db'
),
},
}
This would give the structure you ask for. Note however that the projects are not both "standalone". project_2 is clearly dependent on project_1's database.
In any case, perhaps, you should also take a look at the os.path module for more info.

You just need to declare in your model in class meta the attribute db_table with a name diferent the name of app + model (Which are automatically generated by Django) the twice projects need the same models. before the run makemigrations and migrate.
class MyModel(models.Model):
class Meta:
db_table = 'MyModel'

Django shell doesn't respect cache configuration

In my settings.py I have:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'ws_cache_table',
'TIMEOUT': '3000000',
'OPTIONS': {
'MAX_ENTRIES': 10000000
}
}
}
But if I do this in python manage.py shell:
from django.core.cache import cache
print type(cache)
I'm getting:
django.core.cache.backends.locmem.LocMemCache
Why!???
Now I can't clear my cache...
To prove my configuration is corect I can do:
from django.conf import settings
conf = settings.CACHES.get('default', None)
And I'm getting:
{'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'ws_cache_table',
'OPTIONS': {'MAX_ENTRIES': 10000000},
'TIMEOUT': '3000000'}
It looks like get_cache method is called before CACHES is defined...

First of all you should keep in mind that your local_settings.py would overwrite the settings.py.
Then you should watch out what cache daemon as backend is running, as there are different ones and depending what you run you need the respective specified option.
e.g. for memcached the local_settings.py would read:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211'
'CACHE_TIME': '3600',
}
}
whereas for locmem:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'
'LOCATION': '127.0.0.1:11211'
'TIMEOUT': 3600'
}
}