Develop Reference

Heroku application error on Celery: raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")

I am trying to deploy my Django app on Heroku. Whenever I go to my app I am getting an error:
2020-09-06T20:31:59.000436+00:00 app[web.1]: raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
2020-09-06T20:31:59.000477+00:00 app[web.1]: django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty
I have now two settings file so in my projects folder which contains the wsgi.py. I have a folder called settings/ which is:
settings
--common.py
--development.py
--productions.py
In my production.py I have:
from register.settings.common import *
import os
import django_heroku
DEBUG = False
ALLOWED_HOSTS = ['0.0.0.0', 'localhost', '127.0.0.1','appname.heroku.com']
SECRET_KEY = os.environ.get('SECRET_KEY')
EMAIL_HOST_USER = os.environ.get('EMAIL_HOST_USER')
EMAIL_HOST_PASSWORD = os.environ.get('EMAIL_HOST_PASSWORD')
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
django_heroku.settings(locals())
And in my wsgi.py and manage.py I have:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.production')
Now I have added the Postgres add-on from the dashboard and can check that I also have it using the Heroku CLI.
I have also added a custom domain name of www.myapp.com as well which I do not know if this is causing the issue.
For pushing to master I did:
heroku config:set DISABLE_COLLECTSTATIC=1
and then:
git push heroku master
What is causing this issue? When I do python manage.py run server I also get the error:
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
I also get this error in my Heroku logs when I check it how can I set the secret key?
EDIT 2
The issue seems to be with my celery.py file, however, I have tried changing the settings to point to production.py (register in my projects root folder):
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
import django
app = Celery('register')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks(settings.INSTALLED_APPS)
in my manage.py I have:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.development') Which the command manage.py runserver works however, now I get an error:
8:52:21 PM web.1 | raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
8:52:21 PM web.1 | django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
in this line in my wsgi.py file:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.production')
application = get_wsgi_application()

It looks like you simply haven't set your SECRET_KEY.
You try to import it from the environment in your settings module, but this depends on an environment variable called SECRET_KEY existing:
SECRET_KEY = os.environ.get('SECRET_KEY')
On Heroku, the best way to set this environment varialbe is to set a config var. This can be done with the Heroku CLI:
heroku config:set SECRET_KEY=some_value
or in the web dashboard.

How to properly runserver on different settings for Django?

I have a basic django rest API. I want to separate some of my settings for dev and prod just for organizational purposes. I'm also just learning about separating environments. I've read a few things, but I can't seem to get it working the way I want it to.
The hierarchy looks like this:
- djangorest
- api
- __init__.py
- models.py
- views.py
- urls.py
- etc..
- djangorest
- __init__.py
- settings.py
- urls.py
- wsgi.py
Right now, when I run the server, I do a simple:
python3 manage.py runserver
That command reads the settings from settings.py and runs it appropriately but I've looked around on how to separate the settings into prod vs dev and it doesn't work right.
I want to be able to have:
commonsettings.py
dev.py
prod.py
In commonsettings, it'll just have whatever's in both dev and prod. I've tried running:
python3 manage.py runserver --settings=settings.dev
But it gives me an error saying there is no module named 'setting'.
Please help. Thank you!

For example I open SQL log in dev settings
My file structure:
create settings folder and mv original settings.py to settings/defaults.py
load defaults in init.py
# proj/proj/settings/__init__.py
from .defaults import *
Edit dev.py
# proj/proj/settings/dev.py
from .defaults import *
DEBUG = True
# print sql to the console
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
}
},
'loggers': {
'django.db.backends': {
'handlers': ['console'],
'propagate': True,
'level': 'DEBUG',
}
},
}
run in the python env
./manage.py runserver --settings proj.settings.dev
Django 2.1.7 | Mac

Create a folder called config
config/
commonsettings.py
dev.py
prod.py
make sure that in dev.py and prod.py you import everything from commonsettings.py like this:
from .commonsettings import *
then if you want to run the dev.py settings:
python manage.py runserver --settings=config.dev
if you want to run prod.py:
python manage.py runserver --settings=config.prod
NOTE:
For more readable files many developers call their settings files: local.py (for the local settings) production.py (for the production settings) and base.py (the common settings for both)
Personally I place my setting files in:
config/
settings/
base.py
local.py
production.py
test.py (For tests)

This solution works without extra arguments, and without having to replace/delete files across environments.
Make settings.py decide which file to load
Let's assume your Django project is named MyDjangoApp.
Create the config/ dir in your Django project's root dir, and one .py file per environment, e.g. like this:
config/
local.py
dev.py
prod.py
MyDjangoApp/
settings.py
There is no logical limit to the number of environments, I just added the three I usually have.
Then, inside MyDjangoApp/settings.py we can add the logic to choose which settings file to load.
"""
Django settings for MyDjangoApp project.
"""
import os
# Begin: Custom per-env settings
import socket
# Use the appropriate logic to understand "where" we're running
HOST = socket.gethostname()
configs = {
'localhost': 'local', # replace 'localhost' with whatever your machine name is
'dev.mysite.com': 'dev',
'www.mysite.com': 'production',
}
config = 'config.{}'.format(configs[HOST])
settings_module = __import__(config, globals(), locals(), ['MyDjangoApp', ])
try:
for setting in dir(settings_module):
# Only fully-uppercase variables are supposed to be settings
if setting == setting.upper():
locals()[setting] = getattr(settings_module, setting)
except Exception:
# you may ignore this exception, the print() is for debugging purposes
print(Exception)
# End: Custom per-env settings
# ... the rest of the common settings go here
Please note: this logic chooses the settings file to load based on the hostname of the server which is running the Django app. It's just an example- you can implement any other logic that works for you.
Change only what's needed
The good news is, settings.py can still hold the vast majority of your settings, and the files inside config/ just need to contain the settings that change across your environments, e.g.:
DATABASES
DEBUG
ADMINS
LOGGING
Avoid --settings in the commandline
You can set the Django settings as a shell environment variable, like this:
export DJANGO_SETTINGS_MODULE=MyDjangoApp.settings
The best place to do this is in a .sh file; usually I create a file named env.sh file in the root dir of the Django project, and run it (once) just after activating the virtualenv.
File /env.sh contents
#!/usr/bin/env bash
export DJANGO_SETTINGS_MODULE=MyDjangoApp.settings
To run it from Terminal (Bash, or git-scm Bash on Windows):
. env.sh
Then you can use ./manage.py runserver to start the Django server wherever you are.
Reference: https://code.djangoproject.com/wiki/SplitSettings#DevelopmentMachineDependantSettingsConfiguration

Put all your common configurations in the commonsettings.py file.
add following the line into the __init__.py file in the settings directory
from commonsettings import *
This makes every configuration in the commonsettings file available in all other files in the settings folder
start your server with
python3 manage.py runserver --settings=djangorest.settings.dev

Project Hierarchy will be
- djangorest
- api
- __init__.py
- models.py
- views.py
- urls.py
- etc..
- djangorest
- settings (folder)
- commonsettings.py
- dev.py
- prod.py
- __init__.py
- settings.py
- urls.py
- wsgi.py
After This run following commands before python manage.py runserver
export PYTHONPATH=$PWD
export DJANGO_SETTINGS_MODULE=djangorest.settings.dev
and Finally python manage.py runserver

Replace settings.py with dev.py on dev server and prod.py on production server.
now in init.py files write code
try:
print("Trying import production.py settings...")
from .prod import *
except ImportError:
print("Trying import development.py settings...")
from .dev import *
it find prod.py in production server and import it. On dev server it not find and prod.py and import dev.py
then you don't need to specify configuration file name while running runserver command.

What I've always done is on the local env create a file called local_settings.py that can have it's own database settings, static path and so on and at the end of the settings file include this
try:
from local_settings import *#
except:
pass
so on the server I always make sure not to import the local_settings and it separates the two nicely.

Local version of Heroku-Django app does not see vars in .env

I cloned my Heroku-Django project to a new Windows computer. I don't have the local_settings.py file on this computer, so I am trying to set it up to use local environment variables.
I used heroku config:get SECRET_KEY -s >> .env to put the config var into my .env file. That appears to have worked. The .env file now reads SECRET_KEY='mysecretkey'. The actual key is correct.
Because I am on Windows, I created a separate Procfile called Procfile.windows and it reads: web: python manage.py runserver 0.0.0.0:8000
In my production_settings file, I have this line: SECRET_KEY = os.environ.get('SECRET_KEY'). OS is imported.
When I run heroku local -f Procfile.windows, I ultimately get this error: django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty. However, it does find the .env file: [OKAY] Loaded ENV .env File as KEY=VALUE Format.
Am I missing something?
EDIT: This is what my settings module looks like:
__ init.py __:
from .base_settings import *
from .production_settings import *
try:
from .local_settings import *
except:
pass
base_settings.py:
import os, dj_database_url
# All the basic Dango/Heroku settings.
# SECRET_KEY is not defined here as I only define it in local_settings.py and production_settings.py.
production_settings.py:
SECRET_KEY = os.environ.get('SECRET_KEY')
DEBUG = False
ALLOWED_HOSTS = ['.herokuapp.com', 'localhost']

Django app on AWS Elastic Beanstalk - DJANGO_SETTINGS_MODULE

Where should the DJANGO_SETTINGS_MODULE environment variable be set?
Multiple possible locations:
In a config file (.ebextensions) as follows:
option_settings: aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: "app.settings"
In the wsgi.py file before application is loaded
In manage.py (I also see this on different Django projects on the web)
If I don't specify it in wsgi.py, it looks like it can't be found at all. Therefore, I wonder if the environment variables set in .ebextensions are set before wsgi.py is loaded.
Any idea?

Introduce your wsgi.py in a config file in .ebextensions
django_aws_eb.config:
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: "my_site/wsgi.py"
Specifly your DJANGO_SETTING_MODULE in your `wsgi.py' file
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_site.settings")
application = get_wsgi_application()
aws beanstalk first looks for configs in .ebextension to then locate wsgi.py to then locate settings.
more info: Configure Your Django Application for Elastic Beanstalk

Using wsgi.py correctly in Django

When I deploy my Django site I'm a little confused about the wsgi.py file and how locally it is used vs in production with Gunicorn.
Locally I have this file structure.
demo/
__init__.py
settings.py
urls.py
wsgi.py
manage.py
Settings.py has the following setting...WSGI_APPLICATION = 'demo.wsgi.application'
However once I deployed the app running on Gunicorn I get an error saying it could not find the wsgi.py file, in order to get it to work I have to create the file again like this...
demo/
__init__.py
settings.py
urls.py
wsgi.py
manage.py
wsgi.py
This now works but it suggests to me that Gunicorn ignores the WSGI_APPLICATION setting in Djangos setting file demo/settings.py, right? If so, where does Gunicorn get its own reference to the location for the wsgi.py file and what is different locally?
This is my Gunicorn setup incase that matters...
[program:gunicorn_process]
command=gunicorn wsgi:application -c /srv/test/gunicorn.conf.py
directory=/srv/test
user=root
autostart=true
autorestart=true
redirect_stderr=true
gunicorn.conf.py
bind = "127.0.0.1:8001"
workers = 3
worker_class = 'gevent'

You are correct that Gunicorn ignores the WSGI_APPLICATION setting. This setting is only used to specify the wsgi application that the runserver command uses.
Gunicorn does not know anything about Django, but you can specify in which module Gunicorn should look for the application. Right now it looks for an application attribute in the wsgi module:
command=gunicorn wsgi:application ...
To use the file in demo/, you must specify the full module path in the Gunicorn command:
command=gunicorn demo.wsgi:application ...