Django SECRET_KEY setting must not be empty with github workflow - python

I have a GitHub workflow for Django and when it gets to migrating the database it gives the error
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
the secret key is stored in a .env file and loaded with
from dotenv import load_dotenv
load_dotenv()
from pathlib import Path
env_path = Path('.') / '.env'
load_dotenv(dotenv_path=env_path)
SECRET_KEY = os.getenv("secret_key")
Here is the file tree
C:.
| db.sqlite3
| manage.py
|
\---djangosite
| .env
| asgi.py
| settings.py
| urls.py
| wsgi.py
| __init__.py
|
\---__pycache__
...
This is the manage.py, it is the regular django one with the load .env code from settings.py
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
from dotenv import load_dotenv
load_dotenv()
from pathlib import Path
env_path = Path('.') / '.env'
load_dotenv(dotenv_path=env_path)
import os
import sys
def main():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'djangosite.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if __name__ == '__main__':
main()
when I run manage.py on my PC it loads the key and runs the server, but GitHub gives the error above.
How do I stop this error from happening?


If you have stored the SECRET_KEY in your system's environment variable, then for GitHub workflow, you can add a dummy environment variable in the YAML file.
The settings.py should look like this
import os
...
SECRET_KEY = os.environ.get('SECRET_KEY') # Or the name by which you stored environment variable
...
The steps are given below:
Step 1: Generate a dummy SECRET_KEY. You can create it yourself by
import secrets
print(secrets.token_hex(25))
Or generate from a site like this.
Step 2: In your .github/workflows YAML file (e.g., django.yml), add this
steps:
...
- name: Run Tests
env:
SECRET_KEY: your-genereated-secret_key
run: |
python manage.py test
Then everything will work fine with the same version of code in your local environment, production environment, and GitHub workflow.


Adding to #PhysicistSouravDas's answer. You can alternatively do:
- name: Run Tests
env:
SECRET_KEY: ${{ secrets.SECRET_KEY }}
run: |
python manage.py test
Now, go to the settings of your GitHub repository. Under the secrets menu, click Actions and then click New Repository Secret.
Add a new secret with the name SECRET_KEY and value as the dummy SECRET_KEY generated by the method suggested by #PhysicistSouravDas.
GitHub Actions would pick up the SECRET_KEY from there.


When you run python manage.py runserver 8000 you are using manage.py which is settings your DJANGO_SETTINGS_MODULE to settings. It doesn't appear that you have a settings.py in your root directory, so this line in manage.py:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
should become:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Your-App-Name.settings")
as you have in your wsgi.py file.
You can also check here for some other possible solutions.

Related

Heroku application error on Celery: raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")

I am trying to deploy my Django app on Heroku. Whenever I go to my app I am getting an error:
2020-09-06T20:31:59.000436+00:00 app[web.1]: raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
2020-09-06T20:31:59.000477+00:00 app[web.1]: django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty
I have now two settings file so in my projects folder which contains the wsgi.py. I have a folder called settings/ which is:
settings
--common.py
--development.py
--productions.py
In my production.py I have:
from register.settings.common import *
import os
import django_heroku
DEBUG = False
ALLOWED_HOSTS = ['0.0.0.0', 'localhost', '127.0.0.1','appname.heroku.com']
SECRET_KEY = os.environ.get('SECRET_KEY')
EMAIL_HOST_USER = os.environ.get('EMAIL_HOST_USER')
EMAIL_HOST_PASSWORD = os.environ.get('EMAIL_HOST_PASSWORD')
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
django_heroku.settings(locals())
And in my wsgi.py and manage.py I have:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.production')
Now I have added the Postgres add-on from the dashboard and can check that I also have it using the Heroku CLI.
I have also added a custom domain name of www.myapp.com as well which I do not know if this is causing the issue.
For pushing to master I did:
heroku config:set DISABLE_COLLECTSTATIC=1
and then:
git push heroku master
What is causing this issue? When I do python manage.py run server I also get the error:
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
I also get this error in my Heroku logs when I check it how can I set the secret key?
EDIT 2
The issue seems to be with my celery.py file, however, I have tried changing the settings to point to production.py (register in my projects root folder):
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
import django
app = Celery('register')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks(settings.INSTALLED_APPS)
in my manage.py I have:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.development') Which the command manage.py runserver works however, now I get an error:
8:52:21 PM web.1 | raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
8:52:21 PM web.1 | django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
in this line in my wsgi.py file:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'register.settings.production')
application = get_wsgi_application()

It looks like you simply haven't set your SECRET_KEY.
You try to import it from the environment in your settings module, but this depends on an environment variable called SECRET_KEY existing:
SECRET_KEY = os.environ.get('SECRET_KEY')
On Heroku, the best way to set this environment varialbe is to set a config var. This can be done with the Heroku CLI:
heroku config:set SECRET_KEY=some_value
or in the web dashboard.

Django app doesn't recognize .env file, I tried to set it up through the os, Django-environ, and dotenv but nothing worked

I set up a new django app in a virtual environment and tried to use a .env file with it for the secret key and the database url, however I can't manage to make the django app read the .env file.
I've tried first with os.environ, tried to set a direct path, tried with dotenv (both python-dotnev and django-dotnev, seperately) and last with Django-environ. I tried putting the .env file is in the same folder, the parent folder, the venv folder, but nothing worked.
In myapp/settings.py I have:
import os
import environ
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
env = environ.Env()
environ.Env.read_env()
SECRET_KEY = env('SECRET_KEY')
DEBUG = env('DEBUG')
ALLOWED_HOSTS = []
And then in the same folder I have ".env" file:
Debug=True
SECRET_KEY='some-secret-key'
I tried adding "export" before Debug and SECRET_KEY but it didn't make any difference. I constantly get this error when I run python manage.py runserver:
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable
What am I missing? What step did I not do? Does it not work because the app runs in a virtual envirnoment?

How to properly runserver on different settings for Django?

I have a basic django rest API. I want to separate some of my settings for dev and prod just for organizational purposes. I'm also just learning about separating environments. I've read a few things, but I can't seem to get it working the way I want it to.
The hierarchy looks like this:
- djangorest
- api
- __init__.py
- models.py
- views.py
- urls.py
- etc..
- djangorest
- __init__.py
- settings.py
- urls.py
- wsgi.py
Right now, when I run the server, I do a simple:
python3 manage.py runserver
That command reads the settings from settings.py and runs it appropriately but I've looked around on how to separate the settings into prod vs dev and it doesn't work right.
I want to be able to have:
commonsettings.py
dev.py
prod.py
In commonsettings, it'll just have whatever's in both dev and prod. I've tried running:
python3 manage.py runserver --settings=settings.dev
But it gives me an error saying there is no module named 'setting'.
Please help. Thank you!

For example I open SQL log in dev settings
My file structure:
create settings folder and mv original settings.py to settings/defaults.py
load defaults in init.py
# proj/proj/settings/__init__.py
from .defaults import *
Edit dev.py
# proj/proj/settings/dev.py
from .defaults import *
DEBUG = True
# print sql to the console
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
}
},
'loggers': {
'django.db.backends': {
'handlers': ['console'],
'propagate': True,
'level': 'DEBUG',
}
},
}
run in the python env
./manage.py runserver --settings proj.settings.dev
Django 2.1.7 | Mac

Create a folder called config
config/
commonsettings.py
dev.py
prod.py
make sure that in dev.py and prod.py you import everything from commonsettings.py like this:
from .commonsettings import *
then if you want to run the dev.py settings:
python manage.py runserver --settings=config.dev
if you want to run prod.py:
python manage.py runserver --settings=config.prod
NOTE:
For more readable files many developers call their settings files: local.py (for the local settings) production.py (for the production settings) and base.py (the common settings for both)
Personally I place my setting files in:
config/
settings/
base.py
local.py
production.py
test.py (For tests)

This solution works without extra arguments, and without having to replace/delete files across environments.
Make settings.py decide which file to load
Let's assume your Django project is named MyDjangoApp.
Create the config/ dir in your Django project's root dir, and one .py file per environment, e.g. like this:
config/
local.py
dev.py
prod.py
MyDjangoApp/
settings.py
There is no logical limit to the number of environments, I just added the three I usually have.
Then, inside MyDjangoApp/settings.py we can add the logic to choose which settings file to load.
"""
Django settings for MyDjangoApp project.
"""
import os
# Begin: Custom per-env settings
import socket
# Use the appropriate logic to understand "where" we're running
HOST = socket.gethostname()
configs = {
'localhost': 'local', # replace 'localhost' with whatever your machine name is
'dev.mysite.com': 'dev',
'www.mysite.com': 'production',
}
config = 'config.{}'.format(configs[HOST])
settings_module = __import__(config, globals(), locals(), ['MyDjangoApp', ])
try:
for setting in dir(settings_module):
# Only fully-uppercase variables are supposed to be settings
if setting == setting.upper():
locals()[setting] = getattr(settings_module, setting)
except Exception:
# you may ignore this exception, the print() is for debugging purposes
print(Exception)
# End: Custom per-env settings
# ... the rest of the common settings go here
Please note: this logic chooses the settings file to load based on the hostname of the server which is running the Django app. It's just an example- you can implement any other logic that works for you.
Change only what's needed
The good news is, settings.py can still hold the vast majority of your settings, and the files inside config/ just need to contain the settings that change across your environments, e.g.:
DATABASES
DEBUG
ADMINS
LOGGING
Avoid --settings in the commandline
You can set the Django settings as a shell environment variable, like this:
export DJANGO_SETTINGS_MODULE=MyDjangoApp.settings
The best place to do this is in a .sh file; usually I create a file named env.sh file in the root dir of the Django project, and run it (once) just after activating the virtualenv.
File /env.sh contents
#!/usr/bin/env bash
export DJANGO_SETTINGS_MODULE=MyDjangoApp.settings
To run it from Terminal (Bash, or git-scm Bash on Windows):
. env.sh
Then you can use ./manage.py runserver to start the Django server wherever you are.
Reference: https://code.djangoproject.com/wiki/SplitSettings#DevelopmentMachineDependantSettingsConfiguration

Put all your common configurations in the commonsettings.py file.
add following the line into the __init__.py file in the settings directory
from commonsettings import *
This makes every configuration in the commonsettings file available in all other files in the settings folder
start your server with
python3 manage.py runserver --settings=djangorest.settings.dev

Project Hierarchy will be
- djangorest
- api
- __init__.py
- models.py
- views.py
- urls.py
- etc..
- djangorest
- settings (folder)
- commonsettings.py
- dev.py
- prod.py
- __init__.py
- settings.py
- urls.py
- wsgi.py
After This run following commands before python manage.py runserver
export PYTHONPATH=$PWD
export DJANGO_SETTINGS_MODULE=djangorest.settings.dev
and Finally python manage.py runserver

Replace settings.py with dev.py on dev server and prod.py on production server.
now in init.py files write code
try:
print("Trying import production.py settings...")
from .prod import *
except ImportError:
print("Trying import development.py settings...")
from .dev import *
it find prod.py in production server and import it. On dev server it not find and prod.py and import dev.py
then you don't need to specify configuration file name while running runserver command.

What I've always done is on the local env create a file called local_settings.py that can have it's own database settings, static path and so on and at the end of the settings file include this
try:
from local_settings import *#
except:
pass
so on the server I always make sure not to import the local_settings and it separates the two nicely.

Local version of Heroku-Django app does not see vars in .env

I cloned my Heroku-Django project to a new Windows computer. I don't have the local_settings.py file on this computer, so I am trying to set it up to use local environment variables.
I used heroku config:get SECRET_KEY -s >> .env to put the config var into my .env file. That appears to have worked. The .env file now reads SECRET_KEY='mysecretkey'. The actual key is correct.
Because I am on Windows, I created a separate Procfile called Procfile.windows and it reads: web: python manage.py runserver 0.0.0.0:8000
In my production_settings file, I have this line: SECRET_KEY = os.environ.get('SECRET_KEY'). OS is imported.
When I run heroku local -f Procfile.windows, I ultimately get this error: django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty. However, it does find the .env file: [OKAY] Loaded ENV .env File as KEY=VALUE Format.
Am I missing something?
EDIT: This is what my settings module looks like:
__ init.py __:
from .base_settings import *
from .production_settings import *
try:
from .local_settings import *
except:
pass
base_settings.py:
import os, dj_database_url
# All the basic Dango/Heroku settings.
# SECRET_KEY is not defined here as I only define it in local_settings.py and production_settings.py.
production_settings.py:
SECRET_KEY = os.environ.get('SECRET_KEY')
DEBUG = False
ALLOWED_HOSTS = ['.herokuapp.com', 'localhost']

Django. Using multiple settings files with Heroku

I am trying to follow the advice of the book "Two Scoops of Django" and although it is a really good book, I think it this section is unclear.
So, I split my settings file and created a folder like this:
settings/
__init__.py
base.py (allmost everything there)
local.py (dev. specific settings)
production.py (settings for Heroku)
most of the settings are in the base.py file
in local.py I have this:
# settings/local.py
from .base import *
DEBUG = True
TEMPLATE_DEBUG = DEBUG
INSTALLED_APPS += ("debug_toolbar", "django_extensions", "south",)
in production.py I have this:
from .base import *
INSTALLED_APPS += ("gunicorn",)
When I run locally:
python manage.py runserver 7000 --settings=appname.settings.local
python manage.py runserver 7000 --settings=appname.settings.production
everything works fine.
But when I push changes to Heroku, I get the log:
File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/util.py", line 354, in import_app
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
So, I guess Heroku is not finding my settings files, I don't know how to fix this (it might be very simple).
Two Scoops of Django is kind of ironic here, it writes "Platform as Service - See section 25.2" and then in that section it just writes "read Platform Documentation" : /

After you have logged into heroku with heroku login you can check your configs by running:
heroku config.
If you dont see a SECRET_KEY and DJANGO_SETTINGS_MODULE you can set them by running:
heroku config:set SECRET_KEY='secret_key_goes_here'
and
heroku config:set DJANGO_SETTINGS_MODULE=mysite.settings.production
Finally, make sure that you have the following syntax inside of your production setting file:
SECRET_KEY = os.environ['SECRET_KEY']
The above intstructions are for the following project structure
-myproject
-app1
-app2
-mysite
-settings
__init__.py
base.py
dev.py
production.py
-manage.py
-Pipfile
-Procfile
-requirements.txt

You can use the environment variable DJANGO_SETTINGS_MODULE to specify a default settings module:
https://docs.djangoproject.com/en/dev/topics/settings/#envvar-DJANGO_SETTINGS_MODULE
On local Linux machine:
export DJANGO_SETTINGS_MODULE=settings.local
On Heroku:
heroku config:set DJANGO_SETTINGS_MODULE=settings.production

Categories

Resources