I'm trying to deploy a Django app on Heroku. As recommended for security, I don't want to commit and push my secret key from the settings.py file, so I put the key in a separate file and imported it into settings.py. I put my file containing the secret key into .gitignore so that it doesn't get committed.
Now the problem is when I'm pushing to Heroku, I get a ModuleNotFoundError - which makes sense because this file was never committed. Is there a workaround or best practice to use secret keys without exposing them?
It seems you noticed what happened when you excluded the file: it's not there and this is exactly how the security is achieved. The obvious problem now is how does the program get access to the data anyway.
On Heroku, this in handled by setting environment variables on your app that you will then read out when initializing your program -- how this is done differs between programming languages.
So either go https://dashboard.heroku.com/apps/<your-app>/settings and set the variables there. Or utilize the command-line tool like this:
heroku config:set KEY=value -a your-app
The next question then is how you handle this in for local development. A simple solution could be to check for the existence of the git-ignored file, and if it exists use it, otherwise read the values from the environment.
Related
Some background first:
i am currently testing a class that sends a GET request with a configurable url, which is built like this
url = f"{os.environ["TARGET_URL"]}/api/etc"
For normal operation, my TARGET_URL environment variable is set at project startup from a .env file and everything works. When testing locally, everything is still fine, tests passes and everyone is happy. My issue arose when I discovered that my Drone CI server failed to complete the project's build because the TARGET_URL environment variable wasn't found.
After some digging I found out that I had the wrong (dumb) idea that environment variables were reset at every project/test startup, and I basically was using my production environment variable all this time (even during tests) because it was set at first project startup.
From this story comes my question: given that environment variables are kept between executions, would storing configurations in them result in a bad practice? Is there an equally convenient alternative (no global objects and access from everywhere in the code) that can be used instead?
Thanks everyone for the quick responses, here's a bit of what-happened-next:
environment variables stay loaded after the first initialization, so I needed a way to test my code after loading only the variables I needed, with values that were expected. This would allow me to keep using environment variables loaded from a .env file and keep building my project remotely, where no .env files are present.
The solution was to add a pytest plugin called pytest-dotenv, which when properly configured would allow me to overwrite every variable in my .env files with a custom variable from another file (.env.test in my case). I filled the .env.test file with all the variables found in the .env file, and assigned empty values to each of them.
This allowed my tests to run ensuring no weird edge cases are missed because something had the wrong value.
example .env file
TARGET_URL="http://my.api.dev
example .env.test file
TARGET_URL=
pytest.ini configuration
[pytest]
env_override_existing_values = 1
env_files =
.env.test
Environment variables stored in config files or .env files is not a bad practice.
However, it is recommended that you use a key vault such as Azure Key Vault or AWS Key Management System for production deployments.
This way you further remove the keys away from your server (if env files) as well as code (if it is in config files)
Heroku lets you externalise Django configuration, i.e. to put API IDs and secret keys into config vars, that can be later accessed as in os.environ.get('MYVAR',3)
It is shown how to do that in this tutorial.
However, is it safe to just hardcode the secret variables in the settings.py if I am using a private Git repository for my project?
It's safe as long as no one can get into your server files or Git repository, but neither of those seems wise to assume, unless you are even more worried about someone getting into your Heroku account.
Let's say I have some code running on a Heroku dyno (such as this autoscaling script), that needs access to the Platform API. To access the API, I have to authenticate using my app's API Key.
What's the right way to do this?
That script I referenced hardcoded the API Key in the script itself.
A better practice generally seems to put secrets in environment variables, which is what Heroku normally recommends. However, they say they say:
Setting the HEROKU_API_KEY environment variable on your machine will
interfere with normal functioning of auth commands from Toolbelt.
Clearly I could store the API key with under a different key name.
What's the right way? I couldn't find this in the documentation, but seems like a common issue.
Yes, storing this token into a config var is the right way to go.
As for HEROKU_API_KEY, this will happen because locally, the toolbelt will look for the environment variable as one solution to try to fetch your token.
This won't impact your production environment (the heroku toolbelt isn't available within dynos).
Locally, you can also set it easily with a tool like python-dotenv, which will allow you to have a local .env file (don't check it into source control, or your token could be corrupted), with all of it's values available as env vars in your dev app.
I am working on a Django based application whose location on my disk is home/user/Documents/project/application. Now this application takes in some values from the user and writes them into a file located in a folder which is under the project directory i.e home/user/Documents/project/folder/file. While running the development server using the command python manage.py runserver everything worked fine, however after deployment the application/views.py which accesses the file via open('folder/path','w') is not able to access it anymore, because by default it looks in var/www folder when deployed via apache2 server using mod_wsgi.
Now, I am not putting the folder into /var/www because it is not a good practise to put any python code there as it might become readable clients which is a major security threat. Please let me know, how can I point the deployed application to read and write to correct file.
The real solution is to install your data files in /srv/data/myapp or some such so that you can give the webserver user correct permissions to only those directories. Whether you choose to put your code in /var/www or not, is a separate question, but I would suggest putting at least your wsgi file there (and, of course, specifying your <DocumentRoot..> correctly.
I am using Django, python, virtualenv, virtualenvwrapper and Vagrant.
So far I have simply left my secret_key inside of the settings.py file.
This works file for local files. However I have already placed my files in Git. I know this is not acceptable for production(Apache).
What is the correct way to go about hiding my secret_key?
Should I use virtualenv to hide it?
There's a lot of different methods to hide secrets.
Use another, non-versioned file.
Create a new file secrets.py or what have you and put your secrets in that. Place it alongside your settings file and place everything secret in there; then in your settings file put from secrets import * at the top. Then, like Rahul said, add a .gitignore file and add secrets.py to this file so that it won't be committed.
The disadvantage of this approach is that there is no source control at all on that file; if you lose it you're SOL.
Use environment variables.
Use the Apache SetEnv or PassEnv directives to pass environment variables to your process, then retrieve them with os.environ() in your settings file. This has the advantage in that in development, you can set new variables (as simply as VAR1=whatever VAR2=whatever ... ./manage.py runserver ...) or set them from whatever mechanism you use to launch your development project.
The disadvantage is much the same; if you lose your Apache configs you're boned.
Use a second repository in combination with method 1.
Personally, I like the idea of having a dedicated secrets repository that you put all your secrets into and keep that repo under lock and key. Then as part of your deployment process, you can use git archive or another similar command to extract the proper keys for the place you're deploying to, and you can keep your secrets backed up and under version control easily. You can also add the appropriate files in the secrets repo to the .gitingore file of your site repository so that they don't accidentally get committed.
The downside of this is that you have another extra repository and another deployment step. I think that's worth it, personally, but it's really up to you.
In general, the more secure you want it, the more inconvenient it's going to be to access those secrets. That's really a rule in general, though.
You can create a file named secret_settings.py and place your SECRET_KEY inside this file. Then add this file to .gitignore.
Then in your settings, you can remove the secret key variable and import it from there. This should ensure that SECRET_KEY variable remains out of version control.
Create a file named secret_settings and then place your SECRET_KEY and other secret settings in it.
SECRET_KEY = .. # add your setting here
Then in your settings.py file, import these settings.
from secret_settings import *
Finally, add secret_settings.py to your .gitignore file.
Note:
If you already have committed some sensitive data to your repo, then change it!
As per Github website in the removing sensitive data article:
If you committed a password, change it! If you committed a key,
generate a new one.
Check this link on how to purge a file from your repository's history.
Common approach, if you'd like to configure region, but did not want to store sensitive information in repo, is to pass it through environment variables. When you need it just call os.environ('SECRET') (even in your settings.py). Better with some fallback value.
Virtualenv does not helps you to hide anything, it just prevent you system-wide Python installation from littering by one-project-required-packages.
The solution I use is to create a file sec.py and place it next to my settings.py file. Then in at line 1 of settings.py call from .sec import *. Be sure to include the period in front of the file name. Be sure to list sec.py in your .gitignore file.