I'm setting up flask with docker. I've two ways to set environment variables, one in flask .cfg files and another in docker .env files.
I'm wondering which one is the better practice. Evaluating the pros and cons, if I move my environment variables to docker .env files, I would have to do os.environ.get at all the places in my application code, including handling of defaults which brings extra dependency inside application from os environment variables. On the other hand, adding environment variables like DB password, secret keys, etc inside flask config might not be a right idea, although all my environment variables and defaults would be at one single place.
You don't have to settle for one or the other.
Is not an unusual practice to use environment variables for configuring every aspect of your application, including DB passwords. This is staring to be an often practice in container environments such as Kubernetes or using docker secrets as they give the ability of keeping critical information encrypted and mount it as an environment variable to your container.
You could directly check in your application for the values of the environment variables or another option is to have an entrypoint in docker that checks for those values and ends up creating a configuration file that you use in your application.
This last option allows you to use environment variables to configure your application or if you don't want to you could directly mount a configuration file to your container skipping the envs completely.
This is used for example on the logstash docker image.
Just to add to the great answer by Esteban Garcia, another good way is to use both. Have a configuration file, class-based configs are great here because you can take advantage of config inheritance, and have all non-sensitive options in there. For the sensitive stuff like secrets, database passwords, etc - define them right in the config with os.environ.get to extract these values from the environment. So it ends up looking like this:
class DefaultConfig(Config):
TESTING = False
DEBUG = False
SECRET_KEY = os.getenv('APP_SECRET_KEY')
SQLALCHEMY_DATABASE_URI = os.getenv('APP_DATABASE_URI')
This way you can keep using app.config and not have to do os.environ.get all over your app.
This is my 2 cents to the subject.
The problem
I was working on porting a flask app to docker and wanted to move my settings.py config vars to env vars so they will be picked up by the container's env. This way, it's easier to deploy it on an external service like ECS just by setting env vars and the app will still the same.
Because I had a lot of config vars on settings.py and didn't want to set each of them by hand on the flask app creation, I came up with the following solution.
Also, I will be using docker-compose to execute all the containers.
Migrate from .py to .env
First thing is to move from python code to a syntax that is compatible with docker-compose. So, if you have this in settings.py:
# *****************************
# Environment specific settings
# *****************************
# DO NOT use "DEBUG = True" in production environments
DEBUG = True
# DO NOT use Unsecure Secrets in production environments
SECRET_KEY = 'This is an UNSECURE Secret. CHANGE THIS for production environments.'
# SQLAlchemy settings
SQLALCHEMY_DATABASE_URI = 'postgresql:///../app.postgresql'
Rename that file to settings.env and change the content to:
# *****************************
# Environment specific settings
# *****************************
# DO NOT use "DEBUG=True" in production environments
DEBUG=True
# DO NOT use Unsecure Secrets in production environments
SECRET_KEY=This is an UNSECURE Secret. CHANGE THIS for production environments.
# SQLAlchemy settings
SQLALCHEMY_DATABASE_URI=postgresql:///../app.postgresql
Note that you have to remove all white spaces and quotes (doubles and singles)
After that, you need to load that file on the docker-compose.yaml file:
version: '3.1'
services:
web:
image: web-server
env_file: path_to/settings.env
Then, when you create Flask app, do the following:
# Instantiate Flask
app = Flask(__name__)
for variable, value in os.environ.items():
app.config[env_name] = value
If you don't want to load all available env vars on flask
You can add a prefix to the vars on settings.env like this:
YOURAPP_DEBUG=True
And then on the app creation:
# Instantiate Flask
app = Flask(__name__)
for variable, value in os.environ.items():
if variable.startswith("YOURAPP_"):
env_name = variable.split("YOURAPP_")[1]
app.config[env_name] = value
Related
I want to have different environment variables based on development and production
but i can't seem to find anything related to this topic for FastAPI.
Is it possible that i can have .env, .env.local, .env.prod to have different environment variables
I don't think you need multiple files. Usually how it's done is, have a single config file that has the default values, usually this is your "local" config file. For prod, staging and other environments, you can override these settings by setting environment variables, most hosts support it nowadays. It's more secure and you don't have to expose production secrets and keys in your repository.
This library is one example of what you code use:
https://github.com/theskumar/python-dotenv
EDIT
For example, if your application is hosted in Heroku, the heroku config commands of the Heroku CLI makes it easy to manage your app’s config vars:
heroku config:set SOME_CONFIG_I_NEED=value for production
You can also edit config vars from your app’s Settings tab in the Heroku Dashboard.
Please refer to the Heroku documentation for more information.
After you set the env vars in Heroku, this is how you would access them from your Python code (using python-dotenv):
First, install python-dotenv:
pip install python-dotenv
Now, create a file called .env in the root of your project with the following contents:
# Development settings
SOME_CONFIG_I_NEED=value for development
Now in your python file:
from dotenv import load_dotenv
load_dotenv() # take environment variables from .env.
SOME_CONFIG_I_NEED = os.environ.get("SOME_CONFIG_I_NEED")
print(SOME_CONFIG_I_NEED) # This will print "value for development" when running on local, and will print "value for production" when running in Heroku.
The .env file
ADMIN_EMAIL="deadpool#example.com"
APP_NAME="ChimichangApp"
Read settings from .env¶
And then update your config.py with:
from pydantic import BaseSettings
class Settings(BaseSettings):
app_name: str = "Awesome API"
admin_email: str
class Config:
env_file = ".env"
Here we create a class Config inside of your Pydantic Settings class, and set the env_file to the filename with the dotenv file we want to use.
Read variable
from config import Settings
app = FastAPI()
setting = Settings()
print(setting)
Learn more
An alternative approach could be to use the Pydantic Settings:
https://pydantic-docs.helpmanual.io/usage/settings/
There is also a bit about that in the FastAPI docs, but personally I choose not to 'integrate' the nice Pydantic Settings that way.
https://fastapi.tiangolo.com/advanced/settings/
As per flask documentation, FLASK_ENV environmental variable determines whether flask runs in development or production mode.
Hence I have a .env file like so:
FLASK_ENV="development"
and my app.py looks like this:
load_dotenv(find_dotenv())
app = Flask(__name__)
config = DevConfig() if os.environ.get('FLASK_ENV') == 'development' else ProdConfig()
app.config.from_object(config)
Now here's the problem: if I move .env into another folder (in my case config), flask stops seeing it. More specifically (and weirdly):
The env variable is set ok (I can print it from different parts of the app)
The config loads ok (dev config loads indeed)
But flask app itself says:
Loading .env environment variables…
* Serving Flask app "app.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
How is it possible that env variable is set, but flask still thinks it's running in prod? Again this only happens when I move .env away from the root folder.
I am trying to deploy my django app on Heroku, but the issue is that I cannot access my config vars from python code.
Here are my config vars (obfuscated for obvious reasons)
$ heroku config --remote production
=== myapp Config Vars
DATABASE_URL: postgres://<removed>
DJANGO_DEBUG: 0
GOOGLE_APPLICATION_CREDENTIALS: <removed>
SECRET_KEY: <removed>
Now, in my django settings.py, the code
import os
os.environ["DJANGO_DEBUG"]
for example results in an error because the key "DJANGO_DEBUG" is nowhere to be found in the dictionary os.environ.
My question is: how can I access Heroku config variables in production?
I have tried the package python-decouple, but it is not able to access the config vars neither. (Locally, my variables are put in a .env file and are accessible with the help of the package python-decouple)
Edit: I actually realized that the config variables are accessible from my app and from the command if I run bash from within the app page on heroku, but there are not accessible if I run python on an ssh session, i.e., if I run python like this:
$heroku ps:exec
$python
os.environ does not contain my config variables.
Thank you for your help!
I am the original OP. Turns out the problem was the way I was accessing the remote terminal session. The correct way to do it to have all the config vars accessible is
heroku run bash
as opposed to
heroku ps:exec
You can set the config vars for a Heroku application in the settings of the App. Click the Settings Tab then navigate to the Config Vars section. Click Reveal Config Vars and enter the key pair values for your variables.
These will get injected into your application.
Every time I start up my flask app the environment variable is set to production. I want to have it set to development mode by default. Otherwise every time I start my app i have to run ..
export FLASK_ENV=development
How can I set environment's default value as development in every startup?
EDIT: I am using flask in a virtual environment on a raspberry pi.
You can edit your main flask app file and add these lines:
if __name__ == '__main__':
app.run(debug=True)
Using this method you have to run your flask app with Python interpreter like this => python app.py
Best Practice:
Install python-dotenv package inside your working environment =>pip install python-dotenv
Create a file named .env, put your environment variables in it, for your case it's FLASK_ENV=development
Then add this code to your config.py or some file that will get loaded before Flask main App
from dotenv import load_dotenv
dotenv_path = join(dirname(__file__), '.env') # Path to .env file
load_dotenv(dotenv_path)
Note that: If you are using flask command to run your application, you don't need to do the third step, flask will find .env files in the project directory by itself.
Using this method, it will only set Environment variable for the project that you have added this code to.
On Linux distro, like "Raspberry pi o.s", specify the environment on the terminal with the code below.
Unless you specify the environment, flask will assume production.
export FLASK_ENV=development
flask run
Like the first answer and instead of adding the variable to a .env file which can be forgotten, do this instead.
This way, if you try to run the file in production, you'll get an assertion error to remind you to actually use a dedicated web server (which "imports" the app). If you run locally, not only will you be reminded to use a .env file, but in the case no environment file is needed, the flask env is set to development to avoid any production conflicts.
import os
app = Flask(__name__)
IS_DEV = app.env == 'development' # FLASK_ENV env. variable
# code
if __name__ == '__main__':
# guaranteed to not be run on a production server
assert os.path.exists('.env') # for other environment variables...
os.environ['FLASK_ENV'] = 'development' # HARD CODE since default is production
app.run(debug=True)
While I am running Flask code from my command line, a warning is appearing:
Serving Flask app "hello_flask" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
What does this mean?
As stated in the Flask documentation:
While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time.
Given that a web application is expected to handle multiple concurrent requests from multiple users, Flask is warning you that the development server will not do this (by default). It recommends using a Web Server Gateway Interface (WSGI) server (numerous possibilities are listed in the deployment docs with further instructions for each) that will function as your web/application server and call Flask as it serves requests.
Try gevent:
from flask import Flask
from gevent.pywsgi import WSGIServer
app = Flask(__name__)
#app.route('/api', methods=['GET'])
def index():
return "Hello, World!"
if __name__ == '__main__':
# Debug/Development
# app.run(debug=True, host="0.0.0.0", port="5000")
# Production
http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()
Note: Install gevent using pip install gevent
As of Flask 1.x, the default environment is set to production.
To use the development environment, create a file called .flaskenv and save it in the top-level (root) of your project directory. Set the FLASK_ENV=development in the .flaskenv file. You can also save the FLASK_APP=myapp.py.
Example:
myproject/.flaskenv:
FLASK_APP=myapp.py
FLASK_ENV=development
Then you just execute this on the command line:
flask run
That should take care of the warning.
To remove the "Do not use the development server in a production environment." warning, run:
export FLASK_ENV=development
before flask run.
I was typing flask run and then saw this message after that I solve this issue with these:
1- Add this text in your myproject/.flaskenv :
FLASK_APP=myapp.py
FLASK_ENV=development
also you should type "pip3 install python-dotenv" for using this file .flaskenv
2-in your project folder type in terminal your flask command which one you use :
flask-3 run
First, try to the following :
set FLASK_ENV=development
then run your app.
I have been using flask for quite some time now, and today, suddenly this warning turned up. I found this.
As mentioned here, as of flask version 1.0 the environment in which a flask app runs is by default set to production. If you run your app in an older flask version, you won't be seeing this warning.
New in version 1.0.
Changelog
The environment in which the Flask app runs is set by the FLASK_ENV environment variable. If not set it defaults to production. The other recognized environment is development. Flask and extensions may choose to enable behaviors based on the environment.
in configurations or config you can add this code :
ENV = ""
same as if you try to add debug set to true like this
DEBUG = True
for more detail you can check this http://flask.pocoo.org/docs/1.0/config/#ENV
It means the programe is run on production mode even in developing environment.so to avoid that warning, you need to define this is development environment.for that,Type and run below command in project directory on terminal(linux).
export FLASK_ENV=development
if you are windows user then run,
set FLASK_ENV=development
To disable the message I use:
app.env = "development"
You have to put this in the Python-Script before you run the app with:
app.run(host="localhost")
If you encounter NoAppException and you see lazy loading the following seemed to fix the issue:
cd <project directory>
export FLASK_APP=.
export FLASK_ENV=development
export FLASK_DEBUG=1
You can begin your main script like this :
import os
if __name__ == '__main__':
os.environ.setdefault('FLASK_ENV', 'development')