Develop Reference

Using Flask-migrate for models in multiple files

I am confused about how to use Flask-Migrate when I have multiple models.
Basically my Flask app looks like this:
app
├── __init__.py
├── config.py
├── manage.py
├── migrations
├── models
│   ├── model1.py
│   ├── model2.py
├── resources
├── run.py
└── tests
I've read that for each model its best to create the db = SQLAlchemy() object in the file and then import this db object into the app's__init__.py like so:
from models.model1 import db
db.init_app(app)
from models.model2 import db
db.init_app(app)
However if I do this for multiple model files, how can I add Flasks's migrate functionality, considering I can only use 1 sql alchemy object for the migrate class instantiation:
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
Would it be best in this case to define a single sql alchemy object in the __init__.py file and import that into all my models?

You misread the referenced text. That talks about something completely different. That talks about keeping your db object separate from the app object (and tie the two togther in the create_app factory function). Creating multiple db objects is only complicating matters for you.
All that is needed is a single db = SQLAlchemy() object, and all the files that define models need to be imported. Usually that's done directly or indirectly via your create_app factory function, You need to call the create_app() function anyway to be able to run the flask db command-line tool anyway.
Next, you do not need to create a manager either. The Manager object is a hold-over from the time before the Flask project added support for scripts itself. If you are using Flask 0.12 or newer, you don't want to be using Flask-Script and it's manager.
So, all you need, in your __init_.py, is:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
db = SQLAlchemy()
def create_app(test_config=None):
app = Flask(__name__)
app.config.from_object(f"{__name__}.config")
app.config.from_envvar("PROJECTNAME_SETTINGS", silent=True)
if test_config:
app.config.from_mapping(test_config)
db.init_app(app)
Migrate(app, db)
# importing the models to make sure they are known to Flask-Migrate
from models import models1, models2
# any other registrations; blueprints, template utilities, commands
return app

Flask: application unable to run because of KeyError

Am currently studying how to develop web applications using Flask and i was following this tutorial on how to do it and everything was moving on fine till the concept of Flask-SQLAchemy got introduced.
The structure of my flask application is as below
├── bin
├── bookshelf
│   ├── admin
│   │   ├── controllers.py
│   │   ├── __init__.py
│   ├── data
│   │   ├── __init__.py
│   │   ├── models.py
│   ├── __init__.py
│   ├── main
│   │   ├── controllers.py
│   │   ├── __init__.py
│   ├── static
│   └── templates
├── config.py
├── data-dev.sqlite
├── docs
├── requirements.txt
├── run.py
Am trying to use an sqlite database(data-dev.sqlite) that i designed myself without having to go through the process of SQLAchemy creating for me one and below is it's structure; that was generated using the .dump command in sqlite
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE author (
id INTEGER NOT NULL PRIMARY KEY,
author_names VARCHAR(100) NOT NULL UNIQUE
);
CREATE TABLE book (
id INTEGER NOT NULL PRIMARY KEY,
title VARCHAR(80) NOT NULL,
rating INT,
image VARCHAR(30),
author_id INTEGER NOT NULL,
FOREIGN KEY (author_id) REFERENCES author(id)
);
CREATE TABLE role (
id INTEGER NOT NULL PRIMARY KEY,
role_name VARCHAR(80) NOT NULL UNIQUE,
description VARCHAR(255)
);
CREATE TABLE user (
id INTEGER NOT NULL PRIMARY KEY,
email VARCHAR(75) NOT NULL UNIQUE,
password VARCHAR(255) NOT NULL,
active BOOLEAN NOT NULL DEFAULT 0,
role_id INTEGER NOT NULL,
FOREIGN KEY (role_id) REFERENCES role(id)
);
COMMIT;
The traceback produced when i run the command python run.py runserver -d in the venv is
Traceback (most recent call last):
File "run.py", line 1, in <module>
from bookshelf import create_app
File "/home/mayatsa/environments/flask-test/bookshelf/__init__.py", line 2, in <module>
from bookshelf.main.controllers import main
File "/home/mayatsa/environments/flask-test/bookshelf/main/controllers.py", line 2, in <module>
from bookshelf.data.models import Author, Book
File "/home/mayatsa/environments/flask-test/bookshelf/data/models.py", line 6, in <module>
class Book(db.Model):
File "/home/mayatsa/environments/flask-test/bookshelf/data/models.py", line 8, in Book
__table__ = db.Model.metadata.tables['book']
KeyError: 'book'
Files order of traceback from top to bottom and their contents are as follows
run.py
from bookshelf import create_app
from flask_script import Manager
app = create_app()
manager = Manager(app)
if __name__ == '__main__':
manager.run()
bookshelf/__init__.py
from flask import Flask
from bookshelf.main.controllers import main
from bookshelf.admin.controllers import admin
from bookshelf.data.models import db
def create_app():
app = Flask(__name__)
app.config.from_object('config')
with app.app_context():
db.init_app(app)
db.Model.metadata.reflect(db.engine)
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
bookshelf/main/controllers.py
from flask import Blueprint, render_template, request
from bookshelf.data.models import Author, Book
main = Blueprint('main', __name__, template_folder='templates')
#main.route('/')
def index():
return render_template('main/index.html')
#main.route('books/')
def display_books():
return render_template('main/books.html')
bookshelf/data/models.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Book(db.Model):
__table__ = db.Model.metadata.tables['book']
def __repr__(self):
return '<Book %r>' % (self.title)
class Author(db.Model):
__table__ = db.Model.metadata.tables['author']
def __repr__(self):
return '<Author %r>' % (self.author_names)
class Role(db.Model):
__table__ = db.Model.metadata.tables['role']
def __repr__(self):
return '<Role %r>' % (self.role_name)
class User(db.Model):
__table__ = db.Model.metadata.tables['user']
def __repr__(self):
return '<User %r>' % (self.email)
Additional config.py
import os
basedir = os.path.abspath(os.path.dirname(__file__))
DEBUG = True
TESTING = False
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \
'sqlite:///' + os.path.join(basedir, 'data-dev.sqlite')
SQLALCHEMY_COMMIT_ON_TEARDOWN = True
SQLALCHEMY_TRACK_MODIFICATIONS = False
SECRET_KEY = 'a9eec0e0-23b7-4788-9a92-318347b9a39f'
From all the code provided above, what could be causing my application to produce this error and stopping it from running?
Thank you

I kept probing till i landed on this question and this one and they helped clear up the fog for me.
After installing sqlacodegen and flask-sqlacodegen using
pip install sqlacodegen
pip install flask-sqlacodegen
All i had to do now is run the following command in my Flask application root
sqlacodegen sqlite:///data-dev.sqlite --flask > models.py
And the models corresponding to my database were generated and then after i replaced my the bookshelf/data/models.py with the just generated models.py.
Then finally changed bookshelf/__init__.py create_app function to look like this
def create_app():
app = Flask(__name__)
app.config.from_object('config')
db.init_app(app)
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
And now everything seems to be working fine.

This is because the models.py file is executed before the tables are reflected. In bookshelf/__init__.py file, you're importing your controllers main and admin at the top, which in turn import your data models from bookshelf/data/models.py which subclasses the SQLAlchemy instance db. But the db.reflect() is yet to happen, therefore the metadata of the tables isn't reflecting and hence the key error. To solve this, either import your models in your controllers inside the functions, wherever needed, like this
bookshelf/main/controllers.py
from flask import Blueprint, render_template, request
main = Blueprint('main', __name__, template_folder='templates')
#main.route('/')
def index():
from bookshelf.data.models import Author, Book
return render_template('main/index.html')
#main.route('books/')
def display_books():
from bookshelf.data.models import Book
return render_template('main/books.html')
or import your controllers after you've initialized your db object like this,
bookshelf/init.py
from flask import Flask
from bookshelf.data.models import db
def create_app():
app = Flask(__name__)
app.config.from_object('config')
with app.app_context():
db.init_app(app)
db.Model.metadata.reflect(db.engine)
from bookshelf.admin.controllers import admin, main
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
After this, just add a separate database.py file to create the SQLAlchemy object instead of creating it in your models.py file, since it will cause a circular import issue if you have it in either bookshelf/init.py or bookshelf/data/models.py.
bookshelf/database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
Remove this line from your init.py file
from bookshelf.data.models import db
and instead get it in your init.py from database.py
from bookshelf.database import db

The metadata isn't there yet at the time you're importing bookshelf.data.models, as it's being reflected into the metadata only afterwards.
Maybe using __tablename__ instead of __table__ might help? I'm not that well versed in the intricacies of SQLAlchemy.

variable defined in __init__.py can not be imported

I am trying to follow the instructions in http://flask.pocoo.org/docs/0.12/patterns/celery/ so I can perform flask/socketIO operations in celery tasks. My directory structure is a bit different however and I'm not having any luck with imports.
My directory structure is as follows:
├── app
│   ├── __init__.py
│   ├── __pycache__
│   ├── auth.py
│   ├── ctasks.py
│   ├── helper.py
│   ├── saml.py
│   ├── socks.py
│   ├── templates
│   ├── threads.py
│   └── views.py
├── app.py
├── config.py
├── requirements.txt
└── saml
   ├── dev
   └── prod
I call the app from app.py
from app import socketio, app
if __name__ == '__main__':
socketio.run(app, debug=True, port=443, ssl_context='adhoc')
__init__.py
from flask import Flask, request
from flask_socketio import SocketIO
from .ctasks import subtaskcaller, make_celery
from .helper import wait_to_finish
async_mode = None
app = Flask(__name__)
app.config.from_object('config')
socketio = SocketIO(app, async_mode=async_mode)
cel = make_celery(app)
from .auth import SamlManager
saml_manager = SamlManager()
saml_manager.init_app(app)
from app import views, socks, saml, helper, ctasks
ctasks.py
from celery import Celery
from config import *
from .helper import wait_to_finish, emitter
import time
from app import cel
def make_celery(app):
c = Celery(app.import_name, backend=CELERY_RESULT_BACKEND, broker=CELERY_BROKER_URL)
c.conf.update(app.config)
taskbase = c.Task
class ContextTask(taskbase):
abstract = True
def __call__(self, *args, **kwargs):
with app.app_context():
return taskbase.__call__(self, *args, **kwargs)
c.Task = ContextTask
return c
#cel.task(name='tasks.tester', serializer='pickle')
def tester():
emitter('emit from subsubtask')
for i in range(1, 50):
time.sleep(1)
print('test {0}'.format(i))
x = True
return x
#cel.task(name='task.subtaskcaller', serializer='pickle')
def subtaskcaller():
emitter('emit from subtask')
finished = tester.delay()
wait_to_finish(finished)
return finished
I am getting an error when trying to import cel from app in ctasks.py:
ImportError: cannot import name 'cel'

In your __init__.py you only have cel. You don't have an object called celery in your __init__ file, so it can't be imported to some other file. You can try from app import cel.˛
EDIT:
in __init__ you from .ctasks import subtaskcaller, make_celery
but in ctasks you import cel from app (which doesn't exist yet at that point, only Flask, request, and SocketIO exist at that point in time).
So you need to put your #cel decorated functions in yet another script, which you can import all the way at the bottom of __init__

Organizing a Flask project

This is my first time creating a project using python and flask. I intend to use SQLAlchemy models along too. and this is a fairly bigger project. As of now, I have divided the project in 2 Blueprints : site and the api. After organizing the project, I am confused as to how can I connnect these models with the database and do I need to re-organize the structure as I am not fully aware of nature of flask.
so this is the directory structure of the dir app/ in my base repository:
`
.
├── Blueprints
│   ├── __init__.py
│   ├── __pycache__
│   │   └── __init__.cpython-36.pyc
│   ├── api
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-36.pyc
│   │   │   └── routes.cpython-36.pyc
│   │   └── routes.py
│   ├── config.py
│   └── site
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   └── routes.cpython-36.pyc
│   ├── operations.py
│   ├── routes.py
│   ├── static
│   └── templates
│   ├── about.html
│   ├── contact.html
│   ├── home.html
│   ├── login.html
│   ├── services.html
│   └── stories.html
├── __main__.py
├── __pycache__
│   └── __main__.cpython-36.pyc
└── models
├── Attendance.py
├── Batch.py
├── Course.py
├── Module.py
├── Student.py
├── Test.py
└── __init__.py
`
Please ignore Pycache, as this is auto generated.
now I cannot figure out a way as to how to import and use these models in api and site, neither I can understand as to how am I supposed to fetch the db object created in /Blueprints/__init__.py to all the models.
I understand that this question is not upto the standards of stack overflow questions, BUT I personally feel that organizing a flask project is itself very confusing with each tutorial or forum I see, having their own perspectives of organizing it.

There's several ways to organize a project, but the __init__.py file contained inside the app/ folder is what links a lot of it together. Here's the contents of one of my project's __init__.py file:
from werkzeug.contrib.fixers import ProxyFix
from flask import Flask, session
from app.config import (PERMANENT_SESSION_LIFETIME_MS, Time_Before_Warning,
Min_Ping_Interval)
import datetime
app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app)
# Setup the app with the config.py file
app.config.from_pyfile('config.py')
# Setup the logger
from app.logger_setup import logger, log_view
# Setup the database
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
#setup zipcode database
from pyzipcode import ZipCodeDatabase
zdb = ZipCodeDatabase()
# Setup the mail server
from flask.ext.mail import Mail
mail = Mail(app)
# Setup the debug toolbar
#from flask_debugtoolbar import DebugToolbarExtension
#app.config['DEBUG_TB_TEMPLATE_EDITOR_ENABLED'] = False
#app.config['DEBUG_TB_PROFILER_ENABLED'] = False
#toolbar = DebugToolbarExtension(app)
# Setup the password crypting
from flask.ext.bcrypt import Bcrypt
bcrypt = Bcrypt(app)
# Import the views
from app.views import (main, user, error, request, upload, dashboard, org,
msgs, notifications, download, reports,
direct_send,provider,utils)
app.register_blueprint(user.userbp)
app.register_blueprint(request.requestbp)
app.register_blueprint(upload.uploadbp)
app.register_blueprint(dashboard.dashboardbp)
app.register_blueprint(org.orgbp)
app.register_blueprint(msgs.msgbp)
app.register_blueprint(notifications.notificationsbp)
app.register_blueprint(download.downloadbp)
app.register_blueprint(reports.reportsbp)
app.register_blueprint(direct_send.directsendbp)
app.register_blueprint(provider.providerbp)
app.register_blueprint(utils.utilsbp)
# Setup the user login process
from flask.ext.login import LoginManager, current_user
from app.models import User, View
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.login_view = 'userbp.signin'
#login_manager.user_loader
def load_user(email):
return User.query.filter(User.email == email).first()
from flask.ext.principal import identity_loaded, RoleNeed, UserNeed
#identity_loaded.connect_via(app)
def on_identity_loaded(sender, identity):
# Set the identity user object
identity.user = current_user
# Add the UserNeed to the identity
if hasattr(current_user, 'id'):
identity.provides.add(UserNeed(current_user.id))
# Assuming the User model has a list of roles, update the
# identity with the roles that the user provides
if hasattr(current_user, 'roles'):
identity.provides.add(RoleNeed(current_user.roles.type))
from flask.ext.principal import Principal
# load the extension
principals = Principal(app)
# Create a permission with a single Need, in this case a RoleNeed.
#from app import admin
#app.before_request
def make_session_permanent():
session.permanent = True
lt = PERMANENT_SESSION_LIFETIME_MS / (60*1000)
app.permanent_session_lifetime = datetime.timedelta(minutes=lt)
#app.context_processor
def add_session_config():
"""
Add current_app.permanent_session_lifetime converted to milliseconds
to context.
"""
return {
'PERMANENT_SESSION_LIFETIME_MS': PERMANENT_SESSION_LIFETIME_MS,
'Time_Before_Warning': Time_Before_Warning,
'Min_Ping_Interval': Min_Ping_Interval,
}
And then inside one of the blueprints:
from flask import (Blueprint, render_template, redirect, url_for,
abort, flash, request)
from flask.ext.login import login_required, current_user
from app import app, models, db, log_view, config
from app.models import (Groups, Organizations, OrgHasOwner, UserHasGroups,
GroupHasOwner, User, Fax, FavoriteGroups)
from app.forms import org as org_forms
from app.toolbox import email, misc, s3, fax
from sqlalchemy.sql import func
from werkzeug import secure_filename
from uuid import uuid4
import datetime
import string
import os
# Create a user blueprint
orgbp = Blueprint('orgbp', __name__, url_prefix='/org')
#orgbp.route('/invite_user', methods=['GET','POST'])
#login_required
def invite_user():
[stuff goes here]

Flask teardown request in context of blueprint

I would like to access an sqlite3 database from a Flask application (without using Flask-SQLAlchemy, since I require fts4 functionality). I am using Flask blueprints, and I am not sure where to put the following functions (shamelessly copied from a response to this stackoverflow question):
def request_has_connection():
return hasattr(flask.g, 'dbconn')
def get_request_connection():
if not request_has_connection():
flask.g.dbconn = sqlite3.connect(DATABASE)
# Do something to make this connection transactional.
# I'm not familiar enough with SQLite to know what that is.
return flask.g.dbconn
#app.teardown_request
def close_db_connection(ex):
if request_has_connection():
conn = get_request_connection()
# Rollback
# Alternatively, you could automatically commit if ex is None
# and rollback otherwise, but I question the wisdom
# of automatically committing.
conn.close()
My file structure is:
app
├── __init__.py
├── main
│   ├── forms.py
│   ├── __init__.py
│   ├── views.py
├── models.py
├── static
└── templates
├── base.html
├── index.html
└── login.html
I want the request_has_connection() and get_request_connection() functions accessible from all view functions and maybe from models.py as well. Right now, I'm thinking they all belong in my blueprint init.py, which currently contains:
from flask import Blueprint
main = Blueprint('main',__name__)
from . import views
and that my request teardown function would be registered as
#main.teardown_request
def close_db_connection(ex):
<blah-blah-blah>
Is this right?