Develop Reference

SQLAlchemy create_all() unable to create new tables

I have flask web app which uses mysql db using flask-sqlalchemy.
I have kept separate utility scripts for DDL creation.
My existing app works perfectly fine but this script is unable to create new table.
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from config import connect_strng
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = connect_string
db = SQLAlchemy()
# Added this import just before create_all
from db_models import Test
db.create_all()
db.session.commit()
I have defined model inside db_models
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Test(db.Model):
__tablename__ = 'test'
test_id = db.Column(db.Integer, primary_key=True)
My script is finishing with exit code of 0 indicating no errors, but I don't see table getting generated in mysql database.
$ python create_table.py
$ echo $?
0
I checked answer to the similar question but did not work.

You need to use the same db object across your whole app. Importing it where it is needed.
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy() # <--- This is what is causing your issue
# IT should look something more like...
from create_table import db
class Test(db.Model):
__tablename__ = 'test'
test_id = db.Column(db.Integer, primary_key=True)
However there is a problem with the above suggestion...It will lead to a circular import. To solve that requires restructuring your app a bit. Exactly how to do it is up to you but I'll give you a suggestion.
Create a new file called database.py and put your db object in there. Then you can do from database import db whenever you need db.
database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
As far as how to structure your app, consider the application factory pattern. It takes into account the circular import issue that commonly arises with flask.

I was able to resolve the issue by making use of flask's application context.
As sugested by #noslenkwah, you should use db object from single place by defining into single file database.py.
Here is my solution.
database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
db_models.py
from database import db
class Test(db.Model):
__tablename__ = 'test'
test_id = db.Column(db.Integer, primary_key=True)
create_table.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from config import connect_strng
from database import db
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = connect_string
with app.app_context():
db.init_app(app)
# Added this import just beore create_all
from db_models import Test, CrawlStat
db.create_all()
db.session.commit()

flask_sqlalchemy create_all() fails silently in unit testing

I'm writing unit tests for a REST API written in Flask with the flask_sqlalchemy extension. Because I have a number of model classes, I wrote a TestCase subclass to do the standard setUp/cleanUp of the test database. All my test classes inherit from this. Each test succeeds when run alone, but when I run more than one test in a single class, the second setUp() fails on the self.db.session.commit() (I'm trying to add an entry to the User table) because self.db.create_all() has (silently) failed to create any tables.
Here is my base test class, in the __init__.py of the test package:
import unittest
from .test_client import TestClient
from .. import create_app
from pdb import set_trace as DBG
class ApiTest(unittest.TestCase):
default_username = 'fred'
default_password = 'bloggs'
db = None
def setUp(self):
try:
self.app = create_app('testing')
self.addCleanup(self.cleanUp)
self.ctx = self.app.app_context()
self.ctx.push()
from .. import db
self.db = db
self.db.session.commit()
self.db.drop_all(app=self.app)
from ..models import User, Player, Team, Match, Game
# self.app.logger.debug('drop_all())')
self.db.create_all(app=self.app)
# self.app.logger.debug('create_all())')
user = User(user_name=self.default_username)
user.password = self.default_password
self.db.session.add(u)
self.db.session.commit()
self.client = TestClient(self.app, user.generate_auth_token(), '')
except Exception, ex:
self.app.logger.error("Error during setUp: %s" % ex)
raise
def cleanUp(self):
try:
self.db.session.commit()
self.db.session.remove()
self.db.drop_all(app=self.app)
# self.app.logger.debug('drop_all())')
self.ctx.pop()
except Exception, ex:
self.app.logger.error("Error during cleanUp: %s" % ex)
raise
Can anyone tell me what's wrong here please?
EDIT: Added the code for create_app() as requested.
# chessleague/__init__.py
import os
from flask import Flask, g
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
from . import config
app = None
db = None # The database, initialised in create_app()
def create_app(config_name):
app = Flask(__name__)
app.config.update(config.get_config(config_name))
# if app.config['USE_TOKEN_AUTH']:
# from api.token import token as token_blueprint
# app.register_blueprint(token_blueprint, url_prefix='/auth')
import logging
from logging.handlers import SysLogHandler
syslog_handler = SysLogHandler()
syslog_handler.setLevel(logging.WARNING)
app.logger.addHandler(syslog_handler)
login_manager = LoginManager()
login_manager.login_view = 'auth.login'
login_manager.init_app(app)
global db
db = SQLAlchemy(app)
db.init_app(app)
from .models import User,Player,Game,Match,Team,Post
db.create_all()
from .api import api as api_blueprint
app.register_blueprint(api_blueprint, url_prefix='/chessleague')
return app
`

create_all() applies to the metadata, that is being discovered by importing modules with models. In your case, models' metadata binds to the db from your models.py but you are calling create_all() from chessleague/__init__.db from create_app() function, which is different objects for SqlAlchemy. You can fix that by using db from models.py:
from .models import User,Player,Game,Match,Team,Post, db as models_db
models_db.create_all()

Here's the initialisation sequence that worked for me - comments welcome!
My test class setUp() calls create_app(config_name) from the main app package.
The main app package(__init__.py) creates the app instance at module level, ie app=Flask(my_app_package_name)
Then my function
create_app(config_name)
loads the right config into app.config (including the right SQLACHEMY_DATABASE_URL)
imports the model classes and db (as model_db) from models.py
This import creates the symbol db at module level in models.py, followed by the model class definitions:
# models.py
from . import app
db = SQLAlchemy(app)
...
class User(db.Model)
...
etc
Now everything is set up properly: the symbol 'db' can be imported anywhere from models.py, and I can call db.create_all() successfully from my test setUp().
#Fian, can you post your solution as an answer so I can give you credit?

Flask circular dependency

I am developing a Flask application. It is still relatively small. I had only one app.py file, but because I needed to do database migrations, I divided it into 3 using this guide:
https://realpython.com/blog/python/flask-by-example-part-2-postgres-sqlalchemy-and-alembic/
However, I now can't run my application as there is a circular dependency between app and models.
app.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask import render_template, request, redirect, url_for
import os
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = os.environ['DB_URL']
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.debug = True
db = SQLAlchemy(app)
from models import User
... routes ...
if __name__ == "__main__":
app.run()
models.py:
from app import db
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
def __init__(self, username, email):
self.username = username
self.email = email
def __repr__(self):
return self.username
manage.py:
from flask_script import Manager
from flask_migrate import Migrate, MigrateCommand
from app import app, db
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
if __name__ == "__main__":
manager.run()
They are all in the same directory. When I try to run python app.py to start the server, I receive an error which definitely shows a circular dependency (which is pretty obvious). Did I make any mistakes when following the guide or is the guide wrong? How can I refactor this to be correct?
Thanks a lot.
EDIT: Traceback
Traceback (most recent call last):
File "app.py", line 14, in <module>
from models import User
File "/../models.py", line 1, in <module>
from app import db
File "/../app.py", line 14, in <module>
from models import User
ImportError: cannot import name User

I propose the following structure:
# app/extensions.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
...
# app/app.py
from app.extensions import db
def create_app(config_object=ProdConfig):
app = Flask(__name__.split('.')[0])
app.config.from_object(config_object)
register_extensions(app)
...
def register_extensions(app):
db.init_app(app)
...
# manage.py
from yourapp.app import create_app
app = create_app()
app.debug = True
...
In this case, database, app, and your models are all in separate modules and there are no conflicting or circular imports.

I chased this for a few hours, landing here a few times, and it turned out I was importing my page modules (the ones holding the #app.route commands) before the line where the app was created. This is easy to do since import commands tend to be placed at the very beginning, but it doesn't work in this case.
So this:
# app/__init__.py
print("starting __init__.py")
from flask import Flask
from flask import render_template
import matplotlib.pyplot as plt
import numpy as np
import mpld3
app = Flask(__name__, instance_relative_config=True)
app.config.from_object('config')
from . import index
from . import simple
app.run(threaded=False)
print("finished __init__.py")
Instead of having all imports on top.
Placing this here because this has to be a common error for casual flask users to encounter and they are likely to land here. I have hit it as least twice in the last couple of years.

db.create_all() doesn't create tables defined in separate file

I am trying to create the tables for my models, which are defined in a separate module from my app. I call db.create_all(), but no tables are created and there are no errors. I've defined the models and imported them before calling create_all. Why doesn't this work?
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://postgres:123#localhost/flask'
db = SQLAlchemy(app)
from models import User
db.create_all()
db.session.commit()
models.py:
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class User(db.Model):
__tablename__ = 'users'
uid = db.Column(db.Integer, primary_key = True)

You created two separate db instances, one along with the app and one along with the models. Each instance has it's own metadata that stores the tables defined on it. The one you're using to issue the create table statement was not the one that the models were defined on. You should use only one instance of the extension, importing it when needed.
myapp/__init__.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
# import the models *after* the db object is defined
from myapp import models
myapp/models.py:
from myapp import db
class User(db.Model):
...
create_tables.py:
from myapp import app, db
with app.app_context():
db.create_all()
Other things to note:
You should structure your app as a package, so that everything is importable under one location.
flask.ext is deprecated, import the extension directly from its package name.
Flask-SQLAlchemy automatically generates __tablename__ from the class name, you don't need to define it yourself.
You do not have to call commit after create_all.

Flask-SQLAlchemy import/context issue

I want to structure my Flask app something like:
./site.py
./apps/members/__init__.py
./apps/members/models.py
apps.members is a Flask Blueprint.
Now, in order to create the model classes I need to have a hold of the app, something like:
# apps.members.models
from flask import current_app
from flaskext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(current_app)
class Member(db.Model):
# fields here
pass
But if I try and import that model into my Blueprint app, I get the dreaded RuntimeError: working outside of request context. How can I get a hold of my app correctly here? Relative imports might work but they're pretty ugly and have their own context issues, e.g:
from ...site import app
# ValueError: Attempted relative import beyond toplevel package

The flask_sqlalchemy module does not have to be initialized with the app right away - you can do this instead:
# apps.members.models
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Member(db.Model):
# fields here
pass
And then in your application setup you can call init_app:
# apps.application.py
from flask import Flask
from apps.members.models import db
app = Flask(__name__)
# later on
db.init_app(app)
This way you can avoid cyclical imports.
This pattern does not necessitate the you place all of your models in one file. Simply import the db variable into each of your model modules.
Example
# apps.shared.models
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
# apps.members.models
from apps.shared.models import db
class Member(db.Model):
# TODO: Implement this.
pass
# apps.reporting.members
from flask import render_template
from apps.members.models import Member
def report_on_members():
# TODO: Actually use arguments
members = Member.filter(1==1).all()
return render_template("report.html", members=members)
# apps.reporting.routes
from flask import Blueprint
from apps.reporting.members import report_on_members
reporting = Blueprint("reporting", __name__)
reporting.route("/member-report", methods=["GET","POST"])(report_on_members)
# apps.application
from flask import Flask
from apps.shared import db
from apps.reporting.routes import reporting
app = Flask(__name__)
db.init_app(app)
app.register_blueprint(reporting)
Note: this is a sketch of some of the power this gives you - there is obviously quite a bit more that you can do to make development even easier (using a create_app pattern, auto-registering blueprints in certain folders, etc.)

an original app.py: https://flask-sqlalchemy.palletsprojects.com/en/2.x/quickstart/
...
app = flask.Flask(__name__)
app.config['DEBUG'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = flask.ext.sqlalchemy.SQLAlchemy(app)
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
...
class Computer(db.Model):
id = db.Column(db.Integer, primary_key=True)
...
# Create the database tables.
db.create_all()
...
# start the flask loop
app.run()
I just splitted one app.py to app.py and model.py without using Blueprint. In that case, the above answer dosen't work. A line code is needed to work.
before:
db.init_app(app)
after:
db.app = app
db.init_app(app)
And, the following link is very useful.
http://piotr.banaszkiewicz.org/blog/2012/06/29/flask-sqlalchemy-init_app/