Develop Reference

How to separate flask files when two files depend on each other?

I'm trying to develop a database driven flask app.
I have an api.py file which has the flask app, api and SQLAlchemy db objects and a users.py file which contains the routes ands code to create a database table.
In the users.py file, there's a UserManager Resource which has the routes. I have to add this resource to the API from this file.
So users.py needs to import the db and the api from the api.py file and api.py needs to serve the flask app so it has all the variables users.py needs, but api.py also needs to import users.py in order to use it as a flask blueprint.
api.py:
...
from user import user
app = Flask(__name__)
app.register_blueprint(user)
api = Api(app)
...
db = SQLAlchemy(app)
ma = Marshmallow(app)
...
if __name__ == '__main__':
app.run(debug=True)
users.py:
from api import api
user = Blueprint('user', __name__, template_folder='templates')
db = SQLAlchemy(api.app)
ma = Marshmallow(api.app)
class User(db.Model):
user_id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True, nullable=False)
username = db.Column(db.String(32), unique=True, nullable=False)
password = db.Column(db.String(32))
first_name = db.Column(db.String(32))
last_name = db.Column(db.String(32))
...
...
class UserManager(Resource):
#user.route('/get/<user_id>', methods = ['GET'])
def get_user(user_id):
...
This of course results in errors due to circular imports. Question is, how do I separate the flask files with the routes from the api file when the blueprint has dependencies from the api (the db and the api objects)
I also somehow have to do something like api.add_resource(UserManager, '/api/users') but I'm not sure where that'd go given the circular import.
Tried reducing dependencies between two files, but couldn't achieve described goal without doing a 2 way import.
Either trying to get 2 way import to work or still have same structure of separate files with routes but using 1 way import.

This is a well known problem in flask. The solution is to use application factories.
Cookiecutter Flask does this really well and offers a good template. It is well worth to check out their repo and try to understand what they are doing.
Assuming you have a folder app and this folder contains a file __init__.py and your other files user.py, etc.
Create a file app/extensions.py with this content and any other extension you need to initialize.
...
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
...
Import the db object (which was created, but not initialized) in the files where you need it.
from app.extensions import db
In your app/api.py file
from flask import Flask
from app.extensions import db
from app.user import user as user_bp
def create_app():
app = Flask(__name__)
register_extensions(app)
register_blueprints(app)
return app
def register_extensions(app):
"""Register Flask extensions."""
db.init_app(app)
return None
def register_blueprints(app):
"""Register Flask blueprints."""
app.register_blueprint(user_bp)
return None
if __name__ == '__main__':
app = create_app()
app.run(debug=True)
Add stuff based on this approach as needed.

flask error with SQLAlchemy / ORM -- not seeing db tables

In development (so sqlite3) I'm getting this error on any database access:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: ujs ...
I got here by saying
export FLASK_ENV=development
export FLASK_APP=my_app.py
flask db init
flask db migrate
flask db upgrade
flask run
and then doing an HTTP GET against that dev server.
I believe the migration workflow succeeded, because when I use the sqlite3 commandline client, I can see the (empty) table with a believably correct schema.
╭╴ (get-db-working *%=)╶╮
╰ jeff#starshine:TN_flask_web $ sqlite3 dev.db
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> .table
alembic_version ujs
sqlite> .quit
╭╴ (get-db-working *%=)╶╮
╰ jeff#starshine:TN_flask_web $
I therefore believe I've made a coding error. But I'm not seeing it.
I have this code (pared down to what I believe is the essential bits):
my_app.py:
from app import create_app, db, cli
from app.models import UJS
app = create_app()
cli.register(app)
#app.shell_context_processor
def make_shell_context():
return {'db': db,
'UJS': UJS}
app/models.py:
from app import db
import time
def now_in_microseconds():
"""Return the current time in microseconds since the epoch.
"""
return time.time() * 1000 * 1000
class UJS(db.Model):
id = db.Column(db.Integer, primary_key=True)
timestamp_microseconds = db.Column(db.BigInteger, default=now_in_microseconds)
ip_hash = db.column(db.String(40))
# And then some more columns, all quite boring.
def __repr__(self):
return '<[{tag}]/[{ip}] {microsec}/{city}>'.format(
tag=self.tag, ip=self.ip_hash,
microsec=self.timestamp_microseconds, city=self.city)
app/__init__.py:
from flask import Flask, request, current_app
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from config import Config
db = SQLAlchemy()
migrate = Migrate()
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(config_class)
try:
app.config.from_pyfile("../config_local.py")
except FileNotFoundError:
print('No local config found.')
except:
print('Unexpected error on app.config.from_pyfile()')
db.init_app(app)
migrate.init_app(app, db)
...
return app
from app import models
and app/main/routes.py:
from flask import request, g, current_app, session
from app import db
from app.main import bp
from app.models import UJS
#bp.before_app_request
def before_request():
if 'static' == request.endpoint:
# This should only happen in dev. Otherwise, nginx handles static routes directly.
return
# I expect this to return an empty list, but it throws a 500.
print(UJS.query.all())
Any suggestions what I'm missing?

For anyone who might find this question later on: the problem was about having the right absolute path to your DB in your SQLALCHEMY_DATABASE_URI config value.
Also (this wasnt the case here, but it might possibly gotcha with the same symptoms) - if you omit __tablename__ on Model declaration, SQLAlchemy might autogenerate something you wont expect. Just a thing to keep in mind, if you're working with an existing DB with some schema already in place.

Flask- How to import db=SQLAlchemy in separate file

I want to add a new table or add data by calling db, but i got some problem when i try to import db
db return like this <SQLAlchemy engine=None>
which mean i didnt already doing this db.init_app(app)
this is my file struckture
Root
run.py
------>server/__init__.py
Config.py
import os
class Config:
SECRET_KEY = os.environ.get('SECRET_KEY')
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root:#localhost/flask_py'
MAIL_SERVER = 'smtp.googlemail.com'
MAIL_PORT = 587
MAIL_USE_TLS = True
MAIL_USERNAME = os.environ.get('EMAIL_USER')
MAIL_PASSWORD = os.environ.get('EMAIL_PASS')
Run.py
from server import server_app, db
app = server_app()
if __name__ == '__main__':
app.run(debug=True)
__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from server.config import Config
db = SQLAlchemy()
def server_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
from server.users.routes import users
app.register_blueprint(users)
return app
in my command line using windows i want to import db, i try like this :
D:\PYTHON\root>python
from run import db
db.create_all()
but when i check is :
<SQLAlchemy engine=None>

I have a full working solution here very similar to what you're doing. Check my layout and then look at the create_db.py.
https://github.com/researcher2/stackoverflow_56885380
It appears in your case the "server_app" is not being executed in your interactive shell. Assuming you are running interactive shell in root directory, you would want to do the following:
from server import db, create_app
app = server_app()
with app.app_context():
db.create_all()
The annoying thing about flask-sqlalchemy as opposed to plain sqlalchemy is the db is coupled to a flask app. The db config comes from the flask config and the initiation is done during db_init or just SqlAlchemy(db) if you want the simpler version for single app setup.
Your models would also need to be setup properly. In my example above I just had them in the create_db script.
This post may help you as well regarding factories and blueprints. I created the above github to answer it.
Reflecting different databases in Flask factory setup

Flask migrate doesn't detect models.py

I'm attempting to run flask migrate db in my working directory, and it does not use the model I defined in models.py
Here's the code.
models.py
import sys
sys.path.append("../")
from Talks2 import db
class Talk(db.Model):
presenter = db.Column(db.Text())
talkType = db.Column(db.Text())
desc = db.Column(db.Text(), primary_key=True)
link = db.Column(db.Text())
time = db.Column(db.Integer())
def __repr__(self):
return "Presenter: {}\nType: {}\nDescription:\n{}\nLink: {}".format(self.presenter,self.talkType,self.desc,self.link)
routes.py
import sys
sys.path.append("../")
from flask import Flask, request, render_template
from Talks2 import Config
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
app = Flask(__name__)
app.config.from_object(Config)
db = SQLAlchemy(app)
migrate = Migrate(app,db)
from Talks2 import models
#app.route("/")
def index():
return render_template("index.html")
#app.route("/add", methods=["POST"])
def add():
person = request.form["presenter"]
ttype = request.form["type"]
desc = request.form["desc"]
link = request.form["link"]
print(person, file=sys.stderr)
print(ttype, file=sys.stderr)
print(desc, file=sys.stderr)
print(link, file=sys.stderr)
return render_template("index.html")
if __name__ == "__main__":
app.run()
What do I need to change for it to correctly generate the script?

You are importing db from Talks2.py in models.py file and again in routes.py declaring again.
You haven't shared the code of Talks2.py file. What I am suspecting is you are declaring app and db object multiple times and replacing it with others.
Just do import in the proper way and your model will be detected by the flask.
The simplest solution is to declare app & db in Talks2.py, then import both in models.py and then from models.py import app & db in routes.py. This will resolve your problem.
Also, it should be flask db migrate instead of flask migrate db.
For more information refer to these commands:
To create a migration repository:
flask db init
To Generate Migration Script (Make sure to reviewed and edited, as Alembic currently does not detect every change you make to your models)
flask db migrate
To apply the migration to the database
flask db upgrade
To see all the commands that are available run this command:
flask db --help
For more info refer this official doc.
Let me know if this didn't help.

Flask nosetests - unable to connect to test db

I'm making a simple Flask web application for fun and I wanted to use nosetests. I'm stuck at how to use Flask-SQLAlchemy to connect to an in-memory test database in my tests file. When I run my tests - Flask connects to my main app's database and what is more, fails to clean it up after each test. Here's my tests code:
import nose
from nose.tools import *
from pyquery import PyQuery as pq
from flask.ext.sqlalchemy import SQLAlchemy
from app import site, db
from app.models import Post
class TestApp(object):
def setUp(self):
site.config['TESTING'] = True
site.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
self.test_app = site.test_client()
db.create_all()
def tearDown(self):
# db.session.remove()
db.drop_all()
def test_posts_index(self):
db.session.add(Post('title', 'body'))
db.session.add(Post('title2', 'body'))
db.session.commit() # this writes to production db ie app.db file
# instead of sqlite://
rv = self.test_app.get('/posts')
d = pq(rv.data)
print len(d('h1'))
assert len(d('h1')) == 2
And here's my app/__init__.py code:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from app import config
site = Flask(__name__)
site.config['SQLALCHEMY_DATABASE_URI'] = config.db_uri
db = SQLAlchemy(site)
site.secret_key = 'A0Zr98j/3yX R~XHH!jmN]LWX/,?RT'
from app import db_setup
db_setup.create_db()
import controllers, models
The db_setup.create_db() in app/__init__.py function looks simply like this:
from app import db
from app.models import Post
def create_db():
db.create_all()
db.session.commit()
I tried instantiating the application and database in the tests file, but then my models don't work because they from app import db, where db is the production db object. I also sprinkled a few print statements in the test case like print db and they print out something like <SQLAlchemy engine sqlite://>, but it still writes to the production db anyways.
I'd really appreciate any tips on how to get around this. Thanks!

Why don't you use something about the environment to determine whether the app starts in a testing or live mode?
if 'testing' in os.environ:
site.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
else:
site.config['SQLALCHEMY_DATABASE_URI'] = config.db_uri
There are so many ways to skin this particular cat. If you don't like the idea of having if blocks littering your code you can import your settings from an entirely separate module based on whether the app is started in testing or live mode.

I was able to figure out the problem, it's related to me initiating a connection to the database in my __init__.py file, which I shouldn't do.
The culprit was the
from app import db_setup
db_setup.create_db()
code. Essentially, every time I did an from app import db, I think that app gets instantiated, it calls db_setup.create_db(), which creates the tables using the production config. From there on, despite trying to set the flask app config SQLALCHEMY_DATABASE_URI to an in memory database, the db object would continue to use the database instantiated in the __init__.py file.
All I have to do is call create_all() from the environment my will run in at that time. Hope this helps anyone how might run into something similar.

I had the same problem, but I didn't use a db.create_all() type statement in my init.py file at all.
In the end, the only way I could around the issue was to use
def setUp(self):
with app.app_context():
db.create_all()