Am currently studying how to develop web applications using Flask and i was following this tutorial on how to do it and everything was moving on fine till the concept of Flask-SQLAchemy got introduced.
The structure of my flask application is as below
├── bin
├── bookshelf
│ ├── admin
│ │ ├── controllers.py
│ │ ├── __init__.py
│ ├── data
│ │ ├── __init__.py
│ │ ├── models.py
│ ├── __init__.py
│ ├── main
│ │ ├── controllers.py
│ │ ├── __init__.py
│ ├── static
│ └── templates
├── config.py
├── data-dev.sqlite
├── docs
├── requirements.txt
├── run.py
Am trying to use an sqlite database(data-dev.sqlite) that i designed myself without having to go through the process of SQLAchemy creating for me one and below is it's structure; that was generated using the .dump command in sqlite
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE author (
id INTEGER NOT NULL PRIMARY KEY,
author_names VARCHAR(100) NOT NULL UNIQUE
);
CREATE TABLE book (
id INTEGER NOT NULL PRIMARY KEY,
title VARCHAR(80) NOT NULL,
rating INT,
image VARCHAR(30),
author_id INTEGER NOT NULL,
FOREIGN KEY (author_id) REFERENCES author(id)
);
CREATE TABLE role (
id INTEGER NOT NULL PRIMARY KEY,
role_name VARCHAR(80) NOT NULL UNIQUE,
description VARCHAR(255)
);
CREATE TABLE user (
id INTEGER NOT NULL PRIMARY KEY,
email VARCHAR(75) NOT NULL UNIQUE,
password VARCHAR(255) NOT NULL,
active BOOLEAN NOT NULL DEFAULT 0,
role_id INTEGER NOT NULL,
FOREIGN KEY (role_id) REFERENCES role(id)
);
COMMIT;
The traceback produced when i run the command python run.py runserver -d in the venv is
Traceback (most recent call last):
File "run.py", line 1, in <module>
from bookshelf import create_app
File "/home/mayatsa/environments/flask-test/bookshelf/__init__.py", line 2, in <module>
from bookshelf.main.controllers import main
File "/home/mayatsa/environments/flask-test/bookshelf/main/controllers.py", line 2, in <module>
from bookshelf.data.models import Author, Book
File "/home/mayatsa/environments/flask-test/bookshelf/data/models.py", line 6, in <module>
class Book(db.Model):
File "/home/mayatsa/environments/flask-test/bookshelf/data/models.py", line 8, in Book
__table__ = db.Model.metadata.tables['book']
KeyError: 'book'
Files order of traceback from top to bottom and their contents are as follows
run.py
from bookshelf import create_app
from flask_script import Manager
app = create_app()
manager = Manager(app)
if __name__ == '__main__':
manager.run()
bookshelf/__init__.py
from flask import Flask
from bookshelf.main.controllers import main
from bookshelf.admin.controllers import admin
from bookshelf.data.models import db
def create_app():
app = Flask(__name__)
app.config.from_object('config')
with app.app_context():
db.init_app(app)
db.Model.metadata.reflect(db.engine)
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
bookshelf/main/controllers.py
from flask import Blueprint, render_template, request
from bookshelf.data.models import Author, Book
main = Blueprint('main', __name__, template_folder='templates')
#main.route('/')
def index():
return render_template('main/index.html')
#main.route('books/')
def display_books():
return render_template('main/books.html')
bookshelf/data/models.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Book(db.Model):
__table__ = db.Model.metadata.tables['book']
def __repr__(self):
return '<Book %r>' % (self.title)
class Author(db.Model):
__table__ = db.Model.metadata.tables['author']
def __repr__(self):
return '<Author %r>' % (self.author_names)
class Role(db.Model):
__table__ = db.Model.metadata.tables['role']
def __repr__(self):
return '<Role %r>' % (self.role_name)
class User(db.Model):
__table__ = db.Model.metadata.tables['user']
def __repr__(self):
return '<User %r>' % (self.email)
Additional config.py
import os
basedir = os.path.abspath(os.path.dirname(__file__))
DEBUG = True
TESTING = False
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \
'sqlite:///' + os.path.join(basedir, 'data-dev.sqlite')
SQLALCHEMY_COMMIT_ON_TEARDOWN = True
SQLALCHEMY_TRACK_MODIFICATIONS = False
SECRET_KEY = 'a9eec0e0-23b7-4788-9a92-318347b9a39f'
From all the code provided above, what could be causing my application to produce this error and stopping it from running?
Thank you
I kept probing till i landed on this question and this one and they helped clear up the fog for me.
After installing sqlacodegen and flask-sqlacodegen using
pip install sqlacodegen
pip install flask-sqlacodegen
All i had to do now is run the following command in my Flask application root
sqlacodegen sqlite:///data-dev.sqlite --flask > models.py
And the models corresponding to my database were generated and then after i replaced my the bookshelf/data/models.py with the just generated models.py.
Then finally changed bookshelf/__init__.py create_app function to look like this
def create_app():
app = Flask(__name__)
app.config.from_object('config')
db.init_app(app)
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
And now everything seems to be working fine.
This is because the models.py file is executed before the tables are reflected. In bookshelf/__init__.py file, you're importing your controllers main and admin at the top, which in turn import your data models from bookshelf/data/models.py which subclasses the SQLAlchemy instance db. But the db.reflect() is yet to happen, therefore the metadata of the tables isn't reflecting and hence the key error. To solve this, either import your models in your controllers inside the functions, wherever needed, like this
bookshelf/main/controllers.py
from flask import Blueprint, render_template, request
main = Blueprint('main', __name__, template_folder='templates')
#main.route('/')
def index():
from bookshelf.data.models import Author, Book
return render_template('main/index.html')
#main.route('books/')
def display_books():
from bookshelf.data.models import Book
return render_template('main/books.html')
or import your controllers after you've initialized your db object like this,
bookshelf/init.py
from flask import Flask
from bookshelf.data.models import db
def create_app():
app = Flask(__name__)
app.config.from_object('config')
with app.app_context():
db.init_app(app)
db.Model.metadata.reflect(db.engine)
from bookshelf.admin.controllers import admin, main
app.register_blueprint(main, url_prefix='/')
app.register_blueprint(admin, url_prefix='/admin')
return app
After this, just add a separate database.py file to create the SQLAlchemy object instead of creating it in your models.py file, since it will cause a circular import issue if you have it in either bookshelf/init.py or bookshelf/data/models.py.
bookshelf/database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
Remove this line from your init.py file
from bookshelf.data.models import db
and instead get it in your init.py from database.py
from bookshelf.database import db
The metadata isn't there yet at the time you're importing bookshelf.data.models, as it's being reflected into the metadata only afterwards.
Maybe using __tablename__ instead of __table__ might help? I'm not that well versed in the intricacies of SQLAlchemy.
Related
Hello I am triying to create database and their tables with SQLAlchemy, in order to do this I have the following code and structure:
.
├── api
│ ├── api.py
│ ├── config.py
│ ├── connector.py
│ ├── detector
│ │ ├── __init__.py
│ │ └── route.py
│ └── user
│ ├── __init__.py
│ └── model.py
├── database.db
├── manager.py
├── queue
│ ├── __init__.py
│ └── queue.py
└── requirements.txt
Requirements:
dictalchemy~=0.1.2.7
Flask-RESTful
redis
Flask-SQLAlchemy
Flask-Script
Flask-Migrate
python-dotenv
SQLAlchemy~=1.3.18
Flask~=1.1.2
flask-cors
user.model
from sqlalchemy import Column
from sqlalchemy import Integer
from sqlalchemy import Text
from api.connector import BaseModel, engine
class User(BaseModel):
__tablename__ = 'users'
id = Column(
Integer,
name='id',
nullable=False,
primary_key=True,
autoincrement=True,
)
email = Column(
Text,
name='email',
)
Base.metadata.create_all(engine)
Connector
from contextlib import contextmanager
import functools
from dictalchemy import DictableModel
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
engine = create_engine("sqlite:///database.db", echo=True)
_db_session = sessionmaker(bind=engine)
BaseModel = declarative_base(cls=DictableModel)
Api
from flask import Flask
from flask_restful import Api
from api.detector.route import Detector
from flask_cors import CORS
app = Flask(__name__)
api = Api(app)
cors = CORS(app, resources={r"/api/*": {"origins": "*"}})
api.add_resource(Detector, '/api/v1.0/det')
Manager
from api.api import app
from flask_script import Manager
from api.connector import engine, BaseModel
from api.detector.route import Detector
manager = Manager(app)
app.config['DEBUG'] = True
#manager.command
def create_tables():
"Create relational database tables."
BaseModel.metadata.create_all(engine)
if __name__ == '__main__':
manager.run()
In order to create tables I use this python manager.py create_tables, that return:
2020-07-23 00:45:15,815 INFO sqlalchemy.engine.base.Engine SELECT
CAST('test plain returns' AS VARCHAR(60)) AS anon_1 2020-07-23
00:45:15,815 INFO sqlalchemy.engine.base.Engine () 2020-07-23
00:45:15,816 INFO sqlalchemy.engine.base.Engine SELECT CAST('test
unicode returns' AS VARCHAR(60)) AS anon_1 2020-07-23 00:45:15,816
INFO sqlalchemy.engine.base.Engine ()
The database file is created but this is empty.
Why the models are not loading on the database?
Thanks.
create_all() creates tables for the models it knows about. If it doesn't know about any, it's perfectly happy to make a database with an empty schema.
It appears as though create_all() runs before user.model has been imported. Fix that, and you should get tables.
I am confused about how to use Flask-Migrate when I have multiple models.
Basically my Flask app looks like this:
app
├── __init__.py
├── config.py
├── manage.py
├── migrations
├── models
│ ├── model1.py
│ ├── model2.py
├── resources
├── run.py
└── tests
I've read that for each model its best to create the db = SQLAlchemy() object in the file and then import this db object into the app's__init__.py like so:
from models.model1 import db
db.init_app(app)
from models.model2 import db
db.init_app(app)
However if I do this for multiple model files, how can I add Flasks's migrate functionality, considering I can only use 1 sql alchemy object for the migrate class instantiation:
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
Would it be best in this case to define a single sql alchemy object in the __init__.py file and import that into all my models?
You misread the referenced text. That talks about something completely different. That talks about keeping your db object separate from the app object (and tie the two togther in the create_app factory function). Creating multiple db objects is only complicating matters for you.
All that is needed is a single db = SQLAlchemy() object, and all the files that define models need to be imported. Usually that's done directly or indirectly via your create_app factory function, You need to call the create_app() function anyway to be able to run the flask db command-line tool anyway.
Next, you do not need to create a manager either. The Manager object is a hold-over from the time before the Flask project added support for scripts itself. If you are using Flask 0.12 or newer, you don't want to be using Flask-Script and it's manager.
So, all you need, in your __init_.py, is:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
db = SQLAlchemy()
def create_app(test_config=None):
app = Flask(__name__)
app.config.from_object(f"{__name__}.config")
app.config.from_envvar("PROJECTNAME_SETTINGS", silent=True)
if test_config:
app.config.from_mapping(test_config)
db.init_app(app)
Migrate(app, db)
# importing the models to make sure they are known to Flask-Migrate
from models import models1, models2
# any other registrations; blueprints, template utilities, commands
return app
I'm currently building an application with Flask. I'm struggling to access Flask instance 'app' as well as Flask-mail instance 'mail'.
Below is how my project looks like:
└── my-project
├── application
│ ├── __init__.py
│ ├── admin
│ │ ├── __init__.py
│ │ ├── forms.py
│ │ └── views.py
│ ├── auth
│ │ ├── __init__.py
│ │ ├── forms.py
│ │ └── views.py
│ │ └── token.py
│ │ └── email.py
│ ├── home
│ │ ├── __init__.py
│ │ └── views.py
│ ├── models.py
│ ├── static
│ └── templates
│ └──....
│
├── config.py
├── instance
│ └── config.py
├── migrations
│ ├── README
│ ├── alembic.ini
│ ├── env.py
│ ├── script.py.mako
│ └── versions
│ └── a1a1d8b30202_.py
├── requirements.txt
└── run.py
Flask instance is created in run.py with create_app function (from
run.py
import os
from application import create_app
config_name = os.getenv('FLASK_CONFIG')
app = create_app(config_name)
if __name__ == '__main__':
app.run()
application/__init__.py
# third-party imports
from flask import Flask, render_template
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
from flask_migrate import Migrate
from flask_bootstrap import Bootstrap
from flask_mail import Mail
import stripe
# local imports
from config import app_config
# db variable initialization
db = SQLAlchemy()
login_manager = LoginManager()
LoginManager.user_loader
def create_app(config_name):
app = Flask(__name__, instance_relative_config=True)
app.config.from_object(app_config[config_name])
app.config.from_pyfile('config.py')
Bootstrap(app)
db.init_app(app)
login_manager.init_app(app)
mail = Mail(app)
migrate = Migrate(app,db)
from application import models
from .admin import admin as admin_blueprint
app.register_blueprint(admin_blueprint, url_prefix='/admin')
#the rest of the blueprint import goes here
return app
What I want to do is to use 'app' and 'mail'. For example, in application/auth/token:
application/auth/token.py
from itsdangerous import URLSafeTimedSerializer
from . import auth
def generate_confirmation_token(email):
serializer = URLSafeTimedSerializer(app.config['SECRET_KEY'])
return serializer.dumps(email, salt=app.config['SECURITY_PASSWORD_SALT'])
def confirm_token(token, expiration = 600):
serializer = URLSafeTimedSerializer(app.config['SECRET_KEY'])
try:
email = serializer.loads(
token,
salt=app.config['SECURITY_PASSWORD_SALT'],
max_age=expiration
)
except:
return False
return email
or in application/auth/email.py:
application/auth/email.py
from flask_mail import Message
from . import auth
def send_mail(to, subject, template):
msg = Message(
subject,
recipients=[to],
html=template,
sender=app.config['MAIL_DEFAULT_SENDER']
)
mail.send(msg)
I need both of these function in application/aut/views.py
application/auth/views.py
from flask import flash, redirect, render_template, url_for, request
from flask_login import login_required, login_user, logout_user
from werkzeug.security import check_password_hash
import datetime
from . import auth
from forms import LoginForm, RegistrationForm
from .. import db
from ..models import User
#auth.route('/register', methods=['GET', 'POST'])
def register():
"""
Handle requests to the /register route
Add a user to the database through the registration form
"""
form = RegistrationForm()
form.id = 'form_signup'
if form.validate_on_submit():
user = User(email=form.email.data,
#username=form.username.data,
first_name=form.first_name.data,
last_name=form.last_name.data,
password=form.password.data,
registered_on=datetime.datetime.now(),
confirmed=False,
premium=False)
# add employee to the database
db.session.add(user)
db.session.commit()
flash("We've just sent you an email confirmation. Please activate you account to completly finish your registration", 'succes')
token = generate_confirmation_token(user.email)
confirm_url = url_for('auth.confirm_email', token=token, _external=True)
html = render_template('auth/activate.html', confirm_url=confirm_url)
subject = "Please confirm your email"
send_email(user.email, subject, html)
login_user(user)
flash('A confirmation email has been sent via email.', 'success')
# redirect to the login page
#return redirect(url_for('auth.login'))
return redirect(url_for('home.homepage'))
# load registration template
return render_template('auth/register.html', form=form, title='Register')
#auth.route('/confirm/<token>')
#login_required
def confirm_email(token):
try:
email = confirm_token(token)
except:
flash('The confirmation link is invalid or has expired.', 'danger')
user = User.query.filter_by(email=email).first_or_404()
if user.confirmed:
flash('Account already confirmed. Please login.', 'succes')
else:
user.confirmed =True
user.confirmed_on = datetime.datetime.now()
db.session.add(user)
db.session.commit()
flash("You've confirmed your account. Thanks!", 'succes')
return redirect(url_for('auth.login'))
What I get is an error 'global name app is not defined' or 'global name mail is not defined'. I tried to import the variable with from application import app which return me an import error 'cannot import app'
Thanks for your support
Since you are using an application factory you need to use the .init_app method on the Flask-Mail class like you did the Flask-SQLAlchemy class. from application import app will not work since you are never initializing an instance of the flask application until you call the create_app function in run.py
application/__init__.py
from flask_mail import Mail
mail = Mail()
def create_app(config_lvl):
# stuff
mail.init_app(app)
# more stuff
return app
Also you can use current_app to refer to the application instance instead of the instance itself as long as the block of code you use it in is being ran in a flask application. Here is a more in depth explanation.
application/auth/email.py
from application import mail # you can now import the Mail() object
from flask_mail import Message
from flask import current_app # use this to reference current application context
def send_email(to, subject, template):
msg = Message(
subject,
recipients=[to],
html=template,
sender=current_app.config['MAIL_DEFAULT_SENDER']
)
mail.send(msg)
application/auth/token.py
from itsdangerous import URLSafeTimedSerializer
from flask import current_app
def generate_confirmation_token(email):
serializer = URLSafeTimedSerializer(current_app.config['SECRET_KEY'])
return serializer.dumps(email, salt=current_app.config['SECURITY_PASSWORD_SALT'])
def confirm_token(token, expiration = 600):
serializer = URLSafeTimedSerializer(current_app.config['SECRET_KEY'])
try:
email = serializer.loads(
token,
salt=current_app.config['SECURITY_PASSWORD_SALT'],
max_age=expiration
)
except:
return False
return email
also should note that you don't need from . import auth in any of your modules under the auth blueprint except views.py
EDIT
Side note: You don't have to add the user to the session because it was added when you queried for it earlier in the route. I was unaware of this for the longest time myself.
#auth.route('/confirm/<token>')
#login_required
def confirm_email(token):
try:
email = confirm_token(token)
except:
flash('The confirmation link is invalid or has expired.', 'danger')
user = User.query.filter_by(email=email).first_or_404()
if user.confirmed:
flash('Account already confirmed. Please login.', 'success')
else:
user.confirmed = True
user.confirmed_on = datetime.datetime.now()
# db.session.add(user) # can remove this
db.session.commit()
flash("You've confirmed your account. Thanks!", 'success')
return redirect(url_for('auth.login'))
According to http://flask.pocoo.org/docs/0.12/patterns/appfactories/ you have to use current_app (from flask import current_app) to access your Flask instance for wherever in your application.
Then you can use your config variables: current_app.config['DEBUG'] for instance.
Another point to consider when using application factories, create instance class instances outside of the 'create_app()' function. Then you have to use .init_app method:
Below is an example:
from flask_mail import Mail
mail= Mail()
def create_app():
mail.init_app(app)
I am following the book Mastering flask's recommended file structure.
(The name of my project is Paw)
In Paw/paw/__init__.py:
def create_app(object_name):
app = Flask(__name__)
app.config.from_object(object_name)
db.init_app(app)
robot = LoggingWeRobot(token='banana', enable_session=True)
robot.init_app(app, endpoint='werobot', rule='/wechat')
attach_debugging_logger(app)
app.register_blueprint(poll_blueprint)
app.register_blueprint(wechat_blueprint)
return app
Note that the robot variable is actually needed in my blueprint, wechat, found in: Paw/paw/controllers/wechat.py
#robot.handler
def request_logging_middleware(message, session):
app.logger.debug("\n%s", request.data)
return False # This allows other handlers to continue execution
So my problem is that my blueprint has no access to the robot variable. However, the robot variable should be created in create_app in Paw/paw/__init__.py because I am trying to follow the application factory pattern.
Any recommendation on how to fix this? My project can be found here and I am trying to follow this application structure
Simply use the same pattern you are using for db - create robot elsewhere and import it into your Paw/paw/__init__.py file, just as you do with db:
import db from models
import robot from wechat_setup
# wechat_setup is where you would invoke
# robot = LoggingWeRobot(token='banana', enable_session=True)
def create_app(object_name):
app = Flask(__name__)
app.config.from_object(object_name)
db.init_app(app)
robot.init_app(app, endpoint='werobot', rule='/wechat')
I usually put project global variables in one file (say gvars.py).
Then the project structure will be some sort like this:
.
├── etc
│ └── webapp.py
├── models
│ └── common.py
├── views
│ └── common.py
├── gvars.py
└── webapp.py
In other files we just do this:
from gvars import db, robot # or other variables
How can I register a Flask-Admin BaseView as a module in my application? every time I run my app I get a blueprint collision error!
I also know about ModelView in Flask-Admin, but I want to separate models and views from each other.
init.py
from flask import Flask
import flask_admin as admin
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db = SQLAlchemy(app)
from views.user import user_view, UserView
admin = admin.Admin(app, name='Backend')
user_view.add_view(UserView)
db.create_all()
Package Folder Backend
├── __init__.py
├── models.py
├── static
├── templates
│ └── user
│ └── index.html
└── views
├── __init__.py
└── user.py
models.py
from . import db
class UserModel(db.Model):
'__tablename__' == "User"
id = db.Column(db.Integer, primary_key=True)
first_name = db.Column(db.String(100))
last_name = db.Column(db.String(100))
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
# Required for administrative interface. For python 3 please use __str__ instead.
def __unicode__(self):
return self.username
user.py
from flask_admin import Admin, BaseView, expose
from Backend import app
user_view = Admin(app, name="User")
class UserView(BaseView):
#expose('/')
def index(self):
return self.render('user/index.html')
So I answer my own question. it was just a fallacy.
I just need to import the UserView as described here. And also need to import the package app in the view.
So here is the relation between __init__.py and views/user.py.
init.py
from flask import Flask
import flask_admin as admin
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db = SQLAlchemy(app)
from views.user import UserView
admin = admin.Admin(app, name='Backend')
admin.add_view(UserView(name="User"))
db.create_all()
views/user.py
from Backend import app
from flask_admin import BaseView, expose
class UserView(BaseView):
#expose('/')
def index(self):
return self.render('user/index.html')
This part from Flask Documentaion: was interessting.
Circular Imports:
Every Python programmer hates them, and yet we just added some:
circular imports (That’s when two modules depend on each other. In
this case views.py depends on init.py). Be advised that this is a
bad idea in general but here it is actually fine. The reason for this
is that we are not actually using the views in init.py and just
ensuring the module is imported and we are doing that at the bottom of
the file.