I am using flask_sqlalchemy to manage a database for a small web app. I have been using sqlite locally, but am now migrating to a postgresql database as it is the case in production environment.
Currently, the app has some logic where if the database does not exist, I read in some csv files and populate the tables:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
path_to_db = 'data/database.db'
db = SQLAlchemy()
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = f'sqlite:///{path_to_db}'
db.init_app(app)
if not path.exists(path_to_db):
db.create_all(app)
# Read in csv files and add to database
Now I have created a postgresql database called database using pgAdmin and changed the code to:
path_to_db = 'database'
#...
app.config['SQLALCHEMY_DATABASE_URI'] = f'postgresql://postgres:postgres#localhost:5432/{path_to_db}'
# >> How do I check whether this database is empty?
# if not path.exists(path_to_db):
# Read in csv files and add to database
In the original sqlite code, it creates a database called database.db which I then check if exists. If it doesn't I create one, and read in csv files. However I am confused about on how to implement this logic with postgresql, and how to check if the database is empty.
Any suggestions?
Related
I have built two flask app one for business (APP 1) and another one for admin (APP 2). Both APP 1 and APP 2 share same database and tables except APP 2 is using few subscriptions related tables which are not present in APP 1. Now the problem is whenever i execute SQL Alchemy migrate command in APP 1 it generate drop table syntax for subscriptions tables because those table models are not present in APP 1.
What is the best way to deal with two flask app share same database by using SQL Alchemy migrate?
You can define the tables to be excluded in app1 in you app1's alembic.ini file, example is below:
[alembic:exclude]
tables = table1,table2
Then in you env.py file, you can in include_object function.
include_object is a callable function which is given the chance to return True or False for any object, indicating if the given object should be considered in the autogenerate sweep.
https://alembic.sqlalchemy.org/en/latest/api/runtime.html?highlight=include_object#alembic.runtime.environment.EnvironmentContext.configure.params.include_object
Code in env.py file
exclude_tables = config.get_section('alembic:exclude').get('tables', '').split(',')
def include_object(object, name, type_, reflected, compare_to):
if type_ == "table" and name in exclude_tables:
return False
else:
return True
exclude_tables will contain the list of all the tables, that you have defined in your alembic.ini file
Then include this include_object function in your context.configure function as below:
context.configure(
# ...
include_object = include_object
)
I am new to flask and python in general. I just need help accessing an already existing snowflake database with flask. I just want to query the data. This is my code thus far and its not working:
from flask import Flask, render_template, request, redirect, url_for
from model import InputForm
from compute import preprocess
from sqlalchemy import create_engine, MetaData, Table
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABSE_URI'] = 'snowflake://<user_login_name>:<password>#<account_name>'
db = SQLAlchemy(app)
engine = create_engine()
metadata = MetaData(bind=engine)
engagements = db.Table('ENGAGEMENTS', db.metadata, autoload=True, autoload_with=db.engine)
companies = db.Table('COMPANIES', db.metadata, autoload=True, autoload_with=db.engine)
#app.route('/')
def index():
results = db.session.query(engagements).all()
for r in results:
print(engagements)
return ''
if __name__ == '__main__':
app.run(debug=True)
There may be multiple things going on here.
First, if the error message you get is due to non-existent engagements table in the Snowflake db, it may be exactly that. Before you can execute queries, you need to bring your database at par with the models in your code. https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database is a great source on how to set up your db. It teaches how to set up a normal sqlite database for migrations using Alembic. Luckily snowflake-sqlalchemy has alembic support: https://github.com/snowflakedb/snowflake-sqlalchemy#alembic-support . You can use them to build and export migrations, and your db should be ready.
Second, there's two typos: one in your code, one in the documentation. Your code sets app.config['SQLALCHEMY_DATABSE_URI'] instead of app.config['SQLALCHEMY_DATABASE_URI']. And you also definitely need to mention your db and schema for snowflake as pointed out in https://stackoverflow.com/a/59204661/2928486. But the official documentation has incorrect uri.
'snowflake://<user_login_name>:<password>#<account_name>/<database_name>/<schema_name>?warehouse=<warehouse_name>?role=<role_name>'
Instead of the above, it should be the following:
'snowflake://<user_login_name>:<password>#<account_name>/<database_name>/<schema_name>?warehouse=<warehouse_name>&role=<role_name>'
Basically your connection params should be joined using & instead of ?. I was able to set up my flask application using your code, and it is migrating and updating the snowflake db fine, so hopefully it works for other ddl/dql scenarios as well!
I'm not a flask user, but it is possible you need to include the database and schema. I found a URL example below that includes snowflake account, database, schema, warehouse and role.
'snowflake://<user_login_name>:<password>#<account_name>/<database_name>/<schema_name>?warehouse=<warehouse_name>?role=<role_name>'
Our application uses flask + gunicorn. Now we want to make it able to reload db configuration while it is runing, which means it can switch to a new db without restart process. With the help of config center we can dispatch config at runtime, but how can I re-init the global varibale db?
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
app.config.from_object(dynamic_config)
db.init_app(app)
And assume at some time. We dispatch new config, how can db be init with new config? Or is it safe to just replace it with new SQLAlchemy() instance? Like do this:
from models import set_db # which will set global db to new instance
from app import app
def callback(odl, new):
new_db = SQLAlchemy()
# re-construct config with old, and new
# now app.config is updated
new_db.init_app(app)
set_db(new_db)
Is it ok to do this? As I'm concerned, it will cause something like thread safety and may destroy Transaction.
Help me with this, many thanks
If i were you I would use SQLALCHEMY_BINDS config or different instances of db with different configs instead of changing configuration every time.
and I think it isn't good practice to change db structure using application(in case you are doing that)
I'm trying to test my flask application using unittest. I want to refrain from flask-testing because I don't like to get ahead of myself.
I've really been struggling with this unittest thing now. It is confusing because there's the request context and the app context and I don't know which one I need to be in when I call db.create_all().
It seems like when I do add to the database, it adds my models to the database specified in my app module (init.py) file, but not the database specified in the setUp(self) method.
I have some methods that must populate the database before every test_ method.
How can I point my db to the right path?
def setUp(self):
#self.db_gd, app.config['DATABASE'] = tempfile.mkstemp()
app.config['TESTING'] = True
# app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + app.config['DATABASE']
basedir = os.path.abspath(os.path.dirname(__file__))
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + \
os.path.join(basedir, 'test.db')
db = SQLAlchemy(app)
db.create_all()
#self.app = app.test_client()
#self.app.testing = True
self.create_roles()
self.create_users()
self.create_buildings()
#with app.app_context():
# db.create_all()
# self.create_roles()
# self.create_users()
# self.create_buildings()
def tearDown(self):
#with app.app_context():
#with app.request_context():
db.session.remove()
db.drop_all()
#os.close(self.db_gd)
#os.unlink(app.config['DATABASE'])
Here is one of the methods that populates my database:
def create_users(self):
#raise ValueError(User.query.all())
new_user = User('Some User Name','xxxxx#gmail.com','admin')
new_user.role_id = 1
new_user.status = 1
new_user.password = generate_password_hash(new_user.password)
db.session.add(new_user)
Places I've looked at:
http://kronosapiens.github.io/blog/2014/08/14/understanding-contexts-in-flask.html
http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvi-debugging-testing-and-profiling
And the flask documentation:
http://flask.pocoo.org/docs/0.10/testing/
one issue that your hitting is the limitations of flask contexts, this is the primary reason i think long and hard before including a flask extension into my project, and flask-sqlalchemy is one of the biggest offenders. i say this because in most cases it is completely unnecessary to depend on the flask app context when dealing with your database. Sure it can be nice, especially since flask-sqlalchemy does a lot behind the scenes for you, mainly you dont have to manually manage your session, metadata or engine, but keeping that in mind those things can easily be done on your own, and for doing that you get the benefit of unrestricted access to your database, with no worry about the flask context. here is an example of how to setup your db manually, first i will show the flask-sqlalchemy way, then the manual plain sqlalchemy way:
the flask-sqlalchemy way:
import flask
from flask_sqlalchemy import SQLAlchemy
app = flask.Flask(__name__)
db = SQLAlchemy(app)
# define your models using db.Model as base class
# and define columns using classes inside of db
# ie: db.Column(db.String(255),nullable=False)
# then create database
db.create_all() # <-- gives error if not currently running flask app
the standard sqlalchemy way:
import flask
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
# first we need our database engine for the connection
engine = sa.create_engine(MY_DB_URL,echo=True)
# the line above is part of the benefit of using flask-sqlalchemy,
# it passes your database uri to this function using the config value
# SQLALCHEMY_DATABASE_URI, but that config value is one reason we are
# tied to the application context
# now we need our session to create querys with
Session = sa.orm.scoped_session(sa.orm.sessionmaker())
Session.configure(bind=engine)
session = Session()
# now we need a base class for our models to inherit from
Model = declarative_base()
# and we need to tie the engine to our base class
Model.metadata.bind = engine
# now define your models using Model as base class and
# anything that would have come from db, ie: db.Column
# will be in sa, ie: sa.Column
# then when your ready, to create your db just call
Model.metadata.create_all()
# no flask context management needed now
if you set your app up like that, any context issues your having should go away.
as a separate answer, to actually just force what you need to work, you can just use the test_request_context function, ie: in setup do: self.ctx = app.test_request_context() then just activate it, self.ctx.push() and when your done get rid of it, ie in tearDown: self.ctx.pop()
I am building my first app with Flask Python micro-framework and I have a problem with committing my models to the database. When I test my User model on the command line, all works well. But when I do a db.session.commit(), I have error 1146 : "Table doesn't exist."
I'm using a MySQL database in local mode and there is no error with login/password
Maybe I'm doing it wrong on the configuration or something else. So here is my config application config file
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:admin#localhost/flask_db'
db = SQLAlchemy(app)
from app import views
The error explains it all-- while you may have the models for your data, you haven't yet created the tables in the database to store and query them. Simply import your models then run db.create_all() to generate the tables and you should be good to go.
It'll be worth you reading the quickstart guide for Flask-SQLAlchemy to get your head around the general flow.