I have a Flask app that uses Flask-SQLAlchemy and I'm trying to configure it to use multiple databases with the Flask-Restless package.
According to the docs, configuring your models to use multiple databases with __bind_key__ seems pretty straightforward.
However it doesn't seem to be working for me.
I create my app and initialise my database like this:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
SQLALCHEMY_DATABASE_URI = 'postgres://db_user:db_pw#localhost:5432/db_name'
SQLALCHEMY_BINDS = {
'db1': SQLALCHEMY_DATABASE_URI,
'db2': 'mysql://db_user:db_pw#localhost:3306/db_name'
}
app = Flask(__name__)
db = SQLALchemy(app)
Then define my models including __bind_key__, which should tell SQLAlchemy which DB it needs to use:
class PostgresModel(db.Model):
__tablename__ = 'postgres_model_table'
__bind_key__ = 'db1'
id = db.Column(db.Integer, primary_key=True)
...
class MySQLModel(db.Model):
__tablename__ = 'mysql_model_table'
__bind_key__ = 'db2'
id = db.Column(db.Integer, primary_key=True)
...
Then I fire up Flask-Restless like this:
manager = restless.APIManager(app, flask_sqlalchemy_db=db)
manager.init_app(app, db)
auth_func = lambda: is_authenticated(app)
manager.create_api(PostgresModel,
methods=['GET'],
collection_name='postgres_model',
authentication_required_for=['GET'],
authentication_function=auth_func)
manager.create_api(MySQLModel,
methods=['GET'],
collection_name='mysql_model',
authentication_required_for=['GET'],
authentication_function=auth_func)
The app runs fine and when I hit http://localhost:5000/api/postgres_model/[id] I get the expected JSON response of the object from the Postgres DB (I'm guessing this is because I have it's credentials in SQLALCHEMY_DATABASE_URI).
Although when I hit http://localhost:5000/api/mysql_model/[id], I get a mysql_model_table does not exist error, indicating that it's looking in the Postgres DB, not the MySQL one.
What am I doing wrong here?
This was not working because of a simple typo:
__bind_key = 'db1'
Should have been
__bind_key__ = 'db1'
I've updated the original question and fixed the typo as an example of how this can work for others.
Related
My app.py file
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgres:////tmp/test.db'
db = SQLAlchemy(app) # refer https://flask-sqlalchemy.palletsprojects.com/en/2.x/api/#flask_sqlalchemy.SQLAlchemy
One of my model classes, where I imported db
from app import db
Base = declarative_base()
# User class
class User(db.Model, Base):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
def __repr__(self):
return '<User %r>' % self.username
def get_user_by_id(self, id):
return self.query.get(id)
My database has the same set of tables in different schema (multi-tenancy) and there
I need to select the schema as per the request initiated by a particular tenant on the fly by using before_request (grabbing tenant_id from subdomain URL).
I found Postgres provides selecting the schema name on fly by using
schema_translate_map ref. https://docs.sqlalchemy.org/en/14/core/connections.html#translation-of-schema-names and that is under execution_options https://docs.sqlalchemy.org/en/14/core/connections.html#sqlalchemy.engine.Connection.execution_options
In my above code snippet where you see db = SQLAlchemy(app), as per official documentation, two parameters can be set in SQLAlchemy objct creation and they are - session_options and engine_options, but no execution_options ref. https://flask-sqlalchemy.palletsprojects.com/en/2.x/api/#flask_sqlalchemy.SQLAlchemy
But how do I set schema_translate_map setting when I am creating an object of SQLAlchemy
I tried this -
db = SQLAlchemy(app,
session_options={
"autocommit": True,
"autoflush": False,
"schema_translate_map": {
None: "public"
}
}
)
But obviously, it did not work, because schema_translate_map is under execution_options as mentioned here https://docs.sqlalchemy.org/en/14/core/connections.html#translation-of-schema-names
Anyone has an idea, how to set schema_translate_map at the time of creating SQLAlchemy object.
My goal is to set it dynamically for each request. I want to control it from this
centralized place, rather than going in each model file and specifying it when I execute
queries.
I am aware of doing this differently as suggested here https://stackoverflow.com/a/56490246/1560470
but my need is to set somewhere around db = SQLAlchemy(app) in app.py file only. Then after I import db in all my model classes (as shown above) and in those model classes, all queries execute under the selected schema.
I found a way to accomplish it. This is what needed
db = SQLAlchemy(app,
session_options={
"autocommit": True,
"autoflush": False
},
engine_options={
"execution_options":
{
"schema_translate_map": {
None: "public",
"abc": "xyz"
}
}
}
)
I am trying to create a FAST Api that is reading from an already existing table in PostgreSQL database but it is giving me an internal server error. Would appreciate your direction on what might be wrong with the code
The existing table looks like this:
schema : testSchema
table : test_api
id
email
1
test#***.com
2
test2#***.com
engine = sqlalchemy.create_engine("my_database_connection")
Base = declarative_base()
database = databases.Database("my_database_connection")
metadata = sqlalchemy.MetaData()
metadata.reflect(bind=engine, schema='testSchema')
test_api_tb = metadata.tables['testSchema.test_api']
class testAPI(Base):
__tablename__ = test_api_tb
id = Column(Integer, primary_key=True)
email = Column(String(256))
app = FastAPI()
#app.get("/testing_api/")
def read_users():
query = test_api_tb.select()
return database.execute(query)
The error I am getting from the logs
RecursionError: maximum recursion depth exceeded in comparison
The best thing you can do is to read the official documentation at fastapi.tiangolo.com, it is amazing and explains all the basics in a very detailed way.
SQL Relational Databases are used very often with FastAPI and are also mentioned in the documentation here, you can find step by step tutorial about how to use postgresql with sqlalchemy and FastAPI.
There are a few parts to make this work. The first part is to connect to the database:
engine = create_engine(my_database_connection)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()
we create the engine with the connection string as you did, then we need to create a session in order to connect to the database. At the end we are creating a Base class which will help us to create the models and schemas.
Now we need to create the model using the base class just as you did above.
we need to make sure that the __tablename__ is the same as the name of the table in the database
class testAPIModel(Base):
__tablename__ = "test_api"
id = Column(Integer, primary_key=True)
email = Column(String(256))
Now comes the main part. We need to make sure we bind the engine of the database to the base class using
Base.metadata.create_all(bind=engine)
Now we will create a function that will help us and create a db session instance and will close the connection when we done with the query.
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
Now we can create the FastAPI app instance and get the data from the database.
#app.get("/testing_api/")
def read_users(db:Session = Depends(get_db)):
users = db.query(testAPIModel).all()
return users
We are using the Depends(get_db) to inject the db session from the function we wrote above.
The full code:
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session
from fastapi import Depends, FastAPI
from sqlalchemy import Column, Integer, String
my_database_connection = "postgresql://user:password#server_ip/db_name"
engine = create_engine(my_database_connection)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()
class testAPIModel(Base):
__tablename__ = "test_api"
id = Column(Integer, primary_key=True)
email = Column(String(256))
Base.metadata.create_all(bind=engine)
app = FastAPI()
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
#app.get("/testing_api/")
def read_users(db:Session = Depends(get_db)):
users = db.query(testAPIModel).all()
return users
Good Luck!
I am trying to populate a database with two tables in SQLAlchemy. I have already created the database test_database and now I am trying to create 2 tables inside this. I have already checked that this database is successfully created using \l. Following is the code for the file create.py which creates a database with two tables:
import os
from flask import Flask, render_template, request
from models import *
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = 'postgresql://shammun:my_password#localhost:5432/test_database.db'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
# db = SQLAlchemy()
db.init_app(app)
def main():
db.create_all()
if __name__ == "__main__":
with app.app_context():
main()
This file create.py imports model.py which generates two tables, the code of which is given below:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Flight(db.Model):
__tablename__ = "flights"
id = db.Column(db.Integer, primary_key=True)
origin = db.Column(db.String, nullable=False)
destination = db.Column(db.String, nullable=False)
duration = db.Column(db.Integer, nullable=False)
class Passenger(db.Model):
__tablename__ = "passengers"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False)
flight_id = db.Column(db.Integer, db.ForeignKey("flights.id"), nullable=False)
Now, in the terminal when I run the file create.py, I get the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: database "test_database.db" does not exist
Actually, this is almost the same question that I asked in this post db.create_all() doesn't create a database a month ago. The only difference is that I asked the wrong question that database was not created. Actually the question would be why the database wasn't found and why would it throw an error. As this question was closed and as I have tried so many times for a long time to resolve it and couldn't find any solution, I am asking almost the same question again. I will much appreciate if someone can help me to lift me from this bottleneck where I am stuck for a long time.
Check on what port is your postgres running using this command \conninfo
cause I doubt your PostgreSQL database is running on some different port.
Default port of PostgreSQL is 5432, but if it is already occupied by some other application then it tries next empty port and starts running on 5433
So in your app config variable of SQLALCHEMY_DATABASE_URI, try changing 5432 to 5433 and see if it works.
Edit 1:
Try removing .db from your database name test_database.db, and just put test_database
Change this:
app.config["SQLALCHEMY_DATABASE_URI"] = 'postgresql://shammun:my_password#localhost:5432/test_database.db'
To this:
app.config["SQLALCHEMY_DATABASE_URI"] = 'postgresql://shammun:my_password#localhost:5432/test_database'
The SQLAlchemy provides the Connection.execution_options.schema_translate_map for change the schemas in execution time, as said in docs.
In the examples is shown how to use to perform queries, but want to know how to use it with create_all().
I'm using Flask-Sqlaclhemy and postgresql as database. Let's say I have this:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
app = Flask(...)
...
db.init_app(app)
...
return app
class User(db.Model):
__tablename__ = 'user'
__table_args__ = {'schema':'public'}
company = db.Column(db.String(10))
class SomePublicModel(db.Model):
__tablename__ = 'some_public'
__table_args__ = {'schema':'public'}
...
class SomeModelByDynamicSchema(db.Model):
__tablename__ = 'some_dynamic'
__table_args__ = {'schema':'dynamic'}
...
The dynamic schema will be replace for other value according the user's company in execution time.
Assuming I already have in database the schemas public and dynamic and a I want to create a new schema with the tables, something like this:
def create_new():
user = User(company='foo')
db.session.execute("CREATE SCHEMA IF NOT EXISTS %s" % user.company)
db.session.connection().execution_options(schema_translate_map={'dynamic':user.company})
#I would like to do something of the kind
db.create_all()
I expected the tables to be created in the foo schema as foo.some_dynamic, but the SQLAlchemy still try to create in dynamic schema.
Can someone help me?
When you set execution options, you create copy of connection. This mean what create_all run without schema_translate_map.
>>> c = Base.session.connection()
>>> w = c.execution_options(schema_translate_map={'dynamic':'kek'})
>>> c._execution_options
immutabledict({})
>>> w._execution_options
immutabledict({'schema_translate_map': {'dynamic': 'kek'}})
to achieve your goal, you could try another approach.
get the tables from old grammmar and adapt for new metadata.
metadata = MetaData(bind=engine, schema=db_schema)
for table in db.Model.metadata.tables.values():
table.tometadata(metadata)
metadata.drop_all()
metadata.create_all()
I had a small web service built using Flask and Flask-SQLAlchemy that only held one model. I now want to use the same database, but with a command line app, so I'd like to drop the Flask dependency.
My model looks like this:
class IPEntry(db.Model):
id = db.Column(db.Integer, primary_key=True)
ip_address = db.Column(db.String(16), unique=True)
first_seen = db.Column(db.DateTime(),
default = datetime.datetime.utcnow
)
last_seen = db.Column(db.DateTime(),
default = datetime.datetime.utcnow
)
#validates('ip')
def validate_ip(self, key, ip):
assert is_ip_addr(ip)
return ip
Since db will no longer be a reference to flask.ext.sqlalchemy.SQLAlchemy(app), how can I convert my model to use just SQLAlchemy. Is there a way for the two applications (one with Flask-SQLAlchemy the other with SQLAlchemy) to use the same database?
you can do this to replace db.Model:
from sqlalchemy import orm
from sqlalchemy.ext.declarative import declarative_base
import sqlalchemy as sa
base = declarative_base()
engine = sa.create_engine(YOUR_DB_URI)
base.metadata.bind = engine
session = orm.scoped_session(orm.sessionmaker())(bind=engine)
# after this:
# base == db.Model
# session == db.session
# other db.* values are in sa.*
# ie: old: db.Column(db.Integer,db.ForeignKey('s.id'))
# new: sa.Column(sa.Integer,sa.ForeignKey('s.id'))
# except relationship, and backref, those are in orm
# ie: orm.relationship, orm.backref
# so to define a simple model
class UserModel(base):
__tablename__ = 'users' #<- must declare name for db table
id = sa.Column(sa.Integer,primary_key=True)
name = sa.Column(sa.String(255),nullable=False)
then to create the tables:
base.metadata.create_all()
That is how to use SQLAlchemy without Flask (for example to write a bulk of objects to PostgreSQL database):
from sqlalchemy import Column, Integer, String
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
# Define variables DB_USERNAME, DB_PASSWORD, DB_HOST, DB_PORT, DB_NAME
SQLALCHEMY_DATABASE_URI = f'postgresql://{DB_USERNAME}:{DB_PASSWORD}#{DB_HOST}:
{DB_PORT}/{DB_NAME}'
# ----- This is related code -----
engine = create_engine(SQLALCHEMY_DATABASE_URI, echo=True)
Base = declarative_base()
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
Session.configure(bind=engine)
session = Session()
# ----- This is related code -----
class MyModel(Base):
__tablename__ = 'my_table_name'
id = Column(Integer, primary_key=True)
value = Column(String)
objects = [MyModel(id=0, value='a'), MyModel(id=1, value='b')]
session.bulk_save_objects(objects)
session.commit()
Check this one github.com/mardix/active-alchemy
Active-Alchemy is a framework agnostic wrapper for SQLAlchemy that makes it really easy to use by implementing a simple active record like api, while it still uses the db.session underneath. Inspired by Flask-SQLAlchemy
There is a great article about Flask-SQLAlchemy: how it works, and how to modify models to use them outside of Flask:
http://derrickgilland.com/posts/demystifying-flask-sqlalchemy/
The sqlalchemy docs has a good tutorial with examples that sound like what you want to do.
Shows how to connect to a db, mapping, schema creation, and querying/saving to the db.
This does not completely answer your question, because it does not remove Flask dependency, but you can use SqlAlchemy in scripts and tests by just not running the Flask app.
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import MetaData
test_app = Flask('test_app')
test_app.config['SQLALCHEMY_DATABASE_URI'] = 'database_uri'
test_app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
metadata = MetaData(schema='myschema')
db = SQLAlchemy(test_app, metadata=metadata)
class IPEntry(db.Model):
pass
One difficulty you may encounter is the requirement of using db.Model as a base class for your models if you want to target the web app and independent scripts using same codebase. Possible way to tackle it is using dynamic polymorphism and wrap the class definition in a function.
def get_ipentry(db):
class IPEntry(db.Model):
pass
return IPEntry
As you construct the class run-time in the function, you can pass in different SqlAlchemy instances. Only downside is that you need to call the function to construct the class before using it.
db = SqlAlchemy(...)
IpEntry = get_ipentry(db)
IpEntry.query.filter_by(id=123).one()
Flask (> 1.0) attempt to provide helpers for sharing code between an web application and a command line interface; i personally think it might be cleaner, lighter and easier to build libraries unbound to flask, but you might want to check:
https://flask.palletsprojects.com/en/2.1.x/cli/
https://flask.palletsprojects.com/en/2.1.x/api/#flask.Flask.cli
Create database and table
import os
from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
if os.path.exists('test.db'):
os.remove('test.db')
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
id = Column(Integer(), primary_key=True)
name = Column(String())
engine = create_engine('sqlite:///test.db')
Base.metadata.create_all(engine)
Using Flask_SQLAlchemy directly
from flask import Flask
from sqlalchemy import MetaData
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import Column, Integer, String
app = Flask(__name__)
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db = SQLAlchemy(app, metadata=MetaData())
class Person(db.Model):
__tablename__ = 'person'
id = Column(Integer(), primary_key=True)
name = Column(String())
person = Person(name='Bob')
db.session.add(person)
db.session.commit()
print(person.id)