Develop Reference

Do you still need to create SQLAlchemy models if the DB already exists?

New to Flask and SQL - teaching myself to build a fairly basic web app.
The lessons that I have followed so far haven't made it clear to me how a pre-existing SQLite database interacts with SQLAlchemy.
I have created an SQLite database using SQL in my terminal which already has the tables set up that I want to use.
I am now looking to connect it to my Flask app using SQLAlchemy. For examples sake lets say this is the model of a Book table in my DB has the following SQL setup:
CREATE TABLE "Book" (
"isbn" INTEGER NOT NULL,
"title" TEXT NOT NULL,
"author" TEXT NOT NULL,
PRIMARY KEY("id" AUTOINCREMENT)
)
Is the purpose of outlining the model in the Flask app like below just so that the application knows how to interact with the DB?
class Book(db.Model):
isbn = db.column(db.Integer, primary_key = True)
title = db.column(db.String(50), unique = False, index = True)
author = db.column(db.String(50), unique = True, index = True)
And given the double up in almost all of the information that is presented - is there a more efficient way to import a model like this from a database that is already set up?

You can use a technique called reflection to generate declarative SQLAlchemy classes based on existing tables in your DB.
For example:
"""Example of table reflection with SQLAlchemy"""
from sqlalchemy import Table, create_engine, text
from sqlalchemy.orm import DeclarativeBase, Session
# Create a SQLAlchemy engine connected to an in-memory SQLite3 DB:
engine = create_engine("sqlite:///:memory:")
# Create the table in the DB using DDL:
DDL = text(
"""
CREATE TABLE Book (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
isbn TEXT NOT NULL,
title TEXT NOT NULL,
author TEXT NOY NULL
);
"""
)
with Session(engine) as session:
session.execute(DDL)
# Confirm table is empty:
select_all = text("SELECT * FROM Book;")
with Session(engine) as session:
print(session.execute(select_all).fetchall())
# []
# Use reflection to automatically generate a SQLAlchemy Book model:
class Base(DeclarativeBase):
pass
class Book(Base):
__table__ = Table("Book", Base.metadata, autoload_with=engine)
# Insert a new book using the reflected table:
the_hobbit = Book(
isbn="9780007270613",
title="The Hobbit",
author="J. R. R. Tolkien"
)
with Session(engine) as session:
session.add(the_hobbit)
session.flush()
session.commit()
with Session(engine) as session:
print(session.execute(select_all).one())
# [(1, '9780007270613', 'The Hobbit', 'J. R. R. Tolkien')]
The key part is the use of autoload_with=engine the Book python class definition.
This example was tested with Python 3.10 and SQLAlchemy 2.0, but you can do the same with other versions, and with the Flask-Sqlalchemy plugin.
You can also look at more advanced usage such as deferred reflection and automap.

You add new line tablename, old topic here
Document
class Book(db.Model):
__tablename__ = 'Book'
isbn = db.column(db.Integer, primary_key = True)
title = db.column(db.String(50), unique = False, index = True)
author = db.column(db.String(50), unique = True, index = True)

Using Flask SQLAlchemy models with a regular SQLAlchemy

I have an existing model that I can't change, written in Flask-SQLAlchemy.
I'm writing another app that uses the same model, but without the need for Flask, therefore I'm working with the regular SQLAlchemy module.
Unfortunately, I'm getting a lot of:
'AttributeError: module 'DB' has no attribute 'Model'
for all kind of attributes - such as Column, Integer, etc
Is there a way to use Flask-SQLAlchemy with a regular SQLAlchemy app?
There is an example of one of my Model Class:
class Table_name(Model):
__tablename__ = 'table_name'
id = db.Column(db.INTEGER, primary_key=True)
field1 = db.Column(db.INTEGER, db.ForeignKey('table1.id'), nullable=False)
field2 = db.Column(db.TEXT, nullable=False)
field3 = db.Column(db.INTEGER, db.ForeignKey('table2.id'), nullable=False)
time = db.Column(db.TIMESTAMP, nullable=False)
Unfortunately I can't change them

I've had the same dilemma. If it's a small or one-off, you can hack in a Flask app object without really using it, like so:
from flask_sqlalchemy import SQLAlchemy
throw_away_app = Flask(__name__)
db = SQLAlchemy(throw_away_app)
with self.throw_away_app.app_context():
(perform your db operation)
That works relatively well for simple things like scripts. But if you're sharing a model across multiple projects and you simply cannot alter the existing Flask project, it's unfortunately probably not a good solution.
If that's the case, and you simply cannot alter the existing codebase at all, it probably makes sense to create a new model class and connect to the existing database using vanilla SQLAlchemy.
BTW, for any future programmers wondering how to get some of the benefits of Flask-SQLAlchemy without Flask, consider: sqlalchemy-mixins.

from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, String, Text, TIMESTAMP, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
# connect in memory sqlite database or you can connect your own database
engine = create_engine('sqlite:///:memory:', echo=True)
# create session and bind engine
Session = sessionmaker(bind=engine)
class Table_name(Base):
__tablename__ = 'table_name'
id = Column(Integer, primary_key=True)
field1 = Column(Integer, ForeignKey('table1.id'), nullable=False)
field2 = Column(Text, nullable=False)
field3 = Column(Integer, ForeignKey('table2.id'), nullable=False)
time = Column(TIMESTAMP, nullable=False)
table = Table_name(field1=1, fi....)
session.add(table)
Now you can use your ORM as usual like flask-sqlalchemy .
Docs: https://docs.sqlalchemy.org/en/13/orm/tutorial.html

AmbiguousForeignKeysError even after specifying foreign_keys

I'm setting up some table objects for SQLAlchemy.
I have a user and checkout tables. I want to associate a user object with the checkin and the checkout, which are both recorded in the same checkout object, so I have an in_user and out_user associated with each checkout object.
I've run into a sqlalchemy.exc.AmbiguousForeignKeysError
To quote the exact error message:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child tables on relationship Checkout.out_auth_user - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
I've done as the error message requests (see below), but the error still occurs.
I originally only specified user email because I wanted to be able to remove users in the future without corrupting historical data. However, I tried to add user id, but still got the same error.
There are many similar questions on StackOverflow, but I couldn't find one that addressed my problem and most of them are working with much older versions of sqlalchemy that did not support the foreign_keys argument to relationship. It seems like this often occurs with backreferences, but I'm not using those as far as I'm aware. This is a simple one-way link from a checkout object to two user objects.
Flask foreign_keys still shows AmbiguousForeignKeysError
sqlalchemy , AmbiguousForeignKeysError
The full code is on github at https://github.com/ACMWM/hwcheckout
Below is an MRE
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Boolean, Integer, String, ForeignKey, DateTime
from sqlalchemy.orm import relationship
db = "sqlite:///mre.db"
engine = create_engine(db, convert_unicode=True)
db_session = scoped_session(sessionmaker(autocommit=False,
autoflush=False,
bind=engine))
Base = declarative_base()
Base.query = db_session.query_property()
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
email = Column(String)
class HW(Base):
__tablename__ = "HW"
id = Column(Integer, primary_key=True)
class Checkout(Base):
__tablename__ = "Checkouts"
what = Column(Integer, ForeignKey(HW.id))
hardware = relationship(HW, foreign_keys=[what])
id = Column(Integer, primary_key=True)
out_auth_id = Column(Integer, ForeignKey(User.id))
out_auth_email = Column(String, ForeignKey(User.email))
out_auth_user = relationship(User, foreign_keys=[out_auth_id, out_auth_email])
in_auth_id = Column(Integer, ForeignKey(User.id))
in_auth_email = Column(String, ForeignKey(User.email))
in_auth_user = relationship(User, foreign_keys=[in_auth_id, in_auth_email])
Base.metadata.create_all(bind=engine, checkfirst=True)
u = User(email="test#example.com")
chk = Checkout(out_auth_user_id=u.id,out_auth_user_email=u.email)
I'm using SQLAlchemy 1.3.3
EDIT: Remove double import of models. Same error still occurs
EDIT again: Got the MRE to reproduce the error
Postgres EDIT: Don't know if this helps, but when I tried to move my code to a real database, I got this error:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidForeignKey) there is no unique constraint matching given keys for referenced table "users"
[SQL:
CREATE TABLE "Checkouts" (
id SERIAL NOT NULL,
outdate TIMESTAMP WITHOUT TIME ZONE,
returndate TIMESTAMP WITHOUT TIME ZONE,
who VARCHAR,
reason VARCHAR,
quantity INTEGER,
what INTEGER,
out_auth_id INTEGER,
out_auth_email VARCHAR,
in_auth_id INTEGER,
in_auth_email VARCHAR,
PRIMARY KEY (id),
UNIQUE (id),
FOREIGN KEY(what) REFERENCES "HW" (id),
FOREIGN KEY(out_auth_id) REFERENCES users (id),
FOREIGN KEY(out_auth_email) REFERENCES users (email),
FOREIGN KEY(in_auth_id) REFERENCES users (id),
FOREIGN KEY(in_auth_email) REFERENCES users (email)
)
]

Try to change your Checkout model definition:
class Checkout(Base):
__tablename__ = "Checkouts"
what = Column(Integer, ForeignKey(HW.id))
hardware = relationship(HW, foreign_keys=[what])
id = Column(Integer, primary_key=True)
out_auth_id = Column(Integer, ForeignKey(User.id))
out_auth_email = Column(String, ForeignKey(User.email))
in_auth_id = Column(Integer, ForeignKey(User.id))
in_auth_email = Column(String, ForeignKey(User.email))
out_auth_user = relationship('User', foreign_keys=[out_auth_id])
in_auth_user = relationship('User', foreign_keys=[in_auth_id])
out_auth_user_by_email = relationship('User', foreign_keys=[out_auth_email])
in_auth_user_by_email = relationship('User', foreign_keys=[in_auth_email])
Documentation: https://docs.sqlalchemy.org/en/13/orm/join_conditions.html#handling-multiple-join-paths

SQLAlchemy classes across files, tables are not created

I am trying to create an application using SQLAlchemy. It worked fine as long as I only had one file with one Class. Now I want to have multiple classes/tables in different files. I stumbled upon this question, and tried to do it like it was suggested there: I now have three files
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
blind.py
from sqlalchemy import Column, String
from .base import Base
class Blind(Base):
__tablename__ = 'blinds'
blind = Column(String)
data_processor_uuid = Column(String, primary_key=True)
data_source_uuid = Column(String)
timestamp = Column(String, primary_key=True)
and data.py
from sqlalchemy import Column, Integer, String, Float
from .base import Base
class Datum(Base):
__tablename__ = 'data'
data_source_uuid = Column(Integer, primary_key=True)
sensor_type = Column(String)
timestamp = Column(String, primary_key=True)
value = Column(Float)
I now want to initialize the database in db_setup.py using
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .base import Base
engine = create_engine('sqlite:///test.db', echo=True)
Base.metadata.bind = engine
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
def get_db_session():
return session
This works, however, it does not create the tables in the database as expected. When I try to insert something into the table, I get an error saying "table does not exist". Can someone tell me what I am doing wrong here?

The problem was that I wasn't importing the class definitions for Blinds and Datum anywhere, so they weren't evaluated! Before I split them up into different files, I had imported them to get to Base. Thanks to #IljaEverilä for this answer!

SQLAlchemy: Unnecessary join with joinedload

I'm using SQLAlchemy ORM for a Flask project where I want to join across an eagerly loaded model but this leads to two joins to the same intermediary model. If you run the code below you'll see in the generated SQL that there are two joins between the Author model and the Book model. If the lazy=joined bit is removed the sql generated is perfect.
I don't know if I'm doing something wrong or this is by design. How do I get SQLAlchemy to emit the right SQL while maintaining the joinedload in this case?
Note: I have tried this with MySQL and SQLite and it happens with both those dbs.
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
name = Column(String)
pseudo = Column(String)
books = relationship("Book", lazy='joined')
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>" % (
self.name, self.fullname, self.password)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
author_id = Column(Integer, ForeignKey('authors.id'))
name = Column(String)
user = relationship("Author", back_populates="books")
pages = relationship("Page")
class Page(Base):
__tablename__ = 'pages'
id = Column(Integer, primary_key=True)
book_id = Column(Integer, ForeignKey('books.id'))
text = Column(String)
book = relationship("Book", back_populates="pages")
Base.metadata.create_all(engine)
session = Session()
print(str(session.query(Author).outerjoin(Author.books, Page)))

It is by design – read The Zen of Joined Eager Loading:
It is critical to understand the distinction that while Query.join() is used to alter the results of a query, joinedload() goes through great lengths to not alter the results of the query, and instead hide the effects of the rendered join to only allow for related objects to be present.
There are multiple somewhat similar questions in sqlalchemy, though couldn't find one that'd fit the bill exactly.
If you manually add a join, and want to use it to eager load a relationship as well, you need contains_eager():
session.query(Author).\
outerjoin(Author.books, Book.pages).\
options(contains_eager(Author.books).contains_eager(Book.pages))
Note that the relationship definitions Author.books and Book.pages would seem to be missing the back_populates= argument.