I've defined some models with some association tables for m2m relationships:
from itsdangerous import TimedJSONWebSignatureSerializer
from passlib.hash import bcrypt
from sqlalchemy.ext.declarative import declarative_base
import app
from app import db
Base = declarative_base()
class UserGroupRelationship(Base):
__tablename__ = 'users_groups'
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
group_id = db.Column(db.Integer, db.ForeignKey('groups.id'), primary_key=True)
class FriendRelationship(Base):
__tablename__ = u'users_friends'
id = db.Column(db.Integer, primary_key=True)
user_left = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
user_right = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
class User(db.Model):
__tablename__ = u'users'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
email = db.Column(db.String(120), unique=True)
password = db.Column(db.String(120))
# ...
last_login = db.Column(db.DateTime, default=db.func.now())
friends = db.relationship(FriendRelationship,
primaryjoin=id==FriendRelationship.user_left,
backref='friends', lazy='dynamic')
friends_with = db.relationship(FriendRelationship,
primaryjoin=id==FriendRelationship.user_right,
backref='friends_with', lazy='dynamic')
class Group(db.Model):
__tablename__ = u'groups'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
users = db.relationship(UserGroupRelationship,
primaryjoin=id==UserGroupRelationship.group_id,
backref='groups', lazy='dynamic')
class Device(db.Model):
''' devices linked to users '''
__tablename__ = u'devices'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
uuid = db.Column(db.String(50))
date_added = db.Column(db.DateTime)
user_id = db.Column(db.Integer, db.ForeignKey('groups.id'))
running alembic revision --autogenerate does generate table for classes inheriting from db.Model but not for the tables used for my m2m relationships.
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
INFO [alembic.autogenerate] Detected added table u'groups'
INFO [alembic.autogenerate] Detected added table u'users'
INFO [alembic.autogenerate] Detected added table u'devices'
INFO [alembic.autogenerate] Detected added table u'question'
Generating /Users/rxdazn/w/xxx/xxx-
yyy/migrations/versions/4e47aa7f3050_.py...done
My alembic.ini and env.py files are the default ones. I simply import my models init my project's __init__.py
any idea of what could cause this behaviour?
(Miguel answered this in the comments. I'll delete this answer if he posts it and someone pokes me in a comment. I only posted it so it could be marked as an answer, as I had the same problem and almost left the page before reading the comments.)
Don't inherit the association tables from Base. All models should inherit from db.Model, like this:
class FriendRelationship(db.Model):
This is why:
There are two patterns for many-to-many relationships. The basic one
uses a Table object for the association table. The advanced one uses
models, and has the advantage that you can store additional columns in
the association table. You seem to be using the advanced one, but if
you are using Flask-SQLAlchemy then all your models should inherit
from db.Model. You should not go directly to SQLAlchemy.
Related
FastAPI/SQLAlchemy. I had two tables in sqlLite database: blog and user. I had created the database, but later decided to add relationship between the two and added new foreignKey Column. However, my changes did not appear in existing db (even though I deleted everything in it). Only after I deleted .db file entirely, did my --reload generate new db with new columns. I was wondering if there is analog of makemigrations + migrate from Django where you do not have to delete DB entirely for FastAPI and SQLAlchemy.
I made changes from this
class Blog(Base):
__tablename__ = "blogs"
id=Column(Integer, primary_key=True, index= True)
title= Column(String)
body=Column(String)
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key = True, index=True)
username = Column(String)
email = Column(String)
password= Column(String)
To this
class Blog(Base):
__tablename__ = "blogs"
id=Column(Integer, primary_key=True, index= True)
title= Column(String)
body=Column(String)
user_id= Column(Integer, ForeignKey("users.id")) ### NEW
creator = relationship("User", back_populates= "blogs") ### NEW
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key = True, index=True)
username = Column(String)
email = Column(String)
password= Column(String)
blogs = relationship("Blog", back_populates = "creator") ### NEW
You can used alembic to do that.
Please click this link for the official instructions! This is the link
So I have these two simple classes inside the same python file which I'm trying to map with SQLAlchemy, where User column 'password_id' is a foreign key to the table Password 'password_id' column as well
from sqlalchemy.orm import relationship, declarative_base
from sqlalchemy import Column, Integer, String
from sqlalchemy.sql.schema import ForeignKey
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
user_id = Column(Integer, primary_key=True)
name = Column(String)
email = Column(String)
password_id = Column(Integer, ForeignKey('password.password_id'))
parent = relationship("Password", back_populates="users")
class Password(Base):
__tablename__ = 'passwords'
password_id = Column(Integer, primary_key=True)
password = Column(String)
last_change_date = Column(DateTime)
valid_until = Column(DateTime)
child = relationship("User", back_populates="passwords", uselist=False)
Here's the db schema for context:
I'm following this guide from sql alchemy but for some reason I keep getting the error from the title 'Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.', which is strange because accorind to what I see in the guide, my classes have been mapped correctly so I can't understand why this error is happening.
Thank you in advance
I think the issue is in the following line (the table name in ForeignKey)...
password_id = Column(Integer, ForeignKey('password.password_id'))
should be passwords instead of password.
I have two model classes in separate files, created a One-To-Many relationship between them.
user.py:
class User(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(100), index=True, unique=True, nullable=False)
password = db.Column(db.String(128), nullable=False)
projects = db.relationship('Project', backref='owner', lazy='dynamic')
project.py:
class Project(db.Model):
__tablename__ = 'projects'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), index=True, unique=True, nullable=False)
owner_id = db.Column(db.Integer, db.ForeignKey('User.id'))
This is how I create my app:
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URL
db = SQLAlchemy(app)
#app.before_first_request
def create_tables():
db.create_all()
app.run(debug=True)
However when I create a request I get the following error:
sqlalchemy.exc.NoReferencedTableError:
Foreign key associated with column 'projects.owner_id' could not find
table 'User' with which to generate a foreign key to target column 'id'
I understand this is a duplicate of this question asked before, however I tried that and did not work:
#app.before_first_request
def create_tables():
from projectx.models.auth.user import User
from projectx.models.private_data.project import Project
db.create_all()
I also tried giving the same __table_args__ = {"schema": "testdb"} args to both models (I manually created the db with this name) and refer to db.ForeignKey('testdb.User.id) to no avail, the error is the same.
What am I missing here?
It seems I misunderstood some concepts about how this relationship abstraction works.
The foreign key reference is to the tablename and not the class name naturally.
I should have written db.ForeignKey('users.id').
I have to create an Flask application which need to read data from multiple almost identical schemas placed on this same Oracle instance. I am able to connect to this database only by 'read_only' user.
By do this I am using blueprints.
project
|-warehouse01
| -__init__.py
| - view.py
|-warehouse02
| -__init__.py
| - view.py
|-warehouse999
| -__init__.py
| - view.py
-model.py
run.py
config.py
Of course model for all schemas is identical
class Product(db.Model):
#__table_args__ = {'schema': 'warehouse01'}
__tablename__ = 'PRODUCTS'
prod_no = db.Column(db.Integer, primary_key=True, nullable=False)
prod_name = db.Column(db.String(35), nullable=False)
quantity = db.Column(db.Integer, nullable=False)
...
steps = db.relationship('Shipment', backref='shipments', lazy='dynamic')
class Shipment(db.Model):
__tablename__ = 'SHIPMENTS'
#__table_args__ = {'schema': 'warehouse01'}
shipm_no = db.Column(db.Integer, , primary_key=True, nullable=False)
prod_no = db.Column(db.Integer, db.ForeignKey('PRODUCTS.prod_no'), nullable=False)
customer_id= db.Column(db.Integer, db.ForeignKey('CUSTOMERS.customer_id') nullable=False)
...
Problem is that connecting by different schema (READ_ONLY) I have to declare a schema in table_args
I would like to use this same model to all schemas. Obviously I do not want to create separate models for all blueprints.
Is it possible to use this same model for all schemas or redefine in blueprint a schema name?
I think that you can use a factory function to create your models on the fly:
def create_product_model(schema):
class Product(db.Model):
__table_args__ = {'schema': schema}
__tablename__ = 'PRODUCTS'
prod_no = db.Column(db.Integer, primary_key=True, nullable=False)
prod_name = db.Column(db.String(35), nullable=False)
quantity = db.Column(db.Integer, nullable=False)
...
steps = db.relationship('Shipment', backref='shipments', lazy='dynamic')
return Product
Finally I have found a way to do this. Unfortunately a approach proposed by #Laurent LAPORTE was not proper. It works only when I had just one class in model, and only after adding 'extend_existing': True. Please correct me if I am wrong but in Flask a database model is declared at application start. As schema name is a part of a class declared in __table_args__ = {'schema': schema}, while you would like to change the schema, SqlAlchemy treats it as try to defining new class with existing name in model registry.
My solution is to remove schema from table_args and setup a schema name by sql command like below:
def estimate_schema(schema):
db.engine.execute(f"alter session set current_schema={schema}")
This I use before each route and it works
def index():
estimate_schema('warehouse01')
...
I'm a little new to SQLAlchemy. I've searched around for an answer to my question but I have found nothing that works for my situation.
In short, deleting a record in the Release model will delete all the records in the other models as long as there is no related records in TestResults. However, if there are related records in TestResult, then deleting a Release will not work. It almost seems as if deleting a parent will delete a child and the child's child but not the child's child's child. Here is some code to help highlight this:
class Release(db.Model):
__tablename__ = 'releases'
id = db.Column(db.Integer, primary_key=True)
platform_id=db.Column(db.Integer, db.ForeignKey('platforms.id'))
name = db.Column(db.String(20), unique=True)
builds = db.relationship('ReleaseBuilds', cascade='all,delete', lazy='dynamic', order_by="desc(ReleaseBuilds.date_created)")
class ReleaseBuilds(db.Model):
__tablename__='release_builds'
id = db.Column(db.Integer, primary_key=True)
release_id = db.Column(db.Integer, db.ForeignKey('releases.id'))
name = db.Column(db.String(150), nullable=False)
artifacts = db.relationship('ReleaseBuildArtifacts', cascade='all,delete', backref='builds', lazy='dynamic')
deployments = db.relationship('Deployments', cascade='all,delete', lazy='dynamic')
tests = db.relationship('Test', cascade='delete', lazy='dynamic')
class ReleaseBuildArtifacts(db.Model):
__tablename__='release_build_artifacts'
id = db.Column(db.Integer, primary_key=True)
release_build_id = db.Column(db.Integer, db.ForeignKey('release_builds.id'))
application_id = db.Column(db.Integer, db.ForeignKey('applications.id'))
rpm = db.Column(db.String(300))
build = db.relationship('ReleaseBuilds')
application = db.relationship('Application')
class Deployments(db.Model):
__tablename__ = 'deployments'
release_build_id = db.Column(db.Integer, db.ForeignKey('release_builds.id'), primary_key=True)
environment_id = db.Column(db.Integer, db.ForeignKey('environments.id'), primary_key=True)
date_deployed = db.Column(db.DateTime(timezone=False), default=datetime.datetime.utcnow)
environment = db.relationship('Environment', foreign_keys=[environment_id])
class TestType(db.Model):
__tablename__ = 'test_types'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50), unique=True)
class Test(db.Model):
__tablename__ = 'tests'
id = db.Column(db.Integer, primary_key=True)
release_build_id = db.Column(db.Integer, db.ForeignKey('release_builds.id'), nullable=False)
environment_id = db.Column(db.Integer, db.ForeignKey('environments.id'), nullable=False)
test_type_id = db.Column(db.Integer, db.ForeignKey('test_types.id'))
name = db.Column(db.String(300))
environments = db.relationship('Environment', foreign_keys=[environment_id])
results = db.relationship('TestResult', cascade='all,delete', lazy='dynamic')
__table_args__ = (
ForeignKeyConstraint(['release_build_id', 'environment_id'],['deployments.release_build_id', 'deployments.environment_id']),
)
class TestResult(db.Model):
__tablename__ = 'test_results'
id = db.Column(db.Integer, primary_key=True)
test_id = db.Column(db.Integer, db.ForeignKey('tests.id'), nullable=False)
name = db.Column(db.String(500))
passed = db.Column(db.Boolean)
Any suggestions as to why this cascade delete is not working?
I came across a similar issue in our project, where we define cascades on the ORM level and also use lazy=dynamic relationships. This caused the cascade not to run on the bottom-most children.
Dynamic loading causes the relationship to return a Query object when accessed.
Delete on queries is quite limited, in order to increase performance, as documented here:
https://docs.sqlalchemy.org/en/13/orm/query.html
The method does not offer in-Python cascading of relationships - it
is assumed that ON DELETE CASCADE/SET NULL/etc. is configured for any
foreign key references which require it, otherwise the database may
emit an integrity violation if foreign key references are being
enforced.
After the DELETE, dependent objects in the Session which were impacted
by an ON DELETE may not contain the current state, or may have been
deleted. This issue is resolved once the Session is expired, which
normally occurs upon Session.commit() or can be forced by using
Session.expire_all(). Accessing an expired object whose row has been
deleted
will invoke a SELECT to locate the row; when the row is not found,
an ObjectDeletedError is raised.
Therefore a solution for your problem could be either defining cascades on the database level, or using other types of relationships.
Related question was raised here: SQLAlchemy delete doesn't cascade
EDIT: (Solution I applied is changing the loading type on query level - in options)