Develop Reference

What happens if I specify OneToMany relationship only from one side in flask-sqlalchemy?

I have a OneToMany relationship between 2 entities in flask. I also specified the relationship only on one side. I am unsure what the difference is between the following:
class CustomJob(db.Model):
__tablename__ = "custom_job"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
country_from = db.Column(db.Integer, db.ForeignKey('country.id'))
class Country(db.Model):
__tablename__ = "country"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
custom_jobs = db.relationship('CustomJob', backref="country", lazy=False)
Or just specify the foreign key on master entity:
class CustomJob(db.Model):
__tablename__ = "custom_job"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
country_from = db.Column(db.Integer, db.ForeignKey('country.id'))
will is there performance difference between the two ?

The brilliance behind an ORM like SQLAlchemy is that it can detect relationships between models based on foreign key constraints. So once you've declared your foreign key on the custom_job table, the relationship is configured in the database.
Mapping that relationship to your python objects is another useful part of ORM's. Here, you are doing that with db.relationship. By specifying backref, you are essentially telling the ORM to make the relationship available on the other object.
Let me explain more explicitly using the code provided in your Q:
class Country(db.Model):
__tablename__ = 'country'
...
custom_jobs = db.relationship('CustomJob', backref='custom_job', lazy=False)
...
The Country model you've defined will map all associated rows from the custom_job table through the attribute Country.custom_jobs.
This relationship will propagate to the CustomJob model and allow you to access the associated rows from the country table through an attribute created by the backref parameter --> here CustomJob.custom_job.
I assume this is an error and that you intended to use backref="country"
In this case, access associated objects instead with CustomJob.country

How to construct a SQLAlchemy relationship that takes the record most recently inserted?

Imagine I've got the following:
class User:
id = Column(Integer, primary_key=True)
username = Column(String(20), nullable=False)
password_hash = Column(String(HASH_LENGTH), nullable=False)
class LoginAttempts:
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey(User.id))
attempted_at = Column(DateTime, default=datetime.datetime.utcnow)
Now, I want to add a relationship to User called last_attempt that retrieves the most recent login attempt. How might one do this?

This seems like a use case for a relationship to an aliased class, which was added in SQLAlchemy 1.3 – before that you'd use a non primary mapper, or other methods such as a custom primary join. The idea is to create a subquery representing a derived table of latest login attempts per user that is then aliased to LoginAttempts and used as the target of a relationship. The exact query used to derive the latest attempts depends on your DBMS1, but a generic left join "antijoin" will work in most. Start by generating the (sub)query for latest login attempts:
newer_attempts = aliased(LoginAttempts)
# This reads as "find login attempts for which no newer attempt with larger
# attempted_at exists". The same could be achieved using NOT EXISTS as well.
latest_login_attempts_query = select([LoginAttempts]).\
select_from(
outerjoin(LoginAttempts, newer_attempts,
and_(newer_attempts.user_id == LoginAttempts.user_id,
newer_attempts.attempted_at > LoginAttempts.attempted_at))).\
where(newer_attempts.id == None).\
alias()
latest_login_attempts = aliased(LoginAttempts, latest_login_attempts_query)
Then just add the relationship attribute to your User model:
User.last_attempt = relationship(latest_login_attempts, uselist=False,
viewonly=True)
1: For example in Postgresql you could replace the LEFT JOIN subquery with a LATERAL subquery, NOT EXISTS, a query using window functions, or SELECT DISTINCT ON (user_id) ... ORDER BY (user_id, attempted_at DESC).

Although the selected answer is more robust, another way you could accomplish this is to use a lazy=dynamic and order_by:
User.last_attempted = relationship(LoginAttempts, order_by=desc(LoginAttempts.attempted_at), lazy='dynamic')
Be careful though, because this returns a query object (and will require .first() or equivalent), and you will need to use a limit clause:
last_attempted_login = session.query(User).get(my_user_id).last_attempted.limit(1).first()

AmbiguousForeignKeysError even after specifying foreign_keys

I'm setting up some table objects for SQLAlchemy.
I have a user and checkout tables. I want to associate a user object with the checkin and the checkout, which are both recorded in the same checkout object, so I have an in_user and out_user associated with each checkout object.
I've run into a sqlalchemy.exc.AmbiguousForeignKeysError
To quote the exact error message:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child tables on relationship Checkout.out_auth_user - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
I've done as the error message requests (see below), but the error still occurs.
I originally only specified user email because I wanted to be able to remove users in the future without corrupting historical data. However, I tried to add user id, but still got the same error.
There are many similar questions on StackOverflow, but I couldn't find one that addressed my problem and most of them are working with much older versions of sqlalchemy that did not support the foreign_keys argument to relationship. It seems like this often occurs with backreferences, but I'm not using those as far as I'm aware. This is a simple one-way link from a checkout object to two user objects.
Flask foreign_keys still shows AmbiguousForeignKeysError
sqlalchemy , AmbiguousForeignKeysError
The full code is on github at https://github.com/ACMWM/hwcheckout
Below is an MRE
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Boolean, Integer, String, ForeignKey, DateTime
from sqlalchemy.orm import relationship
db = "sqlite:///mre.db"
engine = create_engine(db, convert_unicode=True)
db_session = scoped_session(sessionmaker(autocommit=False,
autoflush=False,
bind=engine))
Base = declarative_base()
Base.query = db_session.query_property()
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
email = Column(String)
class HW(Base):
__tablename__ = "HW"
id = Column(Integer, primary_key=True)
class Checkout(Base):
__tablename__ = "Checkouts"
what = Column(Integer, ForeignKey(HW.id))
hardware = relationship(HW, foreign_keys=[what])
id = Column(Integer, primary_key=True)
out_auth_id = Column(Integer, ForeignKey(User.id))
out_auth_email = Column(String, ForeignKey(User.email))
out_auth_user = relationship(User, foreign_keys=[out_auth_id, out_auth_email])
in_auth_id = Column(Integer, ForeignKey(User.id))
in_auth_email = Column(String, ForeignKey(User.email))
in_auth_user = relationship(User, foreign_keys=[in_auth_id, in_auth_email])
Base.metadata.create_all(bind=engine, checkfirst=True)
u = User(email="test#example.com")
chk = Checkout(out_auth_user_id=u.id,out_auth_user_email=u.email)
I'm using SQLAlchemy 1.3.3
EDIT: Remove double import of models. Same error still occurs
EDIT again: Got the MRE to reproduce the error
Postgres EDIT: Don't know if this helps, but when I tried to move my code to a real database, I got this error:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidForeignKey) there is no unique constraint matching given keys for referenced table "users"
[SQL:
CREATE TABLE "Checkouts" (
id SERIAL NOT NULL,
outdate TIMESTAMP WITHOUT TIME ZONE,
returndate TIMESTAMP WITHOUT TIME ZONE,
who VARCHAR,
reason VARCHAR,
quantity INTEGER,
what INTEGER,
out_auth_id INTEGER,
out_auth_email VARCHAR,
in_auth_id INTEGER,
in_auth_email VARCHAR,
PRIMARY KEY (id),
UNIQUE (id),
FOREIGN KEY(what) REFERENCES "HW" (id),
FOREIGN KEY(out_auth_id) REFERENCES users (id),
FOREIGN KEY(out_auth_email) REFERENCES users (email),
FOREIGN KEY(in_auth_id) REFERENCES users (id),
FOREIGN KEY(in_auth_email) REFERENCES users (email)
)
]

Try to change your Checkout model definition:
class Checkout(Base):
__tablename__ = "Checkouts"
what = Column(Integer, ForeignKey(HW.id))
hardware = relationship(HW, foreign_keys=[what])
id = Column(Integer, primary_key=True)
out_auth_id = Column(Integer, ForeignKey(User.id))
out_auth_email = Column(String, ForeignKey(User.email))
in_auth_id = Column(Integer, ForeignKey(User.id))
in_auth_email = Column(String, ForeignKey(User.email))
out_auth_user = relationship('User', foreign_keys=[out_auth_id])
in_auth_user = relationship('User', foreign_keys=[in_auth_id])
out_auth_user_by_email = relationship('User', foreign_keys=[out_auth_email])
in_auth_user_by_email = relationship('User', foreign_keys=[in_auth_email])
Documentation: https://docs.sqlalchemy.org/en/13/orm/join_conditions.html#handling-multiple-join-paths

Query items from granchildren objects

I have a Flask python application that has a set of related tables that are chained together through foreign keys. I would like to be able to return an aggregate list of records from one table that are related to a distant table. However, I am struggling to understand how sqlalchemy does this through object relationships.
For example, there are three objects I'd like to join (challenge and badge) with two tables (talent_challenge and badge) to be able to query for all badges related to a specific challenge. In SQL, this would look something like:
SELECT b.id, b.name
FROM badge b
INNER JOIN talent_challenge tc ON tc.talent_id = b.talent_id
WHERE tc.challenge_id = 21
The 'talent' and 'challenge' tables are not needed in this case, since I only need the talent and challenge IDs (in 'talent_challenge') for the relationship. All of the interesting detail is in the badge table.
I am able to use sqlalchemy to access the related talent from a challenge using:
talents = db.relationship('TalentModel', secondary='talent_challenge')
And I can then reference talent.badges for each of those talents to get the relevant badges related to my initial challenge. However, there can be redundancy, and this list of badges isn't contained in a single object.
A stripped-down version of the three models are:
class TalentModel(db.Model):
__tablename__ = 'talent'
# Identity
id = db.Column(db.Integer, primary_key=True)
# Relationships
challenges = db.relationship('ChallengeModel', secondary='talent_challenge',)
# badges (derived as backref from BadgeModel)
class ChallengeModel(db.Model):
__tablename__ = 'challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
member_id = db.Column(db.Integer, db.ForeignKey('member.id'))
# Relationships
talents = db.relationship('TalentModel', secondary='talent_challenge', order_by='desc(TalentModel.created_at)')
class BadgeModel(db.Model):
__tablename__ = 'badge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
# Parents
talent = db.relationship('TalentModel', foreign_keys=[talent_id], backref="badges")
I also have a model for the associative table, 'talent_challenge':
class TalentChallengeModel(db.Model):
__tablename__ = 'talent_challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
challenge_id = db.Column(db.Integer, db.ForeignKey('challenge.id'))
# Parents
talent = db.relationship('TalentModel', uselist=False, foreign_keys=[talent_id])
challenge = db.relationship('ChallengeModel', uselist=False, foreign_keys=[challenge_id])
I would like to better understand sqlalchemy (or specifically, flask-sqlalchemy) to allow me to construct this list of badges from the challenge object. Is db.session.query of BadgeModel my only option?
UPDATED 1/23/2015:
My blocker on my project was solved by using the following:
#property
def badges(self):
from app.models.sift import BadgeModel
from app.models.relationships.talent import TalentChallengeModel
the_badges = BadgeModel.query\
.join(TalentChallengeModel, TalentChallengeModel.talent_id==BadgeModel.talent_id)\
.filter(TalentChallengeModel.challenge_id==self.id)\
.all()
return the_badges
Wrapping the query in a function got around the issues I was having with the name BadgeModel not being defined and not being able to be imported in the model otherwise. The #property decorator allows me to just reference this as challenge.badges later in the view.
However, I am still interested in understanding how to do this as a relationship. Some searching elsewhere led me to believe this would work:
badges = db.relationship('BadgeModel',
secondary="join(BadgeModel, TalentChallengeModel, BadgeModel.talent_id == TalentChallengeModel.talent_id)",
secondaryjoin="remote([id]) == foreign(TalentChallengeModel.challenge_id)",
primaryjoin="BadgeModel.talent_id == foreign(TalentChallengeModel.talent_id)",
viewonly=True,
)
Because of other unresolved issues in my application environment, I can't fully test this (e.g., adding this code breaks Flask-User in my site) but would like to know if this is correct syntax and if there is any disadvantage to this over the query-in-function solution.

Materialized path relationship in declarative SQLAlchemy

I have a hierarchical categories model, where hierarchy is maintained using materialized path (one character per level):
class Category(Base):
__tablename__ = 'categories'
id = Column(SmallInteger, primary_key=True)
path = Column(String, unique=True, nullable=False)
# problematic relationship
all_subcats = relationship('Category', lazy='dynamic', viewonly=True,
primaryjoin=foreign(path).like(remote(path).concat('%')))
When trying to define "all subcategories" relationship I run into a problem:
sqlalchemy.exc.ArgumentError: Can't determine relationship direction for
relationship 'Category.all_subcats' - foreign key columns within the join
condition are present in both the parent and the child's mapped tables.
Ensure that only those columns referring to a parent column are marked as
foreign, either via the foreign() annotation or via the foreign_keys argument.
SQLAlchemy is confused, because I'm joining on the same column. All examples I've managed to find always join on different columns.
Is this sort of relationship possible at all? I want to query through this join, so custom #property is not acceptable.

Use the latest git master or version 0.9.5 or greater of SQLAlchemy. Then:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Element(Base):
__tablename__ = 'element'
path = Column(String, primary_key=True)
related = relationship('Element',
primaryjoin=
remote(foreign(path)).like(
path.concat('/%')),
viewonly=True,
order_by=path)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
sess = Session(e)
sess.add_all([
Element(path="/foo"),
Element(path="/foo/bar1"),
Element(path="/foo/bar2"),
Element(path="/foo/bar2/bat1"),
Element(path="/foo/bar2/bat2"),
Element(path="/foo/bar3"),
Element(path="/bar"),
Element(path="/bar/bat1")
])
e1 = sess.query(Element).filter_by(path="/foo/bar2").first()
print [e.path for e in e1.related]
note that this model, whether you deal with "descendants" or "anscestors", uses collections. You want to keep remote() and foreign() together so that the ORM considers it as one to many.