Develop Reference

Many to many relationship in SQLAlchemy with same foreign key used twice in association object

I have three tables - Competition, Competitor, and Duel. Table Duel is actually an association object between tables Competition and Competitor. The code is following:
class Competition(db.Model):
__tablename__ = 'competitions'
id_competition = db.Column(db.Integer, primary_key=True, autoincrement=True)
# some additional attributes
competitors_duel = relationship('Duel', back_populates='rel_competition')
class Competitor(db.Model):
__tablename__ = 'competitors'
id_competitor = db.Column(db.Integer, primary_key=True, autoincrement=True)
# some additional attributes
competitions_duel = relationship('Duel', back_populates='rel_competitor')
class Duel(db.Model):
__tablename__ = 'duels'
id_competitor1 = db.Column(db.ForeignKey('competitors.id_competitor'), primary_key=True)
id_competitor2 = db.Column(db.ForeignKey('competitors.id_competitor'), primary_key=True)
id_competition = db.Column(db.ForeignKey('competitions.id_competition'), primary_key=True)
phase = db.Column(db.Integer, primary_key=True)
# some additional attributes
rel_competitor = relationship('Competitor', back_populates='competitions_duel')
rel_competition = relationship('Competition', back_populates='competitors_duel')
This is the same association object as example of bidirectional association object in documentation.
Problem that I have is this:
Could not determine join condition between parent/child tables on
relationship Competitor.competitions_duel - there are multiple foreign
key paths linking the tables. Specify the 'foreign_keys' argument,
providing a list of those columns which should be counted as
containing a foreign key reference to the parent table.
The error is gone when I delete id_competitor2 (or 1). But I need both ids because it is a duel, I need to know both competitors. Also, each contest has its phase so those four keys (id_competitor1, id_competitor2, id_competition, and phase) should actually create one composite key which differentiates between duels.
How can I solve this?

Is there any way to filter by the first child using SqlAlchemy?

I'm trying to find a way to filter by the attribute of the first row returned as children.
class Parent(Base):
__tablename__ = "parent_table"
id = Column(Integer, primary_key=True, index=True, autoincrement=True)
children = relationsihp("Child", back_populates="parent")
class Child(Base):
__tablename__ = "child_table"
id = Column(Integer, primary_key=True, index=True, autoincrement=True)
parent_id = Column(Integer, ForeignKey("parent_table"))
type = Column(String(10))
created_at = Column(TIMESTAMP)
parent = relationship("Parent", back_populates="children")
What I need to do is select Parent while selecting their children as well, order the children by created_at, see if type of the first row returned(first child, i guess?) is 'type A', and if not, don't select that Parent.
Is it possible to implement this using SqlAlchemy?
At first I tried to add new temporary columns of each child row's 'type' which has value of created_at but i was not able to make a query comparing two columns of the same record.
Then I tried to use hybrid_property, which returns type of the first child as Parent's property. It didn't work as well because it was not allowed to filter by hybrid_property.(In real case, 'type' is not given. It has to be calculated using other attributes and conditions.)
It would be also highly appreciated if you just give me some hint about making a query using raw SQL syntax.
Thank you.

sorting sqlalchemy relationships by primary key

How can I make a relationship attribute be sorted by the child tables primary key? I have a schema where many of the tables have a common parent and the parent has a list of each of its children using the declaritive system like this
class Scenario(Base):
__tablename__ = 'scenarios'
id = Column(Integer, primary_key=True, autoincrement=True)
class ScenarioMixin(object):
#declared_attr
def scenario_id(cls):
return Column(None, ForeignKey(Scenario.id),
primary_key=True,
index=True)
#declared_attr
def scenario(cls):
return relationship('Scenario',
backref=backref(cls.__tablename__))
# example scenario-specific table
class Child(Base, ScenarioMixin):
__tablename__ = 'children'
id1 = Column(String, primary_key=True)
id2 = Column(String, primary_key=True)
So the scenario object has an attribute children which is a list of children. This works fine, except that the order of the Child objects in the list is arbitrary. This is fine for the semantics of the application, it leads to nondeterministic behavior. What's the best way to configure the mixin so that every child list will be sorted by the primary key of its table, even if it's a compound primary key?

Specify order_by argument in your relationship definition as in the code below.
#declared_attr
def scenario(cls):
return relationship(
'Scenario',
backref=backref(
cls.__tablename__,
order_by=lambda: cls.__table__.primary_key.columns
),
)
In you case you cannot return cls.__table__.primary_key.columns straight away because at the definition time the __table__ does not exist yet. But returning a callable works.
See Relationships API for more information.

SQLAlchemy DELETE Error caused by having a both lazy-load AND a dynamic version of the same relationship

Here is some example code:
users_groups = Table('users_groups', Model.metadata,
Column('user_id', Integer, ForeignKey('users.id')),
Column('group_id', Integer, ForeignKey('groups.id'))
)
class User(Model):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
class Group(Model):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
users = relationship('User', secondary=users_groups, lazy='select', backref='groups')
users_dynamic = relationship('User', secondary=users_groups, lazy='dynamic')
So what happens here is that if you add a bunch of users to a group like so:
g = Group()
g.users = [User(), User(), User()]
session.add(g)
session.commit()
and then try to delete the group
session.delete(g)
session.commit()
You will get some form of this error:
DELETE statement on table 'users_groups' expected to delete 3 row(s); Only 0 were matched.
Removing the 2nd version of the relationship (the dynamic one in my case) fixes this problem. I am not even sure where to begin in terms of understanding why this is happening. I have been using 2 versions of various relationships in many cases throughout my SQLAlchemy models in order to make it easy to use the most appropriate query-strategy given a situation. This is the first time it has caused an unexpected issue.
Any advice is welcome.

both the Group.users and Group.users_dynamic relationships are attempting to reconcile the fact that the Group is being deleted along with being able to manage the User() objects they refer to; one relationship succeeds while the second one fails, as the rows in the association table were already deleted. The most straightforward solution is to mark all but one of the identical relationships as viewonly:
class Group(Base):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
users = relationship('User', secondary=users_groups, lazy='select', backref='groups')
users_dynamic = relationship('User', viewonly=True, secondary=users_groups, lazy='dynamic')
if you're still wanting to have both relationships handle some degree of mutations, you'd need to do this carefully as SQLAlchemy doesn't know how to coordinate among changes in two relationships at the same time, so conflicts like this can continue to happen (like double inserts, etc) if you make equivalent mutations on both relationships. To just take care of the "delete" issue by itself, you can also try setting Group.users_dynamic to passive_deletes=True:
class Group(Base):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
users = relationship('User', secondary=users_groups, lazy='select', backref='groups')
users_dynamic = relationship('User', passive_deletes=True, secondary=users_groups, lazy='dynamic')

I just add another simple workaround.
You can delete the collections before deleting the item itself:
>>> for user in group.users:
group.users.remove(user)
>>> db.session.delete(group)
>>> db.session.commit()
Alternatively, you can also set it as an empty list:
>>> group.users = []
>>> db.session.commit()
>>> db.session.delete(group)
>>> db.session.commit()

Sqlalchemy: One to Many relationship combined with Many to Many relationship

I've got a User and Group table with a many to many relationship
_usergroup_table = db.Table('usergroup_table', db.metadata,
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('group_id', db.Integer, db.ForeignKey('group.id')))
class User(db.Model):
"""Handles the usernames, passwords and the login status"""
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), nullable=False, unique=True)
class Group(db.Model):
"""Used for unix-style access control."""
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), nullable=False)
users = db.relationship('User', secondary=_usergroup_table,
backref='groups')
Now i'd like to add a primary group to the user class. Of course I could just add a group_id column and a relationship to the Group class, but this has drawbacks. I'd like to get all groups when calling User.group, including primary_group. The primary group should always be part of the groups relationship.
Edit:
It seems the way to go is the association object
class User(db.Model, UserMixin):
"""Handles the usernames, passwords and the login status"""
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), nullable=False, unique=True)
primary_group = db.relationship(UserGroup,
primaryjoin="and_(User.id==UserGroup.user_id,UserGroup.primary==True)")
class Group(db.Model):
"""Used for unix-style access control."""
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), nullable=False)
class UserGroup(db.Model):
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
group_id = db.Column(db.Integer, db.ForeignKey('group.id'))
active = db.Column(db.Boolean, default=False)
user = db.relationship(User, backref='groups', primaryjoin=(user_id==User.id))
group = db.relationship(Group, backref='users', primaryjoin=(group_id==Group.id))
I could simplify this with the AssociationProxy, but how do I force only a single primary group per user?

The group_id approach you originally thought of has several advantages here to the "boolean flag" approach.
For one thing, it is naturally constrained so that there is only one primary group per user. For another, loading user.primary_group means the ORM can identify this related row by it's primary key, and can look locally in the identity map for it, or emit a simple SELECT by primary key, instead of emitting a query that has a hard-to-index WHERE clause with a boolean inside of it. Yet another is there's no need to get into the association object pattern which simplifies the usage of the association table and allows SQLAlchemy to handle loads and updates from/to this table more efficiently.
Below we use events, including a new version (as of 0.7.7) of #validates that catches "remove" events, to ensure object-level modifications to User.groups and User.primary_group are kept in sync. (If on an older version of 0.7 you can use the attribute "remove" event or the "AttributeExtension.remove" extension method if you're still on 0.6 or earlier). If you wanted to enforce this at the DB level you could possibly use triggers to verify the integrity you're looking for:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base= declarative_base()
_usergroup_table = Table('usergroup_table', Base.metadata,
Column('user_id', Integer, ForeignKey('user.id')),
Column('group_id', Integer, ForeignKey('group.id')))
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(60), nullable=False, unique=True)
group_id = Column(Integer, ForeignKey('group.id'), nullable=False)
primary_group = relationship("Group")
#validates('primary_group')
def _add_pg(self, key, target):
self.groups.add(target)
return target
#validates('groups', include_removes=True)
def _modify_groups(self, key, target, is_remove):
if is_remove and target is self.primary_group:
del self.primary_group
return target
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
name = Column(String(60), nullable=False)
users = relationship('User', secondary=_usergroup_table,
backref=backref('groups', collection_class=set))
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
g1, g2, g3 = Group(name='g1'), Group(name='g2'), Group(name='g3')
u1 = User(name='u1', primary_group=g1)
u1.groups.update([g2, g3])
s.add_all([
g1, g2, g3, u1
])
s.commit()
u1.groups.remove(g1)
assert u1.primary_group is None
u1.primary_group = g2
s.commit()

How about a GroupMemberships model to hold the association, instead of _usergroup_table? A user could have many groups through Group Memberships, and a group membership can hold additional attributes, such as whether a given Group is the associated User's primary group.
EDIT
In order to enforce a limit of one primary group per user, I would use a validation in the User model, such that any attempt to assign more (or fewer) than one primary group would result in an error when the record is saved. I am not aware of a way of achieving the same result relying purely on the database's integrity system. There are any number of ways of coding the validation check - the documentation shows a nice approach using the validates() decorator.