Develop Reference

How to order query results for parent by related field for one-to-many relationship in SQLAlchemy

I have following DB models
class User(Base):
__tablename__ = "user"
user_id = Column("id", Integer(), primary_key=True)
groups = relationship(
"Group", back_populates="user", lazy="selectin", cascade="all, delete-orphan",
)
class Group(Base):
__tablename__ = "group"
group_id = Column("id", Integer(), primary_key=True)
user_id = Column(
Integer,
ForeignKey("user.id", ondelete="CASCADE"),
index=True,
)
division_id = Column(
String(96),
ForeignKey("division.id", onupdate="CASCADE"),
nullable=False,
)
name = Column(String(64), nullable=False)
user = relationship("User", back_populates="groups", lazy="selectin")
group = relationship("Division", back_populates="groups", lazy="selectin")
class Division(Base):
__tablename__ = "division"
division_id = Column("id", Integer, primary_key=True)
name = Column(String(64), nullable=False)
groups = relationship("Group", back_populates="group", lazy="selectin")
I want to fetch all the users ordered by their groups(can be something else as well, need to come from enduser), which I can easily achieve using the following query
session.query(User).join(Group).join(Division).order_by(Group.name).all()
And it might look that this works just fine but it doesn't, because since a user might have multiple groups, so in order to have correct result I first need to sort the groups for each user object i.e. something like sorted(User.group.order_by(Group.name) and then apply the order_by on the User model based on these sorted groups.
And the same thing can apply to division names as well. I know that we can provide default order_by fields while defining the relationship like below but that's not what I want since the order_by field need to come from enduser and can be any other field as well.
groups = relationship("Group", back_populates="user", lazy="selectin", cascade="all, delete-orphan",order_by=("Group.name"))
I can do this at data layer in python but that would not be ideal since there is already some ordering being done at DB layer.
So how can I achieve this at DB layer using SQLAlchemy or even with raw sql. Or is it even possible with sql?

SQLAlchemy many to many dynamic lazyload not returning AppenderQuery

I'm trying to get a grip on the backref lazyload dynamic feature of SQLAlchemy ORM.
I have 3 tables and two link tables.
course_members = Table('course_members', Base.metadata,
Column('user_id', Integer, ForeignKey('users.id')),
Column('course_id', Integer, ForeignKey('courses.id'))
)
course_roles = Table('course_roles', Base.metadata,
Column('role_id', Integer, ForeignKey('roles.id')),
Column('course_id', Integer, ForeignKey('courses.id'))
)
class User(Base):
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String(200), nullable=False)
class Course(Base, Jsonify):
id = Column(Integer, primary_key=True, autoincrement=True)
members = relationship('User', secondary=course_members, backref=backref('courses', lazy='dynamic'))
roles = relationship('Role', secondary=course_roles, backref=backref('roles', lazy='dynamic'))
class Role(Base):
id = Column(Integer, primary_key=True, autoincrement=True)
role_id = Column(Integer, nullable=False)
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship('User', backref=backref("roles", lazy='dynamic'))
UniqueConstraint('role_id', 'user_id', name='role_combination')
With a user query:
print(type(user.roles))
print(user.roles.filter_by(id=1).first())
# print(user.courses.first() <-- Works fine too.
<class 'sqlalchemy.orm.dynamic.AppenderQuery'>
{id: 1, 'user_id': 1, role_id: 3}
But with a course query:
print(type(course.roles))
print(course.roles.filter_by(id=1).first())
<class 'sqlalchemy.orm.collections.InstrumentedList'>
AttributeError: 'InstrumentedList' object has no attribute 'filter_by'
Same result when I try with members.
Using the members and courses as list objects ofc work:
course.roles[0].user.first()
But I really miss the functions of the AppenderQuery class for my course query.
Is this normal behavior or am I missing something?

While writing the question and researching I found the answer to the problem in this post add dynamic to the other side.
And after seeing the answer I understood more of the relationship functionality as well.
members = relationship('User', secondary=course_members, lazy='dynamic', backref=backref('courses', lazy='dynamic'))
roles = relationship('Role', secondary=course_roles, lazy='dynamic', backref=backref('roles', lazy='dynamic'))
The members and roles relationships are ofc the functionality of the parent class, and the backref=* is ofc the relationship functionality of the child class. It took me more time than I liked to realize this. But by having lazy='dynamic' as a parameter in both the relationship function, and the backref function for the backref parameter effectively applies this to both sides of the relationship.
And now users, courses and roles are returned as AppenderQuery.
Hopefully this will help others in search of the question.

SQLAlchemy many to many relationship with an extra column [duplicate]

I have 3 tables: User, Community, community_members (for relationship many2many of users and community).
I create this tables using Flask-SQLAlchemy:
community_members = db.Table('community_members',
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('community_id', db.Integer, db.ForeignKey('community.id')),
)
class Community(db.Model):
__tablename__ = 'community'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False, unique=True)
members = db.relationship(User, secondary=community_members,
backref=db.backref('community_members', lazy='dynamic'))
Now I want add additional field to community_members like this:
community_members = db.Table('community_members',
db.Column('id', db.Integer, primary_key=True),
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('community_id', db.Integer, db.ForeignKey('community.id')),
db.Column('time_create', db.DateTime, nullable=False, default=func.now()),
)
And now in python shell I can do this:
create community:
> c = Community()
> c.name = 'c1'
> db.session.add(c)
> db.session.commit()
add members to community:
> u1 = User.query.get(1)
> u2 = User.query.get(2)
> c.members.append(u1)
> c.members.append(u2)
> db.session.commit()
> c.members
[<User 1>, <User 2>]
Ok, this works.
But how now I can get time_create of community_members table?

You will have to switch from using a plain, many-to-many relationship to using an "Association Object", which is basically just taking the association table and giving it a proper class mapping. You'll then define one-to-many relationships to User and Community:
class Membership(db.Model):
__tablename__ = 'community_members'
id = db.Column('id', db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
community_id = db.Column(db.Integer, db.ForeignKey('community.id'))
time_create = db.Column(db.DateTime, nullable=False, default=func.now())
community = db.relationship(Community, backref="memberships")
user = db.relationship(User, backref="memberships")
class Community(db.Model):
__tablename__ = 'community'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False, unique=True)
But you may only occasionally be interested in the create time; you want the old relationship back! well, you don't want to set up the relationship twice; because sqlalchemy will think that you somehow want two associations; which must mean something different! You can do this by adding in an association proxy.
from sqlalchemy.ext.associationproxy import association_proxy
Community.members = association_proxy("memberships", "user")
User.communities = association_proxy("memberships", "community")

If you only need query community_members and community table by a known user_id(such as user_id=2), In SQLAlchemy, you can perform:
session.query(community_members.c.time_create, Community.name).filter(community_members.c.user_id==2)
to get the result.

SQLAlchemy join with association tables

In a Flask app that uses SQLAlchemy, I have tables for users and assets. The users belong to zero to multiple groups, and the access to each asset is allowed to zero to several of these groups. These relationships are modeled with association tables (the basic structure is shown below). I can query the database and the association tables work as intended.
The part where I am struggling is to, given a user, retrieve the assets that this user is allowed to access. I understand that I need to join on groups.
In SQL, the following statement gives the result that I need:
select * from user as u
join association_user_group as aug on u.id == aug.user_id
join association_asset_group as aag on aag.group_id = aug.group_id
where username='some_name';
However, I can't figure out how to translate this to Flask-SQLAlchemy, leveraging its benefits (which I like in many contexts).
For simplicity, let's assume the user under consideration is User.query.first() (in my code I have a reference to this object).
The basic database definition is as follows:
association_user_group = db.Table(
'association_user_group',
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('group_id', db.Integer, db.ForeignKey('group.id'))
)
association_asset_group = db.Table(
'association_asset_group',
db.Column('asset_id', db.Integer, db.ForeignKey('asset.id')),
db.Column('group_id', db.Integer, db.ForeignKey('group.id'))
)
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
...
class Asset(db.Model):
__tablename__ = 'asset'
id = db.Column(db.Integer, primary_key=True)
...
class Group(db.Model):
__tablename__ = 'group'
id = db.Column(db.Integer, primary_key=True)
...
users = db.relationship(
'User',
secondary=association_user_group,
backref=db.backref('groups', lazy='dynamic'))
asset = db.relationship(
'Asset',
secondary=association_asset_group,
backref=db.backref('groups', lazy='dynamic'))
Update: In the meanwhile, the following kind of works, but it does not make use of the db.relationships defined in the classes, which seems like a shame:
db.session.query(association_asset_group) \
.join(association_user_group,
association_user_group.c.group_id
== association_asset_group.c.group_id) \
.filter(association_user_group.c.user_id == u.id)

This seems to do the job:
q = db.session.query(Asset) \
.join(Group, User.groups) \
.join(Asset, Group.assets) \
.filter(User.id == u.id) \
.distinct()
But I have to say that I find it difficult to grasp the glue that flask-sqlalchemy provides under the hood (and when not).

sqlalchemy: referencing label()'d column in a filter or clauselement

I'm trying to perform a query that works across a many->many relation ship between bmarks and tags with a secondary table of bmarks_tags. The query involves several subqueries and I have a need to DISTINCT a column. I later want to join that to another table via the DISTINCT'd ids.
I've tried it a few ways and this seems closest:
tagid = alias(Tag.tid.distinct())
test = select([bmarks_tags.c.bmark_id],
from_obj=[bmarks_tags.join(DBSession.query(tagid.label('tagid'))),
bmarks_tags.c.tag_id == tagid])
return DBSession.execute(qry)
But I get an error:
⇝ AttributeError: '_UnaryExpression' object has no attribute 'named_with_column'
Does anyone know how I can perform the join across the bmarks_tags.tag_id and the result of the Tag.tid.distinct()?
Thanks
Schema:
# this is the secondary table that ties bmarks to tags
bmarks_tags = Table('bmark_tags', Base.metadata,
Column('bmark_id', Integer, ForeignKey('bmarks.bid'), primary_key=True),
Column('tag_id', Integer, ForeignKey('tags.tid'), primary_key=True)
)
class Tag(Base):
"""Bookmarks can have many many tags"""
__tablename__ = "tags"
tid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(255), unique=True)

Something like this should work:
t = DBSession.query(Tag.tid.distinct().label('tid')).subquery('t')
test = select([bmarks_tags.c.bmark_id], bmarks_tags.c.tag_id == t.c.tid)
return DBSession.execute(test)

It is hard to tell what you are trying to accomplish, but since you are using orm anyways (and there is not much reason anymore to go with bare selects in sa these days), you should probably start by establishing a many-to-many relation:
bmarks_tags = Table('bmark_tags', Base.metadata,
Column('bmark_id', Integer, ForeignKey('bmarks.bid'), primary_key=True),
Column('tag_id', Integer, ForeignKey('tags.tid'), primary_key=True)
)
class Tag(Base):
"""Bookmarks can have many many tags"""
__tablename__ = "tags"
tid = Column(Integer, primary_key=True)
name = Column(Unicode(255), unique=True)
class BMark(Base):
__tablename__ = 'bmarks'
bid = Column(Integer, primary_key=True)
tags = relation(Tag, secondary=bmarks_tags, backref="bmarks")
Then get your query and go from there:
query = DBSession.query(BMark).join(BMark.tags)
If not, give us the actual sql you are trying to make sqlalchemy emit.