When I try to join a many-to-many table and group it by the main-id I am getting duplicates when I add the second many-to-many table.
Here is how my models look like:
Models
user
class User(UserMixin, db.Model):
id = db.Column(db.Integer, primary_key=True)
user_fistName = db.Column(db.String(64))
...
student_identifier
student_identifier = db.Table('student_identifier',
db.Column('class_id', db.Integer, db.ForeignKey('class.class_id')),
db.Column('id', db.Integer, db.ForeignKey('user.id'))
)
class
class Class(db.Model):
sqlite_autoincrement=True
class_id = db.Column(db.Integer, primary_key=True)
class_name = db.Column(db.String(128), unique=True)
mm_children = db.relationship('User', secondary=student_identifier, backref=db.backref('classes'))
class_course_identifier
class_course_identifier = db.Table('class_course_identifier',
db.Column('course_id', db.Integer, db.ForeignKey('course.course_id')),
db.Column('class_id', db.Integer, db.ForeignKey('class.class_id'))
)
database structure
Well I am using SQLAlchemy to select the desired tables with the data I want. with this session.query
db.session.query(
Class.class_id,
Class.class_name,
func.group_concat(User.user_fistName),
func.group_concat(Course.course_name)
).filter(Class.courses, User.classes).group_by(Class.class_id)
the problem is that I am getting duplicates of both the courses AND names, so if the course has two users it will print the students and the course two times.
Here is how it is looking:
wrong view
And here is how it should look:
correct view
the problem
the problem is coming when I am adding the second many-to-many table, for example users/student-identifier. If I remove the line where I "join" it, I am getting the duplicates. Is there anyway to correct this? Or should I use RAW-SQL instead(and if yes, how?)
Found out the solution, and it is quite simple.
RAW SQL
SELECT
class.class_id,
class.class_name,
GROUP_CONCAT(DISTINCT course.course_name),
GROUP_CONCAT(DISTINCT user.user_fistName)
FROM
class
JOIN class_course_identifier ON class.class_id = class_course_identifier.class_id
JOIN course ON class_course_identifier.course_id = course.course_id
JOIN student_identifier ON class.class_id = student_identifier.class_id
JOIN user ON student_identifier.id = user.id
GROUP BY class.class_id
SQLAlchemy
db.session.query(
Class.class_id,
Class.class_name,
func.group_concat(User.user_fistName.distinct()),
func.group_concat(Course.course_name.distinct())
).filter(Class.courses, User.classes).group_by(Class.class_id)
Simply add the distinct() to the desired column you want to be unique
Related
In a Flask app that uses SQLAlchemy, I have tables for users and assets. The users belong to zero to multiple groups, and the access to each asset is allowed to zero to several of these groups. These relationships are modeled with association tables (the basic structure is shown below). I can query the database and the association tables work as intended.
The part where I am struggling is to, given a user, retrieve the assets that this user is allowed to access. I understand that I need to join on groups.
In SQL, the following statement gives the result that I need:
select * from user as u
join association_user_group as aug on u.id == aug.user_id
join association_asset_group as aag on aag.group_id = aug.group_id
where username='some_name';
However, I can't figure out how to translate this to Flask-SQLAlchemy, leveraging its benefits (which I like in many contexts).
For simplicity, let's assume the user under consideration is User.query.first() (in my code I have a reference to this object).
The basic database definition is as follows:
association_user_group = db.Table(
'association_user_group',
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('group_id', db.Integer, db.ForeignKey('group.id'))
)
association_asset_group = db.Table(
'association_asset_group',
db.Column('asset_id', db.Integer, db.ForeignKey('asset.id')),
db.Column('group_id', db.Integer, db.ForeignKey('group.id'))
)
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
...
class Asset(db.Model):
__tablename__ = 'asset'
id = db.Column(db.Integer, primary_key=True)
...
class Group(db.Model):
__tablename__ = 'group'
id = db.Column(db.Integer, primary_key=True)
...
users = db.relationship(
'User',
secondary=association_user_group,
backref=db.backref('groups', lazy='dynamic'))
asset = db.relationship(
'Asset',
secondary=association_asset_group,
backref=db.backref('groups', lazy='dynamic'))
Update: In the meanwhile, the following kind of works, but it does not make use of the db.relationships defined in the classes, which seems like a shame:
db.session.query(association_asset_group) \
.join(association_user_group,
association_user_group.c.group_id
== association_asset_group.c.group_id) \
.filter(association_user_group.c.user_id == u.id)
This seems to do the job:
q = db.session.query(Asset) \
.join(Group, User.groups) \
.join(Asset, Group.assets) \
.filter(User.id == u.id) \
.distinct()
But I have to say that I find it difficult to grasp the glue that flask-sqlalchemy provides under the hood (and when not).
I am joining two models without a foreign key:
Models:
class Users(db.Model):
__tablename__ = "Users"
userName = db.Column(db.String, primary_key=True)
lastLogin = db.Column(db.DateTime)
class TimeOff
__tablename__ = "timeOff"
timeOffID = db.Column(db.Integer, primary_key=True)
userName = db.Column("userName", db.String, db.ForeignKey('appUsers.userName')),
dayWork = db.Column(db.DateTime)
View:
result = db.session.query(models.Users).join(models.TimeOff)
sqlalchemy.exc.InvalidRequestError: Could not find a FROM clause to join from.
Tried joining to but got: Can't find any foreign key relationships between 'TimeOff' and 'Users'.
I dont have a foreign key defined in table
You need to tell SQLAlchemy how to join the tables. Try something like this:
result = db.session.query(Users).join(TimeOff,Users.userName==TimeOff.userName)
To improve upon #Matt Healy's answer, if you also want to be able to access attributes on the joined object you can do something like:
user, timeOff = db.session.query(Users, TimeOff).join(
TimeOff, Users.userName == TimeOff.userName
).first()
Then timeOff.dayWork etc. will give the information you need.
In my situation, I want to join the two objects so that I can get the Author information of the Posts, but I don't want to use the architecture-level foreign key design to associate the two tables.here is my codes:
class Posts(db.Model):
id= db.Column(db.String, primary_key=True)
author_code= db.Column(db.String)
...
class Author:
id= db.Column(db.Integer, primary_key=True)
name= db.Column(db.String),
code= db.Column(db.String)
address= db.Column(db.String)
...
we can add the code like this:
from sqlalchemy.orm import relationship
class Posts(db.Model):
id= db.Column(db.String, primary_key=True)
author_code= db.Column(db.String)
...
# add this line
author_info = relationship('Author',
foreign_keys=[author_code],
primaryjoin='Author.code == Posts.author_code')
then we can get author_info by query like this:
post_id = xxx # some id you defined
post = Posts.query.filter_by(id=post_id).one_or_none()
# here you can get `author_info` attributes you want.
print(dir(post),post.author_info)
You can learn more about this with link here,and read the documents Configuring how Relationship Joins — SQLAlchemy 1.4 Documentation
Its an old post but I had a similar problem
result = session.query(models.Users).join(models.TimeOff, models.Users.userName == models.TimeOff.userName).all()
with this method, I can reach the features of the first object which is Users but not the TimeOff. I am wondering if it is possible to reach the secondary object's attributes. But I hope this helps.
This worked for me,
I have 3 tables, And I have the rtf_id which is unique to all three tables, you have to use the select_from keyword which tells the table starting from left.
db_data = dbobj.session.query(A, B, C). \
select_from(A).join(B, B.rtf_id == A.rtf_id). \
join(C, C.rtf_id == A.rtf_id).all()
I am using Flask-SQLAlchemy to to query my Postgres database.
I am currently trying to query for suggestions of titles with the following query:
res = Title.query.filter(Titles.name.ilike(searchstring)).limit(20)
So far so good.
Now I would like to order the results by the number of "subscribers" each Title object has.
I am aware of the following SO question: SQLAlchemy ordering by count on a many to many relationship however its solution did not work for me.
I am receiving the following error:
ProgrammingError: (ProgrammingError) column "publishers_1.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ...itles_name, count(titles_users.user_id) AS total, publishers...
(Publisher has a one-to-many relationship with Title models)
I understand that the answer has something to do with subqueries
Below is a simplified example of my two models.
# Many-to-Many relationship for user and titles
titles_users = db.Table('titles_users',
db.Column('user_id', db.Integer, db.ForeignKey('users.id')),
db.Column('title_id', db.Integer, db.ForeignKey('titles.id'))
)
class User(db.Model, UserMixin):
__tablename__ = 'users'
# ids
id = db.Column(db.Integer, primary_key=True)
# Attributes
email = db.Column(db.String(255), unique=True)
full_name = db.Column(db.String(255))
pull_list = db.relationship(
'Title',
secondary=titles_users,
backref=db.backref('users', lazy='dynamic'),
lazy='joined'
)
class Title(db.Model):
__tablename__ = 'titles'
#: IDs
id = db.Column(db.Integer(), primary_key=True)
#: Attributes
name = db.Column(db.String(255))
Code below would work for the model you describe:
q = (db.session.query(Title, func.count(names_users.c.user_id).label("total"))
.filter(Title.name.ilike(searchstring))
.outerjoin(names_users).group_by(Title).order_by('total DESC')
)
for x in q:
print(x)
Your error, however, includes some other data like publisher or like. So if above is not helpful, you should add more information to your question.
I am trying to do a many-to-many relationship table with more information than just the two ids. It doesnt work. I obviously standardised my table with generic names. But the use case could be like a user table , with a post table and a likes relationship table, something like that. when i do these table, python gave me that :
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Table2.rela1 - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
So here is my relationship table
class Relationship(db.Model):
id = db.Column(db.Integer, primary_key = True)
table1_id = db.Column(db.Integer, db.ForeignKey('Table1.id')),
table2_id = db.Column(db.Integer, db.ForeignKey('Table2.id')),
value = db.Column(db.Integer), #a number
date = db.Column(db.DateTime) #actual time of the entry
here is my table1
class Table1(db.Model):
id = db.Column(db.Integer, primary_key = True)
rela1 = db.relationship('Relationship', backref = 'rel1', lazy = 'dynamic')
rela2 = db.relationship('Table2', backref = 'rel2', lazy = 'dynamic')
here is my table 2
class Table2(db.Model):
id = db.Column(db.Integer, primary_key = True)
table1_id = db.Column(db.Integer, db.ForeignKey('table1.id'))
rela1 = db.relationship('Relationship', backref = 'rel3', lazy = 'dynamic')
Thanks for helping me.
If that can be solved then my second issue is creating a function to compute the total 'value' per Table2 object, regardless of the which table1 object post the value. something like a select with sum and group by table2.id , but i dont really understand how to do it with python and flask and sqlalchemy.
Thanks.
EDIT1
Using http://docs.sqlalchemy.org/en/rel_0_9/orm/relationships.html#association-object
i changed to
class Relationship(db.Model):
__tablename__ = 'relationship'
table1_id = db.Column(db.Integer, db.ForeignKey('Table1.id'), primary_key = True),
table2_id = db.Column(db.Integer, db.ForeignKey('Table2.id'), primary_key = True),
value = db.Column(db.Integer), #a number
date = db.Column(db.DateTime) #actual time of the entry
table2obj = db.relationship("Table2", backref="table2_assocs")
then
class Table2(db.Model):
__tablename__ = 'table2'
id = db.Column(db.Integer, primary_key = True)
table1_id = db.Column(db.Integer, db.ForeignKey('table1.id'))
and table 1 is unchanged except the addition of tablename
but now i get
sqlalchemy.exc.ArgumentError: Mapper Mapper|Relationship|relatioship could not assemble any primary key columns for mapped table 'Relationship'
there are two relationship declarations on Table1, but from the code you've posted, I can see only one path to Table2, through Relationship.table2_id.
It looks like you are trying to have a many-to-many relationship with extra metadata on the intermediate relation. This is called the association object pattern in the sqlalchemy docs. For that, everything you've got so far, except the Table1.rela2 is good, you pretty well have it. That "rogue" relationship needs to be removed.
to get the convenience of having a real relationship from Table1 to Table2, you need to use an association proxy.
I have two models in SQLAlchemy, having a many-to-many relationship
team_user_table = Table('team_user', Base.metadata,
Column('user_id', Integer, ForeignKey('users.id')),
Column('team_id', Integer, ForeignKey('teams.id'))
)
class User(Base):
""" The SQLAlchemy declarative model class for a User object. """
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String, unique=True)
class Team(Base):
""" The SQLAlchemy declarative model class for a Team object. """
__tablename__ = 'teams'
id = Column(Integer, primary_key=True)
name = Column(Text, unique=True)
members = relationship("User",
secondary=team_user_table,
backref="memberteams")
I would like to query those users that are not member of a specific team.
In SQL (for example):
SELECT u.id,u.name FROM users u WHERE u.id NOT IN (SELECT tu.user_id FROM team_user tu WHERE tu.team_id=?);
How can I do this in SQLAlchemy?
This does what you want:
team_id = 1
query = session.query(User.id).filter(~User.memberteams.any(Team.id == team_id))
This is the SQL it outputs (on MySQL):
SELECT users.id AS users_id, users.name AS users_name
FROM users
WHERE NOT (EXISTS (SELECT 1
FROM team_user, teams
WHERE users.id = team_user.user_id AND teams.id = team_user.team_id AND teams.id = %s))
This is not exactly what your query looks like now, but I think this is the way it should be done. Check the docs for this.
I tested this on MySQL. However, I think it should work on any other.
To have the exact query you are looking for, you might need to look into using subqueries in combination with the filter statement. I think you will need explicit reference to the team_user table in your code to make it work.
I believe this should work and it does not use subqueries.
session.query(User).join(team_users_table).filter(team_users_table.team_id != OTHER_TEAM)