How do I join 3 tables, Clients, Orders and Deposits and perform func.sum on Orders.total and Deposits.total for each Client.id that exists in the database ? The query result should include columns Clients.email, func.sum(Orders.total) and func.sum(Deposits.total).
So far, I've tried different queries along the lines of :
listeclients = db.session.query(Clients,func.sum(Clients.orders.total).\
label("ctotal"),func.sum((Clients.deposits.total).\
label("dtotal"))).group_by(Client.id).all()
giving me different errors such as:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Clients.orders has an attribute 'total'
I would like to see how one does this in sqlalchemy, but I'd also settle for hints behind the logic of such a query ...
Are my mappings correct? What is the syntax of such a join? Should I use eagerload somewhere? I've had success with simpler queries, but one like this is over my head for now ! Any help is welcome, even just the logic of it in raw SQL. I'm stuck w/ this ...
class Clients(db.Model):
__tablename__ = 'clients'
id = db.Column(db.Integer, primary_key = True)
email = db.Column(db.String(60), index = True, unique = True)
adresse = db.Column(db.String(64), index = True)
telephone = db.Column(db.String(10), index = True)
confirmed = db.Column(db.Boolean, default = False)
orders = db.relationship('Orders')
deposits = db.relationship('Deposits')
class Orders(db.Model):
__tablename__ = 'orders'
id = db.Column(db.Integer, primary_key = True)
client_id = db.Column(db.Integer, db.ForeignKey('clients.id'))
total = db.Column(db.Float)
date = db.Column(db.DateTime, index = True, default=datetime.now)
client = db.relationship('Clients')
class Deposits(db.Model):
__tablename__='deposits'
id = db.Column(db.Integer, primary_key = True)
date = db.Column(db.DateTime, index = True, default=datetime.now)
client_id = db.Column(db.Integer, db.ForeignKey('clients.id'))
total = db.Column(db.Float)
cheque = db.Column(db.Boolean)
client = db.relationship('Clients')
update: updated the query below to handle properly the sum:
sq1 = (db.session.query(Orders.client_id, func.sum(Orders.total).label("ctotal"))
.group_by(Orders.client_id)).subquery("sub1")
sq2 = (db.session.query(Deposits.client_id, func.sum(Deposits.total).label("dtotal"))
.group_by(Deposits.client_id)).subquery("sub2")
q = (db.session.query(Clients, sq1.c.ctotal, sq2.c.dtotal)
.outerjoin(sq1, sq1.c.client_id == Clients.id)
.outerjoin(sq2, sq2.c.client_id == Clients.id)
)
Also, instead of defining relationships twice (which might actually fail on some versions of sqlalchemy), you can simply use backref:
class Clients(db.Model):
orders = db.relationship('Orders', backref='client')
deposits = db.relationship('Deposits', backref='client')
class Orders(db.Model):
# client = db.relationship('Clients')
class Deposits(db.Model):
# client = db.relationship('Clients')
In sql it is straightforward:
select c.email, sum(o.total), sum(d.total)
from Clients c
left join Orders o
on ...
left join Deposits d
on ...
group by c.email
Related
AttributeError: 'Query' object has no attribute 'is_clause_element' when joining table with query
I have a query that counts the amount of keywords a company has and then sorts them by the amount of keywords they have.
query_company_ids = Session.query(enjordplatformCompanyToKeywords.company_id.label("company_id"),func.count(enjordplatformCompanyToKeywords.keyword_id)).group_by(enjordplatformCompanyToKeywords.company_id).order_by(desc(func.count(enjordplatformCompanyToKeywords.keyword_id))).limit(20)
I then want to get information about these companies like image, title, info etc and send it to the frontend (this is done later by looping through companies_query).
Though I have trouble in building the connection between the query_company_ids query and enjordplatformCompanies table.
I have tried two ways of doing this:
companies_query = Session.query(enjordplatformCompanies, query_company_ids).filter(enjordplatformCompanies.id == query_company_ids.company_id).all()
companies_query = Session.query(enjordplatformCompanies, query_company_ids).join( query_company_ids, query_company_ids.c.company_id == enjordplatformCompanies.id).all()
But both of them result in the error: AttributeError: 'Query' object has no attribute 'is_clause_element'
Question
How can I join the query_company_ids query and enjordplatformCompanies table?
Thanks
Here are the table definitions
class enjordplatformCompanies(Base):
__tablename__ = "enjordplatform_companies"
id = Column(Integer, primary_key=True, unique=True)
name = Column(String)
about = Column(String)
image = Column(String)
website = Column(String)
week_added = Column(Integer)
year_added = Column(Integer)
datetime_added = Column(DateTime)
created_by_userid = Column(Integer)
company_type = Column(String)
contact_email=Column(String)
adress=Column(String)
city_code=Column(String)
city=Column(String)
class enjordplatformCompanyToKeywords(Base):
__tablename__ = "enjordplatform_company_to_keywords"
id = Column(Integer, primary_key=True, unique=True)
company_id = Column(Integer,ForeignKey("enjordplatform_companies.id"))
keyword_id = Column(Integer,ForeignKey("enjordplatform_keywords.id"))
I copied your example query above and was getting a lot of weird errors until I realized you use Session instead of session. I guess make sure you are using an instance instead of the class or sessionmaker.
Below I create an explicit subquery() to get the company id paired with its keyword count and then I join the companies class against that, applying the order and limit to the final query.
with Session(engine) as session, session.begin():
subq = session.query(
enjordplatformCompanyToKeywords.company_id,
func.count(enjordplatformCompanyToKeywords.keyword_id).label('keyword_count')
).group_by(
enjordplatformCompanyToKeywords.company_id
).subquery()
q = session.query(
enjordplatformCompanies,
subq.c.keyword_count
).join(
subq,
enjordplatformCompanies.id == subq.c.company_id
).order_by(
desc(subq.c.keyword_count)
)
for company, keyword_count in q.limit(20).all():
print (company.name, keyword_count)
This isn't the exact method but explains the intention of calling .subquery() above:
subquery
I have two databases:
class AcUsers(db.Model):
__tablename__ = "ac_users"
id = Column(Integer, primary_key=True)
b2c_customer_id = Column(Integer, ForeignKey("b2c_customer.id",
ondelete="CASCADE"))
ac_tests = db.relationship(
"AcTests",
back_populates="ac_user",
uselist=False,
lazy="joined",
)
last_test_submit_time = association_proxy("ac_tests",
"last_test_submit_time")
class AcTests(db.Model):
__tablename__ = "ac_tests"
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey("ac_users.id"))
ac_user = db.relationship(
"AcUsers",
back_populates="ac_tests",
uselist=False,
lazy="joined",
)
first_test_started_time = Column(DateTime)
last_test_submit_time = Column(DateTime)
I have tried to query the database like this:
ac = (
db.session.query(
func.count(AcUsers.id),
func.week(AcUsers.last_test_submit_time),
func.year(AcUsers.last_test_submit_time),
)
.filter(AcUsers.results_sent)
.filter(AcUsers.last_test_submit_time != None)
.all()
)
I got this error:
NotImplementedError: The association proxy can't be used as a plain column expression; it only works inside of a comparison expression
How the I get past this error in order to be able to use the group by week and year in order to know exactly the user for the present dispensation.
I have tried to get the daily, monthly, and yearly data by getting the data in the AcUsers and looping through it:
ac_users = (
db.session.query(AcUsers)
.filter(AcUsers.results_sent)
.filter(AcUsers.last_test_submit_time != None)
.all()
)
daily:
for user in ac_users:
actual_date = user.last_test_submit_time.date()
if actual_date not in daily_data:
daily_data[actual_date] = 1
else:
daily_data[actual_date] += 1
results_sent_daily = []
for x, y in daily_data.items():
results_sent_daily.append({"x": x, "y": y})
I did the same for the monthly and yearly.
How do i make the query use the grouping func into func.year, func.week, func.date etc.
Thank you in advance
As stated in the association proxy documentation, the proxies are not meant for ORM querying aside from filtering.
The most simple thing to do is to craft a query using both models, specifying the join between them and the grouping fields.
ac = (
s.dbs.query(
func.count(AcUsers.id).label("count"),
func.week(AcTests.last_test_submit_time).label("week"),
func.year(AcTests.last_test_submit_time).label("year"),
)
.join(AcUsers.ac_tests)
.group_by(
func.week(AcTests.last_test_submit_time),
func.year(AcTests.last_test_submit_time)
)
.filter(AcTests.last_test_submit_time != None)
.all()
)
In a project using Flask-SQLAlchemy, i get some intermittent errors and i think it might be due to not explicitly using transactions.
I have these two model classes, one for locations and another for closures:
class Location(db.Model):
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String)
code = sa.Column(sa.String, unique=True)
class LocationPath(db.Model):
ancestor_id = sa.Column(sa.Integer, sa.ForeignKey('location.id'), nullable=False, primary_key=True)
descendant_id = sa.Column(sa.Integer, sa.ForeignKey('location.id'), nullable=False, primary_key=True)
depth = sa.Column(sa.Integer, default=0, nullable=False)
In a background process, i'm doing a lot of inserts, so i'm bypassing the ORM to use Core:
location_table = Location.__table__
location_path_table = LocationPath.__table__
statement = select([location_table.c.id]).where(code == code)
result = db.session.get_bind().execute(statement)
location_id = result.first()
if location_id is None:
statement = location_table.insert().values(**kwargs)
result = db.session.get_bind().execute(statement)
new_id = result.inserted_primary_key[0]
result.close()
else:
new_id = location_id
# save new_id as an ancestor_id or a descendant_id
path = LocationPath.query.filter_by(
ancestor_id=ancestor_id,
descendant_id=descendant_id
).first()
if path is None:
statement = location_path_table.insert().values(
ancestor_id=ancestor_id,
descendant_id=descendant_id,
depth=depth)
# the line below intermittently generates either of two errors:
# - the inserted primary key (ancestor/descendant) does not exist
# - a duplicate key error where the path already exists
result = db.session.get_bind().execute(statement)
this has resulted in quite a bit of head-scratching on my part, since i get the ancestor_id or descendant_id either from a select or an insert, and i also query the database to see if the path exists before attempting to insert it.
Edit: the code above runs in a loop.
I need help creating SqlAlchemy query.
I'm doing a Flask project where I'm using SqlAlchemy. I have created 3 tables: Restaurant, Dish and restaurant_dish in my models.py file.
restaurant_dish = db.Table('restaurant_dish',
db.Column('dish_id', db.Integer, db.ForeignKey('dish.id')),
db.Column('restaurant_id', db.Integer, db.ForeignKey('restaurant.id'))
)
class Restaurant(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(64), index = True)
restaurant_dish = db.relationship('Dish', secondary=restaurant_dish,
backref=db.backref('dishes', lazy='dynamic'))
class Dish(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(64), index = True)
info = db.Column(db.String(256), index = True)
I have added data to the restaurant_dish table and it should be working correctly. Where I need help is understanding how to correctly get a Dish using Restaurant. Raw SQL would be something like this:
SELECT dish_id FROM restaurant_dish WHERE restaurant_id == id
What I have managed to get done but not working:
x = Restaurant.query.filter_by(Restaurant.restaurant_dish.contains(name)).all()
Thanks for help and I also appreciate tutorials that can point me in the right direction(the official documentation goes over my head).
The semantic of the relationship doesn't look right. I think it should be something like:
class Restaurant(db.Model):
...
dishes = db.relationship('Dish', secondary=restaurant_dish,
backref=db.backref('restaurants'))
Then, to retrieve all the dishes for a restaurant, you can do:
x = Dish.query.filter(Dish.restaurants.any(name=name)).all()
This should generate a query like:
SELECT dish.*
FROM dish
WHERE
EXISTS (
SELECT 1
FROM restaurant_dish
WHERE
dish.id = restaurant_dish.dish_id
AND EXISTS (
SELECT 1
FROM restaurant
WHERE
restaurant_dish.restaurant_id = restaurant.id
AND restaurant.name = :name
)
)
First, the database overview:
competitors - people who compete
competitions - things that people compete at
competition_registrations - Competitors registered for a particular competition
event - An "event" at a competition.
events_couples - A couple (2 competitors) competing in an event.
First, EventCouple, a Python class corresponding to events_couples, is:
class EventCouple(Base):
__tablename__ = 'events_couples'
competition_id = Column(Integer, ForeignKey('competitions.id'), primary_key=True)
event_id = Column(Integer, ForeignKey('events.id'), primary_key=True)
leader_id = Column(Integer)
follower_id = Column(Integer)
__table_args__ = (
ForeignKeyConstraint(['competition_id', 'leader_id'], ['competition_registrations.competition_id', 'competition_registrations.competitor_id']),
ForeignKeyConstraint(['competition_id', 'follower_id'], ['competition_registrations.competition_id', 'competition_registrations.competitor_id']),
{}
)
I have a Python class, CompetitorRegistration, that corresponds to a record/row in competition_registrations. A competitor, who is registered, can compete in multiple events, but either as a "leader", or a "follower". I'd like to add to CompetitorRegistration an attribute leading, that is a list of EventCouple where the competition_id and leader_id match. This is my CompetitorRegistration class, complete with attempt:
class CompetitorRegistration(Base):
__tablename__ = 'competition_registrations'
competition_id = Column(Integer, ForeignKey('competitions.id'), primary_key=True)
competitor_id = Column(Integer, ForeignKey('competitors.id'), primary_key=True)
email = Column(String(255))
affiliation_id = Column(Integer, ForeignKey('affiliation.id'))
is_student = Column(Boolean)
registered_time = Column(DateTime)
leader_number = Column(Integer)
leading = relationship('EventCouple', primaryjoin=and_('CompetitorRegistration.competition_id == EventCouple.competition_id', 'CompetitorRegistration.competitor_id == EventCouple.leader_id'))
following = relationship('EventCouple', primaryjoin='CompetitorRegistration.competition_id == EventCouple.competition_id and CompetitorRegistration.competitor_id == EventCouple.follower_id')
However, I get:
ArgumentError: Could not determine relationship direction for primaryjoin
condition 'CompetitorRegistration.competition_id == EventCouple.competition_id
AND CompetitorRegistration.competitor_id == EventCouple.leader_id', on
relationship CompetitorRegistration.leading. Ensure that the referencing Column
objects have a ForeignKey present, or are otherwise part of a
ForeignKeyConstraint on their parent Table, or specify the foreign_keys parameter
to this relationship.
Thanks for any help, & let me know if more info is needed on the schema.
Also, another attempt of mine is visible in following — this did not error, but didn't give correct results either. (It only joined on the competition_id, and completely ignored the follower_id)
Your leading's condition mixes expression and string to be eval()ed. And following's condition mixes Python and SQL operators: and in Python is not what you expected here. Below are corrected examples using both variants:
leading = relationship('EventCouple', primaryjoin=(
(competition_id==EventCouple.competition_id) & \
(competitor_id==EventCouple.leader_id)))
leading = relationship('EventCouple', primaryjoin=and_(
competition_id==EventCouple.competition_id,
competitor_id==EventCouple.leader_id))
following = relationship('EventCouple', primaryjoin=\
'(CompetitorRegistration.competition_id==EventCouple.competition_id) '\
'& (CompetitorRegistration.competitor_id==EventCouple.follower_id)')