I'm currently executing this query in one process:
SELECT DISTINCT ON (c.api_key, worker_id) worker_id, c.api_key, a.updated_at, b.user_id, a.country
FROM TABLE_A a
INNER JOIN TABLE_B b ON (b.id = a.user)
INNER JOIN TABLE_C c ON (b.owner = c.id)
WHERE 1=1
AND a.platform = 'x'
AND a.country = 'y'
AND a.access_token is not NULL
ORDER BY c.api_key, worker_id, a.updated_at desc
I'm currently wrapping it using from SQLAlchemy import text and then simply executing
query_results = db.execute(query).fetchall()
list_dicts = [r._asdict() for r in query_results]
df = pd.DataFrame(list_dicts)
and it works, but I would really like to see if it's possible to have it in the other notation, like :
db.query(TABLE_A).filter().join()... etc
Yes, it's possible.
But the exact way to do it will depend on your SQLAlchmey version and how you've setup your SQLAlchemy project and models.
You may want to check out the SQLAlchemy ORM querying guide and the Expression Language Tutorial to see which one fits better your case.
Related
I have a SQL query with a subquery inside it. The subquery uses a column name from the outer expression.
The SQL query looks something like this,
SELECT u.id, u.name FROM user u WHERE (SELECT COUNT(*) FROM report r WHERE r.reportee = u.id) = 0;
I am not sure how to convert it into sqlalchemy-core.
The documentation demonstrates how to use columns in the outer query from the inner query. However, I want the vice-versa.
I would appreciate any kind of assistance.
I am having the following query:
select t2.col as tag, count(*)
from my_table t1 JOIN TABLE(SPLIT(tags,'/')) as t2
where t2.col != ''
group by tag
(TABLE(SPLIT(tags,'/')) - create a temp table by splitting the tags field.)
Query works just fine running it on the database directly, but having trouble to create the query with this join clause using SQLAlchemy.
How can i perform a join with a table that created on the fly? and uses functions that aren't defined in SQLAlchemy.
Thanks.
I try to join a second table (PageLikes) on a first Table (PageVisits) after selecting only distinct values on one column of the first table with the python ORM peewee.
In pure SQL I can do this:
SELECT DISTINCT(pagevisits.visitor_id), pagelikes.liked_item FROM pagevisits
INNER JOIN pagelikes on pagevisits.visitor_id = pagelikes.user_id
In peewee with python I have tried:
query = (Page.select(
fn.Distinct(Pagevisits.visitor_id),
PageLikes.liked_item)
.join(PageLIkes)
This gives me an error:
distinct() takes 1 positional argument but 2 were given
The only way I can and have used distinct with peewee is like this:
query = (Page.select(
Pagevisits.visitor_id,
PageLikes.liked_item)
.distinct()
which does not seem to work for my scenario.
So how can I select only distinct values in one table based on one column before I join another table with peewee?
I don't believe you should be encountering an error using fn.DISTINCT() in that way. I'm curious to see the full traceback. In my testing locally, I have no problems running something like:
query = (PageVisits
.select(fn.DISTINCT(PageVisits.visitor_id), PageLikes.liked_item)
.join(PageLikes))
Which produces SQL equivalent to what you're after. I'm using the latest peewee code btw.
As Papooch suggested, calling distinct on the Model seems to work:
distinct_visitors = (Pagevisits
.select(
Pagevisits.visitor_id.distinct().alias("visitor")
)
.where(Pagevisits.page_id == "Some specifc page")
.alias('distinct_visitors')
)
query = (Pagelikes
.select(fn.Count(Pagelikes.liked_item),
)
.join(distinct_visitors, on=(distinct_visitors.c.visitor = Pagelikes.user_id))
.group_by(Pagelikes.liked_item)
)
SELECT *
FROM Residents
WHERE apartment_id IN (SELECT ID
FROM Apartments
WHERE postcode = 2000)
I'm using sqlalchemy and am trying to execute the above query. I haven't been able to execute it as raw SQL using db.engine.execute(sql) since it complains that my relations doesn't exist... But I succesfully query my database using this format: session.Query(Residents).filter_by(???).
I cant not figure out how to build my wanted query with this format, though.
You can create subquery with subquery method
subquery = session.query(Apartments.id).filter(Apartments.postcode==2000).subquery()
query = session.query(Residents).filter(Residents.apartment_id.in_(subquery))
I just wanted to add, that if you are using this method to update your DB, make sure you add the synchronize_session='fetch' kwarg. So it will look something like:
subquery = session.query(Apartments.id).filter(Apartments.postcode==2000).subquery()
query = session.query(Residents).\
filter(Residents.apartment_id.in_(subquery)).\
update({"key": value}, synchronize_session='fetch')
Otherwise you will run into issues.
I'm trying to replicate the following query in SQLAlchemy + MySQL without luck...
delete paths from paths
join paths as p1 on p1.ancestor = <ID>
where paths.descendant = p1.descendant;
SQLAlchemy seems be to ignoring any joins that I put into the delete query. I also tried using a subquery instead of a join, but that's not allowed in MySQL (can't select from same table as you want to delete from). Any help would be much appreciated.
Update:
Yes, I'm trying to use the ORM layer. Here's the queries that I've tried:
p1 = aliased(Path, name="p1")
db.session.query(Path).join(
p1, p1.ancestor==<ID>
)
.filter(
Path.descendant==p1.Path.descendant
).delete()
And the subquery variant, but this doesn't work on MySQL, so it's of no use to me:
q = db.session.query(Path.descendant).filter(Path.ancestor==<ID>).subquery()
db.session.query(Path).filter(Path.descendant.in_(q)).delete(synchronize_session='fetch')
SQLAlchemy currently supports UPDATE..FROM across Postgresql, MySQL and others, but we haven't as of yet tried to support DELETE..JOIN.
However, it does appear to work (almost?), as far as generating the SQL string:
class Path(Base):
__tablename__ = "path"
id = Column(Integer, primary_key=True)
descendant = Column(Integer)
ancestor = Column(Integer)
j = join(Path, p1, p1.ancestor == 5)
d = delete(j).where(Path.descendant == p1.descendant)
print d
prints:
DELETE FROM path JOIN path AS p1 ON p1.ancestor = :ancestor_1
WHERE path.descendant = p1.descendant
However, my MySQL database isn't accepting this, by default it renders INNER JOIN, which fails, but if I modify the MySQL compiler to not do this, still fails :
s.execute(d)
(ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the
right syntax to use near 'JOIN path AS p1 ON p1.ancestor = 5 WHERE
path.descendant = p1.descendant' at line 1") 'DELETE FROM path JOIN
path AS p1 ON p1.ancestor = %s WHERE path.descendant = p1.descendant'
(5,)
looks like your SQL verbatim (oh, except for 'delete paths FROM paths'? is that right?) ?
In any case, if the built in compiler is not doing it, your options are to use session.execute("some sql") or to build a custom construct with the compiler extension.
You can use prefixes keyword argument:
j = join(table1, table2, table1.c.key==table2.c.key)
stmt = delete(j, prefixes=[table1_name])
session.execute(stmt)
prefixes keyword just adds one or more expressions following the statement keyword, i.e. SELECT, INSERT, UPDATE, or DELETE.
In this case, delete(j) statement generates expression: "DELETE FROM table1 INNER JOIN table2 ON table1.key=table2.key".
When we add prefixes argument expression becomes: "DELETE table1 FROM table1 INNER JOIN table2 ON table1.key=table2.key", that is correct MySQL query.