Develop Reference

Using CTE in Python with Postgresql and psycopg2

I'm trying to create a query using CTE where I am creating 2 subtables and then the select statement. I believe the following syntax would work for full SQL, but it isn't working in this situation using psycopg2 in Python.
The idea is that I should be able to pull a query that shows the Name of all events (E.Event), the E.EDate, the E.ETemp and SmithTime. So it should have the full list of Events but the time column only shows times recorded for Smith (not in all Events).
query = ("""WITH cte AS (SELECT E.Event, O.Time AS "SmithTime"
FROM event E JOIN outcome O ON E.EventID = O.EventID
JOIN name N ON N.ID = O.ID
WHERE Name = 'Smith'),
WITH cte2 AS (SELECT E.Event, O.Time, E.EDate, E.ETemp
FROM event E JOIN outcome O ON E.EventID = O.EventID
JOIN name N ON N.ID = O.ID)
SELECT cte2.Event, cte2.EDate, cte2.ETemp, cte.SmithTime
FROM cte JOIN cte2 ON cte.Event = cte2.Event
ORDER BY 2 ASC""")
query = pd.read_sql(query, conn)
print(query)
This is just my latest iteration, I'm not sure what else to try. It is currently generating a DatabaseError:
DatabaseError: Execution failed on sql 'WITH cte AS (SELECT E.Event, O.Time AS "SmithTime"
FROM event E JOIN outcome O ON E.EventID = O.EventID
JOIN name N ON N.ID = O.ID
WHERE Name = 'Smith'),
WITH cte2 AS (SELECT E.Event, O.Time, E.EDate, E.ETemp
FROM event E JOIN outcome O ON E.EventID = O.EventID
JOIN name N ON N.ID = O.ID)
SELECT cte2.Event, cte2.EDate, cte2.ETemp, cte.SmithTime
FROM cte JOIN cte2 ON cte.Event = cte2.Event
ORDER BY 2 ASC': syntax error at or near "WITH"
LINE 6: WITH cte2 AS (SELECT E.Event, O.Time, E.EDate, E.ETemp

I have no idea whether or not your current query even be logically correct. But we can get around the SQL error by inlining the common table expressions:
SELECT cte2.Event, cte2.EDate, cte2.ETemp, cte.SmithTime
FROM (
SELECT E.Event, O.Time AS "SmithTime"
FROM event E
INNER JOIN outcome O ON E.EventID = O.EventID
INNER JOIN name N ON N.ID = O.ID
WHERE Name = 'Smith'
) cte
INNER JOIN (
SELECT E.Event, O.Time, E.EDate, E.ETemp
FROM event E
INNER JOIN outcome O ON E.EventID = O.EventID
INNER JOIN name N ON N.ID = O.ID
) cte2
ON cte.Event = cte2.Event
ORDER BY 2;

It's an SQL syntax error, nothing specific to psycopg2.
There's only one WITH in a CTE query. It should be WITH cte AS (...), cte2 AS (...) SELECT ..., not WITH cte AS (...), WITH cte2 AS (...) SELECT ....

SQLite3 - cross table SELECT queries

What I would like returned is all the seat_ids in the performance table that have a booking_id that matches all the booking_ids where night = 1 in the booking table - is an INNER JOIN the best way to do it?
Or is it more along the lines of """SELECT seat_id FROM performance WHERE booking_id=(SELECT * FROM booking WHERE night = ?""", (night_number))
With the above I get sqlite3.OperationalError: incomplete input error.
connection = sqlite3.connect('collyers_booking_system.db')
cursor = connection.cursor()
cursor.execute(booking_table)
cursor.execute(performance_table)
connection.commit()
booking_table = """CREATE TABLE IF NOT EXISTS
booking(
booking_id TEXT PRIMARY KEY,
customer_id INTEGER,
night INTEGER,
cost REAL,
FOREIGN KEY (customer_id) REFERENCES customer(customer_id)
)"""
performance_table = """CREATE TABLE IF NOT EXISTS
performance(
performance_id TEXT PRIMARY KEY,
seat_id TEXT,
booking_id INTEGER,
FOREIGN KEY (seat_id) REFERENCES seat(seat_id),
FOREIGN KEY (booking_id) REFERENCES booking(booking_id),
)"""
night_number = 1
cursor.execute("""SELECT seat_id FROM performance INNER JOIN booking ON night=?""", (night_number))
booked_seats = cursor.fetchall()
print(booked_seats)
With this I get ValueError: parameters are of unsupported type error.

First, if this is your actual code, there is a typo in the CREATE statement of the table performance.
You must remove the , at the end of:
FOREIGN KEY (booking_id) REFERENCES booking(booking_id),
Then, here:
cursor.execute("""SELECT seat_id FROM performance WHERE booking_id=(SELECT * FROM booking WHERE night = ?""", (night_number))
you missed a closing parenthesis for the sql statement and the subquery may return more than 1 rows, so instead of = you should use IN.
Also, the parameter night_number should passed as a tuple and not just a number, by adding a , inside the paraentheses:
cursor.execute("""SELECT seat_id FROM performance WHERE booking_id IN (SELECT * FROM booking WHERE night = ?)""", (night_number,))
For the join you need a proper ON clause, that links the tables and a , to create the tuple for night_number:
sql = """
SELECT p.seat_id
FROM performance p INNER JOIN booking b
ON b.booking_id = p. booking_id
WHERE b.night=?
"""
cursor.execute(sql, (night_number,))
Both ways, the operator IN and the join will work.
There is another option which sometimes performs better and this is EXISTS:
sql = """
SELECT p.seat_id
FROM performance p
WHERE EXISTS (
SELECT 1 FROM booking b
WHERE b.night=? AND b.booking_id = p.booking_id
)
"""
cursor.execute(sql, (night_number,))

You are comparing a list result with an integer.
this SELECT * FROM booking WHERE night = ? => returns an N rows
and you are wating for an Integer SELECT seat_id FROM performance WHERE booking_id=?.
You have to use something like this :
SELECT seat_id FROM performance WHERE booking_id in (SELECT * FROM booking WHERE night = ?""", (night_number))

SQLALCHEMY: Problem with subquery select, How do i a exact subquery with outerjoin?

I'd like to get a select by a sub-query but, I don't know how I will do that. I searched for every world of internet but not found what i want.
The select is:
SELECT order_status.*
FROM `order`
LEFT OUTER JOIN
(
SELECT *
FROM (
SELECT *
FROM order_status
ORDER BY created_date DESC LIMIT 1) s
WHERE status IN ('NEW', 'FINISH','SENDED','PROCESSING')
) AS order_status ON order.id = order_status.order_id;
my code:
subqy = self.session.query(OrderStatus).order_by(OrderStatus.created_date.desc()).limit(1).subquery()
query = self.session.query(Order).outerjoin(subqy)
return query.filter(and_(in_(conditions))).all()

I'd change the query a little bit to remove one subquery:
SELECT order_status.*
FROM `order`
LEFT OUTER JOIN
(
SELECT *
FROM order_status
ORDER BY created_date DESC
LIMIT 1
) AS order_status ON order.id = order_status.order_id
AND order_status.status IN ('NEW', 'FINISH','SENDED','PROCESSING')
Then the code becomes
subquery = self.session\
.query(OrderStatus)\
.order_by(OrderStatus.created_date.desc())\
.limit(1)\
.subquery()
query = self.session.query(Order)\
.outerjoin(subquery,
(subquery.c.order_id == Order.id)
& subquery_a.c.status.in_(('NEW', 'FINISH','SENDED','PROCESSING')))
return query.all()
I think the thing you missed is that for subqueries, you need to access the columns through table.c.column, instead of Table.column, as you're used to.

Python MYSQL multiple statement execution issue

Suppose I have this multiple-process Python-MySql query:
self.calculateLeadTime = ("""SET #lastDate = (SELECT sessionDate FROM stock
WHERE product = (%s)
ORDER BY stocksessionID DESC LIMIT 1);
SET #secondLastDate = (SELECT sessionDate FROM stock WHERE product = (%s)
ORDER BY stocksessionID DESC LIMIT 1, 1);
SET #leadTime = (SELECT DATEDIFF(#lastDate, #secondLastDate));
SET #lastStockSessionID = (SELECT stocksessionID
FROM stock WHERE product = (%s) ORDER BY stocksessionID DESC LIMIT 1);
UPDATE stock SET leadTime = (#leadTime)
WHERE stocksessionID = #lastStockSessionID;""", (self.productID.get(), self.productID.get(), self.productID.get()))
self.query = self.cur.execute(self.calculateLeadTime, multi=True)
for self.cur in self.results:
print('cursor:', self.cur)
if self.cur.with_rows:
print('result:', self.cur.fetchall())
self.cn.commit()
I am subject to the error:
stmt = operation.encode(self._connection.python_charset)
AttributeError: 'tuple' object has no attribute 'encode'
I have read the MySql Python documentation regarding multi=True when executing multiple SQL statements via Python. However, my implementation does not work. Any ideas?

Currently, you are passing a tuple as first argument in the cursor.execute call when it expects a single scalar query string in first argument and tuple/list of parameters in second argument:
self.calculateLeadTime = """SET #lastDate = (SELECT sessionDate
FROM stock
WHERE product = (%s)
ORDER BY stocksessionID DESC LIMIT 1);
SET #secondLastDate = (SELECT sessionDate
FROM stock WHERE product = (%s)
ORDER BY stocksessionID DESC LIMIT 1, 1);
SET #leadTime = (SELECT DATEDIFF(#lastDate, #secondLastDate));
SET #lastStockSessionID = (SELECT stocksessionID
FROM stock WHERE product = (%s)
ORDER BY stocksessionID DESC LIMIT 1);
UPDATE stock
SET leadTime = (#leadTime)
WHERE stocksessionID = #lastStockSessionID;
"""
self.query = self.cur.execute(self.calculateLeadTime,
params=(self.productID.get(), self.productID.get(), self.productID.get()),
multi=True)
By the way, MySQL supports JOIN in UPDATE statements for a single statement. Run below with two parameters:
UPDATE stock s
INNER JOIN
(SELECT product, MAX(stocksessionID) AS MaxID
FROM stock
WHERE product = %s
GROUP BY product
) agg_id
ON s.stocksessionID = agg_id.MaxID
INNER JOIN
(SELECT product, MAX(sessionDate) As MaxDate
FROM stock
GROUP BY product) max_dt
ON max_dt.product = s.product
INNER JOIN
(SELECT product, MAX(sessionDate) As SecMaxDate
FROM stock
WHERE sessionDate < ( SELECT MAX(sessionDate)
FROM stock
WHERE product = %s )
GROUP BY product
) sec_max_dt
ON max_dt.product = max_dt.product
SET leadTime = DATEDIFF(MaxDate, SecMaxDate);

I am unsure, if this will work or counts even as multiple queries, because Multi = true is for multiple Resultsets from SELECTS.
You must add teh data for the %s
self.query = self.cur.execute(self.calculateLeadTime,('product','product','product') , multi=True)

Add alias to SqlAlchemy join object

I am trying to select a column from a subquery (code below).
import sqlalchemy as sa
table1 = sa.schema.Table('t1',
db,
autoload=True,
autoload_with=db.engine)
si = Query().cte()
pids = sa.select([t1.c.pid], distinct=True).where(t1.c.date > date)
jt = pids.join(
si,
(si.c.pid == t1.c.pid) &
(si.c.startdate <= date) &
(si.c.enddate > date))
return sa.select([si.c.id]).select_from(jt)
However, I always get an error like:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) subquery in FROM must have an alias
LINE 6: FROM (SELECT DISTINCT t1.pid AS pid
^
HINT: For example, FROM (SELECT ...) [AS] foo.
How should I add an alias to my subquery?