I have some problems with a SQL for Python that I hope you can help me with - I'm trying to retrieve some data from wordpress/woocommerce.
My code:
cursor.execute("
SELECT t1.ID, t1.post_date, t2.meta_value AS first_name, t3.meta_value AS last_name
FROM test_posts t1
LEFT JOIN test_postmeta t2
ON t1.ID = t2.post_id
WHERE t2.meta_key = '_billing_first_name' and t2.post_id = t1.ID
LEFT JOIN test_postmeta t3
ON t1.ID = t3.post_id
WHERE t3.meta_key = '_billing_last_name' and t3.post_id = t1.ID
GROUP BY t1.ID
ORDER BY t1.post_date DESC LIMIT 20")
I'm getting the following error:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'LEFT JOIN test_postmeta t3 ON t1.ID = t3.post_id WHERE t3.meta_key = '_billing' at line 1
What am I doing wrong?
Thanks in advance.
There should be only 1 WHERE clause before GROUP BY.
But since you use LEFT joins, setting a condition on the right table like t2.meta_key = '_billing_first_name' you get an INNER join instead because you reject unmatched rows.
So set all the conditions in the ON clauses:
cursor.execute("
SELECT t1.ID, t1.post_date, t2.meta_value AS first_name, t3.meta_value AS last_name
FROM test_posts t1
LEFT JOIN test_postmeta t2
ON t1.ID = t2.post_id AND t2.meta_key = '_billing_first_name'
LEFT JOIN test_postmeta t3
ON t1.ID = t3.post_id AND t3.meta_key = '_billing_last_name'
GROUP BY t1.ID
ORDER BY t1.post_date DESC LIMIT 20")
Although this query may be syntactically correct for MySql, it does not make sense to use GROUP BY since you do not do any aggregation.
Your SQL syntax is incorrect. Try this:
cursor.execute("
SELECT t1.ID, t1.post_date, t2.meta_value AS first_name, t3.meta_value AS last_name
FROM test_posts t1
LEFT JOIN test_postmeta t2 ON t1.ID = t2.post_id
LEFT JOIN test_postmeta t3 ON t1.ID = t3.post_id
WHERE t3.meta_key = '_billing_last_name' and t2.meta_key = '_billing_first_name'
GROUP BY t1.ID
ORDER BY t1.post_date DESC LIMIT 20")
It might be worth reading a little bit about SQL Joins and WHERE statements.
Related
I have this code:
query = """SELECT sp.customer_surname, sp.amount, cp.amount, sp.monthly, sp.date_ FROM set_payment7777 sp JOIN customers_payments7777 cp ON cp.customer_VAT = sp.customer_VAT WHERE sp.date_ = (SELECT MAX(date_) FROM set_payment7777 GROUP BY customer_VAT) GROUP BY sp.customer_VAT"""
mycursor.execute(query)
for row in mycursor:
#do something
but I get the error:
mysql.connector.errors.DataError: 1242 (21000): Subquery returns more
than 1 row
You have several customer_VAT then your subquery return more than a row .. for avoid this you could use a join on the subquery
query = """SELECT sp.customer_surname, sp.amount, cp.amount, sp.monthly, sp.date_
FROM set_payment7777 sp
INNER JOIN customers_payments7777 cp ON cp.customer_VAT = sp.customer_VAT
INNER JOIN (
SELECT MAX(date_) FROM set_payment7777
GROUP BY customer_VAT
) t on t.customer_VAT = sp.customer_VAT
GROUP BY sp.customer_VAT"""
anyway you have a main select without aggregation function then you should avoid an improper use of group by. In this case use DISTINCT if you need not repeated result
query = """SELECT DISTINCT sp.customer_surname, sp.amount, cp.amount, sp.monthly, sp.date_
FROM set_payment7777 sp
INNER JOIN customers_payments7777 cp ON cp.customer_VAT = sp.customer_VAT
INNER JOIN (
SELECT MAX(date_) FROM set_payment7777
GROUP BY customer_VAT
) t on t.customer_VAT = sp.customer_VAT"""
I am trying to retrieve information from a database using a Python tuple containing a set of ids (between 1000 and 10000 ids), but my query uses the IN statement and is subsequently very slow.
query = """ SELECT *
FROM table1
LEFT JOIN table2 ON table1.id = table2.id
LEFT JOIN ..
LEFT JOIN ...
WHERE table1.id IN {} """.format(my_tuple)
and then I query the database using PostgreSQL to charge the result in a Pandas dataframe:
with tempfile.TemporaryFile() as tmpfile:
copy_sql = "COPY ({query}) TO STDOUT WITH CSV {head}".format(
query=query, head="HEADER"
)
conn = db_engine.raw_connection()
cur = conn.cursor()
cur.copy_expert(copy_sql, tmpfile)
tmpfile.seek(0)
df = pd.read_csv(tmpfile, low_memory=False)
I know that IN is not very efficient with a high number of parameters, but I do not have any idea to optimise this part of the query. Any hint?
You could debug your query using explain statement. Probably you are trying to
sequently read big table while needing only a few rows. Is field table1.id indexed?
Or you could try to filter table1 first and then start joining
with t1 as (
select f1,f2, .... from table1 where id in {}
)
select *
from t1
left join ....
I am trying to write the following PosgreSQL query in SQLAlchemy:
SELECT DISTINCT user_id
FROM
(SELECT *, (amount * usd_rate) as usd_amount
FROM transactions AS t1
LEFT JOIN LATERAL (
SELECT rate as usd_rate
FROM fx_rates fx
WHERE (fx.ccy = t1.currency) AND (t1.created_date > fx.ts)
ORDER BY fx.ts DESC
LIMIT 1
) t2 On true) AS complete_table
WHERE type = 'CARD_PAYMENT' AND usd_amount > 10
So far, I have the lateral join by using subquery in the following way:
lateral_query = session.query(fx_rates.rate.label('usd_rate')).filter(fx_rates.ccy == transactions.currency,
transactions.created_date > fx_rates.ts).order_by(desc(fx_rates.ts)).limit(1).subquery('rates_lateral').lateral('rates')
task2_query = session.query(transactions).outerjoin(lateral_query,true()).filter(transactions.type == 'CARD_PAYMENT')
print(task2_query)
This produces:
SELECT transactions.currency AS transactions_currency, transactions.amount AS transactions_amount, transactions.state AS transactions_state, transactions.created_date AS transactions_created_date, transactions.merchant_category AS transactions_merchant_category, transactions.merchant_country AS transactions_merchant_country, transactions.entry_method AS transactions_entry_method, transactions.user_id AS transactions_user_id, transactions.type AS transactions_type, transactions.source AS transactions_source, transactions.id AS transactions_id
FROM transactions LEFT OUTER JOIN LATERAL (SELECT fx_rates.rate AS usd_rate
FROM fx_rates
WHERE fx_rates.ccy = transactions.currency AND transactions.created_date > fx_rates.ts ORDER BY fx_rates.ts DESC
LIMIT %(param_1)s) AS rates ON true
WHERE transactions.type = %(type_1)s
Which print the correct lateral query,but so far I don't know how to add the calculated field (amount*usd_rate), so I can apply the distinct and where statements.
Add the required entity in the Query, give it a label, and use the result as a subquery as you've done in SQL:
task2_query = session.query(
transactions,
(transactions.amount * lateral_query.c.usd_rate).label('usd_amount')).\
outerjoin(lateral_query, true()).\
subquery()
task3_query = session.query(task2_query.c.user_id).\
filter(task2_query.c.type == 'CARD_PAYMENT',
task2_query.c.usd_amount > 10).\
distinct()
On the other hand wrapping it in a subquery should be unnecessary, since you can use the calculated USD amount in a WHERE predicate in the inner query just as well:
task2_query = session.query(transactions.user_id).\
outerjoin(lateral_query, true()).\
filter(transactions.type == 'CARD_PAYMENT',
transactions.amount * lateral_query.c.usd_rate > 10).\
distinct()
How to do this thing?
I have two table
Table1:
-id
-table2_id_1
-table2_id_2
Table2:
-id
-table3_id
Table3:
-id
-table4_id
-table5_id
-table6_id
Table4, Table5 and Table6:
-id
-name
-date
Main table is Table1
db(db.Table1).select()
I need to join twice Table2(colums) in witch i need to join Table3(in each table2_id_1 and table2_id_2 field table3_id is equals), than join Table4,Table5,Table6
I don't know, if I really got, what you are trying to do, but if you just want to join the tables according to the id's, something like that should work:
SELECT *
FROM table1 a JOIN table2 b ON (a.table2_id_1 = b.id) JOIN
table2 c ON (a.table2_id_2 = c.id) JOIN
table3 d ON (b.table3_id = d.id) JOIN
table3 e ON (c.table3_id = e.id) JOIN
table4 f ON (d.table4_id = f.id) JOIN
table5 g ON (d.table5_id = g.id) JOIN
table6 h ON (d.table6_id = h.id) JOIN
table4 i ON (e.table4_id = i.id) JOIN
table5 j ON (e.table5_id = j.id) JOIN
table6 k ON (e.table6_id = k.id)
Even so how to solve this problem:
cursor.execute("""SELECT * FROM Users AS t1
INNER JOIN Users_has_Users AS t ON t.Users_id = t1.id
INNER JOIN Users AS t2 ON t.Users_id1 = t2.id
WHERE t1.email = %s AND t1.id != t2.id AND t2.id >= %s
ORDER BY t2.name {}
LIMIT 10""".format(order), (email, since_id, limit))
Error:
not all arguments converted during string formatting
You cannot use SQL parameters to interpolate anything other than data; you cannot use it for any SQL keywords such as ASC, nor the limit parameter. That is the point of SQL parameters; to avoid their values from being interpreted as SQL instead.
Use string formatting to interpolate your sort direction and query limit instead:
cursor.execute("""SELECT * FROM Users AS t1
INNER JOIN Users_has_Users AS t ON t.Users_id = t1.id
INNER JOIN Users AS t2 ON t.Users_id1 = t2.id
WHERE t1.email = %s AND t1.id != t2.id AND t2.id >= %s
ORDER BY t2.name {}
LIMIT {}""".format(order, limit), (email, since_id))
This does assume that you have full control over the contents of order and limit; never set it from user-supplied data as string formatting like this would open you up to a SQL injection attack otherwise.