Develop Reference

Same PostgreSQL and SQLAlchemy query returning different results

Or at least I think they are equivalent. The PostgreSQL query is:
SELECT
*
FROM community
JOIN thread ON community.id = thread.community_id
LEFT OUTER JOIN thread_vote ON thread.id = thread_vote.thread_id AND thread_vote.user_id=1
WHERE community.name = 'dogs'
The SQLAlchemy query I'm using:
q=Community.query.filter_by(name='dogs').join(Community.threads).outerjoin(Thread.votes.and_(ThreadVote.user_id == 1))
print(q)
> SELECT
community.created AS community_created, community.updated AS community_updated,
community.id AS community_id, community.user_id AS community_user_id,
community.name AS community_name, community.description AS community_description,
community.visibility AS community_visibility
FROM community
JOIN thread ON community.id = thread.community_id
LEFT OUTER JOIN thread_vote ON thread.id = thread_vote.thread_id AND thread_vote.user_id = %(user_id_1)s
WHERE community.name = %(name_1)s
The PostgreSQL query returns all threads in the community, and if user_id '1' voted on a thread, I get data associated with the vote, which is what I want. On the other hand, the SQLAlchemy query returns all threads in the community and all votes per thread regardless of user_id. My questions are:
Why do I observe different behaviors?
How can I fix the SQLAlchemy query to only get thread_vote data for one specific user_id?

How to select the distinct values of three sql tables and perform inner join with python sqlite3?

I'm trying to perform an inner join of distinct values of three tables from an SQLite DB. I tried multiple times and failed. Please guide me.
Below is a pseudo-code of what I'm trying to achieve
sql = '''
SELECT DISTINCT lesson_id, question_id FROM lesson_practice_questions as lpq
INNER JOIN
SELECT DISTINCT topic_id, lesson_id FROM chapter_lessons as cl
WHERE cl.topic_id==2
ON cl.lesson_id = lpq.lesson_id
INNER JOIN
SELECT DISTINCT question_id, subject_id, question_type_id, knowledge_type_ids complexity_level FROM questions as q
ON q.question_id = lpq.question_id;'''
cur.execute(sql)
Many Thanks to #eshirvana for taking the time to help out!.
Perfect Solution with ambiguous error resolution for future stackoverflow reference:
sql = '''SELECT
lpq.lesson_id, cl.lesson_id,
topic_id,
q.question_id, lpq.question_id,
subject_id,
question_type_id,
knowledge_type_ids,
complexity
FROM lesson_practice_questions as lpq
INNER JOIN chapter_lessons as cl on cl.topic_id = 2 and cl.lesson_id = lpq.lesson_id
INNER JOIN questions as q ON q.question_id = lpq.question_id;'''

here is the right sql syntax , however you need to provide sample data and desired output if this is not the right output :
SELECT
lpq.lesson_id,
question_id,
topic_id,
lesson_id,
question_id,
subject_id,
question_type_id,
knowledge_type_ids,
complexity_level
FROM lesson_practice_questions as lpq
INNER JOIN chapter_lessons as cl on cl.topic_id = 2 and cl.lesson_id = lpq.lesson_id
INNER JOIN questions as q ON q.question_id = lpq.question_id;

SQL query returns result multiple times

I'm pretty new to SQL and am trying to join some tables in SQL.
I'm using SQLite3 and Pandas and have the following table structure:
User
|
Measurement - Environment - meas_device - Device
| |
Data Unit_of_Measurement
Why do I get the result of the following SQL-query multiple times (4x)?
query = """
SELECT User.name, Measurement.id, Data.set_id, Data.subset_id, Data.data
FROM Measurement
JOIN Data ON Measurement.id = Data.measurement_id
JOIN User ON Measurement.user_id = user.id
JOIN Environment ON Measurement.Environment_id = Environment.id
JOIN meas_device ON Environment.meas_dev_ids = meas_device.id
JOIN Device ON meas_device.device_id = Device.id
JOIN Unit_of_Measurement ON meas_device.Unit_id = Unit_of_Measurement.id
WHERE User.name = 'nicola'
"""
pd.read_sql_query(query, conn)
My guess is that I did something wrong with the joining, but I can not see what.
I hoped to be able to save a JOIN statement somewhere that works for every possible query, that's why more tables are joined than necessary for this query.
Update
I think the problem lies within the Environment table. Whenever I join this table the results get multiplied. As the Environment is a collection of meas_devices, there are multiple entries with the same Environment id.
(I could save the Environment table with the corresponding meas_device_id's as lists, but then I see no possibility to link the Environment table with the meas_device table.)
id | meas_device_id
1 | 1
1 | 2
1 | 5
2 | 3
2 | 4
Up until now i created the tables with pandas DataFrame.to_sql() therefore the id is not marked as primary key or something like that. Could this be the reason for my problem
Update 2
I found the problem. I don't think that actually helps somebody in the future. But for the sake of completeness, here the explanation. It was not really a question of how to link the tables but I neglected a crucial link. Because the Environment has multiple indices with the same value it created "open ends" that resulted in a multiplication of the results. I needed to add a cross-check between Environment.subset_id and Data.subset_id. The following query works fine:
query = f""" SELECT {SELECT}
FROM Data
JOIN Measurement ON Data.measurement_id = Measurement.id
JOIN User ON Measurement.user_id = User.id
JOIN Environment ON Measurement.Environment_id = Environment.id
JOIN meas_device ON Environment.meas_dev_ids = meas_device.id
JOIN Device ON meas_device.Device_id = Device.id
JOIN Unit_of_Measurement ON meas_device.Unit_id = Unit_of_Measurement.id
WHERE {WHERE} AND Environment.subset_id = Data.subset_id
"""

If you need to filter on tables that produce additional rows in the result the when they are joined, don't join them and instead include them in a sub-query in the WHERE clause.
E.g.
SELECT User.name, Measurement.id, Data.set_id, Data.subset_id, Data.data
FROM
Measurement
JOIN Data ON Measurement.id = Data.measurement_id
JOIN User ON Measurement.user_id = user.id
WHERE
Measurement.Environment_id IN (
SELECT Environment.id
FROM
Environment
JOIN meas_device ON Environment.meas_dev_ids = meas_device.id
JOIN Device ON meas_device.device_id = Device.id
JOIN Unit_of_Measurement ON meas_device.Unit_id = Unit_of_Measurement.id
WHERE Device.name = 'xy'
)
In this subquery you can join many tables without generating additional records.
If this is not an option because you want to select entries from other tables as well, you can simply add a DISTINCT to you original query.
SELECT DISTINCT
User.name, Measurement.id, Data.set_id, Data.subset_id, Data.data
FROM
Measurement
JOIN Data ON Measurement.id = Data.measurement_id
JOIN User ON Measurement.user_id = user.id
JOIN Environment ON Measurement.Environment_id = Environment.id
JOIN meas_device ON Environment.meas_dev_ids = meas_device.id
JOIN Device ON meas_device.device_id = Device.id
JOIN Unit_of_Measurement ON meas_device.Unit_id = Unit_of_Measurement.id
WHERE
User.name = 'nicola'

SQLAlchemy Coalesce and Join

I'm having a lot of trouble converting my sql query to sqlalchemy. I haven't been able to find any resources doing what I am trying to do.
The query I am trying to convert is:
SELECT
COALESCE(d.manager_name, e.name) AS name,
COALESCE(d.department_name, e.department_name) AS department
FROM employee e
LEFT JOIN department d ON e.id = d.id
WHERE e.date = '2018-11-05'
In sqlalchemy I came up with:
query = self.session.query(
func.coalesce(Department.manager_name, Employee.name),
func.coalesce(Department.department_name, Employee.department_name)).join(Department,
Employee.id == Department.id,
).filter(
Employee.date == '2018-11-05',
)
But keep getting the error:
sqlalchemy.exc.InvalidRequestError: Can't join table/selectable 'Department' to itself.
WHY?! The statements are exact!

Since Department is the leftmost item in your query, joins take place against it. To control what is considered the first – or the "left" – entity in the join use Query.select_from():
query = self.session.query(
func.coalesce(Department.manager_name, Employee.name),
func.coalesce(Department.department_name, Employee.department_name)).\
select_from(Employee).\
outerjoin(Department, Employee.id == Department.id).\
filter(Employee.date == '2018-11-05')
This behaviour is also explained in the ORM tutorial under "Querying with Joins", and Query.join(): "Controlling what to Join From".
Your query construct was also using Query.join(), though the raw SQL had LEFT JOIN. In that case Query.outerjoin() or join(..., isouter=True) should be used.

format a string with Python from list elements

I have a list that contains two elements like this :
tr = ['table1', 'table2']
I would like to be able to generate a part of a query and get this :
table1 INNER JOIN table2 ON table1.id = table2.id
How can I do this please in Python ?
Any help would be appreciated.
EDIT :
Here is what I've tried to produce table1 INNER JOIN table2:
join_tables = ('%s LEFT JOIN %s'.format(' '.join('%s' for _ in range(element -1))) for element in tr)

"{0} INNER JOIN {1} ON {0}.id = {1}.id".format("table1", "table2")
Edit
If this was a legitimate example, you need to use cursor.execute("{0} INNER JOIN {1} ON {0}.id = {1}.id", ("table1", "table2")) for MySQL, or cursor.mogrify(...) for Postgres to properly escape the table names and prevent SQL injection.