Develop Reference

pyodbc - cannot delete from MSSQL tables

Trying to delete some of the table entries by using pyodbc in database results in nothing happening. I know for sure that database connection is working as intended, can select data. Perhaps any suggestions what could be the cause?
get_user_id = conn.cursor()
get_user_id.execute('''
SELECT b.UserId
FROM Bindery b
INNER JOIN ActiveUser au
ON au.Id = b.UserId
WHERE au.UserId = ?
''', user_to_kick)
id_list = [id[0] for id in get_user_id.fetchall()]
delete_user = conn.cursor()
#delete from bindery first
delete_user.execute('''
DELETE FROM Bindery
WHERE UserId in (?)
''', id_list)
conn.commit
#delete from active user list
delete_user.execute('''
DELETE FROM ActiveUser
WHERE UserId = ?
''', user_to_kick)
conn.commit
delete_user.close()
conn.close
This is a code block that should imo trigger the delete query, but nothing happens. Select query does indeed get the data.
UPDATE:
After some adjustments and passing list as a parameter fixed, the delete query now indeed works as intended.
get_user_id = conn.cursor()
get_user_id.execute('''
SELECT b.UserId
FROM Bindery b
INNER JOIN ActiveUser au
ON au.Id = b.UserId
WHERE au.UserId = ?
''', user_to_kick)
id_list = [id[0] for id in get_user_id.fetchall()]
placeholders = ", ".join(["?"] * len(id_list))
sql = 'DELETE FROM Bindery\
WHERE UserId in (%s)' % placeholders
delete_user = conn.cursor()
#delete from bindery first
delete_user.execute(sql, id_list)
conn.commit()
#delete from active user list
delete_user.execute('''
DELETE FROM ActiveUser
WHERE UserId = ?
''', user_to_kick)
conn.commit()
get_user_id.close()
delete_user.close()
conn.close()

how to use one query output into other query in python?

I am trying to use one query output into other. but not getting the correct result. Can you please help me how to do this?
Example:
query1 = "select distinct lower(tablename) as tablename from medaff.imedical_metadata where object_type = 'View'"
output of above query is :
tablename
vw_mdcl_insght
vw_fbms_interactions
I want to use above output in other query. Something like this-
query2 = "select * from medaff.imedical_business_metadata where objectname in ('vw_mdcl_insght', 'vw_fbms_interactions')"
How to do this part in python?
I am using below code to run the query:
conn = redshift_conn()
with conn.cursor() as cur:
query1 = "select distinct lower(tablename) as tablename from medaff.imedical_metadata where object_type = 'View'"
cur.execute(sql_query)
result = cur.fetchall()
print(result)
conn.commit()
query2 = "select * from medaff.imedical_business_metadata where objectname in ('vw_mdcl_insght', 'vw_fbms_interactions')"
cur.execute(sql_query)
result = cur.fetchall()
print(result)
conn.commit()

I think you can just use an in query:
select ibm.*
from medaff.imedical_business_metadata ibm
where ibm.objectname in (select lower(im.tablename) as tablename
from medaff.imedical_metadata im
where im.object_type = 'View'
);
It is better to let the database do the work.

I used the below code:
query = "select distinct lower(tablename) from medaff.imedical_metadata where object_type = 'View'"
cur.execute(query)
res = cur.fetchall()
print(res)
res = tuple([item[0] for item in res])
res = str(res)

what slows down parsing?

I have big XML files to parse (about 200k lines and 10MB). The structure is following:
<el1>
<el2>
<el3>
<el3-1>...</el3-1>
<el3-2>...</el3-2>
</el3>
<el4>
<el4-1>...</el4-1>
<el4-2>...</el4-2>
</el4>
<el5>
<el5-1>...</el4-1>
<el5-2>...</el5-2>
</el5>
</el2>
</el1>
Here is my code:
tree = ElementTree.parse(filename)
doc = tree.getroot()
cursor.execute(
'INSERT INTO first_table() VALUES()',
())
cursor.execute('SELECT id FROM first_table ORDER BY id DESC limit 1')
row = cursor.fetchone()
v_id1 = row[0]
for el1 in doc.findall('EL1'):
cursor.execute(
'INSERT INTO second_table() VALUES(v_id1)',
(v_id1))
cursor.execute(
'SELECT id FROM second_table ORDER BY id DESC limit 1')
row = cursor.fetchone()
v_id2 = row[0]
for el2 in el1.findall('EL2'):
cursor.execute(
'INSERT INTO third_table(v_id2) VALUES()',
(v_id2))
cursor.execute(
'SELECT id FROM third_table ORDER BY id DESC limit 1')
row = cursor.fetchone()
v_id3 = row[0]
for el3 in el2.findall('EL3'):
cursor.execute(
'INSERT INTO fourth_table(v_id3) VALUES()',
(v_id3))
cursor.execute(
'SELECT id FROM fourth_table ORDER BY id DESC limit 1')
row = cursor.fetchone()
v_id4 = row[0]
for el4 in el3.findall('EL4'):
cursor.execute(
'INSERT INTO fifth_table(v_id4) VALUES()',
(v_id4))
for el5 in el4.findall('EL5'):
cursor.execute(
'INSERT INTO sixth_table(v_id4) VALUES()',
(v_id4))
cursor.execute(
'SELECT id FROM sixth_table ORDER BY id DESC limit 1')
row = cursor.fetchone()
v_id5 = row[0]
...
conn.commit()
Basically I get values from attributes and send them into the database. When I need to process nested elements, I have to SELECT last inserted ID from the database and INSERT it as a foreign key into the next INSERT statement.
The whole process takes about 50s but apparently it's too long for the data I have. The SELECT statements for sure take some time, but I already selecting only 1 attribute on last row.
I don't know if it can be faster since I'm not good at programming so I ask you guys.

You have 4 nested for loops. That's why. It is O(n^4).

How create dynamic insert if not exists (no of columns and values vary) for postgresql

I use python 2.7 + Psycopg2 2.6 to insert data into a Postgresql 9.4 database, which is working fine on a very basic level. Have created some dynamic INSERT-queries, which take different sets of columns and values from a dictionary (input_cols):
sql_template = "insert into tbl ({}) values %s"
sql = sql_template.format(', '.join(input_cols.keys()))
params = (tuple(input_cols.values()),)
cur.execute(sql, params)
Correct SQL generated:
insert into tbl (col1, col2, ...) values ('val1', 'val2', ...)
Would now like to use dynamic SQL generation also for some INSERT if NOT EXIST queries, but as 'cur.execute(sql, params)' above outputs a value list enclosed by '()' I cannot get it to work:
sql_template = "insert into tbl ({}) select %s where not exists (select id
from tbl where id = %s)"
sql = sql_template.format(', '.join(input_cols.keys()))
params = (tuple(input_cols.values()), input_cols['col1'])
Incorrect SQL generated:
insert into tbl (col1, col2) select ('val1', 'val2')
where not exists (select col1 from tbl where id = 'val1')
How can I output ('val1', 'val2') without () so that I can use it in a SELECT xxx, xxx WHERE NOT EXISTS query?

Use from (values...
input_cols = {'col1':'val1','col2':'val2'}
sql_template = """
insert into tbl ({})
select *
from (values %s) s
where not exists (
select id
from tbl
where id = %s
)
"""
sql = sql_template.format(', '.join(input_cols.keys()))
params = (tuple(input_cols.values()), input_cols['col1'])
print cursor.mogrify(sql, params)
Output:
insert into tbl (col2, col1)
select *
from (values ('val2', 'val1')) s
where not exists (
select id
from tbl
where id = 'val1'
)

select multiple columns using SQLite3 in Python

I have a list that contains the name of columns I want to retrieve from a table in the database.
My question is how to make the cursor select columns specified in the list. Do I have to convert nameList to a string variable before include it in the select statement? Thanks
nameList = ['A','B','C','D',...]
with sqlite3.connect(db_fileName) as conn:
cursor = conn.cursor()
cursor.execute("""
select * from table
""")

As long as you can be sure your input is sanitized -- to avoid SQL injection attack -- you can do:
...
qry = "select {} from table;"
qry.format( ','.join(nameList) )
cursor.execute(qry)
If you're on a really old version of Python do instead:
...
qry = "select %s from table;"
qry % ','.join(nameList)
cursor.execute(qry)

nameList = ["'A(pct)'",'B','C','D',...]
with sqlite3.connect(db_fileName) as conn:
cursor = conn.cursor()
cursor.execute("""
select {} from table
""".format(", ".join(nameList)))