Develop Reference

Insert list of data into SQL table but got an Error : mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement

sql = """INSERT INTO (Product_details) VALUES (%s, %s, %s, %s, %s, %s)"""
val = [name, Spec ,Ratings ,Delivery ,Discount ,Price ] # list of data into list
for values in val: #loping the variables in the list and adding it to database
engine.execute(sql, values)

It looks like you're using Python - Try ? instead of %s - sometimes the parameter marker is not what you would expect it to be so do check which one you need to use for the language you're embedding the SQL in

TypeError: (‘Params must be in a list, tuple, or Row’, ‘HY000’) Error on pyodbc

I am trying to fetch all the duplicates from tokens and load into tokn_Duplicates table. Getting Type error as below, if the sequence/format of the values passed to the insert query is causing this. Tried different options from the forum to pass the values as tuple/list, havent been successful
Code snippet below and error
with conn.cursor() as cur:
cur.execute("""
select case when tokn_type_cd IS NULL Then 'Address' else tokn_type_cd end as tokn_type_cd,case when tokn_val_tx is null then '0' else tokn_val_tx end tokn_val_tx,case when pg_duplicate_ct is null then 0 else pg_duplicate_ct end pg_duplicate_ct from ((SELECT 0 DUMMY ) DM LEFT OUTER JOIN (SELECT tokn_type_cd, tokn_val_tx, COUNT(*) AS pg_duplicate_ct
FROM devt.tokens
GROUP BY tokn_type_cd, tokn_val_tx
HAVING COUNT(*) > 1) on 1=1)
""")
rows2 = cur.fetchall()
cnxn = pyodbc.connect(server='connect to server')
crsr = cnxn.cursor()
if len(rows2):
crsr.executemany("INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)", [str(r) for r in rows2])
cnxn.commit()
print("... Successfully completed...")
Error received below
crsr.executemany(“INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)“, [str(r) for r in rows2])
TypeError: (‘Params must be in a list, tuple, or Row’, ‘HY000’)
Tried the below change from the forum:
crsr.executemany(“INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)“, [str(tokn_type_cd), str(tokn_val_tx) ,pg_duplicate_ct])
received the error below after the change
crsr.executemany(“INSERT INTO secret_tokn_duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)“, [str(tokn_type_cd), str(tokn_val_tx) ,pg_duplicate_ct])
NameError: name ‘tokn_type_cd’ is not defined
Any help would be appreciated.
Thanks

rows2 is the result of a .fetchall() which means that it is a list of pyodbc.Row objects. Such a list is suitable without modification for use as parameters for a subsequent .executemany() operation. In other words, don't mess with rows2; just pass it to the .executemany() for your INSERT:
if len(rows2):
crsr.executemany(
"INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)",
rows2
)
cnxn.commit()
Your initial attempt failed because [str(r) for r in rows2] returns a list of string objects (scalars) instead of a list of Row objects.

Try
crsr.executemany(
"INSERT INTO secret_tokn_duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)",
[(str(row.tokn_type_cd), str(row.tokn_val_tx), row.pg_duplicate_ct) for row in rows2])
or if that doesn't work:
crsr.executemany(
"INSERT INTO secret_tokn_duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)",
[(str(row[0]), str(row[1]), row[2]) for row in rows2])
The list [(str(row.tokn_type_cd), str(row.tokn_val_tx), row.pg_duplicate_ct) for row in rows2] creates a list of 3-tuples from the rows in rows2. I don't have access to your database, so I can't test that it works, but it's my best guess at will work given what's in your question.
Your first attempt
crsr.executemany("INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)", [str(r) for r in rows2])
didn't work because the list you passed to crsr.executemany contained strings, which were what you got if you attempted to convert each row read from the cursor to a string. The elements in the list passed to crsr.executemany must be lists, tuples or Rows: you need to pass it a list of lists, a list of tuples or a list of Rows.
Your second attempt
crsr.executemany("INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct) VALUES (?, ?, ?)", [str(tokn_type_cd), str(tokn_val_tx) ,pg_duplicate_ct])
didn't work because tokn_type_cd, tokn_val_tx and pg_duplicate_ct aren't variables that Python knows about. However, even if you had defined them, you would again end up passing a list of strings to crsr.executemany and would get the first error again.
However, having said all the above, most RDBMSs support INSERT INTO ... SELECT ... statements, which will copy the data directly, so why not just execute the following?
cur.execute("""
INSERT INTO duplicates (tokn_type_cd, tokn_val_tx, pg_duplicate_ct)
select case when tokn_type_cd IS NULL Then 'Address' else tokn_type_cd end as tokn_type_cd,case when tokn_val_tx is null then '0' else tokn_val_tx end tokn_val_tx,case when pg_duplicate_ct is null then 0 else pg_duplicate_ct end pg_duplicate_ct from ((SELECT 0 DUMMY ) DM LEFT OUTER JOIN (SELECT tokn_type_cd, tokn_val_tx, COUNT(*) AS pg_duplicate_ct
FROM devt.tokens
GROUP BY tokn_type_cd, tokn_val_tx
HAVING COUNT(*) > 1) on 1=1)
""")

How to format and build query strings in Python Sqlite?

What is the most used way to create a Sqlite query in Python?
query = 'insert into events (date, title, col3, col4, int5, int6)
values("%s", "%s", "%s", "%s", %s, %s)' % (date, title, col3, col4, int5, int6)
print query
c.execute(query)
Problem: it won't work for example if title contains a quote ".
query = 'insert into events (date, title, col3, col4, int5, int6)
values(?, ?, ?, ?, ?, ?)'
c.execute(query, (date, title, col3, col4, int5, int6))
Problem: in solution 1., we could display/print the query (to log it); here in solution 2. we can't log the query string anymore because the "replace" of each ? by a variable is done during the execute.
Another cleaner way to do it? Can we avoid to repeat ?, ?, ?, ..., ? and have one single values(?) and still have it replaced by all the parameters in the tuple?

You should always use parameter substitution of DB API, to avoid SQL injection, query logging is relatively trivial by subclassing sqlite3.Cursor:
import sqlite3
class MyConnection(sqlite3.Connection):
def cursor(self):
return super().cursor(MyCursor)
class MyCursor(sqlite3.Cursor):
def execute(self, sql, parameters=''):
print(f'statement: {sql!r}, parameters: {parameters!r}')
return super().execute(sql, parameters)
conn = sqlite3.connect(':memory:', timeout=60, factory=MyConnection)
conn.execute('create table if not exists "test" (id integer, value integer)')
conn.execute('insert into test values (?, ?)', (1, 0));
conn.commit()
yields:
statement: 'create table if not exists "test" (id integer, value integer)', parameters: ''
statement: 'insert into test values (?, ?)', parameters: (1, 0)

To avoid formatting problems and SQL injection attacks, you should always use parameters.
When you want to log the query, you can simply log the parameter list together with the query string.
(SQLite has a function to get the expanded query, but Python does not expose it.)
Each parameter markers corresponds to exactly one value. If writing many markers is too tedious for you, let the computer do it:
parms = (1, 2, 3)
markers = ",".join("?" * len(parms))

Python Dynamic Parameterized Query

I'm hoping to create a build a query dynamically that I can populate through paramerters. E.g. Something like:
INSERT INTO Table (Col1, Col2, Col3, ...) VALUES (%s, %s, %s, ...)
Then I want to populate it like shown in
How to put parameterized sql query into variable and then execute in Python? :
sql = "INSERT INTO table VALUES (%s, %s, %s)"
args= var1, var2, var3
cursor.execute(sql, args)
Building the query string above should be pretty easy, however I'm not sure how to build the list of args. Once I can do that, the rest should be easy.

Looks like my question was easier than I thought, it looks like arrays behave like lists in python. The following gave me what I needed:
results = []
results.append("a")
results.append("b")
results.append("c")
results.append("d")
print results
I can use the above to make my arg list.

I typically use something like this to achieve what you describe:
>>> def prepquery(qrystr, args):
... # Assumes Qry Str has a %s in it instead of column placeholders.
... return qrystr % ', '.join('?' * len(args)) # ['%s'] * len(args), depending on DB.
...
>>>
>>> prepquery('Insert Into Table Values (%s)', range(5))
'Insert Into Table Values (?, ?, ?, ?, ?)'

MySQL Dynamic Query Statement in Python

I am trying to accomplish something like the following:
cursor = db.cursor()
cursor.execute('INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)', (fieldlist, valuelist))
cursor.commit()
I have 2 lists, fieldlist and valuelist which each contain the same number of items. What is the best way to generate a dynamic MySQL query statement where the collumns are stored in fieldlist and the values are stored in valuelist?

cursor.execute('INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist), valuelist)
To make it clearer:
sql = 'INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist)
cursor.execute(sql, valuelist)

The cursor expects parameters to be passed as a single sequence, so you need to combine - in order - the field and value lists.
itertools.chain() does exactly that however it returns a generator and I'm not sure if cursor.execute() will accept that as it's param sequence. Try it. If it fails, wrap it with list()
import itertools
sql = 'INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)'
cursor.execute(sql, itertools.chain(fieldlist, valuelist))
EDIT:
This solution will not work. This would cause the field names to be escaped and wrapped with quotes which would cause an sql syntax error.
I'll leave this answer as it might serve as a useful example but look to #Trent's answer for the solution.