Develop Reference

Insert into an unknown mysql column

Am trying to insert values into a mysql table with unknown database columns that are present in the db but can be found - and are passed - from inside a loop, however I am still stuck and I get the error message. I have written sample code that reproduces the error and tries to generate the mysql query dynamically. Is there a simpler way to do this with mysql? Why is my code not running? The final query seems correct
Error
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s,%s,%s,%s)' at line 1
The far that I could imagine
from Learn.callmysql import mycursor, db
datte = {}
datte["schoolfee"] = "amount"
datte["student"] = "name"
testlist = ["parent", "bothnames"] #Notte this will be generated dynamically so I have no idea the variables in it, this is a sample
thelist = []
secondlist = ""
for value in testlist:
datte[value] = "Bch" #here "Bch will be a real unknown value"
print("Final dictionary after adding degree is like: " + str(datte))
for value in datte:
thelist.append(value)
breakdown = "("
count = 0
total_count_should_be = len(thelist)
for value in thelist:
if count == total_count_should_be -1:
breakdown = breakdown + value + ")"
else:
breakdown = breakdown + value+","
count = count + 1
first_part_of_query = breakdown
print("First part of the query will be like: " + first_part_of_query)
for index in range(len(thelist)):
if index == 0:
secondlist = secondlist + "(%s,"
elif index == len(thelist)-1:
secondlist = secondlist + "%s)"
else:
secondlist = secondlist + "%s,"
second_part_of_the_query = secondlist
print("Second part of the query will be like: " + second_part_of_the_query)
#Try to join the queries
query = "INSERT INTO testtale " + first_part_of_query + " VALUES " + second_part_of_the_query
print("Query looks like: " + query)
val = datte
mycursor.execute(query, val)
db.commit()
CODE PRINTS

As #Rob Streeting suggested,
I have converted my dictionary into an ordered list then to a tuple like so
print("Dictionary is: " + str(datte))
list = []
for value in datte.values():
list.append(value)
listintotupple = tuple(list)
print(listintotupple)
Then passed it to the query:
#Try to join the queries
query = "INSERT INTO testtale " + first_part_of_query + " VALUES " + second_part_of_the_query
print("Query looks like: " + query)
val = listintotupple
mycursor.execute(query, val)
db.commit()

Trying to update or insert to SQL Server using pyodbc by iterating through Panda data frame

I am trying to update if record exist and insert if no record is found. using below code
for index, row in df.iterrows():
cols = "],[".join([str(i) for i in df.columns.tolist()])
cols = "([" + cols + "])"
ucols = "] = ?,[".join([str(i) for i in df.columns.tolist()])
ucols = "[" + ucols + "] = ?"
c.execute("SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;")
c.execute("BEGIN TRANSACTION;")
c.execute("UPDATE " + tblname + " SET" + ucols + " WHERE [TESTNUMBER]=" + str(row['TESTNUMBER']) + " AND [ROWNUM] =" + str(row['ROWNUM']) + ";", tuple(row))
sqlr = "IF ##ROWCOUNT = 0 " \
"BEGIN " \
"INSERT INTO " + tblname + cols +" VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?); " \
"END " \
"COMMIT TRANSACTION;"
c.execute(sqlr, tuple(row))
getting below error message
{ProgrammingError}('25000', u'[25000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 2, current count = 1. (266) (SQLExecDirectW)')
no sure what i am doing wrong. appreciate your help

Dynamically passing the column name as well the values in python mysql query

This is the following code
pythonlist = ['Name','Mno']
datalist = ["qwerty",'234']
sql = "SELECT " + ",".join(pythonlist) + " FROM data WHERE name = '"+ "','".join(datalist) + "' INTO OUTFILE filename"
print(sql)
OUTPUT:
SELECT Name,Mno FROM data WHERE Name= 'qwerty','234'
DESIRED OUTPUT:
SELECT Name,Mno FROM data WHERE Name = 'qwerty' and Mno = 234
Do note the removal of quotations marks in 'mno'.
The reason I am doing this is due because the column names, as well as values corresponding it to, will change frequently

Code :
queryparams = {'Name': 'qwerty', 'Mno': '234'}
and_clause = []
[and_clause.append(' %s = %s ') for k,v in queryparams.items()]
and_clause_str = ' and '.join(and_clause)
sql = 'SELECT %s FROM data WHERE ' + and_clause_str
params = [','.join(queryparams.keys())]
for k,v in queryparams.items():
params.append(str(k))
params.append(str(v))
print(sql)
print(params)
cursor.execute(sql, params=tuple(params))
This works if you add 10/20 more items to dictionary .
Aswell as prevents SQL-injection : Using params to pass values instead of string-concatenation .

Try this:
data = {'Name': 'qwerty' , 'Mno' : '234'}
sql = "SELECT " + ", ".join(data.keys()) + " FROM data WHERE " + str(list(data.keys())[0]) + " = '" + \
str(data[list(data.keys())[0]]) + "' and " +\
str(list(data.keys())[1]) + " = " + str(data[list(data.keys())[1]])
print(sql)

]Incorrect syntax near '_283846_2019'. (102) (SQLExecDirectW)")

I have to connect the sql database to python so that I can add new user data via python.
I have tried the int conversion which puts me in further trouble of null types dataset.
i have tried the bracket placement. It doesn't work.
import os
import datetime
import pyodbc
import sqlite3
file_open = open("filenames.txt","r")
path = 'C:\\Users\\Timble\\Desktop\\Face_recognition\\user-id_filenames\\'
flag_loc = 1
flag_proc = 0
flag_vis = 0
file_read_lines = file_open.readlines()
for line in file_read_lines:
for character in line:
if character == "_":
details = line.split("_")
now = datetime.datetime.now()
name = line
print("name:", name) #col-3
print("type of name:", type(name))
user_id = int(details[1])
print("user_id:", details[1]) #col-2
print("type of user_id:", type(user_id))
date = details[2]
print("date on which photo is taken:", details[2]) #col-4
print("type of data:",type(details[2]))
now = now.strftime("%Y-%m-%d %H:%M:%S")
print("Current date and time: ", now) #col-6
print("type of current date:", type(now))
path2 = path + details[1]
if os.path.exists(path2):
print(path2)
else:
os.makedirs(path2)
#break
date = str(date)
print("type of date", type(date))
user_id = str(user_id)
print("type of user_id", type(user_id))
name = str(name)
print("type of name",type(name))
now = str(now)
print("type of now", type(now))
flag_loc = str(flag_loc)
print("type loc flag", type(flag_loc))
flag_proc = str(flag_proc)
print("type proc flag", type(flag_proc))
flag_vis = str(flag_vis)
print("type vis flag", type(flag_vis))
conn = pyodbc.connect(
"DRIVER={SQl Server};"
"server=DESKTOP-3ORBD3I\MSSQL;"
"database=TimbleSecuritySystem;"
"uid=sa;"
"pwd=P#ssword")
cur = conn.cursor()
sqlInsertUser = "Insert Into retraining (date, user_id, image_name,location_flagged, processing_flagged, insert_date, visible)Values( "+ date + " , " + user_id + " , " + name + " , " + flag_loc + " , " + flag_proc + " , " + now + " , " + flag_vis + " )"
print(sqlInsertUser)
cur.execute(sqlInsertUser)
conn.commit()
break
file_open.close()
The actual results tell me that print(sqlInsertUser) prints all the right values.
I am expecting the execute command to work and sql data added there.

This line is the problem:
sqlInsertUser = "Insert Into retraining (date, user_id, image_name,location_flagged, processing_flagged, insert_date, visible)Values( "+ date + " , " + user_id + " , " + name + " , " + flag_loc + " , " + flag_proc + " , " + now + " , " + flag_vis + " )"
For example if name contains some invalid characters e.g. "[" or "]", then the execute call fails because the name string is not properly enclosed. (It should be enclosed in a pair of quote)
You can use the parameter substitution support in pyodbc e.g.
sqlInsertUser = "Insert Into retraining (date, user_id,
image_name, location_flagged, processing_flagged, insert_date,
visible) Values (?,?,?,?,?,?,?)"
then run
cur.execute(sqlInsertUser, date, user_id, name, flag_loc, flag_proc, now, flag_vis)
(My sample code above is untested. You might need to fix some syntax errors)
For more details about the syntax see https://www.python.org/dev/peps/pep-0249/#paramstyle or https://github.com/mkleehammer/pyodbc/wiki/Cursor

How to introduce a switch breaker to perform insert every 50 lines

My current code basically does a bulk insert after iterating around all of the rows in my excel file. I want to introduce a switch breaker that will perform the insert every 50 lines.
db = Database(settings)
elt_insert_line = "INSERT INTO elt_data VALUES"
for row in r:
elt_insert_line = elt_insert_line + "(" + row[2] + ", " + row[3] + "),"
db.execute(elt_insert_line.rstrip(",")).commit().cleanup()

Using modulo operator and IF conditional
not familiar with python but i think you need something like this
db = Database(settings)
elt_insert_line = "INSERT INTO elt_data VALUES"
for row in r:
elt_insert_line = elt_insert_line + "(" + row[2] + ", " + row[3] + "),"
if r % 50 = 0 then
(
db.execute(elt_insert_line.rstrip(",")).commit().cleanup()
elt_insert_line = "INSERT INTO elt_data VALUES"
)
--one aditional at the end of the for
db.execute(elt_insert_line.rstrip(",")).commit().cleanup()