I use Python (2.7) and I work with MySQLdb directory to control my SQL database (MySQL workbench 6)
I have a script that updates an sql table every ten minutes. The script deletes everything on the table at first and then insert all the rows again (with the updated data).
This is a sketch of my code:
db=MySQLdb.Connect("localhost", "admin", "admin")
cursor = db.cursor()
cursor.execute("use my_scheme")
cursor.execute("delete from my_table")
for item in updated_item_list:
cursor.execute("insert into my_table values (%s, %s, %s)", (item[0], item[1], item[2]))
db.commit()
cursor.fetchall()
Now, for some reason, I got weird behavior sometimes: the script starts updating the table and some of the rows are actually being inserted but it stops in the middle with:
_mysql_exceptions.IntegrityError: (1062, "Duplicate entry for key 'PRIMARY'")
Every time, different key is being picked as duplicate. How can it be? All the entries have been deleted before. Any idea?
Thanks a lot!
You seem to be inserting constants into the database, not your actual values.Try this.
cursor.execute("INSERT INTO my_table values " +
"(val1, val2, val3) VALUES (%s, %s, %s, %s)",
(val1, val2, val3))
Related
I'm working in a Jupyter Notebook and using pymysql. I can read off that database, so the connection must be established, but I can't send any INSERT statements.
connection = pymysql.connect(endpoint, user, passwd, db)
insert = [('Popowice',363000),('Wroclaw',389991),('Biskupin',359000)]
sql = "INSERT INTO housing_wroclaw (`District`, `Price`) VALUES (%s, %s)"
cursor = connection.cursor()
cursor.executemany(sql,insert)
This piece of code with my credentials returns 3 - the number of insert tuples and no errors. But the database just doesn't have those records. I tried also looping through values using execute() rather than executemany(), but neither worked and the latter is apparently better.
Below is my working SELECT statement:
cursor = connection.cursor()
cursor.execute('SELECT * from housing_wroclaw')
rows = cursor.fetchall()
How can I INSERT? Why it doesn't work?
You must call connection.commit() after inserting data to make it persistent.
I am trying to commit data retrived from tkinter widgets into a SQL table (which has already been created and has the column names of leaderID, firstname,secondname,age,address,postcode,telephone,email). I keep receiving the error despite remaking the table, rewriting the SQL, resetting the database server etc. This code is contained within a function which is activated by a tkinter button. All of the inputted data is retrieved from each tkinter widget using the .get() syntax.
LeaderID=random.randint(1,10000)
print(LeaderID)
sqlcommand="INSERT INTO leaderinfo (leaderID, firstname,secondname,age,address,postcode,telephone,email) VALUES (%s, %s, %s, %s, %s, &s, &s, %s)"
LeaderInput= (LeaderID,FName.get(),SName.get(),Age.get(),Address.get(),Postcode.get(),TelephoneNum.get(),Email.get())
mycursor.execute(sqlcommand,LeaderInput)
mydb.commit()
print("Completed Transaction")
Produces the error
mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement
Further I was wondering how I could make the code look a little more readiable, as you can see from the code their is quite a long line of code contain a lot of placeholder '%s', could I move those to a different line?
Let me know if you would like me to supply additional information.
Thanks
You have &s instead of %s for some of your arguments. Fix it and you should be OK:
sqlcommand="INSERT INTO leaderinfo (leaderID, firstname,secondname,age,address,postcode,telephone,email) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)"
# Here ------------------------------------------------------------------------------------------------------------------------------^---^
I need to do a daily load from a source db into a new db.
The new db table columns are identical structure to the source table select statement. The new db table is in a separate db so the cursor object is unique per db connection and I can't just do a select into query.
So, for example, if I do a select on some table in some source db:
# Assuming we already connected to the db and have a cursor object...
sql_query = "SELECT val_bin, val_id, val_sel from table"
cursor.execute(sql_query)
I now have the objects I need in the cursor object from the select.
Then to insert, normally I would just grab each value and do an insert statement for each. For example:
for row in cursor.fetchall():
insert_query = "insert into new_table (val_bin, val_id, val_sel) VAULES (%s, %d, %s) % row[0], row[1], row[2]"
destination_cursor.execute(insert_query)
destination_db.commit()
However this seems tedious and slow to loop through everything.
Is there a way I can just insert the entire returned cursor object from the select statement into the new db table? The destination table schema matches exactly with what returned from the select.
If it's not possible, that's fine I am just trying to make this easier and more efficient.
You may want to use .executemany instead of .execute if the goal to insert all the results from a given cursor into a table. The following is an illustration:
cursor = connection.cursor()
destination_cursor = connection.cursor()
sql_query = "SELECT val_bin, val_id, val_sel from table"
cursor.execute(sql_query)
insert_query = "insert into new_table (val_bin, val_id, val_sel) VALUES (%s, %s, %s)"
destination_cursor.executemany(insert_query, cursor)
destination_db.commit()
I hope this proves useful.
I'm using pymssql to connect to a MS SQL DB on Azure and insert records from a CSV file. I've verified that the connection is successful, and that the list used for the executemany parameter contains all of the data in the correct format and with the correct number of values. However, when I run the script 0 rows are inserted - but no error is thrown.
I looked around and it seems like most others that have experienced something similar were missing the commit(), but that isn't the issue here.
Here is the code. Any help is greatly appreciated.
with open('file.csv') as csvfile:
data = csv.reader(csvfile)
next(data)
dicts = ({'col1': line[0], 'col1': line[1], 'col3': line[2], 'col4': int(line[3]), 'col5': int(line[4]), 'col6': float(line[5])} for line in data)
to_db = ((i['col1'], i['col2'], i['col3'], i['col4'], i['col5'], i['col6']) for i in dicts)
cursor.executemany(
'INSERT INTO myTable VALUES (%s, %s, %s, %d, %d, %f)',
to_db)
print str(cursor.rowcount) + " rows inserted"
conn.commit()
Edit: If I execute the query using cursor.execute() and include the values explicitly in the query then I can successfully insert rows into the database (see below for example).
cursor.execute("INSERT INTO myTable VALUES ('4/18/2016','test','test',0,0,0.0)")
But if I user the cursor.executemany(operation,parameters) syntax and pass a list of the values as the parameter then it results in an incorrect syntax error.
cursor.executemany("INSERT INTO myTable VALUES(%s,%s,%s,%d,%d,%f)",list_of_values)
I was just reading in the module reference that only %s and %d are supported. So I'm thinking that might be the issue. But how do I pass a float?
Using the float placeholder (%f) was in fact the issue. Only %s and %d are supported, but are purely placeholders and do not have any impact on formatting the way that they typically do in python, so really only %s is needed. The working code is as follows:
cursor.executemany("INSERT INTO myTable VALUES(%s,%s,%s,%s,%s,%s)",list_of_values)
I am having trouble finding the documentation to be able to add non-SQL variables to a SQL statement The official docs don't go into that at http://dev.mysql.com/doc/refman/5.0/en/user-variables.html.
How would I do the following in python and valid SQL?
id, provider_id, title = row[0], row[2], row[4]
cursor.execute("INSERT INTO vendor_id VALUES (%s,%s,%s);"%(id, provider_id, title))
You're on the right track but it should look like this.
id, provider_id, title = row[0], row[2], row[4]
cursor.execute("INSERT INTO vendor_id VALUES (%s, %s, %s)", id, provider_id, title)
The python SQL database API automatically disables and ask for a reformation of escaping and quoting of variables. String formatting operators are unnecessary and in this case do not do any formatting of variable quoting. You can format the final array of variables any way you like, just do not use the % operator like you use. If I am correct, the reason for this is that the variables are bound, and not formatted into the text, so that an injection cannot occur
cursor.execute("INSERT INTO vendor_id VALUES (%s, %s, %s)", (id, provider_id, title))
cursor.execute("INSERT INTO vendor_id VALUES (?, ?, ?)", (id, provider_id, title))
should work too.