In this program I created a database called mydata and I created a table called mining and I wanted to read my name and last name and age from a list in and if it was not in the database, save it in the database . But I got into trouble and my program gives an error.
In fact, in writing SQL syntax, it would store the data in the database if it did not exist. I have a problem
mycode:
import mysql.connector
cnx = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'mydata'
)
c = cnx.cursor()
list_number = ['mobin','ghanbari',15]
for i in list_number:
c.execute('INSERT INTO mining VALUES ("%s","%s","%s") WHERE NOT EXISTS (SELECT * FROM mining)'%(i,i,i))
Please help me solve this problem
You need to add unique index for the fields you do not want duplicate
Here an example with 3 fields . You can with multiple fields
mysql> ALTER TABLE mining ADD UNIQUE INDEX(your_field1, your_field2,your_field3);
And use INSERT IGNORE instead of INSERT . This prevent query to insert same data two or more times in the table
for i in list_number:
c.execute('INSERT IGNORE INTO mining VALUES ("%s","%s","%s") )
Or execute a query:
SELECT * FROM mining WHERE
your_field1 = 'mobin' AND
your_field2 = 'ghanbari' AND
your_field3 = 15;
Before the query INSERT .
if the SELECT query return something do not execute the INSERT query. Otherwise do the INSERT query.
You should make the columns you don't want duplicate information in unique. This way, when you try putting duplicate information in them, SQL will send a failure message to you in Python.
For example, take this table:
username (unique)
password
myusername
mypassword
When you try adding [newusername,newpassword], it won't give any error.
username (unique)
password
myusername
mypassword
newusername
newpassword
If we insert [anotherusername,mypassword], it won't give an error because password isn't unique
username (unique)
password
myusername
mypassword
newusername
newpassword
anotherusername
mypassword
But when we try inserting [myusername,anotherpassword], SQL will return an error to Python saying that since the username column is unique, it wasn't able to insert duplicate data (myusername is duplicate).
username (unique)
password
myusername
mypassword
newusername
newpassword
anotherusername
mypassword
See this tutorial to learn how to make a column unique.
Access to mysql
$ mysql -u root -p
Creating Database and table as example on mysql from command line.
mysql> CREATE DATABASE sof;
Query OK, 1 row affected (0.00 sec)
mysql> use sof;
Database changed
mysql> CREATE TABLE mining (
-> id int NOT NULL AUTO_INCREMENT,
-> lastname varchar(255) NOT NULL,
-> firstname varchar(255),
-> age int,
-> PRIMARY KEY (id)
-> );
Query OK, 0 rows affected (0.01 sec)
mysql> show tables;
+---------------+
| Tables_in_sof |
+---------------+
| mining |
+---------------+
mysql> explain mining;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| lastname | varchar(255) | NO | | NULL | |
| firstname | varchar(255) | YES | MUL | NULL | |
| age | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
mysql> ALTER TABLE mining ADD UNIQUE INDEX(firstname, lastname, age);
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> SHOW INDEXES FROM mining;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| mining | 0 | PRIMARY | 1 | id | A | 2 | NULL | NULL | | BTREE | | |
| mining | 0 | firstname | 1 | firstname | A | 2 | NULL | NULL | YES | BTREE | | |
| mining | 0 | firstname | 2 | lastname | A | 2 | NULL | NULL | | BTREE | | |
| mining | 0 | firstname | 3 | age | A | 3 | NULL | NULL | YES | BTREE | | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
4 rows in set (0.00 sec)
Please note the field Non_unique
1 if the index can contain duplicates, 0 if it cannot.
Now the Python Script:
import mysql.connector
mydb = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'sof'
)
cursor = mydb.cursor()
listd =[
{'firstname':'mobin1','lastname':'ghanbari1','age':15},
{'firstname':'mobin2','lastname':'ghanbari2','age':16},
{'firstname':'mobin3','lastname':'ghanbari3','age':17},
{'firstname':'mobin4','lastname':'ghanbari4','age':18}
]
for row in listd:
sql = "INSERT INTO mining (firstname, lastname , age) VALUES (%s, %s, %s)"
val = (row['firstname'],row['lastname'], row['age'])
try:
cursor.execute(sql, val)
mydb.commit()
print(cursor.rowcount, "record inserted for "+row["firstname"])
except mysql.connector.Error as err:
print("mysql exception: {}".format(err))
print(cursor.rowcount, "record inserted for "+row["firstname"])
Output:
1 record inserted for mobin1
1 record inserted for mobin2
1 record inserted for mobin3
1 record inserted for mobin4
Checking on mysql if records was inserted
mysql> select * from mining;
+----+-----------+-----------+------+
| id | lastname | firstname | age |
+----+-----------+-----------+------+
| 1 | ghanbari1 | mobin1 | 15 |
| 2 | ghanbari2 | mobin2 | 16 |
| 3 | ghanbari3 | mobin3 | 17 |
| 4 | ghanbari4 | mobin4 | 18 |
+----+-----------+-----------+------+
4 rows in set (0.00 sec)
Adding some data in the list of Python script*
import mysql.connector
mydb = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'sof'
)
cursor = mydb.cursor()
listd =[
{'firstname':'mobin1','lastname':'ghanbari1','age':15},
{'firstname':'mobin2','lastname':'ghanbari2','age':16},
{'firstname':'mobin3','lastname':'ghanbari3','age':17},
{'firstname':'mobin4','lastname':'ghanbari4','age':18},
{'firstname':'mobin5','lastname':'ghanbari5','age':19},
{'firstname':'mobin6','lastname':'ghanbari6','age':20},
{'firstname':'mobin7','lastname':'ghanbari7','age':21}
]
for row in listd:
sql = "INSERT INTO mining (firstname, lastname , age) VALUES (%s, %s, %s)"
val = (row['firstname'],row['lastname'], row['age'])
try:
cursor.execute(sql, val)
mydb.commit()
print(cursor.rowcount, "record inserted for "+row["firstname"])
except mysql.connector.Error as err:
print("mysql exception: {}".format(err))
print(cursor.rowcount, "record inserted for "+row["firstname"])
Running the Python Script
Output:
mysql exception: 1062 (23000): Duplicate entry 'mobin1-ghanbari1-15' for key 'firstname'
-1 record inserted for mobin1
mysql exception: 1062 (23000): Duplicate entry 'mobin2-ghanbari2-16' for key 'firstname'
-1 record inserted for mobin2
mysql exception: 1062 (23000): Duplicate entry 'mobin3-ghanbari3-17' for key 'firstname'
-1 record inserted for mobin3
mysql exception: 1062 (23000): Duplicate entry 'mobin4-ghanbari4-18' for key 'firstname'
-1 record inserted for mobin4
1 record inserted for mobin5
1 record inserted for mobin6
1 record inserted for mobin7
Checking on mysql if we have duplicate entries
mysql> select * from mining;
+----+-----------+-----------+------+
| id | lastname | firstname | age |
+----+-----------+-----------+------+
| 1 | ghanbari1 | mobin1 | 15 |
| 2 | ghanbari2 | mobin2 | 16 |
| 3 | ghanbari3 | mobin3 | 17 |
| 4 | ghanbari4 | mobin4 | 18 |
| 13 | ghanbari5 | mobin5 | 19 |
| 14 | ghanbari6 | mobin6 | 20 |
| 15 | ghanbari7 | mobin7 | 21 |
+----+-----------+-----------+------+
7 rows in set (0.00 sec)
As you can see there are no duplicate entries in the table.
Related
thank you very much! I use 'renew' to replace 'update'. and it success! #MrTux
Thank you for help me.
mysql error:1064,but my type isn't wrong
python3+mysql+pymysql
mysql table：(you just need to see the 'update')
(if I insert into not include update , it will success.)
mysql> desc House;
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| num | varchar(30) | NO | UNI | NULL | |
| url | varchar(150) | YES | | NULL | |
| name | varchar(50) | YES | | NULL | |
| maintain | varchar(30) | YES | | NULL | |
| update | varchar(30) | YES | | NULL | |
+----------+--------------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
Code:
update = "a str"
sql2 = "INSERT INTO House (num,update) VALUES ('{0}' , '{1}' )".format(num,update)
print(sql2)
try:
cursor.execute(sql2)
print("sql2 success")
connect.rollback()
print("rollback success")
except Exception as e:
print("sql2 wrong：" + str(e))
Error:
INSERT INTO House (num,update) VALUES ('NJ2578781985216667648' , 'a str' )
sql2 wrong：(1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'update) VALUES ('NJ2578781985216667648' , 'a str' )' at line 1")
You get a syntax error because update is a keyword in MySQL. You need to use a different name or quote using backticks:
sql2 = "INSERT INTO House (num,`update`) VALUES ('{0}' , '{1}' )".format(num,update)
PS: please also be aware of possible SQL injections.
I have created a small excel file to try this process
+----------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+ | S no | int(11) | NO | PI | NULL | | | Order | int(11)
| YES | | NULL | | | Price | int(11) | YES |
| NULL | | | comments | var char(45) | YES | | NULL
| |
+----------+-------------+------+-----+---------+-------+ 4 rows in set (1.21 sec)
This is the code I am using in python for the same
import xlrd import pymysql book =
xlrd.open_workbook(r'C:\Users\hp\Desktop\excel-python-mysql.xlsx')
sheet = book.sheet_by_index(0)
# Connect to the database
connection = pymysql.connect(host='localhost',
user='root',
password='*****',
db='trial',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
cursor = connection.cursor()
query = """INSERT INTO demo(Sno,Order,Price,Comments)VALUES(%d,%d,%d,%s)"""
for r in range(1,sheet.nrows):
Sno = sheet.cell(r,0).value
Order = sheet.cell(r,1).value
Price = sheet.cell(r,2).value
Comments = sheet.cell(r,3).value
values = (Sno,Order,Price,Comments)
cursor.execute(query,values)
cursor.close()
connection.commit()
connection.close()
but I am facing error stating "%d format: a number is required, not str" I want to move the data to MySQL for further use it at Metabase.
I wrote a function to take a list of lists and insert into my database table, however, if there is a duplicate key, one of the columns is to update. What am I doing wrong?
def insert_or_update(self, data):
try:
self.cursor.executemany('''insert into landing_pages (profile_id, landing_page, keyword_count, unique_key)
values (%s, %s, %s, %s) on duplicate key update keyword_count =values(keyword_count)''', (data))
self.db.commit()
return self.cursor.lastrowid
except Exception as e:
self.db.rollback()
# Rollback in case there is any error
return e
finally:
self.db.close()
Update:
Sorry, forgot to mention the actual issue (been working for all day). It doesn't seem to be updating, instead, it just inserts the data again.
For the unique key, I'm actually creating a salted hash, this is from the profile_id and landing_page columns; the hash is passed into the unique_key column.
My table looks like this:
+---------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+---------------+------+-----+---------+----------------+
| id | mediumint(9) | NO | PRI | NULL | auto_increment |
| profile_id | int(11) | YES | | NULL | |
| landing_page | varchar(2083) | YES | | NULL | |
| keyword_count | int(11) | YES | | NULL | |
| unique_key | varchar(200) | YES | UNI | NULL | |
+---------------+---------------+------+-----+---------+----------------+
Assuming keyword_count is an int, could you change the following in your query:
update keyword_count =values(keyword_count)
to
update keyword_count = <value>
I think your problem in table is self explanatory
your key is id
values you are inserting are
profile_id, landing_page, keyword_count, unique_key
none of which clash with your key
On update that there was unique key:
http://sqlfiddle.com/#!9/881082/1
I am trying to add data to a mysql database. Can someone explain why the query seems to be treating temp as a column instead of the string value it is supposed to be?
pi#raspberrypi ~/Desktop/PiControl $ python writedb.py
rollback: (1054, "Unknown column 'temp' in 'field list'")
done
my table
mysql> select * from logs;
+----+--------+---------+------+
| id | sensor | reading | time |
+----+--------+---------+------+
| 1 | temp | NULL | NULL |
| 2 | NULL | 69.69 | NULL |
| 3 | NULL | 69.69 | NULL |
| 4 | NULL | 1234.56 | NULL |
| 5 | NULL | 1234.56 | NULL |
| 6 | NULL | 1234.56 | NULL |
| 7 | temp | 123 | NULL |
| 8 | temp | 20 | NULL |
+----+--------+---------+------+
8 rows in set (0.00 sec)
The script:
import MySQLdb
conn = MySQLdb.connect(host= "localhost",user="xx",passwd="yy",db="homelog")
x = conn.cursor()
try:
x.execute("INSERT INTO `logs` (`sensor`,`reading`) VALUES (`temp`,`4242`)")
#print x.execute("SELECT * FROM logs") #returns number of rows in table for some reason
conn.commit()
except Exception, e:
print 'rollback: ',e
conn.rollback()
conn.close()
print "done"
Try parameterizing your query avoiding using backticks for column values:
sensor = "temp"
reading = 4242
x.execute("INSERT INTO `logs` (`sensor`,`reading`) VALUES (%s, %s)", (sensor, reading))
print x.execute("SELECT * FROM logs") #returns number of rows in table for some reason
You need to fetch all the matching records from the cursor after executing:
sensor = "temp"
reading = 4242
x.execute("INSERT INTO `logs` (`sensor`,`reading`) VALUES (%s, %s)", (sensor, reading))
conn.commit()
x.execute("SELECT * FROM logs")
print(x.fetchall())
I have a table that has 4 attributes.
+--------------+--------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| url | varchar(255) | YES | | NULL | |
| mail | varchar(255) | YES | | NULL | |
| date_entered | timestamp | NO | | CURRENT_TIMESTAMP | |
| active | tinyint(1) | NO | | 1 | |
+--------------+--------------+------+-----+-------------------+----------------+
Now i want to insert only the data_entered and other attributes to get default values.
I'm doing it for the id field which I need to be exact to another id I insereted to different table.
this is the code:
tx.execute(\
"insert into company_career (date_entered) "
"values (%s)",
(time.time())
)
This is the error:
query = query % db.literal(args)
exceptions.TypeError: not all arguments converted during string formatting
How to fix it?
Try this:
tx.execute(\
"insert into company_career (date_entered) "
"values (FROM_UNIXTIME('%s'))",
(time.time())
)
when you are providing values into a insert query dont forget to enclode it in commas
"insert into company_career (date_entered) values ('%s')"
I did manage to insert with: insert into company(url,mail,date_entered) values("","","now");