Related
In this program I created a database called mydata and I created a table called mining and I wanted to read my name and last name and age from a list in and if it was not in the database, save it in the database . But I got into trouble and my program gives an error.
In fact, in writing SQL syntax, it would store the data in the database if it did not exist. I have a problem
mycode:
import mysql.connector
cnx = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'mydata'
)
c = cnx.cursor()
list_number = ['mobin','ghanbari',15]
for i in list_number:
c.execute('INSERT INTO mining VALUES ("%s","%s","%s") WHERE NOT EXISTS (SELECT * FROM mining)'%(i,i,i))
Please help me solve this problem
You need to add unique index for the fields you do not want duplicate
Here an example with 3 fields . You can with multiple fields
mysql> ALTER TABLE mining ADD UNIQUE INDEX(your_field1, your_field2,your_field3);
And use INSERT IGNORE instead of INSERT . This prevent query to insert same data two or more times in the table
for i in list_number:
c.execute('INSERT IGNORE INTO mining VALUES ("%s","%s","%s") )
Or execute a query:
SELECT * FROM mining WHERE
your_field1 = 'mobin' AND
your_field2 = 'ghanbari' AND
your_field3 = 15;
Before the query INSERT .
if the SELECT query return something do not execute the INSERT query. Otherwise do the INSERT query.
You should make the columns you don't want duplicate information in unique. This way, when you try putting duplicate information in them, SQL will send a failure message to you in Python.
For example, take this table:
username (unique)
password
myusername
mypassword
When you try adding [newusername,newpassword], it won't give any error.
username (unique)
password
myusername
mypassword
newusername
newpassword
If we insert [anotherusername,mypassword], it won't give an error because password isn't unique
username (unique)
password
myusername
mypassword
newusername
newpassword
anotherusername
mypassword
But when we try inserting [myusername,anotherpassword], SQL will return an error to Python saying that since the username column is unique, it wasn't able to insert duplicate data (myusername is duplicate).
username (unique)
password
myusername
mypassword
newusername
newpassword
anotherusername
mypassword
See this tutorial to learn how to make a column unique.
Access to mysql
$ mysql -u root -p
Creating Database and table as example on mysql from command line.
mysql> CREATE DATABASE sof;
Query OK, 1 row affected (0.00 sec)
mysql> use sof;
Database changed
mysql> CREATE TABLE mining (
-> id int NOT NULL AUTO_INCREMENT,
-> lastname varchar(255) NOT NULL,
-> firstname varchar(255),
-> age int,
-> PRIMARY KEY (id)
-> );
Query OK, 0 rows affected (0.01 sec)
mysql> show tables;
+---------------+
| Tables_in_sof |
+---------------+
| mining |
+---------------+
mysql> explain mining;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| lastname | varchar(255) | NO | | NULL | |
| firstname | varchar(255) | YES | MUL | NULL | |
| age | int(11) | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
mysql> ALTER TABLE mining ADD UNIQUE INDEX(firstname, lastname, age);
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> SHOW INDEXES FROM mining;
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| mining | 0 | PRIMARY | 1 | id | A | 2 | NULL | NULL | | BTREE | | |
| mining | 0 | firstname | 1 | firstname | A | 2 | NULL | NULL | YES | BTREE | | |
| mining | 0 | firstname | 2 | lastname | A | 2 | NULL | NULL | | BTREE | | |
| mining | 0 | firstname | 3 | age | A | 3 | NULL | NULL | YES | BTREE | | |
+--------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
4 rows in set (0.00 sec)
Please note the field Non_unique
1 if the index can contain duplicates, 0 if it cannot.
Now the Python Script:
import mysql.connector
mydb = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'sof'
)
cursor = mydb.cursor()
listd =[
{'firstname':'mobin1','lastname':'ghanbari1','age':15},
{'firstname':'mobin2','lastname':'ghanbari2','age':16},
{'firstname':'mobin3','lastname':'ghanbari3','age':17},
{'firstname':'mobin4','lastname':'ghanbari4','age':18}
]
for row in listd:
sql = "INSERT INTO mining (firstname, lastname , age) VALUES (%s, %s, %s)"
val = (row['firstname'],row['lastname'], row['age'])
try:
cursor.execute(sql, val)
mydb.commit()
print(cursor.rowcount, "record inserted for "+row["firstname"])
except mysql.connector.Error as err:
print("mysql exception: {}".format(err))
print(cursor.rowcount, "record inserted for "+row["firstname"])
Output:
1 record inserted for mobin1
1 record inserted for mobin2
1 record inserted for mobin3
1 record inserted for mobin4
Checking on mysql if records was inserted
mysql> select * from mining;
+----+-----------+-----------+------+
| id | lastname | firstname | age |
+----+-----------+-----------+------+
| 1 | ghanbari1 | mobin1 | 15 |
| 2 | ghanbari2 | mobin2 | 16 |
| 3 | ghanbari3 | mobin3 | 17 |
| 4 | ghanbari4 | mobin4 | 18 |
+----+-----------+-----------+------+
4 rows in set (0.00 sec)
Adding some data in the list of Python script*
import mysql.connector
mydb = mysql.connector.connect(
host='localhost',
user = 'root',
password = '',
database = 'sof'
)
cursor = mydb.cursor()
listd =[
{'firstname':'mobin1','lastname':'ghanbari1','age':15},
{'firstname':'mobin2','lastname':'ghanbari2','age':16},
{'firstname':'mobin3','lastname':'ghanbari3','age':17},
{'firstname':'mobin4','lastname':'ghanbari4','age':18},
{'firstname':'mobin5','lastname':'ghanbari5','age':19},
{'firstname':'mobin6','lastname':'ghanbari6','age':20},
{'firstname':'mobin7','lastname':'ghanbari7','age':21}
]
for row in listd:
sql = "INSERT INTO mining (firstname, lastname , age) VALUES (%s, %s, %s)"
val = (row['firstname'],row['lastname'], row['age'])
try:
cursor.execute(sql, val)
mydb.commit()
print(cursor.rowcount, "record inserted for "+row["firstname"])
except mysql.connector.Error as err:
print("mysql exception: {}".format(err))
print(cursor.rowcount, "record inserted for "+row["firstname"])
Running the Python Script
Output:
mysql exception: 1062 (23000): Duplicate entry 'mobin1-ghanbari1-15' for key 'firstname'
-1 record inserted for mobin1
mysql exception: 1062 (23000): Duplicate entry 'mobin2-ghanbari2-16' for key 'firstname'
-1 record inserted for mobin2
mysql exception: 1062 (23000): Duplicate entry 'mobin3-ghanbari3-17' for key 'firstname'
-1 record inserted for mobin3
mysql exception: 1062 (23000): Duplicate entry 'mobin4-ghanbari4-18' for key 'firstname'
-1 record inserted for mobin4
1 record inserted for mobin5
1 record inserted for mobin6
1 record inserted for mobin7
Checking on mysql if we have duplicate entries
mysql> select * from mining;
+----+-----------+-----------+------+
| id | lastname | firstname | age |
+----+-----------+-----------+------+
| 1 | ghanbari1 | mobin1 | 15 |
| 2 | ghanbari2 | mobin2 | 16 |
| 3 | ghanbari3 | mobin3 | 17 |
| 4 | ghanbari4 | mobin4 | 18 |
| 13 | ghanbari5 | mobin5 | 19 |
| 14 | ghanbari6 | mobin6 | 20 |
| 15 | ghanbari7 | mobin7 | 21 |
+----+-----------+-----------+------+
7 rows in set (0.00 sec)
As you can see there are no duplicate entries in the table.
I have 2 tables that look like this:
Table 1
| ID | Tel. | Name |
|:--:|:----:|:-------:|
| 1 | 1234 | Denis |
| 2 | 4567 | Michael |
| 3 | 3425 | Peter |
| 4 | 3242 | Mary |
Table 2
| ID | Contact Date |
|:--:|:------------:|
| 1 | 2014-05-01 |
| 2 | 2003-01-05 |
| 3 | 2020-01-10 |
| 4 | NULL |
Now I want to Compare the First Table with the second table with the ID column to look if contact 1 is already in the list where people were contacted. After this, I want to write the Contact Date into the first table to see the last contact date in the main table.
How would I do this?
Thanks for any answers!!
Here's a solution that might get your interest using MySQL; Then if you want to implement it into Python will be easy.
First, Let's create our T1 Table AS CODED
create table T1(
id integer primary key,
tel integer,
name varchar(100)
);
create table T2(
id integer primary key,
contactDate date
);
Second Table T2
create table T2(
id integer primary key,
contactDate date
);
Insertion Of T1 AND T2:
-- Table 'T1' Insertion
insert into T1(id, tel, name) VALUES
(1, 1234, "Denis"),
(2, 4567, "Michael"),
(3, 3425,"Peter"),
(4, 3242, "Mary");
-- Table 'T2' Insertion
insert into T2(id, contactDate) VALUES
(1, 20140105),
(2, 20030105),
(3, 20201001),
(4, Null);
Then, Create T3 With the Select Statement for both tables using INNER JOIN For Joins Results
CREATE TABLE T3 AS
SELECT T1.id, T1.name, T1.tel, T2.contactDate
FROM T1
INNER JOIN T2 ON T1.id=T2.id;
Then, SELECT to check the results
select * from T3;
OUTPUT
| id | name | tel | contactDate |
|:--:|:-------:|------|-------------|
| 1 | Denis | 1234 | 2014-01-05 |
| 2 | Michael | 4567 | 2003-01-05 |
| 3 | Peter | 3425 | 2020-10-01 |
| 4 | Mary | 3242 | NULL |
I Hope This Helps. I was really trying to merge the T3---> contactDate with T1 for like 3 hours. But it was process heavy. I will attach Links that could help you more.
Reference
INNER JOIN
SQL ALTER TABLE Statement
SQL Server INSERT Multiple Rows
INSERT INTO SELECT statement overview and examples
SQL INSERT INTO SELECT Statement
I have a table(from log file) with emails and three other columns that contains states of that user's interaction with a system, an email(user) may have 100 or 1000 entries, each entries contain those three combinations of values, that might repeat on and on for same email and others.
something look like this:
+---------+---------+---------+-----+
| email | val1 | val2 | val3 |
+---------+---------+---------+-----+
|jal#h | cast | core | cam |
|hal#b |little ja| qar | ja sa |
|bam#t | cast | core | cam |
|jal#h |little ja| qar | jaja |
+---------+---------+---------+-----+
and so, the emails repeat, all values repeat, and there are 40+ possible values for each columns, all strings. so i want to sort distinct email email and then put all possible value as column name, and under it a count for how many this value repeated for a particular email, like so:
+-------+--------+--------+------+----------+-----+--------+-------+
| email | cast | core | cam | little ja| qar | ja sa | blabla |
+-------+--------+--------+------+----------+-----+--------+--------|
|jal#h | 55 | 2 | 44 | 244 | 1 | 200 | 12 |
|hal#b | 900 | 513 | 101 | 146 | 2 | 733 | 833 |
|bam#t | 1231 | 33 | 433 | 411 | 933 | 833 | 53 |
+-------+--------+--------+------+----------+-----+--------+---------
I have tried mysql but i managed to count a certain value total occurances for each email, but not counting all possible values in each columns:
SELECT
distinct email,
count(val1) as "cast"
FROM table1
where val1 = 'cast'
group by email
This query clearly doesn't do it, as it output only on value 'cast' from the first column 'val1', What i'm looking for is all distinct values in first, second, and third columns be turned to columns heads and the values in rows will be the total for that value, for a certain email 'user'.
there is a pivote table thing but i couldn't get it to work.
I'm dealing with this data as a table in mysql, but it is available in csv file, so if it isn't possible with a query, python would be a possible solution, and prefered after sql.
update
in python, is it possible to output the data as:
+-------+--------+---------+------+----------+-----+--------+-------+
| | val1 | val2 | val3 |
+-------+--------+---------+------+----------+-----+--------+-------+
| email | cast |little ja|core | qar |cam | ja sa | jaja |
+-------+--------+---------+------+----------+-----+--------+--------|
|jal#h | 55 | 2 | 44 | 244 | 1 | 200 | 12 |
|hal#b | 900 | 513 | 101 | 146 | 2 | 733 | 833 |
|bam#t | 1231 | 33 | 433 | 411 | 933 | 833 | 53 |
+-------+--------+--------+------+----------+-----+--------+---------
I'm not very familiar with python.
If you use pandas, you can do a value_counts after grouping your data frame by email and then unstack/pivot it to wide format:
(df.set_index("email").stack().groupby(level=0).value_counts()
.unstack(level=1).reset_index().fillna(0))
To get the updated result, you can group by both the email and val* columns after the stack:
(df.set_index("email").stack().groupby(level=[0, 1]).value_counts()
.unstack(level=[1, 2]).fillna(0).sort_index(axis=1))
I'd reconstruct dataframe, then groupby and unstack with pd.value_counts
v = df.values
s = pd.Series(v[:, 1:].ravel(), v[:, 0].repeat(3))
s.groupby(level=0).value_counts().unstack(fill_value=0)
cam cast core ja sa jaja little ja qar
bam#t 1 1 1 0 0 0 0
hal#b 0 0 0 1 0 1 1
jal#h 1 1 1 0 1 1 1
If you know the list you can calculate it using group by:
SELECT email,
sum(val1 = 'cast') as `cast`,
sum(val1 = 'core') as `core`,
sum(val1 = 'cam') as `cam`,
. . .
FROM table1
GROUP BY email;
The . . . is for you to fill in the remaining values.
You can use this Query to generate a PREPARED Statement dynamic from your Values val1-val3 in your table:
SELECT
CONCAT( "SELECT email,\n",
GROUP_CONCAT(
CONCAT (" SUM(IF('",val1,"' IN(val1,val2,val3),1,0)) AS '",val1,"'")
SEPARATOR ',\n'),
"\nFROM table1\nGROUP BY EMAIL\nORDER BY email") INTO #myquery
FROM (
SELECT val1 FROM table1
UNION SELECT val2 FROM table1
UNION SELECT val3 FROM table1
) AS vals
ORDER BY val1;
-- ONLY TO VERIFY QUERY
SELECT #myquery;
PREPARE stmt FROM #myquery;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
sample table
mysql> SELECT * FROM table1;
+----+-------+-----------+------+-------+
| id | email | val1 | val2 | val3 |
+----+-------+-----------+------+-------+
| 1 | jal#h | cast | core | cam |
| 2 | hal#b | little ja | qar | ja sa |
| 3 | bam#t | cast | core | cam |
| 4 | jal#h | little ja | qar | cast |
+----+-------+-----------+------+-------+
4 rows in set (0,00 sec)
generate query
mysql> SELECT
-> CONCAT( "SELECT email,\n",
-> GROUP_CONCAT(
-> CONCAT (" SUM(IF('",val1,"' IN(val1,val2,val3),1,0)) AS '",val1,"'")
-> SEPARATOR ',\n'),
-> "\nFROM table1\nGROUP BY EMAIL\nORDER BY email") INTO #myquery
-> FROM (
-> SELECT val1 FROM table1
-> UNION SELECT val2 FROM table1
-> UNION SELECT val3 FROM table1
-> ) AS vals
-> ORDER BY val1;
Query OK, 1 row affected (0,00 sec)
verify query
mysql> -- ONLY TO VERIFY QUERY
mysql> SELECT #myquery;
SELECT email,
SUM(IF('cast' IN(val1,val2,val3),1,0)) AS 'cast',
SUM(IF('little ja' IN(val1,val2,val3),1,0)) AS 'little ja',
SUM(IF('core' IN(val1,val2,val3),1,0)) AS 'core',
SUM(IF('qar' IN(val1,val2,val3),1,0)) AS 'qar',
SUM(IF('cam' IN(val1,val2,val3),1,0)) AS 'cam',
SUM(IF('ja sa' IN(val1,val2,val3),1,0)) AS 'ja sa'
FROM table1
GROUP BY EMAIL
ORDER BY email
1 row in set (0,00 sec)
execute query
mysql> PREPARE stmt FROM #myquery;
Query OK, 0 rows affected (0,00 sec)
Statement prepared
mysql> EXECUTE stmt;
+-------+------+-----------+------+------+------+-------+
| email | cast | little ja | core | qar | cam | ja sa |
+-------+------+-----------+------+------+------+-------+
| bam#t | 1 | 0 | 1 | 0 | 1 | 0 |
| hal#b | 0 | 1 | 0 | 1 | 0 | 1 |
| jal#h | 2 | 1 | 1 | 1 | 1 | 0 |
+-------+------+-----------+------+------+------+-------+
3 rows in set (0,00 sec)
mysql> DEALLOCATE PREPARE stmt;
Query OK, 0 rows affected (0,00 sec)
mysql>
Is it possible to select whole row from table_1 (without autoincrement ID) and insert it into another table table_2 which has the same relational scheme as table_1 (the same columns)?
I can do that using for example Python but the table has too much rows to do write a code for that.
So this is the example:
table_1:
id | name | age | sex | degree
1 | Pate | 98 | it | doc
2 | Ken | 112 | male| -
table_2:
id | name | age | sex | degree
SQLite3:
INSERT INTO table_2 (SELECT * FROM table_1 WHERE id=2);
RESULT:
table_2:
id | name | age | sex | degree
1 | Ken | 112 | male| -
EDIT:
If this is not possible, it could be done including id so the table_2 would look like:
id | name | age | sex | degree
2 | Ken | 112 | male| -
The INSERT statement indeed has a form that inserts each row returned by a SELECT. However, that SELECT is not a subquery, so you have to omit the parentheses around it, and when you're not inserting all columns, you have to specifiy which columns to use:
INSERT INTO table_2 (name, age, sex, degree)
SELECT name, age, sex, degree
FROM table_1
WHERE id = 2;
I am working on a django based project which created two tables table1 and table2 with a foreign key from table2 to table1. I need to insert some values manually into table2. On trying to do so in MySQL I get the constraint failure error. I checked thoroughly, corresponding entries required for a foreign key constraint already exist in table1. I looked into some previous questions on SO but could not find an appropriate solution.
Table1 Schema:
+------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| email | varchar(100) | NO | | NULL | |
Table 2:
+-------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| km | varchar(1000) | YES | | NULL | |
| owner_id | int(11) | NO | MUL | NULL | |
| receiver_id | int(11) | NO | MUL | NULL | |
+-------------+---------------+------+-----+---------+----------------+
The insert statement insert into crest_recipient values(id,owner_id=5,receiver_id=5,km="hello world");
or similar ones fail with the exact error as
Cannot add or update a child row: a foreign key constraint fails
(crest.crest_recipient CONSTRAINT
crest_recipient_owner_id_4943116a1387be04_fk_crest_user_id FOREIGN
KEY (owner_id) REFERENCES crest_user (id))
You are attempting to update or insert a row in crest.crest_recipient table, a value in column owner_id
that does not exist in crest_user column id
Edit
make it look like this
insert crest_recipient (km,owner_id,receiver_id) values ('test',5,1)
'test',5,1 are whatever you want.
Skip the id column. Don't specify it, don't supply the value for it. It is an auto_increment PK most likely (basically there is nothing else it can be). The db engine chooses it for you. Specifying it will just screw things up 99% of the time.
Focus on parameter 2 for now (the 5)
You are trying to insert a row in child table crest_recipient which does not exist in parent table crest_user.
So first insert corresponding row in master table then you can insert in child table.