Insert from a list after checking in mysql if duplicate - python

From my list, I am looking to insert row by row, but I need to perform a checking before allowing to insert in the database.
let's imagine that is my list
unique_hrefs = [
https://www.linkedin.com/in/123456,
https://www.linkedin.com/in/789013556,
https://www.linkedin.com/in/888888888,
https://www.linkedin.com/in/082b62112,
https://www.linkedin.com/in/5625a1a,
https://www.linkedin.com/in/123456,
https://www.linkedin.com/in/0000000341454,
https://www.linkedin.com/in/55555555,
https://www.linkedin.com/in/55555555,
https://www.linkedin.com/in/66666666,
https://www.linkedin.com/in/666677777
]
I need to check my table for the same string before allowing to inserting and if the string does not exist inserting.
I have my code to inset but struggling to find how to check before inserting it?
query= "INSERT INTO name_links (ulr_name) VALUES (%s)
cursor.executemany(query,[(r,) for r in unique_hrefs])
mydb.commit()

Related

Insert values in table with two excecute commands

trying to insert values into one MySQL table using python.
First inserting values from csvfile; with:
sql = "INSERT INTO mydb.table(time,day,number)values %r" % tuple (values),)
cursor.execute(sql)
then insert into the same table and same row an other value
sql = "INSERT INTO mydb.table(name) values(%s)"
cursor.execute(sql)
with this i get the inserts in two different rows…
But i need to insert it into the same row without using sql = "INSERT INTO mydb.table(time,day,number,name)values %r" % tuple (values),)
Is there a way to insert values into the same row in two 'insert statements'?
INSERT will always add a new row. If you want to change values in this row, you have to specify a unique identifier (key) in the WHERE clause to access this row and use UPDATE or REPLACE instead.
When using REPLACE you need to be careful if your table contains an auto_increment column, since a new value will be generated.

In python script i have insert query but when i want insert multiple columns in the same query it gives error

In python script i have insert query but when i want insert multiple columns in the same query it gives error.
but for single query it works perfectly.
Below is my code.
my database AWS S3.
A = [] #
for score_row in score:
A.append(score_row[2])
print("A=",A)
B = [] #
for day_row in score:
B.append(day_row[1])
print("B=",B)
for x,y in zip(A,B):
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.executemany(sql, (x,),(y,))
when i replace above query with following sql insert statement it works perfect.
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?)"""
cursor.executemany(sql, (x,))
Fix your code like this:
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.execute(sql, (x,y,)) #<-- here
Because is just a onet insert ( not several inserts )
Explanation
I guess you are mistaked about number of inserts ( rows ) and number of parĂ meters ( fields to insert on each row ). When you want to insert several rows, use executemany, just for one row you should to use execute. Second parapeter of execute is the "list" (or sequence ) of values to be inserted in this row.
Alternative
You can try to change syntax and insert all data in one shot using ** syntax:
values = zip(A,B) #instead of "for"
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.executemany(sql, **values )
Notice this approach don't use for statement. This mean all data is send to database in one call, this is more efficient.

psychopg2 to generate insert statements with variable column counts

I am attempting to insert Excel spreadsheets into a Postgres DB using a Python script with psychopg2.
The problem is not all the spreadsheets have the same number of columns, and I need the insert statement to be flexible enough so I don't have to specify them by name.
My approach is to load the columns of the spreadsheet's header row into a tuple, and likewise with the values being inserted. So for example:
sql = ''''INSERT INTO my_table (%s) VALUES (%s);'''
cur.execute(sql, (cols, vals))
where 'cols' and 'vals' are both tuples.
'cols' can have 7, 9, 10, etc. entries, again depending on how many columns the spreadsheet had.
When I attempt to run this, I get:
psycopg2.ProgrammingError: syntax error at or near "'INSERT INTO my_table
(ARRAY['"
LINE 1: 'INSERT INTO my_table...
^
Not sure if the problem is in my calling syntax, or if you simply can't do what I'm trying to do.
There's an apostrophe ' at the beginning of your sql query.
''''INSERT INTO my_table (%s) VALUES (%s);'''
should be
'''INSERT INTO my_table (%s) VALUES (%s);'''
Edit: didn't realize you where trying to query columns dynamically. To do that, you should use text formatting. Asuming cols is a list:
sql = '''INSERT INTO my_table ({}) VALUES (%s)'''.format(','.join(cols))
Then, your execution would be:
cur.execute(sql, (vals,))

How can I do upsert (update and insert) query in MySQL Python?

I'm looking for a simple upsert (Update/Insert).
I have table in which I am inserting rows for books table but next time when I want to insert row I don't want to insert again data for that table just want to update with required columns if it exits there if not then create new row.
How can I do this in Mysql-python?
cursor.execute("""INSERT INTO books (book_code,book_name,created_at,updated_at) VALUES (%s,%s,%s,%s)""", (book_code,book_name,curr_time,curr_time,))
MySQL has REPLACE statement:
REPLACE works exactly like INSERT, except that if an old row in the
table has the same value as a new row for a PRIMARY KEY or a UNIQUE
index, the old row is deleted before the new row is inserted.
cursor.execute("""
REPLACE INTO books (book_code,book_name,created_at,updated_at)
VALUES (%s,%s,%s,%s)""",
(book_code,book_name,curr_time,curr_time,)
)
UPDATE According to comment of #Yo-han, REPLACE is like DELETE and INSERT, not UPSERT. Here's alternative using INSERT ... ON DUPLICATE KEY UPDATE:
cursor.execute("""
INSERT INTO books (book_code,book_name,created_at,updated_at)
VALUES (%s,%s,%s,%s)
ON DUPLICATE KEY UPDATE book_name=%s, created_at=%s, updated_at=%s
""", (book_code, book_name, curr_time, curr_time, book_name, curr_time, curr_time))

Add list to sqlite database

How would I add something in sqlite to an already existing table this is what I have so far
>>> rid
'26539249'
>>> for t in [(rid,("billy","jim"))]:
c.execute("insert into whois values (?,?)",t)
How would I add onto jim and create a list? or is there some way to add onto it so It can have multiple values?
I'll take a guess here, but I suspect I'm wrong.
You can't insert ("billy", "jim") as a column in the database. This is intentional. The whole point of RDBMSs like sqlite is that each field holds exactly one value, not a list of values. You can't search for 'jim' in the middle of a column shared with other people, you can't join tables based on 'jim', etc.
If you really, really want to do this, you have to pick some way to convert the multiple values into a single string, and to convert them back on reading. You can use json.dumps/json.loads, repr/ast.literal_eval, or anything else that seems appropriate. But you have to write the extra code yourself. And you won't be getting any real benefit out of the database if you do so; you'd be better off just using shelve.
So, I'm guessing you don't want to do this, and you want to know what you want to do instead.
Assuming your schema looks something like this:
CREATE TABLE whois (Rid, Names);
What you want is:
CREATE TABLE whois (Rid);
CREATE TABLE whois_names (Rid, Name, FOREIGN KEY(Rid) REFERENCES whois(Rid);
And then, to do the insert:
tt = [(rid,("billy","jim"))]
for rid, names in tt:
c.execute('INSERT INTO whois VALUES (?)', (rid,))
for name in names:
c.execute('INSERT INTO whois_names VALUES (?, ?)', (rid, name))
Or (probably faster, but not as interleaved):
c.executemany('INSERT INTO whois VALUES (?)', (rid for rid, names in tt))
c.executemany('INSERT INTO whois_names VALUES (?, ?),
(rid, name for rid, names in tt for name in names))
Not tested but should do the trick
conn = sqlite3.connect(db)
cur = conn.cursor()
cur.execute('''CREATE TABLE if not exists Data
(id integer primary key autoincrement, List)''')
cur.execute("INSERT INTO Data (id,List) values (?,?)",
(lid, str(map(lambda v : v, My_list) ) ))

Categories

Resources