I am attempting to insert Excel spreadsheets into a Postgres DB using a Python script with psychopg2.
The problem is not all the spreadsheets have the same number of columns, and I need the insert statement to be flexible enough so I don't have to specify them by name.
My approach is to load the columns of the spreadsheet's header row into a tuple, and likewise with the values being inserted. So for example:
sql = ''''INSERT INTO my_table (%s) VALUES (%s);'''
cur.execute(sql, (cols, vals))
where 'cols' and 'vals' are both tuples.
'cols' can have 7, 9, 10, etc. entries, again depending on how many columns the spreadsheet had.
When I attempt to run this, I get:
psycopg2.ProgrammingError: syntax error at or near "'INSERT INTO my_table
(ARRAY['"
LINE 1: 'INSERT INTO my_table...
^
Not sure if the problem is in my calling syntax, or if you simply can't do what I'm trying to do.
There's an apostrophe ' at the beginning of your sql query.
''''INSERT INTO my_table (%s) VALUES (%s);'''
should be
'''INSERT INTO my_table (%s) VALUES (%s);'''
Edit: didn't realize you where trying to query columns dynamically. To do that, you should use text formatting. Asuming cols is a list:
sql = '''INSERT INTO my_table ({}) VALUES (%s)'''.format(','.join(cols))
Then, your execution would be:
cur.execute(sql, (vals,))
Related
trying to insert values into one MySQL table using python.
First inserting values from csvfile; with:
sql = "INSERT INTO mydb.table(time,day,number)values %r" % tuple (values),)
cursor.execute(sql)
then insert into the same table and same row an other value
sql = "INSERT INTO mydb.table(name) values(%s)"
cursor.execute(sql)
with this i get the inserts in two different rows…
But i need to insert it into the same row without using sql = "INSERT INTO mydb.table(time,day,number,name)values %r" % tuple (values),)
Is there a way to insert values into the same row in two 'insert statements'?
INSERT will always add a new row. If you want to change values in this row, you have to specify a unique identifier (key) in the WHERE clause to access this row and use UPDATE or REPLACE instead.
When using REPLACE you need to be careful if your table contains an auto_increment column, since a new value will be generated.
I want to read from MSSQL table then insert in to MySQL table but i couldn't format my MSSQL results to executemany on them
cursor.execute('select * from table') # MSSQL
rows = cursor.fetchall()
many_rows = []
for row in rows:
many_rows.append((row))
sql = "insert into mysql.table VALUES (NULL, %s) on duplicate key update REFCOLUMN=REFCOLUMN" # MYSQL
mycursor.executemany(sql, many_rows)
mydb.commit()
this gives Failed executing the operation; Could not process parameters
First NULL is for id column and %s for other 49 columns. It works with 1 by 1 but takes ages over remote connection
EDIT
my example print output of many_rows:
[
(49 columns' values, all string and separated by comma),
(49 columns' values, all string and separated by comma),
(49 columns' values, all string and separated by comma),
...
]
I was able to fix my issue with appending data like below:
many_rows.append((list(row)))
In python script i have insert query but when i want insert multiple columns in the same query it gives error.
but for single query it works perfectly.
Below is my code.
my database AWS S3.
A = [] #
for score_row in score:
A.append(score_row[2])
print("A=",A)
B = [] #
for day_row in score:
B.append(day_row[1])
print("B=",B)
for x,y in zip(A,B):
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.executemany(sql, (x,),(y,))
when i replace above query with following sql insert statement it works perfect.
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?)"""
cursor.executemany(sql, (x,))
Fix your code like this:
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.execute(sql, (x,y,)) #<-- here
Because is just a onet insert ( not several inserts )
Explanation
I guess you are mistaked about number of inserts ( rows ) and number of paràmeters ( fields to insert on each row ). When you want to insert several rows, use executemany, just for one row you should to use execute. Second parapeter of execute is the "list" (or sequence ) of values to be inserted in this row.
Alternative
You can try to change syntax and insert all data in one shot using ** syntax:
values = zip(A,B) #instead of "for"
sql = """INSERT INTO calculated_corr_coeff(date,Day) VALUES (?,?)"""
cursor.executemany(sql, **values )
Notice this approach don't use for statement. This mean all data is send to database in one call, this is more efficient.
I need to update columns values in csv to mysql database and the csv values are dynamic for one file it may be 10 columns and for other it may be 5 columns.
my understanding in python we need to use list also this question raised earlier is similar to my requirement but here the values are static so it can be predefined where's in my case being dynamic need to a solution to have %%s under VALUES to be multiplied according to my column values dynamically.
MySQL Dynamic Query Statement in Python
ds_list=['name','id','class','chapter','MTH','YEAR']
vl_list=['xxxxx','978000048','x-grade','Science',mar,2017]
sql = 'INSERT INTO ann1 (%s) VALUES (%%s, %%s, %%s, %%s, %%s, %%s)' % ','.join(ds_list)
cur.execute(sql, vl_list)
conn.commit()
So, if you have two lists, one with headers and the other with values – you can create yourself dynamic INSERT query.
query_placeholders = ', '.join(['%s'] * len(vl_list))
query_columns = ', '.join(ds_list)
insert_query = ''' INSERT INTO table (%s) VALUES (%s) ''' %(query_columns, query_placeholders)
and then execute & commit your query by passing list with values in query.
cursor.execute(insert_query, vl_list)
I have a table called sentiments with columns Date, Positive, Negative, Neutral.
How can i write my sql query such that i can insert date values as in my code below?
i think the problem lies with %s for the datetime values because all the other %s values for the columns other than date were able to be inserted.
The code below returns a sql syntax error.
for item in list_of_dates:
d['Period'].append(datetime.datetime.fromtimestamp(int(item)/1000).strftime('%Y-%m-%d %H:%M:%S'))
combined_list=zip(d['period'],d['Positive'],d['Negative'],d['Neutral'])
with con:
cur = con.cursor()
for x,y,w,z in combined_list:
cur.execute("INSERT INTO sentiments(%s,%s,%s,%s) VALUES(%s,%s,%s,%s)"
%("Date","Positive","Negative","Neutral",str(x),str(y),str(w),str(z)))
The problem with the query lies in the Values. it should have small quotations for all the %s as shown below.
VALUES('%s','%s','%s','%s')