I am trying to insert data into a PostgreSQL database table using Python. I don't see any syntax errors but, for some reason, my data isn't getting inserted into the database.
conn = psycopg2.connect(connection)
cursor = conn.cursor()
items = pickle.load(open(pickle_file,"rb"))
for item in items:
city = item[0]
price = item[1]
info = item[2]
query = "INSERT INTO items (info, city, price) VALUES (%s, %s, %s);"
data = (info, city, price)
cursor.execute(query, data)
You have to commit the transaction.
conn.commit()
If there's no reason to think the transaction will fail, it's faster to commit after the for loop finishes.
Related
How to insert data from code below?
I have a code below
latitude1 = -6.208470935786019
longitude1 = 106.81796891087399
new_data = [[latitude1, longitude1]]
preds = model.predict(new_data)
preds
arr = [latitude1,longitude1]
arrcon = np.concatenate((arr,preds))
print(arrcon) #[-6.208470935786019 106.81796891087399 'Not Categorized']
listarcon= arrcon.tolist()
print(listarcon) #[-6.208470935786019, 106.81796891087399, 'Not Categorized']
#make the list into multi list
singlearcon = np.array(listarcon).reshape(1,3)
print(singlearcon) #[['-6.208470935786019' '106.81796891087399' 'Not Categorized']]
This is insert into database code
mycursor = conn.cursor()
sql = "INSERT INTO traveldata (Latitude,Longitude,Wisata) VALUES (%s, %s, %s)"
val = (listarcon[0],listarcon[1],listarcon[2])
mycursor.execute(sql, val)
How to insert it to database? the data didn't seem to get to the database.
After executing a transaction mycursor.execute(sql, val), we should commit the change mycursor.commit()
Reference for the commit method https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlconnection-commit.html
An example of insert code https://dev.mysql.com/doc/connector-python/en/connector-python-example-cursor-transaction.html
import mysql.connector
def add_features_to_db(stockname, timeframe, date, feature):
try:
conn = mysql.connector.connect(
user='root', password='', host='localhost', database='fx003')
cursor = conn.cursor()
dbtable = stockname + timeframe
mySql_insert_query = """INSERT INTO `%s` (date, trend) VALUES ( `%s`, `%s` )"""
record = (dbtable, date, feature)
cursor.execute(mySql_insert_query, record)
conn.commit()
print("Record inserted successfully")
except mysql.connector.Error as error:
print("Failed to insert into MySQL table {}".format(error))
finally:
if conn.is_connected():
cursor.close()
conn.close()
print("MySQL connection is closed")
add_features_to_db("aud-cad", "_30mins", "2021-09-24 21:00:00", "Short")
I have the code above and giving me the below error:
Failed to insert into MySQL table 1146 (42S02): Table 'fx003.'aud-cad_30mins'' doesn't exist
aud-cad_30mins table does exist and an insert query like below doing its job:
mySql_insert_query = """INSERT INTO aud-cad_30mins (date, trend) VALUES ( "2021-09-24 21:00:00","Short" )"""
So when I try to use variables in the query, it gives the error. Why the table name getting unwanted quotes? Checked several tutorials but couldn't find a solution, any ideas?
The table name should be hardcoded in the query string instead of having it there as a placeholder %s, which is meant for the values to be inserted. So if you have the table name in the variable, you can replace it via format() before calling cursor.execute()
dbtable = stockname + timeframe
mySql_insert_query = """INSERT INTO {} (date, trend) VALUES ( %s, %s )""".format(dbtable)
see the examples in the docs
edit: as Bill mentioned in the comment, dont add the backticks around the %s placeholders.
Upsert to MySQL using python and data from excel.
Im working on populating a MySQL DB, using python.
The data is stored on excel sheets.
Because the DB is suppossed to be used for monitoring "projects", there's a posibility for repeated pk, so in that case it need to be updated instead of insert, because a project can have many stages.
Also, there's a value to be inserted in the DB table, that can't be added from the spreadsheet. So i'm wondering if in that case, the insert of this value, most be done using a separated query for it or if theres a way to insert it in the same query. The value is the supplier ID and needs to be inserted between id_ops and cif_store.
And to finish, I need to perform an inner join, to import the store_id using the store_cif, from another table called store. I know how do it, but im wondering if it also must be executed from a sepparated query or can be performed at the sameone.
So far, i have done this.
import xlrd
import MySQLdb
def insert():
book = xlrd.open_workbook(r"C:\Users\DevEnviroment\Desktop\OPERACIONES.xlsx")
sheet = book.sheet_by_name("Sheet1")
database = MySQLdb.connect (host="localhost", user = "pytest", passwd = "password", db = "opstest1")
cursor = database.cursor()
query = """INSERT INTO operation (id_ops, cif_store, date, client,
time_resp, id_area_service) VALUES (%s, %s, %s, %s, %s, %s)"""
for r in range(1, sheet.nrows):
id_ops = sheet.cell(r,0).value
cif_store = sheet.cell(r,1).value
date = sheet.cell(r,2).value
client = sheet.cell(r,3).value
time_resp = sheet.cell(r,4).value
id_area_service = sheet.cell(r,5).value
values = (id_ops, cif_store, date, client, time_resp, id_area_service)
cursor.execute(query, values)
# Close the cursor
cursor.close()
# Commit the transaction
database.commit()
# Close the database connection
database.close()
# Print results
print ("")
print ("")
columns = str(sheet.ncols)
rows = str(sheet.nrows)
print ("Imported", columns,"columns and", rows, "rows. All Done!")
insert()
What you are looking for is INSERT ... ON DUPLICATE KEY UPDATE ...
Take a look here https://dev.mysql.com/doc/refman/8.0/en/insert-on-duplicate.html
Regarding the extraneous data, if its a static value for all rows you can just hard code it right into the INSERT query. If it's dynamic you'll have to write some additional logic.
For example:
query = """INSERT INTO operation (id_ops, hard_coded_value, cif_store, date, client,
time_resp, id_area_service) VALUES (%s, "my hard coded value", %s, %s, %s, %s, %s)"""
cur1 = connection.cursor()
cur3 = connection.cursor()
cur3.execute("SELECT * FROM TABLE1")
connection.commit()
for i in range(0,totalRow-1):
row = cur3.fetchone()
if tempId.__contains__(row[0]):
cur1.execute("insert into summary (id, description, resolution) values (%s, %s, %s)",(row[0],row[1],tempResolution[tempId.index(row[0])]))
The above code is not giving any error but data is not inserting in the database.
instead of last line, try this:
tuple = row[0], row[1], tempResolution[tempId.index(row[0])]
cur1.executemany("insert into summary (id, description, resolution) values (?,?,?)", tuple)
if that doesn't work please expand your code explaining what are totalRow, tempId.__contains__ and types of row[0],row[1],tempResolution[tempId.index(row[0])]
I have a MySQL Table named TBLTEST with two columns ID and qSQL. Each qSQL has SQL queries in it.
I have another table FACTRESTTBL.
There are 10 rows in the table TBLTEST.
For example, On TBLTEST lets take id =4 and qSQL ="select id, city, state from ABC".
How can I insert into the FACTRESTTBL from TBLTEST using python, may be using dictionary?
Thx!
You can use MySQLdb for Python.
Sample code (you'll need to debug it as I have no way of running it here):
#!/usr/bin/python
import MySQLdb
# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
# Select qSQL with id=4.
cursor.execute("SELECT qSQL FROM TBLTEST WHERE id = 4")
# Fetch a single row using fetchone() method.
results = cursor.fetchone()
qSQL = results[0]
cursor.execute(qSQL)
# Fetch all the rows in a list of lists.
qSQLresults = cursor.fetchall()
for row in qSQLresults:
id = row[0]
city = row[1]
#SQL query to INSERT a record into the table FACTRESTTBL.
cursor.execute('''INSERT into FACTRESTTBL (id, city)
values (%s, %s)''',
(id, city))
# Commit your changes in the database
db.commit()
# disconnect from server
db.close()