I'm inserting data into a table where i'm skipping, excluding, data that is already in the table with that price and sku value.
Im running in an issue where when my variables have the same value ex. price and sale_price are both 0 or companiy_id and status are both 1 i get an error Duplicate column name.
sql_insert = "INSERT INTO apps_scraper_data_history (sku, price, company_id, date_time, status, item_key, price_sale) SELECT * FROM (SELECT %s, %s, %s, %s, %s, %s, %s) AS tmp WHERE NOT EXISTS(SELECT sku FROM apps_scraper_data_history WHERE price=%s AND sku=%s)"
val_insert = (sku, price, company, date, status, key, sale_price, price_scrape, sku)
cursor.execute(sql_insert, val_insert)
You didn't define column names for alias tmp. So that, when value of two columns are same those might be conflicting. Please try below query with column aliases.
sql_insert = "INSERT INTO apps_scraper_data_history (sku, price, company_id, date_time, status, item_key, price_sale) SELECT * FROM (SELECT %s sku, %s price, %s company_id, %s date_time, %s status, %s item_key, %s price_sale) AS tmp WHERE NOT EXISTS(SELECT sku FROM apps_scraper_data_history WHERE price=%s AND sku=%s)"
val_insert = (sku, price, company, date, status, key, sale_price, price_scrape, sku)
cursor.execute(sql_insert, val_insert)
Related
In python, I am trying to insert some data into my table;
try:
cursor.execute("INSERT INTO covid_testtable (pid, age, state, city, notes, backnotes, type, nationality, status, announced, changed) " +
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, STR_TO_DATE(%s, '%%d/%%m/%%Y'), STR_TO_DATE(%s, '%%d/%%m/%%Y'))",
(pid, age, item["detectedstate"], item["detectedcity"], item["notes"], item["backupnotes"], item["typeoftransmission"],
item["nationality"], item["currentstatus"], dateannounced, statuschanged));
except ValueError as verr:
print(item["dateannounced"], verr);
break;
This works till a point where one of the date columns have a empty entry.
pymysql.err.InternalError: (1411, "Incorrect datetime value: '' for function str_to_date"
I tried to change the value of the empty date to "00/00/0000" however it seems the issue is with the str_to_date() function.
I am fine with inserting NULL value however, not managing to insert.
The valid values which are succesfully inserted are of this format "30/01/2020" -> "%d/%m/%Y"
My table schema is as below:
create table covid_testtable ( pid int NOT NULL, age int, state VARCHAR(255),
city VARCHAR(255), notes VARCHAR(255), backnotes VARCHAR(255), type VARCHAR(255),
nationality VARCHAR(255), status VARCHAR(255), announced DATE default NULL,
changed DATE default NULL, PRIMARY KEY (pid));
EDIT 2: Selected Answer solved this issue:
def validate_date(val):
try:
return datetime.strptime(val, '%d/%m/%Y').strftime('%Y-%m-%d');
except ValueError as verr:
return None;
dateannounced = validate_date(item["dateannounced"]);
statuschanged = validate_date(item["statuschangedate"]);
try:
cursor.execute("INSERT INTO covid_testtable (pid, age, state, city, notes, backnotes, type, nationality, status, announced, changed) " +
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",
(pid, age, item["detectedstate"], item["detectedcity"],
item["notes"], item["backupnotes"], item["typeoftransmission"],
item["nationality"], item["currentstatus"], dateannounced,
statuschanged));
Rather than processing the dates in the query, you could pre-process them in your python code. For example:
from datetime import datetime
dstr = ''
try:
dateannounced = datetime.strptime(dstr, '%d/%m/%Y').strftime('%Y-%m-%d')
except ValueError:
dateannounced = None
Then your query simply becomes
cursor.execute("INSERT INTO covid_testtable (pid, age, state, city, notes, backnotes, type, nationality, status, announced, changed) " +
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)",
(pid, age, item["detectedstate"], item["detectedcity"],
item["notes"], item["backupnotes"], item["typeoftransmission"],
item["nationality"], item["currentstatus"], dateannounced,
statuschanged));
Check whether the string is empty with if().
cursor.execute("""
INSERT INTO covid_testtable (pid, age, state, city, notes, backnotes, type, nationality, status, announced, changed)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s,
IF(%s = '', NULL, STR_TO_DATE(%s, '%%d/%%m/%%Y')),
IF(%s = '', NULL, STR_TO_DATE(%s, '%%d/%%m/%%Y')))""",
(pid, age, item["detectedstate"], item["detectedcity"], item["notes"], item["backupnotes"], item["typeoftransmission"],
item["nationality"], item["currentstatus"], dateannounced, dateannounced, statuschanged, statuschanged));
Note that I had to repeat dateannounced and statuschanged in the parameter list, since I'm substituting them into the IF functions in addition to the STR_TO_DATE functions.
I have two tables say Doctors and patients. I need to fetch the doctors id (unique) from the doctors table and then pass it to a column of the patient's table during the INSERT operation. How can I write the query for that.
I tried the below way but it gives programming error:
usr = self.current_user
self.db.execute("INSERT INTO patients ("SELECT id FROM doctors WHERE doctors.id = '%s' " % usr )(mrd, name, age, gender, address, phone_number, blood_group,\
registration_date, did) values ('%s', '%s', '%s', '%s','%s','%s','%s','%s', '%s')" % (mrd,name,age,gender,\
address,phoneNumber, bloodGroup,dateOfReg,usr),callback=self.add_response)
I am trying to execute this in python tornado framework with psql backend
SELECT should be in VALUES section
The query will be like so
doctor_id = self.db.execute("SELECT id FROM doctors WHERE doctors.id = '%s' " % usr ).fetchone()
self.db.execute("INSERT INTO patients(mrd, name, age, gender, address, phone_number, blood_group,registration_date, did, doctor_id) values (%s, %s, %s, %s,%s,%s,%s,%s,%s, %s)" % (mrd,name,age,gender,address,phoneNumber, bloodGroup,dateOfReg,usr, doctor_id),callback=self.add_response)
I have a table and I want to translate columns 'topic' and 'review' of a row and store the entire table with their translations into a new table. It seems that the for-loop doesn't iterate over all rows of the input table. Only the first row is stored into the new table. Why?
database = mysql.connector.connect(user='root', password='root', host='localhost', database='test')
DBcursor = database.cursor(buffered=True)
query = ("SELECT * FROM test_de")
DBcursor.execute(query)
for (id, user_name, date, country, version, score, topic, review, url) in DBcursor:
topic_trans = translate(topic, 'en')
review_trans = translate(review, 'en')
add_translation = ("INSERT INTO test_de_en(id, user_name, date, country, version, score, topic, review, url)"
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)")
translation_data = (id, user_name, date, country, version, score, topic_trans, review_trans, url)
DBcursor.execute(add_translation, translation_data)
database.commit()
DBcursor.close()
database.close()
I am trying to insert data to the table that was created earlier using python script. Here is the code I am trying to execute. I want to insert data into table with date as well.
date_today = dt.date.today()
conn = psycopg2.connect(host = serverip, port = port, database = database, user = uid, password = pwd)
cursor = conn.cursor()
cursor.execute("INSERT INTO My_TABLE (Date, Class, Total_students, failed_students, Percent_passed_students) VALUES (date_today, 'Class Name', int1, int2, int3)")
print "Data Inserted successfully"
conn.commit()
conn.close()
Here is the error I see from my job. what am i missing here?
psycopg2.ProgrammingError: column "date_today" does not exist
I created the table using different job with the following query:
cursor.execute("""CREATE TABLE MY_TABL(Date date, Lob varchar(30), Total_Students int, failed_students int, Percent_passed_students int)""")
And the table is created with above five columns.
This line:
cursor.execute("INSERT INTO My_TABLE (Date, Class, Total_students, failed_students, Percent_passed_students) VALUES (date_today, 'Class Name', int1, int2, int3)")
Is the incorrect way to dynamically insert values into a database.
Here's a functional and correct example:
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
And applying it in your case...
cursor.execute("INSERT INTO My_TABLE VALUES (%s, %s, %s, %s, %s)", (date_today, 'Class Name', int1, int2, int3))
I have a simple registration form which creates a new record in "Students" table:
cur = con.cursor()
query = "INSERT INTO students (Name, Address, Major, Phone, Email, Password) values (%s, %s, %s, %s, %s, %s)"
args = [name, address, major,phone, email, password]
cur.execute(query, args)
con.commit()
The column Id is auto-incremented, so in order to get the last Id, I select Max
query = "SELECT MAX(Id) FROM students;"
cur.execute(query)
id = cur.fetchone()[0]
It works in my little homework-project, but how would it be done in a heavy-loaded project, where there is a possibility of something being created before the select statement?
Use lastrowid attribute:
...
cur.execute(query, args)
id = cur.lastrowid