Suppose, I have a modifying statement:
cursor = conn.cursor()
# some code
affected_rows1 = cursor.execute(update_statement1, params1)
# some code
conn.commit()
cursor.close()
Should I wrap the block of code with a try ... except and explicitly rollback a transaction when an exception is raised, and which MySQLdb exceptions should I catch to rollback? I used to catch any StandardError in this case, but now I have a hesitation that the block of code would even need an explicit rollback at all.
The following example is slightly more difficult, and I understand that it does require an explicit rollback if the first update statement succeeded. Still, which exceptions should I catch in this case:
cursor = conn.cursor()
# some code
affected_rows1 = cursor.execute(update_statement1, params1)
# some code
affected_rows2 = cursor.execute(update_statement2, params2)
#some code
conn.commit()
cursor.close()
This link shows the various types of Errors that you can catch. MySQLdb.Error is the standard base class from which all other MySQL Errors are derived.
I usually use MySQLdb.Error because it lets you focus on errors relating to MySQLdb itself. By contrast StandardError will catch almost all the exceptions (not something you want if you want better debugging capability). Plus the use of MySQLdb.Error allows you to display the exact error message (MySQL error number and all) so that you can debug it faster.
Coming to the first part of the question, in case of database statements it is (usually) necessary to rollback transactions (if they are supported) in case of error.
The methodology that I follow is to wrap each execute statement in a try except clause (catching MySQLdb.Error) and using rollback if there is an an error before printing the error message and exiting.
However, there is a catch. In MySQLdb the changes that you make to DB are not actually written to the database until you explicilty call commit. So, logically, rollback is not necessary.
As an example,
conn = MySQLdb.connection(db=, host=, passwd=, user=)
cur = conn.cursor()
#Say you have a table X with one entry id = 1 and total = 50
cur.execute("update X set total = 70 where id = 1")
#Actual DB has not yet changed
cur.execute("update X set total = 80 where id = 1")
#Actual DB has still not changed
If you exit the program without commiting, the value in DB will still be 50 because you never called commit().
This is how you would ideally do it:
conn = MySQLdb.connection(db=, host=, passwd=, user=)
cur = conn.cursor()
#Say you have a table X with one entry id = 1 and total = 50
try:
cur.execute("update X set total = 70 where id = 1")
except MySQLdb.Error,e:
print e[0], e[1]
conn.rollback()
cur.close()
conn.close()
#print lengthy error description!!
sys.exit(2)
#Note: Value in table is still 50
#If you do conn.commit() here, value becomes 70 in table too!!
try:
cur.execute("update X set total = 80 where id = 1")
except MySQLdb.Error,e:
print e[0], e[1]
conn.rollback()
cur.close()
conn.close()
#print lengthy error description!!
sys.exit(2)
#Value in DB will be
#a) 50 if you didn't commit anywhere
#b) 70 if you committed after first execute statement
conn.commit()
#Now value in DB is 80!!
cur.close()
conn.close()
IMHO, you should rollback transactions if you continue to use the same connection. Else everything before the error will get commit when you finish the transactions.
For the exception to catch, I always use MySQLdb.Error but I'm not sure that's correct.
Its advised to wrap execute() in a sub. This is how i do it.
def executeSQL(self, stmt):
cursor = self.dbHand.cursor()
if not stmt.endswith(";"):
stmt += ';'
try:
cursor.execute(stmt)
except MySQLdb.Error, e:
self.logger.error("Caught MYSQL exception :%s: while executing stmt :%s:.\n"%(e,stmt))
return False
Related
I have the code below to read the chat.db iMessage database on Mac. I am pretty sure this is correct but when I print I get an empty list.
try:
messages = sqlite3.connect("<path>\\chat.db")
except Exception as e:
print(e)
cur = messages.cursor()
results = cur.execute("SELECT name FROM sqlite_master WHERE type='table'")
print(results.fetchall())
returns []
I saw /tmp should not be full. I removed about 10% of it (down to 90% full) but it still is not not working
I want to print a message in case the user made a mistake while writing the code but it doesnt work I also tried to add NameError exception, it only works if I raise an exception.Thank you for helping.
`
def cncours(nvcours,num_cours):
try :
sql="Update cours set nomC=%s where num_cours=%s"
result=cursor.fetchone()
cursor.execute(sql,(nvcours,num_cours))
print("Operation Done.")
except TypeError:
print("Plz put the name between quotes")
`
Each DB implementation (MySQL, sqlite,...) may raise their particular exceptions. So instead of catching a general exceptions, you may catch errors depending on the specific DB, then on the particular type (e.g. SyntaxError). I suggest to provoque a syntax error on your SQL statement then see what type is (e.g. errorcode or exception type) then catch it.
For instance, the MySQL connector raises error numbers:
import mysql.connector as cn
try:
#...
except cn.Error as err:
print("Something went wrong: {}".format(err))
if err.errno == errorcode.ER_BAD_TABLE_ERROR: #
Here are some MySQL Error Code Ranges
If you are using MySQLdb:
import MySQLdb
try:
#...
cursor.execute(sql)
res = cursor.fetchone()
# ...
except MySQLdb.Error, e:
print "MySQLdb.Error: {}".format(e.args)
Depending on your schema (column types) and the type from the input variable, you may use:
sql="Update cours set nomC='%s' where num_cours=%s" # Added quotes on the first replacement
Besides what you are asking, I think that the command order is inverted.
sql="Update cours set nomC=%s where num_cours=%s"
cursor.execute(sql,(nvcours,num_cours)) # First
result=cursor.fetchone() # Second
print("Operation Done.")
https://www.tutorialspoint.com/python/python_database_access.htm
# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")
# Fetch a single row using fetchone() method.
data = cursor.fetchone()
I am using pyMysql for saving data to mysql db. I have permanent flow of tick data from financial market. i use cur.executemany for inserting 10 lines by one time. It worked fine just for first 20-30 lines - then it stops write it and don't throw any exception..
self.queue.append((timestamp,side,size,price))
if len(self.queue)>=10:
try:
self.logger.info("Writing 10 lines to sql..")
conn = pymysql.connect(host='localhost', port=3306, user='root', passwd='*****', db='sys')
cur = conn.cursor()
sqlQ=""" INSERT den_trades (date2 , side, size, price) VALUES (%s,%s,%s,%s)"""
cur.executemany(sqlQ, self.queue)
conn.commit()
conn.close()
self.queue=[]
except Exception as e:
self.logger.warning("Exception while cur.executemany... sys.exc_info()[0]: {}".format(sys.exc_info()[0]))
self.logger.warning("e.message ".format(e.message))
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(e).__name__, e.args)
self.logger.warning(message)
conn.rollback()
i am trying to get some exceptions - but no one warning..
strange thing is - when problem appears - "Writing 10 lines to sql.." still work fine for every tick - self.queue continue to grow up - so self.queue=[] never happens.. how can it be? first string of try: block still work, but last string stop occur.. if so - there should be any exception... right?
one more thing. i have another script that's running fine on same machine. that script saving 1000 lines by one time through pyMysql.
Can it be a problem?
Here are data in MySQL databaseļ¼
python3 tableļ¼users_info
But when i use fetchall to get all data in tables it always return none!!
I really really dont konw how to fix it,can someone has met this problem?
the followings are the code files.
encapsulation.py
import MySQLdb
class mysql_encapsulation(object):
def __init__(self,host,port,user,passwd,db,charset):
self.host = host
self.port = port
self.user = user
self.passwd = passwd
self.db = db
self.charset = charset
def open(self):
self.conn = MySQLdb.connect(host=self.host,port=self.port,user=self.user,passwd=self.passwd,db=self.db,charset=self.charset)
self.cursor = self.conn.cursor()
def close(self):
self.cursor.close()
self.conn.close()
def operate(self,sql,params):
try:
self.open()
self.cursor.execute(sql,params)
self.conn.commit()
print(' operate ok')
self.close()
except Exception,e:
print(e.message)
self.conn.rollback()
def get_result_set(self,sql,params=[]):
result=None
try:
self.open()
self.cursor.execute(sql,params)
result = self.cursor.fetchall()
self.close()
except Exception,e:
print('error!')
print(e.message)
return result
use.py(problem in this file)
#coding=utf-8
from encapsulation import *
mysql = mysql_encapsulation(port=3306,host='localhost',user='root',passwd='mysql',
db='python3',charset='utf8')
sql='select id,name from users_info where id=3'
result=mysql.get_result_set(sql)
print (result)
The problem happens in this method;
def get_result_set(self,sql,params=[]):
result=None
try:
self.open()
self.cursor.execute(sql,params)
result = self.cursor.fetchall()
self.close()
except Exception,e:
print('error!')
print(e.message)
return result
One of the first 3 lines in the try block (self.open / self.cursor.exec / result = self.cursor.fetch) generate an error, which you are then catching in the except block and (you can see that it prints "error!"). That's why the result always stays in it's default None value. Remove the except block and it will tell you what kind of error occured.
You should almost never catch the bare Exception but instead catch specific kinds of exception and handle each of them correctly, this problem is a prefect example why.
The error in question probably happens because in your SQL query you are selecting id and name, when the columns in your table are actually id and user_name. So your SQL query should be like this;
sql = 'select id, user_name from users_info where id = 3'
Did you check your sql statement? Based on your table your columns are "id", "user_name" and "passwd", but in your sql you are searching for "id" and "name", and "name" isn't a column so that will throw an error. Change your sql to "sql='select id,user_name from users_info where id=3'"
As ruohola already explained in his answer, your exception handler is hiding all the important informations about what really went wrong - FWIW, there's the "Error !" string printed in your screen capture just above the None. The point is: your try/except block is not only useless, it's actually harmful - it prevents some other code up the call stack to even be aware that there was a problem (and eventually solve it), it also prevents you from knowing what went wrong. As a general rule, only catch exception the exact exception that you expect AND can effectively handle at this point in the code - if you can't fix the error at this point, let the exception propagate and let the calling code deal with it. Also, you want to have as few code as possible in your try clause (the only exception being the application's top-level exception handler of course).
This being said, there is indeed a very valid reason for wanting to be warned of exception, which is to make sure you free resources (and eventually rollback a transaction), but then you want 1/ to use a finally clause for resources cleanup (a finally block is always executed, whatever happens) and 2/ for the rollback part, use an except clause but re-raise the exception, ie:
# leave this one out of the try block -
# there's nothing you can do here about a connection error
# so let the calling code deal with it
self.connect()
try:
self.cursor.execute(sql, params)
except Exception as e:
self.connection.rollback()
finally:
# this will ALWAYS be executed (unless `self.connect()` raised
# but then you don't even want to close the connection <g>
self.close()
wrt/ the real cause of your problem, Kurose's answer is certainly the right one.
Also, there are a quite a few other things that are debatable with your "mysql_encapsulation" class (the naming to start with but let ignore this ATM), the main one being to open and close the connection for each query. Opening a database connection has a rather high cost, so you want to keep it opened as long as possible (and eventually reconnect if you get a "2006 mysql has gone away" error).
I'm making a python program that does Elo calculations for an Azure SQL database. The problem is with the last two 'cursor.execute' commands (the UPDATEs).
I took out part of the code before posting here to make it smaller, but all the variables are properly passed from the find_winner and find_loser methods-- the print commands show the right value.
When I run the program as is, it prints the change in ratings and the message from the except block. When I comment out the UPDATE methods, it doesn't print the except message. The only reason I can come up with is that the variables from the tuple from find_winner and find_loser aren't being entered into the SQL statement properly.
I tried running it with ? and '%s' instead of winner_new_rating and winner_id, but none of the 3 versions worked.
Is there something obvious I'm missing? What's the proper way of entering parameters stored in variables?
def rate():
try:
(winner_rating,winner_name,winner_id) = find_winner()
(loser_rating,loser_name,loser_id) = find_loser()
cursor = conn.cursor()
print(winner_name, "wins", winner_rating, "-->", winner_new_rating)
print(loser_name, "loses:", loser_rating, "-->", loser_new_rating)
cursor.execute("UPDATE KIDS SET Rating = winner_new_rating WHERE LocalID = winner_id")
cursor.execute("UPDATE KIDS SET Rating = loser_new_rating WHERE LocalID = loser_id")
conn.commit()
except:
print("Rate method error")
This is the correct syntax:
try:
cursor.execute("UPDATE KIDS SET Rating = ? WHERE LocalID = ?",
str(winner_new_rating), winner_id)
cursor.execute("UPDATE KIDS SET Rating = ? WHERE LocalID = ?",
str(loser_new_rating), loser_id)
except DatabaseError as e:
print(str(e))