I have a python script which downloads certain data from the internet (from a list of URLS) and stores it in a MySQL database. My code is something along these lines:
try:
con = mdb.connect(host = 'localhost', user = 'root', passwd = 'gatech', db ='SpecialProb', charset = 'utf8');
cur = con.cursor()
print " Database Connection Parameters set"
except mdb.Error, e:
print "Error %d: %s" % (e.args[0],e.args[1])
sys.exit(1)
try:
for item in listURLs[index:]:
try:
r2 = requests.get(item)
except:
print "Could not fetch data"
print "Exception: ", sys.exc_info()[0]
continue
## Some code to fetch Name, ID from the internet, which I have tested and which works correctly. ##
try:
cur.execute("INSERT into TABLENAME (NAME, ID"
") VALUES (%s, %s)",
(totalName, ID))
print("Value inserted in database.\n")
countInsert += 1;
con.commit()
except mdb.IntegrityError:
print "Most likely cause: Database already contains this entry."
print "Message from system: "
print "Error %d: %s\n" % (e.args[0],e.args[1])
continue
except:
print "'For loop exception: ", sys.exc_info()[0]
sys.exit(0)
The data which I fetch could be duplicate and I don't want the duplicate data to be inserted into the database and the code should go over the next iteration etch the next data instead of storing the data. So I have the except mdb.IntegrityError: line there which takes care of the duplicates.
However, after catching the exception for the duplicate entry the code, instead of going over the next iteration goes to the except which I have set for the for loop.
Here's what I get:
Most likely cause: Database already contains this entry.
Message from system:
'For loop exception: <type 'exceptions.NameError'>
Why does this happen? And how do I prevent from happening this?
Thanks a lot!
Your e would not be defined causing exception in your except block.
Use mdb.IntegrityError,e like you did for except mdb.Error, e
Related
I'm new to Python and I'm trying to save the data obtained by the serial port from Arduino to MySQL database, but I can only save once since I have to run the program again to save once more, I tried to use a while loop but I persist on the database only once and keep getting the "Failed to get data from Arduino!".
Here's my code:
import serial
import MySQLdb
dbConn = MySQLdb.connect("localhost","root","sasa","sms") or die ("could not connect to database")
cursor = dbConn.cursor()
device = 'COM3'
try:
print "Trying...",device
arduino = serial.Serial(device, 9600)
except:
print "Failed to connect on",device
try:
data = arduino.readline()
pieces = data.split("\t")
try:
cursor.execute("INSERT INTO industrial (db) VALUES (%s)", (pieces[0]))
dbConn.commit()
cursor.close()
except MySQLdb.IntegrityError:
print "failed to insert data"
finally:
cursor.close()
except:
print "Failed to get data from Arduino!"
I've got it, it was really simple actually, i just put a while True: at the beginning, and it was updating correctly, here's the code if anyone is interested:
import serial
import MySQLdb
while True:
dbConn = MySQLdb.connect("localhost","root","sasa","sms") or die ("could not connect to database")
cursor = dbConn.cursor()
device = 'COM3'
try:
print "Trying...",device
arduino = serial.Serial(device, 9600)
except:
print "Failed to connect on",device
try:
data = arduino.readline()
pieces = data.split("\t")
try:
cursor.execute("INSERT INTO industrial (db) VALUES (%s)", pieces)
dbConn.commit()
cursor.close()
except MySQLdb.IntegrityError:
print "failed to insert data"
finally:
cursor.close()
except:
print "Failed to get data from Arduino!"
Here is a snippet from my code. For some reason it simply won't print out the second line saying "Cracking took 10 seconds" or whatever, but this first bit saying Password Found: does work... Why?
def connect(host, user, password, release):
global Found
global Fails
global startTime
try:
s = pxssh.pxssh()
s.login(host, user, password)
print '[+] Password Found: ' + password
print 'Cracking the password took' + datetime.now()-startTime + 'seconds.'
Found = True
except Exception, e:
if 'read_nonblocking' in str(e):
Fails += 1
time.sleep(5)
connect(host, user, password, False)
elif 'synchronize with original prompt' in str(e):
time.sleep(1)
connect(host, user, password, False)
You are trying to concatenate two different things (datetime and str), try converting the datetime to str as:
def connect(host, user, password, release):
global Found
global Fails
global startTime
try:
s = pxssh.pxssh()
s.login(host, user, password)
print '[+] Password Found: ' + password
print 'Cracking the password took' + str(datetime.now()-startTime) + 'seconds.'
Found = True
except Exception, e:
if 'read_nonblocking' in str(e):
Fails += 1
time.sleep(5)
connect(host, user, password, False)
elif 'synchronize with original prompt' in str(e):
time.sleep(1)
connect(host, user, password, False)
Moreover, you shouldn't trap all kind Exception, just those you need.
The issue is probably that you haven't set startTime, but you masked it by over-broad exception handling. Either remove the try/except, select some other exception to trap, or simply include a bare raise command in your exception handler and you should see a NameError because of the absence of initialization. Fix that and your code has more of a chance.
I am using
def mysql_handling(string):
global cursor
while True:
try:
cursor.execute(string)
if 'SELECT' not in string:
db.commit()
break
except MySQLdb.MySQLError:
cursor.close()
print 'something went wrong!!'
time.sleep(1)
cursor = get_cursor()
To retry the query if the connection fails, but I ONLY want to retry the connection when I have the error “Lost connection to MySQL server during query”. (else the mysql_handling function gets in an infinite loop)
So what should I use instead of 'except MySQLdb.MySQLError:' ?
From this page it would seem that you can't really catch an exception that is that specific but need to go as close as you can (OperationalError) and check errno for the exact error code;
except MySQLdb.OperationalError as err:
if err.errno == errorcode.CR_SERVER_LOST:
# This is the error you're looking for
else:
# This is not the error you're looking for
raise
instead of while True, you can try connecting and commiting again in except block.
def mysql_handling(string):
global cursor
try:
cursor.execute(string)
if 'SELECT' not in string:
db.commit()
except MySQLdb.MySQLError:
cursor.close()
print 'something went wrong!!'
time.sleep(1)
cursor = get_cursor()
cursor.execute(string)
if 'SELECT' not in string:
db.commit()
finally:
if cursor:
cursor.close()
or u can keep max number of retries say like 5.
I have a server which executes properly 9/10 times, but sometimes I get the error “Lost connection to MySQL server during query” and then the whole process stops/freezes.
I'm trying to let a MySQL query execute again if it fails using a function for this, unfortunately it seems that the code 'stops' or get stuck whenever I get an error. Am I doing something wrong?
Here string is something like: SELECT xx FROM xxx WHERE xxx
try:
db = MySQLdb.connect (host = "",
user = "",
passwd = "",
db = "" )
except MySQLdb.Error, e:
print("Error %d: %s" %(e.args[0], e.args[1]))
sys.exit(1);
cursor = db.cursor()
def mysql_handling(string):
while 1:
try:
cursor.execute(string)
if 'SELECT' not in string:
db.commit()
break
except:
mysql_error_tracking(string)
mysql_error_tracking is a function to monitor what queries fail most of the time (not relevant).
Add a timeout to your connection method:
db = MySQLdb.connect (host = HOST, user = USER, passwd = PASS, db = DB,
connect_timeout = TIMEOUT)
Do this in your except block so that you do a re-connect.
To make your life easier, place your connection code in a separate function so you can re-use it. Something like this (untested):
def get_cursor()
try:
db = MySQLdb.connect (host = HOST, user = USER, passwd = PASS, db = DB,
connect_timeout = TIMEOUT)
except MySQLdb.Error, e:
print("Error %d: %s" %(e.args[0], e.args[1]))
sys.exit(1);
return db.cursor()
def mysql_handling(cursor, string):
while True:
try:
cursor.execute(string)
if 'SELECT' not in string:
db.commit()
break
except MySQLdb.MySQLError:
cursor.close()
mysql_error_tracking(string)
cursor = get_cursor()
def main():
mysql_handling(get_cursor(), string)
I am new at python, currently I am working on a GPS tracker with that interacts with Google maps using an Arduino Uno. I am getting this error and it is not letting me run the .py script for my tcpServer this is the whole script.
#!/usr/bin/env python
import socket
import MySQLdb
TCP_IP = 'my machine IP'
TCP_PORT = 32000
BUFFER_SIZE = 40
# ClearDB. Deletes the entire tracking table
def ClearDB(curs,d ):
curs.execute ("""
INSERT INTO gmaptracker (lat, lon)
VALUES (0.0,0.0)""")
d.commit()
# Connect to the mySQL Database
def tServer():
try:
db = MySQLdb.connect (host = "my host",
user = "my user",
passwd = "my password",
db = "gmap" )
except MySQLdb.Error, e:
print "Error %d: %s" %(e.args[0], e.args[1])
sys.exit(1);
cursor = db.cursor()
# Start with a fresh tracking table
ClearDB(cursor,db)
# Set up listening Socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((TCP_IP, TCP_PORT))
print "Listening...."
s.listen(1)
conn, addr = s.accept()
print 'Accepted connection from address:', addr
except socket.error (message):
if s:
s.close()
print "Could not open socket: " + message
cursor.close()
conn.close()
db.close()
sys.exit(1)
try:
while 1:
data = conn.recv(BUFFER_SIZE)
if not data:break
str1,str2 = data.split("Long: ")
str1 = str1.split("Lat: ")[1]
latitude = float(str1)
longitude = float(str2)
cursor.execute ("""
INSERT INTO gmaptracker (lat, lon)
VALUES (%s,%s)""", (latitude,longitude))
db.commit()
except KeyboardInterrupt:
ClearDB(cursor,db);
cursor.close()
conn.close()
db.close()
if __name__ == '__main__':
tServer()
and this is the error that I am getting
Traceback (most recent call last):
File "tcpServer.py", line 79, in <module>
tServer()
File "tcpServer.py", line 48, in tServer
except socket.error(message):
NameError: global name 'message' is not defined
If anyone can help me figure this out I would greatly appreciate it, as I said I am new at python I am also running python 2.7 if that helps. Thanks in advance
You are not using the correct syntax for catching an exception. Instead, use:
except socket.error as serror:
message = serror.message
The socket.error exception has two extra attributes, errno and message. Older code used to catch it like this:
except socket.error, (value, message):
because in Python 2 you can treat an exception like a tuple and unpack it, but that's gone in Python 3 and should really not be used.
Moreover, the older except exceptiontype, targetvariable: has been replaced by the except exceptiontype as targetvariable: syntax as that is less ambiguous when you try to catch more than one exception type in the same statement.
When an exception is thrown, the normal flow of code is interrupted; instead the flow 'jumps' to the exception handler. Because of this jump, you have another problem in your code. In the exception handler you refer to conn.close(), but the variable conn is defined after the point where the socket exception will be thrown (the various socket operations). This will result in a NameError. In this case, there is no path through your code that'll result in conn being assigned an open socket connection, you can remove the conn.close() line altogether.
If there was a need to call .close() on conn, you'd need to detect if it was set in the first place. Set it to None, beforehand, then call .close() only if conn is no longer None:
conn = None
try:
# ... do stuff ...
conn, addr = s.accept()
# ... do more stuff
except socket.error as serror:
# test if `conn` was set
if conn is not None:
conn.close()