I'm writing a python script to monitor a few 1wire sensors off of a Raspberry Pi and store the results in a MySQL database.
Using the MySQL Connector/Python library I can successfully connect to the database, and run the query however the transaction doesn't seem to fully commit. I know the query runs successfully since the out param is set to the new auto-incremented ID.
CREATE TABLE `lamp`.`sensors` (
`SensorID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`SensorSerial` char(15) NOT NULL,
`SensorFamily` tinyint(4) NOT NULL,
PRIMARY KEY (`SensorID`),
UNIQUE KEY `SensorID_UNIQUE` (`SensorID`),
UNIQUE KEY `SensorSerial_UNIQUE` (`SensorSerial`)
)
CREATE PROCEDURE `lamp`.`AddSensor` (sensorSerial char(15),
sensorFamily tinyint, out returnValue int)
BEGIN
INSERT INTO sensors (SensorSerial,SensorFamily) VALUES (sensorSerial,sensorFamily);
SET returnValue=LAST_INSERT_ID();
END
However when I attempt to query the table (Select * from sensors) I get 0 results. If I run the procedure from the MySQL Workbench or from a .Net application everything works as expected. Which means I'm missing something when it comes to the Connector/Python, but I have no clue what. I'm extremely baffled since the auto-increment value does increase but no records are added. There are also no errors reported
def Test(self):
#this line works fine
#self.RunProcedure("INSERT INTO sensors (SensorSerial,SensorFamily) VALUES ('{0}',{1})".format(self.ID,self.Family),False,())
#this line does not?
args=self.RunProcedure('AddSensor',True,(self.ID,self.Family,-1))
if args[2]>=1:
logging.debug("Successfully added sensor data '{1}' for sensor '{0}' to the database".format(self.ID,value))
return True
else:
logging.critical("Failed to add Data to Database for unknown reason. SensorID: {0} Type: {1} Data:{2}".format(self.ID,self.Family,value))
def RunProcedure(self,proc,isStored,args):
try:
logging.debug("Attempting to connect to database.")
connection=mysql.connector.connect(user='root',password='1q2w3e4r',host='localhost',database='LAMP')
except mysql.connector.Error as e:
logging.exception("Failed to connect to mysql database.\r\n {0}".format(e))
else:
logging.debug("Successfully connected to database.")
try:
cursor=connection.cursor()
if isStored:
args = cursor.callproc(proc,args)
return args
else:
cursor.execute(proc,args)
#these do not seem to solve the issue.
#cursor.execute("Commit;")
#connection.commit()
except mysql.connector.Error as e:
logging.exception("Exception while running the command '{0}' with the args of '{1}' exception is {2}".format(proc,args,e))
finally:
logging.debug("Closing connection to database")
cursor.close()
connection.close()
Output from to the log looks like this;
2013-06-09 13:21:25,662 Attempting to connect to database.
2013-06-09 13:21:25,704 Successfully connected to database.
2013-06-09 13:21:25,720 Closing connection to database
2013-06-09 13:21:25,723 Successfully added sensor data '22.25' for sensor '10.85FDA8020800' to the database
**Edit
Not sure why but adding autocommit=True to the connection.open params seems to have resolved the issue. Since it was a problem with committing why didn't connection.commit() or the cursor.execute('commit;') correct the issue?
The problem is actually in your code:
..
try:
cursor=connection.cursor()
if isStored:
args = cursor.callproc(proc,args)
return args
else:
cursor.execute(proc,args)
connection.commit()
except mysql.connector.Error as e:
..
If you are using the Stored Routine, you are immediately returning, so commit() will never be called.
Related
I have connected to database in python successfully but it is not saving the data I provide.Here is the code :
try:
connection = MySQLdb.connect(host="localhost",user="root",passwd="",db="traffic_alert")
except Exception as e:
print('Not connected')
cursor = connection.cursor()
cursor.execute("insert into data (main_status,extracted_info,time) values (%s,%s,%s)",[sentence,final_result,time])
print(sentence)
print(final_result)
print(time)
it can successfully print sentence, final_result and time but not saving them in database.sentence,final_result and time are less than 50 characters.I have created the database manually in mysql with these rows :
id -> auto_increment,pk
main_status -> varchar(255)
extracted_info -> varchar(255)
seen -> int(10),Null=True
time ->varchar(255)
connection.commit()
After
cursor.execute
I am writing and reading from a sql database with tries & excepts. The thought behind the try/except is if for some reason the internet is down or we cannot connect to the server, we will write the sql transactions locally to a text file and then use those statements to update the table. That being said - the try and except only seems to work if there is a connection to the server. We have a table BAR in the DB database on server FOO:
try:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=FOO;DATABASE=DB;UID=user;PWD=password')
cursor = conn.cursor()
cursor.execute("UPDATE BAR SET Date = '"+time+"' WHERE ID = "+ID)
conn.commit()
except:
f = open("vistorlog.txt", "a")
f.write("UPDATE BAR SET Date = '"+time+"' WHERE ID = "+ID+"\n")
f.close()
the only instance where this try&except works is when there is an issue with the sql statement i.e. "Update BARS..." fails because there is no table named BARS. If I change the server to FOOS (or in a real life scenario unplug the ethernet cord and leave the table/serve names legitimate) the try and except doesn't work - the program freezes with no error.
I have a code like this:
import mysql.connector as mysql
from generate_records import generateRecords
devicesQuery = "CALL iot.sp_sensors_overview()"
try:
db = mysql.connect(
user = "username",
password = "password",
host = "hostname",
database="iot"
)
cursor = db.cursor(dictionary=True, buffered=True)
cursor.execute(devicesQuery)
for sensor in cursor:
generateRecords(sensor, db)
cursor.close()
except mysql.connector.Error as error:
print("Error:")
print(error)
else:
db.close()
The purpose of generateRecords function is obviously to generate records and run the INSERT query against the different table.
Seems like I do something wrong, because no matter what I trying, I getting different errors here, like mysql.connector.errors.OperationalError: MySQL Connection not available..
(upd) I also tried to change the code like it was suggested (see example bellow), with no luck - I still receiving the MySQL connection not available. error.
rows = cursor.fetchall()
cursor.close()
for sensor in rows:
cursor2 = db.cursor()
generateRecords(sensor, cursor2)
So, should I create a new connection within generateRecords function, or pass something different within it, or use some kind of different approach here?
Thank you!
Finally I found what was wrong. I'm used the query to call the stored procedure. Using the cursor.callproc("sp_sensors_overview") instead fixed my issue, and now I'm able to create the next cursor without errors.
i am making a script with python were every time it runs starts fresh, there for i want to make sure i delete the database and table and recreate them.
i was following this tutorial: https://www.fullstackpython.com/blog/postgresql-python-3-psycopg2-ubuntu-1604.html
but instead matt as a user a did osboxes, but when i try to create the table i get the error that already exists, how can that be if i previously dropped the database?
# connect to postgres
connection = psycopg2.connect(dbname="postgres", user="osboxes", password="osboxes.org", host="localhost")
connection.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
cursor = connection.cursor()
# create database to work on
name_database = "image_comparation"
cursor.execute("DROP DATABASE IF EXISTS {0};".format(name_database))
cursor.execute("CREATE DATABASE {0};".format(name_database))
name_table = "IMAGES"
cursor.execute("DROP TABLE IF EXISTS {0};".format(name_table))
# here is the issue, never creates the table
try:
# cursor.execute("DROP TABLE IF EXISTS {0};".format(name_table))
create_table_query = """CREATE TABLE {0} (IMAGE_PATH TEXT NOT NULL, IN_PROGRESS BOOLEAN, COMPLETED BOOLEAN, TIME_PROCESS REAL, CREATED_TIME timestamp, MATCHES TEXT NULL, MOVED BOOLEAN);""".format(name_table)
cursor.execute(create_table_query)
connection.commit()
except psycopg2.DatabaseError as e:
print(f'Error {e}')
Error relation "images" already exists
the first time i run the scripts works after that tell me tha relation ship allready exist so i asume that the table some how persisted, so i checked in the cmd
psql (12.3 (Ubuntu 12.3-1.pgdg18.04+1), server 12.2 (Ubuntu 12.2-4))
Type "help" for help.
image_db=> \dt
Did not find any relations.
image_db=>
then i manually copy the create table query into the psql and does work fine. i am missing something?
you are logging into the default database 'postgres', think as the master database that hold information about every thing else that is happening in postgre,
then you create a new database,
but you need to create another connection to that that base to switch to it, right now you are creating a table in the 'master' database,
add
connection = psycopg2.connect(dbname="image_comparation", user="osboxes", password="osboxes.org", host="localhost")
to switch to that database and then create the tale.
I'm still using Flask-mysql.
I'm getting the database context (the mysql variable) just fine, and can query on the database / get results. It's only the insert that is not working: it's not complaining (throwing Exceptions). It returns True from the insert method.
This should be done inserting the record when it commits, but for some reason, as I watch the MySQL database with MySQL Workbench, nothing is getting inserted into the table (and it's not throwing exceptions from the insert method):
I'm passing in this to insertCmd:
"INSERT into user(username, password) VALUES ('test1','somepassword');"
I've checked the length of the column in the database, and copied the command into MySQL Workbench (where it successfully inserts the row into the table).
I'm at a loss. The examples I've seen all seem to follow this format, and I have a good database context. You can see other things I've tried in the comments.
def insert(mysql, insertCmd):
try:
#connection = mysql.get_db()
cursor = mysql.connect().cursor()
cursor.execute(insertCmd)
mysql.connect().commit()
#mysql.connect().commit
#connection.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
You need to keep a handle to the connection; you keep overriding it in your loop.
Here is a simplified example:
con = mysql.connect()
cursor = con.cursor()
def insert(mysql, insertCmd):
try:
cursor.execute(insertCmd)
con.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
If mysql is your connection, then you can just commit on that, directly:
def insert(mysql, insertCmd):
try:
cursor = mysql.cursor()
cursor.execute(insertCmd)
mysql.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
return False
Apparently, you MUST separate the connect and cursor, or it won't work.
To get the cursor, this will work: cursor = mysql.connect().cursor()
However, as Burchan Khalid so adeptly pointed out, any attempt after that to make a connection object in order to commit will wipe out the work you did using the cursor.
So, you have to do the following (no shortcuts):
connection = mysql.connect()
cursor = connection.cursor()
cursor.execute(insertCmd)
connection.commit()