I am executing some sqls in loop . mostly the code works fine and completes the loop. But some time python fails with Traceback issue after 2-3 hours. When i checked the SQL execution in DB, it executed but python did not continue the next execution loop and got stuck. Throws the error after 2 hours
Executing SQL :
sql = f.read()
cursor.execute(sql)
Error :
Traceback (most recent call last):
...
...
File "/home/lib/python2.7/site-packages/pgdb.py", line 480, in __exit__
self.rollback()
File "/home/lib/python2.7/site-packages/pgdb.py", line 519, in rollback
raise _op_error("can't rollback")
pg.OperationalError: can't rollback
Here is the full code in loop:
dt = new_execution_timestamp_for(job, dependencies, cursor)
if dt is not None:
logging.info("Running job %s", job)
with open(os.path.join(os.environ['ROOT'],
"share", job + ".sql")) as f:
sql = f.read()
if weekly is None or weekly == date.today().weekday():
if loops is not None:
loop_items = fetch_list("loops", loops, cursor)
timestamp = fetch_job_timestamp(job, cursor)
for item in loop_items:
cursor.execute(sql, {loops: item,
'last_executed': timestamp})
else:
cursor.execute(sql)
cursor.execute(UPDATE_TIMESTAMP, (dt, job))
else:
logging.info("Skipping job %s", job)
Related
I am learning to use the MySQL Connector/Python. When I insert multiple rows together into the table using the code below, it shows an error. But it works when I insert only one row at at time.
Here's the python code I have written:
import mysql.connector as sql
mydb = sql.connect(host = "localhost", user = "root", passwd = "your_password", database = "testdb")
mycursor = mydb.cursor()
str = "insert into test (Name, Address) values (%s, %s)"
val = [
('Foster', 'Moonland'),
('Toblerone', 'Marstown'),
('Hershey', 'Neptunestadt'),
('Cadbury', 'Jupiter DC')
]
mycursor.execute(str, val)
mydb.commit()
This the error message:
Traceback (most recent call last):
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\conversion.py", line 183, in to_mysql
return getattr(self, "_{0}_to_mysql".format(type_name))(value)
AttributeError: 'MySQLConverter' object has no attribute '_tuple_to_mysql'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 432, in _process_params
res = [to_mysql(i) for i in res]
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 432, in <listcomp>
res = [to_mysql(i) for i in res]
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\conversion.py", line 186, in to_mysql
"MySQL type".format(type_name))
TypeError: Python 'tuple' cannot be converted to a MySQL type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "d:\coding\python\mysql_connector\connectSQL.py", line 24, in <module>
mycursor.execute(str, val)
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 557, in execute
psub = _ParamSubstitutor(self._process_params(params))
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 437, in _process_params
"Failed processing format-parameters; %s" % err)
mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'tuple' cannot be converted to a MySQL type
But if val is a single tuple, the code executes without producing any error.
Inserting multiple MySQL records using Python
Error: Python 'tuple' cannot be converted to a MySQL type
ERROR CODE:
Traceback (most recent call last):
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\conversion.py", line 181, in to_mysql
return getattr(self, "_{0}_to_mysql".format(type_name))(value)
AttributeError: 'MySQLConverter' object has no attribute '_tuple_to_mysql'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 432, in _process_params
res = [to_mysql(i) for i in res]
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 432, in <listcomp>
res = [to_mysql(i) for i in res]
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\conversion.py", line 184, in to_mysql
"MySQL type".format(type_name))
TypeError: Python 'tuple' cannot be converted to a MySQL type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "python_mysql_2.py", line 22, in <module>
my_cursor.execute(mike_placeholders,records_list)
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 557, in execute
psub = _ParamSubstitutor(self._process_params(params))
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 437, in _process_params
"Failed processing format-parameters; %s" % err)
mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'tuple' cannot be converted to a MySQL type
Python Code:
#import sql.connector
import mysql.connector
#Create connection, added db we created#
connection = mysql.connector.connect(
host='localhost',
user='root',
password='123',
database='testdb_1'
)
#Create cursor for the connection
my_cursor = connection.cursor()
#Create SQL statement with placeholders and put in variable
mike_placeholders="INSERT INTO users (name,email,age) VALUES (%s, %s, %s) "
#Create list (array) of records
records_list = [('Tim','Tim#tim.com',32), ('Mary','Mary#mary.com',40), ('Sam','Sam#sam.com',50), ('Fred','Fred#fred.com',22) ]
#Execute cursor, requires SQl statement variable, record variable
my_cursor.execute(mike_placeholders,records_list)
#Commit the connection to make the change on the database
connection.commit()
Ahhh, I used the wrong Python term.
I should have used executemany when working with a tuple.
my_cursor.executemany(mike_placeholders,records_list)
You can't pass a list to my_cursor.execute(), you need to iterate over the list:
for values in records_list:
my_cursor.execute(mike_placeholders, values)
Or you could repeat the (%s, %s, %s) multiple times and do it all in a single query by flattening the list of tuples.
mike_placeholders="INSERT INTO users (name,email,age) VALUES " + ", ".join(["(%s, %s, %s)"] * len(records_list))
my_cursor.execute(mike_placeholders, sum(records_list))
use my_cursor.executemany(mike_placeholders,records_list)
If you have multiple elements which are saved in a list or tuple then use,
cursor.executemany(query,list) or cursor.executemany(query,tuple)
You must use a for loop and INSERT item by item
for x in records_list:
my_cursor.execute(mike_placeholders, x)
With Python 3.6.2 and MySQL Connector 2.1.6 package on Windows 10, not calling the execute method of a database cursor, or calling it on a non SELECT statement (CREATE, DROP, ALTER, INSERT, DELETE, UPDATE, etc.) yields the following results:
>>> import mysql.connector
>>> session = mysql.connector.connect(user = "root", database = "mysql")
>>> cursor = session.cursor()
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
>>> cursor.execute("CREATE TABLE test (x INTEGER)")
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
PEP 249 explicitly states for the fetchone, fetchmany and fetchall methods:
An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.
So why don't fetchone and fetchmany raise an exception like fetchall?
I filed a bug report on bugs.mysql.com and the bug has been fixed in MySQL Connector/Python 8.0.23.
I am working on a raspberry pi project, in which I'm fetching data from plc and storing it into mysql database.
Here is my code:
import minimalmodbus
import serial
import mysql.connector
instrument = minimalmodbus.Instrument('/dev/ttyAMA0',3,mode='rtu')
instrument.serial.baudrate=115200
instrument.serial.parity = serial.PARITY_NONE
instrument.serial.bytesize = 8
instrument.serial.stopbits = 1
instrument.serial.timeout = 0.05
con = mysql.connector.connect(user='root',password='raspberry',host='localhost',
database='Fujiplc')
cursor = con.cursor()
try:
reg_value=instrument.read_register(102)
print reg_value
cursor.execute("insert into Register_Values values(%s)",(reg_value))
print ('One row inserted successfully.')
except IOError:
print("Failed to read from PLC.")
print (cursor.rowcount)
con.commit()
cursor.close()
con.close()
After running this code, I get next error:
Traceback (most recent call last):
File "/home/pi/rpi_to_plc_read.py", line 22, in <module>
cursor.execute("insert into Register_Values values(%d)",(reg_value))
File "/usr/local/lib/python2.7/dist-packages/mysql/connector/cursor.py", line 477, in execute
stmt = operation % self._process_params(params)
File "/usr/local/lib/python2.7/dist-packages/mysql/connector/cursor.py", line 355, in _process_params
"Failed processing format-parameters; %s" % err)
ProgrammingError: Failed processing format-parameters; argument 2 to map() must support iteration
I have gone through so many solutions but problem couldn't solve.
Please help me.
i think should be.
cursor.execute("insert into Register_Values values(%s)",(reg_value))
con.commit()
Pretty common error in python.
(reg_value) is not a tuple
(reg_value,) is a tuple
I have 2 problems with my procedure.
First problem is the procedure doesn't work from python and I don't know why.
Here is my python code:
db = MySQLdb.connect(host="***", # your host, usually localhost
user="***", # your username
passwd="***", # your password
db="***") # name of the data base
# you must create a Cursor object. It will let
# you execute all the queries you need
cursor = db.cursor()
args=('SugarSHape',10024,241)
cursor.callproc('p_add_fb_data', args)
My procedure:
CREATE DEFINER=`leverate`#`%` PROCEDURE `p_add_fb_data`(IN p_username VARCHAR(45),IN p_impressions INT(45),IN p_linkClicks INT(45))
BEGIN
DECLARE id INT(5);
SELECT id_user FROM t_user WHERE t_user.username=p_username;
INSERT INTO t_fb_data(Impressions,LinkClicks)
Values(p_impressions, p_linkClicks);
END
I know that id is useless, but I need it later.
If I start the procedure from workbench it works. But if my Python script starts I don't get a new line in my table.
My second problem is if I start the procedure some more time I got this error:
> Traceback (most recent call last): File "C:\Python27\Leverate.py",
> line 76, in <module>
> cursor.execute("SELECT * FROM t_user") File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 174, in
> execute
> self.errorhandler(self, exc, value) File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in
> defaulterrorhandler
> raise errorclass, errorvalue ProgrammingError: (2014, "Commands out of sync; you can't run this command now")
My code for this problem:
for z in range(0,5):
args=(Clients[z][1],data["results"][z]["impressions"],data["results"][z]["clicks"]);
cursor.callproc('p_add_fb_data', args)
I hope it's not too much and that someone can help me.