Develop Reference

Python sql connection pool not updating value

I have an issue which is related to connection pool but I don't understand it.
Below is my code and this is the behavior:
Starting with empty table, I do SELECT query for non-existing value (no results)
Then I do INSERT query, it successfully inserts the value
HOWEVER, after inserting a new value, if I try to do more SELECT statements it only works 2 out of 3 times, always fails exactly every 3rd try (with pool size=3. ie with pool size=10 it will work exactly 9 out of 10 times)
finally, if i restart the script, with the initial SELECT commented out (but the value is in table before script ones) I get the inserted value and it works every time.
Why does this code seem to 'get stuck returning empty result for the connection that had no result' until restarting the script?
(note that it keep opening and closing connections from connection pool because this is taken from a web application where each connect/close is a different web request. Here i cut the whole 'web' aspect out of it)
#!/usr/bin/python
import mysql.connector
dbvars = {'host':'h','user':'u','passwd':'p','db':'d'}
# db has 1 empty table 'test' with one varchar field 'id'
con = mysql.connector.connect(pool_name="mypool", pool_size=3, pool_reset_session=False, **dbvars)
cur = con.cursor()
cur.execute("SELECT id FROM test WHERE id = '123';")
result = cur.fetchall()
cur.close()
con.close()
con = mysql.connector.connect(pool_name="mypool")
cur = con.cursor()
cur.execute("INSERT INTO test VALUES ('123');")
con.commit()
cur.close()
con.close()
for i in range(12):
con = mysql.connector.connect(pool_name="mypool")
cur = con.cursor()
cur.execute("SELECT id FROM test WHERE id = '123';")
result = cur.fetchall()
cur.close()
con.close()
print result
The output of the above is:
[(u'123',)]
[]
[(u'123',)]
[(u'123',)]
[]
[(u'123',)]
[(u'123',)]
[]
[(u'123',)]
[(u'123',)]
[]
[(u'123',)]
Again, if I don't do the initial SELECT before the insert, then all of them return 123 (if it's already in db). It seems the initial SELECT 'corrupts' one of the connections of the connection pool. Further, if I do 2 SELECTs for empty results before the INSERT, then 2 of the 3 connections are 'corrupt'. Finally if I do 3 SELECTs before the insert, it still works 1 of 3 times, because it seems the INSERT 'fixes' the connection (presumably by having 'results').
Ubuntu 18.04
Python 2.7.17 (released Oct 2019)
mysql-connector-python 8.0.21 (June 2020)
MySql server 5.6.10

It seems to be a rather severe bug in the python driver for MySQL. Perhaps some configuration incompatibility but clearly a bug as no error is shown yet it returns wrong query results.
I filed the bug report with MySQL team and it's status is currently 'verified'.
https://bugs.mysql.com/bug.php?id=102053

"Insert into table values" does not send data

I am new in this and this is my first question. I hope you guys will help.
If my question format is wrong, feel free to comment on that also.
The code is pretty simple. I have DB connection, 2 functions - one for printing and another for choosing how many SQL queries I want to execute and input for those queries.
Idea is to enter a number(INT) of SQL queries - for example, 2 and then in another line user must enter 2 SQL queries.
After that, call_table function will print out current table status/situation/data.
For example - user wants to print out into console table data (table have 2 columns, [name][college], varchar type)
Insert a number of SQL queries you want to execute: 1
Insert SQL statement:
select * from student
('ivan', 'ino')
('nena', 'fer')
('tomislav', 'ino')
('marko', 'fer')
('tomislav', 'ino')
('marko', 'fer')
When I try to insert some values into the same table nothing happens with the table, data is not entered.
The query is 100% correct since I tested it in workbench, also I've tried to create another table from this program and the query was executed normally and the table was created.
I receive no errors.
Code is below:
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='123456', database='test')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container = [input("Insert SQL statement: \n").upper()]
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
call_table(mycursor)
What am I doing wrong?
I tried even more complicated SQL queries but insert into the table is not working.
Thank you

Everything is good with the code, you're just missing cursor.commit()
By default cursor commit is false in python for insert queries.
cursor.execute(y)
cursor.commit()
and if you're done with queries
db.close()

You should append the queries to the container variable
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='123456', database='test')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container.append(input("Insert SQL statement: \n").upper())
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
call_table(mycursor)

At the end of the program, I've added db.commit(), and everything works fine now.
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='45fa6cb2',
database='ivan')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container.append(input("Insert SQL statement: \n").upper())
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
db.commit()
call_table(mycursor)

Python MySQL cursor fails to fetch rows

I am trying to fetch data from AWS MariaDB:
cursor = self._cnx.cursor()
stmt = ('SELECT * FROM flights')
cursor.execute(stmt)
print(cursor.rowcount)
# prints 2
for z in cursor:
print(z)
# Does not iterate
row = cursor.fetchone()
# row is None
rows = cursor.fetchall()
# throws 'No result set to fetch from.'
I can verify that table contains data using MySQL Workbench. Am I missing some step?
EDIT: re 2 answers:
res = cursor.execute(stmt)
# res is None
EDIT:
I created new Python project with a single file:
import mysql.connector
try:
cnx = mysql.connector.connect(
host='foobar.rds.amazonaws.com',
user='devuser',
password='devpasswd',
database='devdb'
)
cursor = cnx.cursor()
#cursor = cnx.cursor(buffered=True)
cursor.execute('SELECT * FROM flights')
print(cursor.rowcount)
rows = cursor.fetchall()
except Exception as exc:
print(exc)
If I run this code with simple cursor, fetchall raises "No result set to fetch from". If I run with buffered cursor, I can see that _rows property of cursor contains my data, but fetchall() returns empty array.

Your issue is that cursor.execute(stmt) returns an object with results and you're not storing that.
results = cursor.execute(stmt)
print(results.fetchone()) # Prints out and pops first row

For the future googlers with the same Problem I found a workaround which may help in some cases:
I didn't find the source of the problem but a solution which worked for me.
In my case .fetchone() also returned none whatever I did on my local(on my own Computer) Database. I tried the exact same code with the Database on our companies server and somehow it worked. So I copied the complete server Database onto my local Database (by using database dumps) just to get the server settings and afterwards I also could get data from my local SQL-Server with the code which didn't work before.
I am a SQL-newbie but maybe some crazy setting on my local SQL-Server prevented me from fetching data. Maybe some more experienced SQL-user knows this setting and can explain.

retrieve updated table from database - MySQLdb

I am trying to retrieve all the data from a table using just one connection. Here is a pseudo code:
import MySQLdb
db = MySQLdb.connect(host,user,pass,db)
def some_fun():
global db
cur=db.cursor()
cur.execute("SELECT * FROM TableName")
result = cur.fetchall()
for i in result:
print i
cur.close()
while True:
some_fun()
This code doesn't work and every times gives the same records as a result even the content of the table has changed.
How can i achieve multiple query with just one single connection and not opening the connection every time i have to get the table contents.
Thanks

If your major concern is reusing a conection without closing and opening it repeatedly, you can do something like the following.
The cursor won't close, and every new query will fetch the updated data inside the table.
import MySQLdb
class Dbconnect(object):
def __init__(self):
self.dbconection = MySQLdb.connect(host='host', port=int('port'), user='user', passwd'passwd', db='schema_db')
self.dbcursor = self.dbconection.cursor()
db = Dbconnect()
# conection made before the infinite loop
while True:
db.dbcursor.execute()
# use an iterator to view results. This consumes the cursor
for row in db.dbcursor:
print row

select from insert into not working with sqlalchemy

I want to insert a record in mytable (in DB2 database) and get the id generated in that insert. I'm trying to do that with python 2.7. Here is what I did:
import sqlalchemy
from sqlalchemy import *
import ibm_db_sa
db2 = sqlalchemy.create_engine('ibm_db_sa://user:pswd#localhost:50001/mydatabase')
sql = "select REPORT_ID from FINAL TABLE(insert into MY_TABLE values(DEFAULT,CURRENT TIMESTAMP,EMPTY_BLOB(),10,'success'));"
result = db2.execute(sql)
for item in result:
id = item[0]
print id
When I execute the code above it gives me this output:
10 //or a increasing number
Now when I check in the database nothing has been inserted ! I tried to run the same SQL request on the command line and it worked just fine. Any clue why I can't insert it with python using sqlalchemy ?

Did you try a commit? #Lennart is right. It might solve your problem.
Your code does not commit the changes you have made and thus are rolled back.
If your Database is InnoDB, it is transactional and thus needs a commit.
according to this, you also have to connect to your engine. so in your instance it would look like:
db2 = sqlalchemy.create_engine('ibm_db_sa://user:pswd#localhost:50001/mydatabase')
conn = db2.connect()
trans = conn.begin()
try:
sql = "select REPORT_ID from FINAL TABLE(insert into MY_TABLE values(DEFAULT,CURRENT TIMESTAMP,EMPTY_BLOB(),10,'success'));"
result = conn.execute(sql)
for item in result:
id = item[0]
print id
trans.commit()
except:
trans.rollback()
raise
I do hope this helps.