Develop Reference

mysql.connector.errors.InternalError: Unread result found when trying to insert into remote database [duplicate]

I am inserting JSON data into a MySQL database
I am parsing the JSON and then inserting it into a MySQL db using the python connector
Through trial, I can see the error is associated with this piece of code
for steps in result['routes'][0]['legs'][0]['steps']:
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
if steps['travel_mode'] == "pub_tran":
travel_mode = steps['travel_mode']
Orig_lat = steps['var_1']['dep']['lat']
Orig_lng = steps['var_1']['dep']['lng']
Dest_lat = steps['var_1']['arr']['lat']
Dest_lng = steps['var_1']['arr']['lng']
time_stamp = leg['_sent_time_stamp']
if steps['travel_mode'] =="a_pied":
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
travel_mode = steps['travel_mode']
Orig_lat = steps['var_2']['lat']
Orig_lng = steps['var_2']['lng']
Dest_lat = steps['var_2']['lat']
Dest_lng = steps['var_2']['lng']
time_stamp = leg['_sent_time_stamp']
cursor.execute(query,(travel_mode, Orig_lat, Orig_lng, Dest_lat, Dest_lng, time_stamp))
leg_no = cursor.fetchone()[0]
print(leg_no)
I have inserted higher level details and am now searching the database to associate this lower level information with its parent. The only way to find this unique value is to search via the origin and destination coordinates with the time_stamp. I believe the logic is sound and by printing the leg_no immediately after this section, I can see values which appear at first inspection to be correct
However, when added to the rest of the code, it causes subsequent sections where more data is inserted using the cursor to fail with this error -
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
The issue seems similar to MySQL Unread Result with Python
Is the query too complex and needs splitting or is there another issue?
If the query is indeed too complex, can anyone advise how best to split this?
EDIT As per #Gord's help, Ive tried to dump any unread results
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng))
leg_no = cursor.fetchone()[0]
try:
cursor.fetchall()
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
pass
else:
raise
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng, time_stamp))
But, I still get
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
[Finished in 3.3s with exit code 1]
scratches head
EDIT 2 - when I print the ie.msg, I get -
No result set to fetch from

All that was required was for buffered to be set to true!
cursor = cnx.cursor(buffered=True)
The reason is that without a buffered cursor, the results are "lazily" loaded, meaning that "fetchone" actually only fetches one row from the full result set of the query. When you will use the same cursor again, it will complain that you still have n-1 results (where n is the result set amount) waiting to be fetched. However, when you use a buffered cursor the connector fetches ALL rows behind the scenes and you just take one from the connector so the mysql db won't complain.

I was able to recreate your issue. MySQL Connector/Python apparently doesn't like it if you retrieve multiple rows and don't fetch them all before closing the cursor or using it to retrieve some other stuff. For example
import mysql.connector
cnxn = mysql.connector.connect(
host='127.0.0.1',
user='root',
password='whatever',
database='mydb')
crsr = cnxn.cursor()
crsr.execute("DROP TABLE IF EXISTS pytest")
crsr.execute("""
CREATE TABLE pytest (
id INT(11) NOT NULL AUTO_INCREMENT,
firstname VARCHAR(20),
PRIMARY KEY (id)
)
""")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Gord')")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Anne')")
cnxn.commit()
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # InternalError: Unread result found.
If you only expect (or care about) one row then you can put a LIMIT on your query
crsr.execute("SELECT firstname FROM pytest LIMIT 0, 1")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # OK now
or you can use fetchall() to get rid of any unread results after you have finished working with the rows you retrieved.
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
try:
crsr.fetchall() # fetch (and discard) remaining rows
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
# no problem, we were just at the end of the result set
pass
else:
raise
crsr.execute("SELECT firstname FROM pytest") # OK now

cursor.reset() is really what you want.
fetchall() is not good because you may end up moving unnecessary data from the database to your client.

The problem is about the buffer, maybe you disconnected from the previous MySQL connection and now it cannot perform the next statement. There are two ways to give the buffer to the cursor. First, only to the particular cursor using the following command:
import mysql.connector
cnx = mysql.connector.connect()
# Only this particular cursor will buffer results
cursor = cnx.cursor(buffered=True)
Alternatively, you could enable buffer for any cursor you use:
import mysql.connector
# All cursors created from cnx2 will be buffered by default
cnx2 = mysql.connector.connect(buffered=True)
cursor = cnx.cursor()
In case you disconnected from MySQL, the latter works for you.
Enjoy coding

If you want to get only one result from a request, and want after to reuse the same connexion for other requests, limit your sql select request to 1 using "limit 1" at the end of your request.
ex "Select field from table where x=1 limit 1;"
This method is faster using "buffered=True"

Set the consume_results argument on the connect() method to True.
cnx = mysql.connector.connect(
host="localhost",
user="user",
password="password",
database="database",
consume_results=True
)
Now instead of throwing an exception, it basically does fetchall().
Unfortunately this still makes it slow, if you have a lot of unread rows.

There is also a possibility that your connection to MySQL Workbench is disconnected. Establish the connection again. This solved the problem for me.

cursor.reset()
and then create tables and load entries

Would setting the cursor within the for loop, executing it, and then closing it again in the loop help?
Like:
for steps in result['routes'][0]['legs'][0]['steps']:
cursor = cnx.cursor()
....
leg_no = cursor.fetchone()[0]
cursor.close()
print(leg_no)

Error executing cursor.execute when quotes are required

fairly new to SQL in general. I'm currently trying to bolster my general understanding of how to pass commands via cursor.execute(). I'm currently trying to grab a column from a table and rename it to something different.
import mysql.connector
user = 'root'
pw = 'test!*'
host = 'localhost'
db = 'test1'
conn = mysql.connector.connect(user=user, password=pw, host=host, database=db)
cursor = conn.cursor(prepared=True)
new_name = 'Company Name'
query = f'SELECT company_name AS {new_name} from company_directory'
cursor.execute(query)
fetch = cursor.fetchall()
I've also tried it like this:
query = 'SELECT company_name AS %s from company_directory'
cursor.execute(query, ('Company Name'),)
fetch = cursor.fetchall()
but that returns the following error:
stmt = self._cmysql.stmt_prepare(statement)
_mysql_connector.MySQLInterfaceError: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '? from company_directory' at line 1
I'm using python and mySQL. I keep reading about database injection and not using string concatenation but every time I try to use %s I get an error similar to the one below where. I've tried switching to ? syntax but i get the same error.
If someone could ELI5 what the difference is and what exactly database injection is and if what I'm doing in the first attempt qualifies as string concatenation that I should be trying to avoid.
Thank you so much!

If a column name or alias contains spaces, you need to put it in backticks.
query = f'SELECT company_name AS `{new_name}` from company_directory'
You can't use a placeholder for identifiers like table and column names or aliases, only where expressions are allowed.

You can't make a query parameter in place of a column alias. The rules for column aliases are the same as column identifiers, and they must be fixed in the query before you pass the query string.
So you could do this:
query = f"SELECT company_name AS `{'Company Name'}` from company_directory'
cursor.execute(query)

Is commit used on the connect or cursor?

The book named "Practical Programming: 2nd Edition" has conflicting code. This is the start of my code:
import sqlite3
con = sqlite3.connect('stackoverflow.db')
cur = conn.cursor()
To commit, would I use con.commit(), cur.commit() or are there different times to use each? From the book:
con.commit() :
cur.commit() :
Documentation shows con.commit() :

I took unutbu's advice and tried it myself.
Sample code:
import sqlite3
con = sqlite3.connect('db.db')
cur = con.cursor()
data = [('data', 3), ('data2', 69)]
cur.execute('CREATE TABLE Density(Name TEXT, Number INTEGER)')
for i in data:
cur.execute('INSERT INTO Density VALUES (?, ?)', (i[0], i[1]))
cur.commit()
PyCharm Run:
Traceback (most recent call last):
File "/Users/User/Library/Preferences/PyCharmCE2018.1/scratches/scratch_2.py", line 13, in <module>
cur.commit()
AttributeError: 'sqlite3.Cursor' object has no attribute 'commit'
Error in textbook. cur.commit() does not exist.
Thanks unutbu and s3n0

con.commit() and conn.commit() are the same ... they are created object types ... in both cases they are otherwise named ... important is mainly .commit() and not the naming that the programmer has specified
There are object types that use a different name (con and cur - as you asked) to calling the method. You can also use a different name in your code, for example:
db = sqlite3.connect('/tmp/filename.db')
cursor = db.cursor()
cursor.execute("CREATE TABLE ....
.... some DB-API 2.0 commands ....
")
db.commit()
Please check again the webpage https://docs.python.org/3/library/sqlite3.html .
You forgot to copy these two lines from the webpage:
import sqlite3
conn = sqlite3.connect('example.db')
And then continuing the code (just copied it):
c = conn.cursor()
# Create table
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, qty real, price real)''')
# Insert a row of data
c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
# Save (commit) the changes
conn.commit()
# We can also close the connection if we are done with it.
# Just be sure any changes have been committed or they will be lost.
conn.close()

I think if you're using a specified cursor to commit changes, in your case, it should be cur.connection.commit().
You can always use connect to commit in the end of your code, whether it's named db, or con or conn.
But when your code gets complicated, you'll have different function to do certain operation to the database, if you only use connection commit, when there is a bug, you gonna have a hard time to find which function failed. So you create specific cursor for specific operation, when that failed, the traceback message will show you which specific cursor when wrong.

To #s3n0 & #DanielYu's point they can be handled two different ways. I had to list these out to better understand the overlap:
Connection Objects
backup
close
commit
create_aggregate
create_collation
create_function
cursor
enable_load_extension
execute
executemany
executescript
in_transaction
interrupt
isolation_level
iterdump
load_extension
rollback
row_factory
set_authorizer
set_progress_handler
set_trace_callback
text_factory
total_changes
Cursor objects
arraysize
close
connection
description
execute
executemany
executescript
fetchall
fetchmany
fetchone
lastrowid
rowcount
setinputsizes
setoutputsize

Insert into with condition

I'm working with python and using pymysql library and i want to write a query that insert an array in a line where a column has some special value.
For example insert 'hi' into a column where user_id is 22
for that query i write this code
from pymysql import *
chat_id = 22
user_first_name = "hi"
db = connect(host="localhost", port=3306, user="root", passwd="",
db='support',charset='utf8')
cursor = db.cursor()
cursor.execute("""INSERT INTO users user_firstname VALUE %s WHERE user_id is
%s""",(user_first_name, chat_id))
db.commit()
how should i write this query in correct form?

If I'm undertanding, correctly, rather than an INSERT INTO, it seems you need an UPDATE:
cursor.execute("""UPDATE users SET user_firstname='%s' WHERE user_id=%s""",(user_first_name, chat_id))

Francisco is right though. If you have a user_id already, then an UPDATE should be used to change the value of and existing record. The INSERT command, creates a new record.

Confirmation that a postgres 'update' query worked in python

I've written my first 'update' query in python, while it seems correct, I'm not sure how to receive back the output to confirm it worked..
This is supposed to load a CSV file and replace the values in the first column with those in the second:
def main():
try:
conn=psycopg2.connect("dbname='subs' user='subs' host='localhost' password=''")
except:
print "I am unable to connect to the database."
sys.exit()
with open("dne.txt", "r+") as f:
for line in f:
old = line.split(',')[0].strip()
new = line.split(',')[1].strip()
cur = conn.cursor()
cur.execute("UPDATE master_list SET subs = '{0}' WHERE subs = '{1}';".format(new, old))
conn.commit()
results = cur.fetchall()
for each in results:
print str(each)
if __name__=="__main__":
main()
I thought the results (UPDATE 1 for each change?) would come back as a tuple, but I got an error instead:
psycopg2.ProgrammingError: no results to fetch
I'm not sure if this means my query just didn't work and there were no updates, or if I can't use fetchall() like I'm trying to.
Any feedback or suggestions welcome!

The UPDATE statement won't return any values as you are asking the database to update its data not to retrieve any data.
By far the best way to get the number of rows updated is to use cur.rowcount. This works with other drivers too, like with Psycopg2 for Postgresql it's the same syntax.
cur.execute("UPDATE master SET sub = ('xyz') WHERE sub = 'abc'")
print(cur.rowcount)
A more roundabout way of checking the update is by running a SELECT against the table after updating it; you should get the data returned. In my example below the first SELECT will return the row(s) where the update will happen. The second SELECT after the update should then return no rows as you have already updated all fields. The third SELECT should return the rows you have updated, plus any that already existed with the 'xyz' value.
import sqlite3
def main():
try:
conn=sqlite3.connect(":memory:")
cur = conn.cursor()
cur.execute("create table master(id text, sub text)")
cur.execute("insert into master(id, sub) values ('1', 'abc')")
cur.execute("insert into master(id, sub) values ('2', 'def')")
cur.execute("insert into master(id, sub) values ('3', 'ghi')")
conn.commit()
except:
print("I am unable to connect to the database.")
sys.exit()
cur.execute("select id, sub from master where sub='abc'")
print(cur.fetchall())
cur.execute("UPDATE master SET sub = ('xyz') WHERE sub = 'abc'")
conn.commit()
cur.execute("select id, sub from master where sub='abc'")
print(cur.fetchall())
cur.execute("select id, sub from master where sub='xyz'")
print(cur.fetchall())
if __name__=="__main__":
main()

In PostgreSQL 9.5 or later you can add RETURNING * to end your query that then returns the modified rows.
PostgreSQL docs: https://www.postgresql.org/docs/9.5/dml-returning.html
Sometimes it is useful to obtain data from modified rows while they
are being manipulated. The INSERT, UPDATE, and DELETE commands all
have an optional RETURNING clause that supports this. Use of RETURNING
avoids performing an extra database query to collect the data, and is
especially valuable when it would otherwise be difficult to identify
the modified rows reliably.