I'm still using Flask-mysql.
I'm getting the database context (the mysql variable) just fine, and can query on the database / get results. It's only the insert that is not working: it's not complaining (throwing Exceptions). It returns True from the insert method.
This should be done inserting the record when it commits, but for some reason, as I watch the MySQL database with MySQL Workbench, nothing is getting inserted into the table (and it's not throwing exceptions from the insert method):
I'm passing in this to insertCmd:
"INSERT into user(username, password) VALUES ('test1','somepassword');"
I've checked the length of the column in the database, and copied the command into MySQL Workbench (where it successfully inserts the row into the table).
I'm at a loss. The examples I've seen all seem to follow this format, and I have a good database context. You can see other things I've tried in the comments.
def insert(mysql, insertCmd):
try:
#connection = mysql.get_db()
cursor = mysql.connect().cursor()
cursor.execute(insertCmd)
mysql.connect().commit()
#mysql.connect().commit
#connection.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
You need to keep a handle to the connection; you keep overriding it in your loop.
Here is a simplified example:
con = mysql.connect()
cursor = con.cursor()
def insert(mysql, insertCmd):
try:
cursor.execute(insertCmd)
con.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
If mysql is your connection, then you can just commit on that, directly:
def insert(mysql, insertCmd):
try:
cursor = mysql.cursor()
cursor.execute(insertCmd)
mysql.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
return False
Apparently, you MUST separate the connect and cursor, or it won't work.
To get the cursor, this will work: cursor = mysql.connect().cursor()
However, as Burchan Khalid so adeptly pointed out, any attempt after that to make a connection object in order to commit will wipe out the work you did using the cursor.
So, you have to do the following (no shortcuts):
connection = mysql.connect()
cursor = connection.cursor()
cursor.execute(insertCmd)
connection.commit()
Related
I have an SQLite3 database that I want to add to with python, this is the code i have to add a row
def create_connection(db_file):
""" create a database connection to a SQLite database """
conn = None
try:
conn = sqlite3.connect(db_file)
return conn
except Error as e:
print(e)
def add_password(conn, data):
"""
Create an entry into the password database
"""
try:
sql = 'INSERT INTO passwords(added,username,password,website,email) VALUES(?,?,?,?,?)'
cur = conn.cursor()
cur.execute(sql, data)
print('done')
return cur.lastrowid
except Error as e:
print(e)
connection = create_connection('passwords.db')
data = (datetime.now(), 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
add_password(connection, data)
When I run it prints done and ends, there are no errors. However, when I open the database to view the table, it has no entries.
If I open the database and run the same SQL code
INSERT INTO passwords(added,username,password,website,email)
VALUES('13-5-2020', 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
it adds to the table. So it must be a problem with my python code. How do I get it to add?
Just make conn.commit() after executing query. It should work
I am trying to create a login function. But it only works ones. Ex- When i give a wrong userid and password I got correct error massage that "Could't login" after canceling that message and giving correct userid and password then I get "pymysql.err.Error: Already closed" below are the sample code.
import pymysql
# Connect to the database
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
class LoginModel:
def check_user(self, data):
try:
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `username` FROM `users` WHERE `username`=%s"
cursor.execute(sql, (data.username))
user = cursor.fetchone()
print(user)
if user:
if (user, data.password):
return user
else:
return False
else:
return False
finally:
connection.close()
You have a mismatch with respect to the number of times you're creating the connection (once) and the number of times you're closing the connection (once per login attempt).
One fix would be to move your:
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
into your def check__user(). It would work because you'd create and close the connection on each invocation (as others have pointed out, the finally clause always gets executed.)
That's not a great design because getting database connections tends to be relatively expensive. So keeping the connection creating outside of the method is preferred.... which means you must remove the connection.close() within the method.
I think you're mixing up connection.close() with cursor.close(). You want to do the latter, not the former. In your example you don't have to explicitly close the cursor because that happens automatically with your with connection.cursor() as cursor: line.
Change finally to except, or remove the try block completely.
This is the culprit code:
finally:
connection.close()
Per the docs:
"A finally clause is always executed before leaving the try statement, whether an exception has occurred or not"
From: https://docs.python.org/2/tutorial/errors.html
You didn't describe alternative behavior for what you would like to see happen instead of this, but my answer addresses the crux of your question.
Had the same issue. The "Finally clause is needed for Postgres with the psycopg2 driver, if used with context manager (with clause), it close the cursor but not the connection. The same does not apply with Pymysql.
I've seen some answers around here that open a new MySQL cursor before each query, then close it.
Is that slow? Shouldn't I be recycling a cursor, by passing it in as a parameter?
I have a program that does an infinite loop, so eventually the connection will time out after the default 8 hours.
Edit:
As requested, this is the relevant code that handles the SQL query:
def fetch_data(query):
try:
cursor = db.Cursor()
cursor.execute(query)
return cursor.fetchall()
except OperationalError as e:
db = fetchDb()
db.autocommit(True)
print 'reconnecting and trying again...'
return fetch_data(query)
Of course, re-connecting a connection for thousands of times will take much more time. You'd better set it as a property of your class, like this:
class yourClass():
self.db = ...
self.cursor = self.con.Cursor()
# do something
def fetch_data(self, query):
try:
if self.cursor:
self.cursor.execute(query)
else:
raise OperationalError
return self.cursor.fetchall()
except OperationalError as e:
self.db = fetchDb()
self.db.autocommit(True)
print 'reconnecting and trying again...'
return fetch_data(query)
I am accessing a MySQL database from python via MySQLdb library. I am attempting to test the database connection as shown below.
db = MySQLdb.connect(self.server, self.user,
self.passwd, self.schema)
cursor = db.cursor()
try:
cursor.execute("SELECT VERSION()")
results = cursor.fetchone()
ver = results[0]
if (ver is None):
return False
else:
return True
except:
print "ERROR IN CONNECTION"
return False
Is this the right way one should test the connectivity when writing unit testcases? If there is a better way, please enlighten!
I could be wrong (or misinterpreting your question :) ), but I believe that a connection-related exception gets thrown on MySQLdb.connect(). With MySQLdb, the exception to catch is MySQLdb.Error. Therefore, I would suggest moving the db setup inside of the try block and catching the proper exception (MySQLdb.Error). Also, as #JohnMee mentions, fetchone() will return None if there are no results, so this should work:
try:
db = MySQLdb.connect(self.server, self.user, self.passwd, self.schema)
cursor = db.cursor()
cursor.execute("SELECT VERSION()")
results = cursor.fetchone()
# Check if anything at all is returned
if results:
return True
else:
return False
except MySQLdb.Error:
print "ERROR IN CONNECTION"
return False
If you don't care about the connection and are just looking to test the execution of the query, I guess you could leave the connection setup outside the try but include MySQLdb.Error in your exception, perhaps as follows:
db = MySQLdb.connect(self.server, self.user, self.passwd, self.schema)
cursor = db.cursor()
try:
cursor.execute("SELECT VERSION()")
results = cursor.fetchone()
# Check if anything at all is returned
if results:
return True
else:
return False
except MySQLdb.Error, e:
print "ERROR %d IN CONNECTION: %s" % (e.args[0], e.args[1])
return False
This would at least give you a more detailed reason of why it failed.
Yes. Looks good to me.
My personal preferences:
actually throw an exception if no connection
you only need to fetchone, the test for None is superfluous (unless you're keen to enforce a minimum version of the database)
I have built a site using Django and I am receiving this annoying error when I am trying to execute a query.
If I restart the Apache server, the error will go away for a short time.
Traceback:
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/home/fran/cron/views/set_caches.py" in set_caches
24. cursor.execute(query, [category['id']])
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
15. return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py" in execute
86. return self.cursor.execute(query, args)
File "build/bdist.linux-i686/egg/MySQLdb/cursors.py" in execute
155. charset = db.character_set_name()
Exception Type: InterfaceError at /blablabla/
Exception Value: (0, '')
This is caused by a global cursor. Try creating and closing the cursor within each method a raw query is needed.
cursor = connection.cursor()
cursor.execute(query)
cursor.close()
You get this error when you have a db.close() call and later try to access the database without creating a new connection. Try to find if you close the connection to the database when you don't mean to.
I agreed with Moberg. This error is caused when we try to access the database after we have closed the connection. This could be caused by some wrong indentation in the code. Below is my code.
conn = connect()
cur = conn.cursor()
tk = get_tickers(cur)
for t in tk:
prices = read_price(t, cur)
if prices != None:
update_price(t, cur)
print 'Price after update of ticker ', t, ':'
p_open, p_high, p_low, p_close = read_price(t, cur)
print p_open, p_high, p_low, p_close
else:
print 'Price for ', t, ' is not available'
conn.close()
I got the same error as reported by Marian. After dedenting conn.close(), everything worked well. Confirmed that global conn is not an issue.
I had the same problem as for April of 2019 using python 3.7 and Mysql 2.7.
At intermittent intervals, the string (0, '') would be added at random to my SQL statements causing errors. I have solved the issue by commenting on the closing of the database connection and just leaving the closing of the cursors across my code.
def set_db():
db = pymysql.connect(host='localhost',
user="root",
passwd="root",
db="DATABASE")
return db
def execute_sql(cnx, sql_clause, fetch_all):
if sql_clause and sql_clause is not None:
try:
cnx.execute(sql_clause)
except Exception as e:
print("Error in sql: " + sql_clause + str(e))
return 0
pass
if fetch_all:
result = cnx.fetchall()
else:
result = cnx.fetchone()
return result
else:
print("Empty sql.")
return 0
db = set_db()
cnx = db.cursor()
sql = "SELECT * FROM TABLE"
result = execute_sql(cnx, sql, 1)
cnx.close() #close the cursor
#db.close #do not close the db connection
...
I had the same issue using threading with Python3 and Pymysql. I was getting deadlocks and then I would get hit with InterfaceError (0, '').
My issue was that I was trying to do a rollback on exception of the query- I believe this rollback was trying to use a connection that no longer existed and it was giving me the interface error. I took this rollback out (because I am OK with not doing rollback for this query) and I just let things go. This fixed my issue.
def delete_q_msg(self, assetid, queuemsgtypeid, msgid):
"""
Given the paramerts below remove items from the msg queue equal to or older than this.
If appropriate send them into a history table to be processed later
:param assetid:
:param queuemsgtypeid:
:param msgid:
:return:
"""
params = (assetid, queuemsgtypeid, msgid,)
db_connection = self._connect_to_db()
sp_sql = "{db}.ps_delete_q_msg".format(db=self._db_settings["database"])
return_value = []
try:
with db_connection.cursor() as cursor:
cursor.callproc(sp_sql, params)
return_value = cursor.fetchall()
db_connection.commit()
except Exception as ex:
# i think we dont want rollback here
# db_connection.rollback()
raise Exception(ex)
finally:
db_connection.close()
return return_value
I can confirm this is caused by a global cursor which is then later used in some functions. My symptoms were the exact same: intermittent interface errors that would temporarily be cleared up by an apache restart.
from django.db import connection
cursor = connection.cursor() # BAD
def foo():
cursor.execute('select * from bar')
But, I am using Django on top of Oracle 11.2 so I do not believe this is a bug in the MySQL/python driver. This is probably due to the caching done by apache/mod_wsgi.
I had the same issue with Flask+pymysql, I was getting an empty tuple as a result in the except: block, something like this "(\"(0, '')\",)" to be specific.
It turned out that the connection was getting closed and later the code tried accessing it which resulted into this error.
So I solved it by referring to above solutions and used a function for connection which assured me a conn every time I had to access the db.
You can recreate this issue by inserting conn.close() just before accessing the cursor.
For reference I used this site which helped me solve this issue.
https://hackersandslackers.com/python-mysql-pymysql/
For me, removing the conn.close() from my function worked. I was trying to access the database again after closing.
I am using flask with AWS.
Also you can try to restart your flask application if it has been running for a long time & if you are also using AWS RDS with MYSQL workbench like in my case, then just check whether your session is expired or not and update the access key and id.
Hope this helps.
I had this same problem and what worked for me in Django was what is described in this answer, which consists of:
Replacing
'ENGINE': 'django.db.backends.mysql'
with
'ENGINE': 'mysql_server_has_gone_away'
on
settings.DATABASES['ENGINE']
and installing with pip the package below:
mysql_server_has_gone_away==1.0.0
with connections.cursor() as cursor:
res=cursor.execute(sql)