I am trying to create a login function. But it only works ones. Ex- When i give a wrong userid and password I got correct error massage that "Could't login" after canceling that message and giving correct userid and password then I get "pymysql.err.Error: Already closed" below are the sample code.
import pymysql
# Connect to the database
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
class LoginModel:
def check_user(self, data):
try:
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `username` FROM `users` WHERE `username`=%s"
cursor.execute(sql, (data.username))
user = cursor.fetchone()
print(user)
if user:
if (user, data.password):
return user
else:
return False
else:
return False
finally:
connection.close()
You have a mismatch with respect to the number of times you're creating the connection (once) and the number of times you're closing the connection (once per login attempt).
One fix would be to move your:
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
into your def check__user(). It would work because you'd create and close the connection on each invocation (as others have pointed out, the finally clause always gets executed.)
That's not a great design because getting database connections tends to be relatively expensive. So keeping the connection creating outside of the method is preferred.... which means you must remove the connection.close() within the method.
I think you're mixing up connection.close() with cursor.close(). You want to do the latter, not the former. In your example you don't have to explicitly close the cursor because that happens automatically with your with connection.cursor() as cursor: line.
Change finally to except, or remove the try block completely.
This is the culprit code:
finally:
connection.close()
Per the docs:
"A finally clause is always executed before leaving the try statement, whether an exception has occurred or not"
From: https://docs.python.org/2/tutorial/errors.html
You didn't describe alternative behavior for what you would like to see happen instead of this, but my answer addresses the crux of your question.
Had the same issue. The "Finally clause is needed for Postgres with the psycopg2 driver, if used with context manager (with clause), it close the cursor but not the connection. The same does not apply with Pymysql.
Related
I'm trying to use psycopg2 in python to drop index in postgresql:
connection = psycopg2.connect(host=hostname, user=username, password=password, dbname=database)
cur = connection.cursor()
statement = "DROP INDEX IF EXISTS idx_my_id"
cur.execute(statement)
connection.commit()
Same statement can be done in pgAdmin4 in one second. But in python, the execution never finished.
"pg_stat_activity" shows wait_event_type is Lock and wait_event is relation.
What went wrong?
won't fit in comments put it here. maybe because you don't commit your connections?
add this to your code , close all the connections and try again:
connection.set_session(autocommit=True)
I have a code like this:
import mysql.connector as mysql
from generate_records import generateRecords
devicesQuery = "CALL iot.sp_sensors_overview()"
try:
db = mysql.connect(
user = "username",
password = "password",
host = "hostname",
database="iot"
)
cursor = db.cursor(dictionary=True, buffered=True)
cursor.execute(devicesQuery)
for sensor in cursor:
generateRecords(sensor, db)
cursor.close()
except mysql.connector.Error as error:
print("Error:")
print(error)
else:
db.close()
The purpose of generateRecords function is obviously to generate records and run the INSERT query against the different table.
Seems like I do something wrong, because no matter what I trying, I getting different errors here, like mysql.connector.errors.OperationalError: MySQL Connection not available..
(upd) I also tried to change the code like it was suggested (see example bellow), with no luck - I still receiving the MySQL connection not available. error.
rows = cursor.fetchall()
cursor.close()
for sensor in rows:
cursor2 = db.cursor()
generateRecords(sensor, cursor2)
So, should I create a new connection within generateRecords function, or pass something different within it, or use some kind of different approach here?
Thank you!
Finally I found what was wrong. I'm used the query to call the stored procedure. Using the cursor.callproc("sp_sensors_overview") instead fixed my issue, and now I'm able to create the next cursor without errors.
With psycopg2, connection and querying the database works like so
conn = psycopg2.connect('connection string')
with conn:
cur=conn.cursor()
cur.execute("SELECT * FROM pg_stat_activity") #simple query
rows = cur.fetchall()
for row in rows:
print (row)
After trial and error, I found out that with conn is absolutely necessary or you will get many unexplained locks.
My question is: is there a way to setup the connection to avoid the need to use it?
From https://www.psycopg.org/docs/usage.html,
Warning
Unlike file objects or other resources, exiting the connection’s with
block doesn’t close the connection, but only the transaction
associated to it. If you want to make sure the connection is closed
after a certain point, you should still use a try-catch block:
conn = psycopg2.connect(DSN)
try:
# connection usage
finally:
conn.close()
In psycopg, the Context manager has been implemented in such a way that the with statement will only terminate the transaction and not close the connection for you. The connection needs to be closed by you separately.
In case of an error with your transaction, you have the option to rollback and raise the error.
One way to do this is to write the connection closing logic yourself.
def with_connection(func):
"""
Function decorator for passing connections
"""
def connection(*args, **kwargs):
# Here, you may even use a connection pool
conn = psycopg.connect(DSN)
try:
rv = func(conn, *args, **kwargs)
except Exception as e:
conn.rollback()
raise e
else:
# Can decide to see if you need to commit the transaction or not
conn.commit()
finally:
conn.close()
return rv
return connection
#with_connection
def run_sql(conn, arg1):
cur = conn.cursor()
cur.execute(SQL, (arg1))
Since Version 2.5, psycopg2 should support the with statement like you expect it to behave.
Docs
conn = psycopg2.connect(DSN)
with conn:
with conn.cursor() as curs:
curs.execute(SQL1)
with conn:
with conn.cursor() as curs:
curs.execute(SQL2)
conn.close()
I'm still using Flask-mysql.
I'm getting the database context (the mysql variable) just fine, and can query on the database / get results. It's only the insert that is not working: it's not complaining (throwing Exceptions). It returns True from the insert method.
This should be done inserting the record when it commits, but for some reason, as I watch the MySQL database with MySQL Workbench, nothing is getting inserted into the table (and it's not throwing exceptions from the insert method):
I'm passing in this to insertCmd:
"INSERT into user(username, password) VALUES ('test1','somepassword');"
I've checked the length of the column in the database, and copied the command into MySQL Workbench (where it successfully inserts the row into the table).
I'm at a loss. The examples I've seen all seem to follow this format, and I have a good database context. You can see other things I've tried in the comments.
def insert(mysql, insertCmd):
try:
#connection = mysql.get_db()
cursor = mysql.connect().cursor()
cursor.execute(insertCmd)
mysql.connect().commit()
#mysql.connect().commit
#connection.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
You need to keep a handle to the connection; you keep overriding it in your loop.
Here is a simplified example:
con = mysql.connect()
cursor = con.cursor()
def insert(mysql, insertCmd):
try:
cursor.execute(insertCmd)
con.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
If mysql is your connection, then you can just commit on that, directly:
def insert(mysql, insertCmd):
try:
cursor = mysql.cursor()
cursor.execute(insertCmd)
mysql.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
return False
Apparently, you MUST separate the connect and cursor, or it won't work.
To get the cursor, this will work: cursor = mysql.connect().cursor()
However, as Burchan Khalid so adeptly pointed out, any attempt after that to make a connection object in order to commit will wipe out the work you did using the cursor.
So, you have to do the following (no shortcuts):
connection = mysql.connect()
cursor = connection.cursor()
cursor.execute(insertCmd)
connection.commit()
I have built a site using Django and I am receiving this annoying error when I am trying to execute a query.
If I restart the Apache server, the error will go away for a short time.
Traceback:
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/home/fran/cron/views/set_caches.py" in set_caches
24. cursor.execute(query, [category['id']])
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
15. return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py" in execute
86. return self.cursor.execute(query, args)
File "build/bdist.linux-i686/egg/MySQLdb/cursors.py" in execute
155. charset = db.character_set_name()
Exception Type: InterfaceError at /blablabla/
Exception Value: (0, '')
This is caused by a global cursor. Try creating and closing the cursor within each method a raw query is needed.
cursor = connection.cursor()
cursor.execute(query)
cursor.close()
You get this error when you have a db.close() call and later try to access the database without creating a new connection. Try to find if you close the connection to the database when you don't mean to.
I agreed with Moberg. This error is caused when we try to access the database after we have closed the connection. This could be caused by some wrong indentation in the code. Below is my code.
conn = connect()
cur = conn.cursor()
tk = get_tickers(cur)
for t in tk:
prices = read_price(t, cur)
if prices != None:
update_price(t, cur)
print 'Price after update of ticker ', t, ':'
p_open, p_high, p_low, p_close = read_price(t, cur)
print p_open, p_high, p_low, p_close
else:
print 'Price for ', t, ' is not available'
conn.close()
I got the same error as reported by Marian. After dedenting conn.close(), everything worked well. Confirmed that global conn is not an issue.
I had the same problem as for April of 2019 using python 3.7 and Mysql 2.7.
At intermittent intervals, the string (0, '') would be added at random to my SQL statements causing errors. I have solved the issue by commenting on the closing of the database connection and just leaving the closing of the cursors across my code.
def set_db():
db = pymysql.connect(host='localhost',
user="root",
passwd="root",
db="DATABASE")
return db
def execute_sql(cnx, sql_clause, fetch_all):
if sql_clause and sql_clause is not None:
try:
cnx.execute(sql_clause)
except Exception as e:
print("Error in sql: " + sql_clause + str(e))
return 0
pass
if fetch_all:
result = cnx.fetchall()
else:
result = cnx.fetchone()
return result
else:
print("Empty sql.")
return 0
db = set_db()
cnx = db.cursor()
sql = "SELECT * FROM TABLE"
result = execute_sql(cnx, sql, 1)
cnx.close() #close the cursor
#db.close #do not close the db connection
...
I had the same issue using threading with Python3 and Pymysql. I was getting deadlocks and then I would get hit with InterfaceError (0, '').
My issue was that I was trying to do a rollback on exception of the query- I believe this rollback was trying to use a connection that no longer existed and it was giving me the interface error. I took this rollback out (because I am OK with not doing rollback for this query) and I just let things go. This fixed my issue.
def delete_q_msg(self, assetid, queuemsgtypeid, msgid):
"""
Given the paramerts below remove items from the msg queue equal to or older than this.
If appropriate send them into a history table to be processed later
:param assetid:
:param queuemsgtypeid:
:param msgid:
:return:
"""
params = (assetid, queuemsgtypeid, msgid,)
db_connection = self._connect_to_db()
sp_sql = "{db}.ps_delete_q_msg".format(db=self._db_settings["database"])
return_value = []
try:
with db_connection.cursor() as cursor:
cursor.callproc(sp_sql, params)
return_value = cursor.fetchall()
db_connection.commit()
except Exception as ex:
# i think we dont want rollback here
# db_connection.rollback()
raise Exception(ex)
finally:
db_connection.close()
return return_value
I can confirm this is caused by a global cursor which is then later used in some functions. My symptoms were the exact same: intermittent interface errors that would temporarily be cleared up by an apache restart.
from django.db import connection
cursor = connection.cursor() # BAD
def foo():
cursor.execute('select * from bar')
But, I am using Django on top of Oracle 11.2 so I do not believe this is a bug in the MySQL/python driver. This is probably due to the caching done by apache/mod_wsgi.
I had the same issue with Flask+pymysql, I was getting an empty tuple as a result in the except: block, something like this "(\"(0, '')\",)" to be specific.
It turned out that the connection was getting closed and later the code tried accessing it which resulted into this error.
So I solved it by referring to above solutions and used a function for connection which assured me a conn every time I had to access the db.
You can recreate this issue by inserting conn.close() just before accessing the cursor.
For reference I used this site which helped me solve this issue.
https://hackersandslackers.com/python-mysql-pymysql/
For me, removing the conn.close() from my function worked. I was trying to access the database again after closing.
I am using flask with AWS.
Also you can try to restart your flask application if it has been running for a long time & if you are also using AWS RDS with MYSQL workbench like in my case, then just check whether your session is expired or not and update the access key and id.
Hope this helps.
I had this same problem and what worked for me in Django was what is described in this answer, which consists of:
Replacing
'ENGINE': 'django.db.backends.mysql'
with
'ENGINE': 'mysql_server_has_gone_away'
on
settings.DATABASES['ENGINE']
and installing with pip the package below:
mysql_server_has_gone_away==1.0.0
with connections.cursor() as cursor:
res=cursor.execute(sql)