import sqlite3
location = 'data'
table_name = 'table_name'
try:
conn = sqlite3.connect(location)
c = conn.cursor()
sql = 'insert into ' + table_name + ' (id) values (%d)' % (1)
c.execute(sql) # Some exception occurs here
conn.commit() # This line is not reached, so database is locked.
except Exception as e:
pass:
So if we run the above code and an exception occurs, the database is locked. So what should we do ?
EDIT: I am aware that this is vulnerable to sql injection. I've posted this example simply so that I can quickly ask the question.
This is a great use case for with:
from contextlib import closing
conn = sqlite3.connect(location)
with closing(conn.cursor()) as c:
# Do anything that may raise an exception here
With works on context objects. These objects define behavior for cleaning up in the event of an exception. In this case, the cursor will be released and the database will be unlocked if an exception occurs.
Note that we need closing from contextlib. This is because sqlite3 doesn't make its cursors contexts (even though it should, and other db's like mysql and postgres do this for their db bindings). closing allows the with to know what to do with the cursor when you leave the with's block.
A similar pattern is the correct way to handle files in Python:
with open('my_file.txt', 'r') as f:
# Do anything you want here, even something that may
# raise an exception
# the with will automatically f.close() whenever control
# leaves the with statement (whether by natural flow or
# exception)
As #IljaEverilä mentions in the comments, the connections themselves are also context managers and you should definitely use them this way to ensure that connections are properly cleaned up:
with sqlite3.connect(location) as conn:
# Do whatever you need with conn
I am using this below code to update the records of my table but it doesn't work and it doesn't update my table
conn = mysql.connector.connect(host='localhost',
database='rps',
user='root',
password='')
cur = conn.cursor()
cur.execute("""UPDATE players SET score=%s WHERE chat_id =%s""", (int(self.p1.score), str(self.p1.chat__id)))
cur.execute("""UPDATE players SET score=%s WHERE chat_id =%s""", (int(self.p2.score), str(self.p2.chat__id)))
conn.commit()
cur.close()
conn.close()
It is completely true but i don't know how to solve this problem
Can you help me to solve it guys ?
Thanks
You can set try/catch on your query execution:
try:
cursor.execute(some_statement)
except MySQLdb.IntegrityError, e:
# handle a specific error condition
except MySQLdb.Error, e:
# handle a generic error condition
except MySQLdb.Warning, e:
# handle warnings, if the cursor you're using raises them
If you don't catch any exceptions, you should check your program logic.
I am using Python/Bottle/SqlAlchemy/MySQL for a web service.
I am trying to catch an IntegrityError raised by calling a stored procedure but I am unable to do that.
Using this
cursor = connection.cursor()
cursor.callproc('my_stored_proc', [arguments])
yields the same result as
try:
cursor = connection.cursor()
cursor.callproc('my_stored_proc', [arguments])
except IntegrityError as e:
print("Error: {}".format(e))
return {"message": e.message}
I get an IntegrityError exception in both cases. Why is the exception not caught in the latter case?
The problem was I was catching the incorrect exception.
It turned out the error raised is actually of type pymysql.err.IntegrityError and not sqlalchemy.exc.IntegrityError as I assumed.
I found out the exception type by doing:
import sys
try:
cursor = connection.cursor()
cursor.callproc('my_stored_proc', [arguments])
except:
print "Unexpected error:", sys.exc_info()[0]
And I saw this printout:
Unexpected error: <class 'pymysql.err.IntegrityError'>
In my case, and also yours, you'd use the sqlalchemy object:
try:
cursor = connection.cursor()
cursor.callproc('my_stored_proc', [arguments])
except sqlalchemy.exc.IntegrityError as e:
print("Error: {}".format(e))
return {"message": e.message}
In my case and may be this helps you too, I was using MYSQL as database so to catch exceptions I need to import exceptions
from django.db.utils import IntegrityError
Then you can try and catch it like this
try:
#your code
except IntegrityError:
#your code when integrity error comes
With pymysql I use:
import pymysql
# some code
try:
#Your code
except pymysql.err.IntegrityError:
# Your exception code
except Exception as e:
# Raise all other exceptions.
raise e
The following cx_Oracle code works fine when the database is up:
#!C:\Python27
import cx_Oracle
try:
conn = cx_Oracle.connect("scott/tiger#oracle")
try:
curs = conn.cursor()
curs.execute("SELECT dummy FROM sys.dual")
print curs.fetchone()[0]
finally:
curs.close()
finally:
conn.close()
But if the database happens to be down when I run this script, a NameError is raised:
Traceback (most recent call last):
File "C:\Users\ArtMetzer\Documents\Code\Python\db_conn_test.py", line 14, in <module>
conn.close()
NameError: name 'conn' is not defined
This makes sense to me: cx_Oracle wasn't able to instantiate a connection, so the variable conn never got set, and hence has no close() method.
In Python, what's the best way to ensure your database connection closes, while still gracefully handling the condition of a down database?
Doing something like the following seems like a massive kludge to me:
finally:
try:
conn.close()
except NameError:
pass
You can try initializing conn to something like None before-hand and testing that in the finally block. This works because the only place the connection is set to something else is when it is opened. So opened implies non-None and None implies not-opened:
#!C:\Python27
import cx_Oracle
conn = None
try:
conn = cx_Oracle.connect("scott/tiger#oracle")
try:
curs = conn.cursor()
curs.execute("SELECT dummy FROM sys.dual")
print curs.fetchone()[0]
finally:
curs.close()
finally:
if conn is not None:
conn.close()
According to the docs, you don't really need to bother closing the connection:
https://cx-oracle.readthedocs.io/en/latest/user_guide/connection_handling.html#closing-connections
The important line being:
Alternatively, you may prefer to let connections be automatically cleaned up when references to them go out of scope. This lets cx_Oracle close dependent resources in the correct order.
(not exactly an answer, but comments don't have nice formatting)
Try this:
#!C:\Python27
import cx_Oracle
try:
conn = cx_Oracle.connect("scott/tiger#oracle")
try:
curs = conn.cursor()
curs.execute("SELECT dummy FROM sys.dual")
print curs.fetchone()[0]
finally:
curs.close()
conn.close()
except Exception as e:
print e
Not ideal, but should work better. I'm also wondering why so much nesting. Why not do this:
#!C:\Python27
import cx_Oracle
try:
conn = cx_Oracle.connect("scott/tiger#oracle")
curs = conn.cursor()
curs.execute("SELECT dummy FROM sys.dual")
print curs.fetchone()[0]
curs.close()
conn.close()
except Exception as e:
print e
BTW, I have this assumption that the connection and the cursor will close automatically on exit, removing the need to close them explicitly.
I got a lot of errors with the message :
"DatabaseError: current transaction is aborted, commands ignored until end of transaction block"
after changed from python-psycopg to python-psycopg2 as Django project's database engine.
The code remains the same, just don't know where those errors are from.
This is what postgres does when a query produces an error and you try to run another query without first rolling back the transaction. (You might think of it as a safety feature, to keep you from corrupting your data.)
To fix this, you'll want to figure out where in the code that bad query is being executed. It might be helpful to use the log_statement and log_min_error_statement options in your postgresql server.
To get rid of the error, roll back the last (erroneous) transaction after you've fixed your code:
from django.db import transaction
transaction.rollback()
You can use try-except to prevent the error from occurring:
from django.db import transaction, DatabaseError
try:
a.save()
except DatabaseError:
transaction.rollback()
Refer : Django documentation
In Flask you just need to write:
curs = conn.cursor()
curs.execute("ROLLBACK")
conn.commit()
P.S. Documentation goes here https://www.postgresql.org/docs/9.4/static/sql-rollback.html
So, I ran into this same issue. The problem I was having here was that my database wasn't properly synced. Simple problems always seem to cause the most angst...
To sync your django db, from within your app directory, within terminal, type:
$ python manage.py syncdb
Edit: Note that if you are using django-south, running the '$ python manage.py migrate' command may also resolve this issue.
Happy coding!
In my experience, these errors happen this way:
try:
code_that_executes_bad_query()
# transaction on DB is now bad
except:
pass
# transaction on db is still bad
code_that_executes_working_query() # raises transaction error
There nothing wrong with the second query, but since the real error was caught, the second query is the one that raises the (much less informative) error.
edit: this only happens if the except clause catches IntegrityError (or any other low level database exception), If you catch something like DoesNotExist this error will not come up, because DoesNotExist does not corrupt the transaction.
The lesson here is don't do try/except/pass.
I think the pattern priestc mentions is more likely to be the usual cause of this issue when using PostgreSQL.
However I feel there are valid uses for the pattern and I don't think this issue should be a reason to always avoid it. For example:
try:
profile = user.get_profile()
except ObjectDoesNotExist:
profile = make_default_profile_for_user(user)
do_something_with_profile(profile)
If you do feel OK with this pattern, but want to avoid explicit transaction handling code all over the place then you might want to look into turning on autocommit mode (PostgreSQL 8.2+): https://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode
DATABASES['default'] = {
#.. you usual options...
'OPTIONS': {
'autocommit': True,
}
}
I am unsure if there are important performance considerations (or of any other type).
just use rollback
Example code
try:
cur.execute("CREATE TABLE IF NOT EXISTS test2 (id serial, qa text);")
except:
cur.execute("rollback")
cur.execute("CREATE TABLE IF NOT EXISTS test2 (id serial, qa text);")
You only need to run
rollback;
in PostgreSQL and that's it!
If you get this while in interactive shell and need a quick fix, do this:
from django.db import connection
connection._rollback()
originally seen in this answer
I encountered a similar behavior while running a malfunctioned transaction on the postgres terminal. Nothing went through after this, as the database is in a state of error. However, just as a quick fix, if you can afford to avoid rollback transaction. Following did the trick for me:
COMMIT;
I've just got a similar error here. I've found the answer in this link https://www.postgresqltutorial.com/postgresql-python/transaction/
client = PsqlConnection(config)
connection = client.connection
cursor = client.cursor
try:
for query in list_of_querys:
#query format => "INSERT INTO <database.table> VALUES (<values>)"
cursor.execute(query)
connection.commit()
except BaseException as e:
connection.rollback()
Doing this the following query's you send to postgresql will not return an error.
I've got the silimar problem. The solution was to migrate db (manage.py syncdb or manage.py schemamigration --auto <table name> if you use south).
In Flask shell, all I needed to do was a session.rollback() to get past this.
I have met this issue , the error comes out since the error transactions hasn't been ended rightly, I found the postgresql_transactions of Transaction Control command here
Transaction Control
The following commands are used to control transactions
BEGIN TRANSACTION − To start a transaction.
COMMIT − To save the changes, alternatively you can use END TRANSACTION command.
ROLLBACK − To rollback the changes.
so i use the END TRANSACTION to end the error TRANSACTION, code like this:
for key_of_attribute, command in sql_command.items():
cursor = connection.cursor()
g_logger.info("execute command :%s" % (command))
try:
cursor.execute(command)
rows = cursor.fetchall()
g_logger.info("the command:%s result is :%s" % (command, rows))
result_list[key_of_attribute] = rows
g_logger.info("result_list is :%s" % (result_list))
except Exception as e:
cursor.execute('END TRANSACTION;')
g_logger.info("error command :%s and error is :%s" % (command, e))
return result_list
I just had this error too but it was masking another more relevant error message where the code was trying to store a 125 characters string in a 100 characters column:
DatabaseError: value too long for type character varying(100)
I had to debug through the code for the above message to show up, otherwise it displays
DatabaseError: current transaction is aborted
In response to #priestc and #Sebastian, what if you do something like this?
try:
conn.commit()
except:
pass
cursor.execute( sql )
try:
return cursor.fetchall()
except:
conn.commit()
return None
I just tried this code and it seems to work, failing silently without having to care about any possible errors, and working when the query is good.
I believe #AnujGupta's answer is correct. However the rollback can itself raise an exception which you should catch and handle:
from django.db import transaction, DatabaseError
try:
a.save()
except DatabaseError:
try:
transaction.rollback()
except transaction.TransactionManagementError:
# Log or handle otherwise
If you find you're rewriting this code in various save() locations, you can extract-method:
import traceback
def try_rolling_back():
try:
transaction.rollback()
log.warning('rolled back') # example handling
except transaction.TransactionManagementError:
log.exception(traceback.format_exc()) # example handling
Finally, you can prettify it using a decorator that protects methods which use save():
from functools import wraps
def try_rolling_back_on_exception(fn):
#wraps(fn)
def wrapped(*args, **kwargs):
try:
return fn(*args, **kwargs)
except:
traceback.print_exc()
try_rolling_back()
return wrapped
#try_rolling_back_on_exception
def some_saving_method():
# ...
model.save()
# ...
Even if you implement the decorator above, it's still convenient to keep try_rolling_back() as an extracted method in case you need to use it manually for cases where specific handling is required, and the generic decorator handling isn't enough.
This is very strange behavior for me. I'm surprised that no one thought of savepoints. In my code failing query was expected behavior:
from django.db import transaction
#transaction.commit_on_success
def update():
skipped = 0
for old_model in OldModel.objects.all():
try:
Model.objects.create(
group_id=old_model.group_uuid,
file_id=old_model.file_uuid,
)
except IntegrityError:
skipped += 1
return skipped
I have changed code this way to use savepoints:
from django.db import transaction
#transaction.commit_on_success
def update():
skipped = 0
sid = transaction.savepoint()
for old_model in OldModel.objects.all():
try:
Model.objects.create(
group_id=old_model.group_uuid,
file_id=old_model.file_uuid,
)
except IntegrityError:
skipped += 1
transaction.savepoint_rollback(sid)
else:
transaction.savepoint_commit(sid)
return skipped
I am using the python package psycopg2 and I got this error while querying.
I kept running just the query and then the execute function, but when I reran the connection (shown below), it resolved the issue. So rerun what is above your script i.e the connection, because as someone said above, I think it lost the connection or was out of sync or something.
connection = psycopg2.connect(user = "##",
password = "##",
host = "##",
port = "##",
database = "##")
cursor = connection.cursor()
It is an issue with bad sql execution which does not allow other queries to execute until the previous one gets suspended/rollback.
In PgAdmin4-4.24 there is an option of rollback, one can try this.
you could disable transaction via "set_isolation_level(0)"