autocommit in SQLAlchemy:
sessionmaker(bind=engine, autocommit=False) # autocommit default False
autocommit in MySQL:
SET AUTOCOMMIT=0 -- autocommit default 1
I am wondering, are the two autocommits the same thing? i.e. SQLAlchemy passes autocommit status to MySQL via something equivolent to SET AUTOCOMMIT?
According to the docs, these aren't identical (although they achieve the same result).
SQLAlchemy always uses transactions, so it always sets AUTOCOMMIT=0 on MySQL. However, if you set autocommit=True, it will automatically call .commit() whenever you make a change to the data.
This is done in that way because every database does autocommit differently (if at all), and SQLAlchemy tries to behave consistently between them.
Related
With default settings in Django (version 3.1) is safe to do:
with connection.cursor() as cursor:
cursor.execute("BEGIN")
# Some SQL operations
commit_or_rollback = "COMMIT" if success else "ROLLBACK"
with connection.cursor() as cursor:
cursor.execute(commit_or_rollback)
Or must I set autocommit to False with set_autocommit method before, as Django's autocommit closes transactions? Or autocommit is isolated and there will be no problem with my code?
In case you're asking why I'm using raw SQL for transactions: I've tried using transactions manually as docs indicates but it had some issues with multi-process enviroment, so I had to implement with raw queries
Ok, I've been reading more, testing in my project and seing the queries executed by Django in the database logs. It seems to be safe to use transactions with raw SQL as autocommit begins a new one with any operation and doesn't interferes with transactions opened by others connections
From https://wiki.postgresql.org/wiki/Psycopg2_Tutorial
PostgreSQL does not have an autocommit facility which means that all
queries will execute within a transaction.
Execution within a transaction is a very good thing, it ensures data
integrity and allows for appropriate error handling. However there are
queries that can not be run from within a transaction. Take the
following example.
#/usr/bin/python2.4
#
#
import psycopg2
# Try to connect
try:
conn=psycopg2.connect("dbname='template1' user='dbuser' password='mypass'")
except:
print "I am unable to connect to the database."
cur = conn.cursor()
try:
cur.execute("""DROP DATABASE foo_test""")
except:
print "I can't drop our test database!"
This code would actually fail with the printed message of "I can't
drop our test database!" PostgreSQL can not drop databases within a
transaction, it is an all or nothing command. If you want to drop the
database you would need to change the isolation level of the database
this is done using the following.
conn.set_isolation_level(0)
You would place the above immediately preceding the DROP DATABASE
cursor execution.
I was wondering why
"PostgreSQL does not have an autocommit facility which means that all queries will execute within a transaction."
"PostgreSQL can not drop databases within a transaction"
"If you want to drop the database you would need to change the isolation level of the database"
Thanks.
Update:
What does autocommit mean in postgresql and psycopg2? answer my question
All the 3 are related to Python and its DB connector library, not the PostgreSQL itself:
PostgreSQL has an autocommit and it is active by default - which means that every SQL statement is immediately executed. When you start a transaction block, this autocommit mode is disabled until you finish the transaction (either by COMMIT or ROLLBACK)
The operation of destroying a database is implemented in a way where you can not run it from inside a transaction block. Also keep in mind that unlike most other databases PostgreSQL allows almost all DDL statements (obviously not the DROP DATABASE one) to be executed inside a transaction.
Actually you can not drop a database if anyone (including you) is currently connected to this database - so it does not matter what is your isolation level, you still have to connect to another database (e.g. postgres)
I have this code to insert data to Database using MySQL. But when I ran that code using Python, there's no error. But when I checked the Database, the data isn't inserted. Is there anyone who can help me? I would appreciate it. :)
This is the code:
import MySQLdb
db=MySQLdb.connect(host="localhost", user="root", passwd="", db="try")
cursor=db.cursor()
insert="INSERT INTO `try`.`try` (`nomor`, `nama`) VALUES (NULL, 'bismillah')"
cursor.execute(insert)
You're not doing a COMMIT anywhere. So, if auto-commit is not on, all you've done is create a transaction that, if later committed, will insert this row.
Since you haven't done a SET AUTOCOMMIT anywhere, whether auto-commit is on depends on how you created the database. With at least some storage types (in particular, InnoDB), you can change the default at creation time, and, because you often want auto-commit disabled with those storage types, your GUI design tool, or the sample code you copied and pasted, or whatever may have done so for you. Also, the server variable that provides the default can itself be set to a different value at server startup/configuration. (See System Server Variables.)
If you want to make sure that auto-commit is on, just execute SET autocommit=1 before any other statements.
If you want to find out whether auto-commit is on, execute SHOW VARIABLES. (And if it's disabled, you may want to try SHOW GLOBAL VARIABLES LIKE 'autocommit' and SHOW SESSION VARIABLES like 'autocommit' to see which context you've disabled it in.)
If you cannot insert into mysql, there are several ways to solve it:
1: check the log
2: check the structure of your table, maybe it must not be null for nomor or other field
3: the last, when insert into mysqldb using program, you need to commit before you close the connection. for here: db.commit()
I'm using Elixir in a project that connects to a postgres database. I want to run the following query on the database I'm connected to, but I'm not sure how to do it as I'm rather new to Elixir and SQLAlchemy. Anyone know how?
VACUUM FULL ANALYZE table
Update
The error is: "UnboundExecutionError: Could not locate a bind configured on SQL expression or this Session". And the same result with session.close() issued before. I did try doing metadata.bind.execute() and that worked for a simple select. But for the VACUUM it said - "InternalError: (InternalError) VACUUM cannot run inside a transaction block", so now I'm trying to figure out how to turn that off.
Update 2
I can get the query to execute, but I'm still getting the same error - even when I create a new session and close the previous one.
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
# ... insert stuff
old_session.commit()
old_session.close()
new_sess = sessionmaker(autocommit=True)
new_sess.configure(bind=create_engine('postgres://user:pw#host/db', echo=True))
sess = new_sess()
sess.execute('VACUUM FULL ANALYZE table')
sess.close()
and the output I get is
2009-12-10 10:00:16,769 INFO sqlalchemy.engine.base.Engine.0x...05ac VACUUM FULL ANALYZE table
2009-12-10 10:00:16,770 INFO sqlalchemy.engine.base.Engine.0x...05ac {}
2009-12-10 10:00:16,770 INFO sqlalchemy.engine.base.Engine.0x...05ac ROLLBACK
finishing failed run, (InternalError) VACUUM cannot run inside a transaction block
'VACUUM FULL ANALYZE table' {}
Update 3
Thanks to everyone who responded. I wasn't able to find the solution I wanted, but I think I'm just going to go with the one described here PostgreSQL - how to run VACUUM from code outside transaction block?. It's not ideal, but it works.
Dammit. I knew the answer was going to be right under my nose. Assuming you setup your connection like I did.
metadata.bind = 'postgres://user:pw#host/db'
The solution to this was as simple as
conn = metadata.bind.engine.connect()
old_lvl = conn.connection.isolation_level
conn.connection.set_isolation_level(0)
conn.execute('vacuum analyze table')
conn.connection.set_isolation_level(old_lvl)
This is similar to what was suggested here PostgreSQL - how to run VACUUM from code outside transaction block?
because underneath it all, sqlalchemy uses psycopg to make the connection to postgres. Connection.connection is a proxy to the psycopg connection. Once I realized this, this problem came back to mind and I decided to take another whack at it.
Hopefully this helps someone.
You need to bind the session to an engine
session.bind = metadata.bind
session.execute('YOUR SQL STATEMENT')
UnboundExecutionError says that your session is not bound to an engine and there is no way to discover engine from query passed to execute(). You can either use engine.execute() directly or pass additional mapper parameter (either mapper or mapped model corresponding to table used in query) to session.execute() to help SQLAlchemy discover proper engine.
The InternalError says that you are trying to execute this statement inside explicitly (with BEGIN statement) started transaction. Have you issued some statements before it without calling commit()? If so, just call commit() or rollback() method to close transaction before doing VACUUM. Also note, that there are several parameter to sessionmaker() that tell SQLAlchemy when transaction should be started.
If you have access to SQLAlchemy session, you can execute arbitrary SQL statements via its execute method:
session.execute("VACUUM FULL ANALYZE table")
(Depending on the Postgres version) you most likely do not want to run "VACUUM FULL".
I'm new to Python and Python's MySQL adapter. I'm not sure if I'm missing something obvious here:
db = MySQLdb.connect(# db details omitted)
cursor = self.db.cursor()
# WORKS
cursor.execute("SELECT site_id FROM users WHERE username=%s", (username))
record = cursor.fetchone()
# DOES NOT SEEM TO WORK
cursor.execute("DELETE FROM users WHERE username=%s", (username))
Any ideas?
I'd guess that you are using a storage engine that supports transactions (e.g. InnoDB) but you don't call db.commit() after the DELETE. The effect of the DELETE is discarded if you don't commit.
See http://mysql-python.sourceforge.net/FAQ.html#my-data-disappeared-or-won-t-go-away:
Starting with 1.2.0, MySQLdb disables
autocommit by default, as required by
the DB-API standard (PEP-249). If you
are using InnoDB tables or some other
type of transactional table type,
you'll need to do connection.commit()
before closing the connection, or else
none of your changes will be written
to the database.
See also this similar SO question: Python MySQLdb update query fails
Perhaps you are violating a foreign key constraint.
To your code above,
just add a call to self.db.commit().
The feature is far from an annoyance:
It saves you from data corruption issues
when there are errors in your queries.
The problem might be that you are not committing the changes. it can be done by
conn.commit()
read more on this here