I am using SQLAlchemy's provided contextmanager to handle sessions for me. What I don't understand is how to get the automatically generated ID because (1) the ID is not created until after commit() is called yet (2) the newly created instance is only available in the context manager's scope:
def save_soft_file(name, is_geo=False):
with session_scope() as session:
soft_file = models.SoftFile(name=name, is_geo=is_geo)
session.add(soft_file)
# id is not available here, because the session has not been committed
# soft_file is not available here, because the session is out of context
return soft_file.id
What am I missing?
Use session.flush() to execute pending commands within the current transaction.
def save_soft_file(name, is_geo=False):
with session_scope() as session:
soft_file = models.SoftFile(name=name, is_geo=is_geo)
session.add(soft_file)
session.flush()
return soft_file.id
If an exception occurs after a flush but before the session goes out of scope, the changes will be rolled back to the beginning of the transaction. In that case your soft_file would not actually be written to the database, even though it had been given an ID.
Related
Per the docs, we should use the following pattern with a sessionmaker object:
Session = sessionmaker(engine)
with Session.begin() as session:
session.add(some_object)
In a multithreaded environment, we are also supposed to use a single scoped_session and share it. So in my __init__.py I create create one and import it everywhere else in my program:
engine = create_engine(config.SQLALCHEMY_DATABASE_URI)
Session = scoped_session(sessionmaker(bind=engine))
The question is, how am I supposed to combine these two approaches? This seems to be the suggested way, but it errors out:
from myapp import Session
with Session.begin() as session:
query_result = session.query(MyModel).all()
----
Exception has occurred: AttributeError
'SessionTransaction' object has no attribute 'query'
I tried the following and it works, but it seems like it doesn't follow the docs, and I'm afraid it breaks something not obvious. Can anyone confirm if this is correct?
from myapp import Session
with Session() as session, session.begin():
query_result = session.query(MyModel).all()
I've been looking around at other replies and seeing very little that addresses the specific question.
From the Session.begin() docs:
The Session object features autobegin behavior, so that normally it is not necessary to call the Session.begin() method explicitly. However, it may be used in order to control the scope of when the transactional state is begun.
You can use Session.begin() (new in 1.4) to obtain a SessionTransaction instance usable as a context manager which will autocommit on successful exit.
Calling the scoped_session returns a SessionTransaction right away, as per your error, so you do not need to begin it again.
All in all, you can definitely do the stacked context manager, but its unnecessary, so you might as well stick to using the original flow:
Session = scoped_session(...)
with Session() as session: # NB. session is a thread local SessionTransaction
...
session.commit()
or the proxied Session
Session = scoped_session(...)
#on_request_end
def remove_session(req):
Session.remove()
#route("/xyz", ...)
def handle_xyz():
instance = Class(...)
Session.add(instance)
Session.commit()
I am using Flask, with the flask-session plugin for server-side sessions stored in a Redis backend. I have flask set up to use persistent sessions, with a session timeout. How can I make an AJAX request to get the time remaining on the session without resetting the timeout?
The idea is for the client to check with the server before displaying a timeout warning (or logging out the user) in case the user is active in a different tab/window of the same browser.
EDIT: after some digging, I found the config directive SESSION_REFRESH_EACH_REQUEST, which it appears I should be able to use to accomplish what I want: set that to False, and then the session should only be refreshed if something actually changes in the session, so I should be able to make a request to get the timeout without the session timeout changing. It was added in 0.11, and I'm running 0.11.1, so it should be available.
Unfortunately, in practice this doesn't appear to work - at least when checking the ttl of the redis key to get the time remain. I checked, and session.modified is False, so it's not just that I am doing something in the request that modifies the session (unless it just doesn't set that flag)
The following works, though it is rather hacky:
In the application __init__.py, or wherever you call Session(app) or init_app(app):
#set up the session
Session(app)
# Save a reference to the original save_session function so we can call it
original_save_session = app.session_interface.save_session
#----------------------------------------------------------------------
def discretionary_save_session(self, *args, **kwargs):
"""A wrapper for the save_session function of the app session interface to
allow the option of not saving the session when calling specific functions,
for example if the client needs to get information about the session
(say, expiration time) without changing the session."""
# bypass: list of functions on which we do NOT want to update the session when called
# This could be put in the config or the like
#
# Improvement idea: "mark" functions on which we want to bypass saving in
# some way, then check for that mark here, rather than having a hard-coded list.
bypass = ['check_timeout']
#convert function names to URL's
bypass = [flask.url_for(x) for x in bypass]
if not flask.request.path in bypass:
# if the current request path isn't in our bypass list, go ahead and
# save the session normally
return original_save_session(self, *args, **kwargs)
# Override the save_session function to ours
app.session_interface.save_session = discretionary_save_session
Then, in the check_timeout function (which is in the bypass list, above), we can do something like the following to get the remaining time on the session:
#app.route('/auth/check_timeout')
def check_timeout():
""""""
session_id = flask.session.sid
# Or however you want to get a redis instance
redis = app.config.get('REDIS_MASTER')
# If used
session_prefix = app.config.get('SESSION_KEY_PREFIX')
#combine prefix and session id to get the session key stored in redis
redis_key = "{}{}".format(session_prefix, session_id)
# The redis ttl is the time remaining before the session expires
time_remain = redis.ttl(redis_key)
return str(time_remain)
I'm sure the above can be improved upon, however the result is as desired: when calling /auth/check_timeout, the time remaining on the session is returned without modifying the session in any way.
I'm relatively new to Twisted and crossbar.io and I'm currently working on some database abstractions using sqlalchemy and alchimia (a layer to use sqlalchemy with twisted). The db abstractions I build so far are working as expected but I get problems when making asynchronous db calls inside my crossbar procedures. I guess that is because i have nested asynchronous calls with the outer call being some crossbar remote procedure and the inner being the database access.
What i want to do is the following:
#inlineCallbacks
def onJoin(self, details):
...
def login(user, password): # <-- outer call
...
db_result = yield db.some_query(user, password) # <-- inner call
for row in db_result: # <-- need to wait for db_result
return 'authentication_ticket'
return 'error'
To be able to authenticate the user i have to wait for the data from the db and then either issue a valid ticket or return an error. Currently I get an error that i cannot iterate over an deferred, because i don't wait for the db query to be finished.
Now, how do i wait inside my login RPC for the inner db call, then authenticate the user and then return. I know it is possible to chain asynchronous calls in twisted, but i don't know how to do this in this special case with crossbar and when my outer function uses the #inlineCallbacks shortcut.
Update:
Tinkering around I now can add a callback to the db query. Unfortunately now I don't know how to pass a return value from the inner function to the outer function (the outer function needs to return the ticket the user gets authenticated with):
#inlineCallbacks
def onJoin(self, details):
...
def login(user, password): # <-- outer call
db_result = db.some_query(user, password) # <-- inner call
def my_callback(query_result):
if not query_result.is_empty():
return 'user_ticket' # <-- needs to be returned by outer method
db_result.addCallback(my_callback)
return my_callback_result # <-- need something like that
I tried defer.returnValue('user_ticket') inside my callback function but this gives me an error.
Your problem appears to be that while login is expecting to yield Deferreds, it isn't decorated with #inlineCallbacks.
I'm trying to do a two-phase commit using SQLalchemy 0.6.8 with Postgresql 8.3.4, but I think I'm missing something...
The workflow goes like this:
session = sessionmaker(engine)(autocommit=True)
tx = session.connection().begin_twophase(xid) # Doesn't issue any SQL
session.begin()
session.add(obj1)
session.flush()
tx.prepare()
then from another session
session = sessionmaker(engine)(autocommit=True)
session.connection().commit_prepared(xid, recover=True) # recover=True because otherwise it complains that you can't issue a COMMIT PREPARED from inside a transaction
This doesn't raise any error, but doesn't write anything to the table either... O_o
What am I missing?
I tried even blocking the application after the prepare() and issuing a COMMIT PREPARED 'xid' from pgadmin, but still nothing gets written.
I managed to get it working, here's how:
session = sessionmaker(engine)(twophase=True)
session.add(obj1)
session.prepare()
# Find transaction id
for k, v in s.transaction._connections.iteritems():
if isinstance(k, Connection):
return v[1].xid
then from another session
session = sessionmaker(engine)(twophase=True)
session.connection().commit_prepared(xid, recover=True)
Using tornado, I want to create a bit of middleware magic that ensures that my SQLAlchemy sessions get properly closed/cleaned up so that objects aren't shared from one request to the next. The trick is that, since some of my tornado handlers are asynchronous, I can't just share one session for each request.
So I am left trying to create a ScopedSession that knows how to create a new session for each request. All I need to do is define a scopefunc for my code that can turn the currently executing request into a unique key of some sort, however I can't seem to figure out how to get the current request at any one point in time (outside of the scope of the current RequestHandler, which my function doesn't have access to either).
Is there something I can do to make this work?
You might want to associate the Session with the request itself (i.e. don't use scopedsession if it's not convenient). Then you can just say, request.session. Still needs to have hooks at the start/end for setup/teardown.
edit: custom scoping function
def get_current_tornado_request():
# TODO: ask on the Tornado mailing list how
# to acquire the request currently being invoked
Session = scoped_session(sessionmaker(), scopefunc=get_current_tornado_request)
(This is a 2017 answer to a 2011 question) As #Stefano Borini pointed out, easiest way in Tornado 4 is to just let the RequestHandler implicitly pass the session around. Tornado will track the handler instance state when using coroutine decorator patterns:
import logging
_logger = logging.getLogger(__name__)
from sqlalchemy import create_engine, exc as sqla_exc
from sqlalchemy.orm import sessionmaker, exc as orm_exc
from tornado import gen
from tornado.web import RequestHandler
from my_models import SQLA_Class
Session = sessionmaker(bind=create_engine(...))
class BaseHandler(RequestHandler):
#gen.coroutine
def prepare():
self.db_session = Session()
def on_finish():
self.db_session.close()
class MyHander(BaseHandler):
#gen.coroutine
def post():
SQLA_Object = self.db_session.query(SQLA_Class)...
SQLA_Object.attribute = ...
try:
db_session.commit()
except sqla_exc.SQLAlchemyError:
_logger.exception("Couldn't commit")
db_session.rollback()
If you really really need to asynchronously reference a SQL Alchemy session inside a declarative_base (which I would consider an anti-pattern since it over-couples the model to the application), Amit Matani has a non-working example here.