HI is there is any way that I can insert a row to db without using session. A simple Example:
try:
db.session.add(user1)
#in here I want to insert a row to my db but I can't do it with session because if i commit in here it will commit all inserts so my transaction not work.
db.session.add(user2)
except:
db.session.rollback()
else:
db.session.commit()
thank you
If you want to commit changes independently of the default db.session there a couple of possibilities.
If you need an actual session, create one using SQLAlchemy and use it for your log entries:
from sqlalchemy import orm
...
#app.route('/')
def index():
model = MyModel(name='M1')
db.session.add(model)
with orm.Session(db.engine).begin() as log_session:
# Session.begin will commit automatically.
log = MyLog(message='hello')
log_session.add(log)
return ''
If you are just inserting entries in the log table you can just connect using the engine.
import sqlalchemy as sa
...
#app.route('/')
def index():
model = MyModel(name='M1')
db.session.add(model)
log_table = sa.Table('my_log', db.metadata, autoload_with=db.engine)
with db.engine.begin() as conn:
conn.execute(log_table.insert(), {'message': 'hello'})
db.session.rollback()
return ''
You could also send a raw SQL statement using the mechanism in (2.), by replacing log_table.insert with sa.text(sql_string)
How ever you choose to do this be aware that:
Due to transaction isolation, you two transactions may have different views of the data in the database
You are responsible for making sure these additional sessions/transactions/connections are rolled back, committed and closed as necessary
You are responsible for handling problem scenarios, for example if an error causes the db.session to roll back, making log messages potentially invalid.
Related
I have a web application that is on top of mysql. When I started it, I built the mysql database from scratch, and connected to it using pymysql.
Fast forward...
I've rewritten everything using sqlalchemy to connect to the db (non-declarative?) I can connect to the db, read the db, update, etc. I also use MySQLWorkbench to view the database in a graphic way.
My webapp has a few tables that will poll the database for changes, and update the table. For instance, I have a job queue which will update the percentage-done. Here's the kicker:
When I send a job through, I will watch the database in MySQLWorkbench. I can verify that the job is going through (I see the percentage climbing). This confirms that the webapp is writing to the database. But! On the other end, where the table is asking for the status, it is not recieving the updated information (I've done print outs in the models, as well as in the flask app, as well as the FE js).
What is going on here? Even though I can see it in MySQLWorkbench is the connection not 'releasing' or something? Here is some of my code:
#models.py
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker, Query
import os
import json
Base = automap_base()
engine = create_engine("mysql+pymysql://user:asdas758ef10d2364d54d7e8#localhost:{}/metadatacontroller".format(3306))
# reflect the tables
Base.prepare(engine, reflect=True)
EncodeQueue = Base.classes.encode_queue
db_session = scoped_session(sessionmaker(bind=engine))
class DataStore:
def __init__(self):
self.db = db_session()
#receive status
def queue_rendering_items(self):
query = self.db.query(EncodeQueue).filter(EncodeQueue.status.in_(['queue', 'rendering', 'rendering thumbnails'])).all()
return_query = []
for item in query:
row = {'artist_id': item.artist_id,
'art_id': item.art_id,
'status': item.status,
'progress': item.errors,
'artwork_title': item.artwork_title,
'artist_name': item.artist_name,
}
return_query.append(row)
return return_query
#update status
def update_render_progress(self, job_id, progress):
self.db.query(EncodeQueue).filter_by(job_id=job_id).update\({'errors': json.dumps(progress)})
self.db.commit()
(ignore the back slash in update_render_progress ... weird SO formatting)
This is how 'status' code is being called on a flask end point:
#app.route('/api/jobs', methods=["GET"])
def get_all_jobs():
db = models.DataStore()
all_art = db.queue_rendering_items()
print(all_art)
return jsonify({'data':all_art})
Lastly, the status is being updated by a different file, on a different machine:
import models
db = models.DataStore()
db.update_render_progress(job_id, {'percentage': percentage_complete, 'time_remain': render.render_estimated_seconds_remaining})
What the heck is happening?
I want to have a Flask route that deletes all instances of a SQLAlchemy model, VisitLog. I call VisitLog.query.delete(), then redirect back to the page, but the old entries are still present. There was no error. Why weren't they deleted?
#app.route('/log')
def log():
final_list = VisitLog.query.all()
return render_template('log.html', loging=final_list)
#app.route('/logclear')
def logclear():
VisitLog.query.delete()
return redirect("log.html", code=302)
Clear database
Just like other write operations, you must commit the session after executing a bulk delete.
VisitLog.query.delete()
db.session.commit()
I wonder how SQLAlchemy tracks changes that are made outside of SQLAlchemy (manual change for example)?
Until now, I used to put db.session.commit() before each value that can be changed outside of SQLAlchemy. Is this a bad practice? If yes, is there a better way to make sure I'll have the latest value? I've actually created a small script below to check that and apparently, SQLAlchemy can detect external changes without db.session.commit() being called each time.
Thanks,
P.S: I really want to understand how all the magics happen behind SQLAlchemy work. Does anyone has a pointer to some docs explaining the behind-the-scenes work of SQLAlchemy?
import os
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
# Use SQLlite so this example can be run anywhere.
# On Mysql, the same behaviour is observed
basedir = os.path.abspath(os.path.dirname(__file__))
db_path = os.path.join(basedir, "app.db")
app.config["SQLALCHEMY_DATABASE_URI"] = 'sqlite:///' + db_path
db = SQLAlchemy(app)
# A small class to use in the test
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100))
# Create all the tables and a fake data
db.create_all()
user = User(name="old name")
db.session.add(user)
db.session.commit()
#app.route('/')
def index():
"""The scenario: the first request returns "old name" as expected.
Then, I modify the name of User:1 to "new name" directly on the database.
On the next request, "new name" will be returned.
My question is: how SQLAlchemy knows that the value has been changed?
"""
# Before, I always use db.session.commit()
# to make sure that the latest value is fetched.
# Without db.session.commit(),
# SQLAlchemy still can track change made on User.name
# print "refresh db"
# db.session.commit()
u = User.query.filter_by(id=1).first()
return u.name
app.run(debug=True)
The "cache" of a session is a dict in its identity_map (session.identity_map.dict) that only caches objects for the time of "a single business transaction" , as answered here https://stackoverflow.com/a/5869795.
For different server requests, you have different identity_map. It is not a shared object.
In your scenario, you requested the server 2 separated times. The second time, the identity_map is a new one (you can easily check it by printing out its pointer), and has nothing in cache. Consequently the session will request the database and get you the updated answer. It does not "track change" as you might think.
So, to your question, you don't need to do session.commit() before a query if you have not done a query for the same object in the same server request.
Hope it helps.
The following query retrieves every reservation in the database, but I only want to get the reservations that the current user made. When I run the query in the mysql command line, it works. Why isn't this working in my app?
from flaskext.mysql import MySQL
from flask.ext.login import LoginManager
from flask_login import current_user
...
db = MySQL()
db.init_app(app)
...
cur = db.connect().cursor()
cur.execute("SELECT ReservationID FROM Reservation WHERE user_name=" + "'" + current_user + "';")
reservation_instance = cur.fetchall()
current_user is a User instance. You can't just add it to a string, you need to use it's username (assuming you called the attribute "username").
You are currently open to SQL injection attacks. Never try to insert parameters into a query string yourself. Always use parameterized queries, which lets the driver do the work of building, quote, and escape the query properly.
cur.execute('SELECT ReservationID FROM Reservation WHERE user_name=?', [current_user.username])
The placeholder character (I used ? in the example) depends on the driver. Consider using SQLAlchemy, which normalizes this sort of thing and is much more powerful. There is the Flask-SQLAlchemy to make integration with Flask easier as well.
I am using peewee along with flask to populate my database with user's inputs and various other stats. How can I retrieve the data in sqlite3 database?
Here is the skeleton:
from peewee import *
database = SqliteDatabase('app4.db', autocommit=True)
class BaseModel(Model):
class Meta:
database = database
class Model1(BaseModel):
class Model2(BaseModel):
class Model3(BaseModel):
app = Flask(__name__)
#app.before_request
def func1():
database.connect()
#app.after_request
def func2(response):
database.close()
return response
#app.route('/', method= ['GET'])
def some_func():
#update pluto and create instances of Model2 and Model3 here
if __name__ == "__main__":
database.connect()
database.create_tables([Model1, Model2, Model3], True)
pluto, created = Model1.get_or_create(key = 2, key2 = 0)
My app is running and showing the updated instances of Model1 and Model2 which means that it has created the models and is continually updating the database.
But I don't know how to retrieve this data offline.
When I enter my sqllite database by sqlite3 app4.db and type .tables it shows me all the tables i.e. Model1, Model2, Model3 but typing in the query : select * from model1; gives nothing. I tried retrieving the data in Ipython shell too making a connection to database as above and querying list(Model1.select()) but it gives an empty list.
It looks like your database filename is being set to app.py:
database = SqliteDatabase('app.py', autocommit=True)
It should probably be app4.db if that's what you want to use:
database = SqliteDatabase('app4.db', autocommit=True)
The commands you are using look like the correct ones for doing it at the command line. Sounds to me like either 1 of 2 things are happening:
You are not actually putting data into the databases
You are not viewing the same database that you are populating.
Try renaming the app4.db file, and re-running your app and see if it creates a new one. This way you can make sure you are using sqlite3 on the correct one. If you are still not seeing any data using the commands you listed, I would assume that your app is not actually putting the data into the tables.