I'm building an application using Flask and Flask-SQLAlchemy.
In the application I use database models written in the SQLAlchemy declarative language, let's say that I have a table called Server.
The application, by design choices, ask the user -via WTForms- to set values for the fields of the Server table between different pages(views) and I need save the instance to the database in the last view.
My problem is: I have 2 'circular' views and would like to store the instances of objects created in the first view directly in the database session in order to be able to query the session in the second view and committing the result only in the last view (the end of the loop), like the pseudocode shows (very simplified, it makes no sense in this form but is to explain the concept):
def first_view():
form = FormOne() #prompt the first form
if form.validate_on_submit():
#form validate, i create the instance in SQLalchemy
server = Server() #Database Model
server.hostname = form.hostname.data
loop_count = form.repetition.data
session['loop'] = loop_count
db.session.add(server) #adding the object to the session
#but i'm not committing the session in fact the next view needs to know about this new object and a third view needs to commit based on the user input
return redirect(url_for('second_view'))
return render_template("first_view.html", form=form)
def second_view():
form = FormTwo() #prompt the second form
if form.validate_on_submit():
hostname_to_search = form.hostname.data #i get some other input
#i use the input to query the session (here different server instace can appear, depends on the user input of the first view)
rslt= db.session.query(Server).filter(Server.hostname==hostname_to_search ).all()
#the session is empty and doesn't contain the instance created in the previous view... <-- :(
if session['loop'] <= 0 :
#end the loop
return redirect(url_for('commit_view'))
else:
loop_count = session.pop('loop',1)
session['loop'] = loop_count-1
#go back to the first page to add another server instance
return redirect(url_for('first_view'))
return render_template("first_view.html", form=form)
def commit_view():
#loop finished, now i can commit my final instances
db.session.commit() <--here i save the data, db.session is empty
return 'DONE!'
but SEEMS that the session in Flask-SQLAlchemy is local to the request, so it appears that between a view and another the db.session is resetted/empty.
The first solution came into my mind is to store even the server object values in the flask.session (in json format) but this means that I need to jsonify and parse back every time the flask.session to build back the objects in each view: I cannot use the query power of the database, but for example I have to manually check if the hostname input by the user is already present in the previously created server objects.
My question is: how is possible(if it is possible and good practice) to keep the db session 'open' between different views:
How can i implement this?
Is it convenient?
Is it thread safe?
Maybe i'm using the wrong approach to solve the problem? (Of course I can do a single page, but the real case in much more complex and needs to be structured between different pages)
Related
I'm working on this small Django project in which I'm using pre_save signals to update a table in which I save the cumulative value of a certain quantity, whenever a new Transaction is created or modified, the corresponding value in the table is updated. If I add a transaction manually from the admin page everything works fine but today I tried to create a new Transaction through a POST request using tastypie generated api, the problem is that when my update_total_if_changed function is called by the signal, the instance parameter is /api/v1/transaction/ instead of the actual python object, therefore I get "Transaction has no FieldName." since the instance actually points to the tastypie entrypoint instead of the newly created object.
Below you can see the code of my signal
#receiver(pre_save, sender=Transaction)
def update_total_if_changed(sender, instance, **kwargs):
try:
obj = sender.objects.get(pk=instance.pk)
except sender.DoesNotExist: #new transaction
tw, new = TotalWaste.objects.get_or_create(depot=instance.depot, waste = instance.waste)
tw.total += instance.quantity
tw.save()
else:
if not obj.quantity == instance.quantity: # Field has changed
tw, new = TotalWaste.objects.get_or_create(depot=instance.depot, waste = instance.waste)
tw.total = tw.total + instance.quantity - obj.quantity
tw.save()
I figured out the problem, I forgot to define the specific Resource, therefore the endpoint was not seen as a set of objects but just as a link that didn't point to anything, therefore I could not access the needed field.
I'll acknowledge that this is a strange question before I ask it. I'm wondering if it's possible to replicate Flask / SQL Alchemy class methods using raw SQL instead of using the methods themselves?
Long story short, my teammates and I are taking a database design course, and we're now in the implementation phase where we are coding the app that is based on our DB schema design. We want to keep things simple, so we opted for using Flask in Python. We're following the Flask Mega Tutorial, which is a kickass-tic tutorial explaining how to build a basic site like we're doing. We've just completed Chapter 5: User Logins, and are moving on.
In the app/routes.py script, the tutorial does something to grab the user information. Here's the example login route for the example app:
from flask_login import current_user, login_user
from app.models import User
# ...
#app.route('/login', methods=['GET', 'POST'])
def login():
if current_user.is_authenticated:
return redirect(url_for('index'))
form = LoginForm()
if form.validate_on_submit():
user = User.query.filter_by(username=form.username.data).first()
if user is None or not user.check_password(form.password.data):
flash('Invalid username or password')
return redirect(url_for('login'))
login_user(user, remember=form.remember_me.data)
return redirect(url_for('index'))
return render_template('login.html', title='Sign In', form=form)
The line user = User.query.filter_by(username=form.username.data).first() is what I'm interested in. Basically, that line instantiates the User class, which is a database model from SQL Alchemy, and grabs information about the user from the email address they entered. Calling those methods generates a SQL statement like the following:
SELECT `User`.`userID` AS `User_userID`,
`User`.user_email AS `User_user_email`,
`User`.user_first_name AS `User_user_first_name`,
`User`.user_last_name AS `User_user_last_name`,
`User`.user_password AS `User_user_password`
FROM `User`
WHERE `User`.user_email = 'test#test.com'
LIMIT 1
And also some information about the user variable itself:
>>> print(type(user))
<class 'myapp.models.User'>
>>> pp(user.__dict__)
{'_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x7f5a026a8438>,
'userID': 1,
'user_email': 'test#test.com',
'user_first_name': 'SomeFirstName',
'user_last_name': 'SomeLastName',
'user_password': 'somepassword'}
On our project, we're not supposed to be using generated SQL statements like the one that comes from calling query.filter_by(username=form.username.data).first() on the instantiated User class; we should be writing the raw SQL ourselves, which normally doesn't make sense, but in our case it does.
Is this possible?
First of all: Talk to your professor or TA. You will save yourself time by not making assumptions about something so major. If the goal of the class is to think about database schema design then using an ORM is probably fine. If you need to write your own SQL, then don't use an ORM to begin with.
To answer the technical question: yes, you can use SQLAlchemy purely as a database connection pool, as a tool to create valid SQL statements from Python objects, and as a full-fledged ORM, and every gradation in between.
For example, using the ORM layer, you can tell a Query object to not generate the SQL for you but instead take text. This is covered in the SQLAlchemy ORM tutorial under the Using Textual SQL section:
Literal strings can be used flexibly with Query, by specifying their use with the text() construct, which is accepted by most applicable methods
For your login example, querying for just the password could look like this:
user = User.query.from_statement(
db.text("SELECT * FROM User where user_email=:email LIMIT 1")
).params(email=form.username.data).first()
if user is None or user.check_password(form.password.data):
# ...
You could also read up on the SQL Expression API (the core API of the SQLAlchemy library) to build queries using Python code; the relationship between Python objects and resulting query is pretty much one on one; you generally would first produce a model of your tables and then build your SQL from there, or you can use literals:
s = select([
literal_column("User.password", String)
]).where(
literal_column("User.user_email") == form.username.data
).select_from(table("User")).limit(1)
and execute such objects with the Session.execute() method
results = db.session.execute(s)
If you wanted to really shoot yourself in the foot, you can pass strings to db.session.execute() directly too:
results = db.session.execute("""
SELECT user_password FROM User where user_email=:email LIMIT 1
""", {'email': form.username.data})
Just know that Session.execute() returns a ResultProxy() instance, not ORM instances.
Also, know that Flask-Login doesn't require you to use an ORM. As the project documentation states:
However, it does not:
Impose a particular database or other storage method on you. You are entirely in charge of how the user is loaded.
So you could just create a subclass of UserMixin that you instantiate each time you queried the database, manually.
class User(flask_login.UserMixin):
def __init__(self, id): # add more attributes as needed
self.id = id
#login_manager.user_loader
def load_user(user_id):
# perhaps query the database to confirm the user id exists and
# load more info, but all you basically need is:
return User(user_id)
# on login, use
user = User(id_of_user_just_logged_in)
login_user(user)
That's it. The extension wants to see instances that implement 4 basic methods, and the UserMixin class provides all of those and you only need to provide the id attribute. How you validate user ids and handle login is up to you.
I am using code like below from the post of Tracking model changes in SQLAlchemy. The only issue I am currently having is my map gets the end user's username through a token and I need to pass that username info into this function so it is logged with the changes. Is there a better method or do I just need to add a column to my tables called last modified by and just scrape that with this listener on update to the table?
#event.listens_for(cls, 'before_update')
def before_udpate(mapper, connection, target):
state = db.inspect(target)
changes = {}
for attr in state.attrs:
hist = state.get_history(attr.key, True)
if not hist.has_changes():
continue
# hist.deleted holds old value
# hist.added holds new value
changes[attr.key] = hist.added
# now changes map keys to new values
Or is there some type of way that I can call this information from my view?
After sleeping on this for the night. I came up with doing a mixin. Similar to what is on this page: http://francesco.pischedda.info/posts/logging-model-changes-with-sqlalchemy-listeners.html
I pass in my request object so that my target object will have the request object.
I have this accounts model in my django project, which stores the account balance(available money) of all users. Almost every deduction from the account of users is preceded by an amount check i.e. check if the user has x amount of money or more. If yes then go ahead and deduct the amount.
account = AccountDetails.objects.get(user=userid)
if int(account.amount) >= fare:
account.amount = account.amount-fare
account.save()
Now I want to put a lock in the first .get() statement, so that race conditions can be avoided. A user makes a request twice, and the application executes the above code twice simultaneously, causing one of the requests to override the other.
I found out that select_for_update() does exactly what I want. It locks the row until the end of the transaction.
account = AccountDetails.objects.select_for_update().get(user=userid)
But it's only available in Django 1.4 or higher and I'm still using Django 1.3 and moving to a new version can't be done right now. Any ideas how can I achieve this in my present Django version?
Looks like you'll have to use raw SQL. I had a look through the current code and I think it would be more hassle to try and backport this yourself than it would be to just write the SQL.
account = AccountDetails.objects.raw(
"SELECT * FROM yourapp_accountdetails FOR UPDATE WHERE user = %s", [user.id]
)
For convenience and to keep your code DRY you can add this as a method to your AccountDetails model or something.
class AccountDetails(models.Model):
#classmethod
def get_locked_for_update(cls, user):
return cls.objects.raw(
"SELECT * FROM yourapp_accountdetails FOR UPDATE WHERE user = %s", [user.id]
)
yourapp is the name of your application that you would have given when you ran startapp. I'm assuming you have a foreign key relationship on your AccountDetails to a user model of some kind.
The current implementation of select_for_update on Django 1.5 looks like this:
def select_for_update(self, **kwargs):
"""
Returns a new QuerySet instance that will select objects with a
FOR UPDATE lock.
"""
# Default to false for nowait
nowait = kwargs.pop('nowait', False)
obj = self._clone()
obj.query.select_for_update = True
obj.query.select_for_update_nowait = nowait
return obj
So that's pretty simple code. Just setting a few flags on the query object. However those flags won't mean anything when the query is executed. So at some point you'll need to write raw SQL. At this point you only need that query on your AccountDetails model. So just put it there for now. Maybe later you'll need it on another model. That's when you'll have to decide how to share the code between models.
Tricky code:
user = User.objects.filter(id=123)
user[0].last_name = 'foo'
user[0].save() # Cannot be saved.
id(user[0]) # 32131
id(user[0]) # 44232 ( different )
user cannot be saved in this way.
Normal code:
user = User.objects.filter(id=123)
if user:
user[0].last_name = 'foo'
user[0].save() # Saved successfully.
id(user[0]) # 32131
id(user[0]) # 32131 ( same )
So, what is the problem?
In first variant your user queryset isn't evaluated yet. So every time you write user[0] ORM makes independent query to DB. In second variation queryset is evalutaed and acts like normal Python list.
And BTW if you want just one row, use get:
user = User.objects.get(id=123)
when you index into a queryset, django fetches the data (or looks in its cache) and creates a model instance for you. as you discovered with id(), each call creates a new instance. so while you can set the properties on these qs[0].last_name = 'foo', the subsequent call to qs[0].save() creates a new instance (with the original last_name) and saves that
i'm guessing your particular issue has to do with when django caches query results. when you are just indexing into the qs, nothing gets cached, but your call if users causes the entire (original) qs to be evaluated, and thus cached. so in that case each call to [0] retrieves the same model instance
Saving is possible, but everytime you access user[0], you actually get it from the database so it's unchanged.
Indeed, when you slice a Queryset, Django issues a SELECT ... FROM ... OFFSET ... LIMIT ... query to your database.
A Queryset is not a list, so if you want to it to behave like a list, you need to evaluate it, to do so, call list() on it.
user = list(User.objects.filter(id=123))
In your second example, calling if user will actually evaluate the queryset (get it from the database into your python program), so you then work with your Queryset's internal cache.
Alternatively, you can use u = user[0], edit that and then save, which will work.
Finally, you should actually be calling Queryset.get, not filter here, since you're using the unique key.