I am new to Python and was studying FastApi and SQL model.
Reference link: https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/#the-with-block
Here, they have something like this
def create_hero(*, session: Session = Depends(get_session), hero: HeroCreate):
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
return db_hero
Here I am unable to understand this part
session.add(db_hero)
session.commit()
session.refresh(db_hero)
What is it doing and how is it working?
Couldn't understand this
In fact, you could think that all that block of code inside of the create_hero() function is still inside a with block for the session, because this is more or less what's happening behind the scenes.
But now, the with block is not explicitly in the function, but in the dependency above:
It's an explanation from docs what is a session
In the most general sense, the Session establishes all conversations
with the database and represents a “holding zone” for all the objects
which you’ve loaded or associated with it during its lifespan. It
provides the interface where SELECT and other queries are made that
will return and modify ORM-mapped objects. The ORM objects themselves
are maintained inside the Session, inside a structure called the
identity map - a data structure that maintains unique copies of each
object, where “unique” means “only one object with a particular
primary key”.
So
# This line just simply create a python object
# that sqlalchemy would "understand".
db_hero = Hero.from_orm(hero)
# This line add the object `db_hero` to a “holding zone”
session.add(db_hero)
# This line take all objects from a “holding zone” and put them in a database
# In our case we have only one object in this zone,
# but it is possible to have several
session.commit()
# This line gets created row from the database and put it to the object.
# It means it could have new attributes. For example id,
# that database would set for this new row
session.refresh(db_hero)
Related
Let's say I have an object in memory from my database that I retrieved using sqlalchemy like this:
user_object = database_session.query(User).first()
I do a few checks on this object that don't change anything, like this (for this example my user_object has name property equal to "John"):
if user_object.name == "John":
do nothing
else:
do something
I then add this object to a list that will get committed to the database at once, with this operation in mind:
objects_list.append(user_object)
database_session.add_all(objects_list)
database_session.commit()
Some of the objects in objects_list will have changes, some will not.
Is this process of committing a number of unchanged objects wasteful, and if so what is a better way to handle this situation?
When you do
user_object = database_session.query(User).first()
the loaded object is already in the session. From there on the session tracks changes to the object through instrumentation. Session.add() and Session.add_all() simply ignore objects that are already in the session, so
objects_list.append(user_object)
database_session.add_all(objects_list)
is at least partly redundant. When you call Session.commit() SQLAlchemy will first flush any pending changes held in the Session to the database and then commits. In other words you're not committing any "unchanged objects".
I am using SQLAlchemy with Postgres and need to create "external ids" that depend on the Postgres-generated (sequential) id. The code looks like this:
# session is passed to the constructor as described in http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it
def make_merchant(merchant_data, session):
merchant = Merchant(**merchant_data)
session.add(merchant)
session.flush() # Is this needed to have Postgres generate the sequential id?
merchant.external_id = 'merchant_' + convert_to_external_id(merchant.id) # Uses the Postgres-generated id
My issue: the flush seems necessary to get the Postgres-generated id, but it introduces a potentially annoying side effect as it flushes not only the newly created merchant but also other objects in the session.
My questions:
Is there a way to access the Postgres id without a flush?
If not, is there a way to only flush this one object? Creating a separate session just for this construction seems like an anti-pattern
Thanks!
For my Django application, I'm looking to keep a full edit history for all objects. As part of this, I've overridden the model's save() method, part of which is shown below:
# Replicate the current version (from the db) with all attributes unchanged
new_ver = self.__class__.objects.get(pk=self.pk).save(force_insert=True)
# Update the current version in the database with the new attributes
super(CodexBaseClass, self).save(*args, force_update=True, **kwargs)
The 'self' that's passed to the save() method is the NEW version of the object that's been generated by the form. What this code is attempting to do is
(1) Make a copy of the object as it currently appears in the database (ie: copy the data as it was before the form modified it), then force an insert of this data so it's copied as a new row
(2) Update the existing row with the new version of the object that's been submitted through the form.
The problem is with the first line of the two lines of code - It generates a DoesNotExist exception. The object does exist, so I'm currently thinking that the issue is that the database row it's trying to read is currently locked.
So my question is: Is there a way I can modify/replace the first line so that I have a copy of the initial data, as it was before the form modified it?
Thanks.
If you want to insert a new object with same attributes, you only need to change the primary key of your object, and save it.
new_ver = self.__class__.objects.get(pk=self.pk)
new_ver.pk = None
new_ver.save()
Using None as primary key will auto generates it. You can have more information if you look at the django documentation.
If you need you can also make a copy of your object, be careful the cost can be expensive :
from copy import deepcopy
ver = self.__class__.objects.get(pk=self.pk)
new_ver = deepcopy(ver)
new_ver.pk = None
new_ver.save()
# Do what you need with ver object
You should take a look at django-reversion.
django-reversion is an extension to the Django web framework that
provides version control for model instances.
Documentation: link
Features
Roll back to any point in a model instance’s history.
Recover deleted model instances.
Simple admin integration.
I am using Pony ORM for a flask solution and I've come across the following.
Consider the following:
#db_session
def get_orders_of_the_week(self, user, date):
q = select(o for o in Order for s in o.supplier if o.user == user)
q2 = q.filter(lambda o: o.date >= date and o.date <= date+timedelta(days=7))
res = q2[:]
#for r in res:
# print r.supplier.name
return res
When I need the result in Jinja2 -- which is looks like this
{% for order in res %}
Supplier: {{ order.supplier.name }}
{% endfor %}
I get a
DatabaseSessionIsOver: Cannot load attribute Supplier[3].name: the database session is over
If I uncomment the for r in res part, it works fine. I suspect there is some sort of lazy loading that doesn't get loaded with res = q2[:].
Am I completely missing a point or what's going on here?
I just added prefetch functionality that should solve your problem. You can take working code from the GitHub repository. This feature will be part of the upcoming release Pony ORM 0.5.4.
Now you can write:
q = q.prefetch(Supplier)
or
q = q.prefetch(Order.supplier)
and Pony will automatically load related supplier objects.
Below I'll show several queries with prefetching, using the standard Pony example with Students, Groups and Departments.
from pony.orm.examples.presentation import *
Loading Student objects only, without any prefetching:
students = select(s for s in Student)[:]
Loading students together with groups and departments:
students = select(s for s in Student).prefetch(Group, Department)[:]
for s in students: # no additional query to the DB is required
print s.name, s.group.major, s.group.dept.name
The same as above, but specifying attributes instead of entities:
students = select(s for s in Student).prefetch(Student.group, Group.dept)[:]
for s in students: # no additional query to the DB is required
print s.name, s.group.major, s.group.dept.name
Loading students and its courses (many-to-many relationship):
students = select(s for s in Student).prefetch(Student.courses)
for s in students:
print s.name
for c in s.courses: # no additional query to the DB is required
print c.name
As a parameters of the prefetch() method you can specify entities and/or attributes. If you specified an entity, then all to-one attributes with this type will be prefetched. If you specified an attribute, then this specific attribute will be prefetched. The to-many attributes are prefetched only when specified explicitly (as in the Student.courses example). The prefetching goes recursively, so you can load long chain of attributes, such as student.group.dept.
When object is prefetched, then by default all of its attributes are loaded, except lazy attributes and to-many attributes. You can prefetch lazy and to-many attributes explicitly if it is needed.
I hope this new method fully covers your use-case. If something is not working as expected, please start new issue on GitHub. Also you can discuss functionality and make feature requests at Pony ORM mailing list.
P.S. I'm not sure that repository pattern that you use give your serious benefits. I think that it actually increase coupling between template rendering and repo implementation, because you may need to change repo implementation (i.e. add new entities to prefetching list) when template code start using of new attributes. With the top-level #db_session decorator you can just send query result to the template and all happens automatically, without the need of explicit prefetching. But maybe I'm missing something, so I will be interested to see additional comments about the benefits of using the repository pattern in your case.
This happens because you're trying to access the related object which was not loaded and since you're trying to access it outside of the database session (the function decorated with the db_session), Pony raises this exception.
The recommended approach is to use the db_session decorator at the top level, at the same place where you put the Flask's app.route decorator:
#app.route('/index')
#db_session
def index():
....
return render_template(...)
This way all calls to the database will be wrapped with the database session, which will be finished after a web page is generated.
If there is a reason that you want to narrow the database session to a single function, then you need to iterate the returning objects inside the function decorated with the db_session and access all the necessary related objects. Pony will use the most effective way for loading the related objects from the database, avoiding the N+1 Query problem. This way Pony will extract all the necessary objects within the db_session scope, while the connection to the database is still active.
--- update:
Right now, for loading the related objects, you should iterate over the query result and call the related object attribute:
for r in res:
r.supplier.name
It is similar to the code in your example, I just removed the print statement. When you 'touch' the r.supplier.name attribute, Pony loads all non-lazy attributes of the related supplier object. If you need to load lazy attributes, you need to touch each of them separately.
Seems that we need to introduce a way to specify what related objects should be loaded during the query execution. We will add this feature in one of the future releases.
I use python in GAP and try to delete one entries in datastore by using db.delete(model_obj). I suppose this operation is undertaken synchronously, since the document tell the difference between delete() and delete_async(), but when I read the source code in the db, the delete method just simply call the delete_async, which is not match what the document says :(
So is there any one to do delete in synchronous flow?
Here is the source code in db:
def delete_async(models, **kwargs):
"""Asynchronous version of delete one or more Model instances.
Identical to db.delete() except returns an asynchronous object. Call
get_result() on the return value to block on the call.
"""
if isinstance(models, (basestring, Model, Key)):
models = [models]
else:
try:
models = iter(models)
except TypeError:
models = [models]
keys = [_coerce_to_key(v) for v in models]
return datastore.DeleteAsync(keys, **kwargs)
def delete(models, **kwargs):
"""Delete one or more Model instances.
"""
delete_async(models, **kwargs).get_result()
EDIT: From a comment, this is the original misbehaving code:
def tearDown(self):
print self.account
db.delete(self.device)
db.delete(self.account)
print Account.get_by_email(self.email, case_sensitive=False)
The result for two print statement is <Account object at 0x10d1827d0> <Account object at 0x10d1825d0>. Even two memory addresses are different but they point to the same object. If I put some latency after the delete like for loop, the object fetched is None.
The code you show for delete calls delete_async, yes, but then it calls get_result on the returned asynchronous handle, which will block until the delete actually occurs. So, delete is synchronous.
The reason the sample code you show is returning an object is that you're probably running a query to fetch the account; I presume the email is not the db.Key of the account? Normal queries are not guaranteed to return updated results immediately. To avoid seeing stale data, you either need to use an ancestor query or look up the entity by key, both of which are strongly consistent.