Let's say I have a User model with attributes id, name, email and a relationship languages.
Is it possible to create a User instance from existing data that behaves like I would have queried it with dbsession.query(User).get(42)?
What I mean in particular is that I want that an access to user.languages creates a subquery and populates the attribute.
Here a code example:
I have a class User:
class User(Base):
id = Column(Integer, primary_key=True)
name = Column(String(64))
email = Column(String(64))
languages = relationship('Language', secondary='user_languages')
I already have a lot of users stored in my DB.
And I know that I have, for example, this user in my DB:
user_dict = {
'id': 23,
'name': 'foo',
'email': 'foo#bar',
}
So I have all the attributes but the relations.
Now I want to make a sqlalchemy User instance
and kind of register it in sqlalchemy's system
so I can get the languages if needed.
user = User(**user_dict)
# Now I can access the id, name email attributes
assert user.id == 23
# but since sqlalchemy thinks it's a new model it doesn't
# lazy load any relationships
assert len(user.languages) == 0
# I want here that the languages for the user with id 23 appear
# So I want that `user` is the same as when I would have done
user_from_db = DBSession.query(User).get(23)
assert user == user_from_db
The use-case is that I have a big model with lots of complex
relationships but 90% of the time I don't need the data from those.
So I only want to cache the direct attributes plus what else I need
and then load those from the cache like above and be able to
use the sqlalchemy model like I would have queried it from the db.
From the sqlalchemy mailing list:
# to make it look like it was freshly loaded from the db
from sqlalchemy.orm.session import make_transient_to_detached
make_transient_to_detached(user)
# merge instance in session without emitting sql
user = DBSession.merge(user, load=False)
This answer was extracted from the question
Related
I want to create a simple app with Falcon that is able to handle small sqlite database with hostname: ip records. I want to be able to replace rows in sqlite, so I decide that hostname is unique field. I have a model.py:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine, Column, Integer, String
Base = declarative_base()
DB_URI = 'sqlite:///clients.db'
class Client(Base):
__tablename__ = 'clients'
id = Column(Integer, primary_key=True)
hostname = Column(String(50), unique=True)
ip = Column(String(50))
My simple resources.py:
from falcon_autocrud.resource import CollectionResource, SingleResource
from models import *
class ClientCollectionResource(CollectionResource):
model = Client
methods = ['GET', 'POST']
When I make a POST-request with updated information about hostname:ip i get an Unique constraint violated error:
req = requests.post('http://localhost:8000/clients',
headers={'Content-Type': 'application/json'},
data=json.dumps({'hostname': 'laptop1', 'ip': '192.168.0.33'}));
req.content
>> b'{"title": "Conflict", "description": "Unique constraint violated"}'
Is there any way to replace existing records using sqlalchemy? Or maybe I was wrong choosing sqlite for these purposes?
When building a REST-ful API you should not use POST to update existing resources, POST to a resource should only ever create new resources. falcon-autocrud is doing the right thing here.
Instead, use PUT on the individual resource (the SingleResource resource registered for .../clients/<identifier>) to alter existing resources.
If you use hostname in your SingleResource definition then falcon-autocrud should automatically use that column as the identifier (assuming that your SingleResource subclass is called ClientResource):
app.add_route('/clients/{hostname}', ClientResource(db_engine))
at which point you can PUT the new ip value directly with:
requests.put('http://localhost:8000/clients/laptop1', json={'ip': '192.168.0.33'})
(Note that requests supports JSON requests directly; the json= keyword argument is encoded to JSON for you, and the Content-Type header is set for you automatically when you use it).
You may want to limit what fields are returned for your Client objects. With a unique hostname you wouldn't want to confuse clients by also sending the primary key column. I'd limit the response fields by setting the response_fields attribute on your resource classes:
class ClientCollectionResource(CollectionResource):
model = Client
response_fields = ['hostname', 'ip']
methods = ['GET', 'POST']
class ClientResource(SingleResource):
model = Client
response_fields = ['hostname', 'ip']
I see that falcon-autocrud doesn't yet support PATCH requests on the collection that alter existing resources (only "op": "add" is supported), otherwise that'd be another route to alter existing entries too.
Generic solution to delete records which not used as a foreign key
Here are the presets, there are several models: 'ParentA', 'ParentB','ChildAA', 'ChildBA' and so on.
the relationship between ParentX and ChildXY is the ChildXY has a foreign key of ParentX,
for example:
#this is ParentA
class ParentA(Base):
__tablename__ = 'parenta'
id = Column(Integer, primary_key=True)
name = Column(String(12))
need_delete = Column(Integer)
children = relationship("ChildAA",
back_populates="parent")
#this is ChildAA
class ChildAA(Base):
__tablename__ = 'childaa'
name = Column(String(12))
id = Column(Integer, primary_key=True)
need_delete = Column(Integer)
parenta_id = Column(Integer, ForeignKey('parenta.id'))
parenta = relationship("ParentA")
#this is ParentB
........
And I wanna delete all the records(all the childx, parentx included) whose attribute 'need_delete' is 1 and record itself havn't been used as a foreign key by child table. I found a direct but complicated way:
I can firstly go through all the childx tables and safely removd records, and then to the parentx tables and delete records with the
code block one by one:
#deletion is for ParentA
for parent in session.query(ParentA).join(ParentA.children).group_by(ParentA).having(func.count(ChildAA.id) == 0):
if parent.need_delete == 1
session.delete(parent)
#deletion is for ParentB
......
#deletion is for ParentC
.....
session.commit()
And this is hard coded, Is there any generic way to delete records which is used as a foreign key at the present?
You could use NOT EXISTS, an antijoin, to query those parents which have no children and need delete:
from sqlalchemy import inspect
# After you've cleaned up the child tables:
# (Replace the triple dot with the rest of your parent types)
for parent_type in [ParentA, ParentB, ...]:
# Query for `parent_type` rows that need delete
q = session.query(parent_type).filter(parent_type.need_delete == 1)
# Go through all the relationships
for rel in inspect(parent_type).relationships:
# Add a NOT EXISTS(...) to the query predicates (the antijoin)
q = q.filter(~getattr(parent_type, rel.key).any())
# Issue a bulk delete. Replace `False` with 'fetch',
# if you do need to synchronize the deletions with the ongoing
# SQLA session. In your example you commit after the deletions,
# which expires instances in session, so no synchronization is
# required.
q.delete(synchronize_session=False)
...
session.commit()
Instead of first querying all the instances to the session and marking for deletion one by one use a bulk delete.
Do note that you must be explicit about your relationships and the parent side must be defined. If you have foreign keys referring to parent tables not defined as an SQLAlchemy relationship on the parent you'll probably get unwanted deletions of children (depends on how the foreign key constraints have been configured).
Another approach could be to configure your foreign key constraints to restrict deletions and handle the raised errors in a subtransaction (savepoint), but I suppose you've already set your schema up and that'd require altering the existing foreign key constraints.
I want to drop all user sessions when user resets his password, but I can't find a way to do that.
My idea was to get all UserTokens of the specific user and delete them, but it seems impossible, because of
user = model.StringProperty(required=True, indexed=False)
in UserToken model
Any ideas how to do that?
I see two ways how to do that.
First is to inherit from the UserToken class making user an indexed property. Then you can set the token_model class property to your new token model in your user class. Here is the code:
class MyToken(UserToken):
user = ndb.StringProperty(required=True)
class MyUser(User):
token_model = MyToken
# etc.
Don't forget to set the user model used by webapp2 to your user class if you do not do it already:
webapp2_config = {
"webapp2_extras.auth": {
"user_model": "models.MyUser"
},
# etc.
}
app = webapp2.WSGIApplication(routes, config=webapp2_config)
The second way is to make a complicated datastore query based on the token key name. Since the key names are of the form <user_id>.<scope>.<random>, it is possible to retrieve all the entities starting with a specific user ID. Have a look at the code:
def query_tokens_by_user(user_id):
min_key = ndb.Key(UserToken, "%s." % user_id)
max_key = ndb.Key(UserToken, "%s/" % user_id) # / is the next ASCII character after .
return UserToken.query(UserToken.key > min_key, UserToken.key < max_key)
This uses the fact that the query by key names works in the lexicographical order.
I am attempting to create an endpoint for a query joining multiple tables. registerSchema takes a sqlalchemy Base object. The solution I came up with was to create a database view for the sql statement, and use the model to reference the view.
Is this supported more natively with a registered schema? I would rather not maintain database views dependencies in my migrations.
sql for the dataview(replaced the table names with contrived examples)
CREATE VIEW v_user_offices AS
SELECT b.id AS building_id,
b.name AS building_name,
o.id AS office_id,
o.name AS office_name,
uo.user_id AS user_id
FROM buildings AS b
INNER JOIN office_buildings AS ob
ON ob.building_id=b.id
INNER JOIN offices AS o
ON o.id=ob.office_id
INNER JOIN user_offices AS uo
ON uo.office_id=o.id;
sqlalchemy Model:
class ViewUserOffices(CommonColumns):
__tablename__ = 'v_user_offices'
building_id = Column(Integer)
building_name = Column(String)
office_id = Column(Integer, primary_key=True)
office_name = Column(String)
user_id = Column(Integer)
settings.py
# The DOMAIN dict explains which resources will be available and how they will
# be accessible to the API consumer.
registerSchema('v_user_offices')(ViewUserOffices)
DOMAIN = {
'user_offices': ViewUserOffices._eve_schema['v_user_offices']
}
DOMAIN['user_offices'].update({
'item_title': 'user_office',
'item_lookup_field': 'user_id',
'resource_methods': ['GET']
})
The sqlalchemy-eve plugin is trying to be as close to sqlalchemy declarative approach as possible. If you can define your models in sqlalchemy you should be able to register them in python-eve.
In your case IMHO the only reasonable approach is to create a view and declare it in sqlalchemy as you did.
If I understand correctly, you can define this schema by taking advantage of sqlalchemy relationships. If your UserOffices class has all of the correct relationships defined, then registerSchema should automagically take care of the relationships between associated objects. All you will have to do is set the embedded parameter in your requests if you would like to embed the associated objects in the response.
A View makes sense if you do not want your client to have to worry about declaring embedded objects.
I want to create a form for many-to-many relations using Flask, SQLAlchemy and WTForms that represents these models:
personaddress = db.Table('personaddress',
db.Column('person', db.Integer, db.ForeignKey('person.id')),
db.Column('address', db.Integer, db.ForeignKey('address.id'))
)
class Person(db.Model):
__tablename__ = "person"
id = db.Column(Integer, primary_key=True)
name = db.Column(String, nullable=False)
addresses = db.relationship('Address', secondary=personaddress, backref=db.backref('person', lazy='dynamic'))
class Address(db.Model):
__tablename__ = "address"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False)
Requirements
Now I want to create a single page that contains forms to achieve the following:
add/edit/delete a person
add/edit/delete a address
add/edit/delete a relation between a person and an address
Important requirement: Using QuerySelectField, I can choose existing addresses for a person. But I want to add new addresses in the same form.
I've played around with model_form for the main models and subforms using FormField for the junction table but I just can't figure out how to update everything including the foreign key relations. The page should have a single submit button for all forms and subforms displayed.
Questions
How are the above requirements typically implemented in Flask?
Is this many-to-many scenario something that Django can handle more easily through its admin interface?
I have also encountered something similar earlier. I tried to solve it by using model_form, but it doesn't quite solve the problem of adding new entries dynamically, and I was having a hard time using it when dealing with relations.
Using the QuerySelectField in WTForms will only help you populating eg. an < select > with id, value pairs corresponding to the existing addresses. But it still renders to a regular html form in the template.
By using some sort of multiselect with the possibility to dynamically add new options in the frontend you can send additional addresses in the same form. The the endpoint will take care of creating new Addresses if they don't exist in the db.
The WTForm form would be:
from app import db
class PersonAddressForm(Form):
id = HiddenField('id')
name = StringField('Name')
addresses = QuerySelectField('Addresses',
query_factory=lambda: db.session.query(Address),
get_pk=lambda a: a.id, get_label=lambda a: a.name)
# Custom validate
def validate(self):
# ... custom validation
return True
And the route something like:
# ... this will be used to create and update a user
#route('create/<userid>', methods=["GET"])
def get_user_form(userid):
# ... Get the Person
user = Person()
if userid:
# ... if userid supplied, use existing Person object
user = Person.query.get(userid)
# ... Populate the form
person_form = PersonAddressForm(obj=user)
# ... return form
return render_template('somepage.html', form=person_form)
#route('create/<userid>', methods=["POST"])
def post_person_form(userid):
person_form = PersonAddressForm(request.form)
if person_form.validate():
# ... Get db object
person = db.session.query(Person).get(form.id)
# ... Add changes to the object from the form
person_form.populate_obj(obj=person_address)
# ... Get addresses
addresses = form.addresses.raw_data
# ... loop over and add to person
for address in addresses:
# Add or create an address
actual_address = db.session.query(Address).get(address.id)
# ... check if address is existing
if not actual_address:
# ... if address not existing, create new one
actual_address = Address(address.name)
db.session.add(actual_address)
# ... Append new or created address to person
person.addresses.append(actual_address)
# ... save changes to the db
db.session.commit()
# ... Update/Create complete
return redirect(url_for('get_users'))
else:
# ... form not valid, notify user
# ...
This will handle edit/create user and create Address. As well as create the relation between. To make it also support delete Address, change
person.addresses.append(actual_address)
to
person.addresses = list_of_actual_addresses
and change this in the person model (cascade='delete-orphan')
addresses = db.relationship('Address', secondary=personaddress, cascade='delete-orphan' backref=db.backref('person', lazy='dynamic'))
This will make the form update the entire address relation each time and the cascade will delete orphaned addresses. So the entire addresses list for a person would be updated each time the form is submitted.
When dealing with WTForms in templates i highly recommend using macros if you don't already. You would have to rewrite it to some degree, but check this out.
Hope this helps