I'm trying to build a very basic management system website for a hypothetical insurance agency and i just cant wrap my head around how i should organize the database to make it so i can assign users to specific policies and have the ability to update/replace the user in case there are re-arrangements within the agency so policies can be reassigned to the proper agents. This would be used to display data based on login as well. There's 3 layers that i think i need. A User table for user data, a client data/policy table to store client and policy info, and then a table for tasks that would be assigned to policies. I need multiple users to have access to a policy and then the policy should have access to 1 row in the task table. Would it just be better to have a user table and large client table with the task columns inside rather than a separate table for the tasks? I've been banging my head with this for days so if anyone can help, i greatly appreciate it.
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SECRET_KEY'] = ''
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///site.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
username = db.Column(db.String(50), unique=True)
password = db.Column(db.String(50))
email = db.Column(db.String(50), unique=True)
#Multiple assigned users can access
class Client(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
policy_number = db.Column(db.String(50), unique = True)
expiration_date = db.Column(db.Datetime)
#Single "client" assigned to single row of tasks based on policy number
class PolicyTasks(db.Model):
id = db.Column(db.Integer, primary_key=True)
step1 = db.Column(db.String(50))
step1_completed = db.Column(db.Boolean)
step2 = db.Column(db.String(50))
step2_completed = db.Column(db.Boolean)
step3 = db.Column(db.String(50))
step3_completed = db.Column(db.Boolean)
step4 = db.Column(db.String(50))
step4_completed = db.Column(db.Boolean)
step5 = db.Column(db.String(50))
step5_completed = db.Column(db.Boolean)
I removed the code i used to attempt to create the relationships because it might honestly be more helpful to look at the base layout
To wrap your head around how you should organize the database to make it so you can assign users to specific policies, take a look at the docs on the Flask website. I think the second example under One-to-Many is very similar to what you're looking to do http://flask-sqlalchemy.pocoo.org/2.3/models/.
I would recommend keeping the tables separate and not jamming it all into one. Usually that leads to a slippy slope (in my experience) with tables with too many attributes to manage.
It can also mean slower query times because Client.query.filter_by(username='peter').first() would then always query the client data, policy data and whatever else you later on end up throwing in that table, when you may've only needed the policy data for that specific view/api route.
This other stackoverflow post might help too:
database design - when to split tables?
Related
I am using flask with flask-restplus and sqlalchemy.
My rest API function looks like the following:
#ns.route('/user')
class UsersCollection(Resource):
#jwt_optional
#permissions.user_has('admin_view')
#ns.marshal_list_with(user_details)
def get(self):
"""
Returns list of users.
"""
users = User.query.all()
return users
I am willing to add additional data to users collection which should be returned by the API. Some of this data may be stored in other DB tables, while other is stored in memory.
What I am usually doing is something like this:
#ns.route('/user')
class UsersCollection(Resource):
#jwt_optional
#permissions.user_has('admin_view')
#ns.marshal_list_with(user_details)
def get(self):
"""
Returns list of users.
"""
users = User.query.all()
# Add additional data
for user in users:
user.activity = UserActivity.query ... # from DB
user.online_status = "Active" # Taken somewhere from memory
return users
Obviously I don't like this approach of adding data to the User object on the fly. But what is the best design pattern to achieve it?
About the design pattern, I recommend a DAO and Service approach, here's a good and short article about it:
https://levelup.gitconnected.com/structuring-a-large-production-flask-application-7a0066a65447
About the models User/Activity relationship, I presume you have a mapped Activity model. If this is a One to One relationship, in the User model create a activityId field for the FK field and a activity relation where your ORM (I'll assume SQLAlchemy) will retrieve when user.activity is accessed.
class Activity(db.Model):
id = db.Column(db.Integer, primary_key=True)
descrption = db.Column(db.String(255))
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
activity = db.relationship(Activity)
activityId = db.Column(db.Integer, db.ForeignKey(Activity.id))
If you have a Many to Many relationship you'll have to create a third class called ActivityUser to map the relation
class ActivityUser(db.Model):
id = db.Column(db.Integer, primary_key=True)
activity = db.relationship(Activity, lazy=True)
user = db.relationship(User, lazy=True)
activityId = db.Column(db.ForeignKey(Activity))
userId = db.Column(db.ForeignKey(User))
and you can retrieve the user activities like this (again assuming you use SQLAlchemy):
ActivityUser.query.filter(ActivityUser.userId == user.id).all()
I'm a beginner with python/Flask/SQLAlchemy so sorry if my questions are dumb.
I want to create an API with Flask using Flask-SQLAlchemy as following:
one sqlite database for users/passwords
SQLALCHEMY_DATABASE_URI = 'sqlite:////path/to/users.db'
class User(DB.Model):
__tablename__ = 'users'
id = DB.Column(DB.Integer, primary_key=True)
username = DB.Column(DB.String(64), index=True)
password = DB.Column(DB.String(128))
Lets say I have multiple "customers" witch a user can create using
$ http POST http://localhost:5000/api/customers/ name=customer1
class Customer(DB.Model):
__tablename__ = 'customer'
customer_id = DB.Column(DB.Integer, primary_key=True)
customer_name = DB.Column(DB.String, unique=True, index=True)
I need to create a separate sqlite file for each "customers" :
SQLALCHEMY_BINDS = {
'customer1' = 'sqlite:////path/customer1.db',
'customer2' = 'sqlite:////path/customer2.db',
...
}
My questions are:
I do not have fixed number of "customers" so I cannot create a model class for each and specify the "bind_key" for each. Is it possible to do this with Flask-SQLAlchemy or I need to use plain SQLAlchemy?
I have 3 "customers" in data/ as customer1.db, customer2.db and customer3.db.
I would start the application and create SQLALCHEMY_BINDS dictionary listing the files in data/ and then DB.create_all() on a request for a specific "customer" .
how can I bind to the right .db file using the Flask-SQLAlchemy
DB.session?
I've read Using different binds in the same class in Flask-SQLAlchemy
Why exactly do you want entirely separate DB files for each customer?
In any case this is easier with straight SQLAlchemy. You can create a getter function which returns a session pointing to your db file.
def get_session(customer_id):
sqlite_url = 'sqlite:////path/customer%s.db' % customer_id
engine = create_engine(sqlite_url)
# initialize the db if it hasn't yet been initialized
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
return session
You can then use and close that session.
But without knowing your specific use case, it is difficult to understand why you would want to do this instead of just using a single SQLite database.
I have SQLAlchemy models in my Flask application:
class User(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
photos = db.relationship('Photo', lazy='joined')
class Photo(db.Model):
__tablename__ = 'photos'
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('users.id'))
photo = db.Column(db.String(255))
When i query user i get user with his photos automatically. But i noticed if user has a lot of photos, query u = User.query.get(1) become very slow. I do the same but manually with lazy='noload' option
u = User.query.get(1)
photos = Photo.query.filter_by(user_id == 1)
and on the same data it works faster in times. Where is problem, is sql join slow(don`t think so, because it start hang on 100-1kk photo objects, not so big data) or something wrong in the SQLAlchemy?
From my experiences I suggest you to get familiar with SQLAlchemy Loading Relationships. Sometimes even if relationship functionality is easy, usefull in larger datasets is better to do not use it, or even execute plain text SQL. This will be better from performance point of view on larger data sets.
I have a Flask python application that has a set of related tables that are chained together through foreign keys. I would like to be able to return an aggregate list of records from one table that are related to a distant table. However, I am struggling to understand how sqlalchemy does this through object relationships.
For example, there are three objects I'd like to join (challenge and badge) with two tables (talent_challenge and badge) to be able to query for all badges related to a specific challenge. In SQL, this would look something like:
SELECT b.id, b.name
FROM badge b
INNER JOIN talent_challenge tc ON tc.talent_id = b.talent_id
WHERE tc.challenge_id = 21
The 'talent' and 'challenge' tables are not needed in this case, since I only need the talent and challenge IDs (in 'talent_challenge') for the relationship. All of the interesting detail is in the badge table.
I am able to use sqlalchemy to access the related talent from a challenge using:
talents = db.relationship('TalentModel', secondary='talent_challenge')
And I can then reference talent.badges for each of those talents to get the relevant badges related to my initial challenge. However, there can be redundancy, and this list of badges isn't contained in a single object.
A stripped-down version of the three models are:
class TalentModel(db.Model):
__tablename__ = 'talent'
# Identity
id = db.Column(db.Integer, primary_key=True)
# Relationships
challenges = db.relationship('ChallengeModel', secondary='talent_challenge',)
# badges (derived as backref from BadgeModel)
class ChallengeModel(db.Model):
__tablename__ = 'challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
member_id = db.Column(db.Integer, db.ForeignKey('member.id'))
# Relationships
talents = db.relationship('TalentModel', secondary='talent_challenge', order_by='desc(TalentModel.created_at)')
class BadgeModel(db.Model):
__tablename__ = 'badge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
# Parents
talent = db.relationship('TalentModel', foreign_keys=[talent_id], backref="badges")
I also have a model for the associative table, 'talent_challenge':
class TalentChallengeModel(db.Model):
__tablename__ = 'talent_challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
challenge_id = db.Column(db.Integer, db.ForeignKey('challenge.id'))
# Parents
talent = db.relationship('TalentModel', uselist=False, foreign_keys=[talent_id])
challenge = db.relationship('ChallengeModel', uselist=False, foreign_keys=[challenge_id])
I would like to better understand sqlalchemy (or specifically, flask-sqlalchemy) to allow me to construct this list of badges from the challenge object. Is db.session.query of BadgeModel my only option?
UPDATED 1/23/2015:
My blocker on my project was solved by using the following:
#property
def badges(self):
from app.models.sift import BadgeModel
from app.models.relationships.talent import TalentChallengeModel
the_badges = BadgeModel.query\
.join(TalentChallengeModel, TalentChallengeModel.talent_id==BadgeModel.talent_id)\
.filter(TalentChallengeModel.challenge_id==self.id)\
.all()
return the_badges
Wrapping the query in a function got around the issues I was having with the name BadgeModel not being defined and not being able to be imported in the model otherwise. The #property decorator allows me to just reference this as challenge.badges later in the view.
However, I am still interested in understanding how to do this as a relationship. Some searching elsewhere led me to believe this would work:
badges = db.relationship('BadgeModel',
secondary="join(BadgeModel, TalentChallengeModel, BadgeModel.talent_id == TalentChallengeModel.talent_id)",
secondaryjoin="remote([id]) == foreign(TalentChallengeModel.challenge_id)",
primaryjoin="BadgeModel.talent_id == foreign(TalentChallengeModel.talent_id)",
viewonly=True,
)
Because of other unresolved issues in my application environment, I can't fully test this (e.g., adding this code breaks Flask-User in my site) but would like to know if this is correct syntax and if there is any disadvantage to this over the query-in-function solution.
A little background: I am creating a web application (using Flask) for use internally in an organization. The webapp will have a very simple message board that allows users to post and comment on posts.
I'm doing this for a couple reasons -- mainly to get experience with Flask and to better understand sqlalchemy.
This is the database schema with some non-important info removed:
class User(db.Model):
id = db.Column(db.Integer, primary_key = True)
# information about user
posts = db.relationship('Post', backref = 'author', lazy = 'dynamic')
comments = db.relationship('Comment', backref = 'author', lazy = 'dynamic')
class Post(db.Model):
id = db.Column(db.Integer, primary_key = True)
# information about posts (title, body, timestamp, etc.)
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
comments = db.relationship('Comment', backref = 'thread', lazy = 'dynamic')
class Comment(db.Model):
id = db.Column(db.Integer, primary_key = True)
# information about comment (body, timestamp, etc)
user_id = db.Column(db.Integer, db.ForeignKey('user.id')) # author
post_id = db.Column(db.Integer, db.ForeignKey('post.id')) # thread
When I render the messages view, I want to be able to display a table of threads with the following information for each message:
Title
Author
# Replies
Time of last modification
Right now, my query to get the messages looks like this:
messages = Post.query.filter_by(post_type = TYPE_MESSAGE).order_by('timestamp desc')
With that query, I get easily get the title and author for each post. However, it currently orders by the date the message was created (I know that is wrong, and I know why) and I can't easily get the number of replies.
If I was looping through the messages to render them in the application, I could access the message.comments attribute and use that to find the length and get the timestamp of the most recent comment, but am I correct in assuming that to get that data it would require another database query (to access message.comments)?
Since that is the case, I could get the list of all of the messages with one query (good) but if I had n messages, it would require n additional database queries to populate the messages view with the information that I want, which is far from efficient.
This brings me to my main question: is it possible to use aggregate operators with SQLAlchemy as you would in a regular SQL query to get COUNT(comments) and MAX(timestamp) in the original query for messages? Or, is there another solution to this that I haven't explored yet? Ideally, I want to be able to do this all in one query. I looked through the SQLAlchemy documentation and couldn't find anything like this. Thanks!
For counting, you can try this (an example):
session.query(Comment).join(Post).filter_by(id=5).count()
or
sess.query(Comment).join(Post).filter(Post.id==5).count()
And, yes you can use aggregates:
sess.query(func.max(Comment.id)).join(Post).filter_by(id=5).all()
or
sess.query(func.max(Comment.id)).join(Post).filter(Post.id==5).all()