Develop Reference

SQL Integrity Error While adding a multiple values to a table in Flask App

I am creating a Flask App using MYSQL Database. The code seems legit from tutorial and online search I am following but the "Create user function didn't work properly. The error always comes back with missing user_id value although I am adding it. Help me through this I am stuck with over 5 days with this error
db = SQLAlchemy(app)
#database connection
class friends(db.Model):
__tablename__ = 'user'
user_id = db.Column(db.String, primary_key=True)
user_name = db.Column(db.String)
age = db.Column(db.Integer)
email = db.Column(db.String)
gender = db.Column(db.String)
def __init__(self, user_id, user_name, age, email,gender):
self.id = user_id
self.name = user_name
self.age = age
self.email = email
self.gender = gender
#route
#app.route('/add', methods=['GET', 'POST'])
def add():
if request.method == 'POST':
add_usr_id = request.form['new_usr_id']
add_usr_name = request.form['new_usr_name']
add_usr_age = request.form['new_usr_age']
add_usr_email = request.form['new_usr_email']
add_usr_gender = request.form['new_usr_gender']
commit_user = friends(user_id = add_usr_id, user_name = add_usr_name, age = add_usr_age, email = add_usr_email, gender = add_usr_gender)
db.session.add(commit_user)
db.session.commit()
message = "The record was added."
return render_template('add.html', message=message)
else:
return render_template('add.html')
#app.route('/show')
def read():
show_friend = friends.query.all()
return render_template('show.html', show_friend = show_friend)
This is the full error traceback :
sqlalchemy.exc.IntegrityError: (pymysql.err.IntegrityError) (1048, "Column 'user_name' cannot be null")
[SQL: INSERT INTO user (user_name, age, email, gender) VALUES (%(user_name)s, %(age)s, %(email)s, %(gender)s)]
[parameters: {'user_name': None, 'age': '32', 'email': 'dfak#ekfl.com', 'gender': 'male'}]
(Background on this error at: https://sqlalche.me/e/14/gkpj)

> add_usr_id = request.form['new_usr_id']
Add a debug print or logging statement there.
It will reveal that None was passed in.
Either arrange for a better value to be
presented, or hack in some default value.
Here is one way to default it:
add_usr_id = request.form['new_usr_id'] or 'default_user'

Flask Webapp - Verify Email after Registration - Best Practice

I've been following along to Corey Schafer's awesome youtube tutorial on the basic flaskblog. In addition to Corey's code, I`d like to add a logic, where users have to verify their email-address before being able to login. I've figured to do this with the URLSafeTimedSerializer from itsdangerous, like suggested by prettyprinted here.
The whole token creation and verification process seems to work. Unfortunately due to my very fresh python knowledge in general, I can't figure out a clean way on my own how to get that saved into the sqlite3 db. In my models I've created a Boolean Column email_confirmed with default=False which I am intending to change to True after the verification process. My question is: how do I best identify the user (for whom to alter the email_confirmed Column) when he clicks on his custom url? Would it be a good practice to also save the token inside a db Column and then filter by that token to identify the user?
Here is some of the relevant code:
User Class in my modely.py
class User(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(20), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
image_file = db.Column(db.String(20), nullable=False, default='default_profile.jpg')
password = db.Column(db.String(60), nullable=False)
date_registered = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
email_confirmed = db.Column(db.Boolean(), nullable=False, default=False)
email_confirm_date = db.Column(db.DateTime)
projects = db.relationship('Project', backref='author', lazy=True)
def get_mail_confirm_token(self, expires_sec=1800):
s = URLSafeTimedSerializer(current_app.config['SECRET_KEY'], expires_sec)
return s.dumps(self.email, salt='email-confirm')
#staticmethod
def verify_mail_confirm_token(token):
s = URLSafeTimedSerializer(current_app.config['SECRET_KEY'])
try:
return s.loads(token, salt='email-confirm', max_age=60)
except SignatureExpired:
return "PROBLEM"
Registration Logic in my routes (using a users blueprint):
#users.route('/register', methods=['GET', 'POST'])
def register():
if current_user.is_authenticated:
return redirect(url_for('dash.dashboard'))
form = RegistrationForm()
if form.validate_on_submit():
hashed_password = bcrypt.generate_password_hash(form.password.data).decode('utf-8')
user = User(username=form.username.data, email=form.email.data, password=hashed_password)
db.session.add(user)
db.session.commit()
send_mail_confirmation(user)
return redirect(url_for('users.welcome'))
return render_template('register.html', form=form)
#users.route('/welcome')
def welcome():
return render_template('welcome.html')
#users.route('/confirm_email/<token>')
def confirm_email(token):
user = User.verify_mail_confirm_token(token)
current_user.email_confirmed = True
current_user.email_confirm_date = datetime.utcnow
return user
The last parts current_user.email_confirmed = True and current_user.email_confirm_date =datetime.utcnow are probably the lines in question. Like stated above the desired entries aren't made because the user is not logged in at this stage, yet.
Im grateful for any help on this!
Thanks a lot in advance!

Thanks to #exhuma. Here is how I eventually got it to work - also in addition I'm posting the previously missing part of email-sending.
User Class in my models.py
class User(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(20), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
image_file = db.Column(db.String(20), nullable=False, default="default_profile.jpg")
password = db.Column(db.String(60), nullable=False)
date_registered = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
email_confirmed = db.Column(db.Boolean(), nullable=False, default=False)
email_confirm_date = db.Column(db.DateTime)
projects = db.relationship("Project", backref="author", lazy=True)
def get_mail_confirm_token(self):
s = URLSafeTimedSerializer(
current_app.config["SECRET_KEY"], salt="email-comfirm"
)
return s.dumps(self.email, salt="email-confirm")
#staticmethod
def verify_mail_confirm_token(token):
try:
s = URLSafeTimedSerializer(
current_app.config["SECRET_KEY"], salt="email-confirm"
)
email = s.loads(token, salt="email-confirm", max_age=3600)
return email
except (SignatureExpired, BadSignature):
return None
Send Mail function in my utils.py
def send_mail_confirmation(user):
token = user.get_mail_confirm_token()
msg = Message(
"Please Confirm Your Email",
sender="noreply#demo.com",
recipients=[user.email],
)
msg.html = render_template("mail_welcome_confirm.html", token=token)
mail.send(msg)
Registration Logic in my routes.py (using a users blueprint):
#users.route("/register", methods=["GET", "POST"])
def register():
if current_user.is_authenticated:
return redirect(url_for("dash.dashboard"))
form = RegistrationForm()
if form.validate_on_submit():
hashed_password = bcrypt.generate_password_hash(form.password.data).decode(
"utf-8"
)
user = User(
username=form.username.data, email=form.email.data, password=hashed_password
)
db.session.add(user)
db.session.commit()
send_mail_confirmation(user)
return redirect(url_for("users.welcome"))
return render_template("register.html", form=form)
#users.route("/welcome")
def welcome():
return render_template("welcome.html")
#users.route("/confirm_email/<token>")
def confirm_email(token):
email = User.verify_mail_confirm_token(token)
if email:
user = db.session.query(User).filter(User.email == email).one_or_none()
user.email_confirmed = True
user.email_confirm_date = datetime.utcnow()
db.session.add(user)
db.session.commit()
return redirect(url_for("users.login"))
flash(
f"Your email has been verified and you can now login to your account",
"success",
)
else:
return render_template("errors/token_invalid.html")
Only missing from my point of view is a simple conditional logic, to check if email_confirmed = True before logging in, as well as the same check inside the confirm_email(token) function to not make this process repeatable in case the user clicks on the confirmation link several times. Thanks again! Hope this is of some help to anyone else!

The key to your question is this:
My question is: how do I best identify the user (for whom to alter the email_confirmed Column) when he clicks on his custom url?
The answer can be seen in the example on URL safe serialisation using itsdangerous.
The token itself contains the e-mail address, because that's what you are using inside your get_mail_confirm_token() function.
You can then use the serialiser to retrieve the e-mail address from that token. You can do that inside your verify_mail_confirm_token() function, but, because it's a static-method you still need a session. You can pass this in as a separate argument though without problem. You also should treat the BadSignature exception from itsdangerous. It would then become:
#staticmethod
def verify_mail_confirm_token(session, token):
s = URLSafeTimedSerializer(current_app.config['SECRET_KEY'])
try:
email = s.loads(token, salt='email-confirm', max_age=60)
except (BadSignature, SignatureExpired):
return "PROBLEM"
user = session.query(User).filter(User.email == email).one_or_none()
return user
Would it be a good practice to also save the token inside a db Column and then filter by that token to identify the user?
No. The token should be short-lived and should not be kept around.
Finally, in your get_mail_confirm_token implementation you are not using the URLSafeTimedSerializer class correctly. You pass in a second argument called expires_sec, but if you look at the docs you will see that the second argument is the salt, which might lead to unintended problems.

Sql Alchemy: Child object doesn't load during before_commit event listener

I am working on configuring my app to automatically update an elasticsearch (es) index anytime specified database tables/columns are updated. I'm basing it off Michael Grinberg's Flask Megatutorial. I am using a SQLAlchemy session event listeners (before_commit and after_commit) to collect the changes and determine when to update the es index.
The table I am indexing is a user table that has an address table as its child. Besides indexing the user's name, I also want to index their city, which comes from the address table. My issue is that address field populates as None, instead of loading the address. When creating a new user (along with its address), I receive the following error: 'This session is in 'committed' state; no further SQL can be emitted within this transaction.' when I try to index the city value.
I have also tried using the after_flush_postexec event, with the same results. Do you have any suggestions for configurations to load the child relationship in either the 'after_commit' or the 'after_flush_postexec' events?
models.py
class SearchableMixin(object):
#classmethod
def before_commit(cls, session):
session._changes = {
'add': list(session.new),
'update': list(session.dirty),
'delete': list(session.deleted)
}
#classmethod
def after_commit(cls, session):
for obj in session._changes['add']:
if isinstance(obj, SearchableMixin):
#do stuff to add User name and city to index
for obj in session._changes['update']:
if isinstance(obj, SearchableMixin):
# do stuff to update User name and city in index
for obj in session._changes['delete']:
if isinstance(obj, SearchableMixin):
# do stuff to delete User name and city from index
session._changes = None
class User(SearchableMixin, db.model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
address = db.relationship("Address", backref="user", lazy='joined', uselist=False)
#hybrid_property
def city(self):
return self.address.city
class Address(db.Model):
id = db.Column(db.Integer, primary_key=True)
city = db.Column(db.String(50))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))

Here's the solution I came up with. While solving it, it presented another issue for which I've asked a [separate question][1]. However, while far from perfect, it does seem to work.
Instead of using sqlalchemy event hooks, I first subclassed db.model, adding in a CRUD mixin that adds create, update, save, delete functions. Then in the Searchable Mixin, I overrode those create, update, and delete functions to call the appropriate indexing function.
models.py
class CRUDMixin(object):
#classmethod
def create(cls, **kwargs):
"""Create a new record and save it the database."""
instance = cls(**kwargs)
saved = instance.save()
return saved
def update(self, commit=True, **kwargs):
for attr, value in kwargs.items():
if value != getattr(self, attr):
setattr(self, attr, value)
return commit and self.save() or self
def save(self, commit=True):
db.session.add(self)
if commit:
try:
db.session.commit()
except Exception:
db.session.rollback()
raise
return self
def delete(self, commit=True):
"""Remove the record from the database."""
db.session.delete(self)
return commit and db.session.commit()
class Model(CRUDMixin, db.Model):
__abstract__ = True
class SearchableMixin(object):
#classmethod
def create(cls, **kwargs):
try:
new = super().create(**kwargs)
add_to_index(new)
db.session.commit()
return new
except Exception:
raise
def update(self, commit=True, **kwargs):
try:
super().update(commit, **kwargs)
add_to_index(self)
db.session.commit()
return self
except Exception:
raise
def delete(self, commit=True):
try:
super().delete(commit)
remove_from_index(self)
db.session.commit()
return None
except Exception:
raise
class User(SearchableMixin, Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
address = db.relationship("Address", backref="user", lazy='joined', uselist=False)
#hybrid_property
def city(self):
return self.address.city
class Address(Model):
id = db.Column(db.Integer, primary_key=True)
city = db.Column(db.String(50))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
[1]: https://stackoverflow.com/questions/62722027/sql-alchemy-wont-drop-tables-at-end-of-test-due-to-metadata-lock-on-db

SQLAlchemy: How do I find second degree relationships in the database?

I'm writing a permission system for my Flask app, and I'm having problems figuring out how to find relationships in the database. My github repo, if you want to see everything. The decorator is intended to limit access to decorated views.
def user_has(attribute):
"""
Takes an attribute (a string name of either a role or an ability) and returns the function if the user has that attribute
"""
def wrapper(func):
#wraps(func)
def inner(*args, **kwargs):
attribute_object = Role.query.filter_by(name=attribute).first() or \
Ability.query.filter_by(name=attribute).first()
if attribute_object in current_user.roles or attribute in current_user.roles.abilities.all():
return func(*args, **kwargs)
else:
# Make this do someting way better.
return "You do not have access"
return inner
return wrapper
I'm using SQLAlchemy and storing Users, Roles, and Abilities in the database. Users may have one or more roles. Roles may have one or more abilities. I want to take the string passed to the decorator and check if the user has that role or if one of the user's roles has that ability. The decorator shouldn't care whether it has been called with a role or ability argument.
Apparently, this method (current_user.roles.abilities.all()) does not work to get through my relational database as I'm attempting to do here to find abilities. I get an error message:
AttributeError: 'InstrumentedList' object has no attribute 'abilities'
How can I compare the string argument with my current user's abilities which are derived from his/her roles?
For reference, my models:
user_role_table = db.Table('user_role',
db.Column(
'user_id', db.Integer, db.ForeignKey('user.uid')),
db.Column(
'role_id', db.Integer, db.ForeignKey('role.id'))
)
role_ability_table = db.Table('role_ability',
db.Column(
'role_id', db.Integer, db.ForeignKey('role.id')),
db.Column(
'ability_id', db.Integer, db.ForeignKey('ability.id'))
)
class Role(db.Model):
__tablename__ = 'role'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(120), unique=True)
abilities = db.relationship(
'Ability', secondary=role_ability_table, backref='roles')
def __init__(self, name):
self.name = name.lower()
def __repr__(self):
return '<Role {}>'.format(self.name)
def __str__(self):
return self.name
class Ability(db.Model):
__tablename__ = 'ability'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(120), unique=True)
def __init__(self, name):
self.name = name.lower()
def __repr__(self):
return '<Ability {}>'.format(self.name)
def __str__(self):
return self.name
class User(db.Model):
__tablename__ = 'user'
uid = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True)
pwdhash = db.Column(db.String(100))
roles = db.relationship('Role', secondary=user_role_table, backref='users')
def __init__(self, email, password, roles=None):
self.email = email.lower()
# If only a string is passed for roles, convert it to a list containing
# that string
if roles and isinstance(roles, basestring):
roles = [roles]
# If a sequence is passed for roles (or if roles has been converted to
# a sequence), fetch the corresponding database objects and make a list
# of those.
if roles and is_sequence(roles):
role_list = []
for role in roles:
role_list.appen(Role.query.filter_by(name=role).first())
self.roles = role_list
# Otherwise, assign the default 'user' role. Create that role if it
# doesn't exist.
else:
r = Role.query.filter_by(name='user').first()
if not r:
r = Role('user')
db.session.add(r)
db.session.commit()
self.roles = [r]
self.set_password(password)
def set_password(self, password):
self.pwdhash = generate_password_hash(password)
def check_password(self, password):
return check_password_hash(self.pwdhash, password)
def is_authenticated(self):
return True
def is_active(self):
return True
def is_anonymous(self):
return False
def get_id(self):
return unicode(self.uid)
def __repr__(self):
return '<User {}>'.format(self.email)
def __str__(self):
return self.email
and the decorated view:
#app.route('/admin', methods=['GET', 'POST'])
#user_has('admin')
def admin():
users = models.User.query.all()
forms = {user.uid: RoleForm(uid=user.uid, roles=[role.id for role in user.roles])
for user in users}
if request.method == "POST":
current_form = forms[int(request.form['uid'])]
if current_form.validate():
u = models.User.query.get(current_form.uid.data)
u.roles = [models.Role.query.get(role)
for role in current_form.roles.data]
db.session.commit()
flash('Roles updated for {}'.format(u))
return render_template('admin.html', users=users, forms=forms)

The solution ended up being dead simple. I feel a bit silly not knowing it from the start.
user_abilities = []
for role in current_user.roles:
user_abilities += [role.ability for ability in role.abilities]
I still feel there's probably a better pattern for this, but the solution works without a hitch.

Any chance this doesn't work because you use attribute instead of attribute_object in the second clause of your if statement?
Instead of this:
if attribute_object in current_user.roles or attribute in current_user.roles.abilities.all():
Try this:
if attribute_object in current_user.roles or attribute_object in current_user.roles.abilities.all():

Convert variable into class variable (no idea what to name it)

Using
Flask
Flask-sqlalchemy
Sqlalchemy
Jquery
Datatables (jquery plugin)
Jeditable (jquery plugin)
Consider this user class ( straight out of flask-sqlalchemy docs):
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
def __init__(self, username, email):
self.username = username
self.email = email
def __repr__(self):
return '<User %r>' % self.username
The datatables makes an ajax request and populates the table. Each td then is made editable in place with jeditable. As soon as a td is modified, jeditable makes a POST request to localhost/modify containing:
The row id(the same id from the user class)
The new modified value
The column of the table that was altered ( for the sake of argument let's assume that there are three columns id/username/email) (int)
Now, i'd like that in the function that handles localhost/modify i take the row id, make a user object and query the db for that specific row, see what property needs to be modified, modify it with the new value and commit to the db. Here's what i got so far:
#app.route('/modify', methods=['GET', 'POST'])
def modify()
if request.method == 'POST' :
user = user.query.get(form.row_id)
if form.column_id == int(0):
user.id = form.value
elif form.column_id == int(1):
user.username = form.value
elif form.column_id == int(2):
user.email = form.value
else:
pass
db.session.commit()
return 'ok'
This way, yes it does work but theremust be amore beautiful approach. This one doesn't seem very...pythonic
Mihai

Use a map of column ID to attribute name.
colmap = {
0: 'id',
1: 'username',
2: 'email',
}
#app.route('/modify', methods=['GET', 'POST'])
def modify()
if request.method == 'POST' :
user = user.query.get(form.row_id)
try:
setattr(user, colmap[form.column_id], form.value)
except KeyError:
pass
db.session.commit()
return 'ok'