Develop Reference

How do I set the value of a PasswordType property?

I've got a User model defined as such:
class User(BaseMixin, db.Model, DictSerializable):
__tablename__ = 'user'
email = db.Column(EmailType, nullable=False)
username = db.Column(db.Unicode(255), nullable=False)
password = db.Column(PasswordType(
schemes=[
'pbkdf2_sha512',
],
))
def __init__(self, email, password):
self.email = email
self.username = email
self.password = password
And all I want to do is set a password via the flask shell (via docker) for a user that already existed before I created the __init__ method.
I've come to realize I can't just do the ol,
from project_name.models import db, User
user = User.query.first()
user.password = 'foo'
db.session.commit()
So please, can anyone help me?

The docs for PasswordType say it relies on sqlalchemy_utils.force_auto_coercion being run before you define your model:
from sqlalchemy_utils import force_auto_coercion
force_auto_coercion()
...
class User(db.Model):
...
Once you do that, the value will actually persist.

SQL Alchemy Error when moving colums to init method

This is my User object that I'm using to write to MySQL using SQLAlchemy
class User(Base):
def __init__(self):
self.id = Column(Integer, primary_key=True)
self.first_name = Column(String)
self.last_name = Column(String)
self.email_id = Column(String)
self.mobile = Column(String)
self.username = Column(String)
self.hashed_password = Column(String)
def set_first_name(self, first_name):
self.first_name = first_name
def set_last_name(self, last_name):
self.last_name = last_name
def set_email_id(self, email):
self.email_id = email
def set_mobile(self, mobile):
self.mobile = mobile
def set_username(self, username):
self.username = username
def set_hashed_password(self, password):
self.hashed_password = password
def __repr__(self):
return "<User(id=%d, first_name=%s, last_name=%s, email_id=%s, mobile=%s, username=%s)>"%(self.id, self.first_name, self.last_name, self.email_id, self.mobile, self.username)
When I run the program, this is what I get the following error,
sqlalchemy.exc.ArgumentError: Mapper Mapper|User|user could not assemble any primary key columns for mapped table 'user'
This code works if I take the attribute definitions out of the init and remove the self prefix. Can someone help me understand what's going on here?

SQLAlchemy's declarative base mechanism establishes a Python metaclass. That means that SQLAlchemy will specially process the definition of your class.
The purpose of that processing is to construct an sqlalchemy.orm.Mapper for each mapped class. That mapper represents the mapping between your database tables and your class.
In order to do that, SQLAlchemy generally needs to be able to find a primary key. This is required in order to define the identity associated with each mapped instance, so that mapped objects can be cached/found in sessions. That at least needs to be possible when your mapped class is constructed.
That means that you need to define the column of at least the primary key on the class.
Other answers have explained that much, although I think I've provided a bit more detail.
There is a more fundamental problem though.
id = Column(Integer, primary_key = True)
is of course a call to the Column function you import from SQLAlchemy. However, the return from the Column function is a schema item. This schema item is converted by declarative base into a descriptor similar to the kind of descriptor that the property decorator gives you. Such descriptors only work on a class, not an instance of that class.
Let's say I have a class mapped to a table called User and an instance of that user in a variable bob.
User.id
Is a description of the identity column. However
bob.id
is the number that identifies Bob in the users table.
That is, columns aren't intended to be assigned to members of self, they are intended to be assigned to classes.
So:
You need to have at least the primary key column on your class when you define it.
It's generally a good idea to have all your Columns there.
You can add a Column definition to your class later, although things will only work if you arrange for that column to get into your table
It's always wrong to add a Column to an instance of a mapped class. self.x = Column is always wrong.

The SQLAlchemy ORM (almost) always requires a primary key. You have indeed defined one inside your __init__() function. The problem is that __init__() doesn't get called until you create a User object. I assume you create your database before a User object ever gets created. Thus as far as the SQLAlchemy ORM is concerned, a primary key does not exist for User (nor any of the other attributes declared inside __init__).
The solution, as I think you already found from your last line, is to declare them as class attributes where as soon you do something like from models import User the attributes are defined and SQLAlchemy can properly build your User table.
class User(Base):
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
email_id = Column(String)
mobile = Column(String)
username = Column(String)
hashed_password = Column(String)
def set_first_name(self, first_name):
self.first_name = first_name
...

You don't have to declare the column definitions inside the __init__() function. Change your class definition to something like this and it should work:
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
email_id = Column(String)
mobile = Column(String)
username = Column(String)
hashed_password = Column(String)

Entity Relational Model Design and Query Method

Current Design:
Users and Groups: Many-to-Many Table
Tools: SQLAlchemy 1.0.9, Python 2.7, Pyramid
The two questions I pose to you:
I am uncertain if I should have the group attribute under User or the user attribute under Group. This is a MANY-TO-MANY relationship; with a third table that relates the two different classes.
SQLAlchemy has a lot of great examples discussing .joins, but I'm finding the task is difficult with this DB design for querying based on the selected username. (see below)
I have designed a SQLAlchemy database and am trying to implement best practices for retrieving data from my table while eliminating redundancy. With that said, I also want this to be an effective design so that when I build a group_finder function using Pyramid's Authorization and Authentication System, so that I have no issue.
I am using a CRUD methodology. As of right now: To add a user to a Group, I update_group_add_user(...) to add a user or update_group_remove_users(...) to remove a user.
base.py
Commented out groups and added users to class Group
association_table = Table('group_user_link', Base.metadata,
Column('group_id', Integer, ForeignKey('groups.id')),
Column('user_id', Integer, ForeignKey('users.id')))
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
username = Column(String(15), nullable=False, unique=True)
email = Column(String(300))
password = Column(String(300), nullable=False)
#groups = relationship('Group', secondary='group_user_link', backref='users')
def __init__(self, username, password, firstname, lastname, email):
self.username = username
self.password = hashlib.sha224(password).hexdigest()
self.email = email
#self._groups = groups
def __repr__(self): # code
class Group(Base):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
group_name = Column(String(50), unique=True)
users = relationship('User', secondary='group_user_link', backref='groups')
def __init__(self, group_name, user=None):
if user is None: user = []
self.group_name = group_name
self._user = user # to group_add_user and group_remove_user
def __repr__(self): # code
Query method (using CRUD):
This prints out ALL the relationships in tuples in a list. I want to only print out (and return) only the user being entered.
def retrieve_user_bygroup(self, username):
query= self.session.query(User, Group).join(association).\
filter(User.id == Group.id).\
order_by(User.id).all()
print "retrieve user by group:", query
return query

I discovered two similarly asked questions that clarified the use of backref, .join, and .query in SQLAlchemy. This helped clarify how the backref line in users = relationship('User', secondary='group_user_link', backref='groups') made groups accessible through User when later querying with a .join and .filter (shown at the bottom).
Discussing the purpose of backref:
backref means a 'poster' attribute will be added to UserPost. Actually
I'd probably call that 'user'. But the point is that 'relation()'
knows how to join between those two tables because of the previously
defined ForeignKey.
Discussing .join and .query:
.join [works] according [to] the relations, and yields 3-tuples. The arguments to
the query() call are essentially the select list in sqlalchemy.
I also found this Stacks answer to be helpful with using .any()
def retrieve_group_byuser(self, username):
# retrieves group by username
query = self.session.query(Group).\
filter(Group.users.any(User.username == username)).all()
print "retrieved by username:", query
return query
And I removed groups from User which now looks like this:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
username = Column(String(15), nullable=False, unique=True)
email = Column(String(300))
password = Column(String(300), nullable=False)
def __init__(self, username, password, firstname, lastname, email):
self.username = username
self.password = hashlib.sha224(password).hexdigest()
self.email = email
def __repr__(self):
# code

I would say group should contain [say a set of] users in it. Not the other way round.
Now, you might have a need to know the group given the user, if you want to cache this mapping in memory (instead of getting it from DB) then I would recommend NOT to create a circular dependency between group and user classes. Instead promote the that dependency to a third class, which just defines the relation between group and users.
E.g.
class user:
def __init__(self, name):
self.name = name
class group:
def __init__(self, name):
self.name = name
class user_group_relation:
def get_users(self, grp_name):
# could get it from an internal map (like return g2u_mapping[grp_name]) or
# run a query on some DB table..
pass
def get_group(self, usr_name):
# could get it from an internal map or run a query on some DB table..
pass
# group to user mapping
u2g_mapping = {user('user1'): group('group1'), user('user2'): group('group1')}
g2u_mapping = {group('group1'): [user('user1'), user('user2')]}
if this is e.g. a representation of normal user authentication mechanism then only users have passwords, groups don't. Groups just represent a way to organizing things. Hence the password field (and any related functionality should remain in user and not promoted to base.

Flask-SQLAlchemy Constructor Confusion

Once again this code is from Miguel Grindberg's book "Flask Web Development". In models.py we have 3 classes, a Role class which has 3 roles (User, Moderator, Administrator), a User class (id, username, email, role_id, password_hash, confirmed), and a Permsission class (code below). In chp 9 page 114, he adds some code to the User class to check if the email address belongs to the admin and if so adds it to the role. If not, the user is added to the default role (user). . .
def __init__(self, **kwargs):
super(User, self).__init__(**kwargs)
if self.role is None:
if self.email == current_app.config['FLASKY_ADMIN']:
self.role = Role.query.filter_by(permissions=0xff).first()
if self.role is None:
self.role = Role.query.filter_by(default=True).first()
My question is, why do we need a constructor for this code? A constructor isn't used in any other part of the file (full code below), so why do we need one now? I've looked at this question on Stack (Flask-SQLAlchemy Constructor) which shed some light on the subject as far as the base class constructor, but NOT why I need a constructor at all for this piece of code. .Again, THANKS for any help.
class Permission:
FOLLOW = 0x01
COMMENT = 0x02
WRITE_ARTICLES = 0x04
MODERATE_COMMENTS = 0x08
ADMINISTER = 0x80
class Role(db.Model):
__tablename__ = 'roles'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(64), unique=True)
default = db.Column(db.Boolean, default=False, index=True)
permissions = db.Column(db.Integer)
users = db.relationship('User', backref='role', lazy='dynamic')
#staticmethod
def insert_roles():
roles = {
'User': (Permission.FOLLOW |
Permission.COMMENT |
Permission.WRITE_ARTICLES, True),
'Moderator': (Permission.FOLLOW |
Permission.COMMENT |
Permission.WRITE_ARTICLES |
Permission.MODERATE_COMMENTS, False),
'Administrator': (0xff, False)
}
for r in roles:
role = Role.query.filter_by(name=r).first()
if role is None:
role = Role(name=r)
role.permissions = roles[r][0]
role.default = roles[r][1]
db.session.add(role)
db.session.commit()
def __repr__(self):
return '<Role %r>' % self.name
class User(UserMixin, db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(64), unique=True, index=True)
username = db.Column(db.String(64), unique=True, index=True)
role_id = db.Column(db.Integer, db.ForeignKey('roles.id'))
password_hash = db.Column(db.String(128))
confirmed = db.Column(db.Boolean, default=False)
def __init__(self, **kwargs):
super(User, self).__init__(**kwargs)
if self.role is None:
if self.email == current_app.config['FLASKY_ADMIN']:
self.role = Role.query.filter_by(permissions=0xff).first()
if self.role is None:
self.role = Role.query.filter_by(default=True).first()

In typical OO lingo a closer "constructor" equivalent in Python would be a __new__ method rather than __init__. Although to be frank that does not map very well to Python concepts really: you have __new__ method for object creation and __init__ is typically used for initialization. So maybe this is a "split" constructor?
Anyway, .__init__ is where in Python we typically adjust instance's attributes, so super() method above does initialization for User superclass and remainder of child class's __init__ does initialization for the child class. You have to do things like say implementing some logic that depends keyword attributes, etc, somewhere - either you do it outside the class (which violates principle of encapsulation) or you do it in some method inside a class or instance. __init__ is typical and I'm rather astonished thinking why someone would NOT use it.
You also list some SQLAlchemy object with things like declarative __tablename__ class attribute that do not use __init__. Well, zzzeek is a black magician doing wicked and morally controversial stuff so he has risen above peon's __init__ methods that us mere mortals use every day. ;-)

def __init__(self, **kwargs):
super(User, self).__init__(**kwargs)
if self.role is None:
if self.email == current_app.config['FLASKY_ADMIN']:
self.role = Role.query.filter_by(permissions=0xff).first()
if self.role is None:
self.role = Role.query.filter_by(default=True).first()
I think there have some kind of preprocessing and that superclassing is for ealrier checking. So everytime when we querying a User this check will happen.

Whenever a new User instance is created, the constructor assigns a role to that User instance. If this constructor wasn't created, then every time a User instance was created, you would have to assign it a role somewhere else (such as in a route). But that assumes that User instances will be created only within your app if a form is submitted and/or a route is called. This can be messy and leaves holes for some Users to have a None role, which if your app isn't expecting could let users perform actions on the site they are not allowed to do.
By default Flask-SQLAlchemy uses SQLAlchemy's base class defines the constructor (per Flask-SQLAlchemy Constructor). This is why you can create models in Flask-SQLAlchemy without ever having to create a constructor, because it is created by default.
So if you wanted to do something specific in the creation of a model instance (such as assign a value to a column based on some logic), then you need to create the constructor yourself but can use super(MODEL, self).__init__(**kwargs) to save you time in typing out the N columns in that model, since super is inheriting the base class.

How do I extend a SQLAlchemy bound declarative model with extra methods?

For example, I have a declarative class on module a:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relationship("Address", backref="user")
Now, in module b I want to use the mapped entity, but add a method:
from a import User
class UserWithExtraMethod(User):
def name_capitalized(self):
return self.name.capitalize()
user = UserWithExtraMethod()
print(user.name_capitalized)
However, when I run the script, I will get the following error:
InvalidRequestError: Multiple classes found for path "User" in the registry of this declarative base. Please use a fully module-qualified path.
What have I missed when declaring the user entity? I would like to reuse the previous declared entity.
I am expecting something would be like:
class UserWithExtraMethod(User):
___magic_reuse_previous_mapper__ = True
def name_capitalized(self):
return self.name.capitalize()

Unless you've got a particular reason to have separate classes, you should just write:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relationship("Address", backref="user")
def name_capitalized(self):
return self.name.capitalize()
Since the name_capitalized is not special as far as SQLAlchemy is concerned (it's not a ColumnExpression or some such), it is completely ignored by the mapper.
Actually, there's an even better way to do this; your version works fine for instances of User, but is of no use in sql expressions.
from sqlalchemy.ext.hybrid import hybrid_property, hybrid_method
class User(Base):
# ... body as before
#hybrid_method
def name_capitalized(self):
return self.name.capitalize()
#name_capitalized.expression
def name_capitalized(cls):
# works for postgresql, other databases spell this differently.
return sqlalchemy.func.initcap(cls.name)
which will allow you to do things like:
>>> print Query(User).filter(User.name_capitalized() == "Alice")
SELECT users.id AS users_id, users.name AS users_name
FROM users
WHERE initcap(users.name) = :initcap_1

Perhaps a little late for this reply. Do you have any other relationships setup that are pointing to User?
For example, if you have Address defined as:
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
address = Column(String(50))
Users = relationship("User", backref="addresses")
when Address is trying to resolve to which User within the declarative base to point to, it will find two of them. To verify try Base._decl_class_registry['User']. This is similar to this topic covered by Michael.
In ./sqlalchemy/ext/declarative/clsregistry.py there is an example on how to use the fully qualified path. In this case it would be changing the relationship within address from Users = relationship("User", backref="addresses") to
Users = relationship("a.User", backref="addresses")
Hope this helps point you in the right direction for debugging.

Hacky, but why not just monkey-patch the User class for your purpose instead of inheriting from it?
# modude b
from a import User
def name_capitalized(self):
return self.name.capitalize()
User.name_capitalized = name_capitalized
user = User() # and it has extra-method as well
print(user.name_capitalized)

This may not work for you. I had a similar issue. I ended up passing an instance of User to UserWithExtraMethod during instantiation
class UserWithExtraMethod(object):
def __init__(self, user):
self.user = user
def name_capitalized(self):
return self.user.name.capitalize()
Hope this helps