Let's say I have the following structure (using Flask-SqlAlchemy):
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False, index=True)
# The following line throws an error at runtime.
variant = db.Column(db.Integer, nullable=False, index=True,
default=select(func.count(User.id)).where(User.name == self.name))
def __init__(self, name):
super(User, self).__init__()
self.name = name
#property
def clause(self):
return '/'.join([str(self.variant), self.name])
Problem is, "User is not defined." I would like to model a system with Users who may choose the same name but add a field to differentiate between users in a systemic way without using (thereby exposing) the "id" field.
Anyone know how to make a self-referential query to use to populate a default value?
The issue of the default not referring to User here is solved by just assigning "default" to the Column once User is available. However, that's not going to solve the problem here because "self" means nothing either, there is no User method being called so you can't just refer to "self". The challenge with this statement is that you want it to be rendered as an inline sub-SELECT but it still needs to know the in-memory value of ".name". So you have to assign that sub-SELECT per-object in some way.
The usual way people approach ORM-level INSERT defaults like this is usually by using a before_insert handler.
Another way that's worth pointing out is by creating a SQL level INSERT trigger. This is overall the most "traditional" approach, as here you need to have access to the row being inserted; triggers define a means of getting at the row values that are being inserted.
As far as using a default at the column level, you'd need to use a callable function as the default which can look at the current value of the row being inserted, but at the moment that means that your SELECT statement will not be rendered inline with the INSERT statement, you'd need to pre-execute the SELECT which is not really what we want here.
Anyway, the basic task of rendering a SQL expression into the INSERT while also having that SQL expression refer to some local per-object state is achieved by assigning that expression to the attribute, the ORM picks up on this at flush time. Below we do this in the constructor, but this can also occur inside of before_insert() as well:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, index=True)
variant = Column(Integer, nullable=False, index=True)
def __init__(self, name):
self.name = name
self.variant = select([func.count(User.id)]).where(User.name == self.name)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
s.add(User(name='n1'))
s.commit()
s.add(User(name='n1'))
s.commit()
print s.query(User.variant).all()
Related
I’m writing a sqlalchemy based python app. I want to override the sqlalchemy init method which will accept a primary key and then init it’s own instance.
Something like this:
class User(Base):
id = Column(String, primary_key=True)
name = Column(String)
def __init__(id):
self = session.query(User).filter(User.id=id).first()
I know I can initialize the object using session.query, but I want to export a nice simple api that will be used by other users (It’s going to be a SDK).
Any ideas? Thanks!
You can do this via the init method, though I would recommend against it.
There are two issues with your code snippet.
You forgot to include the self argument as the first argument of the __init__ method
Assigning to self would not work, but you can replace the dictionary of self with that of other
putting these two things together:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
def __init__(self, user_id):
res = session.query(User).filter(User.id == user_id).first()
self.__dict__ = res.__dict__
However, I'd recommend adding a classmethod for this very specific but oft-repeated usage, i.e. getting an instance from the database by the primary_key
#classmethod
def get_by_id(cls, key):
return session.query(cls).filter(cls.id == key).first()
This way it is general to all your classes that have a single column primary key.
The usage for these versions would be
u1 = User(user_id=1)
u2 = User.get_by_id(key=1)
This is my User object that I'm using to write to MySQL using SQLAlchemy
class User(Base):
def __init__(self):
self.id = Column(Integer, primary_key=True)
self.first_name = Column(String)
self.last_name = Column(String)
self.email_id = Column(String)
self.mobile = Column(String)
self.username = Column(String)
self.hashed_password = Column(String)
def set_first_name(self, first_name):
self.first_name = first_name
def set_last_name(self, last_name):
self.last_name = last_name
def set_email_id(self, email):
self.email_id = email
def set_mobile(self, mobile):
self.mobile = mobile
def set_username(self, username):
self.username = username
def set_hashed_password(self, password):
self.hashed_password = password
def __repr__(self):
return "<User(id=%d, first_name=%s, last_name=%s, email_id=%s, mobile=%s, username=%s)>"%(self.id, self.first_name, self.last_name, self.email_id, self.mobile, self.username)
When I run the program, this is what I get the following error,
sqlalchemy.exc.ArgumentError: Mapper Mapper|User|user could not assemble any primary key columns for mapped table 'user'
This code works if I take the attribute definitions out of the init and remove the self prefix. Can someone help me understand what's going on here?
SQLAlchemy's declarative base mechanism establishes a Python metaclass. That means that SQLAlchemy will specially process the definition of your class.
The purpose of that processing is to construct an sqlalchemy.orm.Mapper for each mapped class. That mapper represents the mapping between your database tables and your class.
In order to do that, SQLAlchemy generally needs to be able to find a primary key. This is required in order to define the identity associated with each mapped instance, so that mapped objects can be cached/found in sessions. That at least needs to be possible when your mapped class is constructed.
That means that you need to define the column of at least the primary key on the class.
Other answers have explained that much, although I think I've provided a bit more detail.
There is a more fundamental problem though.
id = Column(Integer, primary_key = True)
is of course a call to the Column function you import from SQLAlchemy. However, the return from the Column function is a schema item. This schema item is converted by declarative base into a descriptor similar to the kind of descriptor that the property decorator gives you. Such descriptors only work on a class, not an instance of that class.
Let's say I have a class mapped to a table called User and an instance of that user in a variable bob.
User.id
Is a description of the identity column. However
bob.id
is the number that identifies Bob in the users table.
That is, columns aren't intended to be assigned to members of self, they are intended to be assigned to classes.
So:
You need to have at least the primary key column on your class when you define it.
It's generally a good idea to have all your Columns there.
You can add a Column definition to your class later, although things will only work if you arrange for that column to get into your table
It's always wrong to add a Column to an instance of a mapped class. self.x = Column is always wrong.
The SQLAlchemy ORM (almost) always requires a primary key. You have indeed defined one inside your __init__() function. The problem is that __init__() doesn't get called until you create a User object. I assume you create your database before a User object ever gets created. Thus as far as the SQLAlchemy ORM is concerned, a primary key does not exist for User (nor any of the other attributes declared inside __init__).
The solution, as I think you already found from your last line, is to declare them as class attributes where as soon you do something like from models import User the attributes are defined and SQLAlchemy can properly build your User table.
class User(Base):
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
email_id = Column(String)
mobile = Column(String)
username = Column(String)
hashed_password = Column(String)
def set_first_name(self, first_name):
self.first_name = first_name
...
You don't have to declare the column definitions inside the __init__() function. Change your class definition to something like this and it should work:
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
email_id = Column(String)
mobile = Column(String)
username = Column(String)
hashed_password = Column(String)
Quick Summary: I want to have an ordered list of Addresses in SQLAlchemy.
But the order of my list changes when I commit.
Why does this happen and how can I change it?
Long explanation:
I start with a list of Address attached to a User object.
Then I replace the first element of the "addresses" list with a
new Address.
Then I print the list of addresses ... so far the order is what I would expect.
Finally I commit. After my commit I do a query but the order of my
addresses list has changed.
So is this just something about databasing in general that I don't understand? Or does a SQLAlchemy InstrumentedList not act like an actual list? I thought I could change the order of elements in a relationship but I don't see how.
from sqlalchemy import Column, Integer, String
from sqlalchemy import create_engine
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
Session = sessionmaker()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
fullname = Column(String(50))
password = Column(String(12))
addresses = relationship("Address", back_populates="user")
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>" % (
self.name, self.fullname, self.password)
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email_address = Column(String, nullable=False)
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship("User", back_populates="addresses")
def __repr__(self):
return "<Address(email_address='%s')>" % self.email_address
if __name__ == "__main__":
engine = create_engine('sqlite:///:memory:', echo=False)
Session.configure(bind=engine)
Base.metadata.create_all(engine)
session = Session()
user = User(name='ed', fullname='Ed Jones', password='edspassword')
user.addresses = [Address(email_address='jack#google.com'), Address(email_address='j25#yahoo.com')]
session.add(user)
session.commit()
user = session.query(User).filter_by(name='ed').first()
print("Current order of addresses list at start.")
print(user.addresses)
print()
new_primary_address = Address(email_address='primary#google.com')
user.addresses[0] = new_primary_address
print("Current order of addresses list before commit.")
print("But after chaning addresses[0].")
print(user.addresses)
print()
session.commit()
user = session.query(User).filter_by(name='ed').first()
print("Current order of addresses list after commit.")
print(user.addresses)
print()
print("Why is the order of the InstrumentedList not persistent?")
print("Isn't persistent order what makes a list a list?")
It is "databasing" in general. An InstrumentedList does act like an actual list with the added ORM instrumentation Python side, but when you commit the Session's default behaviour is to expire all database loaded state of ORM-managed attributes, and so the list has to be refreshed upon next access. This means that a SELECT such as
2017-05-21 13:32:31,124 INFO sqlalchemy.engine.base.Engine SELECT addresses.id AS addresses_id, addresses.email_address AS addresses_email_address, addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id
is issued to fetch the list contents. In SQL the order of a SELECT is unspecified, if not explicitly chosen, so you may or may not get the items in the same order as before. Also note that the ORM operation
user.addresses[0] = new_primary_address
translates to an UPDATE that sets the user_id of the old address tuple to NULL and INSERTs a new one in the table, so you'd not get the order you thought, even if the rows were returned in insertion order.
If the order of addresses matters to you, you must choose ordering. Use the order_by parameter of relationship:
class User(Base):
...
addresses = relationship("Address", back_populates="user",
order_by="Address.email_address")
would order the addresses by email address, when fetched. SQLAlchemy also provides (thank you for digging that up) a helper collection class for mutable ordered relationships: orderinglist, which helps managing index/position on changes, if used as the ordering.
It seems you'd like the order of addresses to signify which is the primary address of a user. A separate flag column would work for this better.
Here's a SQLAlchemy class:
class MyGroup(Base):
__tablename__ = 'my_group'
group_id = Column(Integer, Sequence('my_seq'), primary_key=True)
group_name = Column(String(200), nullable=False, index=True)
date_created = Column(DateTime, default=func.now())
date_updated = Column(DateTime, default=func.now(), onupdate=func.now())
Anytime I add a group_name or (for example) update the group_name, the date_updated field will get updated. That's great.
But sometimes there are cases where I want to mark a group as "updated" even if the group record itself did not change (for example, if data in a different but related table is updated).
I could do it manually:
group = session.query(MyGroup).filter(MyGroup.group_name=='Some Group').one()
group.date_updated = datetime.datetime.utcnow()
session.commit()
but I'd really rather let the model do it in its own way, rather than recreate a Python process to manually update the date. (For example, to avoid mistakes like where maybe the model uses now() while the manual function mistakenly uses utcnow())
Is there a way with SQLAlchemy to "touch" a record (kind of like UNIX touch) that wouldn't alter any of the record's other values but would trigger the onupdate= function?
Just to add to this answer, the documentation states the following:
The Column.default and Column.onupdate keyword arguments also accept Python functions. These functions are invoked at the time of insert or update if no other value for that column is supplied, and the value returned is used for the column’s value.
Key part being: are invoked at the time of insert or update if no other value for that column is supplied.
Key part of the key part: if no other value for that column is supplied
So with a simple update statement with empty values, does the trick:
from sqlalchemy import update
stmt = update(ModelName).where(ModelName.column.in_(column_values)).values()
db.engine.execute(update_product_mapping_info)
I am using the sqlalchemy.sql.expression.update for this, documentation here.
Here's the Model column definition I have:
from datetime import datetime
last_updated = Column(db.DateTime, onupdate=datetime.utcnow)
To show a complete example, building on #Chayemor's answer I did the following:
import sqlalchemy.sql.functions as func
from sqlalchemy import Column, Integer, String, update
from . import database as db
Base = declarative_base()
class Object(Base):
__tablename__ = "objects"
id = Column(Integer, primary_key=True, nullable=False)
last_update = Column(
DateTime,
server_default=func.now(),
onupdate=func.current_timestamp()
)
def touch(self):
stmt = update(Game).where(Game.id == self.id)
db.engine.execute(stmt)
From here, running obj.touch() updates its last_update field in the database without changing any other data.
Another way to do this is to call orm.attributes.flag_modified on an instance and attribute. SQLAlchemy will mark the attribute as modified even though it is unchanged and generate an update.
with Session() as s, s.begin():
mg = s.execute(sa.select(MyGroup)).scalar_one()
orm.attributes.flag_modified(mg, 'group_name')
Note that the "dummy" update will be included in the generated SQL's SET clause
UPDATE tbl
SET group_name=%(group_name)s,
date_updated=now()
WHERE tbl.group_id = %(tbl_group_id)s
in contrast with that generated by Chayemor's answer:
UPDATE tbl
SET date_updated=now()
WHERE tbl.group_name = %(group_name_1)s
This may be significant (consider triggers for example).
I haven't looked at the source but from the docs it seems that this is only triggered by issuing a SQL UPDATE command:
onupdate – A scalar, Python callable, or ClauseElement representing a default value to be applied to the column within UPDATE statements, which wil be invoked upon update if this column is not present in the SET clause of the update. This is a shortcut to using ColumnDefault as a positional argument with for_update=True.
If your concern is ensuring that your "touch" uses the same function as the onupdate function you could define a method on your model to perform the touch and have the onupdate parameter point this method.
I think something like this would work:
class MyGroup(Base):
__tablename__ = 'my_group'
group_id = Column(Integer, Sequence('my_seq'), primary_key=True)
group_name = Column(String(200), nullable=False, index=True)
date_created = Column(DateTime, default=func.now())
date_updated = Column(
DateTime,
default=self.get_todays_date,
onupdate=self.get_todays_date)
def get_todays_date(self):
return datetime.datetime.utcnow()
def update_date_updated(self):
self.date_updated = self.get_todays_date()
You could then update your record like this:
group.update_date_updated()
session.commit()
I am currently working with some legacy code that looks as follows:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Unicode
from sqlalchemy.dialects.postgresql import ARRAY, TEXT
Base = declarative_base()
class Book(Base):
__tablename__ = 'book'
id = Column(Integer, primary_key=True)
title = Column(Unicode)
keywords = Column('keywords', ARRAY(TEXT), primary_key=False)
The keywords are currently being kept as an array, but I'd like to flatten this out and have them be in their own separate model
class Keyword():
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
book_id = Column(Integer, ForeignKey('book.id', ondelete='cascade'),
nullable=False)
keyword = Column(Unicode)
How can I make it such that when a Book() is created, it also creates the
accompanying keywords? As an intermediate step for migrating the API, I'd like to keep the current array column, but also have the accompanying Keyword() instances be created.
I could do this within an __init__ method, but would need to know what the current Session() was, in order to run a commit. I could also perhaps use a property attribute, attached to keywords, but am not sure how that would work given that I am working with a class that inherits from SQLAlchemy's base, and not with a regular class that I have defined. What's the correct way to do this?
You can use object_session to find out the session of a given instance.
But if you define relationship between a Book and Keywords, you should not need even bother:
class Book(Base):
# ...
rel_keywords = relationship('Keyword', backref='book')
def init_keyword_relationship(self):
for kw in self.keywords:
self.rel_keywords.add(Keyword(keyword=kw))
sess = # ... get_session...
books = sess.query(Book).all()
for book in books:
book.init_keyword_relationship()
sess.commit()
However, I would do a migration once and get rid of the keywords array in order not to add a logic to keep those in sync.