SQLAlchemy provides a very clean interface for defining database tables:
engine = create_engine('sqlite:///:memory:')
metadata = MetaData()
user = Table('user', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('email_address', String(60), key='email'),
Column('password', String(20), nullable = False)
)
user_prefs = Table('user_prefs', metadata,
Column('pref_id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey("user.user_id"), nullable=False),
Column('pref_name', String(40), nullable=False),
Column('pref_value', String(100))
)
And once these tables have been defined, it is very easy to create these tables with the metadata.create_all(engine) function. This is especially nice for testing, when you want to create tables that will not interfere with the existing tables being used in production.
A project I am currently working on relies heavily on user defined functions in postgres. Is there a clean way to define these functions using SQLAlchemy in order for the metadata.create_all(engine) to properly create the functions along with the appropriate tables?
I've been working on something similar today. So far the best way I've found to do it is using sqlalchemy before_create event listeners. For general functions, you can bind the events to the metadata, but for table specific functions you could bind to the tables. For example:
import sqlalchemy
from sqlalchemy.schema import DDL
sqlalchemy.event.listen(
metadata,
'before_create',
DDL('CREATE OR REPLACE FUNCTION myfunc() ...')
)
You can just replace metadata with your table if you wanted to create the function before a table was created.
This DDL seems to run every time you call metadata.create_all so it's important to use CREATE OR REPLACE. If you wanted a bit more control over when functions are created, you might be better looking into migrations with alembic or similar.
Some uses of DDL are described in the sqlalchemy docs here: http://docs.sqlalchemy.org/en/latest/core/ddl.html#custom-ddl
Related
I am having problems understanding the benefit of the usage of declarative classes in SQLAlchemy.
As I understand the ORM is a way to apply the concept of database tables to the class system of OOP. However I don't understand why the table class doesn't already satisfy this requirement.
So to form my question via an example:
What is the benefit of using this:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, String
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(16))
fullname = Column(String(60))
nickname = Column(String(50))
Instead of this:
from sqlalchemy import *
metadata = MetaData()
user = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(16)),
Column('fullname ', String(60)),
Column('nickname ', String(50))
)
The latter one is already a class representation, isn't it? Why are we building another class over the already existing table class? What's the benefit?
I have the same question recently, you can refer to SQLAlchemy doc.
Some examples in the documentation still use the classical approach, but note that the classical as well as Declarative approaches are fully interchangeable.
I think the benefit of using Declarative Mapping is that,
more convinent to use foreign key.
when you want to create table with some table_args/mapper_args, just write in all down in the class.
I want to collect statistical information for songs (rating, play-count, last time played) from a couple of players over different devices and from different users. I use Python and SQL Alchemy.
I came up with the following table layout:
I can access all related Stats objects from my Commit ORM class as a list. I also want to have access to the related Song objects from the Commit class. Following the examples in the SQLALchemy Documentation I came up with an association table for the mtm relationship.
In code it looks like this (full source):
songcommits_table = Table(
'songcommits', Base.metadata,
Column('commit_id', Integer, ForeignKey('commits.commit_id')),
Column('song_id', Integer, ForeignKey('songs.song_id'))
)
class Commit(Base):
__tablename__ = 'commits'
commit_id = Column(Integer, primary_key=True)
# ...
songs = relationship(
"Song", secondary=songcommits_table, backref="commits"
)
stats = relationship('Stat', backref='commit')
def __repr__(self):
return "<Commit {0.commit_id}>".format(self)
It works. But I have the feeling it might work without the table, using the info already stored in the stats table, but I have no idea how to formulate this in the ORM.
So how can I use the information (commit_id and song_id) already present on the Stats ORM class instead of the helper table?
I have this class which I use to create and manage an SQLIte database:
def __init__(self, db_file = None, parent = None):
self.db = None
self.db_connection = None
self.db_file = str(db_file)
def db_open(self):
self.db = create_engine('sqlite:///' + self.db_file)
self.db_connection = self.db.connect()
def db_close(self):
self.db_connection.close()
def db_create_voltdrop(self):
metadata = MetaData()
tb_cable_brands = Table('cable_brands', metadata,
Column('id', Integer, primary_key=True),
Column('brand', String)
)
tb_cable_types = Table('cable_types', metadata,
Column('id', Integer, primary_key=True),
Column('brand_id', None, ForeignKey('cable_brands.id')),
Column('type', String),
Column('alpha', String)
)
tb_cable_data = Table('cable_data', metadata,
Column('id', Integer, primary_key=True),
Column('type_id', None, ForeignKey('cable_types.id')),
Column('size', String),
Column('resistance', Float)
)
metadata.create_all(self.db)
I instantiate this class when my MainWindow opens, use the DB and close the DB when the program quits.
I've just started learning SQLAlchemy. That code works fine. And then I came across sessions in SQLAlchemy which are also used to create and manage databases. Which way is better? What advantage do sessions have over the above way? Thank you.
The best practice for the session management is to declare it in global scope and use it.
The doc given in sqlalchemy says
When you write your application, place the result of the sessionmaker() call at the global level. The resulting Session class, configured for your application, should then be used by the rest of the applcation as the source of new Session instances.
So you have to create in any package level session. You can refer this link.
Short answer: for your example (where create_all issues DDLs) it is not really important (and I am not even sure if SA supports DDL transactions), but whenever you add/delete/modify/query the objects themselves, Sessions are the way to go. See Using the Sessions for more info.
More info:
Technically, the following statement is not correct: ... then I came across sessions in SQLAlchemy which are also *used to create and manage databases*.
Sessions are not used to create and manage databases, but rather to provide a UnitOfWork pattern for database operations.
A simple view is to see sessions as SQL Transactions: SA Sessions to SA objects are what SQL transactions are to DML (data modification) statements. Your particular example is generating DDL statements (data definition), and many RDBMS do not even support transactions for DDLs (you cannot rollback CREATE TABLE statement, but should use DROP TABLE to cancel your work).
How to make dynamic queries in SqlAlchemy ORM (if it is a correct name for them).
I used SqlAlchemy as abstraction for database, with queries in python code, but what if I need to generate these queries dynamically, not only set the parameters of query like "id"?
For example, I need to generate query from list (table names, column names, joined columns) that links three tables like "organisation", "people", "staff". How can I do it properly?
For example, i meant this list:
[{'table':'organisation', 'column':'staff_id'},
{'table':'staff', 'column':'id'}]
And output for example may contain:
organisation.id, organisation.name, organisation.staff_id, staff.id, staff.name
(name column is presented only in output, because I need simple example, recieving all columns of tables, and array must just set joins)
You can use mapper on the result of a call to sqlalchemy.sql.join and/or sqlalchemy.select. This is roughly equivalent to using mapper on a database view; you can query against such classes naturally, but not necessarily create new records. You can also use sqlalchemy.orm.column_property to map computed values to object attributes. As I read your question, a combination of these three techniques should meet your needs.
Haven't tested, but it with the SQLAlchemy ORM, you can link tables together like:
from sqlalchemy import create_engine, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, ForeignKey
from sqlalchemy.orm import relationship
from asgportal.database import Session
Engine = create_engine('mysql+mysqldb://user:password#localhost:3306/mydatabase', pool_recycle=3600)
Base = declarative_base(bind=Engine)
session = Session()
session.configure(bind=Engine)
class DBOrganization(Base):
__tablename__ = 'table_organization'
id = Column(Integer(), primary_key=True)
name = Column(ASGType.sa(ASGType.STRING))
class DBEmployee(Base):
__tablename__ = 'table_employee'
id = Column(Integer(), primary_key=True)
name = Column(String(255))
organization_id = Column(Integer(), ForeignKey('table_organization.id'))
# backref below will be an array[] unless you specify uselist=False
organization = relationship(DBOrganization, backref='employees')
Base.metadata.create_all()
# From here, you can query:
rs = session.query(DBEmployee).join(DBEmployee.organization).filter(DBOrganization.name=='my organization')
for employees in rs:
print '{0} works for {1}'.format(employees.name,employees.organization.name)
Related (for the no-association-object use case): SQLAlchemy Many-to-Many Relationship on a Single Table
Building a many-to-many relationship is easy. Building a many-to-many relationship on the same table is almost as easy, as documented in the above question.
Building a many-to-many relationship with an association object is also easy.
What I can't seem to find is the right way to combine association objects and many-to-many relationships with the left and right sides being the same table.
So, starting from the simple, naïve, and clearly wrong version that I've spent forever trying to massage into the right version:
t_groups = Table('groups', metadata,
Column('id', Integer, primary_key=True),
)
t_group_groups = Table('group_groups', metadata,
Column('parent_group_id', Integer, ForeignKey('groups.id'), primary_key=True, nullable=False),
Column('child_group_id', Integer, ForeignKey('groups.id'), primary_key=True, nullable=False),
Column('expires', DateTime),
)
mapper(Group_To_Group, t_group_groups, properties={
'parent_group':relationship(Group),
'child_group':relationship(Group),
})
What's the right way to map this relationship?
I guess you are getting an error like Could not determine join condition between parent/child tables...
In this case, Change the mapper for Group_To_Group to the following:
mapper(Group_To_Group, t_group_groups, properties={
'parent_group':relationship(Group,
primaryjoin=(t_group_groups.c.parent_group_id==t_groups.c.id),),
'child_group':relationship(Group,
primaryjoin=(t_group_groups.c.child_group_id==t_groups.c.id),),
})
also you might want to add the backref so that you can navigate the relations from the Group objects as well.