I'm fairly new to using relational databases, so I prefer using a good ORM to simplify things. I spent time evaluating different Python ORMs and I think SQLAlchemy is what I need. However, I've come to a mental dead end.
I need to create a new table to go along with each instance of a player I create in my app's player table. I think I know how to create the table by changing the name of the table through the metadata then calling the create function, but I have no clue on how to map it to a new dynamic class.
Can someone give me some tips to help me get past my brain freeze? Is this even possible?
Note: I'm open to other ORMs in Python if what I'm asking is easier to implement.Just show me how :-)
We are spoiled by SQLAlchemy.
What follows below is taken directly from the tutorial,
and is really easy to setup and get working.
And because it is done so often,
the documentation moved to full declarative in Aug 2011.
Setup your environment (I'm using the SQLite in-memory db to test):
>>> from sqlalchemy import create_engine
>>> engine = create_engine('sqlite:///:memory:', echo=True)
>>> from sqlalchemy import Table, Column, Integer, String, MetaData
>>> metadata = MetaData()
Define your table:
>>> players_table = Table('players', metadata,
... Column('id', Integer, primary_key=True),
... Column('name', String),
... Column('score', Integer)
... )
>>> metadata.create_all(engine) # create the table
If you have logging turned on, you'll see the SQL that SQLAlchemy creates for you.
Define your class:
>>> class Player(object):
... def __init__(self, name, score):
... self.name = name
... self.score = score
...
... def __repr__(self):
... return "<Player('%s','%s')>" % (self.name, self.score)
Map the class to your table:
>>> from sqlalchemy.orm import mapper
>>> mapper(Player, players_table)
<Mapper at 0x...; Player>
Create a player:
>>> a_player = Player('monty', 0)
>>> a_player.name
'monty'
>>> a_player.score
0
That's it, you now have a your player table.
It's a very old question. Anyway if you prefer ORM, it's quite easy to generate table class with type:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
Base = declarative_base()
Test = type('Test', (Base,), {
'__tablename__': 'test',
'test_id': Column(Integer, primary_key=True, autoincrement=True),
'fldA': Column(String),
... other columns
}
)
Base.metadata.create_all(engine)
# passed session create with sqlalchemy
session.query(Test).all()
Making a class factory, it's easy to assign names to a class and database table.
If you are looking to create dynamic classes and tables you can use the following technique based from this tutorial URL I found here (http://sparrigan.github.io/sql/sqla/2016/01/03/dynamic-tables.html), I modified how he did it a bit.
from sqlalchemy import create_engine
engine = create_engine('sqlite:///test.db', echo=True)
from sqlalchemy import Column, Integer,Float,DateTime, String, MetaData
metadata = MetaData()
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session() # create a Session
Base = declarative_base()
First include all the needed dependencies and create your session and Base.
The key to creating it dynamically is this here:
attr_dict = {'__tablename__': 'default','id': Column(Integer, primary_key=True, auto_increment=True)}
you could create a table from just this by taking advantage of the 'type' function in python.
myClass = type('ClassnameHere', (Base,), attr_dict)
Note that we are passing in attr_dict, this will give the required tablename and column information to our class, but the difference is we are defining the class name through a string! This means you could create a loop for example going through an array of strings to start creating tables dynamically!
Next all you have to do is simply call
Base.metadata.create_all(engine)
Because the dynamic class we created inherits from Base the command will simply create the tables!
You add to this table for example like this now:
SomeRow = myClass(id='2')
session.add(SomeRow)
session.commit()
This can go even further if you you don't know the column names as well. Just refer to the article to learn how to do that.
You would essentially do something like this though:
firstColName = "Ill_decide_later"
secondColName = "Seriously_quit_bugging_me"
new_row_vals = myClass(**{firstColName: 14, secondColName: 33})
The ** operator takes the object and unpacks it so that firstColName and secondColName are added with assignment operators so it would essentially be the same thing as this:
new_row_vals = myClass(firstColName=14, secondColName=33)
The advantage of this technique is now you can dynamically add to the table without even having to define the column names!
These column names could be stored in a string array for example or whatever you wanted and you just take it from there.
Maybe look at SQLSoup, which is layer over SQLAlchemy.
You can also create the tables using plain SQL, and to dynamically map, use these libraries if they already don't have create table function.
Or alternatively create a dynamic class and map it:
tableClass = type(str(table.fullname), (BaseTable.BaseTable,), {})
mapper(tableClass, table)
where BaseTable can be any Python class which you want all your table classes to inherit from, e.g. such Base class may have some utility or common methods, e.g. basic CRUD methods:
class BaseTable(object): pass
Otherwise you need not pass any bases to type(...).
you can use declarative method for dynamically creating tables in database
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
Base = declarative_base()
class Language(Base):
__tablename__ = 'languages'
id = Column(Integer, primary_key=True)
name = Column(String(20))
extension = Column(String(20))
def __init__(self, name, extension):
self.name = name
self.extension = extension
I faced the same problem when I was trying to automate simple CRUD tasks using SQLAlchemy.
Here is simple explanation and some code: http://www.devx.com/dbzone/Article/42015
maybe i didn't quite understand what you want, but this recipe create identical column in different __tablename__
class TBase(object):
"""Base class is a 'mixin'.
Guidelines for declarative mixins is at:
http://www.sqlalchemy.org/docs/orm/extensions/declarative.html#mixin-classes
"""
id = Column(Integer, primary_key=True)
data = Column(String(50))
def __repr__(self):
return "%s(data=%r)" % (
self.__class__.__name__, self.data
)
class T1Foo(TBase, Base):
__tablename__ = 't1'
class T2Foo(TBase, Base):
__tablename__ = 't2'
engine = create_engine('sqlite:///foo.db', echo=True)
Base.metadata.create_all(engine)
sess = sessionmaker(engine)()
sess.add_all([T1Foo(data='t1'), T1Foo(data='t2'), T2Foo(data='t3'),
T1Foo(data='t4')])
print sess.query(T1Foo).all()
print sess.query(T2Foo).all()
sess.commit()
info in example sqlalchemy
Related
Background:
I want to use SQLAlchemy and apply its imperative mapping style in the context of class inheritance. Querying the database for one of the subclasses gives me an InvalidReuestError.
Description with Code:
There are the following classes in place:
from sqlalchemy import create_engine, MetaData, select, insert
from sqlalchemy import Column, Integer, String
from sqlalchemy.schema import Table
class Products:
def __init__(self, ident, name, group):
self.ident = ident
self.name = name
self.group = group
class ProductsA(Products):
pass
class ProductsB(Products):
pass
The following mappings are in place:
from sqlalchemy.orm import registry
mapper_registry = registry()
products_mapper = mapper_registry.map_imperatively(Products,
products_table,
polymorphic_on=products_table.c.group,)
productsA_mapper = mapper_registry.map_imperatively(ProductsA,
products_table,
inherits=products_mapper,
polymorphic_identity="A")
productsB_mapper = mapper_registry.map_imperatively(ProductsB,
products_table,
inherits=products_mapper,
polymorphic_identity="B")
Then the following query shows an error if it finds a row with group B:
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(engine)
with Session() as session:
results = session.query(ProductsA).all()
Error message:
InvalidRequestError: Row with identity key (<class '__main__.Products'>, (2,), None) can't be loaded into an object; the polymorphic discriminator column 'products.group' refers to mapped class ProductsB->products, which is not a sub-mapper of the requested mapped class ProductsA->products
However, querying for the product class directly gives me no error. Also using a declarative mapping similar to SQLAlchemy how to define two models for the same table works fine.
What can I do to make the approach with the imperative style work?
Remark: I can also provide code to generate a underlying database.
I came across
several approaches on how to use the vanilla SQLAlchemy models in Flask-SQLAlchemy.
It works like a charm to use models that inherit from Base with Flask-SQLAlchemy.
But I really like that convenience stuff...
Job.query.all() # Does not work
db.session.query(Job).all() # Works
So I started to work on this and put together some code, but I am stuck and need some help to get this nice and clean.
The following block is a general definition that does not inherit from either.
It is imported and supposed to be used from Flask-SQLAlchemy and vanilla SQLAlchemy at some point.
class VanillaMachine():
__tablename__ = 'machine'
id = Column(Integer, primary_key=True)
name = Column(String(100))
status = Column(Integer)
And there is a factory that takes either db.Model or Base and return Machine with the correct parent:
class MachineFactory:
def __init__(self, *args, **kwargs):
pass
def __new__(cls, *args, **kwargs):
return type('Machine',(object, VanillaMachine, args[0]), VanillaMachine.__dict__.copy())
I am quite sure that there's something off with that code, but
I am not sure where.
If I use it like
db = SQLAlchemy()
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()#
Machine1 = MachineFactory(db.Model)
Machine2 = MachineFactory(Base)
there is a error message
sqlalchemy.exc.ArgumentError: Column object 'id' already assigned to Table 'machine'
Can help me to get this straight in a nice, reliable way?
I know that you could just use a function, pass the parent as argument into VanillaMachine and use some if statement, but that would be too straightforward, right? :)
Edit:
Other approaches I came across are
using the Flask context to use Flask-SQLAlchemy models
with app.app_context():
pass
or
app.app_context().push()
But this is too focused on Flask for me and does not allow to clearly separate the models, make them independent and adjust to the context.
supplying an alternative Base class to db = SQLAlchemy(app, model_class=Base), see here. This might work for me, but I did not evaluate this so far.
I found a good solution inspired by a Factory pattern and
Declarative Mixins as mentioned in the SQLAlchemy docs.
For complex multi-level inheritance scenarios a different approach is needed, using #declared_attr.cascading.
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Column, Integer, String
from sqlalchemy import MetaData
from sqlalchemy.ext.declarative import declarative_base
from flask_sqlalchemy import SQLAlchemy
SQLALCHEMY_DATABASE_URI = 'sqlite:///' + '/tmp/test_app.db'
engine = create_engine(SQLALCHEMY_DATABASE_URI, echo=True)
# for vanilla
Base = declarative_base()
# for Flask (import from app once initialized)
db = SQLAlchemy()
class MachineMixin:
__tablename__ = 'machine'
id = Column(Integer, primary_key=True)
name = Column(String(100))
status = Column(Integer)
class ModelFactory:
#staticmethod
def create(which_model, which_parent):
if which_parent == 'flask_sqlalchemy':
parent = db.Model
elif which_parent == 'pure_sqlalchemy':
parent = Base
# now use type() to interit, fill __dict__ and assign a name
obj = type(which_model.__name__ + '_' + which_parent,
(which_model, parent),
{})
return obj
test_scenario = 'pure_sqlalchemy' # 'flask_sqlalchemy'
Machine = ModelFactory.create(MachineMixin, test_scenario)
if test_scenario == 'flask_sqlalchemy':
db.metadata.drop_all(bind=engine)
db.metadata.create_all(bind=engine)
elif test_scenario == 'pure_sqlalchemy':
Base.metadata.drop_all(bind=engine)
Base.metadata.create_all(bind=engine)
Session = sessionmaker(bind=engine)
session = Session()
session.add(Machine(name='Bob', status=1))
session.commit()
The question really is how to update a SQLAlchemy declarative model so that it runs the validators. In my case using setters like User.name = name is not really an option.
Below is a runnable example of what I mean
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Column, String, Integer
from sqlalchemy.orm import validates
from sqlalchemy.ext.declarative import declarative_base
some_engine = create_engine('sqlite://')
Session = sessionmaker(bind=some_engine)
session = Session()
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
#validates('name')
def validate_name(self, key, value):
if value != 'asd':
raise ValueError('not asd')
return value
Base.metadata.create_all(bind=some_engine)
user = User(id=1, name='qwe')
# >>> ValueError: not asd
user = User(id=1, name='asd')
session.add(user)
session.commit()
session.query(User).filter(User.id=1).update({'name': 'qwe'})
session.query(User).filter(User.id==1)[0].name
# >>> 'qwe'
You could add a mixin to your models that provides a rather simple update method that just uses setattr() to set attributes of an instance.
class UpdateMixin:
"""
Add a simple update() method to instances that accepts
a dictionary of updates.
"""
def update(self, values):
for k, v in values.items():
setattr(self, k, v)
User class would then be defined as
class User(UpdateMixin, Base):
...
And to update a single instance from a given dictionary you could for example run
session.query(User).get(1).update({ 'name': 'qwe' })
# or since you have the user instance from before
user.update({ 'name': 'qwe' })
Note the use of Query.get(). If there is no user with the given id, it will return None and trying to call the method update on it will raise. Another caveat is that if you do not rollback if any exceptions are raised, you cannot predict what, if any, updates took place (were added to the session) because a dictionary has no ordering. So always rollback on any errors.
I'd also recommend actually naming the method updateSelf or some such to prevent risk of confusing it with Query.update().
The short answer is not to use query.update when you want model level constraints. It's exactly for the times when performance is more important than enforcing those sorts of model level constraints. Other answers have provided specifics on solutions, but the fundamental answer is that Query.update is not intended to enforce python-level constraints.
General categories of solutions are:
Use some session-level method and Query.get or a loop on Query.filter.all
Check constraints
Triggers and stored procedures
I’m stumped. I really must be missing something basic about how the SQLAlchemy ORM works.
For the same reason I want my Java models to be POJOs (Plain Old Java Objects), I want my Python objects to be independent of the ORM. (Can I call them POPO’s?).
That’s why I’ve chosen to use the SQLAlchemy ORM’s “classical” mapping. The documentation repeatedly states that the declarative and classical mapping methods are equivalent and that, in fact, the former uses the latter under the hood.
The problem is that it ain’t workin’ for me. Specifically, when I do a select to read an object, it’s not being mapped to an instance of my object class. I’m getting back a what my debugger labels a “result” object --a raw row object.
As evidence, I submit two minimal applications. The first one uses declarative mapping and works just fine. My model goes in as a class instance and comes out as one. The second uses classical mapping and my model goes in as a class instances and comes out as a row result.
What am I missing here?
I’m using Python 2.7.10 and SQLAlchemy 1.0.11.
Declarative mapping, which does work:
from sqlalchemy import create_engine, Column, Integer, orm
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Thing(Base):
__tablename__ = 'thing'
id = Column(Integer, primary_key=True)
engine = create_engine("sqlite://")
Base.metadata.create_all(engine)
Session = orm.sessionmaker(bind=engine)
session = Session()
thing = Thing()
print "In: " + str(thing)
session.add(thing)
session.commit()
thing = session.query(Thing).one()
print "Out: " + str(thing)
Output:
In: <__main__.Thing object at 0x10c94c490>
Out: <__main__.Thing object at 0x10c94c490>
Classical mapping version, which does not work:
from sqlalchemy import create_engine, MetaData, Table, Column, Integer, orm
class Thing(object):
pass
engine = create_engine("sqlite://")
metadata = MetaData()
thing_table = Table(
'thing',
metadata,
Column('id', Integer, primary_key=True)
)
orm.mapper(Thing, thing_table)
metadata.create_all(engine)
Session = orm.sessionmaker(bind=engine)
session = Session()
thing = Thing()
print "In: " + str(thing)
session.add(thing)
session.commit()
thing = session.query(thing_table).one()
print "Out: " + str(thing)
Output:
In: <__main__.Thing object at 0x10cda2b10>
Out: (1,)
In SQLAlchemy Declarative, how do I set up default values for columns, such that transient or pending object instances will have those default values? A short example:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
word = Column(String, default="adefault")
a = A()
print a.word
Naively, I would expect the output from this to be adefault. Of course, the output is actually None. Even when adding to a session, it staysNone and only gets filled when I commit (or flush) the session, and re-read the instance value from the database.
Is there any way to set an attribute default without flushing the instance to the database? I tried investigating the ColumnDefault documentation, and there doesn't seem to be an obvious way to inspect the type/python value, so as to manually set it in a custom declarative baseclass.
Add a constructor to your class and set the default value there. The constructor doesn't run when the rows are loaded from the database so it is fine to do this.
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
word = Column(String)
def __init__(self):
self.word = "adefault"
a = A()
print a.word
There are examples of using __init__ in similar ways in the SA Docs.