Develop Reference

Flask-SQLAlchemy how to use create_all with schema_translate_map

The SQLAlchemy provides the Connection.execution_options.schema_translate_map for change the schemas in execution time, as said in docs.
In the examples is shown how to use to perform queries, but want to know how to use it with create_all().
I'm using Flask-Sqlaclhemy and postgresql as database. Let's say I have this:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
app = Flask(...)
...
db.init_app(app)
...
return app
class User(db.Model):
__tablename__ = 'user'
__table_args__ = {'schema':'public'}
company = db.Column(db.String(10))
class SomePublicModel(db.Model):
__tablename__ = 'some_public'
__table_args__ = {'schema':'public'}
...
class SomeModelByDynamicSchema(db.Model):
__tablename__ = 'some_dynamic'
__table_args__ = {'schema':'dynamic'}
...
The dynamic schema will be replace for other value according the user's company in execution time.
Assuming I already have in database the schemas public and dynamic and a I want to create a new schema with the tables, something like this:
def create_new():
user = User(company='foo')
db.session.execute("CREATE SCHEMA IF NOT EXISTS %s" % user.company)
db.session.connection().execution_options(schema_translate_map={'dynamic':user.company})
#I would like to do something of the kind
db.create_all()
I expected the tables to be created in the foo schema as foo.some_dynamic, but the SQLAlchemy still try to create in dynamic schema.
Can someone help me?

When you set execution options, you create copy of connection. This mean what create_all run without schema_translate_map.
>>> c = Base.session.connection()
>>> w = c.execution_options(schema_translate_map={'dynamic':'kek'})
>>> c._execution_options
immutabledict({})
>>> w._execution_options
immutabledict({'schema_translate_map': {'dynamic': 'kek'}})

to achieve your goal, you could try another approach.
get the tables from old grammmar and adapt for new metadata.
metadata = MetaData(bind=engine, schema=db_schema)
for table in db.Model.metadata.tables.values():
table.tometadata(metadata)
metadata.drop_all()
metadata.create_all()

Flask-Blogging error Table is already defined

I'm using a flask setup from a while and now trying to install Flask-Blogging module on it. Current modules:
- Flask-sqlalchemy with postgres
- Flask-login
- Flask-Blogging (new)
My application.py looks like this:
from flask import Flask
from flask import session
from flask.ext.blogging import SQLAStorage, BloggingEngine
from flask.ext.login import LoginManager
from flask.ext.sqlalchemy import SQLAlchemy
'''
The main application setup. The order of things is important
in this file.
'''
app = Flask(__name__)
app.config.from_object('config.base')
app.config.from_envvar('APP_CONFIG_FILE')
'''
Initialize database
'''
db = SQLAlchemy(app)
'''
Initialize blogger
'''
storage = SQLAStorage(db=db)
blog_engine = BloggingEngine(app, storage)
the last two lines are the only new things I added (other than the imports). Suddenly now I'm getting error about duplicate table names:
sqlalchemy.exc.InvalidRequestError: Table 'customer' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
Any ideas what am I doing wrong? I couldn't find much documentation about Flask-Blogging other than:
http://flask-blogging.readthedocs.org/en/latest/

You get this error because in SQLAStorage.__init__ there is this line:
self._metadata.reflect(bind=self._engine)
This will look at your database and create sqlalchemy table things for all the tables currently in your database.
Thus if your database contains a table called 'customer' the line in your code:
storage = SQLAStorage(db=db)
will automatically model an sqlalchemy table called 'customer' for you.
Now... no doubt you have your own database model definitions somewhere, probably in another python module, something like:
class Customer(db.Model):
id = db.Column(db.Integer, primary_key=True)
...
Since this class definition defines a table called 'customer' and since SQLAStorage has already defined a table called 'customer' you get the exception as soon as your class Customer is imported.
Some ways to work around this problem are:
Import your database definition modules before instantiating SQLAStorage
'''
Initialize database
'''
db = SQLAlchemy(app)
import ankit.db_models # import my db models here so SQLAStorage doesn't do it first
'''
Initialize blogger
'''
storage = SQLAStorage(db=db)
blog_engine = BloggingEngine(app, storage)
or
Tell SQLAStorage to use its own metadata
By passing the db param to SQLAStorage.__init__ you are telling it to use your metadata. You can instead just pass the engine parameter and it will create its own metadata.
storage = SQLAStorage(engine=db.engine)

How do I create a table in PostgreSQL with SQLAlchemy?

I tried to create a table in the "app" database, but got this error:
app=# CREATE TABLE users;
ERROR: syntax error at or near ";"
LINE 1: LINE 1: CREATE TABLE users;
^
The columns are defined in models.py which is why I don't create them here.

Do not create the tables in PostgreSQL, SQLAlchemy will create the table (and the columns) for you. All you need to do is create the database, which you have done. To create the tables with Flask-SQLAlchemy:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False, unique=True)
with app.app_context():
db.create_all()
To address the error (although as stated above you should not do this), it is because that is not a full CREATE TABLE expression. You need to specify zero or more columns for the table.
CREATE TABLE my_table ();

In my case forgot to import my Model class and code
run without any error:).
If you call create_all() on interactive mode please ensure that you import particular Model class.
Note: Every time you make a change on Model class restart ipython.

SqlAlchemy add tables versioning to existing tables

Imagine that I have one table in my project with some rows in it.
For example:
# -*- coding: utf-8 -*-
import sqlalchemy as sa
from app import db
class Article(db.Model):
__tablename__ = 'article'
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
name = sa.Column(sa.Unicode(255))
content = sa.Column(sa.UnicodeText)
I'm using Flask-SQLAlchemy, so db.session is scoped session object.
I saw in https://github.com/zzzeek/sqlalchemy/blob/master/examples/versioned_history/history_meta.py
but i can't understand how to use it with my existing tables and anymore how to start it. (I get ArgumentError: Session event listen on a scoped_session requires that its creation callable is associated with the Session class. error when I pass db.session in versioned_session func)
From versioning I need the following:
1) query for old versions of object
2) query old versions by date range when they changed
3) revert old state to existing object
4) add additional info to history table when version is creating (for example editor user_id, date_edit, remote_ip)
Please, tell me what are the best practicies for my case and if you can add a little working example for it.

You can work around that error by attaching the event handler to the SignallingSession class[1] instead of the created session object:
from flask.ext.sqlalchemy import SignallingSession
from history_meta import versioned_session, Versioned
# Create your Flask app...
versioned_session(SignallingSession)
db = SQLAlchemy(app)
class Article(Versioned, db.Model):
__tablename__ = 'article'
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
name = sa.Column(sa.Unicode(255))
content = sa.Column(sa.UnicodeText)
The sample code creates parallel tables with a _history suffix and an additional changed datetime column. Querying for old versions is just a matter of looking in that table.
For managing the extra fields, I would put them on your main table, and they'll automatically be kept track of in the history table.
[1] Note, if you override SQLAlchemy.create_session() to use a different session class, you should adjust the class you pass to versioned_session.

I think the problem is you're running into this bug: https://github.com/mitsuhiko/flask-sqlalchemy/issues/182
One workaround would be to stop using flask-sqlalchemy and configure sqlalchemy yourself.

SQLAlchemy declarative syntax with autoload (reflection) in Pylons

I would like to use autoload to use an existings database. I know how to do it without declarative syntax (model/_init_.py):
def init_model(engine):
"""Call me before using any of the tables or classes in the model"""
t_events = Table('events', Base.metadata, schema='events', autoload=True, autoload_with=engine)
orm.mapper(Event, t_events)
Session.configure(bind=engine)
class Event(object):
pass
This works fine, but I would like to use declarative syntax:
class Event(Base):
__tablename__ = 'events'
__table_args__ = {'schema': 'events', 'autoload': True}
Unfortunately, this way I get:
sqlalchemy.exc.UnboundExecutionError: No engine is bound to this Table's MetaData. Pass an engine to the Table via autoload_with=<someengine>, or associate the MetaData with an engine via metadata.bind=<someengine>
The problem here is that I don't know where to get the engine from (to use it in autoload_with) at the stage of importing the model (it's available in init_model()). I tried adding
meta.Base.metadata.bind(engine)
to environment.py but it doesn't work. Anyone has found some elegant solution?

OK, I think I figured it out. The solution is to declare the model objects outside the model/__init__.py. I concluded that __init__.py gets imported as the first file when importing something from a module (in this case model) and this causes problems because the model objects are declared before init_model() is called.
To avoid this I created a new file in the model module, e.g. objects.py. I then declared all my model objects (like Event) in this file.
Then, I can import my models like this:
from PRJ.model.objects import Event
Furthermore, to avoid specifying autoload-with for each table, I added this line at the end of init_model():
Base.metadata.bind = engine
This way I can declare my model objects with no boilerplate code, like this:
class Event(Base):
__tablename__ = 'events'
__table_args__ = {'schema': 'events', 'autoload': True}
event_identifiers = relationship(EventIdentifier)
def __repr__(self):
return "<Event(%s)>" % self.id

I just tried this using orm module.
Base = declarative_base(bind=engine)
Base.metadata.reflect(bind=engine)
Accessing tables manually or through loop or whatever:
Base.metadata.sorted_tables
Might be useful.

from sqlalchemy import MetaData,create_engine,Table
engine = create_engine('postgresql://postgres:********#localhost/db_name')
metadata = MetaData(bind=engine)
rivers = Table('rivers',metadata,autoload=True,auto_load_with=engine)
from sqlalchemy import select
s = select([rivers]).limit(5)
engine.execute(s).fetchall()
worked for me. I was getting the error because of not specifying bind when creating MetaData() object.

Check out the Using SQLAlchemy with Pylons tutorial on how to bind metadata to the engine in the init_model function.
If the meta.Base.metadata.bind(engine) statement successfully binds your model metadata to the engine, you should be able to perform this initialization in your own init_model function. I guess you didn't mean to skip the metadata binding in this function, did you?