I am trying to integrate SqlAlchemy with SQLAlchemy-continuum.
I use Automap feature instead of making declarative classes, I have been unable to use continuum with the automap feature. Also there are no examples or citations in the docs regarding the same.
Has anybody used SqlAlchemy-continuum with this feature.
Note - I am using custom schemas for postrgesql, the schema is not the default the postrges ships with.
Also adding some code for reference:
#Imports
from sqlalchemy.ext.automap import automap_base
from sqlalchemy_continuum import make_versioned
from sqlalchemy_continuum import version_class, parent_class
make_versioned(user_cls=None) #Currently trying to not make user relationship with transactions table.
#Created the engine and queried the schema metadata from there in _metdadata.
_metdadata.reflect(views=True)
Base = automap_base(metadata=_metdadata)
class Map1(Base):
__tablename__ = 'test_pg_audit'
__table_args__ = {'extend_existing': 'True'}
__versioned__ = {}
id = Column(Integer, primary_key=True)
Base.prepare()
rec = TestPGAudit(name='test')
dbsession.add(rec)
dbsession.flush()
dbsession.commit()
#The History Class is not found, and sqlalchemy_continuum.exc.ClassNotVersioned is raise here.
at = version_class(TestPGAudit)
recs = dbsession.query(at).all()
print recs
I have also tried configuring the mappers at different places including after Base.prepare, but to no avail.
Also tried creating history tables in database manually.
Any help is appreciated.
Related
I am just trying to get started using sqlalchemy. For whatever reason I can't get anything to work.
I installed sqlalchemy the import alone works. I tried to start following the code on this site:
https://www.pythoncentral.io/introductory-tutorial-python-sqlalchemy/
The code is as follows:
import os
import sys
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy import create_engine
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
# Here we define columns for the table person
# Notice that each column is also a normal Python instance attribute.
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
class Address(Base):
__tablename__ = 'address'
# Here we define columns for the table address.
# Notice that each column is also a normal Python instance attribute.
id = Column(Integer, primary_key=True)
street_name = Column(String(250))
street_number = Column(String(250))
post_code = Column(String(250), nullable=False)
person_id = Column(Integer, ForeignKey('person.id'))
person = relationship(Person)
# Create an engine that stores data in the local directory's
# sqlalchemy_example.db file.
engine = create_engine('sqlite:///sqlalchemy_example.db')
# Create all tables in the engine. This is equivalent to "Create Table"
# statements in raw SQL.
Base.metadata.create_all(engine)
I copied and pasted the code to create a table and I'm getting the following error
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to
open database file (Background on this error at:
http://sqlalche.me/e/e3q8)
I went to http://sqlalche.me/e/e3q8 and it seems to believe that adding pool_pre_ping=True to the engine would help resolve issue. It mentions a connection issues, but don't really understand how that can be since it's just creating the sqlite database.
I would really appreciate any advice on how I can fix this issue.
Edit: I put the specific code into my question.
Also I tried performing the code in pythonanywhere and it works as expected. Any guidance on what could be wrong with my machine would be appreciated.
So for whatever reason I needed to designate the absolute path of where the database needed to be. I updated my engine to be:
sqlite:///C:\user\file_path\test.db
this allowed it to create the database. However I'd really prefer it just create the database in the current directory. If someone knows what I need to do to get that to work that would be great.
I have a Pyramid application that does CRUD with SQLAlchemy via pyramid_basemodel. All seems to work nicely.
I then pip installed SQLAlchemy-Continuum, to provide history for certain objects. All I did to configure it was make the following alterations to my models.py file:
import sqlalchemy as sa
from sqlalchemy import (event, Column, Index, Integer, Text, String, Date, DateTime, \
Float, ForeignKey, Table, Boolean,)
from sqlalchemy.orm import (relationship, backref, mapper, scoped_session, sessionmaker,)
from pyramid_basemodel import Base, BaseMixin, Session, save
from pyramid_fullauth.models import User
from sqlalchemy_continuum import make_versioned
from colanderalchemy import setup_schema
from zope.sqlalchemy import ZopeTransactionExtension
DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
event.listen(mapper, 'mapper_configured', setup_schema)
# Continuum setup
make_versioned()
# FOR EACH VERSIONED MODEL I ADD __versioned__ = {} at the start of each model def. Eg:
class Thing(Base):
__versioned__ = {}
__tablename__ = 'thing'
id = sa.Column(Integer, primary_key=True)
related_id = sa.Column(Integer, ForeignKey('OtherThing.id'))
other_thing = sa.orm.relationship("OtherThing", backref="thing")
description = sa.Column(String(length=100))
a_date = sa.Column(Date)
some_hours = sa.Column(Integer)
b_date = sa.Column(Date)
more_hours = sa.Column(Integer)
sa.orm.configure_mappers()
(Sorry for the slightly redundant imports; I decided to totally follow the Continuum example and import sqlalchemy as sa, and switch to using that notation in the models that I versioned. I may also be doing stupid, monkey-see monkey-do stuff based on a half-understanding of different tutorials.)
This setup allowed me to run alembic revision --autogenerate and produce ModelHistory tables in the database, but when I go to some of the pages that read the now-versioned models, they give the error
sqlalchemy.exc.UnboundExecutionError: This session is not bound to a single Engine or Connection, and no context was provided to locate a binding.
For some reason it reads one model added in the same way, but then trying to update it fails with the same error.
My guess is that I need to configure whatever Continuum uses for a SQLAlchemy session to point to the existing one configured in Pyramid, but I'm not sure. Am I getting warm?
You're generating a session when you call:
DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
but not binding it to an engine. Your pyramid template, pyramid_basemodel, is already generating a session for you and binding it to the engine.
Try removing the DBSession and using Session imported from pyramid_basemodel.
FWIW, if anyone these days is looking how to make SQLAlchemy-Continuum work with Pyramid, here's how you do it:
Assuming you have followed the official Pyramid tutorial is the following:
install SQLAlchemy-Continuum
add make_versioned(user_cls=None) to the top of models/__init__.py
add __versioned__ = {} to MyModel class in models/mymodel.py
And that's it!
I've created a repo that has all the needed bits in place: https://github.com/zupo/tutorial/tree/exploration/sqlalchemy-continuum
Imagine that I have one table in my project with some rows in it.
For example:
# -*- coding: utf-8 -*-
import sqlalchemy as sa
from app import db
class Article(db.Model):
__tablename__ = 'article'
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
name = sa.Column(sa.Unicode(255))
content = sa.Column(sa.UnicodeText)
I'm using Flask-SQLAlchemy, so db.session is scoped session object.
I saw in https://github.com/zzzeek/sqlalchemy/blob/master/examples/versioned_history/history_meta.py
but i can't understand how to use it with my existing tables and anymore how to start it. (I get ArgumentError: Session event listen on a scoped_session requires that its creation callable is associated with the Session class. error when I pass db.session in versioned_session func)
From versioning I need the following:
1) query for old versions of object
2) query old versions by date range when they changed
3) revert old state to existing object
4) add additional info to history table when version is creating (for example editor user_id, date_edit, remote_ip)
Please, tell me what are the best practicies for my case and if you can add a little working example for it.
You can work around that error by attaching the event handler to the SignallingSession class[1] instead of the created session object:
from flask.ext.sqlalchemy import SignallingSession
from history_meta import versioned_session, Versioned
# Create your Flask app...
versioned_session(SignallingSession)
db = SQLAlchemy(app)
class Article(Versioned, db.Model):
__tablename__ = 'article'
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
name = sa.Column(sa.Unicode(255))
content = sa.Column(sa.UnicodeText)
The sample code creates parallel tables with a _history suffix and an additional changed datetime column. Querying for old versions is just a matter of looking in that table.
For managing the extra fields, I would put them on your main table, and they'll automatically be kept track of in the history table.
[1] Note, if you override SQLAlchemy.create_session() to use a different session class, you should adjust the class you pass to versioned_session.
I think the problem is you're running into this bug: https://github.com/mitsuhiko/flask-sqlalchemy/issues/182
One workaround would be to stop using flask-sqlalchemy and configure sqlalchemy yourself.
I would like to use autoload to use an existings database. I know how to do it without declarative syntax (model/_init_.py):
def init_model(engine):
"""Call me before using any of the tables or classes in the model"""
t_events = Table('events', Base.metadata, schema='events', autoload=True, autoload_with=engine)
orm.mapper(Event, t_events)
Session.configure(bind=engine)
class Event(object):
pass
This works fine, but I would like to use declarative syntax:
class Event(Base):
__tablename__ = 'events'
__table_args__ = {'schema': 'events', 'autoload': True}
Unfortunately, this way I get:
sqlalchemy.exc.UnboundExecutionError: No engine is bound to this Table's MetaData. Pass an engine to the Table via autoload_with=<someengine>, or associate the MetaData with an engine via metadata.bind=<someengine>
The problem here is that I don't know where to get the engine from (to use it in autoload_with) at the stage of importing the model (it's available in init_model()). I tried adding
meta.Base.metadata.bind(engine)
to environment.py but it doesn't work. Anyone has found some elegant solution?
OK, I think I figured it out. The solution is to declare the model objects outside the model/__init__.py. I concluded that __init__.py gets imported as the first file when importing something from a module (in this case model) and this causes problems because the model objects are declared before init_model() is called.
To avoid this I created a new file in the model module, e.g. objects.py. I then declared all my model objects (like Event) in this file.
Then, I can import my models like this:
from PRJ.model.objects import Event
Furthermore, to avoid specifying autoload-with for each table, I added this line at the end of init_model():
Base.metadata.bind = engine
This way I can declare my model objects with no boilerplate code, like this:
class Event(Base):
__tablename__ = 'events'
__table_args__ = {'schema': 'events', 'autoload': True}
event_identifiers = relationship(EventIdentifier)
def __repr__(self):
return "<Event(%s)>" % self.id
I just tried this using orm module.
Base = declarative_base(bind=engine)
Base.metadata.reflect(bind=engine)
Accessing tables manually or through loop or whatever:
Base.metadata.sorted_tables
Might be useful.
from sqlalchemy import MetaData,create_engine,Table
engine = create_engine('postgresql://postgres:********#localhost/db_name')
metadata = MetaData(bind=engine)
rivers = Table('rivers',metadata,autoload=True,auto_load_with=engine)
from sqlalchemy import select
s = select([rivers]).limit(5)
engine.execute(s).fetchall()
worked for me. I was getting the error because of not specifying bind when creating MetaData() object.
Check out the Using SQLAlchemy with Pylons tutorial on how to bind metadata to the engine in the init_model function.
If the meta.Base.metadata.bind(engine) statement successfully binds your model metadata to the engine, you should be able to perform this initialization in your own init_model function. I guess you didn't mean to skip the metadata binding in this function, did you?
I am writing a multimedia archive database backend and I want to use joined table inheritance. I am using Python with SQLAlchemy with the declarative extension. The table holding the media record is as follows:
_Base = declarative_base()
class Record(_Base):
__tablename__ = 'records'
item_id = Column(String(M_ITEM_ID), ForeignKey('items.id'))
storage_id = Column(String(M_STORAGE_ID), ForeignKey('storages.id'))
id = Column(String(M_RECORD_ID), primary_key=True)
uri = Column(String(M_RECORD_URI))
type = Column(String(M_RECORD_TYPE))
name = Column(String(M_RECORD_NAME))
The column type is a discriminator. Now I want to define the child class AudioRecord from the Record class, but I don't how to setup the polymorphic mapper using the declarative syntax. I am looking for an equivalent for the following code (from SQLAlchemy documentation):
mapper(Record, records, polymorphic_on=records.c.type, polymorphic_identity='record')
mapper(AudioRecord, audiorecords, inherits=Record, polymorphic_identity='audio_record')
How can I pass the polymorphic_on, polymorphic_identity and inherits keywords to the mapper created by the declarative extension?
Thank you
Jan
I finally found the answer in the manual.
http://www.sqlalchemy.org/docs/05/reference/ext/declarative.html#joined-table-inheritance