Develop Reference

SQLAlchemy Create View in PostgresQL

I am trying create a view with SQLAlchemy with Postgresql as the underlying DB. The separate select query to create the view works well and returns results but when I use it in the create view, I get the error sqlalchemy.exc.NoSuchTableError: popular which means the view is not being selected. I am getting the error when I try to select from the view. Creating the view does not throw any error but it does not create the view. Here is my code:
from sqlalchemy import *
import sqlalchemy as db
from sqlalchemy import func
from sqlalchemy import desc
from sqlalchemy import Table
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.expression import Executable, ClauseElement
try:
engine = db.create_engine('postgresql://user:pass#localhost:5432/db_name')
connection = engine.connect()
except:
print('Error establishing DB connection')
# Import metadata
metadata = db.MetaData()
# Import articles, authors and log tables
art = db.Table('articles', metadata, autoload=True, autoload_with=engine)
aut = db.Table('authors', metadata, autoload=True, autoload_with=engine)
log = db.Table('log', metadata, autoload=True, autoload_with=engine)
class CreateView(Executable, ClauseElement):
def __init__(self, name, select):
self.name = name
self.select = select
#compiles(CreateView)
def visit_create_view(element, compiler, **kw):
return "CREATE VIEW %s AS %s" % (
element.name,
compiler.process(element.select, literal_binds=True)
)
# Method to create view with top three articles
def view_top_three():
top_three_view = CreateView('popular', db.select([art.columns.title, func.count(log.columns.path)]) \
.where(func.concat('/article/', art.columns.slug) == log.columns.path) \
.where(log.columns.path != "/") \
.group_by(log.columns.path, art.columns.title) \
.order_by(desc(func.count(log.columns.path))) \
.limit(3))
engine.execute(top_three_view)
v = Table('popular', metadata, autoload=True, autoload_with=engine)
for r in engine.execute(v.select()):
print(r)
# Call the method which creates view and selects from view
view_top_three()
Any help will be appreciated.

Since your CreateView inherits from Executable, and ClauseElement, it is not considered a data changing operation. In other words
engine.execute(top_three_view)
executes the CREATE VIEW statement and then implicitly rollbacks, when the connection is returned to the pool.
Instead it should be a subclass of DDLElement, as shown in the usage recipes wiki. Simply changing the baseclass will allow SQLAlchemy autocommit to work properly.

I found the solution. The issue was to do with autocommit. Setting autocommit to true when creating the engine solved the issue as follows:
engine = db.create_engine('postgresql://user:pass#localhost:5432/db_name').execution_options(autocommit=True)
Special mention to #ilja-everilä

sqlalchemy empty schema and no tables

I am trying to use sqlalchemy via a Database first approach and generate the models for the existing database structure. The db is a standard SQLServer(express).
I can connect to my database and query it via the following
from sqlalchemy import create_engine, MetaData, Table
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
__connectionString = "DSN=databaseDSN;Trusted_Connection=yes"
db_engine = create_engine('mssql+pyodbc:///?odbc_connect=%s' % __connectionString, echo=True)
db_session = scoped_session(sessionmaker(bind=db_engine))
result = db_session.execute("SELECT * FROM debug.BasicTable")
for row in result.fetchmany(10):
print(row)
When I try to reflect the database structure below I am unable to see any of the actual tables and the following raises a NoSuchTableError
myTable= Table('debug.BasicTable', meta, autoload=True, autoload_with=db_engine)
from a common tutorial http://pythoncentral.io/sqlalchemy-faqs/
I should be able to reflect the table objects
Base = declarative_base()
Base.metadata.reflect(db_engine)
meta = MetaData()
meta.reflect(bind=db_engine)
However there are no table objects in meta.tables at all.

This is because debug.BasicTable is most likley not the name of your table. The name of your table is BasicTable and debug is its schema. So:
Table('BasicTable', meta, schema="debug", autoload=True, autoload_with=db_engine)

Reflecting Oracle DB with SQLalchemy, doesn't reflect the columns [duplicate]

I'm a new learner on Python and SQLAlchemy, and I met a curious problem as below.
user = Table('users', meta, autoload=True, autoload_with=engine)
then I
print(user.columns)
it works fine, the output are user.ID, user.Name, etc. But then:
Session = sessionmaker(bind=engine)
session = Session()
session.query(user).order_by(user.id)
shows error:
AttributeError: 'Table' object has no attribute 'id'
I change the "id" to "Name", it's the same error.
I also tried the filter_by method, the same error.
Why this happened?

You could use:
session.query(user).order_by(user.c.id)

Since this is the top answer when you search for this error, I will drop my solution for this as well here.
For my problem, I had to reference the class instead of the table in my models remote_side
so it had to be "Transaction.id" instead of transactions.id
class Transaction(Base):
__tablename__ = "transactions"
...
offset_transaction = relationship(
...
remote_side="Transaction.id",
)
...

For people tumbling into this after many things have changed, this is a solution that works now.
I'm also a new to this, and for me all of those answers failed to work, but using select instead of query worked. Sorry for different table and column names, but this was from an actual query.
Using MSSQL database with ODCB driver.
from sqlalchemy import select, create_engine, MetaData, Table, URL
from sqlalchemy.orm import Session
connect_url = URL.create(
'mssql+pyodbc',
host = 'server',
port = 'port',
database = 'database',
query = dict(driver = 'SQL Server Native Client 11.0'))
engine = create_engine(connect_url)
connection = engine.connect()
metadata = MetaData()
reqgroups = Table('ReqGroup', metadata, autoload_with=engine)
# Print the column names. This works and shows REQGROUPID as one of the columns
print(reqgroups.columns.keys())
with Session(engine) as session:
# Filter with columns REQGROUPID
stmt = select(reqgroups).filter_by(REQGROUPID = "Tarve_ts")
ResultSet = session.execute(stmt).fetchone()
print(ResultSet)

Sqlalchemy if table does not exist

I wrote a module which is to create an empty database file
def create_database():
engine = create_engine("sqlite:///myexample.db", echo=True)
metadata = MetaData(engine)
metadata.create_all()
But in another function, I want to open myexample.db database, and create tables to it if it doesn't already have that table.
EG of the first, subsequent table I would create would be:
Table(Variable_TableName, metadata,
Column('Id', Integer, primary_key=True, nullable=False),
Column('Date', Date),
Column('Volume', Float))
(Since it is initially an empty database, it will have no tables in it, but subsequently, I can add more tables to it. Thats what i'm trying to say.)
Any suggestions?

I've managed to figure out what I intended to do. I used engine.dialect.has_table(engine, Variable_tableName) to check if the database has the table inside. IF it doesn't, then it will proceed to create a table in the database.
Sample code:
engine = create_engine("sqlite:///myexample.db") # Access the DB Engine
if not engine.dialect.has_table(engine, Variable_tableName): # If table don't exist, Create.
metadata = MetaData(engine)
# Create a table with the appropriate Columns
Table(Variable_tableName, metadata,
Column('Id', Integer, primary_key=True, nullable=False),
Column('Date', Date), Column('Country', String),
Column('Brand', String), Column('Price', Float),
# Implement the creation
metadata.create_all()
This seems to be giving me what i'm looking for.

Note that in 'Base.metadata' documentation it states about create_all:
Conditional by default, will not attempt to recreate tables already
present in the target database.
And if you can see that create_all takes these arguments: create_all(self, bind=None, tables=None, checkfirst=True), and according to documentation:
Defaults to True, don't issue CREATEs for tables already present in
the target database.
So if I understand your question correctly, you can just skip the condition.

The accepted answer prints a warning that engine.dialect.has_table() is only for internal use and not part of the public API. The message suggests this as an alternative, which works for me:
import os
import sqlalchemy
# Set up a connection to a SQLite3 DB
test_db = os.getcwd() + "/test.sqlite"
db_connection_string = "sqlite:///" + test_db
engine = create_engine(db_connection_string)
# The recommended way to check for existence
sqlalchemy.inspect(engine).has_table("BOOKS")
See also the SQL Alchemy docs.

For those who define the table first in some models.table file, among other tables.
This is a code snippet for finding the class that represents the table we want to create ( so later we can use the same code to just query it )
But together with the if written above, I still run the code with checkfirst=True
ORMTable.__table__.create(bind=engine, checkfirst=True)
models.table
class TableA(Base):
class TableB(Base):
class NewTableC(Base):
id = Column('id', Text)
name = Column('name', Text)
form
Then in the form action file:
engine = create_engine("sqlite:///myexample.db")
if not engine.dialect.has_table(engine, table_name):
# Added to models.tables the new table I needed ( format Table as written above )
table_models = importlib.import_module('models.tables')
# Grab the class that represents the new table
# table_name = 'NewTableC'
ORMTable = getattr(table_models, table_name)
# checkfirst=True to make sure it doesn't exists
ORMTable.__table__.create(bind=engine, checkfirst=True)

engine.dialect.has_table does not work for me on cx_oracle.
I am getting AttributeError: 'OracleDialect_cx_oracle' object has no attribute 'default_schema_name'
I wrote a workaround function:
from sqlalchemy.engine.base import Engine
def orcl_tab_or_view_exists(in_engine: Engine, in_object: str, in_object_name: str,)-> bool:
"""Checks if Oracle table exists in current in_engine connection
in_object: 'table' | 'view'
in_object_name: table_name | view_name
"""
obj_query = """SELECT {o}_name FROM all_{o}s WHERE owner = SYS_CONTEXT ('userenv', 'current_schema') AND {o}_name = '{on}'
""".format(o=in_object, on=in_object_name.upper())
with in_engine.connect() as connection:
result = connection.execute(obj_query)
return len(list(result)) > 0

This is the code working for me to create all tables of all model classes defined with Base class
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
class YourTable(Base):
__tablename__ = 'your_table'
id = Column(Integer, primary_key = True)
DB_URL="mysql+mysqldb://<user>:<password>#<host>:<port>/<db_name>"
scoped_engine = create_engine(DB_URL)
Base = declarative_base()
Base.metadata.create_all(scoped_engine)

SQLAchemy is never able to locate specified column in row

I'm new to SQLAlchemy (but not ORM's in general) and running into an issue where I literally cannot do anything useful. All I'm able to do is hard code my query, which is what I want to do.
Here is exactly what I'm attempting to do (see Python comments to see what does/doesn't work -
from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
from sqlalchemy import MetaData, Table
import pypyodbc
connection_string = 'mssql+pyodbc://U:P#SERVER/DB'
Base = declarative_base()
class Item(Base):
__tablename__ = 'Items'
id = Column('Id', Integer, primary_key=True)
engine = create_engine(connection_string, echo=True, module=pypyodbc)
sm = sessionmaker(engine)
session = sm()
# WORKS
for row in engine.execute('select top 1 Id from Items'):
print(row)
# DOES NOT WORK
print(session.query(Item).get(111))
The error I'm getting is sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'Items.Id'".
The generated SQL is correct -
SELECT [Items].[Id] AS [Items_Id]
FROM [Items]
WHERE [Items].[Id] = 111
I know I'm pointing to the right database, because if I change __tablename to something non-existent I get an invalid object name error.
Why can't SQLAlchemy find my columns?