I'm using async_engine. When I try to execute anything:
async with self.async_engine.connect() as con:
query = "SELECT id, name FROM item LIMIT 50;"
result = await con.execute(f"{query}")
I'm getting:
Exception has occurred: ObjectNotExecutableError
Not an executable object: 'SELECT id, name FROM item LIMIT 50;'
This question was asked before by user #stilmaniac but it is now deleted from SO.
I found it in Google Search cache, here is copy.
I have the same issue so I'm reasking it, but the original version is below:
I'm trying to create tables from metadata as follows:
Base = declarative_base()
properties = Table(
'properties', Base.metadata,
# ...
Column('geolocation', Geography(geometry_type='POINT', srid=4326)),
# ...
)
engine = create_async_engine("postgresql+asyncpg://user:password#postgres/")
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
Gives me the following error:
sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'CREATE INDEX "idx_properties_geolocation" ON "properties" USING GIST ("geolocation")'
Considering this doc
Versions:
OS: macOS 11.4 ARM
SQLAlchemy: 1.4.22
Python: 3.6
As the exception message suggests, the str 'SELECT id, name FROM item LIMIT 50;' is not an executable object. To make it executable, wrap it with sqlalchemy.text.
from sqlalchemy import text
async with self.async_engine.connect() as con:
query = "SELECT id, name FROM item LIMIT 50;"
result = await con.execute(text(query))
async.connection.execute requires that its statement argument
[...] is always an object that is in both the ClauseElement and
Executable hierarchies, including:
Select
Insert, Update, Delete
TextClause and TextualSelect
DDL and objects which inherit from DDLElement
The synchronous connection.execute method permits raw strings, but this is deprecated in v1.4 and has been removed in SQLAlchemy 2.0.
Related
I'm using async_engine. When I try to execute anything:
async with self.async_engine.connect() as con:
query = "SELECT id, name FROM item LIMIT 50;"
result = await con.execute(f"{query}")
I'm getting:
Exception has occurred: ObjectNotExecutableError
Not an executable object: 'SELECT id, name FROM item LIMIT 50;'
This question was asked before by user #stilmaniac but it is now deleted from SO.
I found it in Google Search cache, here is copy.
I have the same issue so I'm reasking it, but the original version is below:
I'm trying to create tables from metadata as follows:
Base = declarative_base()
properties = Table(
'properties', Base.metadata,
# ...
Column('geolocation', Geography(geometry_type='POINT', srid=4326)),
# ...
)
engine = create_async_engine("postgresql+asyncpg://user:password#postgres/")
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
Gives me the following error:
sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'CREATE INDEX "idx_properties_geolocation" ON "properties" USING GIST ("geolocation")'
Considering this doc
Versions:
OS: macOS 11.4 ARM
SQLAlchemy: 1.4.22
Python: 3.6
As the exception message suggests, the str 'SELECT id, name FROM item LIMIT 50;' is not an executable object. To make it executable, wrap it with sqlalchemy.text.
from sqlalchemy import text
async with self.async_engine.connect() as con:
query = "SELECT id, name FROM item LIMIT 50;"
result = await con.execute(text(query))
async.connection.execute requires that its statement argument
[...] is always an object that is in both the ClauseElement and
Executable hierarchies, including:
Select
Insert, Update, Delete
TextClause and TextualSelect
DDL and objects which inherit from DDLElement
The synchronous connection.execute method permits raw strings, but this is deprecated in v1.4 and has been removed in SQLAlchemy 2.0.
I'm using SQLAlchemy 1.4 to execute the following commands below. When I run the table insertion. It works fine as it is.
# Execute table insertion
with engine.connect() as conn:
insert_stmt = table.insert().values(**data)
conn.execute(insert_stmt)
conn.commit()
However, when I run the command inside a function, load, in this case, I'm getting an error that This method is not implemented for SQLAlchemy 2.0
# Execute table insertion inside the load function
def load(conn, table, data):
insert_stmt = table.insert().values(**data)
conn.execute(insert_stmt)
conn.commit()
with engine.connect() as conn:
load(engine, table, data)
Why is this happening?
One of our queries that was working in Python 2 + mxODBC is not working in Python 3 + pyodbc; it raises an error like this: Maximum number of parameters in the sql query is 2100. while connecting to SQL Server. Since both the printed queries have 3000 params, I thought it should fail in both environments, but clearly that doesn't seem to be the case here. In the Python 2 environment, both MSODBC 11 or MSODBC 17 works, so I immediately ruled out a driver related issue.
So my question is:
Is it correct to send a list as multiple params in SQLAlchemy because the param list will be proportional to the length of list? I think it looks a bit strange; I would have preferred concatenating the list into a single string because the DB doesn't understand the list datatype.
Are there any hints on why it would be working in mxODBC but not pyodbc? Does mxODBC optimize something that pyodbc does not? Please let me know if there are any pointers - I can try and paste more info here. (I am still new to debugging SQLAlchemy.)
Footnote: I have seen lot of answers that suggest to chunk the data, but because of 1 and 2, I wonder if I am doing the correct thing in the first place.
(Since it seems to be related to pyodbc, I have raised an internal issue in the official repository.)
import sqlalchemy
import sqlalchemy.orm
from sqlalchemy import MetaData, Table
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.session import Session
Base = declarative_base()
create_tables = """
CREATE TABLE products(
idn NUMERIC(8) PRIMARY KEY
);
"""
check_tables = """
SELECT * FROM products;
"""
insert_values = """
INSERT INTO products
(idn)
values
(1),
(2);
"""
delete_tables = """
DROP TABLE products;
"""
engine = sqlalchemy.create_engine('mssql+pyodbc://user:password#dsn')
connection = engine.connect()
cursor = engine.raw_connection().cursor()
Session = sqlalchemy.orm.sessionmaker(bind=connection)
session = Session()
session.execute(create_tables)
metadata = MetaData(connection)
class Products(Base):
__table__ = Table('products', metadata, autoload=True)
try:
session.execute(check_tables)
session.execute(insert_values)
session.commit()
query = session.query(Products).filter(
Products.idn.in_(list(range(0, 3000)))
)
query.all()
f = open("query.sql", "w")
f.write(str(query))
f.close()
finally:
session.execute(delete_tables)
session.commit()
When you do a straightforward .in_(list_of_values) SQLAlchemy renders the following SQL ...
SELECT team.prov AS team_prov, team.city AS team_city
FROM team
WHERE team.prov IN (?, ?)
... where each value in the IN clause is specified as a separate parameter value. pyodbc sends this to SQL Server as ...
exec sp_prepexec #p1 output,N'#P1 nvarchar(4),#P2 nvarchar(4)',N'SELECT team.prov AS team_prov, team.city AS team_city, team.team_name AS team_team_name
FROM team
WHERE team.prov IN (#P1, #P2)',N'AB',N'ON'
... so you hit the limit of 2100 parameters if your list is very long. Presumably, mxODBC inserted the parameter values inline before sending it to SQL Server, e.g.,
SELECT team.prov AS team_prov, team.city AS team_city
FROM team
WHERE team.prov IN ('AB', 'ON')
You can get SQLAlchemy to do that for you with
provinces = ["AB", "ON"]
stmt = (
session.query(Team)
.filter(
Team.prov.in_(sa.bindparam("p1", expanding=True, literal_execute=True))
)
.statement
)
result = list(session.query(Team).params(p1=provinces).from_statement(stmt))
I have a DDL object (create_function_foo) that contains a create function statement. In first line of it I put DROP FUNCTION IF EXISTS foo; but engine.execute(create_function_foo) returns:
sqlalchemy.exc.InterfaceError: (InterfaceError) Use multi=True when executing multiple statements
I put multi=True as parameter for create_engine, engine.execute_options and engine.execute but it doesn't work.
NOTE: engine if my instance of create_engine
NOTE: I'm using python 3.2 + mysql.connector 1.0.12 + sqlalchemy 0.8.2
create_function_foo = DDL("""\
DROP FUNCTION IF EXISTS foo;
CREATE FUNCTION `foo`(
SID INT
) RETURNS double
READS SQL DATA
BEGIN
...
END
""")
Where I should put it?
multi=True is a requirement for MySql connector. You can not set this flag passing it to SQLAlchemy methods. Do this:
conn = session.connection().connection
cursor = conn.cursor() # get mysql db-api cursor
cursor.execute(sql, multi=True)
More info here: http://www.mail-archive.com/sqlalchemy#googlegroups.com/msg30129.html
Yeah... This seems like a bummer to me. I don't want to use the ORM so the accepted answer didn't work for me.
I did this instead:
with open('sql_statements_file.sql') as sql_file:
for statement in sql_file.read().split(';'):
if len(statement.strip()) > 0:
connection.execute(statement + ';')
And then this failed for a CREATE function.... YMMV.
There are some cases where SQLAlchemy does not provide a generic way at accessing some DBAPI functions, such as as dealing with multiple result sets. In these cases, you should deal with the raw DBAPI connection directly.
From SQLAlchemy documentation:
connection = engine.raw_connection()
try:
cursor = connection.cursor()
cursor.execute("select * from table1; select * from table2")
results_one = cursor.fetchall()
cursor.nextset()
results_two = cursor.fetchall()
cursor.close()
finally:
connection.close()
You can also do the same using mysql connector as seen here:
operation = 'SELECT 1; INSERT INTO t1 VALUES (); SELECT 2'
for result in cursor.execute(operation, multi=True):
if result.with_rows:
print("Rows produced by statement '{}':".format(
result.statement))
print(result.fetchall())
else:
print("Number of rows affected by statement '{}': {}".format(
result.statement, result.rowcount))
How do you execute raw SQL in SQLAlchemy?
I have a python web app that runs on flask and interfaces to the database through SQLAlchemy.
I need a way to run the raw SQL. The query involves multiple table joins along with Inline views.
I've tried:
connection = db.session.connection()
connection.execute( <sql here> )
But I keep getting gateway errors.
Have you tried:
result = db.engine.execute("<sql here>")
or:
from sqlalchemy import text
sql = text('select name from penguins')
result = db.engine.execute(sql)
names = [row[0] for row in result]
print names
Note that db.engine.execute() is "connectionless", which is deprecated in SQLAlchemy 2.0.
SQL Alchemy session objects have their own execute method:
result = db.session.execute('SELECT * FROM my_table WHERE my_column = :val', {'val': 5})
All your application queries should be going through a session object, whether they're raw SQL or not. This ensures that the queries are properly managed by a transaction, which allows multiple queries in the same request to be committed or rolled back as a single unit. Going outside the transaction using the engine or the connection puts you at much greater risk of subtle, possibly hard to detect bugs that can leave you with corrupted data. Each request should be associated with only one transaction, and using db.session will ensure this is the case for your application.
Also take note that execute is designed for parameterized queries. Use parameters, like :val in the example, for any inputs to the query to protect yourself from SQL injection attacks. You can provide the value for these parameters by passing a dict as the second argument, where each key is the name of the parameter as it appears in the query. The exact syntax of the parameter itself may be different depending on your database, but all of the major relational databases support them in some form.
Assuming it's a SELECT query, this will return an iterable of RowProxy objects.
You can access individual columns with a variety of techniques:
for r in result:
print(r[0]) # Access by positional index
print(r['my_column']) # Access by column name as a string
r_dict = dict(r.items()) # convert to dict keyed by column names
Personally, I prefer to convert the results into namedtuples:
from collections import namedtuple
Record = namedtuple('Record', result.keys())
records = [Record(*r) for r in result.fetchall()]
for r in records:
print(r.my_column)
print(r)
If you're not using the Flask-SQLAlchemy extension, you can still easily use a session:
import sqlalchemy
from sqlalchemy.orm import sessionmaker, scoped_session
engine = sqlalchemy.create_engine('my connection string')
Session = scoped_session(sessionmaker(bind=engine))
s = Session()
result = s.execute('SELECT * FROM my_table WHERE my_column = :val', {'val': 5})
docs: SQL Expression Language Tutorial - Using Text
example:
from sqlalchemy.sql import text
connection = engine.connect()
# recommended
cmd = 'select * from Employees where EmployeeGroup = :group'
employeeGroup = 'Staff'
employees = connection.execute(text(cmd), group = employeeGroup)
# or - wee more difficult to interpret the command
employeeGroup = 'Staff'
employees = connection.execute(
text('select * from Employees where EmployeeGroup = :group'),
group = employeeGroup)
# or - notice the requirement to quote 'Staff'
employees = connection.execute(
text("select * from Employees where EmployeeGroup = 'Staff'"))
for employee in employees: logger.debug(employee)
# output
(0, 'Tim', 'Gurra', 'Staff', '991-509-9284')
(1, 'Jim', 'Carey', 'Staff', '832-252-1910')
(2, 'Lee', 'Asher', 'Staff', '897-747-1564')
(3, 'Ben', 'Hayes', 'Staff', '584-255-2631')
You can get the results of SELECT SQL queries using from_statement() and text() as shown here. You don't have to deal with tuples this way. As an example for a class User having the table name users you can try,
from sqlalchemy.sql import text
user = session.query(User).from_statement(
text("""SELECT * FROM users where name=:name""")
).params(name="ed").all()
return user
For SQLAlchemy ≥ 1.4
Starting in SQLAlchemy 1.4, connectionless or implicit execution has been deprecated, i.e.
db.engine.execute(...) # DEPRECATED
as well as bare strings as queries.
The new API requires an explicit connection, e.g.
from sqlalchemy import text
with db.engine.connect() as connection:
result = connection.execute(text("SELECT * FROM ..."))
for row in result:
# ...
Similarly, it’s encouraged to use an existing Session if one is available:
result = session.execute(sqlalchemy.text("SELECT * FROM ..."))
or using parameters:
session.execute(sqlalchemy.text("SELECT * FROM a_table WHERE a_column = :val"),
{'val': 5})
See "Connectionless Execution, Implicit Execution" in the documentation for more details.
result = db.engine.execute(text("<sql here>"))
executes the <sql here> but doesn't commit it unless you're on autocommit mode. So, inserts and updates wouldn't reflect in the database.
To commit after the changes, do
result = db.engine.execute(text("<sql here>").execution_options(autocommit=True))
This is a simplified answer of how to run SQL query from Flask Shell
First, map your module (if your module/app is manage.py in the principal folder and you are in a UNIX Operating system), run:
export FLASK_APP=manage
Run Flask shell
flask shell
Import what we need::
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
from sqlalchemy import text
Run your query:
result = db.engine.execute(text("<sql here>").execution_options(autocommit=True))
This use the currently database connection which has the application.
Flask-SQLAlchemy v: 3.0.x / SQLAlchemy v: 1.4
users = db.session.execute(db.select(User).order_by(User.title.desc()).limit(150)).scalars()
So basically for the latest stable version of the flask-sqlalchemy specifically the documentation suggests using the session.execute() method in conjunction with the db.select(Object).
Have you tried using connection.execute(text( <sql here> ), <bind params here> ) and bind parameters as described in the docs? This can help solve many parameter formatting and performance problems. Maybe the gateway error is a timeout? Bind parameters tend to make complex queries execute substantially faster.
If you want to avoid tuples, another way is by calling the first, one or all methods:
query = db.engine.execute("SELECT * FROM blogs "
"WHERE id = 1 ")
assert query.first().name == "Welcome to my blog"