I'm using SQLAlchemy (Version: 1.4.44) and I'm having some unexpected results when trying to select columns and using cast on those columns.
First, most of the examples and even current documentation suggests column selection should work by passing an array to the select function like this:
s = select([table.c.col1])
However, I get the following error if I try this:
s = my_table.select([my_table.columns.user_id])
sqlalchemy.exc.ArgumentError: SQL expression for WHERE/HAVING role expected, got [Column('user_id', String(), table=<my_table>)].
Some examples suggest just placing the field directly in the select query.
s = select(table.c.col1)
But this seems to do nothing more than create an idle where-clause out of the field.
I eventually was able to achieve column selection with this approach:
s = my_table.select().with_only_columns(my_table.columns.created_at)
But I am not able to use cast for some reason with this approach.
s = my_table.select().with_only_columns(cast(my_table.columns.created_at, Date))
ValueError: Couldn't parse date string '2022' - value is not a string.
All help appreciated!
I don't think table.select() is common usage. SQLAlchemy is in a big transition right now on its way to 2.0. In 1.4 (and in 2) the following syntax should work, use whatever session handling you already have working I just mean the select(...):
from sqlalchemy.sql import select, cast
from sqlalchemy.dialects.postgresql import INTEGER
class User(Base):
__tablename__ = "users"
id = Column(
Integer, nullable=False, primary_key=True
)
name = Column(Text)
with Session(engine) as session:
u1 = User(name="1")
session.add(u1)
session.commit()
with Session(engine) as session:
my_table = User.__table__
# Cast user name into integer.
print (session.execute(select(cast(my_table.c.name, INTEGER))).all())
Related
The autoincrement argument in SQLAlchemy seems to be only True and False, but I want to set the pre-defined value aid = 1001, the via autoincrement aid = 1002 when the next insert is done.
In SQL, can be changed like:
ALTER TABLE article AUTO_INCREMENT = 1001;
I'm using MySQL and I have tried following, but it doesn't work:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
autoincrement=1001, primary_key=True)
So, how can I get that? Thanks in advance!
You can achieve this by using DDLEvents. This will allow you to run additional SQL statements just after the CREATE TABLE ran. Look at the examples in the link, but I am guessing your code will look similar to below:
from sqlalchemy import event
from sqlalchemy import DDL
event.listen(
Article.__table__,
"after_create",
DDL("ALTER TABLE %(table)s AUTO_INCREMENT = 1001;")
)
According to the docs:
autoincrement –
This flag may be set to False to indicate an integer primary key column that should not be considered to be the “autoincrement” column, that is the integer primary key column which generates values implicitly upon INSERT and whose value is usually returned via the DBAPI cursor.lastrowid attribute. It defaults to True to satisfy the common use case of a table with a single integer primary key column.
So, autoincrement is only a flag to let SQLAlchemy know whether it's the primary key you want to increment.
What you're trying to do is to create a custom autoincrement sequence.
So, your example, I think, should look something like:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.schema import Sequence
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
Sequence('article_aid_seq', start=1001, increment=1),
primary_key=True)
Note, I don't know whether you're using PostgreSQL or not, so you should make note of the following if you are:
The Sequence object also implements special functionality to accommodate Postgresql’s SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a Table object defines a Sequence on its primary key column so that it works with Oracle and Firebird, the Sequence would get in the way of the “implicit” sequence that PG would normally use. For this use case, add the flag optional=True to the Sequence object - this indicates that the Sequence should only be used if the database provides no other option for generating primary key identifiers.
I couldn't get the other answers to work using mysql and flask-migrate so I did the following inside a migration file.
from app import db
db.engine.execute("ALTER TABLE myDB.myTable AUTO_INCREMENT = 2000;")
Be warned that if you regenerated your migration files this will get overwritten.
I know this is an old question but I recently had to figure this out and none of the available answer were quite what I needed. The solution I found relied on Sequence in SQLAlchemy. For whatever reason, I could not get it to work when I called the Sequence constructor within the Column constructor as has been referenced above. As a note, I am using PostgreSQL.
For your answer I would have put it as such:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Sequence, Column, Integer
import os
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Sequence, Integer, create_engine
Base = declarative_base()
def connection():
engine = create_engine(f"postgresql://postgres:{os.getenv('PGPASSWORD')}#localhost:{os.getenv('PGPORT')}/test")
return engine
engine = connection()
class Article(Base):
__tablename__ = 'article'
seq = Sequence('article_aid_seq', start=1001)
aid = Column('aid', Integer, seq, server_default=seq.next_value(), primary_key=True)
Base.metadata.create_all(engine)
This then can be called in PostgreSQL with:
insert into article (aid) values (DEFAULT);
select * from article;
aid
------
1001
(1 row)
Hope this helps someone as it took me a while
You can do it using the mysql_auto_increment table create option. There are mysql_engine and mysql_default_charset options too, which might be also handy:
article = Table(
'article', metadata,
Column('aid', INTEGER(unsigned=True, zerofill=True), primary_key=True),
mysql_engine='InnoDB',
mysql_default_charset='utf8',
mysql_auto_increment='1001',
)
The above will generate:
CREATE TABLE article (
aid INTEGER UNSIGNED ZEROFILL NOT NULL AUTO_INCREMENT,
PRIMARY KEY (aid)
)ENGINE=InnoDB AUTO_INCREMENT=1001 DEFAULT CHARSET=utf8
If your database supports Identity columns*, the starting value can be set like this:
import sqlalchemy as sa
tbl = sa.Table(
't10494033',
sa.MetaData(),
sa.Column('id', sa.Integer, sa.Identity(start=200, always=True), primary_key=True),
)
Resulting in this DDL output:
CREATE TABLE t10494033 (
id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 200),
PRIMARY KEY (id)
)
Identity(..) is ignored if the backend does not support it.
* PostgreSQL 10+, Oracle 12+ and MSSQL, according to the linked documentation above.
I'd like to use the following raw SQL to create an index in PostgreSQL:
CREATE INDEX ix_action_date ON events_map ((action ->> 'action'), date, map_id);
I tried to put this line into the model class's __table_args__ part, but I couldn't. Then I simply solved it by using raw SQL in Alembic migration.
conn = op.get_bind()
conn.execute(text("CREATE INDEX ..."))
and just using a dummy index in __table_args__ like:
Index('ix_action_date')
My only problem is that Alembic doesn't accept the dummy index with the same name, and every time I run a revision --autogenerate, it tells me the following:
SAWarning: Skipped unsupported reflection of expression-based index ix_action_date
% idx_name)
and then it adds the autogenerated index to the migration file:
op.create_index('ix_action_date', 'events_map', [], unique=False)
My question is:
How can I write raw SQL into a __table_args__ Index?
How can I really make my dummy index concept work ? I mean an index which is only compared by name?
How can I write raw SQL into a __table_args__ Index?
To specify formula indexes, you have to provide a text element for the expression
example:
class EventsMap(Base):
__tablename__ = 'events_map'
__table_args__ = (Index('ix_action_date', text("(action->>'action'), date, map_id")),)
map_id = Column(Integer, primary_key=True)
date = Column(DateTime)
action = Column(JSONB)
How can I really make my dummy index concept work ? I mean an index which is only compared by name?
It seems unnecessary to make your dummy index concept work. Either specify the full index expression in the __table_args__ as I've shown above, or omit it completely from the model & delegate index creation as a database migration handled by sqlalchemy.
I am accessing Postgre database using SQLAlchemy models. In one of models I have Column with UUID type.
id = Column(UUID(as_uuid=True), default=uuid.uuid4(), nullable=False, unique=True)
and it works when I try to insert new row (generates new id).
Problem is when I try to fetch Person by id I try like
person = session.query(Person).filter(Person.id.like(some_id)).first()
some_id is string received from client
but then I get error LIKE (Programming Error) operator does not exist: uuid ~~ unknown.
How to fetch/compare UUID column in database through SQLAlchemy ?
don't use like, use =, not == (in ISO-standard SQL, = means equality).
Keep in mind that UUID's are stored in PostgreSQL as binary types, not as text strings, so LIKE makes no sense. You could probably do uuid::text LIKE ? but it would perform very poorly over large sets because you are effectively ensuring that indexes can't be used.
But = works, and is far preferable:
mydb=>select 'd796d940-687f-11e3-bbb6-88ae1de492b9'::uuid = 'd796d940-687f-11e3-bbb6-88ae1de492b9';
?column?
----------
t
(1 row)
I'm having difficulties writing what should be a simple SQL update statement in SQLAlchemy Core. However, I can't find any documentation, examples or tutorials that show how to combine multiple where conditions. I'm sure it's there - just can't find it.
Here's the table:
self.struct = Table('struct',
metadata,
Column('schema_name', String(40), nullable=False,
primary_key=True),
Column('struct_name', String(40), nullable=False,
primary_key=True),
Column('field_type', String(10), nullable=True),
Column('field_len', Integer, nullable=True) )
Here's the insert & update statement:
def struct_put(self, **kv):
try:
i = self.struct.insert()
result = i.execute(**kv)
except exc.IntegrityError: # row already exists - update it:
u = self.struct.update().\
where((self.struct.c.struct_name==kv['struct_name']
and self.struct.c.schema_name==kv['schema_name'])).\
values(field_len=kv['field_len'],
field_type=kv['field_type'])
result = u.execute()
The code handles the insert fine, but updates all rows in the table. Can you help me understand the syntax of this where clause? All suggestions are welcome - thanks in advance.
EDIT: The corrected clause looks like this:
where((and_(self.struct.c.parent_struct_name==kv['parent_struct_name'],
self.struct.c.struct_name==kv['struct_name'],
self.struct.c.schema_name==kv['schema_name']))).\
It's a very simple syntax, but given the many layers of SQLAlchemy it was surprisingly difficult to determine what exactly applied within this context.
It looks to me like you are using the Python "and" operation, which will evaluate to a only one of the clauses surrounding it. You should try using the "and_" function from SQLAlchemy instead. Put those two clauses inside the "and_" function.
You can also use & python operator.
For example:
query.where(
(ModelName.c.column_name == "column_value") &
(ModelName.c.column_name == "column_value)
)
For example, if you had a query like this
user_query = User.select().where(
(User.c.id == 12) &
(User.c.email == "myemail#gmail.com")
)
This will generate a raw SQL like this
select * from users where id = 12 and email = "myemail#gmail.com"
In SQLAlchemy, tablename.c is a special value that you use when constructing conditions that will be treated by SQLAlchemy at runtime.
In this particular case, you're simply saying "update all the rows where the column named struct_name matches the value passed in to struct_put(struct_name="struct_value", schema_name="schema_value"), and the column named schema_name matches the value passed in as schema_name.
In a little script I'm writing using SQLAlchemy and Elixir, I need to get all the distinct values for a particular column. In ordinary SQL it'd be a simple matter of
SELECT DISTINCT `column` FROM `table`;
and I know I could just run that query "manually," but I'd rather stick to the SQLAlchemy declarative syntax (and/or Elixir) if I can. I'm sure it must be possible, I've even seen allusions to this sort of thing in the SQLAlchemy documentation, but I've been hunting through that documentation for hours (as well as that of Elixir) and I just can't seem to actually figure out how it would be done. So what am I missing?
You can query column properties of mapped classes and the Query class has a generative distinct() method:
for value in Session.query(Table.column).distinct():
pass
For this class:
class Assurance(db.Model):
name = Column(String)
you can do this:
assurances = []
for assurance in Assurance.query.distinct(Assurance.name):
assurances.append(assurance.name)
and you will have the list of distinct values
I wanted to count the distinct values, and using .distinct() and .count() would count first, resulting in a single value, then do the distinct. I had to do the following
from sqlalchemy.sql import func
Session.query(func.count(func.distinct(Table.column))
For class,
class User(Base):
name = Column(Text)
id = Column(Integer, primary_key=True)
Method 1: Using load_only
from sqlalchemy.orm import load_only
records= (db_session.query(User).options(load_only(name)).distinct().all())
values = [record[0] if len(record) == 1 else record for record in records] # list of distinct values
Method2: without any imports
records = db_session.query(User.name).distinct().all()
l_values = [record.__dict__[l_columns[0]] for record in records]
for user in session.query(users_table).distinct():
print user.posting_id
SQL Alchemy version 2 encourages the use of the select() function. You can use an SQL Alchemy table to build a select statement that extracts unique values:
select(distinct(table.c.column_name))
SQL Alchemy 2.0 migration ORM usage:
"The biggest visible change in SQLAlchemy 2.0 is the use of Session.execute() in conjunction with select() to run ORM queries, instead of using Session.query()."
Reproducible example using pandas to collect the unique values.
Define and insert the iris dataset
Define an ORM structure for the iris dataset, then use pandas to insert the
data into an SQLite database. Pandas inserts with if_exists="append" argument
so that it keeps the structure defined in SQL Alchemy.
import seaborn
import pandas
from sqlalchemy import create_engine
from sqlalchemy import MetaData, Table, Column, Text, Float
from sqlalchemy.orm import Session
Define metadata and create the table
engine = create_engine('sqlite://')
meta = MetaData()
meta.bind = engine
iris_table = Table('iris',
meta,
Column("sepal_length", Float),
Column("sepal_width", Float),
Column("petal_length", Float),
Column("petal_width", Float),
Column("species", Text))
iris_table.create()
Load data into the table
iris = seaborn.load_dataset("iris")
iris.to_sql(name="iris",
con=engine,
if_exists="append",
index=False,
chunksize=10 ** 6,
)
Select unique values
Re using the iris_table from above.
from sqlalchemy import distinct, select
stmt = select(distinct(iris_table.c.species))
df = pandas.read_sql_query(stmt, engine)
df
# species
# 0 setosa
# 1 versicolor
# 2 virginica
the marked solution showed me an error so I just specified the column and it worked here is the code
for i in (session.query(table_name.c.column_name).distinct()):
print(i)