I'd like to use the following raw SQL to create an index in PostgreSQL:
CREATE INDEX ix_action_date ON events_map ((action ->> 'action'), date, map_id);
I tried to put this line into the model class's __table_args__ part, but I couldn't. Then I simply solved it by using raw SQL in Alembic migration.
conn = op.get_bind()
conn.execute(text("CREATE INDEX ..."))
and just using a dummy index in __table_args__ like:
Index('ix_action_date')
My only problem is that Alembic doesn't accept the dummy index with the same name, and every time I run a revision --autogenerate, it tells me the following:
SAWarning: Skipped unsupported reflection of expression-based index ix_action_date
% idx_name)
and then it adds the autogenerated index to the migration file:
op.create_index('ix_action_date', 'events_map', [], unique=False)
My question is:
How can I write raw SQL into a __table_args__ Index?
How can I really make my dummy index concept work ? I mean an index which is only compared by name?
How can I write raw SQL into a __table_args__ Index?
To specify formula indexes, you have to provide a text element for the expression
example:
class EventsMap(Base):
__tablename__ = 'events_map'
__table_args__ = (Index('ix_action_date', text("(action->>'action'), date, map_id")),)
map_id = Column(Integer, primary_key=True)
date = Column(DateTime)
action = Column(JSONB)
How can I really make my dummy index concept work ? I mean an index which is only compared by name?
It seems unnecessary to make your dummy index concept work. Either specify the full index expression in the __table_args__ as I've shown above, or omit it completely from the model & delegate index creation as a database migration handled by sqlalchemy.
Related
I'm using SQLAlchemy (Version: 1.4.44) and I'm having some unexpected results when trying to select columns and using cast on those columns.
First, most of the examples and even current documentation suggests column selection should work by passing an array to the select function like this:
s = select([table.c.col1])
However, I get the following error if I try this:
s = my_table.select([my_table.columns.user_id])
sqlalchemy.exc.ArgumentError: SQL expression for WHERE/HAVING role expected, got [Column('user_id', String(), table=<my_table>)].
Some examples suggest just placing the field directly in the select query.
s = select(table.c.col1)
But this seems to do nothing more than create an idle where-clause out of the field.
I eventually was able to achieve column selection with this approach:
s = my_table.select().with_only_columns(my_table.columns.created_at)
But I am not able to use cast for some reason with this approach.
s = my_table.select().with_only_columns(cast(my_table.columns.created_at, Date))
ValueError: Couldn't parse date string '2022' - value is not a string.
All help appreciated!
I don't think table.select() is common usage. SQLAlchemy is in a big transition right now on its way to 2.0. In 1.4 (and in 2) the following syntax should work, use whatever session handling you already have working I just mean the select(...):
from sqlalchemy.sql import select, cast
from sqlalchemy.dialects.postgresql import INTEGER
class User(Base):
__tablename__ = "users"
id = Column(
Integer, nullable=False, primary_key=True
)
name = Column(Text)
with Session(engine) as session:
u1 = User(name="1")
session.add(u1)
session.commit()
with Session(engine) as session:
my_table = User.__table__
# Cast user name into integer.
print (session.execute(select(cast(my_table.c.name, INTEGER))).all())
I have MS Access DB file (.accdb), and inside of file stored photos of persons as attachments. In table constructor I see only one field "photos" with type "Attachment". Actually there three hidden fields with names: photos.FileData, photos.FileName, photos.FileType. For parsing these fields I created following class:
class Person:
__tablename__ = 'persons'
name = Column(String(255), name='name')
photos_data = Column(String, name='photos.FileData', quote=False)
....
If I try to get all attributes of Person in same time, as following:
persons = session.query(Person)
I get error in following generated piece of SQL statement:
SELECT ... [persons].photos.FileData AS persons_photos.FileData ...;
As you can see there dot sign present in alias, which raises ODBC error. I can avoid such behavior to request FileData as separate value:
persons = session.query(Person.photos_data.label('photos_data'))
Or I can use raw SQL without aliases. But This is not normal ORM way that I need, because I have to manually construct Persons object each time after request to DB.
Is it possible to set own label to Column during its declaration or even disable label for selected column?
I saw this great answer, but seems this is not applicable to me. Below statement doesn't work properly:
photos_data = Column(String, name='photos.FileData', quote=False).label('photos_data')
How does one add a UniqueConstraint which is Case Insensitive using SQLalchemy?
In some databases, string columns are case-insensitive by default (MySQL, SQL Server), so you wouldn't need to do anything extra.
In others, you can create a functional index that enforces the case-insensitive unique constraint:
Index('myIndex', func.lower(mytable.c.myColumn), unique=True)
You can also specify a case-insensitive collation for the column if the database supports it. For instance SQLite has a 'NOCASE' collation:
myColumn = Column(String(255), collation='NOCASE', nullable=False)
See http://docs.sqlalchemy.org/en/latest/core/type_basics.html?highlight=collation#sqlalchemy.types.String.params.collation
You may also specify a user-defined type for your column if your database provides a suitable one. PostgreSQL has a citext data type which is case-insensitive. See https://github.com/mahmoudimus/sqlalchemy-citext
Finally you can customize the DDL to create the constraint for your specific database.
To add upon #jspcal's answer, if the model is defined using a class, then one you would have to either instantiate it independently after declaring the model or use the text construct.
i.e.
from sqlalchemy.sql.expressions import func
class User(Base):
__tablename__ = 'user'
username = Column('username', String(24), nullable=False)
Index('user_username_index', func.lower(User.username), unique=True)
using the text construct:
from sqlalchemy.sql.expressions import text
class User(Base):
__tablename__ = 'user'
__table_args__ = (
Index('ix_user_name', text('LOWER(username)')),
)
username = Column('username', String(24), nullable=False)
NB: table_args needs to be a tuple or dict, hence the need for that trailing comma inside the parenthesis.
That will create an index on username column of table user in lowercase form. Therefore, data stored in this column is unique and case insensitive.
If I have a SQLAlchemy declarative model like below:
class Test(Model):
__tablename__ = 'tests'
id = Column(Integer, Sequence('test_id_seq'), primary_key=True)
...
Atest_id = Column(Integer, ForeignKey('Atests.id'), nullable=True)
Btest_id = Column(Integer, ForeignKey('Btests.id'), nullable=True)
Ctest_id = Column(Integer, ForeignKey('Ctests.id'), nullable=True)
Dtest_id = Column(Integer, ForeignKey('Dtests.id'), nullable=True)
Etest_id = Column(Integer, ForeignKey('Etests.id'), nullable=True)
...
date = Column(DateTime)
status = Column(String(20)) # pass, fail, needs_review
And I would like to ensure that only one of the *test_id foreign keys is present in a given row, how might I accomplish that in SQLAlchemy?
I see that there is an SQLAlchemy CheckConstraint object (see docs), but MySQL does not support check constraints.
The data model has interaction outside of SQLAlchemy, so preferably it would be a database-level check (MySQL)
Well, considering your requisites "The data model has interaction outside of SQLAlchemy, so preferably it would be a database-level check (MySQL)" and 'ensure that only one [..] is not null'. I think the best approach is to write a trigger like this:
DELIMITER $$
CREATE TRIGGER check_null_insert BEFORE INSERT
ON my_table
FOR EACH ROW BEGIN
IF CHAR_LENGTH(CONCAT_WS('', NEW.a-NEW.a, NEW.b-NEW.b, NEW.c-NEW.c)) = 1 THEN
UPDATE `Error: Only one value of *test_id must be not null` SET z=0;
END IF;
END$$
DELIMITER ;
Some tricks and considerations:
IF STATEMENT: In order to avoid the tedious writing of check each column is not null while others are null, I did this trick: Reduce each column to one character and check how many characters exist. Note that NEW.a-NEW.a always returns 1 character if NEW.a is an Integer, NULL returns 0 characters and the operation NULL-NULL returns NULL on MySQL.
ERROR TRIGGERING: I suppose you want to raise an error, so how to do this on MySQL? You didn't mention the MySQL version. Only on MySQL 5.5 you can use the SIGNAL syntax to throw an exception. So the more portable way is issuing an invalid statement like: UPDATE xx SET z=0. If you are using MySQL 5.5 you could use: signal sqlstate '45000' set message_text = 'Error: Only one value of *test_id must be not null'; instead of UPDATE `Error: Only one value of *test_id must be not null` SET z=0;
Also, I think you want to check this on updates too, so use:
DELIMITER $$
CREATE TRIGGER check_null_update BEFORE UPDATE
ON my_table
FOR EACH ROW BEGIN
IF CHAR_LENGTH(CONCAT_WS('', NEW.a-NEW.a, NEW.b-NEW.b, NEW.c-NEW.c)) = 1 THEN
UPDATE `Error: Only one value of *test_id must be not null` SET z=0;
END IF;
END$$
DELIMITER ;
Or create a stored procedure and call it.
Update
For databases that supports check constraints, the code is more simple, see this example for SQL Server:
CREATE TABLE MyTable (col1 INT NULL, col2 INT NULL, col3 INT NULL);
GO
ALTER TABLE MyTable
ADD CONSTRAINT CheckOnlyOneColumnIsNull
CHECK (
LEN(CONCAT(col1-col1, col2-col2, col3-col3)) = 1
)
GO
I am accessing Postgre database using SQLAlchemy models. In one of models I have Column with UUID type.
id = Column(UUID(as_uuid=True), default=uuid.uuid4(), nullable=False, unique=True)
and it works when I try to insert new row (generates new id).
Problem is when I try to fetch Person by id I try like
person = session.query(Person).filter(Person.id.like(some_id)).first()
some_id is string received from client
but then I get error LIKE (Programming Error) operator does not exist: uuid ~~ unknown.
How to fetch/compare UUID column in database through SQLAlchemy ?
don't use like, use =, not == (in ISO-standard SQL, = means equality).
Keep in mind that UUID's are stored in PostgreSQL as binary types, not as text strings, so LIKE makes no sense. You could probably do uuid::text LIKE ? but it would perform very poorly over large sets because you are effectively ensuring that indexes can't be used.
But = works, and is far preferable:
mydb=>select 'd796d940-687f-11e3-bbb6-88ae1de492b9'::uuid = 'd796d940-687f-11e3-bbb6-88ae1de492b9';
?column?
----------
t
(1 row)