I'm having difficulties writing what should be a simple SQL update statement in SQLAlchemy Core. However, I can't find any documentation, examples or tutorials that show how to combine multiple where conditions. I'm sure it's there - just can't find it.
Here's the table:
self.struct = Table('struct',
metadata,
Column('schema_name', String(40), nullable=False,
primary_key=True),
Column('struct_name', String(40), nullable=False,
primary_key=True),
Column('field_type', String(10), nullable=True),
Column('field_len', Integer, nullable=True) )
Here's the insert & update statement:
def struct_put(self, **kv):
try:
i = self.struct.insert()
result = i.execute(**kv)
except exc.IntegrityError: # row already exists - update it:
u = self.struct.update().\
where((self.struct.c.struct_name==kv['struct_name']
and self.struct.c.schema_name==kv['schema_name'])).\
values(field_len=kv['field_len'],
field_type=kv['field_type'])
result = u.execute()
The code handles the insert fine, but updates all rows in the table. Can you help me understand the syntax of this where clause? All suggestions are welcome - thanks in advance.
EDIT: The corrected clause looks like this:
where((and_(self.struct.c.parent_struct_name==kv['parent_struct_name'],
self.struct.c.struct_name==kv['struct_name'],
self.struct.c.schema_name==kv['schema_name']))).\
It's a very simple syntax, but given the many layers of SQLAlchemy it was surprisingly difficult to determine what exactly applied within this context.
It looks to me like you are using the Python "and" operation, which will evaluate to a only one of the clauses surrounding it. You should try using the "and_" function from SQLAlchemy instead. Put those two clauses inside the "and_" function.
You can also use & python operator.
For example:
query.where(
(ModelName.c.column_name == "column_value") &
(ModelName.c.column_name == "column_value)
)
For example, if you had a query like this
user_query = User.select().where(
(User.c.id == 12) &
(User.c.email == "myemail#gmail.com")
)
This will generate a raw SQL like this
select * from users where id = 12 and email = "myemail#gmail.com"
In SQLAlchemy, tablename.c is a special value that you use when constructing conditions that will be treated by SQLAlchemy at runtime.
In this particular case, you're simply saying "update all the rows where the column named struct_name matches the value passed in to struct_put(struct_name="struct_value", schema_name="schema_value"), and the column named schema_name matches the value passed in as schema_name.
Related
I'm using SQLAlchemy (Version: 1.4.44) and I'm having some unexpected results when trying to select columns and using cast on those columns.
First, most of the examples and even current documentation suggests column selection should work by passing an array to the select function like this:
s = select([table.c.col1])
However, I get the following error if I try this:
s = my_table.select([my_table.columns.user_id])
sqlalchemy.exc.ArgumentError: SQL expression for WHERE/HAVING role expected, got [Column('user_id', String(), table=<my_table>)].
Some examples suggest just placing the field directly in the select query.
s = select(table.c.col1)
But this seems to do nothing more than create an idle where-clause out of the field.
I eventually was able to achieve column selection with this approach:
s = my_table.select().with_only_columns(my_table.columns.created_at)
But I am not able to use cast for some reason with this approach.
s = my_table.select().with_only_columns(cast(my_table.columns.created_at, Date))
ValueError: Couldn't parse date string '2022' - value is not a string.
All help appreciated!
I don't think table.select() is common usage. SQLAlchemy is in a big transition right now on its way to 2.0. In 1.4 (and in 2) the following syntax should work, use whatever session handling you already have working I just mean the select(...):
from sqlalchemy.sql import select, cast
from sqlalchemy.dialects.postgresql import INTEGER
class User(Base):
__tablename__ = "users"
id = Column(
Integer, nullable=False, primary_key=True
)
name = Column(Text)
with Session(engine) as session:
u1 = User(name="1")
session.add(u1)
session.commit()
with Session(engine) as session:
my_table = User.__table__
# Cast user name into integer.
print (session.execute(select(cast(my_table.c.name, INTEGER))).all())
I'm currently executing this query in one process:
SELECT DISTINCT ON (c.api_key, worker_id) worker_id, c.api_key, a.updated_at, b.user_id, a.country
FROM TABLE_A a
INNER JOIN TABLE_B b ON (b.id = a.user)
INNER JOIN TABLE_C c ON (b.owner = c.id)
WHERE 1=1
AND a.platform = 'x'
AND a.country = 'y'
AND a.access_token is not NULL
ORDER BY c.api_key, worker_id, a.updated_at desc
I'm currently wrapping it using from SQLAlchemy import text and then simply executing
query_results = db.execute(query).fetchall()
list_dicts = [r._asdict() for r in query_results]
df = pd.DataFrame(list_dicts)
and it works, but I would really like to see if it's possible to have it in the other notation, like :
db.query(TABLE_A).filter().join()... etc
Yes, it's possible.
But the exact way to do it will depend on your SQLAlchmey version and how you've setup your SQLAlchemy project and models.
You may want to check out the SQLAlchemy ORM querying guide and the Expression Language Tutorial to see which one fits better your case.
Trying to move some queries I run daily into an automated script. I have one in Postgres like the below:
SELECT regexp_split_to_array(col1, "|")[1] AS item, COUNT(*) AS itemcount FROM Tabel1 GROUP BY item ORDER BY itemcount
In SqlAlchemy I have this:
session.query((func.regexp_split_to_array(model.table1.col1, "|")[1]).label("item"), func.count().label("itemcount")).group_by("item").order_by("itemcount")
Python can't "get_item" since it's not actually a collection. I've looked through the docs and can't seem to find something that would let me do this without running raw SQL using execute (which I can do and works, but was looking for a solution for next time).
SQLAlchemy does support indexing with [...]. If you declare a type of a column that you have to be of type postgresql.ARRAY, then it works:
table2 = Table("table2", meta, Column("col1", postgresql.ARRAY(String)))
q = session.query(table2.c.col1[1])
print(q.statement.compile(dialect=postgresql.dialect()))
# SELECT table2.col1[%(col1_1)s] AS anon_1
# FROM table2
The reason why your code doesn't work is that SQLAlchemy does not know that func.regexp_split_to_array(...) returns an array, since func.foo produces a generic function for convenience. To make it work, we need to make sure SQLAlchemy knows the return type of the function, by specifying the type_ parameter:
q = session.query(func.regexp_split_to_array(table1.c.col1, "|", type_=postgresql.ARRAY(String))[1].label("item"))
print(q.statement.compile(dialect=postgresql.dialect()))
# SELECT (regexp_split_to_array(table1.col1, %(regexp_split_to_array_1)s))[%(regexp_split_to_array_2)s] AS item
# FROM table1
For the web app we are building we used SQLite for testing purposes. Recently we wanted to migrate to PostgreSQL. That's where the problems started:
We have this SQLAlchemy model (simplified)
class Entity(db.Model):
id = db.Column(db.Integer, primary_key=True)
i_want_this = db.Column(db.String)
some_value = db.Column(db.Integer)
I want to group all Entitys by some_value which i did like this (simplified):
db.session.query(Entity, db.func.count()).group_by(Entity.some_value)
In SQLite this worked. In retrospect I see that it does not make sense but SQLite did make sense of it. I can't say for sure which of the entities was returned.
Now in PostgrSQL we get this error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column "entity.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT entity.id AS entity_id, entity.i_want_this AS entity_not...
^
[SQL: 'SELECT entity.id AS entity_id, entity.i_want_this AS entity_i_want_this, count(*) AS count_1 \nFROM entity GROUP BY entity.some_value']
And that error totally makes sense.
So my first question is: Why does SQLite allow this and how does it do it (what hidden aggregation is used)?
My second question is obvious: How would I do it with PostgreSQL?
I'm actually only interested in the count and the first i_want_this value. So I could do this:
groups = db.session.query(db.func.min(Entity.id), db.func.count()).group_by(Entity.some_value)
[(Entity.query.get(id_), count) for id_, count in groups]
But I don't want these additional get queries.
So I want to select the first entity (The entity with the minimal id) and the number of entities grouped by some_value or the first i_want_this and the count grouped by some_value
EDIT to make it clear:
I want to group by some_value (Done)
I want to get the number of entities in each group (Done)
I want to get the entity with the lowest id in each group (Need help on this)
Alternatively I want to get the i_want_this value of the entity with the lowest id in each group (Need help on this)
Concerning your first question, check the documentation:
Each expression in the result-set is then evaluated once for each
group of rows. If the expression is an aggregate expression, it is
evaluated across all rows in the group. Otherwise, it is evaluated
against a single arbitrarily chosen row from within the group. If
there is more than one non-aggregate expression in the result-set,
then all such expressions are evaluated for the same row.
Concerning the second question, you'll probably have to explain what you actually want to achieve, considering that your current query also in SQLite returns more or less random results.
EDIT:
To get the entities with minimum id per group, you can use the Query.select_from construct:
import sqlalchemy.sql as sa_sql
# create the aggregate/grouped query
grouped = sa_sql.select([sa_sql.func.min(Entity.id).label('min_id')])\
.group_by(Entity.some_value)\
.alias('grouped')
# join it with the full entities table
joined = sa_sql.join(Entity, grouped, grouped.c.min_id == Entity.id)
# and let sqlalchemy pull the entities from this statement:
session.query(Entity).select_from(joined)
This will produce the following SQL:
SELECT entities.id AS entities_id,
entities.i_want_this AS entities_i_want_this,
entities.some_value AS entities_some_value
FROM entities JOIN (SELECT min(entities.id) AS min_id
FROM entities GROUP BY entities.some_value) AS grouped
ON grouped.min_id = entities.id
I am accessing Postgre database using SQLAlchemy models. In one of models I have Column with UUID type.
id = Column(UUID(as_uuid=True), default=uuid.uuid4(), nullable=False, unique=True)
and it works when I try to insert new row (generates new id).
Problem is when I try to fetch Person by id I try like
person = session.query(Person).filter(Person.id.like(some_id)).first()
some_id is string received from client
but then I get error LIKE (Programming Error) operator does not exist: uuid ~~ unknown.
How to fetch/compare UUID column in database through SQLAlchemy ?
don't use like, use =, not == (in ISO-standard SQL, = means equality).
Keep in mind that UUID's are stored in PostgreSQL as binary types, not as text strings, so LIKE makes no sense. You could probably do uuid::text LIKE ? but it would perform very poorly over large sets because you are effectively ensuring that indexes can't be used.
But = works, and is far preferable:
mydb=>select 'd796d940-687f-11e3-bbb6-88ae1de492b9'::uuid = 'd796d940-687f-11e3-bbb6-88ae1de492b9';
?column?
----------
t
(1 row)