Develop Reference

Select specific columns with cast using SQLAlchemy

I'm using SQLAlchemy (Version: 1.4.44) and I'm having some unexpected results when trying to select columns and using cast on those columns.
First, most of the examples and even current documentation suggests column selection should work by passing an array to the select function like this:
s = select([table.c.col1])
However, I get the following error if I try this:
s = my_table.select([my_table.columns.user_id])
sqlalchemy.exc.ArgumentError: SQL expression for WHERE/HAVING role expected, got [Column('user_id', String(), table=<my_table>)].
Some examples suggest just placing the field directly in the select query.
s = select(table.c.col1)
But this seems to do nothing more than create an idle where-clause out of the field.
I eventually was able to achieve column selection with this approach:
s = my_table.select().with_only_columns(my_table.columns.created_at)
But I am not able to use cast for some reason with this approach.
s = my_table.select().with_only_columns(cast(my_table.columns.created_at, Date))
ValueError: Couldn't parse date string '2022' - value is not a string.
All help appreciated!

I don't think table.select() is common usage. SQLAlchemy is in a big transition right now on its way to 2.0. In 1.4 (and in 2) the following syntax should work, use whatever session handling you already have working I just mean the select(...):
from sqlalchemy.sql import select, cast
from sqlalchemy.dialects.postgresql import INTEGER
class User(Base):
__tablename__ = "users"
id = Column(
Integer, nullable=False, primary_key=True
)
name = Column(Text)
with Session(engine) as session:
u1 = User(name="1")
session.add(u1)
session.commit()
with Session(engine) as session:
my_table = User.__table__
# Cast user name into integer.
print (session.execute(select(cast(my_table.c.name, INTEGER))).all())

How to fix 'Invalid input syntax for integer' issue while using sqlalchemy Declarative API

I'm actually building a little utils which aims to take flat csv/excel file and populate a target database on MS Access - as I'm working on a Mac, i'm developping it using Postgres...
So I developped a part which deals with messy input (csv/excel) forms (several heading, etc) but that's not my issue at the moment.
On the other hand, I made my Database model using SQLAlchemy Declarative Base API.
I'm facing issue when importing data in some tables:
- Split flat record to several objects
- Check (SELECT) if the record doesn't exists yet based on uniqueness contraints
- If it doesn't exists I create object else I use the existing one
- Propagate keys information to related object
For some tables I'm using the auto_increment arguments but sometimes the record has its own ID (in input file) so I should it for insert/select in my tables and sometimes no ID so I have to create a new technical Id for my table.
Example: I have a record with for primary key -obsr25644- and sometimes nothing so I use a default value created with uuid.
So below the stacktrace when doing selectoperation on a my table. The same error occurs when working on existing data - obsr25644 - and generated uuid - 'a8098c1a-f86e-11da-bd1a-00112444be1e'
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation) **invalid input syntax for integer**: "obsr25644"
LINE 3: WHERE "Location"."Id_observer" = 'obsr25644'
As you can see below, "Location"."Id_observer" is declared as String(255). I don't understand why the error is related to 'integer'.
[SQL: SELECT "Location"."Id_location" AS "Location_Id_location", [...], "Location"."Id_observer" AS "Location_Id_observer",
FROM "Location"
WHERE "Location"."Id_observer" = %(Id_observer_1)s
LIMIT %(param_1)s]
[parameters: {'Id_observer_1': 'obsr25644', 'param_1': 1}]
class LocationModel(UniqueMixin, Base):
__tablename__ = 'Location'
# Primary key
Id_location = Column(Integer, primary_key=True, autoincrement=True)
[...]
Id_observer = Column(String(255), ForeignKey('Observer.Id_observer'))
observer = relationship("ObserverModel", load_on_pending=True, back_populates="location")
class ObserverModel(UniqueMixin, Base):
__tablename__ = 'Observer'
# Primary key
Id_observer = Column(String(255), primary_key=True, default=UniqueMixin.unique_hash())
[...]
# Relationship
location = relationship("LocationModel", load_on_pending=True, back_populates="observer")
Note :UniqueMixin.unique_hash() returns uuid.uuid4().hex

Is it possible to have a list or array in a database using SQLAlchemy?

I'm trying to build a system that allows users to make 'projects'. These projects have a fairly simple syntax, they just need an ID, a name, optionally a description and the participants.
Because I want users to be able to add or remove other users from the project without having to input the entire list of users again, I want to make use of a string or array or some such method instead of a string.
However, I'm stuck with trying to input it. I initially tried a regular list, but SQLalchemy didn't accept that. After a google search, it appears that's not possible? Unless I simply haven't come upon it.
I am now trying to use an Array instead. My current code looks like this:
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.ARRAY(db.Integer))
But this gives the error: in _compiler_dispatch raise exc.UnsupportedCompilationError(visitor, cls)
sqlalchemy.exc.CompileError: (in table 'project', column 'project_participants'): Compiler can't render element of type
Before, I tried leaving (db.Integer) out or replacing it with just Integer (because I had seen this with other people with similar problems) like this:
project_participants = db.Column(db.ARRAY(Integer))
Which gives the error that 'Integer' is not defined or, in case of leaving it out altogether, this error:
TypeError: init() missing 1 required positional argument:
'item_type'
I'm hoping to add the array to the database so that I can use it to append new users or delete users without having to make the system's user input all allowed users all over again when he just wants to add or delete one.

First i recommend you strongly to save your participants data in an additional table.
You can add a m:n relation between your DBProject-Table and your Participants-Table.
Anything else would be against any best practice use of databases.
Saving your participants as an Array in your table makes it impossible or at least very uncomfortable to filter by participants in a SQL-query.
But if you have a good reason to ignore that recommendation you can use pickle to make SQLAlchemy transform your array into a string while writing into your database.
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.PickleType, nullable=True)
Using that construct you can basicalliy put any object (if not exceeding a database specific maximum size) into a database field.
Save data:
dbproject_object = DBProject()
dbproject_object.name = "a_name"
dbproject_object.participants = ["you","him","another one"]
session.add(dbproject_object)
session.commit()
Read Data:
participants_array = db.session.query(DBProject).filter(name == "a_name").one().participants
Result:
participants_array : ["you","him","another one"]

Can't find django auth user with PostgreSQL

from django.contrib.auth.models import User as DjangoUser
class Ward(models.Model):
user = models.ForeignKey(DjangoUser, related_name='wards')
group = models.ForeignKey(Group, related_name='wards')
This is my django model and I use this filter.
Group.objects.filter(wards__user=_user).all()
I used this code in sqlite3, it works well.
But, it doesn't work in PostgreSQL.
operator does not exist: character varying = integer
LINE 1: ...rchive_ward"."group_id" ) WHERE "archive_ward"."user_id" = 1
I think it is caused by user_id field in archive_ward tables.
I found this field's data type is character.varying(20).
What can I do for this code?

Try removing the user table in the database and adding it again.
create a new one from scratch. Syncing database again will work..
or else You can do like this Way raw_query
You cannot compare an integer with a varchar. PostgreSQL is strict and does not do any magic typecasting for you. I'm guessing SQLServer does typecasting automagically (which is a bad thing).
If you want to compare these two different beasts, you will have to cast one to the other using the casting syntax ::

The Postgres error means you're comparing an integer to a string:
operator does not exist: character varying = integer
You could change the database model so user_id is of an integer type. Or you could cast the integer to string in Python:
Group.objects.filter(wards__user=str(_user)).all()

sqlalchemy postgresql where int = string

I have 0 experience with postgresql and am deploying an app written in python using sqlalchemy to a server with postgres.
For development, I used an sqlite server.
Things are going pretty smoothly, but I hit a bump I don't know how to resolve.
I have three tables that look like that
class Car(db.Model):
id= db.Column(db.Integer, primary_key=True)
...
class Truck(db.Model):
id= db.Column(db.String(32), primary_key=True)
...
class Vehicles(db.Model):
id= db.Column(db.Integer, primary_key=True)
type= db.Column(db.String) #This is either 'car' or 'truck'
value= db.Column(db.String) #That's the car or truck id
...
I have a query that selects from Vehicles where type = 'car' AND value = 10
This is throwing an error:
sqlalchemy.exc.ProgrammingError: (ProgrammingError) operator does not exist: integer = character varying
So I guess this is because Car.id is an int and Vehicle.value is a string..
How to write this query in sqlalchemy? Is there a way to write it and make it compatible with my sqlite dev environment and the pgsql production?
currently it looks like that
db.session.query(Vehicle).filter(Car.id == Vehicle.value)
PS: The truck id has to be a string and the car id has to be an int. I don't have control over that.

Simply cast to a string:
db.session.query(Vehicle).filter(str(Car.id) == Vehicle.value)
if Car.id is a local variable that is an int.
If you need to use this in a join, have the database cast it to a string:
from sqlalchemy.sql.expression import cast
db.session.query(Vehicle).filter(cast(Car.id, sqlalchemy.String) == Vehicle.value)
If the string value in the other column contains digits and possibly whitespace you may have to consider trimming, or instead casting the string value to an integer (and leave the integer column an integer).