Develop Reference

Getting (sqlite3.IntegrityError) NOT NULL constraint failed when trying to POST using FastAPI/ SQLite

I am getting the following error
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: SignalJourneyAudienceConstraints.constraintId
[SQL: INSERT INTO "SignalJourneyAudienceConstraints" ("sourceId", "constraintTypeId", "constraintValue", targeting, frequency, period) VALUES (?, ?, ?, ?, ?, ?)]
[parameters: (None, 10, 'STRING', 1, None, None)]
When trying to use the below endpoint. I have made all fields except:
constraintTypeId: int
constraintValue: str
targeting: bool
As optional for the time being.
Endpoint
# mail.py
...
#app.post("/add-new-audience-constraint", status_code=status.HTTP_200_OK)
def add_new_audience_constraint(
sjac: schema.SignalJourneyAudienceConstraints, db: Session = Depends(get_db)
):
"""Sets the main query data."""
new_audience_constraint = models.SignalJourneyAudienceConstraints(
constraintTypeId=sjac.constraintTypeId,
constraintValue=sjac.constraintValue.upper(),
targeting=True,
)
db.add(new_audience_constraint)
db.commit()
return {"message": "Data added successfully."}
Model
# models.py
class SignalJourneyAudienceConstraints(Base):
"""Signal Journey Audience Constraints"""
__tablename__ = "SignalJourneyAudienceConstraints"
constraintId = Column(Integer, primary_key=True)
audienceId = Column(
Integer,
ForeignKey("SignalJourneyAudiences.audienceId"),
primary_key=True,
)
sourceId = Column(Integer, ForeignKey("SignalJourneySources.sourceId"))
constraintTypeId = Column(
Integer, ForeignKey("SignalJourneyConstraintType.constraintTypeId")
)
constraintValue = Column(String)
targeting = Column(Boolean)
frequency = Column(Integer)
period = Column(Integer)
Schema
# schema.py
class SignalJourneyAudienceConstraints(BaseModel):
"""SignalJourneyAudienceConstraints BaseModel."""
constraintId: Optional[int] # PK
audienceId: Optional[int] # FK - SignalJourneyAudiences -> audienceId
sourceId: Optional[int] # FK - SignalJourneySources -> sourceId
constraintTypeId: int # FK - SignalJourneyConstraintType -> constraintTypeId
constraintValue: str
targeting: bool
frequency: Optional[int]
period: Optional[int]
class Config:
"""config class"""
orm_mode = True
I have looked into other SO questions, but none seem to fit what I am experiencing. Hopefully someone can explain what's going on and how to resolve it
ERD

You have two fields listed as primary_key; are they both primary keys? Since you have two fields, the primary key field will not automagically defined as an auto increment field; for that you want to only have one primary key, integer field.
SQLAlchemy will only set the auto_increment property for SQLite automagically when there is a single primary key column and it's defined as an integer.
Since defining a composite key wasn't what you wanted, using constraintId as a single auto incrementing key (as shown in your ERD) is what you want - remove the primary_key entry for audienceId field.

SQLAlchemy: Could not locate any relevant foreign key columns for primary join condition

I have built two tables in my postgres CLI (test2_table: id, name, age, profession, city, country). The second is referenced to my 1st table's id and has a column called wage. When I run the syntax everything is fine.
SELECT name, age, profession, city, country, wage FROM test2_table JOIN salary ON salary.test2_id = test2_table.id;
This works - a table with wages is printed for me to see.
My only problem is when I try the same thing to my SQLAlchemy database class. When I try to join the classes together I get an error:
sqlalchemy.exc.ArgumentError: Could not locate any relevant foreign key columns for primary join condition 'test2_table.id = salary.id' on relationship Salary.wages.
Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation.
my database classes:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Test_db_02(Base):
__tablename__ = 'test2_table'
id = Column('id', Integer, primary_key=True)
name = Column('name', String(40))
age = Column('age', Integer)
profession = Column('profession', String(60))
city = Column('city', String(60))
country = Column('country', String(40))
class Salary(Base):
__tablename__ = 'salary'
id = Column('id', Integer, primary_key=True)
wage = Column('wage', String(20))
test2_id = Column('test2_id', Integer, ForeignKey('test2_table.id'))
wages = relationship("Test_db_02", backref="salary", primaryjoin="Test_db_02.id == Salary.id")
I have a simple report page that prompts the user to chose from a drop-down option selector. Before I tried to join the two tables togteher, my queries were working well, now after my attempt to join I get the error. The reports #app has been scaled down for this example.
Reports #app.route
Session = sessionmaker(bind=engine)
session = Session()
#app.route('/reports', methods=['GET', 'POST'])
if request.method == 'GET':
return render_template('reports.html')
else:
if request.form.get("report_options") == "all_db":
db_entry = session.query(Test_db_02).order_by(Test_db_02.id)
db_entry = db_entry.all()
return render_template('reports.html', db_entry=db_entry)
elif request.form.get("report_options") == "name":
db_entry = session.query(Test_db_02).order_by(Test_db_02.id)
data_name = db_entry.all()
return render_template('reports.html', data_name=data_name)
elif request.form.get("report_options") == "media_prof":
db_entry = session.query(Test_db_02).join(test2_table.salary)
media_prof = db_entry.all()
return render_template('reports.html', media_prof=media_prof)
If I am honest I have read through the sqlalchemy documentation on relationships, joins & foreign keys (and watched some YouTube tutorials) but it still seems a bit confusing..
The main challenge for me is to be able to join two tables together. Once I have accomplished that, I will attempt to iterate through them with the flask/jinja set up.

Your primaryjoin condition in Salary.wages is wrong. Presumably, Salary.idis an autoincrement primary key column and as you have no ForeignKey constraining values of Salary.id to values of Test_db_02.id, its unlikely that you want to include that column in the join condition as you have done:
primaryjoin="Test_db_02.id == Salary.id"
Its more likely that you want to relate the two models through the Salary.test2_id attribute, as you have the ForeignKey constraint on that column:
primaryjoin="Test_db_02.id == Salary.test2_id"

How do I insert the database's default for a column in SQLAlchemy?

I'm trying to insert a row into a Postgresql table that looks like this:
CREATE TABLE launch_ids(
id SERIAL PRIMARY KEY,
launch_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT
(now() at time zone 'utc')
);
My class looks like this:
class LaunchId(Base):
"""Launch ID table for runs"""
__tablename__ = 'launch_ids'
id = Column(Integer, primary_key=True)
launch_time = Column(DateTime)
The launch_time should be managed by the database. I know it's possible to use default=datetime.datetime.utcnow(), but that uses the current time on the client. I know it's possible to use default=func.now(), but that means that if the database's definition of the default changes, then I need to change the default in two places.
Here is what I get when I try to insert a row in launch_ids without specifying a value:
l = LaunchId()
session.add(l)
session.commit()
IntegrityError: (psycopg2.IntegrityError) null value in column "launch_time" violates not-null constraint
DETAIL: Failing row contains (1, null).
[SQL: 'INSERT INTO launch_ids (launch_time) VALUES (%(launch_time)s) RETURNING launch_ids.id'] [parameters: {'launch_time': None}]

Use FetchedValue:
from sqlalchemy.schema import FetchedValue
class LaunchId(Base):
...
launch_time = Column(DateTime, FetchedValue())

Specify the server_default on the column like this:
class LaunchId(Base):
"""Launch ID table for runs"""
__tablename__ = 'launch_ids'
id = Column(Integer, primary_key=True)
launch_time = Column(DateTime, nullable=False
server_default=text("(now() at time zone 'utc')"))
Then adding a new launch_id through the session will work. server_default works differently from default in that it is generated on the server side. Official SQLAlchemy documentation: http://docs.sqlalchemy.org/en/latest/core/defaults.html#server-side-defaults
By specifying nullable=False, this model also becomes a true reflection of the CREATE TABLE you specified, and thus can be generated through Base.metadata.create_all or using alembic.

How to define a table without primary key with SQLAlchemy?

I have a table that does not have a primary key. And I really do not want to apply this constraint to this table.
In SQLAlchemy, I defined the table class by:
class SomeTable(Base):
__table__ = Table('SomeTable', meta, autoload=True, autoload_with=engine)
When I try to query this table, I got:
ArgumentError: Mapper Mapper|SomeTable|SomeTable could not assemble any primary key columns for mapped table 'SomeTable'.
How to loss the constraint that every table must have a primary key?

There is only one way that I know of to circumvent the primary key constraint in SQL Alchemy - it's to map specific column or columns to your table as a primary keys, even if they aren't primary key themselves.
http://docs.sqlalchemy.org/en/latest/faq/ormconfiguration.html#how-do-i-map-a-table-that-has-no-primary-key.

There is no proper solution for this but there are workarounds for it:
Workaround 1
Adding parameter primary_key to the existing column that is not having a primary key will work.
class SomeTable(Base):
__table__ = 'some_table'
some_other_already_existing_column = Column(..., primary_key=True) # just add primary key to it whether or not this column is having primary key or not
Workaround 2
Just declare a new dummy column on the ORM layer, not in actual DB. Just define in SQLalchemy model
class SomeTable(Base):
__table__ = 'some_table'
column_not_exist_in_db = Column(Integer, primary_key=True) # just add for sake of this error, dont add in db

Disclaimer: Oracle only
Oracle databases secretly store something called rowid to uniquely define each record in a table, even if the table doesn't have a primary key. I solved my lack of primary key problem (which I did not cause!) by constructing my ORM object like:
class MyTable(Base)
__tablename__ = 'stupid_poorly_designed_table'
rowid = Column(String, primary_key=True)
column_a = Column(String)
column_b = Column(String)
...
You can see what rowid actually looks like (it's a hex value I believe) by running
SELECT rowid FROM stupid_poorly_designed_table
GO

Here is an example using __mapper_args__ and a synthetic primary_key. Because the table is time-series oriented data, there is no need for a primary key. All rows can be unique addresses with a (timestamp, pair) tuple.
class Candle(Base):
__tablename__ = "ohlvc_candle"
__table_args__ = (
sa.UniqueConstraint('pair_id', 'timestamp'),
)
#: Start second of the candle
timestamp = sa.Column(sa.TIMESTAMP(timezone=True), nullable=False)
open = sa.Column(sa.Float, nullable=False)
close = sa.Column(sa.Float, nullable=False)
high = sa.Column(sa.Float, nullable=False)
low = sa.Column(sa.Float, nullable=False)
volume = sa.Column(sa.Float, nullable=False)
pair_id = sa.Column(sa.ForeignKey("pair.id"), nullable=False)
pair = orm.relationship(Pair,
backref=orm.backref("candles",
lazy="dynamic",
cascade="all, delete-orphan",
single_parent=True, ), )
__mapper_args__ = {
"primary_key": [pair_id, timestamp]
}

MSSQL Tested
I know this thread is ancient but I spent way too long getting this to work to not share it :)
from sqlalchemy import Table, event
from sqlalchemy.ext.compiler import compiles
from sqlalchemy import Column
from sqlalchemy import Integer
class RowID(Column):
pass
#compiles(RowID)
def compile_mycolumn(element, compiler, **kw):
return "row_number() OVER (ORDER BY (SELECT NULL))"
#event.listens_for(Table, "after_parent_attach")
def after_parent_attach(target, parent):
if not target.primary_key:
# if no pkey create our own one based on returned rowid
# this is untested for writing stuff - likely wont work
logging.info("No pkey defined for table, using rownumber %s", target)
target.append_column(RowID('row_id', Integer, primary_key=True))

https://docs-sqlalchemy-org.translate.goog/en/14/faq/ormconfiguration.html?_x_tr_sl=auto&_x_tr_tl=ru&_x_tr_hl=ru#how-do-i-map-a-table-that-has-no-primary-key
One way from there:
In SQLAlchemy ORM, to map to a specific table, there must be at least one column designated as the primary key column; multi-column composite primary keys are of course also perfectly possible. These columns do not need to be known to the database as primary key columns. The columns only need to behave like a primary key, such as a non-nullable unique identifier for a row.
my code:
from ..meta import Base, Column, Integer, Date
class ActiveMinutesByDate(Base):
__tablename__ = "user_computer_active_minutes_by_date"
user_computer_id = Column(Integer(), nullable=False, primary_key=True)
user_computer_date_check = Column(Date(), default=None, primary_key=True)
user_computer_active_minutes = Column(Integer(), nullable=True)

The solution I found is to add an auto-incrementing primary key column to the table, then use that as your primary key. The database should deal with everything else beyond that.

Sqlalchemy Sqlite Autoincrement not behaving as expected

I have some trouble making my script incrementing my PK in a correct way. Following the sqlalchemy documentation some special configuration has to be done in order to make it work with sqlite. Here is my script:
def db_stuff():
engine = create_engine('sqlite:///test.db', echo=True)
metadata = MetaData()
db = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
Column('fullname', String),
Column('password', String),
sqlite_autoincrement=True)
metadata.create_all(engine)
return engine.connect(),db
def add_to_db():
ret = db_stuff()
conn = ret[0]
db = ret[1]
try:
conn.execute("INSERT INTO users VALUES ('1','john','smith john','23')")
result = conn.execute(db.select())
for row in result:
print row
finally:
conn.close()
It would be cool if you could help me figuring out what I'm missing here, I start to be desperate...
The problem is that the "id" is not incremented each time and i get an error that it should be unique when I run the script twice.
TIA

try
conn.execute("INSERT INTO users(name, fullname, password) VALUES ('john','smith john','23')")
id is autoincrement hence we should not pass it, however we need to specify what other parameters represent in the table i.e. where should the values ('john', 'smith john', '23') should go.
It should work.

Do this:
conn.execute("INSERT INTO users VALUES ('john','smith john','23')")
You are setting the id to 1 - always. Just leave it and it will be filled due auto-increment.