Develop Reference

How set start of auto increment in flask-sqlalchemy [duplicate]

The autoincrement argument in SQLAlchemy seems to be only True and False, but I want to set the pre-defined value aid = 1001, the via autoincrement aid = 1002 when the next insert is done.
In SQL, can be changed like:
ALTER TABLE article AUTO_INCREMENT = 1001;
I'm using MySQL and I have tried following, but it doesn't work:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
autoincrement=1001, primary_key=True)
So, how can I get that? Thanks in advance!

You can achieve this by using DDLEvents. This will allow you to run additional SQL statements just after the CREATE TABLE ran. Look at the examples in the link, but I am guessing your code will look similar to below:
from sqlalchemy import event
from sqlalchemy import DDL
event.listen(
Article.__table__,
"after_create",
DDL("ALTER TABLE %(table)s AUTO_INCREMENT = 1001;")
)

According to the docs:
autoincrement –
This flag may be set to False to indicate an integer primary key column that should not be considered to be the “autoincrement” column, that is the integer primary key column which generates values implicitly upon INSERT and whose value is usually returned via the DBAPI cursor.lastrowid attribute. It defaults to True to satisfy the common use case of a table with a single integer primary key column.
So, autoincrement is only a flag to let SQLAlchemy know whether it's the primary key you want to increment.
What you're trying to do is to create a custom autoincrement sequence.
So, your example, I think, should look something like:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.schema import Sequence
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
Sequence('article_aid_seq', start=1001, increment=1),
primary_key=True)
Note, I don't know whether you're using PostgreSQL or not, so you should make note of the following if you are:
The Sequence object also implements special functionality to accommodate Postgresql’s SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a Table object defines a Sequence on its primary key column so that it works with Oracle and Firebird, the Sequence would get in the way of the “implicit” sequence that PG would normally use. For this use case, add the flag optional=True to the Sequence object - this indicates that the Sequence should only be used if the database provides no other option for generating primary key identifiers.

I couldn't get the other answers to work using mysql and flask-migrate so I did the following inside a migration file.
from app import db
db.engine.execute("ALTER TABLE myDB.myTable AUTO_INCREMENT = 2000;")
Be warned that if you regenerated your migration files this will get overwritten.

I know this is an old question but I recently had to figure this out and none of the available answer were quite what I needed. The solution I found relied on Sequence in SQLAlchemy. For whatever reason, I could not get it to work when I called the Sequence constructor within the Column constructor as has been referenced above. As a note, I am using PostgreSQL.
For your answer I would have put it as such:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Sequence, Column, Integer
import os
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Sequence, Integer, create_engine
Base = declarative_base()
def connection():
engine = create_engine(f"postgresql://postgres:{os.getenv('PGPASSWORD')}#localhost:{os.getenv('PGPORT')}/test")
return engine
engine = connection()
class Article(Base):
__tablename__ = 'article'
seq = Sequence('article_aid_seq', start=1001)
aid = Column('aid', Integer, seq, server_default=seq.next_value(), primary_key=True)
Base.metadata.create_all(engine)
This then can be called in PostgreSQL with:
insert into article (aid) values (DEFAULT);
select * from article;
aid
------
1001
(1 row)
Hope this helps someone as it took me a while

You can do it using the mysql_auto_increment table create option. There are mysql_engine and mysql_default_charset options too, which might be also handy:
article = Table(
'article', metadata,
Column('aid', INTEGER(unsigned=True, zerofill=True), primary_key=True),
mysql_engine='InnoDB',
mysql_default_charset='utf8',
mysql_auto_increment='1001',
)
The above will generate:
CREATE TABLE article (
aid INTEGER UNSIGNED ZEROFILL NOT NULL AUTO_INCREMENT,
PRIMARY KEY (aid)
)ENGINE=InnoDB AUTO_INCREMENT=1001 DEFAULT CHARSET=utf8

If your database supports Identity columns*, the starting value can be set like this:
import sqlalchemy as sa
tbl = sa.Table(
't10494033',
sa.MetaData(),
sa.Column('id', sa.Integer, sa.Identity(start=200, always=True), primary_key=True),
)
Resulting in this DDL output:
CREATE TABLE t10494033 (
id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 200),
PRIMARY KEY (id)
)
Identity(..) is ignored if the backend does not support it.
* PostgreSQL 10+, Oracle 12+ and MSSQL, according to the linked documentation above.

SQLAlchemy cannot autoload an mssql temporary table

I'm not able to connect to temporary tables created on an SQL server using SQLAlchemy.
I connect to the server:
engine = create_engine(URL, poolclass=StaticPool)
I fill a temporary table with data from a pandas dataframe:
df_tmp.to_sql('#table_test', con=engine)
The table exists on the server:
res = engine.execute('SELECT * FROM tempdb..#table_test')
print(res)
which returns a list of tuples of my data. But then when I try to make an SQLAlchemy table it fails with a NoSuchTableError:
from sqlalchemy import create_engine, MetaData, Table
metadata = MetaData(engine)
metadata.create_all()
table = Table('#table_test', metadata, autoload=True, autoload_with=engine)
I also tried this, which gives the same error:
table = Table('tempdb..#table_test', metadata, autoload=True, autoload_with=engine)
And I also tried creating a blank table with an SQL command, which gives the same error when I try to read it with SQLAlchemy:
engine.execute('CREATE TABLE #table_test (id_number INT, name TEXT)')
Does SQLAlchemy support temporary tables? If so what is going wrong here? I'd like to have the temporary table as an sqlalchemy.schema.Table object if possible, as then it fits with all my other code.

(re: comments to the question)
Actually, it is a limitation of the current mechanism by which SQLAlchemy's mssql dialect checks for the existence of a table. It queries INFORMATION_SCHEMA.TABLES for the current catalog (database), and #temp tables do not appear in that view. They do appear — after a fashion, and in a not-particularly-helpful way — if we USE tempdb and then query INFORMATION_SCHEMA.TABLES from there.
For now, I have created a GitHub issue here to see if we can improve on this.
Update 2020-09-01
The changes for the above GitHub issue have been merged into SQLAlchemy's master branch and will be included in version 1.4. If you want to take advantage of this feature before 1.4 is officially released you can install SQLAlchemy via
pip install git+https://github.com/sqlalchemy/sqlalchemy.git

How to create "GENERATED BY DEFAULT AS IDENTITY" field in Firebird 3 via SQLAlchemy?

I want create column ID via SQLAlchemy.
SQL seems like:
CREATE TABLE OBJECT (
ID INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
NAME VARCHAR(15)
)
Or how to create table with autoincrement field?
Column(autoincrement='auto')
default for primary_key, but that not work for Firebird.
I add fdb to github.
In this string SQL-code what needed for that, but i don't know how setup Column in SQLAlchemy to use that.
Screenshot from IBExpert showing what create after use "GENERATED BY DEFAULT AS IDENTITY". That not same as a simple generator.

This postgresql work around might work:
https://docs.sqlalchemy.org/en/13/dialects/postgresql.html
I believe you would have to modify the example for firebird by changing the compile dialect like this:
from sqlalchemy import MetaData
from sqlalchemy.schema import CreateColumn
from sqlalchemy.ext.compiler import compiles
#compiles(CreateColumn, 'firebird')
def use_identity(element, compiler, **kw):
text = compiler.visit_create_column(element, **kw)
text = text.replace("SERIAL", "INT GENERATED BY DEFAULT AS IDENTITY")
return text
m = MetaData()
t = Table(
't', m,
Column('id', Integer, primary_key=True),
Column('data', String)
)

SQLAlchemy StaleDataError on deleting items inserted via ORM sqlalchemy.orm.exc.StaleDataError

I'm having an issue where I get a an error such as this one:
"MyPyramidApplication Error"<class 'sqlalchemy.orm.exc.StaleDataError'>: DELETE statement on table 'page_view' expected to delete 6 row(s); Only 0 were matched.
So, I have a good idea what is causing the issue but I have been unable to solve it.
I have a page_view model, that has a foreign key on page_id and a user_id.
Here's what the model looks like:
page_view_table = sa.Table(
'page_view',
metadata,
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('page_id', sa.Integer, sa.ForeignKey('guide.id')),
sa.Column('user_id', sa.Integer, sa.ForeignKey('user.id')),
sa.Column('last_view', sa.DateTime, nullable=False),
sa.UniqueConstraint('user_id', 'page_id'),
mysql_engine='InnoDB',
mysql_charset='utf8mb4'
)
Here's what the relations look like
orm.mapper(Page, page_table,
properties = {
'users_viewed': sa.orm.relation(
User,
secondary=page_view_table,
backref='page'),
}
)
I am adding some items to my database using an insert statement, something similar to this:
ins = model.page_view_table.insert()
sql = str(ins)
sql += ' ON DUPLICATE KEY UPDATE last_view = :last_view'
session = model.Session()
session.execute(sql, page_views)
mark_changed(session)
As far as I can tell from the logs, the transactions gets committed properly and I see the items in the DB.
However, when I try to delete the page item using the ORM, I get the StaleDataError exception. Looking at the logs, I see the ORM issuing a delete statement but then rolling back due to the error.
I have tried experimenting with session.expire_all() as well as session.expunge_all() right after the insert statement but they weren't very helpful and I still the error.
Here's what I see in the SQLAlchemy logs.
2011-11-05 18:06:08,031 INFO [sqlalchemy.engine.base.Engine][worker 3] DELETE FROM page_view WHERE page_view.page_id = %s AND page_view.user_id = %s
2011-11-05 18:06:08,031 INFO [sqlalchemy.engine.base.Engine][worker 3] (13818L, 259L)
2011-11-05 18:06:08,032 INFO [sqlalchemy.engine.base.Engine][worker 3] DELETE FROM page_view WHERE page_view.page_id = %s AND page_view.user_id = %s
2011-11-05 18:06:08,033 INFO [sqlalchemy.engine.base.Engine][worker 3] (13818L, 259L)
2011-11-05 18:06:08,033 INFO [sqlalchemy.engine.base.Engine][worker 3] ROLLBACK
I thought the double delete statement was a suspect, maybe pointing to a misconfigured ORM relation but I don't think that's the case.

I guess I can give a hint on this problem. The short version is: "You will probably have to modify data in the Database manually to solve the issue".
The longer version: I had a similar issue with SQLite. I had the following table mapped:
ingredients = Table('ingredients', metadata,
Column('recipe_title', Unicode, ForeignKey('recipes.title'), primary_key=True),
Column('product_title', Unicode, ForeignKey('products.title'), primary_key=True),
Column('amount', Integer, nullable=False),
Column('unit_title', Unicode, ForeignKey('units.title')))
see that composite primary key? I somehow managed to insert two rows with the same recipe_title/product_title pair. I was surprized to find out that there were not a single constraint on the side of SQLite for this table (no primary key, no fereign key - it was just a plain vanilla table), but well - thats the way sqlalchemy goes, not my business.
Then when I tried to delete a persited object involving those two rows, sqlalchemy saw that it's constraints were violated and it threw the 'StaleDataError'. Finally I just had to remove one duplicatinng row manually from SQLite table.

Although columns can be marked as primary_key, make sure this is enforced on the database level as well (for example when the database was created by some different tool). In MySQL this means ensuring they are PRIMARY KEY and not only KEY.
In my case there were 2 columns marked as primary_key (composite) but there were multiple rows containing the same (supposedly) unique id's.

How do I delete a foreign key constraint in SQLAlchemy?

I'm using SQLAlchemy Migrate to keep track of database changes and I'm running into an issue with removing a foreign key. I have two tables, t_new is a new table, and t_exists is an existing table. I need to add t_new, then add a foreign key to t_exists. Then I need to be able to reverse the operation (which is where I'm having trouble).
t_new = sa.Table("new", meta.metadata,
sa.Column("new_id", sa.types.Integer, primary_key=True)
)
t_exists = sa.Table("exists", meta.metadata,
sa.Column("exists_id", sa.types.Integer, primary_key=True),
sa.Column(
"new_id",
sa.types.Integer,
sa.ForeignKey("new.new_id", onupdate="CASCADE", ondelete="CASCADE"),
nullable=False
)
)
This works fine:
t_new.create()
t_exists.c.new_id.create()
But this does not:
t_exists.c.new_id.drop()
t_new.drop()
Trying to drop the foreign key column gives an error: 1025, "Error on rename of '.\my_db_name\#sql-1b0_2e6' to '.\my_db_name\exists' (errno: 150)"
If I do this with raw SQL, i can remove the foreign key manually then remove the column, but I haven't been able to figure out how to remove the foreign key with SQLAlchemy? How can I remove the foreign key, and then the column?

You can do it with sqlalchemy.migrate.
In order to make it work, I have had to create the foreign key constraint explicitly rather than implicitely with Column('fk', ForeignKey('fk_table.field')):
Alas, instead of doing this:
p2 = Table('tablename',
metadata,
Column('id', Integer, primary_key=True),
Column('fk', ForeignKey('fk_table.field')),
mysql_engine='InnoDB',
)
do that:
p2 = Table('tablename',
metadata,
Column('id', Integer, primary_key=True),
Column('fk', Integer, index=True),
mysql_engine='InnoDB',
)
ForeignKeyConstraint(columns=[p2.c.fk], refcolumns=[p3.c.id]).create()
Then the deletion process looks like this:
def downgrade(migrate_engine):
# First drop the constraint
ForeignKeyConstraint(columns=[p2.c.fk], refcolumns=[p3.c.id]).drop()
# Then drop the table
p2.drop()

I was able to accomplish this by creating a separate metadata instance and using Session.execute() to run raw SQL. Ideally, there would be a solution that uses sqlalchemy exclusively, so I wouldn't have to use MySQL-specific solutions. But as of now, I am not aware of such a solution.

I believe you can achieve this with SQLAlchemy-Migrate. Note that a ForeignKey is on an isolated column. A ForeignKeyConstraint is at the table level and relates the columns together. If you look at the ForeignKey object on the column you will see that it references a ForeignKeyConstraint.
I would unable to test this idea because of the two databases I use MS SQL isn't supported by SqlAlchemy-Migrate and sqlite doesn't support "alter table" for constraints. I did get SQLAlchemy to try to remove a FK via a drop on the references constraint on a sqlite table so it was looking good. YMMV.

Well, you can achieve this in sqlalchemy: just drop() the all the constraints before you drop() the column (theoretically, you might have multiple constraints):
def drop_column(column):
for fk in column.table.foreign_keys:
if fk.column == column:
print 'deleting fk ', fk
fk.drop()
column.drop()
drop_column(t_exists.c.new_id)