Develop Reference

How set start of auto increment in flask-sqlalchemy [duplicate]

The autoincrement argument in SQLAlchemy seems to be only True and False, but I want to set the pre-defined value aid = 1001, the via autoincrement aid = 1002 when the next insert is done.
In SQL, can be changed like:
ALTER TABLE article AUTO_INCREMENT = 1001;
I'm using MySQL and I have tried following, but it doesn't work:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
autoincrement=1001, primary_key=True)
So, how can I get that? Thanks in advance!

You can achieve this by using DDLEvents. This will allow you to run additional SQL statements just after the CREATE TABLE ran. Look at the examples in the link, but I am guessing your code will look similar to below:
from sqlalchemy import event
from sqlalchemy import DDL
event.listen(
Article.__table__,
"after_create",
DDL("ALTER TABLE %(table)s AUTO_INCREMENT = 1001;")
)

According to the docs:
autoincrement –
This flag may be set to False to indicate an integer primary key column that should not be considered to be the “autoincrement” column, that is the integer primary key column which generates values implicitly upon INSERT and whose value is usually returned via the DBAPI cursor.lastrowid attribute. It defaults to True to satisfy the common use case of a table with a single integer primary key column.
So, autoincrement is only a flag to let SQLAlchemy know whether it's the primary key you want to increment.
What you're trying to do is to create a custom autoincrement sequence.
So, your example, I think, should look something like:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.schema import Sequence
Base = declarative_base()
class Article(Base):
__tablename__ = 'article'
aid = Column(INTEGER(unsigned=True, zerofill=True),
Sequence('article_aid_seq', start=1001, increment=1),
primary_key=True)
Note, I don't know whether you're using PostgreSQL or not, so you should make note of the following if you are:
The Sequence object also implements special functionality to accommodate Postgresql’s SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a Table object defines a Sequence on its primary key column so that it works with Oracle and Firebird, the Sequence would get in the way of the “implicit” sequence that PG would normally use. For this use case, add the flag optional=True to the Sequence object - this indicates that the Sequence should only be used if the database provides no other option for generating primary key identifiers.

I couldn't get the other answers to work using mysql and flask-migrate so I did the following inside a migration file.
from app import db
db.engine.execute("ALTER TABLE myDB.myTable AUTO_INCREMENT = 2000;")
Be warned that if you regenerated your migration files this will get overwritten.

I know this is an old question but I recently had to figure this out and none of the available answer were quite what I needed. The solution I found relied on Sequence in SQLAlchemy. For whatever reason, I could not get it to work when I called the Sequence constructor within the Column constructor as has been referenced above. As a note, I am using PostgreSQL.
For your answer I would have put it as such:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Sequence, Column, Integer
import os
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Sequence, Integer, create_engine
Base = declarative_base()
def connection():
engine = create_engine(f"postgresql://postgres:{os.getenv('PGPASSWORD')}#localhost:{os.getenv('PGPORT')}/test")
return engine
engine = connection()
class Article(Base):
__tablename__ = 'article'
seq = Sequence('article_aid_seq', start=1001)
aid = Column('aid', Integer, seq, server_default=seq.next_value(), primary_key=True)
Base.metadata.create_all(engine)
This then can be called in PostgreSQL with:
insert into article (aid) values (DEFAULT);
select * from article;
aid
------
1001
(1 row)
Hope this helps someone as it took me a while

You can do it using the mysql_auto_increment table create option. There are mysql_engine and mysql_default_charset options too, which might be also handy:
article = Table(
'article', metadata,
Column('aid', INTEGER(unsigned=True, zerofill=True), primary_key=True),
mysql_engine='InnoDB',
mysql_default_charset='utf8',
mysql_auto_increment='1001',
)
The above will generate:
CREATE TABLE article (
aid INTEGER UNSIGNED ZEROFILL NOT NULL AUTO_INCREMENT,
PRIMARY KEY (aid)
)ENGINE=InnoDB AUTO_INCREMENT=1001 DEFAULT CHARSET=utf8

If your database supports Identity columns*, the starting value can be set like this:
import sqlalchemy as sa
tbl = sa.Table(
't10494033',
sa.MetaData(),
sa.Column('id', sa.Integer, sa.Identity(start=200, always=True), primary_key=True),
)
Resulting in this DDL output:
CREATE TABLE t10494033 (
id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 200),
PRIMARY KEY (id)
)
Identity(..) is ignored if the backend does not support it.
* PostgreSQL 10+, Oracle 12+ and MSSQL, according to the linked documentation above.

Using alembic to migrate pre-existing and populated tables

I have this tables I created models for using vanilla SQLAlchemy to operate the postgre DB.
class table:
def __init__(self):
self.engine = create_engine(DB_ADDRESS, poolclass=NullPool)
self.meta = MetaData()
self.table = Table(
"employeetable",
self.meta,
Column("id", Integer, primary_key=True),
Column("employeeId", String),
Column("name", String),
Column("email", String),
Column("place", String),
Column("sector", String),
Column("function", String),
Column("phone", String),
Column("status", String),
Column("type", String),
Column(
"lastChange",
DateTime(timezone=True),
server_default=func.now(),
),
Column("receptorId", String),
Column("syncFlag", Boolean),
)
self.meta.create_all(self.engine)
each class also have its methods for me to search and alter the the table and whatnot.
The table already is populated with data and takes care of the data sync between two systems.
The problem is now I need to add another column to check the sync of another field.
I know I have to migrate the table to do this, and also know Alembic is an option to this. I just can't figure out how I get Alembic to "import" this class/table so I can work with it inside Alembic to make this migration to alter the table.
Can someone give me some directions on how this works and how i can configure it to receive this?

SQLAlchemy: Table already exists

when do we generally get the following error in SQLAlchemy?
sqlalchemy.exc.OperationalError: (OperationalError) (1050, "Table 'foobar' already exists")
The foobar table does already exists, but why SQLAlchemy is trying to create table when already present. I'm assuming it shouldn't create table if already exists.
I'm using following syntax to create table:
t = Table('foobar', metadata,
Column('col1', String(50), primary_key=True),
Column('col2', String(100)),
mysql_engine='InnoDB',
mysql_charset='utf8')
(I'm calling the same program in parallel 10 times)

Just use schema object's (Table,Index and Sequence) create and drop methods with checkfirst=True keyword and table will automatically add an "IF NOT EXISTS or IF EXISTS CLAUSE" whichever is appropriate to SQL.
FOR EXAMPLE:
t = Table('foobar', metadata,
Column('col1', String(50), primary_key=True),
Column('col2', String(100)),
mysql_engine='InnoDB',
mysql_charset='utf8')
t.create(checkfirst=True)

If the foobar table already existed, you could instead have done:
users = Table('foobar', metadata, autoload=True)
and SQLAlchemy would have automatically figured out the table's structure from the database.
Check first using autoload, table is there or not, if it is not there then create the table.

Here is my guess with some troubleshooting ideas. My guess is that the client thinks the table does not exist, because it can't see it, but when trying to create it, cannot because it in fact does exist.
Troubleshooting ideas:
Check to see if maybe some other part of the code is writing to the same log file or whatever and is trying to create these tables.
Manually login with the same ID and password as the client, and see if you can see the
table.
Pass echo=True to create_engine to learn the exact queries the client performs, then repeat all the queries in your own SQL shell to see what the client is seeing. Hopefully that will lead you to a conclusion.

Another option is to use metadata:
database_engine = create_engine(
'mssql+pyodbc://username:password=#server:1433/databaseName?driver=ODBC+Driver+17+for+SQL+Server?trusted_connection=no')
with database_engine.connect() as database_connection:
metaData = MetaData(bind=database_engine, reflect=True)
metaData.reflect()
try:
Table("tbl_stuff",
metaData,
Column("id", Integer, primary_key=True, autoincrement=False),
Column("Name", String(255)),
extend_existing=True)
metaData.create_all(database_connection, checkfirst=True)
metaData.reflect()
except Exception as an_exc:
logging.exception("Exception creating tbl_stuff: " + str(an_exc))
For details please see https://docs.sqlalchemy.org/en/13/core/metadata.html

SQLAlchemy StaleDataError on deleting items inserted via ORM sqlalchemy.orm.exc.StaleDataError

I'm having an issue where I get a an error such as this one:
"MyPyramidApplication Error"<class 'sqlalchemy.orm.exc.StaleDataError'>: DELETE statement on table 'page_view' expected to delete 6 row(s); Only 0 were matched.
So, I have a good idea what is causing the issue but I have been unable to solve it.
I have a page_view model, that has a foreign key on page_id and a user_id.
Here's what the model looks like:
page_view_table = sa.Table(
'page_view',
metadata,
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('page_id', sa.Integer, sa.ForeignKey('guide.id')),
sa.Column('user_id', sa.Integer, sa.ForeignKey('user.id')),
sa.Column('last_view', sa.DateTime, nullable=False),
sa.UniqueConstraint('user_id', 'page_id'),
mysql_engine='InnoDB',
mysql_charset='utf8mb4'
)
Here's what the relations look like
orm.mapper(Page, page_table,
properties = {
'users_viewed': sa.orm.relation(
User,
secondary=page_view_table,
backref='page'),
}
)
I am adding some items to my database using an insert statement, something similar to this:
ins = model.page_view_table.insert()
sql = str(ins)
sql += ' ON DUPLICATE KEY UPDATE last_view = :last_view'
session = model.Session()
session.execute(sql, page_views)
mark_changed(session)
As far as I can tell from the logs, the transactions gets committed properly and I see the items in the DB.
However, when I try to delete the page item using the ORM, I get the StaleDataError exception. Looking at the logs, I see the ORM issuing a delete statement but then rolling back due to the error.
I have tried experimenting with session.expire_all() as well as session.expunge_all() right after the insert statement but they weren't very helpful and I still the error.
Here's what I see in the SQLAlchemy logs.
2011-11-05 18:06:08,031 INFO [sqlalchemy.engine.base.Engine][worker 3] DELETE FROM page_view WHERE page_view.page_id = %s AND page_view.user_id = %s
2011-11-05 18:06:08,031 INFO [sqlalchemy.engine.base.Engine][worker 3] (13818L, 259L)
2011-11-05 18:06:08,032 INFO [sqlalchemy.engine.base.Engine][worker 3] DELETE FROM page_view WHERE page_view.page_id = %s AND page_view.user_id = %s
2011-11-05 18:06:08,033 INFO [sqlalchemy.engine.base.Engine][worker 3] (13818L, 259L)
2011-11-05 18:06:08,033 INFO [sqlalchemy.engine.base.Engine][worker 3] ROLLBACK
I thought the double delete statement was a suspect, maybe pointing to a misconfigured ORM relation but I don't think that's the case.

I guess I can give a hint on this problem. The short version is: "You will probably have to modify data in the Database manually to solve the issue".
The longer version: I had a similar issue with SQLite. I had the following table mapped:
ingredients = Table('ingredients', metadata,
Column('recipe_title', Unicode, ForeignKey('recipes.title'), primary_key=True),
Column('product_title', Unicode, ForeignKey('products.title'), primary_key=True),
Column('amount', Integer, nullable=False),
Column('unit_title', Unicode, ForeignKey('units.title')))
see that composite primary key? I somehow managed to insert two rows with the same recipe_title/product_title pair. I was surprized to find out that there were not a single constraint on the side of SQLite for this table (no primary key, no fereign key - it was just a plain vanilla table), but well - thats the way sqlalchemy goes, not my business.
Then when I tried to delete a persited object involving those two rows, sqlalchemy saw that it's constraints were violated and it threw the 'StaleDataError'. Finally I just had to remove one duplicatinng row manually from SQLite table.

Although columns can be marked as primary_key, make sure this is enforced on the database level as well (for example when the database was created by some different tool). In MySQL this means ensuring they are PRIMARY KEY and not only KEY.
In my case there were 2 columns marked as primary_key (composite) but there were multiple rows containing the same (supposedly) unique id's.

Can I CREATE TEMPORARY TABLE in SQLAlchemy without appending to Table._prefixes?

I'd like to create a temporary table in SQLAlchemy. I can build a CREATE TABLE statement with a TEMPORARY clause by calling table._prefixes.append('TEMPORARY') against a Table object, but that's less elegant than table.select().prefix_with() used to add a prefix to data manipulation language expressions.
Is there an equivalent to .prefix_with() for DDL?

No, prefix_with() is defined for SELECT and INSERT only. But convenient way to add prefix to CREATE TABLE statement is passing it into table definition:
t = Table(
't', metadata,
Column('id', Integer, primary_key=True),
# ...
prefixes=['TEMPORARY'],
)