Develop Reference

Export SQL Script from SQLAlchemy

I am using SQLAlchemy with ORM and DeclarativeMeta to connect to my Database.
Is there a way to generate or export a .sql file that contains all the Create Tables Commands?
Thank you!
I tried to get that information from my Meta Object or even from my SQLAlchemy Engine but they don't hold information like that.
Even the Meta.metadate._create_all() does not return a string or something else

Found an answers in the documentation of sqlalchemy.
from sqlalchemy.schema import CreateTable
print(CreateTable(my_mysql_table).compile(mysql_engine))
CREATE TABLE my_table (
id INTEGER(11) NOT NULL AUTO_INCREMENT,
...
)ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
SQLAlchemy Documentation!

Python SQLAlchemy INSERT after DELETE violates constraint

I have this pattern for deletion of all rows in a Postgresql table and subsequent insertion with SQLAlchemy:
db = create_engine("postgresql://...", echo=False).connect()
metadata = MetaData(db)
my_table = Table('my_table', metadata, autoload_with=db)
...
db.execute(my_table.delete())
db.execute(my_table.insert(), values)
where values is a list.
I can't uderstand why I get a psycopg2.errors.UniqueViolation when trying to insert.
The data which is inserted is not duplicated, so I guess the problem is that the delete is not committed?
I don't use a Session: what do I need to do to get this simple pattern working correctly?

I found the solution by completely disabling automatic SQLAlchemy transactions (which are not needed in my case of bulk deletions/insertions) with the supported DBAPI isolation_level="AUTOCOMMIT":
db = create_engine("postgresql://...", echo=False).connect().execution_options(isolation_level="AUTOCOMMIT")
See https://docs.sqlalchemy.org/en/14/core/connections.html#setting-transaction-isolation-levels-including-dbapi-autocommit

Escape a table name in sql alchemy

I need to escape table names for any sqlalchemy engine (I want to expand a library I made for other databases than postgres, see Details section at the end of my post) automatically.
It is possible with columns like this:
from sqlalchemy.sql import column
from sqlalchemy import create_engine
engine = create_engine("sqlite:///test.db")
escaped_column = column('%"my_column123?').compile(dialect=engine.dialect)
str(escaped_column)
'"%""my_column123?"'
I (naively) tried the following but it does not work (gives back an empty string):
from sqlalchemy.sql import table
from sqlalchemy import create_engine
engine = create_engine("sqlite:///test.db")
escaped_table_name = table("%table?&;").compile(dialect=engine.dialect)
str(escaped_table_name)
''
Thanks in advance!
Details
I made a library to update PostGres table using pandas DataFrames (see https://github.com/ThibTrip/pangres) and realized that a part of the code is not SQL injection safe (if you are curious here is the part I am talking about: https://github.com/ThibTrip/pangres/blob/7cfa2d2190cf65a1ede8ef30868883f0da3fa5fc/pangres/helpers.py#L270-L290).

I found a way to add columns in a "sqlalchemy way" (adding columns to any table was the reason why I wanted to escaped table names). Unfortunately it is not perfect as it creates a table "alembic_version" for some reasons:
# alembic is a library from the creator of sqlalchemy to migrate databases
from alembic.runtime.migration import MigrationContext # pip install alembic
from alembic.operations import Operations
from sqlalchemy import Column, TEXT, create_engine # pip install sqlalchemy
# create engine
engine = create_engine('sqlite:///test.db')
# add column "some_new_column" of type TEXT in the table 'test'
with engine.connect() as con:
ctx = MigrationContext.configure(con, )
op = Operations(ctx)
op.add_column('test', column=Column('some_new_column', TEXT))
EDIT: it seems the table "alembic_version" has been added somehow by previous tests as I could not reproduce this behavior after dropping the table. So this solution seems good :)!!!

Can I somehow query all the existing tables in peewee / postgres?

I am writing a basic gui for a program which uses Peewee. In the gui, I would like to show all the tables which exist in my database.
Is there any way to get the names of all existing tables, lets say in a list?

Peewee has the ability to introspect Postgres, MySQL and SQLite for the following types of schema information:
Table names
Columns (name, data type, null?, primary key?, table)
Primary keys (column(s))
Foreign keys (column, dest table, dest column, table)
Indexes (name, sql*, columns, unique?, table)
You can get this metadata using the following methods on the Database class:
Database.get_tables()
Database.get_columns()
Database.get_indexes()
Database.get_primary_keys()
Database.get_foreign_keys()
So, instead of using a cursor and writing some SQL yourself, just do:
db = PostgresqlDatabase('my_db')
tables = db.get_tables()
For even more craziness, check out the reflection module, which can actually generate Peewee model classes from an existing database schema.

To get a list of the tables in your schema, make sure that you have established your connection and cursor and try the following:
cursor.execute("SELECT table_name FROM information_schema.tables WHERE table_schema='public'")
myables = cursor.fetchall()
mytables = [x[0] for x in mytables]
I hope this helps.

How do I drop a table in SQLAlchemy when I don't have a table object?

I want to drop a table (if it exists) before writing some data in a Pandas dataframe:
def store_sqlite(in_data, dbpath = 'my.db', table = 'mytab'):
database = sqlalchemy.create_engine('sqlite:///' + dbpath)
## DROP TABLE HERE
in_data.to_sql(name = table, con = database, if_exists = 'append')
database.close()
The SQLAlchemy documentation all points to a Table.drop() object - how would I create that object, or equivalently is there an alternative way to drop this table?
Note : I can't just use if_exists = 'replace' as the input data is actually a dict of DataFrames which I loop over - I've suppressed that code for clarity (I hope).

From the panda docs;
"You can also run a plain query without creating a dataframe with execute(). This is useful for queries that don’t return values, such as INSERT. This is functionally equivalent to calling execute on the SQLAlchemy engine or db connection object."
http://pandas.pydata.org/pandas-docs/version/0.18.0/io.html#id3
So I do this;
from pandas.io import sql
sql.execute('DROP TABLE IF EXISTS %s'%table, engine)
sql.execute('VACUUM', engine)
Where "engine" is the SQLAlchemy database object (the OP's "database" above). Vacuum is optional, just reduces the size of the sqlite file (I use the table drop part infrequently in my code).

You should be able to create a cursor from your SQLAlchemy engine
import sqlalchemy
engine = sqlalchemy.create_engine('sqlite:///' + dbpath)
connection = engine.raw_connection()
cursor = connection.cursor()
command = "DROP TABLE IF EXISTS {};".format(table)
cursor.execute(command)
connection.commit()
cursor.close()
# Now you can chunk upload your data as you wish
in_data.to_sql(name=table, con=engine, if_exists='append')
If you're loading a lot of data into your db, you may find it faster to use pandas' to_csv() and SQL's copy_from function. You can also use StringIO() to hold it in memory and having to write the file.