How to fix incorrect SQLAlchemy metadata after migration - python

I have defined and created a SQLite database using SQLAlchemy, and added some data. This database has been working fine. However, I made a change to a column name in one the models ("User") and migrated this change using alembic. The current model looks like this, where the old name "analogy" was changed to "metaphor":
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
username = Column(String(32), nullable=False, index=True, unique=True)
registered_on = Column(DateTime, index=True, default=datetime.now())
metaphor = Column(String(16))
The migration seemed to go fine, and I also doublechecked using the SQLite command line to confirm that the schema actually changed.
sqlite > .schema
CREATE TABLE "user" (
id INTEGER NOT NULL,
username VARCHAR(32) NOT NULL,
registered_on DATETIME,
metaphor VARCHAR(16),
PRIMARY KEY (id),
UNIQUE (wc_id)
);
CREATE UNIQUE INDEX ix_user_username ON user (username);
CREATE INDEX ix_user_registered_on ON user (registered_on);
But now when I try to query the table in Python, I get the following error, referencing the old column name and saying that there is no such column
>>> from background.database import db_session
>>> from data.models import User
>>> users = db_session.query(User).all()
Traceback (most recent call last):
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such column: user.analogy
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2843, in all
return list(self)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2995, in __iter__
return self._execute_and_instances(context)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 3018, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/export/scratch/pt/power/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: user.goal_period [SQL: 'SELECT user.id AS user_id, user.username AS user_username, user.registered_on AS user_registered_on, user.analogy AS user_analogy \nFROM user'] (Background on this error at: http://sqlalche.me/e/e3q8)
After tracing this error, it looks like the name change to the model is not reflected in the metadata imported from SQLAlchemy. So, SQLAlchemy is still using the metadata from the prior version of the table, while the actual database schema and the defined model have a different column name.
I've tried to delete all caches (.pyc, py_cache) to see if that was the source of the problem.
However, after doing this, there were still no changes to the metadata.
when I open a new interprer session and query the metadata, it shows the old schema, not the new one.
>>> from SQLAlchemy import MetaData
>>> metadata= MetaData()
>>> metadata.tables['user']
Table('user', MetaData(bind=None), Column('id', Integer(), table=<user>, primary_key=True, nullable=False), Column('username', String(length=32), table=<user>, nullable=False), Column('registered_on', DateTime(), table=<user>, default=ColumnDefault(datetime.datetime(2019, 6, 5, 12, 33, 57, 11763))), Column('goal_period', String(length=16), table=<user>, default=ColumnDefault('daily'))), schema=None)
Why is the metadata is not updated, and how do fix this problem?

Related

SqlAlchemy SQL Server Ambiguous Column Name Error

I have an issue that I could not manage. I am developing an FastApi SqlAlchemy App connected to SQL Server. I have created my models and views on locally. But when I deployed an server The following error occurred. I have tried Ubuntu, Linux, Windows VMs but the situation is same.
from sqlalchemy import create_engine
from sqlalchemy import Integer, String, Text, Column, DATE
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import select, MetaData, Table
from sqlalchemy.orm import scoped_session, sessionmaker, Query
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.dialects.mssql import pymssql
import urllib
import datetime
from sqlalchemy.sql.sqltypes import Date
params = urllib.parse.quote_plus("DRIVER={ODBC Driver 17 for SQL Server};"
"SERVER=MYSERVER"
"UID=MYUSERNAME"
"PWD=MYPASSWD"
"DATABASE=MYDB"
"Trusted_Connection=no;"
"Integrated Security=false;")
engine = create_engine("mssql+pyodbc:///?odbc_connect={}".format(params))
engine.connect()
Base = declarative_base()
metadata = Base.metadata
session = scoped_session(sessionmaker(bind=engine))
class Users(Base):
__table__ = Table('astencube_users', Base.metadata, autoload=True, autoload_with=engine)
print(session.query(*Users.__table__.columns).all())
This is a part of my app I am trying to print a table but when I try to make any query I got error below. On my local machine there is no error when I deploy any machine it appears. Also the error gives me a parameter which represents my table but when I check the other tables still problem occurs. On my local machine there is no problem. Why is it happening?
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context
self.dialect.do_execute(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
cursor.execute(statement, parameters)
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Ambiguous column name 'constraint_schema'. (209) (SQLExecDirectW)")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "file.py", line 35, in <module>
class Users(Base):
File "file.py", line 36, in Users
__table__ = Table('astencube_users', Base.metadata, autoload=True, autoload_with=engine)
File "<string>", line 2, in __new__
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 298, in warned
return fn(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 607, in __new__
metadata._remove_table(name, schema)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 602, in __new__
table._init(name, metadata, *args, **kw)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 677, in _init
self._autoload(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 712, in _autoload
conn_insp.reflect_table(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 795, in reflect_table
self._reflect_fk(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 948, in _reflect_fk
fkeys = self.get_foreign_keys(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 564, in get_foreign_keys
return self.dialect.get_foreign_keys(
File "<string>", line 2, in get_foreign_keys
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 55, in cache
ret = fn(self, con, *args, **kw)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/dialects/mssql/base.py", line 2498, in wrap
return _switch_db(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/dialects/mssql/base.py", line 2522, in _switch_db
return fn(*arg, **kw)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/dialects/mssql/base.py", line 3400, in get_foreign_keys
for r in connection.execute(s).fetchall():
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1286, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1478, in _execute_clauseelement
ret = self._execute_context(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
self._handle_dbapi_exception(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2023, in _handle_dbapi_exception
util.raise_(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context
self.dialect.do_execute(
File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Ambiguous column name 'constraint_schema'. (209) (SQLExecDirectW)")
[SQL: WITH fk_info AS (
SELECT
ischema_ref_con.constraint_schema,
ischema_ref_con.constraint_name,
ischema_key_col.ordinal_position,
ischema_key_col.table_schema,
ischema_key_col.table_name,
ischema_ref_con.unique_constraint_schema,
ischema_ref_con.unique_constraint_name,
ischema_ref_con.match_option,
ischema_ref_con.update_rule,
ischema_ref_con.delete_rule,
ischema_key_col.column_name AS constrained_column
FROM
INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS ischema_ref_con
INNER JOIN
INFORMATION_SCHEMA.KEY_COLUMN_USAGE ischema_key_col ON
ischema_key_col.table_schema = ischema_ref_con.constraint_schema
AND ischema_key_col.constraint_name =
ischema_ref_con.constraint_name
WHERE ischema_key_col.table_name = CAST(? AS NVARCHAR(max))
AND ischema_key_col.table_schema = CAST(? AS NVARCHAR(max))
),
constraint_info AS (
SELECT
ischema_key_col.constraint_schema,
ischema_key_col.constraint_name,
ischema_key_col.ordinal_position,
ischema_key_col.table_schema,
ischema_key_col.table_name,
ischema_key_col.column_name
FROM
INFORMATION_SCHEMA.KEY_COLUMN_USAGE ischema_key_col
),
index_info AS (
SELECT
sys.schemas.name AS index_schema,
sys.indexes.name AS index_name,
sys.index_columns.key_ordinal AS ordinal_position,
sys.schemas.name AS table_schema,
sys.objects.name AS table_name,
sys.columns.name AS column_name
FROM
sys.indexes
INNER JOIN
sys.objects ON
sys.objects.object_id = sys.indexes.object_id
INNER JOIN
sys.schemas ON
sys.schemas.schema_id = sys.objects.schema_id
INNER JOIN
sys.index_columns ON
sys.index_columns.object_id = sys.objects.object_id
AND sys.index_columns.index_id = sys.indexes.index_id
INNER JOIN
sys.columns ON
sys.columns.object_id = sys.indexes.object_id
AND sys.columns.column_id = sys.index_columns.column_id
)
SELECT
fk_info.constraint_schema,
fk_info.constraint_name,
fk_info.ordinal_position,
fk_info.constrained_column,
constraint_info.table_schema AS referred_table_schema,
constraint_info.table_name AS referred_table_name,
constraint_info.column_name AS referred_column,
fk_info.match_option,
fk_info.update_rule,
fk_info.delete_rule
FROM
fk_info INNER JOIN constraint_info ON
constraint_info.constraint_schema =
fk_info.unique_constraint_schema
AND constraint_info.constraint_name =
fk_info.unique_constraint_name
AND constraint_info.ordinal_position = fk_info.ordinal_position
UNION
SELECT
fk_info.constraint_schema,
fk_info.constraint_name,
fk_info.ordinal_position,
fk_info.constrained_column,
index_info.table_schema AS referred_table_schema,
index_info.table_name AS referred_table_name,
index_info.column_name AS referred_column,
fk_info.match_option,
fk_info.update_rule,
fk_info.delete_rule
FROM
fk_info INNER JOIN index_info ON
index_info.index_schema = fk_info.unique_constraint_schema
AND index_info.index_name = fk_info.unique_constraint_name
AND index_info.ordinal_position = fk_info.ordinal_position
ORDER BY constraint_schema, constraint_name, ordinal_position
]
[parameters: ('astencube_users', 'dbo')]
(Background on this error at: https://sqlalche.me/e/14/f405)
Wading through your code above we find this exact error message:
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Ambiguous column name 'constraint_schema'. (209) (SQLExecDirectW)")
So when we look to see where you've referenced constraint_schema in your code, there are a few places.
For most of them you have correctly used 2-part naming to avoid any ambiguity.
However your final order by clause does not:
ORDER BY constraint_schema, constraint_name, ordinal_position
P.S. the best way to debug this stuff yourself in the future is to take the SQL code and run it directly against the SQL Server i.e. eliminate Python from the equation as it just muddies the waters.
the problem was because of the SqlAlchemy version.The version in my local machine was 1.4.25 when I install newest version to any where I got the sql error shown above.So this problem was happened because of the version.Thanks.

Flask SQL no such table

I've tried this before and it worked, now I'm doing it again but it doesn't. I use a virtual env.
The db file is in the same directory as the init and models. I've also executed the db.create_all() in the python shell after importing db. The error occurs when I execute Students.query.all(). I'm really confused now. I compared my codes with my existing projects and it is the same. I don't understand why it doesn't work.
Traceback (most recent call last):
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: students
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\orm\query.py", line 3211, in all
return list(self)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\orm\query.py", line 3367, in __iter__
return self._execute_and_instances(context)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\orm\query.py", line 3392, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 982, in execute
return meth(self, multiparams, params)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\sql\elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1101, in _execute_clauseelement
distilled_params,
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1250, in _execute_context
e, statement, parameters, cursor, context
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1476, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "C:\Users\Martin\Desktop\regtry\venv\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: students
[SQL: SELECT students.id AS students_id, students.last_name AS students_last_name, students.college AS students_college, students.email AS students_email
FROM students]
(Background on this error at: http://sqlalche.me/e/e3q8)
_ _ init _ _.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
main = Flask(__name__)
main.config['SECRET_KEY'] = '3bce79429ad6b79670e4e800fb4a57b9'
main.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///site.db'
db = SQLAlchemy(main)
from reg import routes
models.py:
from reg import db
class Students(db.Model):
id = db.Column(db.Integer, primary_key=True)
last_name = db.Column(db.String(60), nullable=False)
college = db.Column(db.String(60), nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
def __repr__(self):
return f"Student('{self.last_name}','{self.college}','{self.email}')"
It probably has to do with where you are calling db.create_all(). You have to call it after you have defined all of your models. Here is a working example in one module:
import flask
from flask_sqlalchemy import SQLAlchemy
app = flask.Flask(__name__)
app.config['SECRET_KEY'] = '3bce79429ad6b79670e4e800fb4a57b9'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///site.db'
db = SQLAlchemy(app)
class Students(db.Model):
id = db.Column(db.Integer, primary_key=True)
last_name = db.Column(db.String(60), nullable=False)
college = db.Column(db.String(60), nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
def __repr__(self):
return f"Student('{self.last_name}','{self.college}','{self.email}')"
db.create_all()
#app.route("/students", methods=["GET"])
def all_students():
students = Students.query.all()
resp = []
for student in students:
resp.append(dict(
id=student.id,
last_name=student.last_name,
college=student.college,
email=student.email
))
return flask.jsonify(resp), 200
#app.route("/students/<_id>", methods=["GET"])
def student_by_id(_id):
student = Students.query.get(_id)
if student:
resp = dict(
id=student.id,
last_name=student.last_name,
college=student.college,
email=student.email
)
return flask.jsonify(resp), 200
else:
return "", 404
#app.route("/students", methods=["POST"])
def add_student():
req = flask.request.get_json()
student = Students(**req)
db.session.add(student)
db.session.commit()
resp = flask.make_response()
resp.headers["Location"] = flask.url_for("student_by_id", _id=student.id)
return resp, 201
In your example I did not see a call to db.create_all() but I suspect that it is in the wrong spot. Using your example, I believe it will work if you move the call from where you have it to the end of your __init__.py:
__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
main = Flask(__name__)
main.config['SECRET_KEY'] = '3bce79429ad6b79670e4e800fb4a57b9'
main.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///site.db'
db = SQLAlchemy(main)
from reg import routes
db.create_all()
In the shell, import the database and the models first before executing db.create_all(). Apparently, error occurs when create you create database without importing the models.
My mistake was:
>>from APPFOLDER import db
>>db.create_all()
>>from APPFOLDER.models import MODELNAME
>>MODELNAME.query.all()
***ERROR MSG***
What you should do:
>>from APPFOLDER import db
>>from APPFOLDER.models import MODELNAME
>>db.create_all()
>>MODELNAME.query.all()
[]

can't render element of type <class 'sqlalchemy.dialects.postgresql.base.UUID'>

I saw this snippet of code from Flask-common at line 57:
id = db.Column(UUID, default=lambda: str(uuid.uuid4()), primary_key=True)
So I thought I give it a try and use it in my app's models.py (because I prefer to have uuid type for my id's)
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from sqlalchemy.dialects.postgresql import UUID
import uuid
from app import db
class CostCenter(db.Model):
__tablename__ = "costcenter"
id = db.Column('id', UUID(as_uuid=True), default=lambda: str(uuid.uuid4()), primary_key=True)
name = db.Column('name', db.Text)
def __init__(self, name):
self.name = name
def __repr__(self):
return '<id {}>'.format(self.id)
But when I try to run python manage.py db upgrade, it results to an error:
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 190, in __init__
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 213, in process
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 2164, in visit_create_table
File "build/bdist.linux-x86_64/egg/sqlalchemy/util/compat.py", line 199, in raise_from_cause
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 2153, in visit_create_table
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 213, in process
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 2184, in visit_create_column
File "build/bdist.linux-x86_64/egg/sqlalchemy/dialects/sqlite/base.py", line 847, in get_column_specification
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/compiler.py", line 261, in process
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 79, in _compiler_dispatch
sqlalchemy.exc.CompileError: (in table 'costcenter', column 'id'): Compiler <sqlalchemy.dialects.sqlite.base.SQLiteTypeCompiler object at 0x7fc16
c1f1a50> can't render element of type <class 'sqlalchemy.dialects.postgresql.base.UUID'>
Why can't it render the type UUID? It is recognized in SQLAlchemy but how come it can't in Flask-SQLAlchemy?
Your column definition is using a function that only works with postgresql, and your database type is sqlite.
You need this:
id = db.Column(
'id',
db.Text(length=36),
default=lambda: str(uuid.uuid4()),
primary_key=True)
You can use UUID just in case of using postgresql database.

peewee foreign key error

I am trying to create a simple model for my existing database with peewee.
The database that I'm working on is described in this paper.
I've generated a model with pwiz which works fine, however it didn't create foreign keys for me which enable joining. Therefore I edited the Model so it looks like this:
from peewee import *
database = MySQLDatabase('enron', **{'password': '...', 'user': 'root'})
class UnknownField(object):
pass
class BaseModel(Model):
class Meta:
database = database
class Employee(BaseModel):
email = CharField(db_column='Email_id', unique=True)
eid = PrimaryKeyField()
class Meta:
db_table = 'employeelist'
class Message(BaseModel):
mid = PrimaryKeyField()
sender = ForeignKeyField(Employee,
related_name='messages',
to_field='email') # was CharField() before my edit
class Meta:
db_table = 'message'
However when I try to run:
for message in Message.select():
print message.mid
I get an error:
Traceback (most recent call last):
File "DBModelEnron.py", line 62, in <module>
for message in Message.select():
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2514, in __iter__
return iter(self.execute())
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2507, in execute
self._qr = ResultWrapper(model_class, self._execute(), query_meta)
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2203, in _execute
return self.database.execute_sql(sql, params, self.require_commit)
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2833, in execute_sql
self.commit()
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2688, in __exit__
reraise(new_type, new_type(*exc_value.args), traceback)
File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2825, in execute_sql
cursor.execute(sql, params or ())
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
peewee.OperationalError: (1054, "Unknown column 't1.sender_id' in 'field list'")
I found a similar problem here, however I have a proper PrimaryKey defined.
Just add a db_column='' to your foreign keys:
sender = ForeignKeyField(Employee,
db_column='sender', # Added this.
related_name='messages',
to_field='email') # was CharField() before my edit

alembic migration error flask

I am making a test blog using the guide found here. It's pretty comprehensive. However, I'm having trouble with the alembic migrations. I can erase all the versions, and spin up a new database with all of the columns just fine. But, when I add a new column I have problems. Here is the code in my models.py:
models.py
....
class Person(db.Model):
__tablename__ = 'person'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(100), unique=True)
pwdhash = db.Column(db.String(100))
name = db.Column(db.String(100), unique=True)
def __init__(self, email, name, password):
self.email = email
self.name = name.title()
self.set_password(password)
def __repr__(self):
return '<User %r>' % (self.name)
def set_password(self, password):
self.pwdhash = generate_password_hash(password)
def check_password(self, password):
return check_password_hash(self.pwdhash, password)
#classmethod
def all(cls):
return Person.query.all()
class Category(db.Model):
__tablename__ = 'category'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), unique=True)
description = db.Column(db.Text)
def __unicode__(self):
return self.name
class Article(db.Model):
__tablename__ = 'articles'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(100))
body = db.Column(db.Text)
created = db.Column(db.DateTime, default=datetime.datetime.now)
category_name = db.Column(db.String(10), db.ForeignKey(Category.name))
category = db.relationship(Category)
person_name = db.Column(db.String(100), db.ForeignKey(Person.name, onupdate="CASCADE", ondelete="CASCADE"))
person = db.relationship(Person)
#property
def slug(self):
return urlify(self.title)
#classmethod
def all(cls):
return Article.query.order_by(desc(Article.created)).all()
#classmethod
def find_by_category(cls, category):
return Article.query.filter(Article.category_name == category).all()
It's all pretty standard. However, if I was to add a random column to my People table, like this:
class Person(db.Model):
....
random_column = db.Column(db.Integer())
then run a python manage.py db migrate (which works fine) then run a python manage.py db upgrade then I get the following error:
Traceback (most recent call last):
File "manage.py", line 7, in <module>
manager.run()
File "/Library/Python/2.7/site-packages/flask_script/__init__.py", line 397, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File "/Library/Python/2.7/site-packages/flask_script/__init__.py", line 376, in handle
return handle(app, *positional_args, **kwargs)
File "/Library/Python/2.7/site-packages/flask_script/commands.py", line 145, in handle
return self.run(*args, **kwargs)
File "/Library/Python/2.7/site-packages/flask_migrate/__init__.py", line 82, in upgrade
command.upgrade(config, revision, sql = sql, tag = tag)
File "/Library/Python/2.7/site-packages/alembic/command.py", line 124, in upgrade
script.run_env()
File "/Library/Python/2.7/site-packages/alembic/script.py", line 199, in run_env
util.load_python_file(self.dir, 'env.py')
File "/Library/Python/2.7/site-packages/alembic/util.py", line 198, in load_python_file
module = load_module(module_id, path)
File "/Library/Python/2.7/site-packages/alembic/compat.py", line 55, in load_module
mod = imp.load_source(module_id, path, fp)
File "migrations/env.py", line 72, in <module>
run_migrations_online()
File "migrations/env.py", line 65, in run_migrations_online
context.run_migrations()
File "<string>", line 7, in run_migrations
File "/Library/Python/2.7/site-packages/alembic/environment.py", line 652, in run_migrations
self.get_context().run_migrations(**kw)
File "/Library/Python/2.7/site-packages/alembic/migration.py", line 225, in run_migrations
change(**kw)
File "migrations/versions/4171a9f6ed2a_.py", line 19, in upgrade
op.drop_index('category_name_key', 'category')
File "<string>", line 7, in drop_index
File "<string>", line 1, in <lambda>
File "/Library/Python/2.7/site-packages/alembic/util.py", line 293, in go
return fn(*arg, **kw)
File "/Library/Python/2.7/site-packages/alembic/operations.py", line 716, in drop_index
self._index(name, table_name, ['x'], schema=schema)
File "/Library/Python/2.7/site-packages/alembic/ddl/impl.py", line 164, in drop_index
self._exec(schema.DropIndex(index))
File "/Library/Python/2.7/site-packages/alembic/ddl/impl.py", line 76, in _exec
conn.execute(construct, *multiparams, **params)
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 662, in execute
params)
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 720, in _execute_ddl
compiled
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
context)
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
exc_info
File "/Library/Python/2.7/site-packages/sqlalchemy/util/compat.py", line 196, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 867, in _execute_context
context)
File "/Library/Python/2.7/site-packages/sqlalchemy/engine/default.py", line 324, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (InternalError) cannot drop index category_name_key because constraint category_name_key on table category requires it
HINT: You can drop constraint category_name_key on table category instead.
'\nDROP INDEX category_name_key' {}
It doesn't even mention the name of the column I created in the stacktrace, so that leads me to believe that something is wrong with the other tables. It mentions the dropping of an index, but I'm not doing anything like that in the migration, just adding a column to the People table. Is it something in alembic that I don't understand?
Like I said it works perfectly find when I spin up a brand new database and load in the configuration. It's only when I make a change and try to migrate that alembic throws me these errors. Does anyone have any idea why this is happening?
EDIT
Just in case people need to see my config.py file:
config.py
import os
basedir = os.path.abspath(os.path.dirname(__file__))
#-----Config the app
SECRET_KEY = 'my_key'
CSRF_ENABLED = True
#-----Config Database
SQLALCHEMY_DATABASE_URI = 'postgresql://username:changeme#localhost/test'
SQLALCHEMY_COMMIT_ON_TEARDOWN = True
SQLALCHEMY_MIGRATE_REPO = os.path.join(basedir, 'db_repository')
#-----Config Upload folder
UPLOAD_FOLDER = os.path.realpath('./snb/static') + '/uploads'
I followed the link to the doc on Flask-Migrate to see how it's using alembic, and it says:
The migrate command adds a new migration script. You should review it
and edit it to be accurate, as Alembic cannot detect all changes that
you make to your models. In particular it does not detect indexes, so
those need to be added manually to the script.
Flask-Migrate uses a feature of alembic called "autogenerate", where it tries to compare the state of your database with your models and automatically create a diff.
We use Alembic directly, and I found autogenerate really convenient, but I have to edit it by hand a lot, especially when you're dealing with indexes and constraints.
So my solution would be to do what it says and edit the migration file by hand and remove the lines you don't want or are spurious.

Categories

Resources