BACKGROUND INFOS:
I have an app which is hosted by heroku on a postgresql DB.
I have already some data in this DB and now I have to add a new row in one of my tables.
Usually I deleted the old DB and recreated it. But in future if the project is live I will have to update tables without losing the data.
I could create a DUMP and delete the old database and recreate it as always. Then I could use a script and upload all existing data into the new DB. But this feels wrong.
WHAT I NEED:
In my current situation there is blog data table=blog on the database and I need to insert a new column into my table=zimmer so blog isnt even affected.
class Zimmer(Base):
__tablename__ = 'zimmer'
id = Column(Integer, primary_key=True)
infofeld = Column(Text, nullable=False)
land = Column(Text, nullable=False)
bundesland = Column(Text, nullable=False)
stadt = Column(Text, nullable=False)
plz = Column(Text, nullable=False)
strasse = Column(Text, nullable=False)
hausnr = Column(Text, nullable=True)
eigener_link = Column(Text, nullable=True)
zimmer_lat = Column(Float, nullable=False)
zimmer_lng = Column(Float, nullable=False)
reingestellt_am = Column(Date, nullable=False)
This is the new value: eigener_link = Column(Text, nullable=True)
I am currently experimenting on localhost but so far I am only getting ProgrammingError because everytime I try to load a site where zimmer is shown it says there is no column eigener_link (that is logical).
WHAT I TRIED:
I tryed to try except the ProgrammingError in the line where it occured, which gave me an InternalError. Here I tryed to update the zimmer table and add the new column eigener_link:
try:
for page in paginator:
pages_list.append(page.number)
except ProgrammingError:
update(Zimmer).values(eigener_link='Ihr Link')
db_session.commit()
It gave me an InternalError. I checked the DB via pgAdmin and the new value has not been added.
try:
for page in paginator:
pages_list.append(page.number)
except ProgrammingError:
db_session.execute('ALTER TABLE zimmer ADD eigener_link TEXT')
db_session.commit()
This gave me also InternalError the transaction has been canceled.
Okay I used alembic to solve this problem and it was really easy and took me like 10 min.
You install it for example via pip:
pip install alembic
And then follow the tutorial.
Basicly you go into your folder with your app and init alembic once, so it creates all necessary alembic files.
In alembic.ini you change the path to your database (you make the changes localy, no need to push something to heroku).
Then you use alembic revision to create a script which applies the changes to your DB. You have to open this created script with an editor to you can add changes. (more in the tutorial).
And finaly you run alembic upgrade head and that is it!
In my case this were the necessary changes in the script:
def upgrade():
op.add_column('zimmer', sa.Column('eigener_link', sa.Text))
def downgrade():
op.drop_column('zimmer', 'eigener_link')
Related
I am using sql-alchemy with flask-migrate. I set up my database class with sth like this:
class Candidate(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(32), index=False, unique=False)
interviewer = db.Column(db.String(32), index=False, unique=False)
Unfortunately some of values was too long so I got the error:
sqlalchemy.exc.DataError: (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(32)
Wasnt sure in which column was the problem (actual class is much more complicated) I changed the class like this:
class Candidate(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(64), index=False, unique=False)
interviewer = db.Column(db.String(64), index=False, unique=False)
and then ran:
flask db migrate
flask db upgrade
to apply changes. Unfortunately I am still receiving the same error. What am I doing wrong?
Alembic (the migration engine behind Flask-Migrate) does not put data type changes in migrations by default, so your change from 32 to 64 was probably ignored. You can confirm by looking at the generated migration script.
To configure Alembic to watch for column type changes, you have to add the compare_type=True option when you create the Migrate class:
migrate = Migrate(app, db, compare_type=True)
After you do this, regenerate your migration, and it should have the code to make your columns larger.
I have run flask db upgrade for creating Todos table. Now I added a new table and established a relationship with the existing table and also added a new field in the existing table.
I would expect flask db migrate to record the differences (adding new table Todoslist and adding a new field in Todo) however it says name error - table not defined.
class TodoList(db.Model):
__tablename__ = 'todolists'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), nullable=False)
todos.db.relationship('Todo', backref='list', lazy = True)
class Todo(db.Model):
__tablename__ = 'todos'
id = db.Column(db.Integer, primary_key=True)
description = db.Column(db.String(), nullable=False)
completed = db.Column(db.Boolean, nullable=False, default=False)
list_id = db.Column(db.Integer, db.ForeignKey(todolists.id), nullable=True)
- Error details:
File "C:\Users\rg\anaconda3\lib\site-packages\flask\_compat.py", line 39, in reraise
raise value
File "C:\Users\rg\class_demos\Todoapp\app.py", line 26, in <module>
class Todo(db.Model):
File "C:\Users\rg\class_demos\Todoapp\app.py", line 32, in Todo
list_id = db.Column(db.Integer, db.ForeignKey(todolists.id), nullable=True)
NameError: name 'todolists' is not defined
Solutions tried
Flask app is set to the current python module
Tried swapping the create table (one before the other)
Searched extensively and this error seems to be common but not able to find a matching case like the one mentioned above.
Any help is much appreciated.
The error stems from how you are creating a relationship between Todo and TodoList models.
list_id needs to be properly initialized as a foreign key to TodoList.id, which means that it references an id value from the TodoList table. You have incorrectly used todolists.id, which is a table that does not exist. Rather, you should define the list_id as:
list_id = db.Column(db.Integer, db.ForeignKey("todoList.id"))
To create the relationship in TodoList table, you will need to initialize the field todos with db.relationship. This is not an actual database field, but a high-level view of the relationship between TodoList and Todo, and for that reason it isn't in the database diagram. So, you can properly create the field as:
todos = db.relationship('Todo', backref='list', lazy='True')
My bad. Missed the quotes for todolists.id
list_id = db.Column(db.Integer, db.ForeignKey("todolists.id"), nullable=True)
This fixed the issue :)
I am inserting data to a remote postgres db and recently added a new column to one of the tables and now I'm getting this error inserting new data into the table with new column
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column "build_name" of relation "DryRun" does not exist
LINE 1: ...tance_id, profile_name, host_box, default_config, build_name...
I tried what I found online such as setting nullable=True or giving it a default value default='Unknown', but still hitting the same error.
Here's my DryRun model.
class DryRun(Base):
__tablename__ = 'DryRun'
id = Column(Integer, primary_key=True, autoincrement=True)
instance_id = Column(Integer, nullable=True)
profile_name = Column(String, nullable=False)
host_box = Column(String, nullable=False)
default_config = Column(JSON, nullable=False)
build_name = Column(String, nullable=True)
This was originally posted as a comment.
The issue is that now the object differs from what's in the database. You can run an ALTER TABLE command to add the column within the command-line tool psql, or using the pgadmin GUI. And if you're anticipating more such changes, consider reading about migrations which are supported by some other modules; for instance, alembic: https://alembic.sqlalchemy.org/en/latest/
For just one column addition, migrations most likely don't make sense though. Here's an example of an ALTER TABLE command for your use-case:
# run this command to get to a text-based interface to your Postgres database
$ psql
# alter table command for your use-case:
ALTER TABLE "DryRun" ADD COLUMN build_name VARCHAR;
I'm using SQLAlchemy ORM framework from a Flask project. I wanted to add another column to one of my models. After I did, I used db.session.drop_all() to drop all current entries in the database, and then I tried recreating a new instance with the new field.
Whenever I tried I get this error
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no
such table: user
[SQL: SELECT user.id AS user_id, user.username AS user_username, user.email AS user_email, user.password AS user_password, user.image_file AS user_image_file
FROM user]
I think I might need to reconstruct the database in some way, but I'm not sure how. I looked into the documentation, but could not find anything useful.
__init__.py
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///site.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
models.py
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(20), unique=True, nullable=False)
email = db.Column(db.String(40), unique=True, nullable=False) #newly added field
password = db.Column(db.String(60), nullable=False)
image_file = db.Column(db.String(20), nullable=False, default='default.jpg')
messages = db.relationship('Message', backref='user', lazy=True)
When you used db.drop_all() you dropped all tables so now you can't insert data because there is no table. You need create the table again with db.create_all() as #SuperShoot mentioned.
You should use something for migrations like alembic or flask-sqlalchemy.
This way you could add new column to your ORM. Run flask db migrate -m 'new column', this will detect changes. Then run flask db upgrade head to apply those changes to your database.
I am using Flask-SQLAlchemy to define my models, and then using Flask-Migrate to auto-generate migration scripts for deployment onto a PostgreSQL database. I have defined a number of SQL Views on the database that I use in my application like below.
However, Flask-Migrate now generates a migration file for the view as it thinks it's a table. How do I correctly get Flask-Migrate / Alembic to ignore the view during autogenerate?
SQL View name: vw_SampleView with two columns: id and rowcount.
class ViewSampleView(db.Model):
__tablename__ = 'vw_report_high_level_count'
info = dict(is_view=True)
id = db.Column(db.String(), primary_key=True)
rowcount = db.Column(db.Integer(), nullable=False)
Which means I can now do queries like so:
ViewSampleView.query.all()
I tried following instructions on http://alembic.zzzcomputing.com/en/latest/cookbook.html and added the info = dict(is_view=True) portion to my model and the following bits to my env.py file, but don't know where to go from here.
def include_object(object, name, type_, reflected, compare_to):
"""
Exclude views from Alembic's consideration.
"""
return not object.info.get('is_view', False)
...
context.configure(url=url,include_object = include_object)
I think (though haven't tested) that you can mark your Table as a view with the __table_args__ attribute:
class ViewSampleView(db.Model):
__tablename__ = 'vw_report_high_level_count'
__table_args__ = {'info': dict(is_view=True)}
id = db.Column(db.String(), primary_key=True)
rowcount = db.Column(db.Integer(), nullable=False)