I am inserting data to a remote postgres db and recently added a new column to one of the tables and now I'm getting this error inserting new data into the table with new column
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column "build_name" of relation "DryRun" does not exist
LINE 1: ...tance_id, profile_name, host_box, default_config, build_name...
I tried what I found online such as setting nullable=True or giving it a default value default='Unknown', but still hitting the same error.
Here's my DryRun model.
class DryRun(Base):
__tablename__ = 'DryRun'
id = Column(Integer, primary_key=True, autoincrement=True)
instance_id = Column(Integer, nullable=True)
profile_name = Column(String, nullable=False)
host_box = Column(String, nullable=False)
default_config = Column(JSON, nullable=False)
build_name = Column(String, nullable=True)
This was originally posted as a comment.
The issue is that now the object differs from what's in the database. You can run an ALTER TABLE command to add the column within the command-line tool psql, or using the pgadmin GUI. And if you're anticipating more such changes, consider reading about migrations which are supported by some other modules; for instance, alembic: https://alembic.sqlalchemy.org/en/latest/
For just one column addition, migrations most likely don't make sense though. Here's an example of an ALTER TABLE command for your use-case:
# run this command to get to a text-based interface to your Postgres database
$ psql
# alter table command for your use-case:
ALTER TABLE "DryRun" ADD COLUMN build_name VARCHAR;
Related
I have run flask db upgrade for creating Todos table. Now I added a new table and established a relationship with the existing table and also added a new field in the existing table.
I would expect flask db migrate to record the differences (adding new table Todoslist and adding a new field in Todo) however it says name error - table not defined.
class TodoList(db.Model):
__tablename__ = 'todolists'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), nullable=False)
todos.db.relationship('Todo', backref='list', lazy = True)
class Todo(db.Model):
__tablename__ = 'todos'
id = db.Column(db.Integer, primary_key=True)
description = db.Column(db.String(), nullable=False)
completed = db.Column(db.Boolean, nullable=False, default=False)
list_id = db.Column(db.Integer, db.ForeignKey(todolists.id), nullable=True)
- Error details:
File "C:\Users\rg\anaconda3\lib\site-packages\flask\_compat.py", line 39, in reraise
raise value
File "C:\Users\rg\class_demos\Todoapp\app.py", line 26, in <module>
class Todo(db.Model):
File "C:\Users\rg\class_demos\Todoapp\app.py", line 32, in Todo
list_id = db.Column(db.Integer, db.ForeignKey(todolists.id), nullable=True)
NameError: name 'todolists' is not defined
Solutions tried
Flask app is set to the current python module
Tried swapping the create table (one before the other)
Searched extensively and this error seems to be common but not able to find a matching case like the one mentioned above.
Any help is much appreciated.
The error stems from how you are creating a relationship between Todo and TodoList models.
list_id needs to be properly initialized as a foreign key to TodoList.id, which means that it references an id value from the TodoList table. You have incorrectly used todolists.id, which is a table that does not exist. Rather, you should define the list_id as:
list_id = db.Column(db.Integer, db.ForeignKey("todoList.id"))
To create the relationship in TodoList table, you will need to initialize the field todos with db.relationship. This is not an actual database field, but a high-level view of the relationship between TodoList and Todo, and for that reason it isn't in the database diagram. So, you can properly create the field as:
todos = db.relationship('Todo', backref='list', lazy='True')
My bad. Missed the quotes for todolists.id
list_id = db.Column(db.Integer, db.ForeignKey("todolists.id"), nullable=True)
This fixed the issue :)
I'm using Python 2.7.5 and SqlAlchemy 0.9.9 (against an Oracle 11 database) and am trying to figure out how I can create a relationship between two tables where the join value is a string in one table and an integer in another. Here's mockup of my two tables:
class BatchInput(db.Model):
__tablename__ = 'batchinput'
batchinput_id = Column(Integer, primary_key=True)
item_id = Column(Integer)
class SubBatch(db.Model):
__tablename__ = 'subbatch'
subbatch_id = Column(Integer, primary_key=True)
subbatch_no = Column(String)
batch_input = relationship('BatchInput',
primaryjoin='SubBatch.subbatch_no == cast(BatchInput.item_id, VARCHAR)')
When I run a query to get subbatches I get this message:
Internal Server Error, Could not locate any relevant foreign key
columns for primary join condition
'subbatch.subbatchno = CAST(forgebatchinput.item_id AS VARCHAR)'
on relationship SubBatch.forge_batch_input. Ensure that
referencing columns are associated with a ForeignKey or
ForeignKeyConstraint, or are annotated in the join condition
with the foreign() annotation
I've tried a couple variations of this adding foreignkeys and remote, etc., but always get this message. I'm not sure what it's trying to tell me as I do have a primaryjoin specified.
BACKGROUND INFOS:
I have an app which is hosted by heroku on a postgresql DB.
I have already some data in this DB and now I have to add a new row in one of my tables.
Usually I deleted the old DB and recreated it. But in future if the project is live I will have to update tables without losing the data.
I could create a DUMP and delete the old database and recreate it as always. Then I could use a script and upload all existing data into the new DB. But this feels wrong.
WHAT I NEED:
In my current situation there is blog data table=blog on the database and I need to insert a new column into my table=zimmer so blog isnt even affected.
class Zimmer(Base):
__tablename__ = 'zimmer'
id = Column(Integer, primary_key=True)
infofeld = Column(Text, nullable=False)
land = Column(Text, nullable=False)
bundesland = Column(Text, nullable=False)
stadt = Column(Text, nullable=False)
plz = Column(Text, nullable=False)
strasse = Column(Text, nullable=False)
hausnr = Column(Text, nullable=True)
eigener_link = Column(Text, nullable=True)
zimmer_lat = Column(Float, nullable=False)
zimmer_lng = Column(Float, nullable=False)
reingestellt_am = Column(Date, nullable=False)
This is the new value: eigener_link = Column(Text, nullable=True)
I am currently experimenting on localhost but so far I am only getting ProgrammingError because everytime I try to load a site where zimmer is shown it says there is no column eigener_link (that is logical).
WHAT I TRIED:
I tryed to try except the ProgrammingError in the line where it occured, which gave me an InternalError. Here I tryed to update the zimmer table and add the new column eigener_link:
try:
for page in paginator:
pages_list.append(page.number)
except ProgrammingError:
update(Zimmer).values(eigener_link='Ihr Link')
db_session.commit()
It gave me an InternalError. I checked the DB via pgAdmin and the new value has not been added.
try:
for page in paginator:
pages_list.append(page.number)
except ProgrammingError:
db_session.execute('ALTER TABLE zimmer ADD eigener_link TEXT')
db_session.commit()
This gave me also InternalError the transaction has been canceled.
Okay I used alembic to solve this problem and it was really easy and took me like 10 min.
You install it for example via pip:
pip install alembic
And then follow the tutorial.
Basicly you go into your folder with your app and init alembic once, so it creates all necessary alembic files.
In alembic.ini you change the path to your database (you make the changes localy, no need to push something to heroku).
Then you use alembic revision to create a script which applies the changes to your DB. You have to open this created script with an editor to you can add changes. (more in the tutorial).
And finaly you run alembic upgrade head and that is it!
In my case this were the necessary changes in the script:
def upgrade():
op.add_column('zimmer', sa.Column('eigener_link', sa.Text))
def downgrade():
op.drop_column('zimmer', 'eigener_link')
I initially defined one of my SQLAlchemy models as:
class StreetSegment(db.Model):
id = db.Column(db.Integer, autoincrement=True) # surrogate ID
seg_id = db.Column(db.Integer, primary_key=True) # assigned in another system; don't increment
not realizing that seg_id would become a SERIAL field in my Postgres database. What I really wanted was an INTEGER field with a PK constraint (no autoincrement). I turned off autoincrement like so:
class StreetSegment(db.Model):
id = db.Column(db.Integer, autoincrement=True)
seg_id = db.Column(db.Integer, primary_key=True, autoincrement=False) # <--- see here
but the change isn't reflected when I run migrate in Alembic. I've also tried writing a custom migration with the following operations:
def upgrade():
op.alter_column('street_segment', 'seg_id', autoincrement=False)
def downgrade():
op.alter_column('street_segment', 'seg_id', autoincrement=True)
but that gives the warning autoincrement and existing_autoincrement only make sense for MySQL. So my question is: is there any way of using Alembic to convert a SERIAL to an INTEGER in Postgres?
Just set the type explicitly to the one you want. This should work:
op.alter_column('street_segment', 'seg_id', _type=Integer)
I'm using SQLAlchemy + alembic to manage my database. I had a string field which was 10 characters long and later on found out that it has to be 20. So I updated the model definition.
class Foo(db.Model):
__tablename__ = 'foos'
id = db.Column(db.Integer, primary_key=True)
foo_id = db.Column(db.Integer, db.ForeignKey('users.id'))
name = db.Column(db.String(80))
When I run alembic revision --autogenerate, this was not detected. Now I did read the documentation and suspected that this might not be supported. How do I managed such changes in DB gracefully?
You need to enable optional column type checking.
See this for notes on what is checked by default
context.configure(
# ...
compare_type = True
)