I have a fairly large code base in django, with several applications, and at some point I had a problem: every time I change or add models and try to do a migration, an error appears: django.db.utils.ProgrammingError: relation "appname_modelname" already exists. The problem has been going on for some time - the migration file is always created under the same number - 0114 (I could not find this file, by the way), and all new fixes are recorded in it along with the previous ones, which is why the problem grows, like snowball.
I did not want to delve into the solution and just manually removed all the innovations from the database - everything that caused the "already exists" conflict to appear. So, in order for the migration to succeed, I had to manually delete all models or table fields that I created after this problem appeared. But now I'm starting to work in prod with this code and fill it with data, and it is no longer possible to delete all the data from the corresponding tables. I have no idea why this problem appeared and how to solve it and would really appreciate your advice.
I tried to make a fake, but then, obviously, the fixes I need simply do not get into the database.
I have a hypothesis as to what caused this: I have three docker containers, one api, and two for celery. In the api container, I wrote the command in the bash file at startup
python3 /usr/srv/h_api/src/manage.py makemigrations && python3 /usr/srv/h_api/src/manage.py migrate
instead of
python3 /usr/srv/h_api/src/manage.py makemigrations
so I don't have to do it manually every time. I don't understand why, but judging by the coincidence of time, it seems that the problem started because of this. It's been weeks since I changed that line back, but the problem remains.
How do I commit previous changes to the database and have django create the next migration file numbered 0115?
It may be a bit risky but it has worked for me in the past. I suggest creating a copy of your project in another folder and trying this safely away from the original project. Also if you are using a postgresql database then just switch to a dummy database although it shouldn't make a difference but just to be on the safer side.
Inside of your migrations folder, inside the app folder, try deleting the all the files inside of pycache EXCEPT init.cpython-39.pyc and 0001_initial.cpython-39.pyc and inside of the migrations folder delete all files EXCEPT init.py and 0001_initial.py.
IMPORTANT: Delete from the pycache INSIDE of migrations NOT the one outside of it.
The approach below has often helped me out, if its a production database its good to backup and also your migrations folder before getting started.
Start by deleting your current applications migrations folder
temporarily add a field to your models.
Makemigrations then migrate
Delete the temporarily added field from your model
Makemigrations and migrate
Every time I do a: python manage.py runserver
And I load the site, python gets data and puts this in my database.
Even when I already filled some info in the database. Enough to get a view of what I am working on.
Now it is not loading the information I want and instead putting in new information to add to the database so it can work with some data.
What is the reason my data in the database is not being processed?
And how do I stop new data being loaded into the database.
May be it is happening due to migration file first sometimes when you migrate models into database query language with same number
python manage.py makemigrations 0001
This "0001" has to be changed everytime
To solve your problem once delete the migrations file and then again migrate all models and then try
Tell if this work
Can django delete/erease/clean data of single database/class in an app managing more disjunctive classes? Or is it safer to build separate app for each separate/disjunctive database?
I have found how to delete all data from all project databases, i.e.:
python manage.py flush
I was only surpruised that user IDs and class IDs have not been reset to 1, but previously used ID numbers stayed somewhere in project memory.
I have also found that it is possible to delete/clean all databases of signle app, e.g. here or here.
python manage.py migrate <app> zero
Here even IDs were reset. Does somebody know how to reset user IDs to start again from number 1?
But nowhere, I have found how to delete/erase/clean/reset signle database/class of the app with more than one database. I have only found for ruby that there it could be possible like:
Model.delete_all
or
Model.destroy_all
But nobody confirmed there, that commads really work. Will/Can it work also in django?
Kind regards, Rene
I'm using Django 1.6 with PostgreSQL and I want to use a two different Postgres users - one for creating the initial tables (syncdb) and performing migrations, and one for general access to the database in my application. Is there a way of doing this?
From ./manage.py help syncdb:
--database=DATABASE Nominates a database to synchronize. Defaults to the
"default" database.
You can add another database definition in your DATABASES configuration, and run ./manage.py syncdb --database=name_of_database_definition. You might want to create a small wrapper script for running that command, so that you don't have to type out the --database=... parameter by hand every time.
south also supports that option, so you can also use it to specify the database for your migrations.
I've setup a local Postgres DB with SQLAlchemy and cannot commit my first entry. I keep on getting this error...
ProgrammingError: (ProgrammingError) relation "user" does not exist
LINE 1: INSERT INTO "user" (name, email, facebook_id, facebook_token...
It seems like the fields aren't matching to those in the database. I'm trying to migrate using flask-migrate but, when I run $ python app.py db migrate I get this error...
raise util.CommandError("No such revision '%s'" % id_)
alembic.util.CommandError: No such revision '39408d6b248d'
It may be best to delete everything and start from scratch as it seems I have botched my database setup and / or migration but I'm not sure how to.
UPDATE: The database has started working now (I dropped and created it again). However, I'm still getting the same error trying to run migrations and it turns out the "no such revision '39408d6b248d' is referring to a migration from an unrelated project. I re-installed flask-migrate but same error.
flask-migrate will create a table named "alembic_version" in your database.
so you should drop this table and delete migrations folder in your project.
and then use $ python app.py db init again...
I think $ python app.py db migrate will work fine.
Alembic keeps the migration history in your database, this is why it still recognises there is another revision there. I keep my project on Heroku so I was able to just do heroku pg:pull ... to be able to get a new copy of my database. Prior to this you will have to drop your local db. In case you don't want to drop your local, I think dropping the table should work too. I use the PG Commander as a GUI tool to quickly browse my databases.
the first step to do is run this command
alembic current
you should get an error as mentioned above (the goal is to make sure that this command returns a valid response).
the reason why u're getting this is bc alembic is confused about your current state.. it's assuming that you should be in revision 39408d6b248d but then decides that that revision is invalid.
to investigate this, let's find out which revisions are considered valid by alembic, run this command:
alembic history --verbose
you'll get a list of all previous revisions (note: it's a good idea to attach a message beside each revision.. think about it as a good git commit message)
Rev: 594cc72f56fd (head)
Parent: 262f40e28682
Path: ***************
adjust context_id in log table so that it is a substring of the object_id
Revision ID: 594cc72f56fd
Revises: 262f40e28682
Create Date: 2015-07-22 14:31:52.424862
Rev: 262f40e28682
Parent: 1dc902bd1c2
Path: ***************
add context_id column to log table
Revision ID: 262f40e28682
Revises: 1dc902bd1c2
Create Date: 2015-07-22 11:05:37.654553
Rev: 1dc902bd1c2
Parent: <base>
Path: ***************
Initial database setup
Revision ID: 1dc902bd1c2
Revises:
Create Date: 2015-07-06 09:55:11.439330
the revision 39408d6b248d clearly doesn't exist in the above revisions. This revision is stored in the alembic_table in the database.. you can verify by going to your dbase and running:
$ select * from alembic_version;
version_num
--------------
57ac999dcaa7
so now you should review the state of your database and see where it fits vis-a-vis the revisions outputted above:
in my case, by poking around my dbase it becomes obvious which revision i'm in right now.. which is that dbase has been setup,, but the other revisions haven't been included yet.
so now i replace the value on the dbase with the one i found from the history command above:
vibereel=> update alembic_version set version_num = '1dc902bd1c2';
and now running alembic current returns
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
1dc902bd1c2
done.
It means that the entry in table alembic_version of your db is "39408d6b248d" and there's no migration file related to it in migrations folder (by default migrations/versions).
So better drop the table alembic_version from your db and do
$ python app.py db history to get the new head of migrations, say, 5301c31377f2
Now run $ python app.py db stamp 5301c31377f2 to let alembic know that it's your migration head (which gets stored in table alembic_version).
Assuming that you have checked that the database exists using psql or pgAdmin, this error usually means exactly what it says. That can be due to either:
not connecting to the correct database instance (check your db url: host/port and db name)
not correctly configuring SQLAlchemy (see: SQLAlchemy create_all() does not create tables)
I got the same error yesterday, for my case the revision number '39408d6b248d' is due to your previous migration upgrade actions, each time your ran upgrade script then an Alembic version will be generated and stored in data.sqlite.
you must have done any upgrade which generated above '39408d6b248d', but then you deleted whole migrations/ directory and removed all upgrade scripts. The database e.g. data.sqlite still stores '39408d6b248d' but there aren't according migration script exists.
for my solution I delete whole Alembic versions in database and did all upgrades from scratch.
I ran into a similar issue. After performing the python manage.py db migrate command the database tables were not created, there was only an alembic table in the database.
I found the solution in the flask-migrate documentation:
https://flask-migrate.readthedocs.org/en/latest/
The migration script needs to be reviewed and edited, as Alembic
currently does not detect every change you make to your models. In
particular, Alembic is currently unable to detect indexes. Once
finalized, the migration script also needs to be added to version
control.
Then you can apply the migration to the database:
python manage.py db upgrade
This command created the tables and applied the database migrations.
I had the same issue. According to the version in alembic_version table in db, migration action is looking for that version in /migrations/versions folder which has already been deleted. Therefore solution is to delete alembic_version table:
If you are using sqlite,
1. Open your xxx.sqlite db file.
sqlite3 xxx.sqlite
2. check tables
.tables
3. you will see alembic_version, delete it
DROP TABLE alembic_version