I want to revert my last migration (0157) by running its Migration.backwards() method. Since I am reverting the migration in production server I want to run it automatically during code deployment. Deployment script executes these steps:
Pull code changes
Run migrations: manage.py migrate <app>
Refresh Apache to use newest code: touch django.wsgi
If I could, I would create new migration file which would tell South to backward migrate to 0156:
migrations/0158_backward__migrate_to_0156.py
This commited migration would be deployed to production and executed during manage.py migrate <app> command. In this case I wouldn't have to execute backward migration by hand, like suggested in these answers.
Lets say, I have created two data migrations, first for user's Payment, second for User model. I have implemented backwards() methods for both migrations in case I'd have to revert these data migrations. I've deployed these two migrations to production. And suddenly find out that Payment migration contains an error. I want to revert my two last data migrations as fast as possible. What is the fastest safe way to do it?
Since I am reverting the migration in production server I want to run
it automatically during code deployment.
IMHO the safest path is
run manage.py migrate <app> (i.e. apply all existing migrations, i.e. up to 0156)
undo the changes in your model
run manage.py schemamigration <app> --auto
This will create a new migration 0157 that effectively reverts the previous migration 0156. Then simply apply the new migration by running manage.py migrate <app> again. As I understand, your code deployment will just do that.
Apparently the codeline has migrations up to #157 and now the developer decided that the last one was not a good idea after all. So the plan is to go back to #156.
Two scenarios:
(a) migration #157 was not released or deployed anywhere yet.
Simply revert the last change from models.py and delete migration #157.py from the source archive. Any deployment will take the system to level 156; "157 was never there".
(b) there have been deployments of the latest software with migration #157.
In this case the previous strategy will obviously not work. So you need to create a migration #158 to undo #157. Revert the change in models.py and run
django manage.py migrate <app> 0157
django manage.py schemamigration <app> --auto
This will auto-generate a new migration #158, which will contain the inverse schema migration compared to #157.
If schemamigration is giving trouble because of django Model validation (something that can happen if you have custom validators which check stuff outside the ORM box), I suggest the following workaround:
<django project>/<app>/management/commands/checkmigrations.py
from south.management.commands import schemamigration
class Command(schemamigration.Command):
requires_model_validation = False
help = "schemamigration without model validation"
This command becomes available in manage.py:
django manage.py checkmigrations <app> --auto
There's no silver bullet here. The simplest solution I can think of would be to - in your dev env of course - manually migrate back to 0156, manually update your migration's history table (sorry I can't remember the table's name now) to fool south in thinking you're still #0158, then run schemamigration again. Not garanteed to work but might be worth trying.
Related
I had an Django2.2.3 app, it was working fine. But I had to chane the name of a field in a table, and add another field. Then I ran ./manage.py makemigrations && ./manage.py migrate. Besides the terminal prompt:
Running migrations:
No migrations to apply.
No error is throwed. But then when I go to the MySQLWorkbench to check the database, it is exactly as I didn't make any change. I tried deleting the migrations and making again, the process ends with no errors but the database don't change. I create another empty database, change the name on settings.py and make migrations and migrate again, and it worked, but when I put the old database name on the settings, it just did not work.
Can someone explain this behavior for me? There is any kind of cache for these information migrations or something? I realy want to know why this is not winrkig as I espect.
Make sure the app with the migrations is in the INSTALLED_APPS. Django won't look at the app for changes otherwise.
Adding new few fields to an existing model (table) is one reason for this problem. A way to go about this is simply as follows:
a) un-apply the migrations for that app:
python3 manage.py migrate --fake <app-name> zero
b) migrate the required migrations (you've already deleted previous migrations and you've done 'makemigrations' for the newly added column. So, you just migrate:
python3 manage.py migrate <app-name>
If the steps above didn't solve the problems, then drop the table first;
i) python3 manage.py dbshell
ii) DROP TABLE appname_tablename
close the shell and repeat a and b again.
I have a table in my database (postgresql) that is associated with a model called feedback in my Django app (which is also called feedback).
I deleted a couple of columns of the feedback model in the models.py file and then created a migration using:
python manage.py makemigrations feedback
And then tried to "merge" it with my database using:
python manage.py migrate feedback
But i got the error:
django.db.utils.ProgrammingError: relation "feedback" already exists
Of course it exists, but i want to apply the changes i've made. In the migrations folder i have the following files:
__init__.py
0001_initial.py
0002_remove_feedback_created_on.py
0003_remove_feedback_is_read.py
The last one contains my most recent changes. What should i do?
Usually this relation x already exists problem appears when you've applied the change at some point in the database, but the Django migrations system hasn't got a clue about the fact that you already did that. How could it happen? For example if you add a ForeignKey field to your model, make a migration, migrate it, but then change your mind and delete the migration without migrating backwards before that. Nothing breaks in the project, but the relationship and the field in the database remain (you might run into trouble, if the field is non-nullable). Now if you change your mind again and decide that shiiit, I needed that field anyhows and add it again, make a new migration and try to migrate it, then you get the relation x already exists error.
Ok, what then? Usually the simplest way is to fake the failing migration, but BEWARE that if you have any other unapplied changes in the migrations, then those won't be applied to the database either. Before you start faking migrations, make sure that you understand what and why you are faking. Ideally you'd have only one AddField in the migration file that is soon to be faked, because otherwise all changes in the migration file will be faked (faking migrations means, that the changes are not applied to database, but Django will consider them done).
If you have two unapplied migrations, then you definitely shouldn't run migrate --fake without specifying the migration number, because then the latter unapplied migration will be unapplied as well. It SEEMS in your case, that running manage.py migrate 0002 --fake and manage.py migrate afterwards might fix everything, but without knowing everything about what you've done, it's impossible to say for certain.
If I picked up your project, here's what I would do:
1) Check if 0002 contains more than one change.
If yes, then copy everything except the failing field to 0003.
If no, then proceed to manage.py migrate 0002 --fake.
2) Run manage.py migrate again to migrate 0003.
It could also help in the case of migrations.
python3 manage.py migrate --run-syncdb
I have a Python/Django proyect. Due to some rolls back, and other mixed stuff we ended up in a kind of odd scenario.
The current scenario is like this:
DB has the correct tables
DB can't be rolled back or dropped
Code is up to date
Migrations folder is behind the DB by one or two migrations. (These migrations were applied from somewhere else and that "somewhere else" doesn't exist anymore)
I add and alter some models
I run makemigrations
New migrations are created, but it's a mix of new tables and some tables that already exist in the DB.
If I run migrate it will complain that some of the tables that I'm trying to create already exist.
What I need:
To be able to run the migrations and kind of "ignore" the existing tables and apply the new ones. Or any alternative way to achieve this.
Is that possible?
When you apply a migration, Django inserts a row in a table called django_migrations. That's the only way Django knows which migrations have been applied already and which have not. So the rows in that table have to match the files in your migrations directory. If you've lost the migration files after they were applied, or done anything else to get things out of sync, you'll have problems.. because the migration numbers in your database refer to different migration files than the ones in your project.
So before you do anything else, you need to bring things back into sync by deleting the django_migrations table rows for any migration files that you've lost somehow and can't get back. The table should contain rows for only those migrations that you do have and that were actually applied to the database correctly.
Now you need to deal with any changes in your database that Django Migrations doesn't know about.. and for that there are a few options:
If things worked out such that the database changes that were already applied to the database are in different migration files than the ones that weren't, then you can fix it by running your migrations one at a time using the --fake option on any changes that are in reality already in the database. The fake option just writes the row to the django_migrations table marking the migration as done. Only do this if the database does in fact already have all the changes contained in that migration file.
And those migration files that contain only changes which have not been applied to the database, run without the --fake option and Django will apply them. eg:
# database already has it
manage.py migrate myapp 0003 --fake
# need it
manage.py migrate myapp 0004
# database already has it
manage.py migrate myapp 0005 --fake
If you have migration files where some but not all of the changes have been applied, then you have a bigger problem. In that case, there are several ways to go about it (choose ONLY ONE):
Edit the migration files to put changes that have already been applied (whether Django did it or you did it manually does not matter) into lower number migrations, and put everything you need done into higher numbered files. Now you can --fake the lower number ones, and run the higher numbered ones as normal. Let's say you have 10 changes you made to your models, and 5 of those changes are actually in the database already, but Django doesn't know about them.. so when you run makemigrations, a new migration is created with all 10 changes. This will normally fail because the database server can't for example add a column that already exists. Move these already-applied changes out of your new migration file, into the previous (already applied) migration file. Django will then assume that these were applied with the previous migration and will not try to apply them again. You can then migrate as normal and the new changes will be applied.
If you don't want to touch your older migration file, a cleaner way to do this is to first run makemigrations --empty appname to create an empty migration file. Then run makemigrations which will create another migration with all the changes that Django thinks need to be done. Move the already done migrations from that file into the empty migration you created.. then --fake that one. This will put Django's understanding of what the database looks like will be in sync with reality and you can then migrate as normal, applying the changes in the last migration file.
Get rid of any new migrations you just created using makemigrations. Now, comment out or put back anything in your models that has not been applied to the database, leaving your code matching what's actually in the database. Now you can do makemigrations and migrate appname --fake and you will get things back in sync. Then uncomment your new code and run 'makemigrations' then migrate as normal and the changes will be applied. If the changes are small (for example, adding a few fields), sometimes this is easiest. If the changes are large, it isn't....
You can go ahead and (carefully) make the database changes yourself, bringing the database up to date. Now just run migrate --fake and if you didn't mess up then everything will be ok. Again, this is easy for smaller changes, not as easy for complicated ones.
You can run manage.py sqlmigrate > mychanges.sql. This generates mychanges.sql containing all the SQL Django WOULD have executed against the database. Now edit that file to remove any changes that have already been applied, leaving what needs to be done. Execute that SQL using pgadmin or psql (you're using postgresql I hope). Now the changes have all been made.. so you can run manage.py migrate --fake, this will bring Django into sync with reality and you should be all set. If your SQL skills are sufficient, this is probably the most straightforward solution.
I should add two warnings:
First, if you apply a later migration, eg 0003_foobar.py, and then things don't work out and you decide to try going back and apply 0002_bazbuz.py, then Django will TAKE STUFF OUT OF YOUR DATABASE. For example a column you might have added in 0003 will be dropped along with its data. Since you say you can't lose data, be very careful about going back.
Second, do not rush into running --fake migrations. Make sure that the entire migration you are about to fake is actually in the database already. Else it gets very confusing. If you do regret faking migrations and don't want to roll back, you can erase django's knowledge of the faked migration by deleting that row from the django_migrations table. It is ok to do this.. if you understand what you are doing. If you know that the migration really was not applied, then it's ok.
This blog post really nails it. https://simpleisbetterthancomplex.com/tutorial/2016/07/26/how-to-reset-migrations.html
Let me summarize the steps in his scenario 2 (you have a production database and want to change schema/models in one or more apps). In my case, I had two apps, queue and routingslip, that had model modifications that I needed to apply to a production system. Key was I already had the database, so this is where --fake-initial comes into play.
Here are the steps I followed. As always, backup everything before starting. I do work in a VM so I just took a snapshot before going forward.
1) Remove the migration history for each app.
python manage.py migrate --fake queue zero
python manage.py migrate --fake routingslip zero
2) Blow away any migration files in the entire project within which the app(s) reside.
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
3) Make migrations
python manage.py makemigrations
4) Apply the migrations, faking initial because the database already exists and we just want the changes:
python manage.py migrate --fake-initial
Worked great for me.
If you don't have any migration files or you lost the previous file and want to migrate new changes, then you need to follow the following steps carefully:
# To create a new migration file before changing the models.
cmd: python manage.py makemigrations app_name
# Fake migrate
cmd: python manage.py migrate app_name 0005 --fake #[0005 is the migration file number created just now. It'll seem like 0005_add_address or something like this.]
# To create a new migration file after changing the models.
cmd: python manage.py makemigrations app_name
# database already has it
cmd: python manage.py migrate app_name 0006 #[0006 is the migration file number created just now.]
I am trying to setup a Django app locally in a new machine but migrations seem to be totally broken. They need to be performed in a particular order, which worked in the first machine I set the environment in a couple months ago, but now there are inconsistencies (although I am pretty sure no new migrations were generated).
So the only solution I can think of is exporting the database from the old machine, where it is working, to the new one. Would that work?
This would not solve the broken migrations issue, but at least I can work on the code till there's a proper soltuion.
Answering this question:
So the only solution I can think of is exporting the database from the old machine, where it is working, to the new one. Would that work?
Yes, this can work if you are sure that your database is in sync with your models. It is actually the way to go, if you want to be best prepared of updating your production environment.
get a dump from the current production machine
create a new database and load the dump
check whether there are differences between the models and the migration history (this is more reliable with the new Django migrations, South was an external tool and had not all of the possibilities) (e.g. ./manage.py showmigrations (1.10), ./manage.py migrate --list (1.7-1.9 and South)
If you are confident that no migrations have to be run but the listing shows differences then do: ./manage.py migrate --fake
Note, in newer versions you can do ./manage.py migrate and it will report that everything is in order if the models and the migrations are in sync. This can be a sanity check before you deploy onto production.
I mistakenly deleted all the .py file under path projectName/appName/migrations, it include like:
0001_initial.py
0011_auto_20150918_0723.py
0002_auto_20150819_1301.py
...
Now, even I updated the Model file, and run the command python manage.py makemigrations, it always prompt with No changes detected.
How can I recover everything?
First of all, even if this happened to you in production, do not panic.
When you deleted all migrations django forgot that this app is supposed to be managed by migrations. Django defaults back to the legacy python manage.py syncdb migrationless behaviour and will not attempt to detect changes or generate new migrations when you run python manage.py makemigrations
In order to make it aware of the migrations, you have to run the command specifically for your app:
python manage.py makemigrations appName
However, for your running application, django will not be able to detect that the new migrations were already applied in the database, and will try to run them again when you run python manage.py migrate.
When this happens the migrations fail saying that the relation appName.XYZ already exists!.
To make django understand that your migration is already reflected in the database you have to fake them:
python manage.py migrate appName --fake
This will update the migration history table and mark your migrations as applied, but will not attempt to create the tables in the database.
Update (thanks to #GwynBleiD):
One concern here is that deleted migrations which were already applied, will be listed in the migrations history table in the database. This will not be a problem for the initial python manage.py makemigrations myApp, however for any other migrations it will raise an error about inconsistent migration history.
To avoid that, you must remove by hand any row from django_migrations table in database that refers to nonexistent migrations.