How could I rework Django migrations without affecting database? [duplicate] - python

I changed a field on a model class (which has no classes point to it, only one foreign key pointing out of it). Somehow, this stuffed up my migrations and it keeps saying "django.db.migrations.graph.NodeNotFoundError:" looking for migration files that do not exist.
I accidentally deleted several files in my 'migrations' folder.
My database contains a lot of data, and I do not want to break it.
Will I lose any data if I:
Remove the table that caused the problem in the first place (psql, \d, DROP TABLE tablename)
delete all my migration files
Re run the migration from the start?
./manage.py makemigrations
./manage.py migrate
Can anyone recommend another way of fixing this?
Here is the traceback:
http://dpaste.com/0Y1YDXS

Aren't you using git so that you can get your migration files back? If not, install and use it, starting now
I would suggest:
make a backup/dump of your database first, in case something goes wrong
Delete all migrations
Empty migration table in psql
call makemigrations
call migrate --fake-initial

Empty the django_migrations table:
delete from django_migrations;
Remove all the files in migrations folders in each and every app of your project.
Reset the migrations for the "built-in" apps:
python manage.py migrate --fake
Create initial migrations for each and every app:
python manage.py makemigrations your_app_name
Final step is to create fake initial migrations:
python manage.py migrate --fake-initial
See here https://micropyramid.com/blog/how-to-create-initial-django-migrations-for-existing-schema/

Related

Django migration not applied to the DB

I had an Django2.2.3 app, it was working fine. But I had to chane the name of a field in a table, and add another field. Then I ran ./manage.py makemigrations && ./manage.py migrate. Besides the terminal prompt:
Running migrations:
No migrations to apply.
No error is throwed. But then when I go to the MySQLWorkbench to check the database, it is exactly as I didn't make any change. I tried deleting the migrations and making again, the process ends with no errors but the database don't change. I create another empty database, change the name on settings.py and make migrations and migrate again, and it worked, but when I put the old database name on the settings, it just did not work.
Can someone explain this behavior for me? There is any kind of cache for these information migrations or something? I realy want to know why this is not winrkig as I espect.
Make sure the app with the migrations is in the INSTALLED_APPS. Django won't look at the app for changes otherwise.
Adding new few fields to an existing model (table) is one reason for this problem. A way to go about this is simply as follows:
a) un-apply the migrations for that app:
python3 manage.py migrate --fake <app-name> zero
b) migrate the required migrations (you've already deleted previous migrations and you've done 'makemigrations' for the newly added column. So, you just migrate:
python3 manage.py migrate <app-name>
If the steps above didn't solve the problems, then drop the table first;
i) python3 manage.py dbshell
ii) DROP TABLE appname_tablename
close the shell and repeat a and b again.

New Django app initial DB migrations of an existing database

I have an existing database filled with a bunch of data. I want to migrate to Django so I went ahead and created the necessary models with python manage.py inspectdb. I haven't changed anything besides removed the managed = False since I want Django to manage the tables (mistake perhaps for initial migration?).
So now that the models are ready, how can I generate the first migration file so that I can start changing the fields to generate additional migrations (renaming a field here and there). I understand python manage.py migrate will handle the creation of Django-specific models but does not actually create any migration files? Some sources indicate the first migration file should be run with --fake so it's not applied. Will the next migration files remember to run the first one as fake and only apply the next ones?
You want makemigrations to create the migrations. The migrate command applies migrations, it does not create them.
You can use the --fake-initial option so that Django does not try to create the tables that already exist. Note that --fake and --fake-initial are two different commands.
When you run migrate, the django_migrations table is updated to store the currently applied migrations. The migration files themselves are not changed. The --fake command updates the django_migrations table without running the migration. That means that if you use it incorrectly your database and django_migrations table can get out of sync, which can be difficult to fix.

How to force migrations to a DB if some tables already exist in Django?

I have a Python/Django proyect. Due to some rolls back, and other mixed stuff we ended up in a kind of odd scenario.
The current scenario is like this:
DB has the correct tables
DB can't be rolled back or dropped
Code is up to date
Migrations folder is behind the DB by one or two migrations. (These migrations were applied from somewhere else and that "somewhere else" doesn't exist anymore)
I add and alter some models
I run makemigrations
New migrations are created, but it's a mix of new tables and some tables that already exist in the DB.
If I run migrate it will complain that some of the tables that I'm trying to create already exist.
What I need:
To be able to run the migrations and kind of "ignore" the existing tables and apply the new ones. Or any alternative way to achieve this.
Is that possible?
When you apply a migration, Django inserts a row in a table called django_migrations. That's the only way Django knows which migrations have been applied already and which have not. So the rows in that table have to match the files in your migrations directory. If you've lost the migration files after they were applied, or done anything else to get things out of sync, you'll have problems.. because the migration numbers in your database refer to different migration files than the ones in your project.
So before you do anything else, you need to bring things back into sync by deleting the django_migrations table rows for any migration files that you've lost somehow and can't get back. The table should contain rows for only those migrations that you do have and that were actually applied to the database correctly.
Now you need to deal with any changes in your database that Django Migrations doesn't know about.. and for that there are a few options:
If things worked out such that the database changes that were already applied to the database are in different migration files than the ones that weren't, then you can fix it by running your migrations one at a time using the --fake option on any changes that are in reality already in the database. The fake option just writes the row to the django_migrations table marking the migration as done. Only do this if the database does in fact already have all the changes contained in that migration file.
And those migration files that contain only changes which have not been applied to the database, run without the --fake option and Django will apply them. eg:
# database already has it
manage.py migrate myapp 0003 --fake
# need it
manage.py migrate myapp 0004
# database already has it
manage.py migrate myapp 0005 --fake
If you have migration files where some but not all of the changes have been applied, then you have a bigger problem. In that case, there are several ways to go about it (choose ONLY ONE):
Edit the migration files to put changes that have already been applied (whether Django did it or you did it manually does not matter) into lower number migrations, and put everything you need done into higher numbered files. Now you can --fake the lower number ones, and run the higher numbered ones as normal. Let's say you have 10 changes you made to your models, and 5 of those changes are actually in the database already, but Django doesn't know about them.. so when you run makemigrations, a new migration is created with all 10 changes. This will normally fail because the database server can't for example add a column that already exists. Move these already-applied changes out of your new migration file, into the previous (already applied) migration file. Django will then assume that these were applied with the previous migration and will not try to apply them again. You can then migrate as normal and the new changes will be applied.
If you don't want to touch your older migration file, a cleaner way to do this is to first run makemigrations --empty appname to create an empty migration file. Then run makemigrations which will create another migration with all the changes that Django thinks need to be done. Move the already done migrations from that file into the empty migration you created.. then --fake that one. This will put Django's understanding of what the database looks like will be in sync with reality and you can then migrate as normal, applying the changes in the last migration file.
Get rid of any new migrations you just created using makemigrations. Now, comment out or put back anything in your models that has not been applied to the database, leaving your code matching what's actually in the database. Now you can do makemigrations and migrate appname --fake and you will get things back in sync. Then uncomment your new code and run 'makemigrations' then migrate as normal and the changes will be applied. If the changes are small (for example, adding a few fields), sometimes this is easiest. If the changes are large, it isn't....
You can go ahead and (carefully) make the database changes yourself, bringing the database up to date. Now just run migrate --fake and if you didn't mess up then everything will be ok. Again, this is easy for smaller changes, not as easy for complicated ones.
You can run manage.py sqlmigrate > mychanges.sql. This generates mychanges.sql containing all the SQL Django WOULD have executed against the database. Now edit that file to remove any changes that have already been applied, leaving what needs to be done. Execute that SQL using pgadmin or psql (you're using postgresql I hope). Now the changes have all been made.. so you can run manage.py migrate --fake, this will bring Django into sync with reality and you should be all set. If your SQL skills are sufficient, this is probably the most straightforward solution.
I should add two warnings:
First, if you apply a later migration, eg 0003_foobar.py, and then things don't work out and you decide to try going back and apply 0002_bazbuz.py, then Django will TAKE STUFF OUT OF YOUR DATABASE. For example a column you might have added in 0003 will be dropped along with its data. Since you say you can't lose data, be very careful about going back.
Second, do not rush into running --fake migrations. Make sure that the entire migration you are about to fake is actually in the database already. Else it gets very confusing. If you do regret faking migrations and don't want to roll back, you can erase django's knowledge of the faked migration by deleting that row from the django_migrations table. It is ok to do this.. if you understand what you are doing. If you know that the migration really was not applied, then it's ok.
This blog post really nails it. https://simpleisbetterthancomplex.com/tutorial/2016/07/26/how-to-reset-migrations.html
Let me summarize the steps in his scenario 2 (you have a production database and want to change schema/models in one or more apps). In my case, I had two apps, queue and routingslip, that had model modifications that I needed to apply to a production system. Key was I already had the database, so this is where --fake-initial comes into play.
Here are the steps I followed. As always, backup everything before starting. I do work in a VM so I just took a snapshot before going forward.
1) Remove the migration history for each app.
python manage.py migrate --fake queue zero
python manage.py migrate --fake routingslip zero
2) Blow away any migration files in the entire project within which the app(s) reside.
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
3) Make migrations
python manage.py makemigrations
4) Apply the migrations, faking initial because the database already exists and we just want the changes:
python manage.py migrate --fake-initial
Worked great for me.
If you don't have any migration files or you lost the previous file and want to migrate new changes, then you need to follow the following steps carefully:
# To create a new migration file before changing the models.
cmd: python manage.py makemigrations app_name
# Fake migrate
cmd: python manage.py migrate app_name 0005 --fake #[0005 is the migration file number created just now. It'll seem like 0005_add_address or something like this.]
# To create a new migration file after changing the models.
cmd: python manage.py makemigrations app_name
# database already has it
cmd: python manage.py migrate app_name 0006 #[0006 is the migration file number created just now.]

makemigrations does not work after deleting migrations

I mistakenly deleted all the .py file under path projectName/appName/migrations, it include like:
0001_initial.py
0011_auto_20150918_0723.py
0002_auto_20150819_1301.py
...
Now, even I updated the Model file, and run the command python manage.py makemigrations, it always prompt with No changes detected.
How can I recover everything?
First of all, even if this happened to you in production, do not panic.
When you deleted all migrations django forgot that this app is supposed to be managed by migrations. Django defaults back to the legacy python manage.py syncdb migrationless behaviour and will not attempt to detect changes or generate new migrations when you run python manage.py makemigrations
In order to make it aware of the migrations, you have to run the command specifically for your app:
python manage.py makemigrations appName
However, for your running application, django will not be able to detect that the new migrations were already applied in the database, and will try to run them again when you run python manage.py migrate.
When this happens the migrations fail saying that the relation appName.XYZ already exists!.
To make django understand that your migration is already reflected in the database you have to fake them:
python manage.py migrate appName --fake
This will update the migration history table and mark your migrations as applied, but will not attempt to create the tables in the database.
Update (thanks to #GwynBleiD):
One concern here is that deleted migrations which were already applied, will be listed in the migrations history table in the database. This will not be a problem for the initial python manage.py makemigrations myApp, however for any other migrations it will raise an error about inconsistent migration history.
To avoid that, you must remove by hand any row from django_migrations table in database that refers to nonexistent migrations.

Updating django application, migrations

Is it possible to remove schemamigrations from the Queue? For example. When I run the command ./manage.py migrate --list, it returns a list of all migrations including items that have yet to be migrated.
I have 5 items that have not been migrated yet, but I do not want to migrate 2 of them because I know they will throw an error.
Is is possible to remove those migrations from the 'queue' so to speak?
Thanks!
Remove the corresponding files from yourapp/migrations. There are all files stored after you called
python manage.py makemigrations
These files are named like 0001_initial.py.
If there is no file, nothing happens when you call
python manage.py migrate
When your model is ready, you can create new migrations for your database.
You should use the --fake option when you run migrate to mark those migrations as applied.

Categories

Resources