Develop Reference

How to deal with `Migration is applied before its dependency`

I have been assigned the task to work on project involving react UI and django backend. There are two versions of this app:
older version which runs on older versions of react and python (v2)
newer version which runs on newer versions of react and python (v3)
I have been tasked to move some functionality from older version to newer version and test it with postgres database dump from a live environment running older version. The challenge could be differences in the database schema of newer and older versions. (But, most possibly there wont be much differences if some minor ones.)
So I proceeded to take the database dump, attached it to the database running on my laptop and specified its name in my django's settings.ini. But when I started my django app, it gave me error
You have 7 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, lab, otherlabs. Run 'python manage.py migrate' to apply them.
When I ran python manage.py migrate, it gave me an error:
Migration abc.0001_initial is applied before its dependency auth.0011_xyz on database 'default'.
So, I deleted record corresponding to abc.0001_initial from django_migrations table and reran python manage.py migrate. Now I got same error for migration of another project def but with the same dependency auth.0011_xyz:
Migration def.0001_initial is applied before its dependency auth.0011_xyz on database 'default'.
Should I proceed with deleting record corresponding to def.0001_initial too? Am afraid that this will continue till I delete all such 0001_initial-suffixed records. There are 35 projects and hence 35 such 0001_initial-suffixed records.
Q1. Is it safe to delete them all and run the migrations? Is deleting all migrations and rerunning them really bad idea if they are schema-only migrations (we use fixtures for importing data)? Or is it simply impossible to do as most of those migrations will fail as tables corresponding to most models are already there in the database (or this wont give any error?), so the only solution remained is to delete migrations record one by one from db till I get no errors. But wont this still give error as I am only deleting migrations record and not actual tables created by migrations? Is there any other approach?
Q2. Are migrations completely safe to "re"-run? I mean if one fails because corresponding table / columns are already there or for some other reasons, will it leave database in consistent state or will it mess up the database?
Q3. Also will it continue to execute rest of the migrations after one fails? OR
Q4. Can / should I execute rest of the migrations if one fails? If yes, do I need to pass some arguments to python manage.py migrate or its default behavior?
Update
I tried removing all migrations from the django_migrations table. And then running python manage.py makemigrations followed by python manage.py migrate. But migration did not complete and terminated with error "relation already exist". So applied fake migration python manage.py migrate --fake. It succeeded (seems that it faked / skipped all migrations as all corresponding tables were present, so it did not create new columns in already existing tables). But then my REST API calls started failing with error in response saying: "column xyz does not exist".
Manually copying data seems impossible since there are 156 tables 🥲
Please let me know if there are any solution to this.

You need to delete the migration row from django_migrations for migration def.0001_initial, then apply migrate auth 0011_xyz to fix the dependency and then you should can to do migrate def 0001_initial --fake to fake the migration and add the row you deleted before to django_migrations.

Delete db.sqlite via terminal (Django)

I'm currently deploying a project to Heroku, however it returns me the following error: ValueError: Related model 'store.user' cannot be resolved.
But there is a way to avoid such error locally. You simply do:
py manage.py migrate store
And then
py manage.py migrate
In other words, if I migrate separetely, I won't face such error. However, Heroku migrates all together, then the deploy fails because of this error.
If I proceed locally as Heroku does, i.e. run
py manage.py migrate
I can effortlessly click on and delete the db.sqlite3 file and makemigrations and migrate separetely. Then the issue would be resolved. However, that's not possible when deploying to Heroku. Thus, how can I delete this file only via terminal? I have been searching, but people only say you click on the file and the delete it, which for me it is not possible.
Thanks

However, Heroku migrates all together, then the deploy fails because of this error
Heroku does nothing of the sort.
Migrations only run on Heroku when you tell them to, either by running something like
heroku run python manage.py migrate
or because you have declared a release process that should run when you deploy a new version, e.g.:
web: gunicorn app.wsgi
release: python manage.py migrate
In both cases you are in charge of what that command is. If you don't want to run python manage.py migrate every time you deploy, remove the release process from your Procfile.
if I migrate separately, I won't face such error
This is concerning.
There could be several causes for this. One common issue is missing dependencies in your migration. If you write a migration in app1 that depends on certain models and tables from app2 existing, you need to add a dependency to the relevant migration in app1, e.g. something like this example from the documentation:
class Migration(migrations.Migration):
dependencies = [
('app1', '0001_initial'),
# added dependency to enable using models from app2 in move_m1
('app2', '0004_foobar'),
]
operations = [
migrations.RunPython(move_m1),
]
If migrations are written properly, such that they accurately reflect the code in each commit, build off each other, have dependencies declared, etc. you should always be able to safely apply them to your database.
Finally, you ask about deleting db.sqlite.
Heroku's filesystem is ephemeral. It gets baked into your application slug at build time, and any changes you make to it get reset every time your dyno restarts. This happens frequently (at least once per day).
This means you can't effectively delete your database file. But it also means you can't save data and expect it to be there when you look it up later! SQLite isn't a good fit for Heroku.
A client-server database like PostgreSQL is a much better choice. Heroku offers its own Postgres service with a free tier.
If you switch, I strongly urge you to switch in development as well. Django's ORM makes it relatively easy to change, but database engines aren't drop-in replacements for each other. I've seen real-world examples where developing on SQLite and deploying to Postgres or MySQL causes things to fail in production that worked in development.

Django migration not applied to the DB

I had an Django2.2.3 app, it was working fine. But I had to chane the name of a field in a table, and add another field. Then I ran ./manage.py makemigrations && ./manage.py migrate. Besides the terminal prompt:
Running migrations:
No migrations to apply.
No error is throwed. But then when I go to the MySQLWorkbench to check the database, it is exactly as I didn't make any change. I tried deleting the migrations and making again, the process ends with no errors but the database don't change. I create another empty database, change the name on settings.py and make migrations and migrate again, and it worked, but when I put the old database name on the settings, it just did not work.
Can someone explain this behavior for me? There is any kind of cache for these information migrations or something? I realy want to know why this is not winrkig as I espect.

Make sure the app with the migrations is in the INSTALLED_APPS. Django won't look at the app for changes otherwise.

Adding new few fields to an existing model (table) is one reason for this problem. A way to go about this is simply as follows:
a) un-apply the migrations for that app:
python3 manage.py migrate --fake <app-name> zero
b) migrate the required migrations (you've already deleted previous migrations and you've done 'makemigrations' for the newly added column. So, you just migrate:
python3 manage.py migrate <app-name>
If the steps above didn't solve the problems, then drop the table first;
i) python3 manage.py dbshell
ii) DROP TABLE appname_tablename
close the shell and repeat a and b again.

DJANGO HEROKU :django.db.migrations.exceptions.InconsistentMigrationHistory:

i have pushed my django application to heroku
and when i tried to fireup the url where my application is
i just got this error
ProgrammingError at /
relation "accounts_user" does not exist
LINE 1: ..."."is_active", "accounts_user"."date_joined" FROM "accounts_...
so i have figured out that the issue is with migration so i tried to run the makemigrations command in heroku
but unfortunately it returned as a failure with an error message
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration posts.0001_initial is applied before its dependency accounts.0001_initial on database 'default'.
so what i have tried is i have completely removed the migrations and database and runnned makemigration and migrate command in my local server and then i pushed it back to heroku but still its of no use
my local is running perfectly fine
i'm not sure where exactly the migrations issue is , in my heroku django application
i want to remove the previous migrations and have a clean database
but i have no idea on how to do that

I was facing the same issue yesterday. I was using Django rest and sqlite3 DB.
To solve that error I have clear all migration and delete a migration folder and clear DB file. After that, I have run migrations specific for that module and then run migrate command then it works fine for me. I hope it will work for you as well.
e.g.
$ python manage.py makemigrations module_name
$ python manage.py migrate module_name

South: how to revert migrations in production server?

I want to revert my last migration (0157) by running its Migration.backwards() method. Since I am reverting the migration in production server I want to run it automatically during code deployment. Deployment script executes these steps:
Pull code changes
Run migrations: manage.py migrate <app>
Refresh Apache to use newest code: touch django.wsgi
If I could, I would create new migration file which would tell South to backward migrate to 0156:
migrations/0158_backward__migrate_to_0156.py
This commited migration would be deployed to production and executed during manage.py migrate <app> command. In this case I wouldn't have to execute backward migration by hand, like suggested in these answers.
Lets say, I have created two data migrations, first for user's Payment, second for User model. I have implemented backwards() methods for both migrations in case I'd have to revert these data migrations. I've deployed these two migrations to production. And suddenly find out that Payment migration contains an error. I want to revert my two last data migrations as fast as possible. What is the fastest safe way to do it?

Since I am reverting the migration in production server I want to run
it automatically during code deployment.
IMHO the safest path is
run manage.py migrate <app> (i.e. apply all existing migrations, i.e. up to 0156)
undo the changes in your model
run manage.py schemamigration <app> --auto
This will create a new migration 0157 that effectively reverts the previous migration 0156. Then simply apply the new migration by running manage.py migrate <app> again. As I understand, your code deployment will just do that.

Apparently the codeline has migrations up to #157 and now the developer decided that the last one was not a good idea after all. So the plan is to go back to #156.
Two scenarios:
(a) migration #157 was not released or deployed anywhere yet.
Simply revert the last change from models.py and delete migration #157.py from the source archive. Any deployment will take the system to level 156; "157 was never there".
(b) there have been deployments of the latest software with migration #157.
In this case the previous strategy will obviously not work. So you need to create a migration #158 to undo #157. Revert the change in models.py and run
django manage.py migrate <app> 0157
django manage.py schemamigration <app> --auto
This will auto-generate a new migration #158, which will contain the inverse schema migration compared to #157.
If schemamigration is giving trouble because of django Model validation (something that can happen if you have custom validators which check stuff outside the ORM box), I suggest the following workaround:
<django project>/<app>/management/commands/checkmigrations.py
from south.management.commands import schemamigration
class Command(schemamigration.Command):
requires_model_validation = False
help = "schemamigration without model validation"
This command becomes available in manage.py:
django manage.py checkmigrations <app> --auto

There's no silver bullet here. The simplest solution I can think of would be to - in your dev env of course - manually migrate back to 0156, manually update your migration's history table (sorry I can't remember the table's name now) to fool south in thinking you're still #0158, then run schemamigration again. Not garanteed to work but might be worth trying.