Develop Reference

Delete db.sqlite via terminal (Django)

I'm currently deploying a project to Heroku, however it returns me the following error: ValueError: Related model 'store.user' cannot be resolved.
But there is a way to avoid such error locally. You simply do:
py manage.py migrate store
And then
py manage.py migrate
In other words, if I migrate separetely, I won't face such error. However, Heroku migrates all together, then the deploy fails because of this error.
If I proceed locally as Heroku does, i.e. run
py manage.py migrate
I can effortlessly click on and delete the db.sqlite3 file and makemigrations and migrate separetely. Then the issue would be resolved. However, that's not possible when deploying to Heroku. Thus, how can I delete this file only via terminal? I have been searching, but people only say you click on the file and the delete it, which for me it is not possible.
Thanks

However, Heroku migrates all together, then the deploy fails because of this error
Heroku does nothing of the sort.
Migrations only run on Heroku when you tell them to, either by running something like
heroku run python manage.py migrate
or because you have declared a release process that should run when you deploy a new version, e.g.:
web: gunicorn app.wsgi
release: python manage.py migrate
In both cases you are in charge of what that command is. If you don't want to run python manage.py migrate every time you deploy, remove the release process from your Procfile.
if I migrate separately, I won't face such error
This is concerning.
There could be several causes for this. One common issue is missing dependencies in your migration. If you write a migration in app1 that depends on certain models and tables from app2 existing, you need to add a dependency to the relevant migration in app1, e.g. something like this example from the documentation:
class Migration(migrations.Migration):
dependencies = [
('app1', '0001_initial'),
# added dependency to enable using models from app2 in move_m1
('app2', '0004_foobar'),
]
operations = [
migrations.RunPython(move_m1),
]
If migrations are written properly, such that they accurately reflect the code in each commit, build off each other, have dependencies declared, etc. you should always be able to safely apply them to your database.
Finally, you ask about deleting db.sqlite.
Heroku's filesystem is ephemeral. It gets baked into your application slug at build time, and any changes you make to it get reset every time your dyno restarts. This happens frequently (at least once per day).
This means you can't effectively delete your database file. But it also means you can't save data and expect it to be there when you look it up later! SQLite isn't a good fit for Heroku.
A client-server database like PostgreSQL is a much better choice. Heroku offers its own Postgres service with a free tier.
If you switch, I strongly urge you to switch in development as well. Django's ORM makes it relatively easy to change, but database engines aren't drop-in replacements for each other. I've seen real-world examples where developing on SQLite and deploying to Postgres or MySQL causes things to fail in production that worked in development.

Django: broken migrations

I am trying to setup a Django app locally in a new machine but migrations seem to be totally broken. They need to be performed in a particular order, which worked in the first machine I set the environment in a couple months ago, but now there are inconsistencies (although I am pretty sure no new migrations were generated).
So the only solution I can think of is exporting the database from the old machine, where it is working, to the new one. Would that work?
This would not solve the broken migrations issue, but at least I can work on the code till there's a proper soltuion.

Answering this question:
So the only solution I can think of is exporting the database from the old machine, where it is working, to the new one. Would that work?
Yes, this can work if you are sure that your database is in sync with your models. It is actually the way to go, if you want to be best prepared of updating your production environment.
get a dump from the current production machine
create a new database and load the dump
check whether there are differences between the models and the migration history (this is more reliable with the new Django migrations, South was an external tool and had not all of the possibilities) (e.g. ./manage.py showmigrations (1.10), ./manage.py migrate --list (1.7-1.9 and South)
If you are confident that no migrations have to be run but the listing shows differences then do: ./manage.py migrate --fake
Note, in newer versions you can do ./manage.py migrate and it will report that everything is in order if the models and the migrations are in sync. This can be a sanity check before you deploy onto production.

How can I use Flask-migrate across multiple development environments

I have flask-migrate (version 1.8.0) working well with a sqlite database in a development environment. Now I would like to migrate our data to MySQL and maintain all of our migration history (so it stays in sync with our Flask-SQLAlchemy models in our git repository).
I created an empty MySQL database, and after changing my SQLALCHEMY_DATABASE_URI, I tried running:
python manage.py db upgrade
That resulted in an error about not being able to drop the table migrate_version. (Which makes sense, since this is a new database, although sqlite actually contains the table 'alembic_version' not 'migrate_version'.)
So, I tried to initialize this new database:
python manage.py db init
Now I get an error: "Directory migrations already exists".
I can rename that folder and re-run the command with no problem, but then I lose all of my previous migrations. I think we would have the same issues when we also transition to our test and production environments.
I've seen in the docs Flask-Migrate has multiple database support, but I think that looks to be more for maintaining multiple databases in a single development environment. Is there a way to have Flask-Migrate track changes across multiple development environments?

To address the real issue in the OP's question, you need to use the --directory flag to initiate a migrations directory specific to your each environment's database.
From the flask-migrate documentation:
All commands also take a --directory DIRECTORY option that points to
the directory containing the migration scripts. If this argument is
omitted the directory used is migrations.
So:
flask db init --directory=[DIRECTORY NAME]
Flask-Migrate itself has no memory of your database, so when running migration commands with flask db, it will reference the specified migrations directory (by default, when the --directory flag is not used, this is called 'migrations').
flask db migrate --directory=[DIRECTORY_NAME]
etc.
It goes without saying that the flask command will reference the application context as configured by your config file or environment variables.
I typically create a migration directory for each environment with an explicit reference to the environment: e.g. development and staging, with something like 'migrations_dev' and 'migrations_stg'.
Hope this is helpful.

Here are the steps I took to transition from SQLite to MySQL and maintain all the migration history. I highly suspect there is a better way to do this, but it worked for me.
Initialize the new, blank database using another folder for your "new" migrations
python manage.py db init -d tmp
Create a migration
python manage.py db migrate -d tmp -m "Bring MySQL up to date"
Apply the migration
python maange.py db upgrade -d tmp
Now, you can delete the "tmp" migrations folder. You no longer need it. Find the HEAD migration. Look for 'Rev: your_revision_num (head)'
python manage.py db show
Run an update statement against your MySQL database
update alembic_version set version_num = 'your_revision_num'
Now your MySQL database schema should match your old SQLite schema and you'll have your full migration history as well.

The table migrate_version is used to track migrations by package sqlalchemy-migrate. Alembic, the package used by Flask-Migrate, uses a alembic_version table, as you know.
So my guess, is that this MySQL database that you want to use has been used before by an application that was under sqlalchemy-migrate control.
I recommend that you delete the MySQL database and make a brand new one.

I also had the same need on my side. I wanted to reproduce the command that exists in the laravel framework to make a migration in different environments:
php artisan migrate --env prod
With this kind of command, you can launch a migration in different environments. I have not found a directly equivalent command in flask.
THE "flask db upgrade --env prod" command does not exist. In particular, the --env argument .
As a workaround, I created a command that allows me to change the environment:
flask env --name prod
That command is a custom flask command that will copy the content of the .env.prod file to .env.
This way, the application is in the prod environment. And we can then launch the migration command on the prod environment.
How to use the custom env command to migrate to different environments?
To start the migration in the staging environment, just run these two commands:
flask env --name staging
flask db updgrade
Then if you want to start the migration in the production environment, just run these two commands:
flask env --name prod
flask db updgrade
How to create the custom command flask env?
First, you need to know how to create custom command in flask. Just follow the official falsk documentation
Here is the content of my custom command which allows to change the environment:
from flask.cli import with_appcontext
import shutil
#click.command('env')
#click.option("--name", is_flag=False, flag_value="Flag", default="")
#with_appcontext
def env(name):
if name == '':
print(os.getenv('FLASK_ENV'))
return
if name not in ['dev', 'prod', 'local', 'staging']:
print('That env does not exist')
return
shutil.copyfile('.env.' + name, '.env')
return
In my setup, I have 4 environments: local, dev, staging, prod.
And I have 4 corresponding .env files: .env.local, .env.staging, .env.prod, .env.dev
The custom flask env command also copies the contents of the environment files into the .env file that the flask application loads at start-up.

Python social auth in Django, makemigrations detects no changes

I was following the documentation to get python social auth on my django project
https://python-social-auth.readthedocs.org/en/latest/configuration/django.html
And after adding 'social.apps.django_app.default', to the INSTALLED_APPS in my settings.py I run this:
python manage.py makemigrations
I get this
No changes detected
Shouldn't this command be doing something. Because without this I can't migrate to create the tables that are needed for the auth.
EDIT:
I've also tried this command, and still ended up getting the same result
python manage.py makemigrations main
where 'main' is the name of my app

Today i ran into this problem. The error is in the documentation itself.
You should run $ python manage.py migrate directly. It creates tables in the database.
All the old tutorials used makemigrations, I think it was used in earlier versions of django.

My answer will be cover some basics so one can understand this types of errors easily.
Let me clear some basic terminology about migrations in the latest versions of Django(From 1.7 to under development ones).
In older versions of Django when One has to make changes in Models (Eventually in Database) then One has to use South app to apply changes in Database without affecting older database.
Django developer community has included this south app in and after Django 1.7 and also provided some simple commands to apply migrations.
When one install New app(Above question scenario) or one make changes in existing models and wish to apply changes in database then one has to tell database about what changes they want to make. To do so one has to make migrations and below is the command.
$ python manage.py makemigrations app_name
or If it is initial then no need to specify app_name, it will consider all apps.
This command will generate migrations files which will include instructions for database to make what tables and what are the attributes of that table and what will be the relationships between tables and what are changes in the current tables etc etc.
Now one has to run below command to apply this all changes in database.
$ python manage.py migrate app_name
or If it is initial then no need to specify app_name.
Please shoot any questions If you have any and please take a look at Django's official documentation on migrations for more information.

The possible reason is your project doesn't use any database models of 'social' application yet. Add a URL in your urls.py, link it to 'social' urls.

to Django < 1.8
INSTALLED_APPS ['social.apps.Django_appConfig',]

South: how to revert migrations in production server?

I want to revert my last migration (0157) by running its Migration.backwards() method. Since I am reverting the migration in production server I want to run it automatically during code deployment. Deployment script executes these steps:
Pull code changes
Run migrations: manage.py migrate <app>
Refresh Apache to use newest code: touch django.wsgi
If I could, I would create new migration file which would tell South to backward migrate to 0156:
migrations/0158_backward__migrate_to_0156.py
This commited migration would be deployed to production and executed during manage.py migrate <app> command. In this case I wouldn't have to execute backward migration by hand, like suggested in these answers.
Lets say, I have created two data migrations, first for user's Payment, second for User model. I have implemented backwards() methods for both migrations in case I'd have to revert these data migrations. I've deployed these two migrations to production. And suddenly find out that Payment migration contains an error. I want to revert my two last data migrations as fast as possible. What is the fastest safe way to do it?

Since I am reverting the migration in production server I want to run
it automatically during code deployment.
IMHO the safest path is
run manage.py migrate <app> (i.e. apply all existing migrations, i.e. up to 0156)
undo the changes in your model
run manage.py schemamigration <app> --auto
This will create a new migration 0157 that effectively reverts the previous migration 0156. Then simply apply the new migration by running manage.py migrate <app> again. As I understand, your code deployment will just do that.

Apparently the codeline has migrations up to #157 and now the developer decided that the last one was not a good idea after all. So the plan is to go back to #156.
Two scenarios:
(a) migration #157 was not released or deployed anywhere yet.
Simply revert the last change from models.py and delete migration #157.py from the source archive. Any deployment will take the system to level 156; "157 was never there".
(b) there have been deployments of the latest software with migration #157.
In this case the previous strategy will obviously not work. So you need to create a migration #158 to undo #157. Revert the change in models.py and run
django manage.py migrate <app> 0157
django manage.py schemamigration <app> --auto
This will auto-generate a new migration #158, which will contain the inverse schema migration compared to #157.
If schemamigration is giving trouble because of django Model validation (something that can happen if you have custom validators which check stuff outside the ORM box), I suggest the following workaround:
<django project>/<app>/management/commands/checkmigrations.py
from south.management.commands import schemamigration
class Command(schemamigration.Command):
requires_model_validation = False
help = "schemamigration without model validation"
This command becomes available in manage.py:
django manage.py checkmigrations <app> --auto

There's no silver bullet here. The simplest solution I can think of would be to - in your dev env of course - manually migrate back to 0156, manually update your migration's history table (sorry I can't remember the table's name now) to fool south in thinking you're still #0158, then run schemamigration again. Not garanteed to work but might be worth trying.