So I am using Django with mysql database (most basic tables, like auth, user, admin and few custom models) but I need to migrate those tables with data to a existing PostgreSQL database. The issue is that there are tables created there already. Should I create additional models in the models.py file for those models existing in the database already so that migrations will be applied correctly?
I am trying to figure what are the steps that I should take to properly switch databases in the Django app. So far what I have done is saved the data from current (mysql) database:
python manage.py dumpdata > dump.json
Now the next step I took was to change database in settings.py to postgres.
After this I save current table schemas using inspectdb.
Here is where I want to ask what should be next steps(?)
Merge old and new models.py files.
Apply migrations
Add data from json dump file to the new database.
First export table data and take a backup or for this, he needs to clear his table data from Postgres so he or she can migrate all the tables with the existing table and also with the previous table if the data table is there at that time and create new migration with make migration first.
Related
Using PostgresSQL and Django, I dropped two tables from postgresql manually.
drop table ...
Then I did make migrations and migrate.
But my tables are not recreated. Can not access from shell_plus. Also in this page, django return relation 'table name' does not exists.
I want to makemigrations and migrate to create tables again.
How can I solve my issue ?
It depends on your current migrations files tree structure.
If your migration file containing CreateModel for those tables you deleted directly from PostgreSQL are the leaf nodes, i.e, there was no other migration file after that, you can simply delete the entry of the migration file in the django_migrations table and run migrate.
For example,
app/migrations/0002_20210813_122.py is the file having commands for the creation of your tables, and this is the last node ( how do we know if this is the last file? so you just check if there's any other migration file in your project which has this filename 0002_20210813_122 under its dependencies field, if no then this file is the leaf node ). If it's a leaf node, go to django_migrations table in your database and delete an entry with value 0002_20210813_122 under column name and column app should be your app_name.
Now run python manage.py migrate, the tables will be recreated.
If your migration file isn't a leaf node, then kindly share the tree structure of your migrations file, for us to help you out.
I am already having one mysql database with a lot of data, of which tables and migrations are written in sql. Now I want to use the same mysql database in django so that I can use the data in that database.I am expecting that there will not be need for making the migrations as I am not going to write the models in Django again, also what will be the changes/modification I will have to do. For eg: as in middlewares?. Can anyone please help me in this?
From what I know there is no 100 % automatic way to achieve that.
You can use the following command
python manage.py inspectdb
It will generate a list of unmanaged models that you can export to a model.py file and integrate in your django project.
However it is not magical and there are a lot of edge cases so the generated list of model should be manually inspected before being integrated.
More info here : https://docs.djangoproject.com/en/3.0/ref/django-admin/#django-admin-inspectdb
This question already has answers here:
Is it possible to generate django models from the database?
(3 answers)
Closed 5 years ago.
I am practising to develop a complex app, i have previously developed the same with PHP and have a sql file. i'd like to know how do i create new models and populate the data from the sql file. I have read tutorials about grabbing from JSON files, But is there any online resources for the same.
You can use manage.py inspectdb command.
It will connect to the DB and create model classes.
So you need to create the database manually and use DB tools to read structure and data from sql file. After that you should put connection settings into your settings.py file and execute:
$ python manage.py inspectdb > models.py
Doc - Integrating with legacy database
UPD as I understand you want to move from your SQL file to django-only solution. So this is how you can achieve it.
Deploy the temporary DB and load dump with some tool like PgAdmin.
Setup your django app so it can connect to the same DB
Use inspectdb to create models
Use dumpdata to save values as json file
Remove managed = False from your scaffolded models (from point 3)
Create migrations with makemigrations command.
After that you will be able to deploy your application with Django tools: migrations to create DB structure and loaddata to load data from json.
With this setup:
-Development environment
-Flask
-SQLAlchemy
-Postgres
-Possibility Alembic
If I have a database with some tables populated with random data. As far as I know the Flask-Migrate, that will use Alembic, will not preserve the data, only keep the models and database synchronized.
But what is the difference between the use of Alembic or just delete > create all tables?
Something like:
db.create_all()
The second question:
What happens to the data when something change in models? The data will be lost? Or the Alembic can preserve the previous populated data?
Well, my idea is just populate the database with some data, and then avoid any lost of data
when the models change. Alembic is the solution?
Or I need to import the data, from a .sql file, for example, when I change the models and database?
I am the Flask-Migrate author.
You are not correct. Flask-Migrate (through Alembic) will always preserve the data that you have in your database. That is the whole point of working with database migrations, you do not want to lose your data.
It seems you already have a database with data in it and you want to start using migrations. You have two options to incorporate Flask-Migrate into your project:
Only track migrations going forward, i.e. leave your initial database schema outside of migration tracking.
For this you really have nothing special to do. Just do manage.py db init to create the migrations repository and when you need to migrate your database do so normally with manage.py db migrate. The disadvantage of this method is that Flask-Migrate/Alembic do not have the initial schema of the database, so it is not possible to recreate a database from scratch.
Implement an initial migration that brings your database to your current state, then continue tracking future migrations normally.
This requires a little bit of trick. Here you want Alembic to record an initial migration that defines your current schema. Since Alembic creates migrations by comparing your models to your database, the trick is to replace your real database with an empty database and then generate a migration. After the initial migration is recorded, you restore your database, and from then on you can continue migrating your database normally.
I hope this helps. Let me know if you have any more questions.
I need to refresh a table every day so I want to make a python script to drop a table form the database or clear it, recreate it and the repopulate it with a new data from a csv file.
How can I clear the data from the table or drop the table and then recreate it?
Either you can use sqlclear like so:
python manage.py sqlclear my_app | python manage.py dbshell
but then you need to create the table again with syncdb.
You could also just delete all posts in your table with MyApp.objects.all().delete()