Develop Reference

Migrate django database to existing one

So I am using Django with mysql database (most basic tables, like auth, user, admin and few custom models) but I need to migrate those tables with data to a existing PostgreSQL database. The issue is that there are tables created there already. Should I create additional models in the models.py file for those models existing in the database already so that migrations will be applied correctly?
I am trying to figure what are the steps that I should take to properly switch databases in the Django app. So far what I have done is saved the data from current (mysql) database:
python manage.py dumpdata > dump.json
Now the next step I took was to change database in settings.py to postgres.
After this I save current table schemas using inspectdb.
Here is where I want to ask what should be next steps(?)
Merge old and new models.py files.
Apply migrations
Add data from json dump file to the new database.

First export table data and take a backup or for this, he needs to clear his table data from Postgres so he or she can migrate all the tables with the existing table and also with the previous table if the data table is there at that time and create new migration with make migration first.

When doing runserver, keep getting new data loaded in my database

Every time I do a: python manage.py runserver
And I load the site, python gets data and puts this in my database.
Even when I already filled some info in the database. Enough to get a view of what I am working on.
Now it is not loading the information I want and instead putting in new information to add to the database so it can work with some data.
What is the reason my data in the database is not being processed?
And how do I stop new data being loaded into the database.

May be it is happening due to migration file first sometimes when you migrate models into database query language with same number
python manage.py makemigrations 0001
This "0001" has to be changed everytime
To solve your problem once delete the migrations file and then again migrate all models and then try
Tell if this work

OperationalError: cursor "_django_curs_<id>" does not exist

We have an online store web-app which is powered by django, postgresql and heroku.
For a specific campaign (you can think a campaign like a product to purchase), we have sold 10k+ copies successfully. Yet some of our users are encountered this error according to our Sentry reports. Common specification of these users is; none of them have address information before the purchase. Generally, users fill out address form right after registering. If they don't, they need to fill the form while purchasing the product and submit them together.
This is how the trace looks like:
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
(66 additional frame(s) were not displayed)
...
File "store/apps/store_main/templatetags/store_form_filters.py", line 31, in render_form
return render_to_string('widgets/store_form_renderer.html', ctx)
File "store/apps/store_main/templatetags/store_form_filters.py", line 20, in render_widget
return render_to_string('widgets/store_widget_renderer.html', ctx)
File "store/apps/store_main/widgets.py", line 40, in render
attrs=attrs) + "<span class='js-select-support select-arrow'></span><div class='js-select-support select-arrow-space'><b></b></div>"
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
So another weird common thing, there are exception messages between sql queries before the failure. You can see it in the image below:
I'm adding it if they are somehow related. What may also be related is, the users who get this error are the users who tries to purchase the campaign right after a bulk mailing. So, extensive traffic might be the reason yet we are also not sure.
We asked Heroku about the problem since they are hosting the postgres, yet they do not have any clue either.
I know the formal reason of this error is trying to reach the cursor after a commit. Since it is destroyed after transaction, trying to reach it cause this error yet I don't see this in our scenario. We are not touching the cursor in any way. What am I missing? What may produce this error? How to prevent it? Any ideas would be appreciated.

The reason for your error might be that you added fields to a model and forgot to makemigrations and migrate.
Have a look at this answer: When in run test cases then I will get this error: psycopg2.OperationalError: cursor "_django_curs_140351416325888_23" does not exist

If you're using django-pytest and have enabled the optimization --reuse-db and have made DB migrations between test runs you need to re-create the DB tables again.
pytest --create-db

Most likely you have forgotten to makemigrations and migrate them to the database.
If you are sure you did make the migrations and running python manage.py makemigrations and python manage.py migrate will not find the changes you made then the database and models are not in sync.
Sometimes this situation can be very frustrating if you have a big database. You will need to manually inspect your models.
To help out you can try this trick, which has been working for me.
Step 1 (delete all migrations from apps)
I am going to assume you are using the unix terminal. run sudo rm -rv */migrations/*. This removes all the migrations files and caches.
Step 2 (make the migration folders in each app)
run the command mkdir <app-folder>/migrations && touch <app-folder>/__init__.py. Replace with the name of the app that you have in the INSTALLED_APPS list in your default django settings file.
Step 3 (Make Migrations)
Here we populate the migrations folders in each app with migration files. Run python manage.py makemigrations. Run the second command. python manage.py migrate --fake. We are using the --fake flag because the data already exists in the database and so we do not want to populate name database tables that are already there or you will be greeted with the already exists error
If this did not work, then you will need to temper with some additional fields on the database. such as the django_migrations or a similarly named table. This is not recommended as there are other database tables that depend on it such as the django_contenttypes and this will throw you in a chain of related tables that you will manually inspect which is very painful.

Schema Migration after dropping table

I added a column to my models.py and it was giving me issues. On the road to trying to solve the problems I've done a couple things.
Dropped the table: ./manage.py sqlclear app | ./manage.py dbshell
Tried to "reset" the schema: ./manage.py schemamigration app --initial
Tried to migrate: ./manage.py migrate app
After doing all these things, I get this error after trying to migrate:
FATAL ERROR - The following SQL query failed: CREATE TABLE "projects_project" ("id" integer NOT NULL PRIMARY KEY)
The error was: table "projects_project" already exists
Question: How do I repair my database? I don't care about any of the data in the db.
Edit:
One of the related posts took me to this link Django South - table already exists . Apparently if you fake the migration all is well.
./manage.py migrate myapp --fake
I'm still unsure of all the reprocussions of this but I guess thats what docs are for.

Welll, the error points it out pretty clearly:
table "projects_project" already exists
You can either do it the quick and dirty way and drop the table. In that case log into your DMBS. If it's MySQL, you'll just open the terminal and typ:
mysql -u root -p YOURPASSWORD
Then select the database:
use your_database;
Finally, drop the table:
DROP TABLE projects_project;
You should be able to migrate now.
The elegant way would be to undo the migration. But every framework has it's own way to do that. You need to figure that out first - or give us more information.

How to prevent Django fixtures from conflicting with existing data

I'm trying to do manage.py loaddata myfixture.json to purposely bulk overwrite several records in my database. However, it gives me the error:
IntegrityError: Problem installing fixture 'myfixture.json': Could not load myapp.Person(pk=1): (1062, "Duplicate entry 'Bob' for key 'name'")
I thought the behavior of loaddata was to overwrite the records if primary key matches an existing record? Am I wrong, or was this changed in Django 1.5? How do I get it to overwrite?

I found a way to modify Django's loaddata command to properly parse natural keys while loading, and avoid re-loading duplicates. The modified command is published here. The only differences between the official loaddata are lines 189-201.