I'm trying to do manage.py loaddata myfixture.json to purposely bulk overwrite several records in my database. However, it gives me the error:
IntegrityError: Problem installing fixture 'myfixture.json': Could not load myapp.Person(pk=1): (1062, "Duplicate entry 'Bob' for key 'name'")
I thought the behavior of loaddata was to overwrite the records if primary key matches an existing record? Am I wrong, or was this changed in Django 1.5? How do I get it to overwrite?
I found a way to modify Django's loaddata command to properly parse natural keys while loading, and avoid re-loading duplicates. The modified command is published here. The only differences between the official loaddata are lines 189-201.
Related
I have inserted data into table from postgresql directly. Now when I try to insert data from django application, it's generating primary key duplication error. How can I resolve this issue?
Run
python manage.py sqlsequencereset [app_name]
and execute all or just one for the required table SQL statements in the database to reset sequences.
Explanation:
You probably inserted with primary keys already present in it, not letting postgresql to auto-generate ids. This is ok.
This means, internal Postgresql sequence used to get next available id has old value. You need to rest with sequence to start with maximum id present in the table.
Django manage.py has command intended just for that - print sql one can execute in db to reset sequences.
I think problem is not in database. please check your django code probably you use get_or_create
We have an online store web-app which is powered by django, postgresql and heroku.
For a specific campaign (you can think a campaign like a product to purchase), we have sold 10k+ copies successfully. Yet some of our users are encountered this error according to our Sentry reports. Common specification of these users is; none of them have address information before the purchase. Generally, users fill out address form right after registering. If they don't, they need to fill the form while purchasing the product and submit them together.
This is how the trace looks like:
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
(66 additional frame(s) were not displayed)
...
File "store/apps/store_main/templatetags/store_form_filters.py", line 31, in render_form
return render_to_string('widgets/store_form_renderer.html', ctx)
File "store/apps/store_main/templatetags/store_form_filters.py", line 20, in render_widget
return render_to_string('widgets/store_widget_renderer.html', ctx)
File "store/apps/store_main/widgets.py", line 40, in render
attrs=attrs) + "<span class='js-select-support select-arrow'></span><div class='js-select-support select-arrow-space'><b></b></div>"
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
So another weird common thing, there are exception messages between sql queries before the failure. You can see it in the image below:
I'm adding it if they are somehow related. What may also be related is, the users who get this error are the users who tries to purchase the campaign right after a bulk mailing. So, extensive traffic might be the reason yet we are also not sure.
We asked Heroku about the problem since they are hosting the postgres, yet they do not have any clue either.
I know the formal reason of this error is trying to reach the cursor after a commit. Since it is destroyed after transaction, trying to reach it cause this error yet I don't see this in our scenario. We are not touching the cursor in any way. What am I missing? What may produce this error? How to prevent it? Any ideas would be appreciated.
The reason for your error might be that you added fields to a model and forgot to makemigrations and migrate.
Have a look at this answer: When in run test cases then I will get this error: psycopg2.OperationalError: cursor "_django_curs_140351416325888_23" does not exist
If you're using django-pytest and have enabled the optimization --reuse-db and have made DB migrations between test runs you need to re-create the DB tables again.
pytest --create-db
Most likely you have forgotten to makemigrations and migrate them to the database.
If you are sure you did make the migrations and running python manage.py makemigrations and python manage.py migrate will not find the changes you made then the database and models are not in sync.
Sometimes this situation can be very frustrating if you have a big database. You will need to manually inspect your models.
To help out you can try this trick, which has been working for me.
Step 1 (delete all migrations from apps)
I am going to assume you are using the unix terminal. run sudo rm -rv */migrations/*. This removes all the migrations files and caches.
Step 2 (make the migration folders in each app)
run the command mkdir <app-folder>/migrations && touch <app-folder>/__init__.py. Replace with the name of the app that you have in the INSTALLED_APPS list in your default django settings file.
Step 3 (Make Migrations)
Here we populate the migrations folders in each app with migration files. Run python manage.py makemigrations. Run the second command. python manage.py migrate --fake. We are using the --fake flag because the data already exists in the database and so we do not want to populate name database tables that are already there or you will be greeted with the already exists error
If this did not work, then you will need to temper with some additional fields on the database. such as the django_migrations or a similarly named table. This is not recommended as there are other database tables that depend on it such as the django_contenttypes and this will throw you in a chain of related tables that you will manually inspect which is very painful.
I'm looking to write tests for my application. I would like to work with a clean database for all my tests. For various reasons, I cannot create a separate test database.
What I currently do is run everything in a transaction and never commit to the db. However some tests read from the db, so I'd like to delete all rows at the start of the transaction and start from there.
The problem I am running into is with foreign key constraints. Currently I just go through each table and do
cursor.execute("DELETE FROM %s" % tablename)
which gives me
IntegrityError: (1451, u'Cannot delete or update a parent row: a
foreign key constraint fails (`testing`.`app_adjust_reason`,
CONSTRAINT `app_adjust_reason_ibfk_2` FOREIGN KEY (`adjust_reason_id`)
REFERENCES `adjust_reason` (`id`))')
edit: I would like something generic that could be applied to any database. Otherwise I would specifically drop the constraints
A more general approach is to create a database from scratch before the test run and drop it after, use CREATE DATABASE_NAME and DROP DATABASE_NAME. This way, you are always starting with a clean database state and you would not worry about the foreign key or other constraints.
Note that you would also need to create your table schema and (possibly test data) after you create a database.
As a real world example, this is what Django does when you run your tests. The table schema is recreated by the Django ORM from your models, then the fixtures or/and the schema and data migrations are applied.
When trying to run my tests (python manage.py test) I am getting:
CommandError: Database test_db couldn't be flushed. Possible reasons:
* The database isn't running or isn't configured correctly.
* At least one of the expected database tables doesn't exist.
* The SQL was invalid.
Hint: Look at the output of 'django-admin.py sqlflush'. That's the SQL this command wasn't able to run.
The full error: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "install_location_2015_05_13" references "app".
HINT: Truncate table "install_location_2015_05_13" at the same time, or use TRUNCATE ... CASCADE.
I am using partitions in our project which are generated on the fly via a python function (so I can run it periodically). I don't have any models for these partitions.
The partition maintenance function is invoked after syncdb triggers the post_syncdb signal (so it is executed when the test database is set up).
How can I make Django clear the additional tables (partitions)? or
How can I tell Django to use CASCADE while running the tests?
Mainly problem occur where we change relation in M2M field. Old constraint remain and new constraint for new relation was created. Fixed in Django 1.8
I just joined a project using Django, and am attempting to initialize my own development server. When I attempt to do so, the migration fails for one of my apps. A model for this app has a sorl.thumbnail.ImageField, to add a logo. When the migration is attempted, I get the following error message:
FATAL ERROR - The following SQL query failed: ALTER TABLE "accounts_account" ADD CONSTRAINT "logo_id_refs_file_ptr_id_7c3d1997" FOREIGN KEY ("logo_id") REFERENCES "filer_image" ("file_ptr_id") DEFERRABLE INITIALLY DEFERRED;
The error was: relation "filer_image" does not exist
I'm not sure what the problem could be, as apparently there was no problem in creating the production database or subsequent migrations.
I believe I have correctly installed sorl-thumbnail, and all of its dependencies.
Your help would be much apperciated.
I'm pulling the answer provided by Johndt6 from a comment to this answer for future search-ability.
The solution is to add filer to the INSTALLED_APPS tuple.