Django Migrating to a new Database - python

I just joined a project using Django, and am attempting to initialize my own development server. When I attempt to do so, the migration fails for one of my apps. A model for this app has a sorl.thumbnail.ImageField, to add a logo. When the migration is attempted, I get the following error message:
FATAL ERROR - The following SQL query failed: ALTER TABLE "accounts_account" ADD CONSTRAINT "logo_id_refs_file_ptr_id_7c3d1997" FOREIGN KEY ("logo_id") REFERENCES "filer_image" ("file_ptr_id") DEFERRABLE INITIALLY DEFERRED;
The error was: relation "filer_image" does not exist
I'm not sure what the problem could be, as apparently there was no problem in creating the production database or subsequent migrations.
I believe I have correctly installed sorl-thumbnail, and all of its dependencies.
Your help would be much apperciated.

I'm pulling the answer provided by Johndt6 from a comment to this answer for future search-ability.
The solution is to add filer to the INSTALLED_APPS tuple.

Related

SQLAlchemy not updating changes causing issues

So I've made some changes to my schema on a Flask Server, using SQLite and SQLAlchemy. The database is generally functional and I'm able to add some of the models, as well as update and query all models without issues.
I've changed the id in two of my models from Integer to String in order to implement uuid ids, and since I've received IntegrityError for the mismatching parameter types when I do db.session.add(new_post) and db.session.commit().
If I do flask db migrate, it reports that no changes have been detected. Should I manually fill out a revision file or is there something else I am missing?
In addition to migrating your database, you'll need to upgrade:
flask db upgrade
If this results in an error, you might be encountering database specific constraints. For example, some primary keys cannot be removed without setting cascade rules on deletion for relationships.

OperationalError: cursor "_django_curs_<id>" does not exist

We have an online store web-app which is powered by django, postgresql and heroku.
For a specific campaign (you can think a campaign like a product to purchase), we have sold 10k+ copies successfully. Yet some of our users are encountered this error according to our Sentry reports. Common specification of these users is; none of them have address information before the purchase. Generally, users fill out address form right after registering. If they don't, they need to fill the form while purchasing the product and submit them together.
This is how the trace looks like:
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
(66 additional frame(s) were not displayed)
...
File "store/apps/store_main/templatetags/store_form_filters.py", line 31, in render_form
return render_to_string('widgets/store_form_renderer.html', ctx)
File "store/apps/store_main/templatetags/store_form_filters.py", line 20, in render_widget
return render_to_string('widgets/store_widget_renderer.html', ctx)
File "store/apps/store_main/widgets.py", line 40, in render
attrs=attrs) + "<span class='js-select-support select-arrow'></span><div class='js-select-support select-arrow-space'><b></b></div>"
OperationalError: cursor "_django_curs_140398688327424_146" does not exist
So another weird common thing, there are exception messages between sql queries before the failure. You can see it in the image below:
I'm adding it if they are somehow related. What may also be related is, the users who get this error are the users who tries to purchase the campaign right after a bulk mailing. So, extensive traffic might be the reason yet we are also not sure.
We asked Heroku about the problem since they are hosting the postgres, yet they do not have any clue either.
I know the formal reason of this error is trying to reach the cursor after a commit. Since it is destroyed after transaction, trying to reach it cause this error yet I don't see this in our scenario. We are not touching the cursor in any way. What am I missing? What may produce this error? How to prevent it? Any ideas would be appreciated.
The reason for your error might be that you added fields to a model and forgot to makemigrations and migrate.
Have a look at this answer: When in run test cases then I will get this error: psycopg2.OperationalError: cursor "_django_curs_140351416325888_23" does not exist
If you're using django-pytest and have enabled the optimization --reuse-db and have made DB migrations between test runs you need to re-create the DB tables again.
pytest --create-db
Most likely you have forgotten to makemigrations and migrate them to the database.
If you are sure you did make the migrations and running python manage.py makemigrations and python manage.py migrate will not find the changes you made then the database and models are not in sync.
Sometimes this situation can be very frustrating if you have a big database. You will need to manually inspect your models.
To help out you can try this trick, which has been working for me.
Step 1 (delete all migrations from apps)
I am going to assume you are using the unix terminal. run sudo rm -rv */migrations/*. This removes all the migrations files and caches.
Step 2 (make the migration folders in each app)
run the command mkdir <app-folder>/migrations && touch <app-folder>/__init__.py. Replace with the name of the app that you have in the INSTALLED_APPS list in your default django settings file.
Step 3 (Make Migrations)
Here we populate the migrations folders in each app with migration files. Run python manage.py makemigrations. Run the second command. python manage.py migrate --fake. We are using the --fake flag because the data already exists in the database and so we do not want to populate name database tables that are already there or you will be greeted with the already exists error
If this did not work, then you will need to temper with some additional fields on the database. such as the django_migrations or a similarly named table. This is not recommended as there are other database tables that depend on it such as the django_contenttypes and this will throw you in a chain of related tables that you will manually inspect which is very painful.

Django can't flush database during tests

When trying to run my tests (python manage.py test) I am getting:
CommandError: Database test_db couldn't be flushed. Possible reasons:
* The database isn't running or isn't configured correctly.
* At least one of the expected database tables doesn't exist.
* The SQL was invalid.
Hint: Look at the output of 'django-admin.py sqlflush'. That's the SQL this command wasn't able to run.
The full error: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "install_location_2015_05_13" references "app".
HINT: Truncate table "install_location_2015_05_13" at the same time, or use TRUNCATE ... CASCADE.
I am using partitions in our project which are generated on the fly via a python function (so I can run it periodically). I don't have any models for these partitions.
The partition maintenance function is invoked after syncdb triggers the post_syncdb signal (so it is executed when the test database is set up).
How can I make Django clear the additional tables (partitions)? or
How can I tell Django to use CASCADE while running the tests?
Mainly problem occur where we change relation in M2M field. Old constraint remain and new constraint for new relation was created. Fixed in Django 1.8

Django inspectdb on fails on table's foreign key which refs a table outside the current database (MySQL 5.0)

I tested with Python 2.6 or 2.7 with Django 1.5.1. My database is on MySQL 5.0. I've created the settings but now I can't run "inspectdb" on the database. I get
DatabaseError: (1146, "Table 'db1.tableName' doesn't exist")
This happens on a table which has a foreign key referencing a table in another DB. So it should not be db1 there since tableName lives in db2. I saw references to this bug from 5 years ago:
https://code.djangoproject.com/ticket/7556
But the patch is outdated by now and I figured it must have been finished in a later release. Is it something wrong with my setup?
Unfortunately, Django still does not currently does not support this feature.
Cross-database relations:
Django doesn’t currently provide any support for foreign key or
many-to-many relationships spanning multiple databases. If you have
used a router to partition models to different databases, any foreign
key and many-to-many relationships defined by those models must be
internal to a single database.
However, a fix could be found on this patch here
Basically, updating the router settings self.rel.to

How to prevent Django fixtures from conflicting with existing data

I'm trying to do manage.py loaddata myfixture.json to purposely bulk overwrite several records in my database. However, it gives me the error:
IntegrityError: Problem installing fixture 'myfixture.json': Could not load myapp.Person(pk=1): (1062, "Duplicate entry 'Bob' for key 'name'")
I thought the behavior of loaddata was to overwrite the records if primary key matches an existing record? Am I wrong, or was this changed in Django 1.5? How do I get it to overwrite?
I found a way to modify Django's loaddata command to properly parse natural keys while loading, and avoid re-loading duplicates. The modified command is published here. The only differences between the official loaddata are lines 189-201.

Categories

Resources