I've got a Django model like the following..
class ExampleModel(models.Model):
name = models.CharField(...)
related_user = models.ForeignKey(UserTypeA, related_name='related_example', blank=True, null=True, on_delete=models.SET_NULL)
where I recently had to make a change to the related_user field by changing the ForeignKey from UserTypeA to UserTypeB.
Of course, this raises an error in the terminal when I attempt to python manage.py makemigration...
django.db.utils.IntegrityError: insert or update on table "models_examplemodel" violates foreign key constraint "models_examplemodel_related_user_id_ac0c6018_fk_accounts_"
DETAIL: Key (related_user_id)=(13) is not present in table "accounts_usertypea".
What's the safest way to go about making these changes? Currently I'm in development so I'm happy to delete my data/migrations/whatever, but I imagine in production this would be difficult.
The ideal behaviour I'd like to see here is the relations from ExampleModel and UserTypeA just being deleted, and so the current relationships would be set to NULL. Thoughts?
if you simply want to drop UserTypeA and use UserTypeB with None values simply do this:
remove related_user field
generate migrations
add related_user field
generate migrtions
If you want to do something more complecated (fill UserTypeB based on UserTypeA) these are the steps
add realted_user_b field with default as null
generate migration file
write a data migrations file which fills realted_user_b based on current data docs
remove realted_user field
generate migration file
rename realted_user_b to realted_user
generate migration file
Related
I am new to django.
I have changed some fields in my already created Django model. But It says this message when I try to apply migrations on it:
It is impossible to add a non-nullable field 'name' to table_name without specifying a default. This is because the database needs something to populate existing rows.
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
2) Quit and manually define a default value in models.py.
Although I have deleted the data of this table from database. I cannot set it's default value because the field has to store unique values. Do I need to delete my previous migration file related to that table?
I have applied data migrations, but still getting the same error when applying migrations again:
def add_name_and_teacher(apps, schema_editor):
Student = apps.get_model('app_name', 'Student')
Teacher = apps.get_model('app_name', 'Teacher')
for student in Student.objects.all():
student.name = 'name'
student.teacher = Teacher.objects.get(id=1)
student.save()
class Migration(migrations.Migration):
dependencies = [
('app', '0045_standup_standupupdate'),
]
operations = [
migrations.RunPython(add_name_and_teacher),
]
So, before you had a nullable field "name". This means that it's possible to have null set as that field's value.
If you add a not null constraint to that field (null=False), and you run the migrations, you will get an integrity error from the database because there are rows in that table that have null set as that field's value.
In case you just made two migrations where first you added a nullable field, but then remembered it mustn't be nullable and you added the not null constraint, you should simply revert your migrations and delete the previous migration. It's the cleanest solution.
You can revert by running python manage.py migrate <app name> <the migration that you want to keep>
Then you simply delete the new migrations and run python manage.py makemigrations again.
In case the migration with the nullable field was defined very early on and there is already data there and it's impossible to delete that migration, you will need to figure out how to populate that data. Since you say that there is also the unique constraint, you can't just provide a default because it will cause issues with that constraint.
My suggestion is to edit the migration file and add migrations.RunSQL where you write custom SQL code which will insert values to the field. Make sure you place the RunSQL operation before the operation that adds the not null constraint (it should be AlterField or AddConstraint) as they are run in order.
You could also use migrations.RunPython, but I prefer the RunSQL because future changes in the code might break your migrations which is a hassle to deal with.
Docs for RunSQL
I have two Django (foreign key) fields - FieldA and FieldB - referring to the same class but being different objects. Then I wanted to rename FieldB to FieldC. I renamed this in one migration (automatic detection).
Then I realised I actually need a new field with the same name as the old FieldB (also foreign key to the same class). Therefore I created a second migration to add a new field: FieldB. Since I just renamed the other one I assumed that this would not give any issues in the DB.
Locally I develop on an SQLite DB and this works fine. When I pushed it to our Postgres DB, this returned an error.
Model class
class ModelClass(Model):
field_a: ForeignClassA = models.ForeignKey(ForeignClassA, on_delete=models.SET_NULL, blank=True, null=True, related_name='FieldA')
# this one should be renamed to field_c after which I create another field with the same name and properties.
field_b: ForeignClassA = models.ForeignKey(ForeignClassA, on_delete=models.SET_NULL, blank=True, null=True, related_name='FieldB')
Migration one: rename
operations = [
migrations.RenameField(
model_name='modelname',
old_name='FieldB',
new_name='FieldC',
),]
Migration two: add field
operations = [
migrations.AddField(
model_name='modelname',
name='FieldB',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='FieldB', to='app.othermodel'),
),]
When I run this migration I get the following error
Applying app.xxx1_previous... OK
Applying app.xxx2_rename_field... OK
Applying app.xxx3_add_field...Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.DuplicateTable: relation "app_modelname_fieldB_id_8c448c6a" already exists
Normally this column should have been deleted and replaced with a new column with the new name. I found this issue describing a similar issue in Postgress and now I wonder if this is a bug in Django. Could it be that the cleanup from the rename is not done correctly?
EDIT 1
After a closer inspection of the Postgres DB I can see that after the rename I still have two columns in my table: FieldA_id and FieldB_id, while I would expect to have FieldA_id and FieldC_id. Obviously this creates an issue if I subsequently try to add FieldB again.
Could it be that Postgres (or the Django controller) does not rename this column for some reason?
EDIT 2
I inspected the SQL query to the Postgres DB. The following SQL is produced:
BEGIN;
--
-- Rename field field_b on modelclass to field_c
--
SET CONSTRAINTS "app_modelclass_field_b_id_8c448c6a_fk_otherapp_otherclass_id" IMMEDIATE; ALTER TABLE "app_modelclass" DROP CONSTRAINT "app_modelclass_field_b_id_8c448c6a_fk_otherapp_otherclass_id";
ALTER TABLE "app_modelclass" RENAME COLUMN "field_b_id" TO "field_c_id";
ALTER TABLE "app_modelclass" ADD CONSTRAINT "app_modelclass_field_c_id_9f82ac2c_fk_otherapp_otherclass_id" FOREIGN KEY ("field_c_id") REFERENCES "otherapp_otherclass" ("id") DEFERRABLE INITIALLY DEFERRED;
COMMIT;
This rename however is seems to be only executed partially since the next step reports an issue, but the migration succeeds.
However, when I manually run this SQL query, the command succeeds and the upgrade is partly successful. In summary:
The SQL query works
While running the SQL, the column name is updated
While running the SQL, the constraint is updated
Still the next step complains about a reference to this rename. Still clueless at the moment what this might be.
After a long search down the SQL rabbit hole, I found out that the rename migration for PostgresQL does not drop the old index.
When I wanted to create a new field, it tried to create a new index with the same name as the old index (which wasn't removed).
A simple way to avoid it is to 'sandwich' the rename with two alter operations. One to unset the index and one to set it back afterwards.
operations = [
migrations.AlterField(
model_name='modelclass',
name='field_b',
field=models.ForeignKey(blank=True, db_index=False, null=True, on_delete=django.db.models.deletion.SET_NULL,
to='otherapp.otherclass'),
),
migrations.RenameField(
model_name='modelclass',
old_name='field_b',
new_name='field_c',
),
migrations.AlterField(
model_name='modelclass',
name='field_c',
field=models.ForeignKey(blank=True, db_index=True, null=True, on_delete=django.db.models.deletion.SET_NULL,
to='otherapp.otherclass'),
),
]
I reported this issue also on the Django bug tracker after which they closed it since there exists a duplicate from 7 years ago.
I need to delete fields from an existing django model that already have a few objects associated with it. Deleting the fields from models.py gives me an error (obviously as there are table columns still associated with them). The data in body2 and body3 are not necessary for my app.
I have copied that data from those fields to the body field. How would I go about deleting these fields without dropping the table entirely?
class Post(models.Model):
#some fields
body =EditorJsField(editorjs_config=editorjs_config)
body2 =EditorJsField(editorjs_config=editorjs_config)
body3 =EditorJsField(editorjs_config=editorjs_config)
I deleted body2 and body3 and ran migrations and when creating a new object, I get errors such as this.
django.db.utils.IntegrityError: null value in column "body2" of relation "second_posts" violates not-null constraint
DETAIL: Failing row contains (20, Wave | Deceptiveness, and unpredictability of nature, 2021-07-19 13:40:32.274815+00, 2021-07-19 13:40:32.274815+00, {"time":1626702023175,"blocks":[{"type":"paragraph","data":{"tex..., null, null, Just how unpredictable is nature? Nature is all around us, yet, ..., image/upload/v1626702035/dfaormaooiaa8felspqd.jpg, wave--deceptiveness-and-unpredictability-of-nature, #66c77c, l, 1, 1, 0).
This is the code that I'm using to save the sanitized data(after I've deleted those fields of course.)
post = Posts.objects.create(
body=form.cleaned_data.get('body'),
#
)
Since nobody seemed to have an answer, and since it looked like this error was an anomaly, I went the non-python way and ran SQL queries and dropped the columns. For those of you who ran into the same problem,
Warning, you will lose all the data in the fields you would like to delete using this method
First, make Django aware of the changes
Delete the fields you want to be deleted and run migrations.
Before
class Post(models.Model):
#some fields
body =EditorJsField(editorjs_config=editorjs_config)
body2 =EditorJsField(editorjs_config=editorjs_config)
body3 =EditorJsField(editorjs_config=editorjs_config)
After
class Post(models.Model):
#some fields
body =EditorJsField(editorjs_config=editorjs_config)
Command Prompt
python manage.py makemigrations
python manage.py migrate
Drop the columns using SQL queries
First connect to your database(I used postgres). The name of the table should be something like appname_model. My app was called "Second" and the model was Post. So, the table was called second_post.
See if the columns still persist after the migrations using,
In the SQL command prompt
/d second_post
This should give you a nice diagram of the database with all the columns listed on the left side. To drop those columns, type,
ALTER TABLE second_post DROP COLUMN body2;
ALTER TABLE second_post DROP COLUMN body3;
After entering each query, the prompt should return a string ALTER TABLE if successful.
If you want to drop the data completely, you need to create a Django migration (using ./manage.py makemigrations preferably) that will remove those columns from the database.
Alternatively, if you want to play safe and persist the old data, you can first make those fields as nullable, then create migrations for them and at the end either just don't use them anymore or remove those columns from the model, but don't reflect it in migrations (you'll need to fake the removal of those columns in migrations though if you'll ever need to run another migration in this app).
In a Django project with postgresql, I once inserted en entry in the db, and one day. Someone else who has access to it, has manually added 36 other rows in the db with pgadmin4. When I want to add new row with the Django project, I got IntegrityError obviously, because Django tries to add 1 to the last entry id added by the Django project (This is what I think it is trying to do). Here is the traceback:
duplicate key value violates unique constraint "register_catalog_pkey"
DETAIL: Key (id)=(2) already exists.
How to tell Django that the last entry id is 36 (the last value manually added with pgadmin4)?
I also tried to have the id field in Django Admin so I could edit it, but it did not show up.
class RegisterAdmin(admin.ModelAdmin)
list_display_links = ['id',]
list_display = ('id',)
Do I need to remove the primary key handled by Django, and define an IntegerField as primary key?
register_id = models.AutoField(primary_key=True)
to
register_id = models.IntegerField(primary_key=True)
Use ALTER SEQUENCE command
ALTER SEQUENCE <table_name>_id_seq RESTART WITH 37
Ex: ALTER SEQUENCE myapp_register_id_seq RESTART WITH 37
Say, I have a model with this field:
is_retired = models.NullBooleanField()
It has never been set for any of the records i the database, so I think all the records will have a null value for this field.
And now want to change it to a string field:
is_retired = models.CharField(max_length=50)
I am using Django 1.7 and ran makemigrations which seemed to go fine, said something about altered field. However, when I actually run the migrate command it fails with
django.db.utils.IntegrityError: column "is_retired" contains null values
How can I resolve this?
If you want to enable null/empty values, change it to:
is_retired = models.CharField(max_length=50, null=True, blank=True)
You might also want to change the null values to empty strings ('') in another migration.