In one of my tables I have a field game_fen_when_leave = models.TextField(). But it gives me an error "You are trying to add a non-nullable field 'game_fen_when_leave' to game without a default; we can't do that (the database needs something to populate existing rows)". Is it necessary for this field to have a default value? I saw an example without having a default.
Short answer
When creating a new model: No it is not
When adding it to an existing model: Yes it is
A bit more on the topic:
With the information given I guess your are about to add this new field to an existing table.
When adding a new non-nullable fields to an existing model you will need to provide a default value. This is because there might already be rows in that particular table and those would need a default value to populate this new field with. (I'm actually just repeating the error message here.)
In the example that you are referring:
The model is new and there cannot be existing rows that would need to be populated with default values. Therefore default value for the TextField is not needed.
Couple of possibilities
Remove and create the model from scratch: If you remove the table by migrations and create it again as a completely new table. You don't have to provide a default value as there cannot be existing rows.
Add a default value: Default value could simply be an empty string and that probably is the way to go.
By default Django TextField is a non-nullable yes. You have the power to change that, but it is not advised to do so:
https://docs.djangoproject.com/en/3.0/ref/models/fields/#null
If a string-based field has null=True, that means it has two possible
values for “no data”: NULL, and the empty string.
Related
Note:
I understand and am well aware of the difference between passing a function as a parameter and invoking a function and passing the result as a parameter. I believe I am passing the function correctly.
Specs
Django 1.11
PostgreSQL 10.4
Scenario:
I have dozens of models in my application, with many existing records. I need to add a random seed to each of these models that will get created and set when a new model instance is created. I also want to generate the random seed for the existing instances.
My understanding of how Django model defaults and Migrations work is that when a new field is added to a model, if that field has a default value set, Djano will update all existing instances with the new field and corresponding default.
However, despite the fact that I'm definitely passing a function as the default, and the function produces a random number, Django is using the same random number when updating existing rows (e.g. it seems that Djano is only calling the function once, then using the return value for all entries).
Example
A shortened version of my code:
def get_random():
return str(random.random())
class Response(models.Model):
# various properties
random = models.CharField(max_length=40, default=get_random)
user = models.ForeignKey(User, on_delete=models.CASCADE)
content = JSONField(null=True)
The random field is being added after the model and many instances of it have already been created. A makemigrations command appears to generate the proper migration file, with a migrations.AddField() call, passing in default=get_random as a parameter.
However, after running makemigrations, all existing existing Response instances contain the exact same number in their random field. Creating and saving new instances of the model work as expected (with a pseudo-unique random number).
Workaround
An easy workaround is to just run a one-time script that does a
for r in Response.objects.all():
r.random = get_random()
r.save()
Or override the model's save() method and then do a mass save. But I don't think these workarounds should be necessary. It also means that if I want to make a unique field with a random default, then I will need multiple migrations. First I would have to add the field with the assigned default. Next I would need to apply the migration and manually re-initialize the field values. Then a second migration to add the unique=True property.
It seems that if Django is to apply default values to existing instances upon a makemigrations then it should apply them using the same semantics as creating a new instance. Is there any way to force Django to call the function for each model instance when migrating?
To add a non-null column to an existing table, Django needs to use an ALTER TABLE ADD COLUMN ... DEFAULT <default_value>. This only allows Django to call the default function once, that's why you see every row having the same value.
Your workaround is pretty much spot on, except that you can populate the existing rows with unique values using a data migration, so that it doesn't require any manual steps. The entire procedure for this use-case is described in the docs: https://docs.djangoproject.com/en/2.1/howto/writing-migrations/#migrations-that-add-unique-fields
Note:
I understand and am well aware of the difference between passing a function as a parameter and invoking a function and passing the result as a parameter. I believe I am passing the function correctly.
Specs
Django 1.11
PostgreSQL 10.4
Scenario:
I have dozens of models in my application, with many existing records. I need to add a random seed to each of these models that will get created and set when a new model instance is created. I also want to generate the random seed for the existing instances.
My understanding of how Django model defaults and Migrations work is that when a new field is added to a model, if that field has a default value set, Djano will update all existing instances with the new field and corresponding default.
However, despite the fact that I'm definitely passing a function as the default, and the function produces a random number, Django is using the same random number when updating existing rows (e.g. it seems that Djano is only calling the function once, then using the return value for all entries).
Example
A shortened version of my code:
def get_random():
return str(random.random())
class Response(models.Model):
# various properties
random = models.CharField(max_length=40, default=get_random)
user = models.ForeignKey(User, on_delete=models.CASCADE)
content = JSONField(null=True)
The random field is being added after the model and many instances of it have already been created. A makemigrations command appears to generate the proper migration file, with a migrations.AddField() call, passing in default=get_random as a parameter.
However, after running makemigrations, all existing existing Response instances contain the exact same number in their random field. Creating and saving new instances of the model work as expected (with a pseudo-unique random number).
Workaround
An easy workaround is to just run a one-time script that does a
for r in Response.objects.all():
r.random = get_random()
r.save()
Or override the model's save() method and then do a mass save. But I don't think these workarounds should be necessary. It also means that if I want to make a unique field with a random default, then I will need multiple migrations. First I would have to add the field with the assigned default. Next I would need to apply the migration and manually re-initialize the field values. Then a second migration to add the unique=True property.
It seems that if Django is to apply default values to existing instances upon a makemigrations then it should apply them using the same semantics as creating a new instance. Is there any way to force Django to call the function for each model instance when migrating?
To add a non-null column to an existing table, Django needs to use an ALTER TABLE ADD COLUMN ... DEFAULT <default_value>. This only allows Django to call the default function once, that's why you see every row having the same value.
Your workaround is pretty much spot on, except that you can populate the existing rows with unique values using a data migration, so that it doesn't require any manual steps. The entire procedure for this use-case is described in the docs: https://docs.djangoproject.com/en/2.1/howto/writing-migrations/#migrations-that-add-unique-fields
I am using a custom primary key for a model that has a few ManyToManyFields. When I update the model and add an object to a ManyToManyField (using add(new_object)), I get an error signifying that its looking up the primary key using the id field (which perhaps exists in the intermediary table, but not in the model).
psycopg2.DataError: invalid input syntax for integer: "TL98GK"
LINE 1: ...WHERE ("placedir_place_place_categ"."place_id" = 'TL98GK' A...
I have been searching on SO for a while but havent been able to zero in the exact issue. I guess I may have to use custom through table for ManytoManyFields (as a punishment for using custom primary key) but I honestly dont want to go down that route.
Using Django 1.10 and Python 3
It seems the migration doesn't detect Foreign Key type changes well. It's a known bug. A workaround is mentioned here (i.e to explicitly change the field type to varchar(32)).
However, you may need to do more (like updating the constraints etc. on the table) depending on your use case. (For those interested to go that route, here is one example case with corresponding migration code).
(p.s I just decided to not use a custom primary key on models with manytomany fields)
I am new to Django.
I was trying to design a model in django. First I did with adding some fields, then I migrated the code. Later I found that I should have some other fields. I added some new fields, let say a CharField. Then while I was doing the migration, its showing error like you are trying to add a non-nullable field without a default. Can anybody tell should I add every time a default value to a new field OR Is there any other way to handle this?
You would need to add null=True to the field parameter that gives you this error in your model to allow for null values or give it a default value default=<value> and then re-run your migration
So I am trying to add a point field to a model in django which looks like this:
class Location(models.Model):
lat = models.FloatField(blank=True, null=True)
lng = models.FloatField(blank=True,null=True)
geometry = models.PointField(srid=4326,default='POINT(0.0,0.0)')
objects = models.GeoManager()
I already have database with lots of locations with the lat lng coordinates but they don't need a point field. I am getting this error django.db.utils.IntegrityError: column "geometry" contains null values
I set the default but it still gives me that error when I run 'python manage.py migrate' How can I fix this?
The default does not change old rows in the database (that existed before the migration). You should set the the geometry field to be nullable and then migrate. Once migrated you can either fill in the null values with the default and remove null=True or leave as is
UPDATE
Based on this post it seems very possible that PointField cannot be null (even if it's specified to be which really grinds my gears...). GIVEN that your model does not need to change but it doesn't look like django can handle this super well for you.
I see two options (better ones might exist)
Do the data migration on the actual table yourself (I THINK with most databases when you add in a new column you can set a default there and it will populate that for you)
Create a new model which is essentially the same with the new columns, copy the data over, delete the old model and columns.
Neither of these seem ideal but the googles have not yet shown me a better way