This is from django's documentation:
Field.unique
If True, this field must be unique throughout the table.
This is enforced at the database level and by model validation.
If you try to save a model with a duplicate value in a unique field, a django
.db.IntegrityError will be raised by the model’s save() method.
Here is my models.py
class MyModel(models.Model):
# my pk is an auto-incrementing field
url = models.URLField("URL", unique=True)
text = models.TextField(max_length=1000)
# my model is just two fields, one pk (unique), and another unique field,
#, the url
Here my is manage.py sqlall (I ran syncdb)
CREATE TABLE `MyModel_mymodel` (
`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`url` varchar(200) NOT NULL UNIQUE,
`text` varchar(1000) NOT NULL,
However, in the manage.py shell, I can freely do this:
>>> from MyModel.models import MyModel
>>> MyModel().save() # it works fine!? Not even the text was checked for!
>>> MyModel(url="blah").save()
>>> MyModel(url="blah").save() # it still works!
# I checked the mysql database afterwards, the models were saved just fine, they
# however did have different PK's (auto incrementing fields).
I'm using mysql, django 1.5. Does anyone have an idea what could possible be causing this?
I am using a custom manager, but I doubt that's the issue.
Thanks.
For django 1.9+
Running makemigrations then migrate applies the unique constraint to sqlite3
For django < 1.9
Since you are using django 1.5, this solution will apply.
If you added the unique=True after the table was already created, then even if you do syncdb later, the unique condition will not be added to your table.
I can confirm with sqlite3 that Django 1.5 happily saves duplicate objects with MyModel(url="blah").save() if the unique constraint does not exist in the database, which seems to contradict with the docs.
The best solution for you is to create the constraint manually in your database using this command.
ALTER TABLE MyModel_mymodel ADD UNIQUE (url);
Or if you don't mind, you can recreate your table. (Drop the table and then run syncdb.)
Running sql scripts directly on the db can be avoided. Rather add the sql execution in your migration:
from __future__ import unicode_literals
from django.db import migrations, connection
def alter_table(apps, schema_editor):
query ="ALTER TABLE <your table> ADD UNIQUE (unique_col);"
cursor = connection.cursor()
cursor.execute(query)
cursor.close()
class Migration(migrations.Migration):
dependencies = [
('app', 'yourlastmigration'),
]
operations = [
migrations.RunPython(alter_table),
]
The solution is pretty simple, Just follow their steps.
1 - Dell all the files in the migration folder
2 - Then run the command "python manage.py makemigrations"
3 - Then run the command "python manage.py migrate"
OR
Do it by the help of a simple SQL-lite Query Adding index Example
alter table test add index index_name(col1(255),col2(255));
Adding unique index Example
alter table test add unique index_name(col1(255),col2(255));
Related
So I'm trying to add a SearchVectorField to a model in Django:
class JobPosting(models.Model):
...
...
search_vector = SearchVectorField()
I get that it should either be nullable or have a default value to be able to migrate so I deleted all entries in the table to prevent this problem.
However, I'm getting the following error when running makemigrations:
You are trying to add a non-`nullable` field 'search_vector' to jobposting without a default;
we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now
(will be set on all existing rows with a null value for this column)
2) Quit, and let me add a default in models.py
Select an option:
Why is it saying this if the table is empty? I don't want to make the column nullable, and I'd rather not have a default value if I can avoid it.
My question is, is there any way to force makemigrations and migrate as I don't understand the problem if the table is empty. I have other tables with data in them which I don't want to delete so can't delete all info in the database.
Alternatively, if option 1) is the solution, how would I format a default value for this type of field? I assume it's not a normal text field?
Thanks for any help.
I am not entirely sure why you do not want to have a default value, but I will assume this as given.
My question is, is there any way to force makemigrations and migrate
as I don't understand the problem if the table is empty.
Your current database table might be empty, but migrations are supposed to be repeatable on other database instances. Therefore Django cannot assume that the same holds on any other database.
A workaround might be to do define a migration that creates the field as nullable, indexes all entries and then updates it to be non-nullable.
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.contrib.postgres.search import SearchVector, SearchVectorField
from django.db import migrations
def index_entries(apps, schema_editor):
Entry = apps.get_model("mymodel", "Entry")
Entry.objects.update(search_vector=SearchVector('body_text'))
class Migration(migrations.Migration):
dependencies = [
('mymodel', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='entry',
name='search_vector',
field=SearchVectorField(null=True),
),
migrations.RunPython(index_entries),
migrations.AlterField(
model_name='entry',
name='search_vector',
field=SearchVectorField(null=False),
),
]
I'd just make the field nullable (and probably not editable, since you're not going to change it in admin interface nor via forms):
class JobPosting(models.Model):
...
...
search_vector = SearchVectorField(null=True, editable=False)
And there will be no issues with migration.
Later you can make this field not nullable, but there is no real reason to do this because you will update it programmatically anyway.
Here's simplified task and setup (Django 1.8, MySQL, Python 2.7), I've got:
class Car(models.Model):
make = models.ForeignKey(CarMake)
class Bike(models.Model):
make = models.ForeignKey(BikeMake)
class CarMake(models.Model):
name = models.CharField(max_length=32)
class BikeMake(models.Model):
name = models.CharField(max_length=32)
Now, I need to ditch the BikeMake model completely so I update the CarMake model with values from BikeMake and also update the foreign key relationship in Bike.
I've created the following migration, which updates the CarMake with names from BikeMake, adds temporary field Bike.car_make, migrates data from Bike.make to Bike.car_make, removes the Bike.make field and renames Bike.car_make to Bike.make.
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def update_car_makes(apps, schema_editor):
"""Update CarMakes with BikeMakes"""
BikeMake = apps.get_model('my_app', 'BikeMake')
CarMake = apps.get_model('my_app', 'CarMake')
for item in BikeMake.objects.all():
if not CarMake.objects.filter(name=item.name).exists():
CarMake.objects.create(name=item.name)
def remove_car_makers(apps, schema_editor):
"""Restore original CarMake (exclude BikeMake)"""
pass
def migrate_to_car_make(apps, schema_editor):
"""Set Bike.car_make according to Bike.make"""
CarMake = apps.get_model('my_app', 'CarMake')
Bike = apps.get_model('my_app', 'Bike')
for item in Bike.objects.all():
old_make = item.make
new_make = CarMake.objects.get(name=old_make.name)
item.car_make = new_make
item.save()
def reverse_migrate_to_car_make(apps, schema_editor):
pass
def dummy_forwards(apps, schema_editor):
# Empty forward migration needed for having custom backwards migration
pass
def restore_make_column_data(apps, schema_editor):
BikeMake = apps.get_model('products', 'BikeMake')
Bike = apps.get_model('products', 'Bike')
for item in Bike.objects.all():
old_make = item.bike_make
new_make = BikeMake.objects.get(name=old_make.name)
item.make = new_make
item.save()
class Migration(migrations.Migration):
dependencies = [('my_app', '0001_blah_blah')]
operations = [
migrations.RunPython(
update_car_makers,
reverse_code=remove_car_makers
),
migrations.AddField(
model_name='bike',
name='car_make',
field=models.ForeignKey(default=1, to='my_app.CarMake'),
preserve_default=False
),
migrations.RunPython(
migrate_to_car_make,
reverse_code=reverse_migrate_to_car_make
),
migrations.RunPython(
dummy_forwards,
reverse_code=restore_make_column_data
),
migrations.RemoveField(
model_name='bike',
name='make',
),
migrations.RenameField(
model_name='bike',
old_name='car_make',
new_name='make'
)
]
And when I try to run it, I get the #1072 error on running the last operation: migrations.RenameField. Now the interesting part is that from DB POV everything is complete, data migrated, column renamed, only the migration isn't marked as done and error is thrown.
Also if I just move the migrations.RenameField to a separate migration file and run two migrations in a row – everything works fine and it doesn't raise the #1072 error.
In addition, I've tried inserting a breakpoint just before the migrations.RenameField and I verified that Bike.car_make column exists and I can fetch normally all objects of Bike model at that point.
The MySQL query, that results in error is following:
CREATE INDEX `my_app_bike_c2036163` ON `my_app_bike` (`car_make_id`)
Any ideas how to fix it and have it within one migration file? Thanks in advance!
UPDATE 04.02.16
As #kvikshaug pointed out, this happens because Django creates indexes and constraints after performing all operations i.e. raw SQL for creating index and/or constraints is generated at the time, the corresponding operation is executed (in my case AddField), but that query is actually run at the very end, thus the error.
One possible solution for relatively small schemas could be to use Django's RunSQL and type the raw queries yourself, but that's quite cumbersome + you've got to create constraints yourself.
So I went with separating the renaming migration.
Django migrations create indexes after performing all operations. Your second operation, adding field car_make, makes Django add the CREATE INDEX command which you noted causes the error:
CREATE INDEX `my_app_bike_c2036163` ON `my_app_bike` (`car_make_id`)
Even though you later renamed the field, Django still tries to create the index for the now-missing car_make field, that's why you get the error. You can see this clearly by running sqlmigrate:
$ ./manage.py sqlmigrate my_app 0002_blah_blah
BEGIN;
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
--
-- Add field car_make to bike
--
ALTER TABLE "my_app_bike" ADD COLUMN "car_make_id" integer DEFAULT 1 NOT NULL;
ALTER TABLE "my_app_bike" ALTER COLUMN "car_make_id" DROP DEFAULT;
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
--
-- Remove field make from bike
--
ALTER TABLE "my_app_bike" DROP CONSTRAINT "my_app_bike_make_id_5615ed11_fk_my_app_bikemake_id";
ALTER TABLE "my_app_bike" DROP COLUMN "make_id" CASCADE;
--
-- Rename field car_make on bike to make
--
ALTER TABLE "my_app_bike" RENAME COLUMN "car_make_id" TO "make_id";
CREATE INDEX "my_app_bike_78e8ca60" ON "my_app_bike" ("car_make_id");
ALTER TABLE "my_app_bike" ADD CONSTRAINT "my_app_bike_car_make_id_6c42be09_fk_my_app_carmake_id" FOREIGN KEY ("car_make_id") REFERENCES "my_app_carmake" ("id") DEFERRABLE INITIALLY DEFERRED;
COMMIT;
You could try to report this as a bug (or search; maybe it's already reported), but you're probably best off following Alasdairs suggestion and just separate the migrations.
I added these fields to my model:
class WatchList(models.Model):
name = models.CharField(max_length=20)
class Thing(models.Model):
watchlist = models.ForeignKey(WatchList)
Ran the schemamigration successfully:
>>> $ python2.7 manage.py schemamigration myapp --auto
+ Added model myapp.WatchList
? The field 'Thing.watchlist' does not have a default specified, yet is NOT NULL.
? Since you are adding this field, you MUST specify a default
? value to use for existing rows. Would you like to:
? 1. Quit now, and add a default to the field in models.py
? 2. Specify a one-off value to use for existing columns now
? Please select a choice: 2
? Please enter Python code for your one-off default value.
? The datetime module is available, so you can do e.g. datetime.date.today()
>>> 0
+ Added field watchlist on myapp.Thing
Created 0004_auto__add_watchlist__add_field_thing_watchlist.py. You can now apply this migration with: ./manage.py migrate myapp
I haven't had a problem before trying to do this, but for some reason I received the following error:
>>> $ python2.7 manage.py migrate myapp
FATAL ERROR - The following SQL query failed: ALTER TABLE "myapp_thing" ADD CONSTRAINT "watchlist_id_refs_id_1b2eef756112b8e" FOREIGN KEY ("watchlist_id") REFERENCES "myapp_watchlist" ("id") DEFERRABLE INITIALLY DEFERRED;
The error was: insert or update on table "myapp_thing" violates foreign key constraint "watchlist_id_refs_id_1b2eef756112b8e"
DETAIL: Key (watchlist_id)=(0) is not present in table "myapp_watchlist".
How can I successfully migrate the changes to the models? Thanks for any ideas that might help!
It's because you specified 0 as the default value in the migration. Delete the migration and run it again, this time specify an empty string as the default.
If you already have some data in the database you'll need to specify null=True and blank=True in the ForeignKey or it'll break.
tl;dr: We want to add a new table to the database, without losing previous data.
First we took a dump of the database using:
$ psql -h localhost dbname > dump_file
Then we loaded it to our local db. This is working fine.
Problem starts:
On adding a new model to models.py, syncdb doesn't create the table. However, if we create a new db (by deleting the data and running syncdb), it shows up as a created table.
This is surprising, because we don't think that adding a new table is a 'schema migration' that syncdb cannot handle. Hence, we don't think South will help here.
Is there something I'm missing/getting wrong?
This is what our code looks like:
class Person(models.Model):
# attributes
class A(models.Model):
# attributes
class B(models.Model):
a_fk = models.ForeignKey(A)
# more attributes
class ProblemModel(models.Model)
p_fk = models.ForeignKey(Person)
a_fk = models.ForeignKey(A)
b_fk = models.ForeignKey(B)
The output for the concerned table using the command:
$ ./manage.py sqlall appname
is:
CREATE TABLE "appname_problemmodel" (
"id" serial NOT NULL PRIMARY KEY,
"p_fk" integer NOT NULL REFERENCES "appname_person" ("p_fk") DEFERRABLE INITIALLY DEFERRED,
"a_fk" integer NOT NULL REFERENCES "appname_a" ("id") DEFERRABLE INITIALLY DEFERRED,
"b_fk" integer NOT NULL REFERENCES "appname_b" ("id") DEFERRABLE INITIALLY DEFERRED
)
Django version 1.5.1
ProblemModel is the new model we're trying to add.
Thanks in advance for the help!
First of all Your data restore command have wrong input symbol. It should be :-
$ psql -h localhost dbname < dump_file
[django-admin-flush] (https://docs.djangoproject.com/en/1.5/ref/django-admin/#=django-admin-flush) has an example of getting database to previous state with options - * --no-initial-data*.
It is best to use south with following command execution -
('python manage.py migrate --no-initial-data')
('python manage.py syncdb')
To restore data to database you can use dump with admin.py or directly use your
database command.
Hence, we don't think South will help here.
Yes it will, just use it - the official documentation should get you started.
No idea why this error is popping up. Here are the models I created -
from django.db import models
from django.contrib.auth.models import User
class Shows(models.Model):
showid= models.CharField(max_length=10, unique=True, db_index=True)
name = models.CharField(max_length=256, db_index=True)
aka = models.CharField(max_length=256, db_index=True)
score = models.FloatField()
class UserShow(models.Model):
user = models.ForeignKey(User)
show = models.ForeignKey(Shows)
Here is the view from which I access these models -
from django.http import HttpResponse
from django.template import Context
from django.template.loader import get_template
from django.http import HttpResponse, Http404
from django.contrib.auth.models import User
def user_page(request, username):
try:
user = User.objects.get(username=username)
except:
raise Http404('Requested user not found.')
shows = user.usershow_set.all()
template = get_template('user_page.html')
variables = Context({
'username': username,
'shows' : shows})
output = template.render(variables)
return HttpResponse(output)
At this point I get an error -
OperationalError: (1054, "Unknown column 'appname_usershow.show_id' in 'field list'")
As you see this column is not even present in my models? Why this error?
maybe your tables schema has been changed? Also, running syncdb does not update already created tables.
You might need to drop all the tables & then run syncdb again. Also remember to take backup of your data!!
As #inception said my tables schema had changed & running syncdb did not update already created tables.
Apparently any changes to the models when updated through syncdb does not change (as in update/modify) the actual tables. So I dropped the relevant DB & ran syncdb on empty DB. Now it works fine. :)
For others, SOUTH data migration tool for Django seems to be favorite option. It seems to provide options which django models & syncdb falls short on. Must check out...
Update 29th Sept 2019: From Django 1.7 upwards, migrations are built into the core of Django. If you are running a previous lower version of Django, you can find the repository on BitBucket.
Normally I get this when when I'm trying to access field which doesn't exist in Database.
Check if the field exist in the database. If you change model and perform syncdb it won't update the database, I'm not sure if that's the case.
On other note Django offers shortcut to replace try/except block in your code using get_object_or_404. (available in django.shortcuts )
try:
user = User.objects.get(username=username)
except:
raise Http404('Requested user not found.')
can be changed to:
user = get_object_or_404(User, username=username)
I have met the same problems:
First, run
manage.py sqlall [appname]
and you can find:
`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
and I add the column manual:
ALTER TABLE tb_realtime_data ADD id integer AUTO_INCREMENT NOT NULL PRIMARY KEY FIRST;
and then it worked.
I think django will add the column called id itself.
For convenience, each model is given an autoincrementing primary key field named id unless you explicitly specify primary_key=True on a field (see the section titled “AutoField” in Appendix A).
you can click here for details.
Good luck.
I faced the same error like you posted above with MySQL database back-end, after long time of resolving this error, I ended up with below solution.
Manually added column to database through workbench with name exactly the same as it is shown in your error message.
After that run below command
python manage.py makemigrations
Now migrations will be successfully created
After that run below command
python manage.py migrate --fake
Now every thing works fine without any data lose issue
In the appropriate field explicitly set
primary_key = True
This is quite an old question but you might find the below useful anyway:
Check the database, you're most likely missing the show_id in the appname_usershow table.
This could occur when you change or add a field but you forget running migration.
OR
When you're accessing a non managed table and you're trying to add a ForeignKey to your model.
Following the example above:
class UserShow(models.Model):
# some attributes ...
show = models.ForeignKey(Show, models.DO_NOTHING)
class Meta:
managed = False
db_table = 'appname_departments'
This is a late response but hopefully someone will find this useful.
I was working on Django REST APIs using DB models. When I added a new column to my existing model(said column already existed in the db table from the start), I received the error shown below:
"Unknown column ',column name>' in 'field list'"executed".
What I missed was migrating the model over to the database.
So I executed the following commands from python terminal:
py -3 manage.py makemigrations ---> It will not allow NULL values for the new column added to the model, even though the column is present in the database from the start. So you need to add a default value, mine was an Integerfield, so I updated my column definition in the model as IntegerField(default=0).
From here onwards it should be straightforward, if there are no other errors.
py -3 manage.py migrate
py -3 manage.py runserver
After executing these steps when I called my REST APIs they were working properly.
I created a model file for my app and then did several sqlall as I refined my tables. One of the changes I made was I set primary_key=True to one of my fields. At the end called for syncdb. Added a dummy value and and tried to access it with User.objects.all(), User being my model class. Although this worked fine, this error came up while printing the list returned by the all() call. It read DatabaseError: (1054, "Unknown column 'polls_user.id' in 'field list'")
Surprisingly, I tried and could get it resolved with another call to syncdb. I remember not having seen the id column in the table at any time throughout the exercise when I checked it through the mysql command line client.
I received this error while trying to use Django Rest Framework serializers. Make sure if you have a serializer that subclasses ModelSerializer, that you migrate any changes to the Models before writing your serializer classes (or just comment anything related to the serializer, migrate, then uncomment).
PS F:\WebApp> python manage.py makemigrations
You are trying to add a non-nullable field 'price' to destination without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
2) Quit, and let me add a default in models.py
Select an option: 2
PS F:\WebApp> python manage.py sqlmigrate travello 0001
BEGIN;
-- Create model Destination
CREATE TABLE travello_destination (id integer AUTO_INCREMENT NOT NULL PRIMARY KEY, name varchar(100) NOT NULL, img varchar(100) NOT NULL, desc longtext NOT NULL, offer bool NOT NULL);
COMMIT;
PS F:\WebApp> python manage.py makemigrations
Migrations for 'travello':
travello\migrations\0002_destination_price.py
- Add field price to destination
PS F:\WebApp> python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, travello
Running migrations:
Applying travello.0002_destination_price... OK
I had this issue when using a composite Primary Key of several VarChar fields and trying to call a table_set.all() queryset.
Django wanted a table_name_id PK column for this queryset, there wasn't one so it threw out this error.
I fixed it by manually creating the table_name_id and setting it to an auto-incremented, integer PK column instead of the composite PK.
I then specified those VarChar composite columns as unique_together in the Model's meta section so they act like a PK.
The direct solution is delete files under folder ../Project/App/migrations, and the method to avoid this problem is create new databaes table column instead of chagne the existing one.