Develop Reference

Django creates a migration that seems already reflected in original postgresql schema

I've modified the foreign key calendar as nullable in my Django model CalendarAssign. \
# ---------------------------------------------------------------------------- #
class Calendars(models.Model):
id = models.CharField(primary_key=True, max_length=100)
cms_id = models.CharField(max_length=100)
default_program = models.ForeignKey(ControlPrograms, models.CASCADE, blank=True, null=True)
timestamp = models.DateTimeField(auto_now_add=True)
class Meta:
managed = True
db_table = 'calendars'
# ---------------------------------------------------------------------------- #
class CalendarAssign(models.Model):
device_mac = models.ForeignKey(Device, models.CASCADE)
calendar = models.ForeignKey(Calendars, models.CASCADE, null=True)
timestamp = models.DateTimeField(auto_now_add=True)
class Meta:
managed = True
db_table = 'calendar_assign'
When applying the migration generated by Django it gives me an error.
operations = [
migrations.AlterField(
model_name='calendarassign',
name='calendar',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='smartbridge.Calendars'),
)
Generated sql code uses unsupported feature 'WITH ORDINALITY'.
It's because Django doesn't support the Postrges version we are using.
WITH ORDINALITY appears in psql 9.4 but we use version 9.1.
Both Postgres and Django cannot be upgraded right now. So I need to write the migration manually (without 'WITH ORDINALITY' feature).
migrations.RunSQL("DO $$DECLARE r record;\
BEGIN\
FOR r IN SELECT table_name,constraint_name \
FROM information_schema.constraint_table_usage \
WHERE table_name IN ('calendars') AND constraint_name like '%calendar_assign_calendar_id%'\
LOOP\
EXECUTE 'ALTER TABLE calendar_assign DROP CONSTRAINT '|| quote_ident(r.constraint_name) || ';';\
END LOOP;\
ALTER TABLE calendar_assign ALTER COLUMN calendar_id DROP NOT NULL; \
ALTER TABLE calendar_assign \
ADD CONSTRAINT calendar_assign_calendar_id_fk_calendars_id FOREIGN KEY (calendar_id) REFERENCES calendars(id);\
END$$;")
Migration seems to work fine.
calendar is now nullable but Django still detect some difference.
If a ask Django to generate the migration corresponding to the difference it generates the same as before my manual migration.
I would like Django to see no difference after my migration.
Thanks

I think you will have to set managed = False for the time being, otherwise the makemigrations command will each time think it has not been made nullable yet.
The migration construction command looks to the previous migration files, and thus constructs a conceptual model how a table will look like in the database if all the previous migrations took place. Based on that model it will look for differences with your Django model that you constructed, and thus create a migration file for that.
As long as you thus do not migrate with the AlterField command, Django will think you did not make the field nullable. It can not parse SQL so even if you made it nullable over there, it will still assume that that the field is non-NULLable.
By setting it to managed=False [Django-doc], Django will no longer manage the migrations of that file. You can create an empty migration [Django-doc] with:
python3 manage.py makemigrations --empty
and use this to define SQL queries to perform on the table.

Adding db_table to model added a last_login field

Background: I started a project with a custom User model. However, noob that I am, I was unaware of the AbstractBaseUser class. So I just wrote my own. The app has been deployed to prod and working fine. But now I want to switch to using AbstractBaseUser so I can take advantage of some of the built-in Django utilities (like the pre-made password resetting process). I had done this with a different app and it worked fine. But that one wasn't in prod while I made the change. Because this one is, I needed to keep the old user table while I made the changes with a copy of it. So my first step was to add db_table = test_users to my old user model, so as to keep the prod app running with an unchanged table. I ran the migration, and two unexpected things happened (I'm a noob, and that's why they were unexpected):
The old user table was renamed. I thought a new table would be created. No problem, I quickly copied the new table and named the copy with the old table's name so the prod app could still find its users
A column last_login was added. Why??
Here's my model, with the added db_table
class User(models.Model):
first_name = models.CharField(max_length=255)
last_name = models.CharField(max_length=255)
email = models.CharField(max_length=255)
password = models.CharField(max_length=255)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
client_id = models.IntegerField(blank=True, null=True)
is_admin = models.BooleanField(default=False)
is_super = models.BooleanField(default=False)
is_active = models.BooleanField(default=True)
class Meta:
db_table = "test_users"
The big problem with this is that when I change to AbstractBaseUser and run the migration, I get an error. Looking at the migration file I see that this change creates a migration that all it tries to do is to add last_login to the table. So, of course, the error I get is "Duplicate column name 'last_login'"
So, my question is two-fold:
Why was that column added in the first migration?
If I just run migrate --fake and keep going, will it have unintended consequences? I thought this could be a good solution, given that the migration file shows nothing else is being done, and if the field already exists, then no harm done?

Maybe because you've changed the parent class django automatically change all the migrations that connected to your user class

Django model tries to auto-create a primary key field even though it is already specified

It is stated in the Django documentation that each model requires a Primary Key. In my MySQL schema it is specified that for my table the primary key is made up of multiple columns. Using inspectdb to auto-generate the model creates the unique_together Meta information.
class Connections(models.Model):
router = models.ForeignKey('Routers', models.CASCADE)
src_mac = models.CharField(db_column='src_MAC', max_length=17) # Field name made lowercase.
src_ip = models.CharField(max_length=45, blank=True, null=True)
src_port = models.CharField(max_length=45)
dest_mac = models.CharField(db_column='dest_MAC', max_length=17, blank=True, null=True) # Field name made lowercase.
dest_ip = models.CharField(max_length=45)
dest_port = models.CharField(max_length=45)
first_activity = models.DateTimeField(blank=True, null=True)
last_activity = models.DateTimeField(blank=True, null=True)
hits_counter = models.IntegerField(blank=True, null=True)
def __unicode__(self):
return self.src_ip
class Meta:
db_table = 'Connections'
unique_together = (('router', 'src_mac', 'src_port', 'dest_ip', 'dest_port'),)
When I go to run
> python manage.py shell
>>> Connections.objects.all()
it gives me the OperationalError: (1054, "Unknown column 'Connections.id' in 'field list'") to my understanding this means that Django is trying to auto-create the primary key column for Connections.
How do I get around this?

tl;dr: Recreate the table or add the "id" field to the database manually.
I don't believe it's possible to create a primary key in all databases.
For example when using SQLite, if I create a simplified version of your table, altering it to add a PK returns an error:
> CREATE TABLE connections(src_MAC CHAR(17) NOT NULL, first_activity DATETIME NULL);
> INSERT INTO connections VALUES ('abcd1234',date('now'));
> ALTER TABLE connections add column id integer NOT NULL PRIMARY KEY AUTOINCREMENT;
Error: Cannot add a PRIMARY KEY column
Similarly, running a Django migration to create the field with SQLite simply fails to add the field silently (possibly a Django issue).
If you're unlucky and use a database that does not support adding a primary key (if you're not sure, check by running the ALTER command), your best bet would be to dump the table data, drop the table, create a new table with a primary key and finally reload the table data. This seems also to be the safest way to do it generally.
On the other hand if you do use a database that supports adding a primary key, you can either try to make migrations or alter the database manually.
Creating the id field manually in Postgres is as simple as:
ALTER TABLE connections ADD COLUMN id SERIAL PRIMARY KEY;
This here is sufficient to make the Model usable in Django. Now for the hard way.
It seems that makemigrations assumes an ID field already exists, and thus if you opt to use migrations, the initial migration would need a fix (remove the id field). You can create the CreateField migration manually thus:
migrations/0002_connections_id.py
# -*- coding: utf-8 -*-
# Generated by Django 1.9.7 on 2016-07-08 08:56
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('testapp', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='connections',
name='id',
field=models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
),
]
Unfortunately, this still doesn't seem to be enough, since the created column does not auto-increment (possibly another Django issue). The solution now would be to alter the created field like so:
> ALTER TABLE connections ALTER id SET default nextval('connections_id_seq');
Finally the values make sense to Django and you can now query and create "Connections" instances as usual.
Just in case anyone needs this later, I've pushed the code I used to test this to this GitHub repo.

Django migrate does not work when a schema name is passed along with the table name

I have an error when I run:
$ python manage.py migrate
This is the scenario:
I have 2 schemas in my Postgres database. It could be 2 schemas anywhere, right? could be 2 schemas in SQL Server,Mysql or Oracle.
I want to put the default django tables in one schema and I think I was able to do it. Schema name = :django".
My own models in a different schema. This is where I am having problems. Schema name = "data"
My model works just fine if I do not use schema in the Meta class in my model. To add a schema name in the db_table option you have to use "" quotes and this is the problem. You can actually create your model in the desired schema using "" quotes ("schema_name"."table_name"), but the problem comes when Django attempts to create the index of your model. PostgreSQL does not require or does not want the schema name to be passed, that is why it errors out.
I saw many posts but they assume I will use PostgreSQL forever so they suggest to use tools like tenants... blablabla which I believe it work for PostgreSQL only. The reason why I do not want to use that is because my project will not be portable anymore, what about if I want to move to SQL Server, Mysql or Oracle? I'd have to do modifications in my project and I do not want that.
I also read about using the option db_schema in my meta but it just does not work at all. It seems that Django does not recognized the db_schema attribute.
Here is my code:
Model:
class ClientCode(models.Model):
client_code_id = models.AutoField(primary_key=True)
client_code_name = models.CharField(max_length=255)
client_name = models.CharField(max_length=255)
is_inactive = models.BooleanField(default=False)
is_billable = models.BooleanField(default=True)
is_internal = models.BooleanField(default=True)
company = models.ForeignKey('companies.Company',related_name='clientcodes')
slug = models.SlugField(unique=True, default='')
def get_absolute_url(self):
return reverse("clientcodes:details", kwargs = {"slug": self.slug})
class Meta:
unique_together = ('company','client_code_name')
ordering = ['client_code_name']
db_table = '"data"."client_code"'
# app_label = 'company_clientcodes'
def __str__(self):
return '%s: %s' % (self.client_code_name, self.client_name)
This is the Error:
django.db.utils.ProgrammingError: zero-length delimited identifier at or near """"
LINE 1: CREATE INDEX ""data"."client_code"_447d3092" ON "data"."clie...
SQLMigrate code result:
BEGIN;
--
-- Create model ClientCode
--
CREATE TABLE "data"."client_code" ("client_code_id" serial NOT NULL PRIMARY KEY, "client_code_name" varchar(255) NOT NULL, "client_name" varchar(255) NOT NULL, "is_inactive" boolean NOT NULL, "is_billable" boolean NOT NULL, "is_internal" boolean NOT NULL, "slug" varchar(50) NOT NULL UNIQUE, "company_id" integer NOT NULL);
--
-- Alter unique_together for clientcode (1 constraint(s))
--
ALTER TABLE "data"."client_code" ADD CONSTRAINT "data_client_code_company_id_e066ecc7_uniq" UNIQUE ("company_id", "client_code_name");
ALTER TABLE "data"."client_code" ADD CONSTRAINT "data_client_code_company_id_bf78b136_fk_data_company_company_id" FOREIGN KEY ("company_id") REFERENCES "data"."company" ("company_id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX ""data"."client_code"_447d3092" ON "data"."client_code" ("company_id");
CREATE INDEX "data_client_code_slug_d0a8e64d_like" ON "data"."client_code" ("slug" varchar_pattern_ops);
COMMIT;
Any help is appreciated.

I solved the same error, I had something like this:
db_table = "'sellers'"
so I changed it to
db_table = 'sellers'
and then the migration worked fine. Maybe you could try changing your code in the same way.

This is what solved it for me:
class Meta:
db_table = 'data"."client_code'
This way, the generated SQL is correct.

Django is ignoring a field item for a Model with a fresh DB

I've completely wiped all my database tables in order to add a new field to a model since wiping SQL tables clean is the fastest way to do so without going the 'south' route when the SQL data only contains dummy data for testing purposes.
So here's my Model:
class Student(models.Model):
uid = models.ForeignKey(User)
title = models.CharField(max_length=250, help_text='The student name.')
resume = models.FileField(upload_to=get_upload_resume_name)
location = models.ForeignKey(Location)
country = models.ForeignKey(Country)
prim_interest = models.CharField(max_length=250, help_text='Maximum 250 characters.')
sec_interest = models.CharField(max_length=250, help_text='Maximum 250 characters.')
cellphone = models.IntegerField(default=0)
email_verified = models.BooleanField(default=False)
thumbnail = models.FileField(upload_to=get_upload_file_name)
def __unicode__(self):
return self.title
and the last field that I've added called 'thumbnail' is not being created by Django when I call syncdb right after deleting all my tables.
My method of completely wiping the tables has always worked no matter what drastic changes are applied to models.py and suddenly this is not the case for this instance. I could show you more code snippets but I have no clue where else in the Django project has a direct effect on generating models.
What could be causing syncdb to be refusing to write this new field 'thumbnail' to the Student table in the DB?

Could be That south is interfering. You habe no migrations waiting. If I remember well syncdb dos not create tables That south is scheduled to create.
Why not make a south migrate after your syncdb?