I see that when you add a column and want to create a schemamigration, the field has to have either null=True or default=something.
What I don't get is that many of the fields that I've written in my models initially (say, before initial schemamigration --init or from a converted_to_south app, I did both) were not run against this check, since I didn't have the null/default error.
Is it normal?
Why is it so? And why is South checking this null/default thing anyway?
If you add a column to a table, which already has some rows populated, then either:
the column is nullable, and the existing rows simply get a null value for the column
the column is not nullable but has a default value, and the existing rows are updated to have that default value for the column
To produce a non-nullable column without a default, you need to add the column in multiple steps. Either:
add the column as nullable, populate the defaults manually, and then mark the column as not-nullable
add the column with a default value, and then remove the default value
These are effectively the same, they both will go through updating each row.
I don't know South, but from what you're describing, it is aiming to produce a single DDL statement to add the column, and doesn't have the capability to add it in multiple steps like this. Maybe you can override that behaviour, or maybe you can use two migrations?
By contrast, when you are creating a table, there clearly is no existing data, so you can create non-nullable columns without defaults freely.
When you have existing records in your database and you add a column to one of your tables, you will have to tell the database what to put in there, south can't read your mind :-)
So unless you mark the new field null=True or opt in a default value it will raise an error. If you had an empty database, there are no values to be set, but a model field would still require basic properties. If you look deeper at the field class you're using you will see django sets some default values, like max_length and null (depending on the field).
Related
I am developing an ERP with groups of developers and we need to preserve customers data when deleting existing columns or table for Dajngo models and DB.
For Example:
I added a column named columns1 and I gave the customer a release product of the System and then, but a week later I had to delete that new column but the customer have data stores in the column1 column, here how can I preserve data or solve this situation.
Another Example:
I have a new column name column2 with unique attr but here the customer have data, but I can not add new column with out allowed it to store the null data, but in this situation I do not want to allow the null data in column column2 and ether I can't put default attr because it has unique attr.
How to solve these things in Django.
i think you need to add one Boolean field in the table field name delete if you want to delete the column don't delete it put delete field value to true. when you query add a filter with condition delete = false. i think this will work for 1st condition
Data migrations will help you solve these. You will have to write scripts in migrations to ensure the data is in the desired state before enforcing constraints e.g check column2 to ensure the data is unique and proceed to add the unique constraint. Refer to this tutorial for more
Usually in these case, you would need to backup your database to an sql file for example https://dev.mysql.com/doc/mysql-backup-excerpt/5.7/en/mysqldump-sql-format.html.
After the backup you can change database structure however you want without worrying about the loss of data(because you saved in the backup sql). If you want to rollback the old version you can just apply the old DB(remember to backup your latest version of database too!). Or if you want to reapply deleted column just check the backup db data.
I'm developing a generic interface that will allow other devs to define (with a list of just field names) which fields they want the end user to be able to sort and filter.
Now, sorting and filtering works best when there's an index on the database column, but I would like to warn the developer that they are allowing filtering/sorting on a column that does not have a index, which may hurt performance, and then suggest they should add an index parameter to the column definition or create an Index() instance on that column.
But inspecting the model, I can only see if Column.index is True, but I can't tell if the column has an index when using the Index() constructor.
Note that I can't access the actual database, I have to work with the declarative stuff only.
Thanks!
https://docs.djangoproject.com/en/1.10/topics/migrations/
Here it says:
"PostgreSQL is the most capable of all the databases here in terms of schema support; the only caveat is that adding columns with default values will cause a full rewrite of the table, for a time proportional to its size.
"For this reason, it’s recommended you always create new columns with null=True, as this way they will be added immediately."
I am asking if I get it correct.
From what I understand, I should first create the field with null=True and no default value then migrate it and then give it a default value and migrate it again, the values will be added immediately, but otherwise the whole database would be rewritten and Django migration doesn't want to do the trick by itself?
It's also mentioned in that same page that:
In addition, MySQL will fully rewrite tables for almost every schema
operation and generally takes a time proportional to the number of
rows in the table to add or remove columns. On slower hardware this
can be worse than a minute per million rows - adding a few columns to
a table with just a few million rows could lock your site up for over
ten minutes.
and
SQLite has very little built-in schema alteration support, and so
Django attempts to emulate it by:
Creating a new table with the new schema Copying the data across
Dropping the old table Renaming the new table to match the original
name
So in short, what that statement you are referring to above really says is
postgresql exhibits mysql like behaviour when adding a new column with
a default value
The approach you are trying would work. Adding a column with a null would mean no table re write. You can then alter the column to have a default value. However existing nulls will continue to be null
The way I understand it, on the second migration the default value will not be written to the existing rows. Only when a new row is created with no value for the default field it will be written.
I think the warning to use null=True for new column is only related to performance. If you really want all the existing rows to have the default value just use default= and accept the performance consequence of a table rewrite.
I have created 31 objects in my database. Now, for some reason, if I create a new object through the Django admin page, the new object will have an id of 33. Now, suppose I change I change its id and then delete it. If I try to create a new object, it's id will be 34. So, the id is always shifted by 2. I'm very new to databases and Django, is there any reason for behaving like this? Thanks
Note: I didn't upload any code, because I don't think that's the problem...
By default, the id is an integer that is always incremented at the creation of an object. It is also incremented such that ids of deleted objects are never used again.
The incrementation is handled by the database itself, not Django. For example, with PostgreSQL, the corresponding database field corresponding the "id" has the "PRIMARY KEY" constraint. It basically means that the field should be not null, and with no duplicates. Moreover the field will be associated with a sequence, that stores the id to use for the next row creation. To change this number, run this in the database shell:
ALTER SEQUENCE yourobjectstable_id_seq RESTART WITH 1234;
However, as emphasized in the comments to your question, this is something you should not do: it is better to keep the "uniqueness" feature of the primary key, even for deleted objects, since other tables may use the id to refer to a row in your main table.
suppose there was a database table with one column, and it's a PK. To make things more specific this is a django project and the database is in mysql.
If I needed an additional column with all unique values, should I create a new UniqueField with unique integers, or just write a hash-like function to convert the existing PK's for each existing row (model instance) into a new unique variable. The current PK is a varchar/ & string.
With creating a new column it consumes more memory but I think writing a new function and converting fields frequently has disadvantages also. Any ideas?
Having a string-valued PK should not be a problem in any modern database system. A PK is automatically indexed, so when you perform a look-up with a condition like table1.pk = 'long-string-key', it won't be a string comparison but an index look-up. So it's ok to have string-valued PK, regardless of the length of the key values.
In any case, if you need an additional column with all unique values, then I think you should just add a new column.