I have a Django model in my python project with a meta class detailing it's indexes. I'm curious if there's a way to create the index using the nested path of the json object. In this case we know the structure of our json and I wanted to stick with a BTree or Hash index on the specific element.
If I were simply running this as raw sql, I'd expect to just do something like:
CREATE INDEX ON foster_data(root->'level_1'->'level_2'->>'name');
I was hoping I could do something like this in my model:
from django.db import models
from django.contrib.postgres import indexes
class ParentGuardians(Facilitators): # which extends models.Model
parent_identifier = models.IntegerField(db_column='p_id', default=None, blank=True,
null=True)
class Meta:
constraints = [
models.UniqueConstraint(fields=['table_id', name='UniqueConstraint for Parents')
]
indexes = [
models.Index(fields=['p_id', ]),
indexes.BTreeIndex(fields=[models.JSONField('{"root": {"level_1": {"level_2": "name"}}}'), ]
, name="jsonb_p_id_idx"),
]
or even:
...
indexes.BTreeIndex(fields=["root->'level_1'->'level_2'->>'name'", ]
...
But the named field fields only wants strings and only wants them to be the top level field defined in the model.
I'm aware of this questions: Indexing JSONField in Django PostgreSQL but it seems more of a hack and wanted the result generated from the codebase and makemigrations, not to manually edit it. Is this possible more recently?
Django 3.2 introduced native support for these indexes.
The question as asked presently doesn't seem to have the definition of the JSONField, but assuming it is something like
from django.db import models
class Facilitators(models.Model):
foster_data = models.JSONField()
To index a particular key, you combine an F expression with a JSONField path lookup on the model's Meta indexes option:
from django.contrib.postgres.fields import JSONField
class Facilitators(models.Model):
foster_data = models.JSONField()
class Meta:
indexes = [
models.Index(models.F("foster_data__root__level_1__level2__name"), name="foster_data__root__level_1__level2__name_idx"),
]
This will create a B-Tree index. If you are adding these to an existing model, be sure to makemigrations and migrate.
See this answer as well https://stackoverflow.com/a/74619523/
Related
Let's say I have the following model and I have already some instances of the model saved in the database -
class Comment(models.Model):
comment_text = models.CharField(max_length=255)
Now, if I want to change the max_length argument to 127 instead of 255 and apply the migrations, the older instances will still be in the database where some might have the length more than 127 which is not obeying the new migrations.
What is the best approach to migrate all previous data along with the new migration rules?
You have to write your migration and apply that before the generated migration (which alters the model).
For this purpose, you have first to write your migration. Place it in the migrations folder of the app. Then add its name to the dependencies field of the generated migration.
This example updates the uuid field of all MyModel Objects.
And is located in my_migration.py
# Generated by Django A.B on YYYY-MM-DD HH:MM
from django.db import migrations
import uuid
def gen_uuid(apps, schema_editor):
MyModel = apps.get_model('myapp', 'MyModel')
for row in MyModel.objects.all():
row.uuid = uuid.uuid4()
row.save(update_fields=['uuid'])
class MyMigration(migrations.Migration):
dependencies = [
('myapp', '0004_add_uuid_field'),
]
operations = [
# omit reverse_code=... if you don't want the migration to be reversible.
migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
]
Assume that your generated migration that alters MyModel is called GeneratedMigration.. edit the generated migrations file as below.
class GeneratedMigration(migrations.Migration):
dependencies = [
('myapp', 'my_migration'),
]
You can also use the run_before attribute.
Checkout djano documentation on writing migrations for more information.
As stated in https://docs.djangoproject.com/en/1.10/ref/models/fields/#django.db.models.Field.db_index, you can add indexes to your table using db_index=True. But how do you force hash vs btree indexes?
Things have changed a lot since the question was asked. It is now possible to define an index in the model layer through the metaclass of your model (at least since django 1.11):
https://docs.djangoproject.com/en/dev/ref/contrib/postgres/indexes/
For example:
from django.contrib.postgres.indexes import HashIndex
from django.db import models
class User(models.Model):
name = models.CharField(max_length=100)
class Meta:
indexes = (HashIndex(fields=('name',)),)
Django has no implemented particular index type selection through models.
Workaround would be to create empty migration and write SQL statement in your migration for consistency
https://docs.djangoproject.com/en/1.10/howto/writing-migrations/
manage.py makemigrations --empty app
Inside of migration in operations put following
migrations.RunSQL('Query to add index')
RunSQL docs
Edit: There seems to be some confusion about what I'm asking. That model is for the Postgres view that I created in migration 0009. I was under the impression that Django won't generate a migration for a model if it has the managed = False option. However, it's still trying to create it.
Also, I'm using Django 1.8 with Python 3.4.
I'm having trouble creating a Django model for a Postgres view, using these links as a guide: drdaeman and eceppda's answer in Can I use a database view as a model in django. I also looked up the Options.managed entry in Django's API docs. However, even with this, it's creating a migration that adds a table for the view's model.
This is my code so far:
foo/models.py
class RelevantModel(models.Model):
rebate_pool_total = models.OneToOneField('foo.VirtualTotal', null=True)
total = models.DecimalField(null=True, decimal_places=2, max_digits=32)
class VirtualTotal(models.Model):
relevant_model = models.ForeignKey('foo.RelevantModel')
total = models.DecimalField(null=True, decimal_places=2, max_digits=32)
class Meta:
managed = False
foo/migrations/0009_add_foo_view.py
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('foo', '0008_previous_migration'),
]
sql = """
create VIEW foo_virtualtotal AS
SELECT rest of view...
"""
operations = [
migrations.RunSQL('DROP VIEW IF EXISTS foo_virtualtotal;'),
migrations.RunSQL(sql)
]
What am I doing wrong?
Django does create a migration for each newly added table in your app regardless of whether it's a managed model or not. However there is a very important and subtle difference when you use the managed=False setting. The resultant migration is a dummy entry. It does not execute any SQL at all.
To confirm this add a new model that is unmanaged
class Dummy(models.Model):
something = models.IntegerField()
class Meta:
managed = False
now when you do makemigrations followed by sqlimigrate *myapp* *migration_number* you will see that it doesn't produce any sql.
If on the other hand, you do find that Django is trying to create a table for you, that usually means that you had the same model in existence earlier but at the time the model was managed. To confirm this, search your migrations folder for VirtualTotal which is the name of the model in question.
I am currently working with one Django project.
I have the dictionary in it:
from models.py
class teltab(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comment=models.CharField(max_length=255,blank=True)
and a form to add data to the dictionary:
class newtelescopesform(forms.ModelForm):
class Meta:
model=teltab
Usually I get a comment from the form and writes it to the dictionary:
from views.py
if len(request.GET['comment'])>0:
commentq=request.GET['comment']
tel_list.update(comment=commentq)
for item in tel_list:
item.save()
But now I need to append a new comment to an already existing cell in the resulting table.
Namely my table looks like this
and I want to get this
In fact either you define new comment model with a ForeignKey: teltab
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
class CommentModel(models.Model):
teltab = models.ForeignKey('telTabModel', related_name='comments')
# ...
Or if you are using PostgreSQL you can use ArrayField as explained:
from django.contrib.postgres.fields import ArrayField
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comments = ArrayField(models.CharField(max_length=200), blank=True),
If you are not using PosgreSQL and you still want to use array you I will recommend Jsonfield
pip install jsonfield
from jsonfield import JSONField
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comments = JSONField(default=[])
I suppose u should change your model comment field type to TextField:
comment=models.TextField(blank=True)
Then just add "/n{new line}" to it
Edit: Dan's right, it s not a good idea there's opportunity to store lists in model if i'm not mistaken.
I have the following model:
from django.db import models
class PopulationData(models.Model):
slot = models.IntegerField(db_index=True)
sample = models.IntegerField()
value = models.FloatField()
class Meta:
unique_together = (('slot', 'sample'),)
And I would like to create also a compound index on the column pair that have the UNIQUE constraint, like so:
CREATE INDEX my_compound_index ON myapp_populationdata (slot, sample);
Right now I have a separate code connected to the post_syncdb signal that issues the previous SQL statement. Is there any way to indicate it from the model specification? (Note: I'm using the 1.3 branch).
Starting from django-1.5 you can make compound index using index_together meta option:
https://docs.djangoproject.com/en/dev/ref/models/options/#index-together
Note index_together is deprecated since django-4.2. Instead, use much more powerful indexes meta option. With it you can create not only compound indexes, but also other types of indexes, e.g. function-based indexes, partial (conditional) indexes, covering indexes:
https://docs.djangoproject.com/en/dev/ref/models/options/#django.db.models.Options.indexes
Starting from Django-1.11 use Meta.indexes option https://docs.djangoproject.com/en/1.11/ref/models/indexes/:
from django.db import models
class PopulationData(models.Model):
slot = models.IntegerField(db_index=True)
sample = models.IntegerField()
value = models.FloatField()
class Meta:
unique_together = (('slot', 'sample'),)
indexes = [
models.Index(fields=['slot', 'sample']),
]
Since a unique constraint also creates an index, it would be counterproductive to create both.
for example, from the postgres docs:
There's no need to manually create indexes on unique columns; doing so would just duplicate the automatically-created index.
Credit to Mark Byers for the doc link
If for some reason you still want to create a multi-column index, you can do so via index_together:
class PopulationData(models.Model):
...
class Meta:
index_together = [['slot', 'sample']]
I think that's not currently implemented in the django ORM.
If you use a migration tool (like south) that might be a good place to add that sql statement or if you preffer to avoid raw sql you could use sqlalchemy (core) but this case sounds simple enough to just go with sql.
Note: index_together may be deprecated in the future:
https://docs.djangoproject.com/en/dev/ref/models/options/#index-together
Use indexes in the Meta class instead:
from django.db import models
class Customer(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
class Meta:
indexes = [
models.Index(fields=['last_name', 'first_name']),
models.Index(fields=['first_name'], name='first_name_idx'),
]