Develop Reference

Create index on nested element JSONField for Postgres in Django

I have a Django model in my python project with a meta class detailing it's indexes. I'm curious if there's a way to create the index using the nested path of the json object. In this case we know the structure of our json and I wanted to stick with a BTree or Hash index on the specific element.
If I were simply running this as raw sql, I'd expect to just do something like:
CREATE INDEX ON foster_data(root->'level_1'->'level_2'->>'name');
I was hoping I could do something like this in my model:
from django.db import models
from django.contrib.postgres import indexes
class ParentGuardians(Facilitators): # which extends models.Model
parent_identifier = models.IntegerField(db_column='p_id', default=None, blank=True,
null=True)
class Meta:
constraints = [
models.UniqueConstraint(fields=['table_id', name='UniqueConstraint for Parents')
]
indexes = [
models.Index(fields=['p_id', ]),
indexes.BTreeIndex(fields=[models.JSONField('{"root": {"level_1": {"level_2": "name"}}}'), ]
, name="jsonb_p_id_idx"),
]
or even:
...
indexes.BTreeIndex(fields=["root->'level_1'->'level_2'->>'name'", ]
...
But the named field fields only wants strings and only wants them to be the top level field defined in the model.
I'm aware of this questions: Indexing JSONField in Django PostgreSQL but it seems more of a hack and wanted the result generated from the codebase and makemigrations, not to manually edit it. Is this possible more recently?

Django 3.2 introduced native support for these indexes.
The question as asked presently doesn't seem to have the definition of the JSONField, but assuming it is something like
from django.db import models
class Facilitators(models.Model):
foster_data = models.JSONField()
To index a particular key, you combine an F expression with a JSONField path lookup on the model's Meta indexes option:
from django.contrib.postgres.fields import JSONField
class Facilitators(models.Model):
foster_data = models.JSONField()
class Meta:
indexes = [
models.Index(models.F("foster_data__root__level_1__level2__name"), name="foster_data__root__level_1__level2__name_idx"),
]
This will create a B-Tree index. If you are adding these to an existing model, be sure to makemigrations and migrate.
See this answer as well https://stackoverflow.com/a/74619523/

How to change field type from dict to ReferenceField

I have two documents defined like this:
from mongoengine import Document, StringField, IntField, DictField
class PetModel(Document):
name = StringField()
age = IntField()
class PersonModel(Document):
name = StringField()
pet = DictField()
Is there a way to change the pet field type to ReferenceField if I already created PersonModel objects?

You need to write a migration script, easiest is probably to:
collect all PetModel and build a mapping {pet_id: pet_dict}
loop over the PersonModel documents using pymongo (the driver used by Mongoengine) and convert the pet field from DBRef to dict using the mapping built above
save/update that document
MongoEngine has a page explaining some migration use case: http://docs.mongoengine.org/guide/migration.html#documents-migration

Add text to a cell without overwriting existing data

I am currently working with one Django project.
I have the dictionary in it:
from models.py
class teltab(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comment=models.CharField(max_length=255,blank=True)
and a form to add data to the dictionary:
class newtelescopesform(forms.ModelForm):
class Meta:
model=teltab
Usually I get a comment from the form and writes it to the dictionary:
from views.py
if len(request.GET['comment'])>0:
commentq=request.GET['comment']
tel_list.update(comment=commentq)
for item in tel_list:
item.save()
But now I need to append a new comment to an already existing cell in the resulting table.
Namely my table looks like this
and I want to get this

In fact either you define new comment model with a ForeignKey: teltab
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
class CommentModel(models.Model):
teltab = models.ForeignKey('telTabModel', related_name='comments')
# ...
Or if you are using PostgreSQL you can use ArrayField as explained:
from django.contrib.postgres.fields import ArrayField
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comments = ArrayField(models.CharField(max_length=200), blank=True),
If you are not using PosgreSQL and you still want to use array you I will recommend Jsonfield
pip install jsonfield
from jsonfield import JSONField
class telTabModel(models.Model):
code=models.CharField(max_length=255)
telescope=models.CharField(max_length=255)
comments = JSONField(default=[])

I suppose u should change your model comment field type to TextField:
comment=models.TextField(blank=True)
Then just add "/n{new line}" to it
Edit: Dan's right, it s not a good idea there's opportunity to store lists in model if i'm not mistaken.

Cleaner / reusable way to emit specific JSON for django models

I'm rewriting the back end of an app to use Django, and I'd like to keep the front end as untouched as possible. I need to be consistent with the JSON that is sent between projects.
In models.py I have:
class Resource(models.Model):
# Name chosen for consistency with old app
_id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
#property
def bookingPercentage(self):
from bookings.models import Booking
return Booking.objects.filter(resource=self)
.aggregate(models.Sum("percent"))["percent__sum"]
And in views.py that gets all resource data as JSON:
def get_resources(request):
resources = []
for resource in Resource.objects.all():
resources.append({
"_id": resource._id,
"name": resource.first,
"bookingPercentage": resource.bookingPercentage
})
return HttpResponse(json.dumps(resources))
This works exactly as I need it to, but it seems somewhat antithetical to Django and/or Python. Using .all().values will not work because bookinPercentage is a derived property.
Another issue is that there are other similar models that will need JSON representations in pretty much the same way. I would be rewriting similar code and just using different names for the values of the models. In general is there a better way to do this that is more pythonic/djangothonic/does not require manual creation of the JSON?

Here's what I do in this situation:
def get_resources(request):
resources = list(Resource.objects.all())
for resource in resources:
resource.booking = resource.bookingPercentage()
That is, I create a new attribute for each entity using the derived property. It's only a local attribute (not stored in the database), but it's available for your json.dumps() call.

It sounds like you just want a serialisation of your models, in JSON. You can use the serialisers in core:
from django.core import serializers
data = serializers.serialize('json', Resource.objects.all(), fields=('name','_id', 'bookingPercentage'))
So just pass in your Model class, and the fields you want to serialize into your view:
get_resources(request, model_cls, fields):
documentation here https://docs.djangoproject.com/en/dev/topics/serialization/#id2

How to introspect django model fields?

I am trying to obtain class information on a field inside a model, when I only know name of the field and name of the model (both plain strings). How is it possible?
I can load the model dynamically:
from django.db import models
model = models.get_model('myapp','mymodel')
Now I have field - 'myfield' - how can I get the class of that field?
If the field is relational - how to get related field?
Thanks a bunch!

You can use model's _meta attribute to get field object and from field you can get relationship and much more e.g. consider a employee table which has a foreign key to a department table
In [1]: from django.db import models
In [2]: model = models.get_model('timeapp', 'Employee')
In [3]: dep_field = model._meta.get_field_by_name('department')
In [4]: dep_field[0].target_field
Out[4]: 'id'
In [5]: dep_field[0].related_model
Out[5]: <class 'timesite.timeapp.models.Department'>
from django/db/models/options.py
def get_field_by_name(self, name):
"""
Returns the (field_object, model, direct, m2m), where field_object is
the Field instance for the given name, model is the model containing
this field (None for local fields), direct is True if the field exists
on this model, and m2m is True for many-to-many relations. When
'direct' is False, 'field_object' is the corresponding RelatedObject
for this field (since the field doesn't have an instance associated
with it).
Uses a cache internally, so after the first access, this is very fast.
"""

The answer from Anurag Uniyal to use get_field_by_name is now (5 years later) outdated as get_field_by_name is deprecated.
Django will give you the following hint:
RemovedInDjango110Warning: 'get_field_by_name is an unofficial API
that has been deprecated. You may be able to replace it with
'get_field()'
API docs for get_field are here.

If you would like to see ALL fields on a Django model object you can simply introspect it by calling ._meta.get_fields() on the class (or an instantiated model object) to get at list of all fields. This API is current with the latest version of Django.
Example:
from django.contrib.auth.models import User
User._meta.get_fields()
This will return a tuple of all model fields. Documentation can be found HERE.

django.db.models.loading.get_model() has been removed in django 1.9.
You are supposed to use django.apps instead.
'get_field_by_name is an unofficial API that has been deprecated in Django 1.10. You may be able to replace it with 'get_field()'
>>> from django.apps import apps
>>> from polls.models import Question
>>> QuestionModel = apps.get_model('polls','Question')
>>> QuestionModel._meta.get_field('pub_date')
>>> QuestionModel._meta.get_fields()
(<ManyToOneRel: polls.choice>, <django.db.models.fields.AutoField: id>, <django.db.models.fields.CharField: question_text>, <django.db.models.fields.DateTimeField: pub_date>)
link to issue