Develop Reference

Optimize Django ORM query to get object if a specific related object does not exist

I have the following table structures:
class Library:
id = models.CharField(...)
bookcase = models.ForeignKey(
Bookcase,
related_name="libraries"
)
location = models.ChoiceField(...)
# Other attributes...
class Bookcase:
# some attributes
type = models.ChoiceField(..)
class Book:
bookcase = models.ForeignKey(
Bookcase,
related_name="books"
)
title=models.CharField(...)
status=models.ChoiceField(...) # borrowed | missing | available
Say if I want to get all Library objects that does not have a book with title "Foo" that is NOT missing, how can I optimize this query? I have the following:
libraries = Library.objects.select_related('bookcase').filter(location='NY', bookcase__type='wooden')
libraries_without_book = []
for library in libraries:
has_non_missing_book = Book.objects.filter(
bookcase=library.bookcase,
title="Foo",
).exclude(status='missing').exists()
if not has_non_missing_book:
libraries_without_book.append(library.id)
Unfortunately, this performs an extra query for every Library object that matches the initial filtering condition. Is there a more optimized method I can use here that makes use of prefetch_related in some way?

Book.objects.filter(~Q(status='missing'),bookcase=library.bookcase,title='Foo')
This query should be sufficient

Why is it so slow when update ListField in mongoengine?

It's too slow when I update a ListField with mongoengine.Here is an example
class Post(Document):
_id = StringField()
txt = StringField()
comments = ListField(EmbeddedDocumentField(Comment))
class Comment(EmbeddedDocument):
comment = StringField()
...
...
position = 3000
_id = 3
update_comment_str = "example"
#query
post_obj = Post.objects(_id=str(_id)).first()
#update
post_obj.comments[position].comment = update_comment_str
#save
post_obj.save()
The time it cost increases with the increase of the length of post_obj.comments.
How to optimize it?

Post.objects(id=str(_id)).update(**{"comments__{}__comment".format(position): update_comment_str})
In your code.
You fetched the whole document into python instance which will take place in RAM.
Then update 3000 th comments which will do some magic in mongoengine(marking changed fields and so on).
Then saves document.
In my answer,I have sent the update instruction to mongodb instead of fetching whole documents with N comments into Python which will save memory(RAM) and time.
The mongoengine/MongoDB supports index support update like
set__comments__1000__comment="blabla"
In order to give position using variable, I've used python dictionary and kwargs trick.

use existing field as _id using elasticsearch dsl python DocType

I have class, where I try to set student_id as _id field in elasticsearch. I am referring persistent example from elasticsearch-dsl docs.
from elasticsearch_dsl import DocType, String
ELASTICSEARCH_INDEX = 'student_index'
class StudentDoc(DocType):
'''
Define mapping for Student type
'''
student_id = String(required=True)
name = String(null_value='')
class Meta:
# id = student_id
index = ELASTICSEARCH_INDEX
I tied by setting id in Meta but it not works.
I get solution as override save method and I achieve this
def save(self, **kwargs):
'''
Override to set metadata id
'''
self.meta.id = self.student_id
return super(StudentDoc, self).save(**kwargs)
I am creating this object as
>>> a = StudentDoc(student_id=1, tags=['test'])
>>> a.save()
Is there any direct way to set from Meta without override save method ?

There are a few ways to assign an id:
You can do it like this
a = StudentDoc(meta={'id':1}, student_id=1, tags=['test'])
a.save()
Like this:
a = StudentDoc(student_id=1, tags=['test'])
a.meta.id = 1
a.save()
Also note that before ES 1.5, one was able to specify a field to use as the document _id (in your case, it could have been student_id), but this has been deprecated in 1.5 and from then onwards you must explicitly provide an ID or let ES pick one for you.

Peewee - How to Convert a Dict into a Model

Lets say I have
import peewee
class Foo(Model):
name = CharField()
I would like to do the following:
f = {id:1, name:"bar"}
foo = Foo.create_from_dict(f)
Is this native in Peewee? I was unable to spot anything in the source code.
I've wrote this function which works but would rather use the native function if it exists:
#clazz is a string for the name of the Model, i.e. 'Foo'
def model_from_dict(clazz, dictionary):
#convert the string into the actual model class
clazz = reduce(getattr, clazz.split("."), sys.modules[__name__])
model = clazz()
for key in dictionary.keys():
#set the attributes of the model
model.__dict__['_data'][key] = dictionary[key]
return model
I have a web page that displays all the foos and allows the user to edit them. I would like to be able to pass a JSON string to the controller, where I would convert it to a dict and then make Foos out of it, so I can update as necessary.

If you have a dict, you can simply:
class User(Model):
name = CharField()
email = CharField()
d = {'name': 'Charlie', 'email': 'foo#bar.com'}
User.create(**d)

You could use PickledKeyStore which allows you to save any value as a python dict and it works like Python's pickle library.

Indexing not supported?

I get this error message:
TypeError: 'City' object does not support indexing
when this is my model:
class City(db.Model):
region = db.ReferenceProperty()
name = db.StringProperty()
image = db.BlobProperty()
vieworder = db.IntegerProperty()
areacode = db.IntegerProperty()
and this is my query
items = Item.all().filter('modified >', timeline).filter('published =', True).order('-modified').filter('cities =',city[0].key()).fetch(PAGESIZE + 1)`
Can you tell my how I should make my query or model my class definitions? I use listproperty(db.Key) to model relationships with references and I thought I could filter like the above query since it worked for one item. What should I do?

city[0].key()
is the cause of your error. Chances are you thought city was a list, but it's actually a single item, or some such.