Develop Reference

Pymongo UUID search not returning documents that definitely exist

Trying to define a function in python that can search for a given UUID like so:
def getid(in_id):
return list(CollectionVar.find({"_id":UUID(in_id)}))
And passing in a UUID. I can take a UUID I know exists from Studio 3T like so:
db.getCollection("CollectionName").find({"_id":UUID("5002aa11-eeb7-4e68-a121-dd51497d2572")})
And the above query returns precisely one document. That same UUID in the python query returns absolutely nothing. I can find documents on other (non UUID) fields easily enough, for example the following works fine on that same document from earlier:
def getname(fn,sn):
return list(CollectionVar.find({"Firstname":re.compile(fn, re.IGNORECASE), "Surname":re.compile(sn, re.IGNORECASE)}))
This seems like a problem with the uuid.UUID class rather than a pymongo issue? Can anyone see the problem?
PyMongo Version 3.6.1

The issue is that PyMongo uses a legacy method of encoding/decoding UUID values by default. You probably want to configure the PyMongo client to use the more modern, cross-language compatible "standard" UUID representation:
client = MongoClient(MONGODB_URI, uuidRepresentation="standard")
Now you should be able to query directly using Python uuid.UUID instances:
from uuid import UUID
items = client["item_database"]["items"].find_one({
"uuid": UUID("187382af-1369-43e6-a0ba-d345886c986c")
})

I've solved this. For anyone else who hits this issue the solution is below:
from bson.binary import Binary, UUID_SUBTYPE
def getcust(inid):
newuuid=uuid.UUID(inid).bytes
return list(DealershipConsumer.find({"_id": Binary(bytes(bytearray(newuuid)), UUID_SUBTYPE)}))
UUID_SUBTYPE needs to be set to whatever subtype of UUID you use - in my case it's 4.

You could specify the UUID type you're using when getting the db :
import bson
import pymongo
mongo_client = pymongo.MongoClient(mongo_uri, document_class=dict)
db = mongo_client.get_database(
"my_db_name",
bson.codec_options.CodecOptions(uuid_representation=bson.binary.UUID_SUBTYPE),
)
If you want to read more about Mongo best practices while using UUIDs, this article might help.
Also, here are the docs about codec_options

I've scraped this from pymongo-2.8.1 in the bson.binary.UUIDLegacy class's docstring/comment, might be usefull
>>> import uuid
>>> from bson.binary import Binary, UUIDLegacy, UUID_SUBTYPE
>>> my_uuid = uuid.uuid4()
>>> coll = db.test
>>> coll.uuid_subtype = UUID_SUBTYPE
>>> coll.insert({'uuid': Binary(my_uuid.bytes, 3)})
ObjectId('...')
>>> coll.find({'uuid': my_uuid}).count()
0
>>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count()
1
>>> coll.find({'uuid': UUIDLegacy(my_uuid)})[0]['uuid']
UUID('...')
>>>
>>> # Convert from subtype 3 to subtype 4
>>> doc = coll.find_one({'uuid': UUIDLegacy(my_uuid)})
>>> coll.save(doc)
ObjectId('...')
>>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count()
0
>>> coll.find({'uuid': {'$in': [UUIDLegacy(my_uuid), my_uuid]}}).count()
1
>>> coll.find_one({'uuid': my_uuid})['uuid']
UUID('...')

You need to use ObjectId instead of UUID.
Try this, it works for me:
from bson.objectid import ObjectId
def getid(in_id):
return list(CollectionVar.find({"_id":ObjectId(in_id)}))

Django Querying MongoDB ObjectIds in views from json object

I am currently working on querying MongoDB objects in Python Django and had no trouble in creating queries if it's the other attributes needed.
However I need to modify my queries to specifically filter through the ObjectIds returning one or no object found.
From my Javascript I am passing a json data to my Django views.py here's how it currently looks like:
def update(request):
#AJAX data
line = json.loads(request.body)
_id = line['_id']
print("OBJECT_ID: %s" % (_id))
another_id = line['another_id']
print("ANOTHER_ID: %s" % (another_id))
*Don't confuse the another_id, there are objects that has the same another_id s and unfortunately has to remain like that. That's why I can't query it for update since it will update all duplicates. This is the reason why I need the ObjectId.
For checking here's what it prints out:
{u'$oid': u'582fc95bb7abe7943f1a45b2'}
ANOTHER_ID: LTJ1277
Therefore I appended the query in views.py like this:
try:
Line.objects(_id=_id).update(set__geometry=geometry, set__properties=properties)
print("Edited: " + another_id)
except:
print("Unedited.")
But it didn't return any object.
So I was wondering if the query itself can't recognize the $oidin the json body as "_id" : ObjectId("582fc95bb7abe7943f1a45b2")?
*Edit:
from bson.objectid import ObjectId
where I edited my views.py with:
_id = line['_id']
print("VALUES: %s" % (_id.get('$oid')))
try:
Line.objects(_id=ObjectId(_id.get('$oid'))).update(set__geometry=geometry, set__properties=properties)
Output:
VALUES: 582fc95bb7abe7943f1a498c
No luck. Still not querying/not found.

According to this Using MongoDB with Django reference site:
Notice that to access the unique object ID, you use "id" rather than "_id".
I tried revising the code from:
Line.objects(_id=ObjectId(_id.get('$oid'))).update(set__geometry=geometry, set__properties=properties)
to
Line.objects(id=ObjectId(_id.get('$oid'))).update(set__geometry=geometry, set__properties=properties)
...And it now works fine. Keeping this question for others who might need this.

Using JSON module to encode an object in Python

I have this String that I need to pass into a REST request
{"notification":{"tag":"MyTag"}}
I'm trying to turn into an object using the JSON module in python.
This is my attempt so far
import json
obj = json.dumps([{'notification': ('{tag : MyTag}')}])
But it isn't parsed correctly so the REST request won't work. Anyone have any ideas?

Just dump your dictionary as is, replace:
obj = json.dumps([{'notification': ('{tag : MyTag}')}])
with:
obj = json.dumps({"notification": {"tag": "MyTag"}})

Search document in MongoDB by _id using Flask-pymongo extension

I am baffled with the following problem.
I am using Flask, flask-pymongo extension, mongodb version v2.2.0-rc0, pdfile version 4.5
This is my route:
#app.route("/check/<id>")
def check(id):
doc=conn.db.msg.find_one({'_id':id})
return render_template('base.html',name=doc)
the id is a valid _id from a document in the msg collection, but ALWAYS return None.
I have tried:
pasing the ObjectId(id) but returns errortype: ObjectId not callable
pasing the id as str(id) returns None
Any thoughts?
UPDATE:
this how the full url looks like:
http://127.0.0.1:8000/check/5030b628895b800de1a9a792
UPDATE2:
I found a similar question with (answer) for ruby. Not sure how I can translate it to python, what sort imports/modules do I need?
How can I retrieve a document by _id?
UPDATE3:
I tried:
import bson
#app.route("/check/<id>")
def check(id):
id2="'"+id+"'"
doc=conn.db.msg.find_one({'_id':bson.objectid(id2) })
return render_template('base.html',name=doc)
but I get TypeError: 'module' object is not callable (it doesnt work with id either)
when I reached 1500 I will suggest a frustation tag :-S
UPDATE4:
Finally I got it up & running!
here it is my solution:
import bson
#app.route("/check/<id>")
def check(id):
doc=conn.db.msg.find_one({'_id':bson.ObjectId(oid=str(id))})
return render_template('base.html',name=doc)

You might also want to try using ObjectId from the bson.objectid module, like so:
from bson.objectid import ObjectId
In that case, you won't need to provide the oid kwarg. You'll just do something like this:
db_conn.msg.find_one({'_id': ObjectId(my_oid)})

How do I turn MongoDB query into a JSON?

for p in db.collection.find({"test_set":"abc"}):
posts.append(p)
thejson = json.dumps({'results':posts})
return HttpResponse(thejson, mimetype="application/javascript")
In my Django/Python code, I can't return a JSON from a mongo query because of "ObjectID". The error says that "ObjectID" is not serializable.
What do I have to do?
A hacky way would be to loop through:
for p in posts:
p['_id'] = ""

The json module won't work due to things like the ObjectID.
Luckily PyMongo provides json_util which ...
... allow[s] for specialized encoding and
decoding of BSON documents into Mongo
Extended JSON's Strict mode. This lets
you encode / decode BSON documents to
JSON even when they use special BSON
types.

Here is a simple sample, using pymongo 2.2.1
import os
import sys
import json
import pymongo
from bson import BSON
from bson import json_util
if __name__ == '__main__':
try:
connection = pymongo.Connection('mongodb://localhost:27017')
database = connection['mongotest']
except:
print('Error: Unable to Connect')
connection = None
if connection is not None:
database["test"].insert({'name': 'foo'})
doc = database["test"].find_one({'name': 'foo'})
return json.dumps(doc, sort_keys=True, indent=4, default=json_util.default)

It's pretty easy to write a custom serializer which copes with the ObjectIds. Django already includes one which handles decimals and dates, so you can extend that:
from django.core.serializers.json import DjangoJSONEncoder
from bson import objectid
class MongoAwareEncoder(DjangoJSONEncoder):
"""JSON encoder class that adds support for Mongo objectids."""
def default(self, o):
if isinstance(o, objectid.ObjectId):
return str(o)
else:
return super(MongoAwareEncoder, self).default(o)
Now you can just tell json to use your custom serializer:
thejson = json.dumps({'results':posts}, cls=MongoAwareEncoder)

Something even simpler which works for me on Python 3.6 using
motor==1.1
pymongo==3.4.0
from bson.json_util import dumps, loads
for mongo_doc in await cursor.to_list(length=10):
# mongo_doc is a <class 'dict'> returned from the async mongo driver, in this acse motor / pymongo.
# result of executing a simple find() query.
json_string = dumps(mongo_doc)
# serialize the <class 'dict'> into a <class 'str'>
back_to_dict = loads(json_string)
# to unserialize, thus return the string back to a <class 'dict'> with the original 'ObjectID' type.