I am using Pymongo to access Mongo db. I want to search for all people nearby a specified location with name contains a string. For example, I want to search all people nearby [105.0133, 21.3434] and name contains 'Mark'. So I write the query like this:
db.users.find({ "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS }, "name": "/Mark/" })
(I have an index "location.coords" in my "users" collection)
The query works fine in Mongodb console, but while execute by Pymongo, the dictionary being re-sort like this:
{ "name": "/Mark/", "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS } }
(The "name" key is before "location.coords", that is not what I expected - also Mongodb expected)
That causes Mongodb cannot understand the query and returns no results. Can anyone help me to figure out how to force the Pymongo does not re-sort my dictionary.
Thanks and regards
The dictionary type is inherently orderless. From the python documentation:
It is best to think of a dictionary as an unordered set of key: value
pairs, with the requirement that the keys are unique (within one
dictionary).
If you want to index your dictionary in a specific order, you'll have to store your order somehow. One easy way to do this is to keep your keys in a list, like:
mongo_keys = ["location.coords", "name"]
for k in mongo_keys:
do_something(mongo_result[k])
You also might want to investigate:
class collections.OrderedDict([items])
Return an instance of a dict
subclass, supporting the usual dict methods. An OrderedDict is a dict
that remembers the order that keys were first inserted. If a new entry
overwrites an existing entry, the original insertion position is left
unchanged. Deleting an entry and reinserting it will move it to the
end.
Unfortunately if you need more help than that, you'll need to provide more details of your situation.
The issue isn't the ordering, it's "/Mark/". The notation with forward slashes is a convenience provided by the javascript shell, and don't constitute a part of the regular expression pattern itself (unless you meant for them to be literal slashes, in which case I've misunderstood your question).
To use a regular expression ("contains") filter in PyMongo, you need to pass a Python regular expression object. Try this:
{ "name": re.compile("Mark"), "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS } }
Related
I am starting using dynamoDB with python, and I have a doubt I can't find an answer online. I have a JSON with this format:
{
"version": "1.0.0",
"trips":[
{
"paymentMode": "ONLINE",
"type":"A",
"locations":{
"name": "Zona A",
"payment":1000
"company": ["XXX", "YYY"]
},
{
"paymentMode": "CASH",
"type":"B",
"locations":{
"name": "Zona D",
"payment":200
"company": ["XXX", "YYY"]
}
]
}
I can store it like that directly but I don't know if there is a way I can access individually each of the elements in trips in a get_item operation or similar? Like, having for example paymentMode and type as primary and sort key, even if they are inside the field "trips".
The input will be a Json like this, but I would like to put those fields as PK (considering they are unique) so I can retrieve only one item.
I know you can just scan everything and then iterate trips, but maybe with more elements, it will take too long. Is there a way to do this directly? if so, how can I do it? does dynamoDb do this automatically or i have to name it like trips.PaymentMode or something like that?
Thank you very much
When designing how to structure your data in a database, you need to consider A) what data you are storing B) how you will be accessing your data.
Additionally, you need to consider the characteristics of the database in questions. For DynamoDB, the main constraint is your primary key. The primary key is either the partition key or the combination of partition key and the sort key, and it must be unique for all items.
Although it's not entirely clear, based on your JSON your items are trips with fields paymentMode, type and locations. If those are your fields on the item, what should be your key? You mention using paymentMode and type, however, these likely won't be suitable, as they probably won't be unique. If you have a time associated with these, a primary key of paymentMode_type and sort key of time might do the job for you, but that depends on the volume of the data.
What you might be better off doing, is assign a unique identifier to each item (e.g. a uuid) and add secondary indices over the fields you want to use to query (paymentMode, type). This will allow you to query efficiently without having to scan the entire database. Do however keep in mind, that you will incur additional costs for the secondary indices.
I need to load a list of dicts (see below) into a mongoDB. Within mongo, you have to define an int type as NumberInt(). Python doesn't recognize this as a valid type for a dict. I've found pages on custom encoding for pymongo that don't actually do what I need. I'm totally stuck. Someone has to have encountered this before!
Need to insert a list of dicts like this into mongoDB from python.
agg = {
'_id' : unique_id_str,
'total' : NumberInt(int(total)),
'mode' : NumberInt(int(mymode))
}
You should be able to just insert the dict with an int, I've never needed to use NumberInt to insert documents using pymongo.
Also, fwiw, folks at mongodb told me that letting mongo create the _id itself tends to be more efficient but obviously it may work better for you to define in your case.
agg = {
'_id' : unique_id_str,
'total' : int(total),
'mode' : int(mymode)
}
should work
Now,I try to insert data with pymongo in mongoldb.
get_db().users.update({'_id':ObjectId(session['user_id'])},{'$push':{'hme':ObjectId(id)}},upsert=True)
but,the method produce duplicate ObjectID.before try find_one().
if not ObjectId(id) in get_db().users.find_one({'_id':ObjectId(session['user_id'])})['hme']:
get_db().users.update({'_id':ObjectId(session['user_id'])},{'$push':{'hme':ObjectId(id)}},upsert=True)
better method request..
may be use forEach.but syntax error
yang
If the hme key holds arrays of ObjectIds then you could try the $addToSet operator instead of the $push since it adds a value to an array unless the value is already present, in which case $addToSet does nothing to that array thus it only ensures that there are no duplicate items added to the set and does not affect existing duplicate elements:
get_db().users.update(
{'_id':ObjectId(session['user_id'])},
{
'$addToSet':{
'hme':ObjectId(id)
}
},
upsert=True
)
I am using PyMongo to insert data (title, description, phone_number ...) into MongoDB. However, when I use mongo client to view the data, it displays the properties in a strange order. Specifically, phone_number property is displayed first, followed by title and then comes description. Is there some way I can force a particular order?
The above question and answer are quite old. Anyhow, if somebody visits this I feel like I should add:
This answer is completely wrong. Actually in Mongo Documents ARE ordered key-value pairs. However when using pymongo it will use python dicts for documents which indeed are not ordered (as of cpython 3.6 python dicts retain order, however this is considered an implementation detail). But this is a limitation of the pymongo driver.
Be aware, that this limitation actually impacts the usability. If you query the db for a subdocument it will only match if the order of the key-values pairs is correct.
Just try the following code yourself:
from pymongo import MongoClient
db = MongoClient().testdb
col = db.testcol
subdoc = {
'field1': 1,
'field2': 2,
'filed3': 3
}
document = {
'subdoc': subdoc
}
col.insert_one(document)
print(col.find({'subdoc': subdoc}).count())
Each time this code gets executed the 'same' document is added to the collection. Thus, each time we run this code snippet the printed value 'should' increase by one. It does not because find only maches subdocuemnts with the correct ordering but python dicts just insert the subdoc in arbitrary order.
see the following answer how to use ordered dict to overcome this: https://stackoverflow.com/a/30787769/4273834
Original answer (2013):
MongoDB documents are BSON objects, unordered dictionaries of key-value pairs. So, you can't rely on or set a specific fields order. The only thing you can operate is which fields to display and which not to, see docs on find's projection argument.
Also see related questions on SO:
MongoDB field order and document position change after update
Can MongoDB and its drivers preserve the ordering of document elements
Ordering fields from find query with projection
Hope that helps.
Mongodb updates provide the $push modifier to append to an array. My problem is that i want this to happen on a dict e.g
If my record looks like this initially:
{"collaborations":{'id1':{'role':'dev','scope':'dev'}}}
I want to add another item("id2" below) to the "collaborations" field dict to look something like this:
{"collaborations":{'id1':{'role':'dev','scope':'dev'},'id2':{'role':'qa','scope':'qa'}}}
I am trying with $push:
my_record.update({match_criteria},{$push,{"collaborations":{'id2':{'role':'qa','scope':'qa'}}}})
and also with $addToSet:
my_record.update({match_criteria},{$,{"collaborations":{'id2':{'role':'qa','scope':'qa'}}}})
With both of these, mongodb throws as error "Cannot apply $addToSet($push) modifier to non-array".
How can this be done for dict types? Any ideas?
The problem is that $addToSet and $push modifiers work with arrays.
To update sub-document (that is what you need here) just use $set modifier with dot notation to access sub-document (field.subfield):
my_record.update({
match_criteria
}, {
'$set': {
'collaborators.id2': {
// new document fields here
}
}
})