I am using PyMongo to insert data (title, description, phone_number ...) into MongoDB. However, when I use mongo client to view the data, it displays the properties in a strange order. Specifically, phone_number property is displayed first, followed by title and then comes description. Is there some way I can force a particular order?
The above question and answer are quite old. Anyhow, if somebody visits this I feel like I should add:
This answer is completely wrong. Actually in Mongo Documents ARE ordered key-value pairs. However when using pymongo it will use python dicts for documents which indeed are not ordered (as of cpython 3.6 python dicts retain order, however this is considered an implementation detail). But this is a limitation of the pymongo driver.
Be aware, that this limitation actually impacts the usability. If you query the db for a subdocument it will only match if the order of the key-values pairs is correct.
Just try the following code yourself:
from pymongo import MongoClient
db = MongoClient().testdb
col = db.testcol
subdoc = {
'field1': 1,
'field2': 2,
'filed3': 3
}
document = {
'subdoc': subdoc
}
col.insert_one(document)
print(col.find({'subdoc': subdoc}).count())
Each time this code gets executed the 'same' document is added to the collection. Thus, each time we run this code snippet the printed value 'should' increase by one. It does not because find only maches subdocuemnts with the correct ordering but python dicts just insert the subdoc in arbitrary order.
see the following answer how to use ordered dict to overcome this: https://stackoverflow.com/a/30787769/4273834
Original answer (2013):
MongoDB documents are BSON objects, unordered dictionaries of key-value pairs. So, you can't rely on or set a specific fields order. The only thing you can operate is which fields to display and which not to, see docs on find's projection argument.
Also see related questions on SO:
MongoDB field order and document position change after update
Can MongoDB and its drivers preserve the ordering of document elements
Ordering fields from find query with projection
Hope that helps.
Related
questions = Question.objects.filter(category=category)
seen_questions_query = SeenQuestions.objects.filter(worker=user_id).only('question_id')
seen_questions_list = list(seen_questions_query)
questions_list = list(questions.exclude(id__in=seen_questions_list).values())
random_sample = random.sample(questions_list, question_count)
I have this code above that does not work. I'm more curious about good Django practices than getting this to work.
The goal is to take a "user_id" and query for questions they've seen in the "SeenQuestions" model then turn that into a list, which will be used to query the "Question" model for all questions that don't have a primary key on the list. Then I convert that to a list and randomly sample "question_count" amount of questions, which get turned into a JSON and returned to whomever is making the GET request.
I've taken the approach above, but I don't feel like this is best Django practices, it seems like there is a better way rather than converting query sets to list then querying on that list then converting to a list again.
The above does not work because it complains about converting "seen_questions_query" to list on line 3.
You can do this using one single queryset.
I think you are missing related_name usage (see docs), assuming you have not set anything, you should access related SeeQuestions through question_instance.seenquestions_set.
Below code should do the trick:
seen_questions = SeenQuestion.objects.filter(worker=user_id)
questions = Question.objects.filter(category=category).exclude(seenquestions_set__in=seen_questions)
Finally, to dump you data as JSON you can use:
from django.core.serializers.json import DjangoJSONEncoder
from django.core.serializers import serialize
serialize('json', questions, cls=DjangoJSONEncoder)
It's possible to write a query that always returns all of the elements in a collection, to use pymongo as an example:
MongoClient()["database"]["collection"].find({})
However, due to the structure of my code, I would quite like to be able to construct a query that does the opposite, a query that will necessarily return zero elements in all situations:
MongoClient()["database"]["collection"].find(null_query)
How can I define null_query, such that this is correct?
You can ask for any field to be in an empty list. It seems reasonable to use the _id field for this:
db.collection.find({_id: {$in: []}})
If you want a shorter query you don't need to use the _id field
at all:
db.collection.find({_:{$in:[]}})
Alternative if MongoDB version >= 3.4:
Arguably one can also ask if the _id field does not exists, which has been suggested by #Marco13:
db.collection.find({_id: {$exists: false}})
However, this assumes that all documents have the _id field, which is not necessarily true for MongoDB versions before 3.4 where a collection could be created with db.createCollection("mycol", {autoIndexID : false}) so all documents were not automatically given an _id field.
I want to select all data or select with conditional in table random but I can't find any guide in MongoDB in Python to do this.
And I can't show all data was select.
Here my code:
def mongoSelectStatement(result_queue):
client = MongoClient('mongodb://localhost:27017')
db = client.random
cursor = db.random.find({"gia_tri": "0.5748676522161966"})
# cursor = db.random.find()
inserted_documents_count = cursor.count()
for document in cursor:
result_queue.put(document)
There is a quite comprehensive documentation for mongodb. For python (Pymongo) here is the URL: https://api.mongodb.org/python/current/
Note: Consider the version you are running. Since the latest version has new features and functions.
To verify pymongo version you are using execute the following:
import pymongo
pymongo.version
Now. Regarding the select query you asked for. As far as I can tell the code you presented is fine. Here is the select structure in mongodb.
First off it is called find().
In pymongo; if you want to select specific rows( not really rows in mongodb they are called documents. I am saying rows to make it easy to understand. I am assuming you are comparing mongodb to SQL); alright so If you want to select specific document from the table (called collection in mongodb) use the following structure (I will use random as collection name; also assuming that the random table has the following attributes: age:10, type:ninja, class:black, level:1903):
db.random.find({ "age":"10" }) This will return all documents that have age 10 in them.
you could add more conditions simply by separating with commas
db.random.find({ "age":"10", "type":"ninja" }) This will select all data with age 10 and type ninja.
if you want to get all data just leave empty as:
db.random.find({})
Now the previous examples display everything (age, type, class, level and _id). If you want to display specific attributes say only the age you will have to add another argument to find called projection eg: (1 is show, 0 is do not show):
{'age':1}
Note here that this returns age as well as _id. _id is always returned by default. You have to explicitly tell it not to returning it as:
db.random.find({ "age":"10", "name":"ninja" }, {"age":1, "_id":0} )
I hope that could get you started.
Take a look at the documentation is very thorough.
Need help in understanding what is happening here and a suggestion to avoid this!
Here is my snippet:
result = [list of dictionary objects(dictionary objects have 2 keys and 2 String values)]
copyResults = list(results);
## Here I try to insert each Dict into MongoDB (Using PyMongo)
for item in copyResults:
dbcollection.save(item) # This is all saving fine in MongoDB.
But when I loop thru that original result list again it shows dictionary objects with a new field added
automatically which is ObjectId from MongoDB!
Later in code I need to transform that original result list to json but this ObjectId is causing issues.No clue why this is getting added to original list.
I have already tried copy or creating new list etc. It still adds up ObjectId in the original list after saving.
Please suggest!
every document saved in mongodb requires '_id' field - which has to be unique among documents in the collection. if you don't provide one, mongodb will automatically create one with ObjectId (bson.objectid.ObjectId for pymongo)
If you need to export documents to json, you have to pop '_id' field before jsonifying it.
Or you could use:
rows['_id'] = str(rows['_id'])
Remember to set it back if you then need to update
I am using Pymongo to access Mongo db. I want to search for all people nearby a specified location with name contains a string. For example, I want to search all people nearby [105.0133, 21.3434] and name contains 'Mark'. So I write the query like this:
db.users.find({ "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS }, "name": "/Mark/" })
(I have an index "location.coords" in my "users" collection)
The query works fine in Mongodb console, but while execute by Pymongo, the dictionary being re-sort like this:
{ "name": "/Mark/", "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS } }
(The "name" key is before "location.coords", that is not what I expected - also Mongodb expected)
That causes Mongodb cannot understand the query and returns no results. Can anyone help me to figure out how to force the Pymongo does not re-sort my dictionary.
Thanks and regards
The dictionary type is inherently orderless. From the python documentation:
It is best to think of a dictionary as an unordered set of key: value
pairs, with the requirement that the keys are unique (within one
dictionary).
If you want to index your dictionary in a specific order, you'll have to store your order somehow. One easy way to do this is to keep your keys in a list, like:
mongo_keys = ["location.coords", "name"]
for k in mongo_keys:
do_something(mongo_result[k])
You also might want to investigate:
class collections.OrderedDict([items])
Return an instance of a dict
subclass, supporting the usual dict methods. An OrderedDict is a dict
that remembers the order that keys were first inserted. If a new entry
overwrites an existing entry, the original insertion position is left
unchanged. Deleting an entry and reinserting it will move it to the
end.
Unfortunately if you need more help than that, you'll need to provide more details of your situation.
The issue isn't the ordering, it's "/Mark/". The notation with forward slashes is a convenience provided by the javascript shell, and don't constitute a part of the regular expression pattern itself (unless you meant for them to be literal slashes, in which case I've misunderstood your question).
To use a regular expression ("contains") filter in PyMongo, you need to pass a Python regular expression object. Try this:
{ "name": re.compile("Mark"), "location.coords": { "$nearSphere": [105.0133, 21.3434], "$maxDistance": 10/EARTH_RADIUS } }