I am using python to query a mongo collection and retrieve a value from it:
subquery = db.partsupp.aggregate([
{"$match": {"r_name": region }},
{"$group": {
"_id" : 0,
"minim": {"$min": "$supplycost"}
}
}
])
This query works just fine and it outputs:
[{'_id': 0, 'minim': 10}]
What I am trying to do now is to get the minim value from this aggregation.
Initially what I wanted was an 'if' to check if the query had any results, like this:
if len(subselect['result']) > 0 :
minim = subquery['result'][0]['minim']
else:
return subselect
But doing this only gets me the following error:
Traceback (most recent call last):
File "query2.py", line 195, in <module>
pprint( list(query2('Catalonia', 1, 1)) )
File "query2.py", line 72, in query2
if len(subquery['result']) > 0 :
TypeError: 'CommandCursor' object is not subscriptable
It looks like the result from the subselect query is not iterable or something like that, how can I solve this?
I am using Python 3.4.3 and pymongo 3.0.1.
Pymongo 3.0.1 returns aggregation results as cursor, which means you can't access the result with subquery['result']. To disable cursor and force pymongo to return a document with {'result':{...}} instead of a cursor, use this:
subquery = db.partsupp.aggregate([
{"$match": {"r_name": region }},
{"$group": {
"_id" : 0,
"minim": {"$min": "$supplycost"}
}
}
], useCursor=False)
From pymongo 4.0, useCursor is no longer available, use list() to convert cursor to a list:
cursor = db.partsupp.aggregate([
{"$match": {"r_name": region }},
{"$group": {
"_id" : 0,
"minim": {"$min": "$supplycost"}
}
}
])
subquery['result'] = list(cursor)
Since useCursor is deprecated and will be removed in PyMongo 4.0, I suggest iterating over the results:
subquery = db.partsupp.aggregate([
{"$match": {"r_name": region }},
{"$group": {
"_id" : 0,
"minim": {"$min": "$supplycost"}
}
}
])
results = [doc for doc in subquery]
Related
An existing collection like as below:
"_id" : "12345",
"vals" : {
"dynamickey1" : {}
}
I need to add
"vals" : {
"dynamickey2" : {}
}
I have tried in python 2.7 with pymongo 2.8:
col.update({'_id': id)},{'$push': {'vals': {"dynamickey2":{"values"}}}})
Error log:
pymongo.errors.OperationFailure: The field 'vals' must be an array but is of type object in document
Expected Output:
"_id" : "12345",
"vals" : {
"dynamickey1" : {},
"dynamickey2" : {}
}
Edited following question edit:
Two options; use $set with the dot notation, or use python dict manipulation.
The first method is more MongoDB native and is one line of code; the second is a bit more work but gives more flexilbility if you use case is more nuanced.
Method 1:
from pymongo import MongoClient
from bson.json_util import dumps
db = MongoClient()['mydatabase']
db.mycollection.insert_one({
"_id": "12345",
"vals": {
"dynamickey1": {},
}
})
db.mycollection.update_one({'_id': '12345'},{'$set': {'vals.dynamickey2':{}}})
print(dumps(db.mycollection.find_one({}), indent=4))
Method 2:
from pymongo import MongoClient
from bson.json_util import dumps
db = MongoClient()['mydatabase']
db.mycollection.insert_one({
"_id": "12345",
"vals": {
"dynamickey1": {},
}
})
record = db.mycollection.find_one({'_id': '12345'})
vals = record['vals']
vals['dynamickey2'] = {}
record = db.mycollection.update_one({'_id': record['_id']}, {'$set': {'vals': vals}})
print(dumps(db.mycollection.find_one({}), indent=4))
Either way gives:
{
"_id": "12345",
"vals": {
"dynamickey1": {},
"dynamickey2": {}
}
}
Previous answer
Your expected output has an object with duplicate fields (vals); this isn't allowed.~
So whatever you are trying to do, it isn't going to work.
Is there any way to use $cond along with ($set, $inc, ...) operators in update? (MongoDB 4.2)
I want to update a field in my document by $inc it with "myDataInt" if a condition comes true, otherwise keeps it as it is:
db.mycoll.update(
{"_id" : "5e9e5da03da783817d231dc4"},
{"$inc" : {
"my_data_sum" : {
"$cond" : [
{
"$ne" : ["snapshot_time", new_snapshot_time)]
},myDataInt, 0]
]
}
},
{upsert=True, multi=False}
)
However, this gives an error in pymongo:
raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: The dollar ($) prefixed field '$cond' in 'my_data_sum.$cond' is not valid for storage.
Any idea to avoid using find() before update in this case?
Update:
If I use the approach that Joe has mentioned, an exception will be raised in PyMongo (v3.10.1) due to using 'list' as a parameter in update_many() instead of 'dict':
from pymongo import MongoClient
db = MongoClient()['mydb']
db.mycoll.update_many(
{"_id" : "5e9e5da03da783817d231dc4"},
[{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}],
upsert:true
)
That ends up with this error:
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 1076, in update_many session=session),
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 856, in _update_retryable _update, session)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1491, in _retryable_write return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session return func(session, sock_info, retryable)
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 852, in _update retryable_write=retryable_write)
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 823, in _update _check_write_command_response(result)
File "/usr/local/lib64/python3.6/site-packages/pymongo/helpers.py", line 221, in _check_write_command_response _raise_last_write_error(write_errors)
File "/usr/local/lib64/python3.6/site-packages/pymongo/helpers.py", line 203, in _raise_last_write_error raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: Modifiers operate on fields but we found type array instead. For example: {$mod: {<field>: ...}} not {$set: [ { $set: { my_data_sum: { $sum: [ "$my_data_sum", { $cond: [ { $ne: [ "$snapshot_time", 1586910283 ] }, 1073741824, 0 ] } ] } } } ]}
If you are using MongoDB 4.2, you can use aggregation operators with updates. $inc is not an aggregation operator, but $sum is. To specify a pipeline, pass an array as the second argument to update:
db.coll.update(
{"_id" : "5e9e5da03da783817d231dc4"},
[{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}],
{upsert:true, multi:false}
)
After spending some time and searching online, I figured that the update_many(), update_one(), and update() methods of Collection object in PyMongo do not accept type list as parameters to support the new Aggregation Pipeline feature of the Update operation in MongoDB 4.2+. (At least this option is not available in PyMongo v3.10 yet.)
However, looks like I could use the command method of the Database object in PyMongo which is an instance of the (MongoDB runCommand) and it worked just fine for me:
from pymongo import MongoClient
db = MongoClient()['mydb']
result = db.command(
{
"update" : "mycoll",
"updates" : [{
"q" : {"_id" : "5e9e5da03da783817d231dc4"},
"u" : [
{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}
],
"upsert" : True,
"multi" : True
}],
"ordered": False
}
)
The command method of the database object gets a dict object of all the required commands as its first argument, and then the list of Aggregation Pipeline can be included inside the dict object (q is the update query, and the u defined the fields to be updated).
result is a dictionary of Ack message from MongoDB which contains 'nModified', 'upserted', and 'writeErrors'.
https://mongoplayground.net/p/1AklFKuhFi6
[
{
"id": 1,
"like": 3
},
{
"id": 2,
"like": 1
}
]
let value = 1,
if you want to increment then use
value = -1 * value
db.collection.aggregate([
{
"$match": {
"id": 1
}
},
{
"$set": {
"count": {
$cond: {
if: {
$gt: [
"$like",
0
]
},
then: {
"$subtract": [
"$like",
value
]
},
else: 0
}
}
}
}
])
I am a newbie to mongodb. I want to retrieve the datas of a certain fields on a specified date from mongodb using python. My Mongodb Collection looks like this
{
"_id" : ObjectId("5d9d7eec7c6265a42e352d6d"),
"browser" : "Chrome",
"countryCode" : "IN",
"Page" : "http://192.168.1.34/third.html",
"date" : "2019-10-09T10:32:08.438660"
}
{
"_id" : ObjectId("5d9d7eec7c6265a42e352d6e"),
"browser" : "Chrome",
"countryCode" : "IN",
"Page" : "http://192.168.1.14/fourth.html",
"date" : "2019-10-12T10:32:08.438662"
}
and so on
I retrieved the data from mongodb by using the following query in mongodb
db.collection_name.find({"date": {'$gte': "2019-10-09T10:32:08.438660", '$lte': "2019-10-10T10:32:08.438661"}},{}, {Page:[], _id:0})
I want to get that data using pymongo in python. Here's the Code I tried,
from pymongo import MongoClient
import pymongo
from bson.raw_bson import RawBSONDocument
myclient = pymongo.MongoClient(
"mongodb://localhost:27017/", document_class=RawBSONDocument)
mydb = myclient['smackcoders']
mycol = mydb['logs']
from_date = "2019-10-09T10:32:08.438663"
to_date = "2019-10-12T10:32:08.438671"
for doc in mycol.find({"date": {'$gte': from_date, '$lte': to_date}}, {}, {'Page': [], '_id': 0}):
print(doc)
It shows error:
Traceback (most recent call last):
File "temp3.py", line 20, in <module>
for doc in mycol.find({"date": {'$gte': from_date, '$lte': to_date}}, {}, {'url': [], '_id': 0}):
File "/home/paulsteven/.local/lib/python3.7/site-packages/pymongo/collection.py", line 1460, in find
return Cursor(self, *args, **kwargs)
File "/home/paulsteven/.local/lib/python3.7/site-packages/pymongo/cursor.py", line 145, in __init__
raise TypeError("skip must be an instance of int")
TypeError: skip must be an instance of int
Output Required:
["http://192.168.1.34/third.html","http://192.168.1.14/fourth.html",.....and goes on for a specified date]
I don't Know how to make it work. Query works in mongodb but in python, it fails. Help me with some solutions.
You've got 3 parameters in your find function; you probably only need 2; a query and a projection. The third parameter is skip which is why it's failing with that error.
Mongo shell only takes 2 parameters so it is likely ignoring the third which is why it looks like it is working.
I wondering how to convert the follow mongodb query to pymongo syntax
db.articles.find(
{ $text: { $search: "cake" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } ).limit(3)
I tried this:
results = \
mongo.db.products.find({ '$text': { '$search': 'cake' } }, { 'score': { '$meta': 'textScore' } }) \
.sort({ 'score': { '$meta': 'textScore' } }) \
.limit(3)
But I got the follow error on sort:
raise TypeError("second item in each key pair must be 1, -1, "
TypeError: second item in each key pair must be 1, -1, '2d', 'geoHaystack', or another valid MongoDB index specifier.
Anyone can help me?
Thanks in advance
I think that the solution is here: https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/cursor.py#L658 . To add list of (key, direction) for new approach with "new" feature '$text':
Beginning with MongoDB version 2.6, text search results can be
sorted by relevance::
cursor = db.test.find(
{'$text': {'$search': 'some words'}},
{'score': {'$meta': 'textScore'}})
# Sort by 'score' field.
cursor.sort([('score', {'$meta': 'textScore'})]) #<<<< HERE
I upgraded pymongo to the latest version, and it worked.
Assuming that this one item of my database:
{"_id" : ObjectID("526fdde0ef501a7b0a51270e"),
"info": "foo",
"status": true,
"subitems : [ {"subitem_id" : ObjectID("65sfdde0ef501a7b0a51e270"),
//more},
{....}
],
//more
}
I want to find (or find_one, doesn't matter) the document(s) with "subitems.subitem_id" : xxx.
I have tried the following. All of them return an empty list.
from pymongo import MongoClient,errors
from bson.objectid import ObjectId
id = '65sfdde0ef501a7b0a51e270'
db.col.find({"subitems.subitem_id" : id } ) #obviously wrong
db.col.find({"subitems.subitem_id" : Objectid(id) })
db.col.find({"subitems.subitem_id" : {"$oid":id} })
db.col.find({"subitems.subitem_id.$oid" : id })
db.col.find({"subitems.$.subitem_id" : Objectid(id) })
In mongoshell this one works however:
find({"subitems.subitem_id" : { "$oid" : "65sfdde0ef501a7b0a51e270" } })
The literal 65sfdde0ef501a7b0a51e270 is not hexadecimal, hence, not a valid ObjectId.
Also, id is a Python built-in function. Avoid reseting it.
Finally, you execute a find but do not evaluate it, so you do not see any results. Remember that pymongo cursors are lazy.
Try this.
from pymongo import MongoClient
from bson.objectid import ObjectId
db = MongoClient().database
oid = '65cfdde0ef501a7b0a51e270'
x = db.col.find({"subitems.subitem_id" : ObjectId(oid)})
print list(x)
Notice I adjusted oid to a valid hexadecimal string.
Same query in the Mongo JavaScript shell.
db.col.find({"subitems.subitem_id" : new ObjectId("65cfdde0ef501a7b0a51e270")})
Double checked. Right answer is db.col.find({"subitems.subitem_id" : Objectid(id)})
Be aware that this query will return full record, not just matching part of sub-array.
Mongo shell:
a = ObjectId("5273e7d989800e7f4959526a")
db.m.insert({"subitems": [{"subitem_id":a},
{"subitem_id":ObjectId()}]})
db.m.insert({"subitems": [{"subitem_id":ObjectId()},
{"subitem_id":ObjectId()}]})
db.m.find({"subitems.subitem_id" : a })
>>> { "_id" : ObjectId("5273e8e189800e7f4959526d"),
"subitems" :
[
{"subitem_id" : ObjectId("5273e7d989800e7f4959526a") },
{"subitem_id" : ObjectId("5273e8e189800e7f4959526c")}
]}