Is there any way to use $cond along with ($set, $inc, ...) operators in update? (MongoDB 4.2)
I want to update a field in my document by $inc it with "myDataInt" if a condition comes true, otherwise keeps it as it is:
db.mycoll.update(
{"_id" : "5e9e5da03da783817d231dc4"},
{"$inc" : {
"my_data_sum" : {
"$cond" : [
{
"$ne" : ["snapshot_time", new_snapshot_time)]
},myDataInt, 0]
]
}
},
{upsert=True, multi=False}
)
However, this gives an error in pymongo:
raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: The dollar ($) prefixed field '$cond' in 'my_data_sum.$cond' is not valid for storage.
Any idea to avoid using find() before update in this case?
Update:
If I use the approach that Joe has mentioned, an exception will be raised in PyMongo (v3.10.1) due to using 'list' as a parameter in update_many() instead of 'dict':
from pymongo import MongoClient
db = MongoClient()['mydb']
db.mycoll.update_many(
{"_id" : "5e9e5da03da783817d231dc4"},
[{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}],
upsert:true
)
That ends up with this error:
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 1076, in update_many session=session),
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 856, in _update_retryable _update, session)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1491, in _retryable_write return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session return func(session, sock_info, retryable)
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 852, in _update retryable_write=retryable_write)
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 823, in _update _check_write_command_response(result)
File "/usr/local/lib64/python3.6/site-packages/pymongo/helpers.py", line 221, in _check_write_command_response _raise_last_write_error(write_errors)
File "/usr/local/lib64/python3.6/site-packages/pymongo/helpers.py", line 203, in _raise_last_write_error raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: Modifiers operate on fields but we found type array instead. For example: {$mod: {<field>: ...}} not {$set: [ { $set: { my_data_sum: { $sum: [ "$my_data_sum", { $cond: [ { $ne: [ "$snapshot_time", 1586910283 ] }, 1073741824, 0 ] } ] } } } ]}
If you are using MongoDB 4.2, you can use aggregation operators with updates. $inc is not an aggregation operator, but $sum is. To specify a pipeline, pass an array as the second argument to update:
db.coll.update(
{"_id" : "5e9e5da03da783817d231dc4"},
[{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}],
{upsert:true, multi:false}
)
After spending some time and searching online, I figured that the update_many(), update_one(), and update() methods of Collection object in PyMongo do not accept type list as parameters to support the new Aggregation Pipeline feature of the Update operation in MongoDB 4.2+. (At least this option is not available in PyMongo v3.10 yet.)
However, looks like I could use the command method of the Database object in PyMongo which is an instance of the (MongoDB runCommand) and it worked just fine for me:
from pymongo import MongoClient
db = MongoClient()['mydb']
result = db.command(
{
"update" : "mycoll",
"updates" : [{
"q" : {"_id" : "5e9e5da03da783817d231dc4"},
"u" : [
{"$set" : {
"my_data_sum" : {
"$sum": [
"$my_data_sum",
{"$cond" : [
{"$ne" : ["snapshot_time", new_snapshot_time]},
myDataInt,
0
]}
]
}
}}
],
"upsert" : True,
"multi" : True
}],
"ordered": False
}
)
The command method of the database object gets a dict object of all the required commands as its first argument, and then the list of Aggregation Pipeline can be included inside the dict object (q is the update query, and the u defined the fields to be updated).
result is a dictionary of Ack message from MongoDB which contains 'nModified', 'upserted', and 'writeErrors'.
https://mongoplayground.net/p/1AklFKuhFi6
[
{
"id": 1,
"like": 3
},
{
"id": 2,
"like": 1
}
]
let value = 1,
if you want to increment then use
value = -1 * value
db.collection.aggregate([
{
"$match": {
"id": 1
}
},
{
"$set": {
"count": {
$cond: {
if: {
$gt: [
"$like",
0
]
},
then: {
"$subtract": [
"$like",
value
]
},
else: 0
}
}
}
}
])
Related
i tried this but it didn't work, i dont know how to get the array labels which is in the object chartData to pass the data to it
state = {
chartData: {
labels:[],
datasets:[{
label:'items',
data:[]
}]
}
}
componentDidMount() {
axios.get('http://127.0.0.1:8000/orders/chart/').then(result => {
this.setState({
labels:result.data.label,
data:result.data.quantity
});
console.log(this.state.chartData)
});
}
and console.log() gave me an empty arrays and here is the result
Object
labels: Array(0)
length: 0
__proto__: Array(0)
datasets: Array(1)
0: {label: "items", data: Array(0), _meta: {…}}
length: 1
__proto__: Array(0)
__proto__: Object
and here is my backend function:
class chart(ListAPIView):
serializer_class=chartserializer
def get(self,*args,**kwrgs):
names=[]
quant=[]
od=order_details.objects.order_by('product__name','quantity').distinct('product__name')
for i in od :
name=products.objects.get(name=i.product.name)
qty=list(order_details.objects.filter(product=i.product).aggregate(Sum('quantity')).values())[0]
names.append(name.name)
quant.append(qty)
data={
"label":names,
"quantity":quant
}
return Response(data)
and it's result:
{
"label": [
"dress",
"jacket"
],
"quantity": [
10,
17
]
}
From what I see, I think you are not organizing you data correctly when returning a json. They should be grouped by order_detail so you can get meaning from each of them i.e. I would do return something like this from backend.
[
{
"label": "dress",
"quantity": 10
},
{
"label": "jacket",
"quantity": 17
},
]
There your json is self explaining. You have an array of order_details, each object inside the array is an order_detail which has a label and a quantity.
Then in the frontend you would just iterate on each of them. Otherwise you will have to keep indexes to refer to each of them since you are receiving two arrays of not directly related values.
this being used in arrow brackets will point to the function result
so your function should be
componentDidMount() {
self = this;
axios.get('http://127.0.0.1:8000/orders/chart/').then(result => {
self.setState({
labels:result.data.label,
data:result.data.quantity
});
console.log(this.state.chartData)
});
}
I need to get a value inside an url (/some/url/value as a Sub Resource) usable as a parameter in an aggregation $match :
event/mac/11:22:33:44:55:66 --> {value:'11:22:33:44:55:66'}
and then:
{"$match":{"MAC":"$value"}},
here is a non-working example :
event = {
'url': 'event/mac/<regex("([\w:]+)"):value>',
'datasource': {
'source':"event",
'aggregation': {
'pipeline': [
{"$match": {"MAC":"$value"}},
{"$group": {"_id":"$MAC", "total": {"$sum": "$count"}}},
]
}
}
}
this example is working correctly with :
event/mac/blablabla?aggregate={"$value":"aa:11:bb:22:cc:33"}
any suggestion ?
The real quick and easy way would be to
path = "event/mac/11:22:33:44:55:66"
value = path.replace("event/mac/", "")
# or
value = path.split("/")[-1]
I need to run the following query on a MongoDB server:
QUERY = {
"$and" : [
{"x" : {'$gt' : 1.0}},
{"y" : {'$gt' : 0.1}},
{"$where" : 'this.s1.length < this.s2.length+3'}
]
}
This query is very slow, due to the JavaScript expression which the server needs to execute on every document in the collection.
Is there any way for me to optimize it?
I thought about using the $size operator, but I'm not really sure that it works on strings, and I'm even less sure on how to compare its output on a pair of strings (as is the case here).
Here is the rest of my script, in case needed:
from pymongo import MongoClient
USERNAME = ...
PASSWORD = ...
SERVER_NAME = ...
DATABASE_NAME = ...
COLLECTION_NAME = ...
uri = 'mongodb://{}:{}#{}/{}'.format(USERNAME,PASSWORD,SERVER_NAME,DATABASE_NAME)
mongoClient = MongoClient(uri)
collection = mongoClient[DATABASE_NAME][COLLECTION_NAME]
cursor = collection.find(QUERY)
print cursor.count()
The pymongo version is 3.4.
You can use aggregation framework, which provides $strLenCP to get length of a string and $cmp to compare them:
db.collection.aggregate(
[
{
$match: {
"x" : {'$gt' : 1.0},
"y" : {'$gt' : 0.1}
}
},
{
$addFields: {
str_cmp: { $cmp: [ { $strLenCP: "$s1" }, { $add: [ { $strLenCP: "$s2" }, 3 ] } ] }
}
},
{
$match: {
"str_cmp": -1,
}
}
]
)
I'm unwinding one field which is an array of date objects, however in some cases there are empty array's which is fine. I'd like the same treatment using a pipeline, but in some cases, I want to filter the results which have an empty array.
pipeline = []
pipeline.append({"$unwind": "$date_object"})
pipeline.append({"$sort": {"date_object" : 1}})
I want to use the pipeline format, however the following code does not return any records:
pipeline.append({"$match": {"date_object": {'$exists': False }}})
nor does the following work:
pipeline.append({"$match": {"date_object": []}})
and then:
results = mongo.db.xxxx.aggregate(pipeline)
I'm also trying:
pipeline.append({ "$cond" : [ { "$eq" : [ "$date_object", [] ] }, [ { '$value' : 0 } ], '$date_object' ] } )
But with this I get the following error:
.$cmd failed: exception: Unrecognized pipeline stage name: '$cond'
However if I query using find such as find({"date_object": []}), I can get these results. How can I make this work with the pipeline.
I've done in MongoDB shell, but it can be translated into Python easily in python language.
Is it your requirements?
I suppose you have such structure:
db.collection.save({foo:1, date_object:[new Date(), new Date(2016,1,01,1,0,0,0)]})
db.collection.save({foo:2, date_object:[new Date(2016,0,16,1,0,0,0),new Date(2016,0,5,1,0,0,0)]})
db.collection.save({foo:3, date_object:[]})
db.collection.save({foo:4, date_object:[new Date(2016,1,05,1,0,0,0), new Date(2016,1,06,1,0,0,0)]})
db.collection.save({foo:5, date_object:[]})
// Get empty arrays after unwind
db.collection.aggregate([
{$project:{_id:"$_id", foo:"$foo",
date_object:{
$cond: [ {"$eq": [{ $size:"$date_object" }, 0]}, [null], "$date_object" ]
}
}
},
{$unwind:"$date_object"},
{$match:{"date_object":null}}
])
// Get empty arrays before unwind
db.collection.aggregate([
{$match:{"date_object.1":{$exists:false}}},
{$project:{_id:"$_id", foo:"$foo",
date_object:{
$cond: [ {"$eq": [{ $size:"$date_object" }, 0]}, [null], "$date_object" ]
}
}
},
{$unwind:"$date_object"}
])
Only empty date_object
[
{
"_id" : ObjectId("56eb0bd618d4d09d4b51087a"),
"foo" : 3,
"date_object" : null
},
{
"_id" : ObjectId("56eb0bd618d4d09d4b51087c"),
"foo" : 5,
"date_object" : null
}
]
At the end, if you need only empty date_object, you don't need to aggregate, you can easely achieve it with find:
db.collection.find({"date_object.1":{$exists:false}},{date_object:0})
Output
{
"_id" : ObjectId("56eb0bd618d4d09d4b51087a"),
"foo" : 3
}
{
"_id" : ObjectId("56eb0bd618d4d09d4b51087c"),
"foo" : 5
}
I have a Mongo Collection that I need to update, and I'm trying to use the collection.update command to no avail.
Code below:
import pymongo
from pymongo import MongoClient
client = MongoClient()
db = client.SensorDB
sensors = db.Sensor
for sensor in sensors.find():
lat = sensor['location']['latitude']
lng = sensor['location']['longitude']
sensor['location'] = {
"type" : "Feature",
"geometry" : {
"type" : "Point",
"coordinates" : [lat ,lng]
},
"properties": {
"name": sensor['name']
}
}
sensors.update({'webid': sensor['webid']} , {"$set": sensor}, upsert=True)
However, running this gets me the following:
Traceback (most recent call last):
File "purgeDB.py", line 21, in <module>
cameras.update({'webid': sensor['webid']} , {"$set": sensor}, upsert=True)
File "C:\Anaconda\lib\site-packages\pymongo\collection.py", line 561, in update
check_keys, self.uuid_subtype), safe)
File "C:\Anaconda\lib\site-packages\pymongo\mongo_client.py", line 1118, in _send_message
rv = self.__check_response_to_last_error(response, command)
File "C:\Anaconda\lib\site-packages\pymongo\mongo_client.py", line 1060, in __check_response_to_last_error
raise OperationFailure(details["err"], code, result)
pymongo.errors.OperationFailure: Mod on _id not allowed
Change this line:
for sensor in sensors.find():
to this:
for sensor in sensors.find({}, {'_id': 0}):
What this does is prevent Mongo from returning the _id field, since you aren't using it, and it's causing your problem later in your update() call since you cannot "update" _id.
An even better solution (Only write the data that is needed)
for sensor in sensors.find():
lat = sensor['location']['latitude']
lng = sensor['location']['longitude']
location = {
"type" : "Feature",
"geometry" : {
"type" : "Point",
"coordinates" : [lat ,lng]
},
"properties": {
"name": sensor['name']
}
}
sensors.update({'webid': sensor['webid']} , {"$set": {'location': location}})
Edit:
As mentioned by Loïc Faure-Lacroix, you also do not need the upsert flag in your case - your code in this case is always updating, and never inserting.
Edit2:
Surrounded _id in quotes for first solution.