I am having trouble updating document in MongoDB that involves adding to list and updating some fields, using Pymongo.
To summarize, I would like to:
Add a value to the a list.
Update some fields.
Using a single update statement.
I have tried 2 methods, but both doesn't work:
key = {'username':'user1'}
user_detail = {
'name':{'first':'Marie', 'last':'Bender'},
'items':{'$addtoset':{'cars':'BMW'}}
}
user_detail2 = {
'name':{'first':'Marie', 'last':'Bender'},
'$addtoset':{'items.cars':'BMW'}
}
mongo_collection.update(key, user_detail, upsert=True)
mongo_collection.update(key, user_detail2, upsert=True)
error message: dollar ($) prefixed field '$addToSet' in '$addToSet' is not valid for storage.
My intended outcome:
Before:
{
'username':'user1',
'item': {'cars':['Merc','Ferrari'],'house':1}
}
Intended After:
{
'username':'user1',
'name': {'first':'Marie', 'last':'Bender'},
'item': {'cars':['Merc','Ferrari','BMW'],'house':1}
}
Your second attempt is closer, but you need to use the $set operator to set the value of name:
user_detail2 = {
'$set': {'name': {'first': 'Marie', 'last': 'Bender'}},
'$addtoset': {'items.cars': 'BMW'}
}
Related
How do I get the value of a dict item within a list, within a dict in Python? Please see the following code for an example of what I mean.
I use the following lines of code in Python to get data from an API.
res = requests.get('https://api.data.amsterdam.nl/bag/v1.1/nummeraanduiding/', params)
data = res.json()
data then returns the following Python dictionary:
{
'_links': {
'next': {
'href': null
},
'previous': {
"href": null
},
'self': {
'href': 'https://api.data.amsterdam.nl/bag/v1.1/nummeraanduiding/'
}
},
'count': 1,
'results': [
{
'_display': 'Maple Street 99',
'_links': {
'self': {
'href': 'https://api.data.amsterdam.nl/bag/v1.1/nummeraanduiding/XXXXXXXXXXXXXXXX/'
}
},
'dataset': 'bag',
'landelijk_id': 'XXXXXXXXXXXXXXXX',
'type_adres': 'Hoofdadres',
'vbo_status': 'Verblijfsobject in gebruik'
}
]
}
Using Python, how do I get the value for 'landelijk_id', represented by the twelve Xs?
This should work:
>>> data['results'][0]['landelijk_id']
"XXXXXXXXXXXXXXXX"
You can just chain those [] for each child you need to access.
I'd recommend using the jmespath package to make handling nested Dictionaries easier. https://pypi.org/project/jmespath/
import jmespath
import requests
res = requests.get('https://api.data.amsterdam.nl/bag/v1.1/nummeraanduiding/', params)
data = res.json()
print(jmespath.search('results[].landelijk_id', data)
When I use the watch() function on my collection, I am passing a aggregation to filter what comes through. I was able to get operationType to work correctly, but I also only want to include documents in which the city field is equal to Vancouver. The current syntax I am using does not work:
change_stream = client.mydb.mycollection.watch([
{
'$match': {
'operationType': { '$in': ['replace', 'insert'] },
'fullDocument': {'city': {'$eq': 'Vancouver'} }
}
}
])
And for reference, this is the what the dictionary that I'm aggregating looks like:
{'_id': {'_data': '825F...E0004'},
'clusterTime': Timestamp(1595565179, 2),
'documentKey': {'_id': ObjectId('70fc7871...')},
'fullDocument': {'_id': ObjectId('70fc7871...'),
'city': 'Vancouver',
'ns': {'coll': 'notification', 'db': 'pipeline'},
'operationType': 'replace'}
I found I just have to use a dot to access the nested dictionary:
change_stream = client.mydb.mycollection.watch([
{
'$match': {
'operationType': { '$in': ['replace', 'insert'] },
'fullDocument.city': 'Vancouver' }
}
}
])
So say i have the following document:
test_obj = {
'my_things':{
'id17': {
'blah': 3,
'weird': 'yay',
'thechallenge': ObjectId('5712d06fdb4d0856551300d2')
},
'id32': {
'blah': 62,
'weird': 'hoorah',
'thechallenge': ObjectId('5712d06fdb4d0856551300d4')
}
},
'_id': 12,
'an_extra_field': 'asdf'
}
for this document i have the following schema:
API.config['DOMAIN']['test_obj']['schema'] = {
'id': {'type': 'int'},
'an_extra_field': {'type': 'string'},
'my_things': {
'type': 'dict',
'valueschema': {
'type': 'dict',
'schema': {
'blah': {'type': 'dict'},
'weird': {'type': 'string'},
'thechallenge': {'type': 'objectid'}
}
}
}
}
Now say i make a patch with the following pseudocode:
data = {
'mythings': {
'id17': {
'thechallenge': '5712d06fdb4d0856551300d8'
}
}
}
PATCH(url='/v1/test_objs/12', data=data)
When I make this patch Cerberus raises an error during validation, saying "value '5712d06fdb4d0856551300d8' cannot be converted to a ObjectId". Now this is a valid object id, and i find that if I make a patch to other non-valueschema fields it does not raise this error. It seems like valueschema was not meant to have a value of dict, and adding an extra 'schema' attribute was the only way i could get around cerberus raising a schemaerror/having cerberus actually validate my fields. But eve does not appear to actually be serializing my fields in my dictionary correctly. It should be of type ObjectId when it gets passed to Cerberus.
The way i'm temporarily getting around this is by manipulating my the code in Eve. In common.py (module) in serialize (function) in line 398 i added, where it checks if the field schema is a 'valueschema':
elif field_type == 'dict' and 'schema' in field_schema['valueschema']:
for subdocument in document[field].values():
serialize(subdocument, schema=field_schema['valueschema']['schema'])
Should i not be using type dict for the valueschema? If not how else should i handle this scenario? I would like to not have to maintain my own fork of Eve, so if others do want the ability to have valueschema be of type dict should i submit a pull-request for this change?
This has been fixed with Eve v0.6.4, which has just been released.
I'm need a little help reordering this JSON data:
I've this file at this moment:
[{"aName":{"name":"somename","vendor":"vendor1"}},
{"bName":{"name":"somename","vendor":"vendor2"}},
{"cName":{"name":"othername","vendor":"vendor1"}
}]
I need re order it like this:
[{"name":"vendor1"},
{"child":[{"name":"somename","id":"aName"},
{"name":"othername","id":"cName"}]},
{"name":"vendor2"},
{"child":[{"name":"somename","id":"bName"}]}]
I was trying with this:
new format = []
for i in old_format:
new_format.setdefault(i['vendor'], []).append({"children":{"name":i['name'],"id":i}})
it's "closer", but not what I want
{
"vendor1":[
{
"children":{
"name":"somename",
"id":"aName"
}
},
{
"children":{
"name":"othername",
"id":"cName"
}
}
],
"vendor2":[
{
"children":{
"name":"somename",
"id":"bName"
}
}
]
}
I'll appreciate any advice.
The data representation you're using is a little strange... You have a lot of dictionaries with one entry, which suggests to me that you'd be better off rethinking how you store your data.
That said, this bit of code should do what you want:
vendors = {}
for entry in data:
for identifier, info in entry.items():
children = vendors.setdefault(info['vendor'], [])
children.append({
'name': info['name'],
'id': identifier
})
res = []
for vendor, children in vendors.items():
res.append({'name': vendor})
res.append({'child': children})
Here data is your input -- a list of dictionaries -- and res is the result:
[{'name': 'vendor2'},
{'child': [{'id': 'bName', 'name': 'somename'}]},
{'name': 'vendor1'},
{'child': [{'id': 'aName', 'name': 'somename'},
{'id': 'cName', 'name': 'othername'}]}]
I wondering how to convert the follow mongodb query to pymongo syntax
db.articles.find(
{ $text: { $search: "cake" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } ).limit(3)
I tried this:
results = \
mongo.db.products.find({ '$text': { '$search': 'cake' } }, { 'score': { '$meta': 'textScore' } }) \
.sort({ 'score': { '$meta': 'textScore' } }) \
.limit(3)
But I got the follow error on sort:
raise TypeError("second item in each key pair must be 1, -1, "
TypeError: second item in each key pair must be 1, -1, '2d', 'geoHaystack', or another valid MongoDB index specifier.
Anyone can help me?
Thanks in advance
I think that the solution is here: https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/cursor.py#L658 . To add list of (key, direction) for new approach with "new" feature '$text':
Beginning with MongoDB version 2.6, text search results can be
sorted by relevance::
cursor = db.test.find(
{'$text': {'$search': 'some words'}},
{'score': {'$meta': 'textScore'}})
# Sort by 'score' field.
cursor.sort([('score', {'$meta': 'textScore'})]) #<<<< HERE
I upgraded pymongo to the latest version, and it worked.