I wanted to add new keys to an existing object in a MongoDB docuemnt, I am trying to update the specific abject with update query but I don't see new keys in database.
I have a object like this:
{'_id': 'patent_1023',
'raw': {'id': 'CN-109897889-A',
'title': 'A kind of LAMP(ring mediated isothermal amplification) product visible detection method',
'assignee': '北京天恩泽基因科技有限公司',
'inventor/author': '徐堤',
'priority_date': '2019-04-17',
'filing/creation_date': '2019-04-17',
'publication_date': '2019-06-18',
'grant_date': None,
'result_link': 'https://patents.google.com/patent/CN109897889A/en', 'representative_figure_link': None
},
'source': 'Google Patent'}
I added two new keys in raw and want to update only 'raw' with new keys 'abstract' and 'description'
Here is what I have done.
d = client.find_one({'_id': {'$in': ids}})
d['raw'].update(missing_data) # missing_data contain new keys to be added in raw.
here = client.find_one_and_update({'_id': d['_id']}, {'$set': {"raw": d['raw']}})
Both update_one and update_many will work with this:
missing_data = {'abstract':'a book', 'description':'a fun book'};
ids = [ 'patent_1023', 'X'];
rc=db.foo.update_one(
{'_id': {'$in': ids}},
# Use pipeline form of update to exploit richer agg framework
# function like $mergeObjects. Below we are saying "take the
# incoming raw object, overlay the missing_data object on top of
# it, and then set that back into raw and save":
[ {'$set': {
'raw': {'$mergeObjects': [ '$$ROOT.raw', missing_data ] }
}}
]
)
Related
i added a document to marqo add_documents() but i didn't pass an id and now i am trying to get the document but i don't know what the document_id is?
Here is what my code look like:
mq = marqo.Client(url='http://localhost:8882')
mq.index("my-first-index").add_documents([
{
"Title": title,
"Description": document_body
}]
)
i tried to check whether the document got added or not but ;
no_of_docs = mq.index("my-first-index").get_stats()
print(no_of_docs)
i got;
{'numberOfDocuments': 1}
meaning it was added.
if you don't add the "_id" as part of key/value then by default marqo will generate a random id for you, to access it you can search the document using the document's Title,
doc = mq.index("my-first-index").search(title_of_your_document, searchable_attributes=['Title'])
you should get a dictionary as the result something like this;
{'hits': [{'Description': your_description,
'Title': title_of_your_document,
'_highlights': relevant part of the doc,
'_id': 'ac14f87e-50b8-43e7-91de-ee72e1469bd3',
'_score': 1.0}],
'limit': 10,
'processingTimeMs': 122,
'query': 'The Premier League'}
the part that says _id is the id of your document.
I'm currently trying to carry over object relationship functionality from a php backend to a python backend. In the php application you were able to have a call like so:
~api/v1/cats/5b40b6eb-538f-a015-9196-1cc63cf8d7ae
That would return a result like this:
{
id: "5b40b6eb-538f-a015-9196-1cc63cf8d7ae",
name: "mittens",
hatId: "0845de76-9d68-cfee-0ebb-473ccddf16bc",
}
The functionality that I am trying to replicate is that if you added an include param to the call with the name of foreign object like this:
~api/v1/cats/5b40b6eb-538f-a015-9196-1cc63cf8d7ae?include=hat
Then it would return the original object including the foreign key object
{
id: "5b40b6eb-538f-a015-9196-1cc63cf8d7ae",
name: "mittens",
hatId: "0845de76-9d68-cfee-0ebb-473ccddf16bc",
hat: {
id: "0845de76-9d68-cfee-0ebb-473ccddf16bc",
name: 'top hat',
}
}
All that I've come across so far is select_related, HyperlinkedRelatedField, and prefetch_related; none of these seem to be able to get the above result when implemented.
I would strongly recommend using Django Rest Framework, and specifically look at nested relationships.
The example given in the DRF docs matches what you're looking to do almost exactly (though note, it's a one-to-many field, while your example is a one-to-one field):
>>> album = Album.objects.create(album_name="The Grey Album", artist='Danger Mouse')
>>> Track.objects.create(album=album, order=1, title='Public Service Announcement', duration=245)
<Track: Track object>
>>> Track.objects.create(album=album, order=2, title='What More Can I Say', duration=264)
<Track: Track object>
>>> Track.objects.create(album=album, order=3, title='Encore', duration=159)
<Track: Track object>
>>> serializer = AlbumSerializer(instance=album)
>>> serializer.data
{
'album_name': 'The Grey Album',
'artist': 'Danger Mouse',
'tracks': [
{'order': 1, 'title': 'Public Service Announcement', 'duration': 245},
{'order': 2, 'title': 'What More Can I Say', 'duration': 264},
{'order': 3, 'title': 'Encore', 'duration': 159},
...
],
}
I'm running a script to get information from an URL and then create a Json file. With that, I'll read thought the info, save the ones that I need and then insert into the database.
But, I'm having problem with a part of this info.
Info from the URL, saved in JSON
"images": [
{
"type": "PosterPortrait",
"url": "https://ingresso-a.akamaihd.net/img/cinema/cartaz/22455-cartaz.jpg"
},
{
"type": "PosterHorizontal",
"url": "https://ingresso-a.akamaihd.net/img/cinema/cartaz/22455-destaque.jpg"
}
],
"trailers": []
This is the part and with that I have to:
Check if there is something inside trailers
Save it in a array, regarding for their "type" and "url"
Save them in my database
I made a code to do that, the problem is, I'm new to python and I don't think it's working quiet right.
insert-events.py
if(i['trailers'][0]):
a = array.arr(
array.arr('url' = i['images'][0]['url'], 'type' = i['images'][0]['type']),
array.arr('url' = i['images'][1]['url'], 'type' = i['images'][1]['type']),
array.arr('url' = i['trailers'][0]['url'], 'type' = Trailer),
array.arr('url' = i['trailers'][1]['url'], 'type' = Trailer),
)
else:
a = array.arr(
array.arr('url' = i['images'][0]['url'], 'type' = i['images'][0]['type']),
array.arr('url' = i['images'][1]['url'], 'type' = i['images'][1]['type']),
)
This is the part of the code to get the info from the JSON and, then, save it in as an array of arrays.
Error
File "insert-events.py", line 46
array.arr('url' = i['images'][0]['url'], 'type' = i['images'][0]['type']),
SyntaxError: keyword can't be an expression
So, what I'm doing it wrong? It's the array structure or when I try to save the info?
Any help would be appreciate. Thanks!
The error you are seeing is because it looks like you are trying to call a function/class with keyword arguments, but not using the keywords, instead using the string.
def foo(a, b):
pass
foo(a=1, b=2) # Correct
foo('a'=1, 'b'=2) # Incorrect
If you want to save stuff with key/value associations, I would recommend using a dict (or maybe a list of dicts in your case)
if(i['trailers'][0]):
a = [
{'url': i['images'][0]['url'], 'type': i['images'][0]['type']},
{'url': i['images'][1]['url'], 'type': i['images'][1]['type']},
] # etc.
else:
a = [
{'url': i['images'][0]['url'], 'type': i['images'][0]['type']},
{'url': i['images'][1]['url'], 'type': i['images'][1]['type']},
]
Lastly as a semi-unrelated note, checking for i['trailers'][0] will throw an IndexError if i['trailers'] is an empty list ([]). You can check for something in i['trailers'] using just if i['trailers']:.
I'm working on a python client program to Cloundant.
I'd like to retrieve a doc, not based on "_id",but on my own field.
Still, it does not work causing Key Error. Any help to solve this error is highly appreciated!
Here is my code:
from cloudant.client import Cloudant
from cloudant.error import CloudantException
from cloudant.result import Result,ResultByKey
...
client.connect()
databaseName = "mydata1"
myDatabase = client[databaseName]
# As direct access like 'doc = myDatabase[<_id>]' cannot work for my key,
# let's check on by one ...
for document in myDatabase:
# if document['_id']== "20170928chibikasmall": <= if I use _id it's ok
if document['gokigenField']== 111:
This cause
KeyError :'gokigenField'
In advance, I've created gokigenField index using dashboard, then confirm the result via my postman with REST API
GET https://....bluemix.cloudant.com/mydata1/_index
the result is as follows:
{"total_rows":2,"indexes":[{"ddoc":null,"name":"_all_docs","type":"special","def":{"fields":[{"_id":"asc"}]}},{"ddoc":"_design/f7fb53912eb005771b736422f41c24cd26c7f06a","name":"gokigen-index","type":"text","def":{"default_analyzer":"keyword","default_field":{},"selector":{},"fields":[{"gokigenField":"number"}],"index_array_lengths":true}}]}
Also, I've confirmed I can use this gokigenField as query index nicely on cloudant dashboard as well as POST query .
My newly created "gokigenField" is not included in all the document in DB, as there are automatically created doc ("_design/xxx) without that field.
I guess this might cause KeyError, when I call this from my Python client.
I cannot find Cloudant API for checking 'if a specific key exists or not in a document', in the reference.. So, cannot have any idea how to by-pass such docs...
This is how to index an query data from the Python client. Let's assume we already have the library imported and have a database client in myDatabase.
First of all I created some data:
#create some data
data = { 'name': 'Julia', 'age': 30, 'pets': ['cat', 'dog', 'frog'], 'gokigenField': 'a' }
myDatabase.create_document(data)
data = { 'name': 'Fred', 'age': 30, 'pets': ['dog'], 'gokigenField': 'b' }
myDatabase.create_document(data)
data = { 'name': 'Laura', 'age': 31, 'pets': ['cat'], 'gokigenField': 'c' }
myDatabase.create_document(data)
data = { 'name': 'Emma', 'age': 32, 'pets': ['cat', 'parrot', 'hamster'], 'gokigenField': 'c' }
myDatabase.create_document(data)
We can check the data is there in the Cloudant dashboard or by doing:
# check the data is there
for document in myDatabase:
print(document)
Next we can opt to index the field gokigenField like so:
# create an index on the field 'gokigenField'
mydb.create_query_index(fields=['gokigenField'])
Then we can query the database:
# do a query
selector = {'gokigenField': {'$eq': 'c'}}
docs = mydb.get_query_result(selector)
for doc in docs:
print (doc)
which outputs the two matching documents.
The python-cloudant documentation is here.
I have a pretty big dictionary which looks like this:
{
'startIndex': 1,
'username': 'myemail#gmail.com',
'items': [{
'id': '67022006',
'name': 'Adopt-a-Hydrant',
'kind': 'analytics#accountSummary',
'webProperties': [{
'id': 'UA-67522226-1',
'name': 'Adopt-a-Hydrant',
'websiteUrl': 'https://www.udemy.com/,
'internalWebPropertyId': '104343473',
'profiles': [{
'id': '108333146',
'name': 'Adopt a Hydrant (Udemy)',
'type': 'WEB',
'kind': 'analytics#profileSummary'
}, {
'id': '132099908',
'name': 'Unfiltered view',
'type': 'WEB',
'kind': 'analytics#profileSummary'
}],
'level': 'STANDARD',
'kind': 'analytics#webPropertySummary'
}]
}, {
'id': '44222959',
'name': 'A223n',
'kind': 'analytics#accountSummary',
And so on....
When I copy this dictionary on my Jupyter notebook and I run the exact same function I run on my django code it runs as expected, everything is literarily the same, in my django code I'm even printing the dictionary out then I copy it to the notebook and run it and I get what I'm expecting.
Just for more info this is the function:
google_profile = gp.google_profile # Get google_profile from DB
print(google_profile)
all_properties = []
for properties in google_profile['items']:
all_properties.append(properties)
site_selection=[]
for single_property in all_properties:
single_propery_name=single_property['name']
for single_view in single_property['webProperties'][0]['profiles']:
single_view_id = single_view['id']
single_view_name = (single_view['name'])
selections = single_propery_name + ' (View: '+single_view_name+' ID: '+single_view_id+')'
site_selection.append(selections)
print (site_selection)
So my guess is that my notebook has some sort of json parser installed or something like that? Is that possible? Why in django I can't access dictionaries the same way I can on my ipython notebooks?
EDITS
More info:
The error is at the line: for properties in google_profile['items']:
Django debug is: TypeError at /gconnect/ string indices must be integers
Local Vars are:
all_properties =[]
current_user = 'myemail#gmail.com'
google_profile = `the above dictionary`
So just to make it clear for who finds this question:
If you save a dictionary in a database django will save it as a string, so you won't be able to access it after.
To solve this you can re-convert it to a dictionary:
The answer from this post worked perfectly for me, in other words:
import json
s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
json_acceptable_string = s.replace("'", "\"")
d = json.loads(json_acceptable_string)
# d = {u'muffin': u'lolz', u'foo': u'kitty'}
There are many ways to convert a string to a dictionary, this is only one. If you stumbled in this problem you can quickly check if it's a string instead of a dictionary with:
print(type(var))
In my case I had:
<class 'str'>
before converting it with the above method and then I got
<class 'dict'>
and everything worked as supposed to