how do i insert data to a specific user in python flask mongodb
{
"_id" : ObjectId("5e208aa1f86973bbd7db6e8a"),
"worker_name" : "user1",
"location" : "location1",
"detection" : [ ]
},
{
"_id" : ObjectId("5e208aa1f86973bbd7db6e8b"),
"worker_name" : "user2",
"location" : "location2",
"detection" : [ ]
}
above are my users, I want to insert some data in user1 detection list, below are my code i had tried
def face_detection():
face_module = mongo.db.face_modules
user = mongo.db.users
stream_link = request.form['stream_link']
location = request.form['location']
camera = request.form['camera']
result = {
"location": location,
"stream_url": stream_link,
"worker_name": "user1",
"date": "1/1/2020",
"hour": "9",
"minute": "10",
"second": "25"
}
if user.find({"worker_name": result['worker_name']}).count() > 0:
update_user = user.detection.insert(result)
output = "user updated"
return jsonify({'result': output})
Since the document already exists you need to update it rather than insert, Also as you wanted to insert something into an array it should be through $push, Plus instead of making two DB calls you can use find_one_and_update which will return updated document with this option :: return_document = ReturnDocument.AFTER or will return none incase of no matching document found. Based on that you can return the response. In general you would use insert or insert_one for inserting a new document to colleciton. I'm a bit new to pymongo, Please add the code to check error scenario from DB, Plus test this code & you're feel free to update this answer with any findings..
Try this :
def face_detection():
face_module = mongo.db.face_modules
user = mongo.db.users
stream_link = request.form['stream_link']
location = request.form['location']
camera = request.form['camera']
result = {
"location": location,
"stream_url": stream_link,
"worker_name": "user1",
"date": "1/1/2020",
"hour": "9",
"minute": "10",
"second": "25"
}
resp = user.find_one_and_update(
{"worker_name": result['worker_name']},
{ '$push': {'detection' : result} },
return_document = ReturnDocument.AFTER)
if resp :
return jsonify({'result': resp})
else :
return jsonify({'result': 'No document found'})
Related
The dictionary I am trying to iterate through has the following structure:
d = {
"main_key_1": {
"name": "Name1",
"context": "Context1",
"message": "Message1",
"date": "Date1",
"reference": "Reference1"
},
"main_key_2": {
"name": "Name2",
"context": "Context2",
"message": "Message2",
"date": "Date2",
"reference": "Reference2"
}
}
This is the way I tried to iterate:
for item in d.items():
from_context = f"from {item[1]['context']}"
with context('given a descriptor'):
with context(from_context):
with before.all:
self.descriptor = item[1]['message']
with context('that contains a date'):
with it('recognizes the date'):
adapter = MessageToDatetAdapter(self.descriptor)
result = adapter.is_a_date()
expect(result).to(equal(True))
with it('extracts the date data'):
adapter = MessageToDatetAdapter(self.descriptor)
result = adapter.adapt()
expect(result['date']).to(equal(item[1]['date']))
expect(result['reference']).to(item[1]['reference'])
The first iteration would be something like below:
with context('given a descriptor'):
with context('from Context1'):
with before.all:
self.descriptor = 'Message1'
with context('that contains a date'):
with it('recognizes the date'):
adapter = MessageToDatetAdapter(self.descriptor)
result = adapter.is_a_date()
expect(result).to(equal(True))
with it('extracts the date data'):
adapter = MessageToDatetAdapter(self.descriptor)
result = adapter.adapt()
expect(result['date']).to('Date1')
expect(result['reference']).to('Reference1')
However, it seems like this is not correct. It looks like I cannot iterate through all the dictionary items.
I have elasticsearch documents like below where I need to rectify age value based on creationtime currentdate
age = creationtime - currentdate
:
hits = [
{
"_id":"CrRvuvcC_uqfwo-WSwLi",
"creationtime":"2018-05-20T20:57:02",
"currentdate":"2021-02-05 00:00:00",
"age":"60 months"
},
{
"_id":"CrRvuvcC_uqfwo-WSwLi",
"creationtime":"2013-07-20T20:57:02",
"currentdate":"2021-02-05 00:00:00",
"age":"60 months"
},
{
"_id":"CrRvuvcC_uqfwo-WSwLi",
"creationtime":"2014-08-20T20:57:02",
"currentdate":"2021-02-05 00:00:00",
"age":"60 months"
},
{
"_id":"CrRvuvcC_uqfwo-WSwLi",
"creationtime":"2015-09-20T20:57:02",
"currentdate":"2021-02-05 00:00:00",
"age":"60 months"
}
]
I want to do bulk update based on each document ID, but the problem is I need to correct 6 months of data & per data size (doc count of Index) is almost 535329, I want to efficiently do bulk update on age based on _id for each day on all documents using python.
Is there a way to do this, without looping through, all examples I came across using Pandas dataframes for update is based on a known value. But here _id I will get as and when the code runs.
The logic I had written was to fetch all doc & store their _id & then for each _id update the age . But its not an efficient way if I want to update all documents in bulk for each day of 6 months.
Can anyone give me some ideas for this or point me in the right direction.
As mentioned in the comments, fetching the IDs won't be necessary. You don't even need to fetch the documents themselves!
A single _update_by_query call will be enough. You can use ChronoUnit to get the difference after you've parsed the dates:
POST your-index-name/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": """
def created = LocalDateTime.parse(ctx._source.creationtime, DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss"));
def currentdate = LocalDateTime.parse(ctx._source.currentdate, DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
def months = ChronoUnit.MONTHS.between(created, currentdate);
ctx._source._age = months + ' month' + (months > 1 ? 's' : '');
""",
"lang": "painless"
}
}
The official python client has this method too. Here's a working example.
🔑 Try running this update script on a small subset of your documents before letting in out on your whole index by adding a query other than the match_all I put there.
💡 It's worth mentioning that unless you search on this age field, it doesn't need to be stored in your index because it can be calculated at query time.
You see, if your index mapping's dates are properly defined like so:
{
"mappings": {
"properties": {
"creationtime": {
"type": "date",
"format": "yyyy-MM-dd'T'HH:mm:ss"
},
"currentdate": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
...
}
}
}
the age can be calculated as a script field:
POST ttimes/_search
{
"query": {
"match_all": {}
},
"script_fields": {
"age_calculated": {
"script": {
"source": """
def months = ChronoUnit.MONTHS.between(
doc['creationtime'].value,
doc['currentdate'].value );
return months + ' month' + (months > 1 ? 's' : '');
"""
}
}
}
}
The only caveat is, the value won't be inside of the _source but rather inside of its own group called fields (which implies that more script fields are possible at once!).
"hits" : [
{
...
"_id" : "FFfPuncBly0XYOUcdIs5",
"fields" : {
"age_calculated" : [ "32 months" ] <--
}
},
...
I have a scenario where I am trying to extract data from json response which is obtained from the GET request and then rebuilding the json data by changing some values and then sending a PUT request at same time after rebuilding the json data(i.e, after changing idter value)
below is the target json response.
target_json = {
"name": "toggapp",
"ts": [
1234,
3456
],
"gs": [
{
"id": 4491,
"con": "mno"
},
{
"id": 4494,
"con": "hkl"
}
],
"idter": 500,
"datapart": false
}
from the above json I am trying to change the idter value to my custom value and rebuild it into json data again and post the new json data.
Here is what I have tried :
headers = {'Authorization': 'bearer ' + auth_token, 'Content-Type':'application/json', 'Accept':'application/json'}
tesstid =[7865, 7536, 7789]
requiredbdy = []
for key in testid:
get_metadata_targetjson= requests.get('https://myapp.com/%s' %key, headers = headers)
metadata=get_metadata_target.json()
for key1 in metadata:
requiredbdy.append(
{
"metadata" : [{
"name": key1['name'],
"ts": key1['ts'],
"gs": key1[gs],
"idter": 100, #custom value which I want to change
"datapart": false
} ]
}
)
send_metadata_newjson= requests.put('https://myapp.com/%s' %key, headers = headers data = requiredbdy)
print(send_metadata_newjson.status_code)
Is this approach fine or How do I proceed in order to achieve this scenario.
You can use the built-in json module for this like so
import json
my_json = """
{
"name": "toggapp",
"ts": [
1234,
3456
],
"gs": [
{
"id": 4491,
"con": "mno"
},
{
"id": 4494,
"con": "hkl"
}
],
"idter": 500,
"datapart": false
}
"""
json_obj = json.loads(my_json)
json_obj['idter'] = 600
print(json.dumps(json_obj))
Prints
{"name": "toggapp", "ts": [1234, 3456], "gs": [{"id": 4491, "con": "mno"}, {"id": 4494, "con": "hkl"}], "idter": 600, "datapart": false}
There's this small script used it to find entries in some very long and unnerving JSONs. not very beautifull und badly documented but maybe helps in your scenario.
from RecursiveSearch import Retriever
def alter_data(json_data, key, original, newval):
'''
Alter *all* values of said keys
'''
retr = Retriever(json_data)
for item_no, item in enumerate(retr.__track__(key)): # i.e. all 'value'
# Pick parent objects with a last element False in the __track__() result,
# indicating that `key` is either a dict key or a set element
if not item[-1]:
parent = retr.get_parent(key, item_no)
try:
if parent[key] == original:
parent[key] = newval
except TypeError:
# It's a set, this is not the key you're looking for
pass
if __name__ == '__main__':
alter_data(notification, key='value',
original = '********** THIS SHOULD BE UPDATED **********',
newval = '*UPDATED*')
In the JSON below, I want to access the email-id and 'gamesplayed' field for each user.
"UserTable" : {
"abcd#gmailcom" : {
"gameHistory" : {
"G1" : [ {
"category" : "1",
"questiontext" : "What is the cube of 2 ?"
}, {
"category" : "2",
"questiontext" : "What is the cube of 4 ?"
} ]
},
"gamesplayed" : 2
},
"xyz#gmailcom" : {
"gameHistory" : {
"G1" : [ {
"category" : "1",
"questiontext" : "What is the cube of 2 ?"
}, {
"category" : "2",
"questiontext" : "What is the cube of 4 ?"
} ]
},
"gamesplayed" : 2
}
}
Following is the code that I using to try and access the users email-id:
for user in jp.match("$.UserTable[*].[0]", game_data):
print("User ID's {}".format(user_id))
This is the error I'm getting:
File "C:\ProgramData\Anaconda3\lib\site-packages\jsonpath_rw\jsonpath.py", line 444, in find
return [DatumInContext(datum.value[self.index], path=self, context=datum)]
KeyError: 0
And when I run the following line to and access the 'gamesplayed' field for each user, the IDE Crashes.
print (parser.ExtentedJsonPathParser().parse("$.*.gamesplayed").find(gd_info))
If you like to use JSONPath. Please try this.
Python code:
with open(json_file) as json_file:
raw_data = json.load(json_file)
jsonpath_expr = parse('$.UserTable')
players = [match.value for match in jsonpath_expr.find(raw_data)][0]
emails = players.keys()
result = [{'email': email, 'gamesplayed': players[email]['gamesplayed']} for email in emails ]
print (result)
Output:
[{'email': 'abcd#gmailcom', 'gamesplayed': 2}, {'email': 'xyz#gmailcom', 'gamesplayed': 2}]
Python can handle valid json's as dictionaries. Therefore you have to parse to json string to a python dictionary.
import json
dic = json.loads(json_str)
You can now access a value by using the specific key as an index value = dict[key].
for user in dic:
email = user
gamesplayed = dic[user][gamesplayed]
print("{} played {} game(s).".format(email, gamesplayed))
>>> abcd#gmailcom played 2 game(s).
xyz#gmailcom played 2 game(s).
I am having a mongodb database that I want to update. Actually, my code for creating and updating the database is the following:
from pymongo import MongoClient
client = MongoClient()
client = MongoClient('localhost', 27017)
db = client['my_db_values']
res = collection = db['db_values']
res = collection.find({"User": "2145"})
if res.count() == 0:
json_file = {"User": "2145", "Item": {"123456": {"process1": [],"process2": []}}}
temp_json1 = {"timestamp": "2123532158", "process1_value": 0.4, "state": {"B": 0.1, "F": 0.2, "E": 0.3}}
temp_json2 = {"timestamp": "2323532158", "process2_value": 0.2, "P": 0.8}
json_file ["Item"][str(123464)]["process1"].append(temp_json1)
json_file ["Item"][str(123464)]["process2"].append(temp_json2)
temp = db.values
temp_id = temp.insert_one(json_file).inserted_id
else:
for line in res:
counter = 0
for key in line["Item"].keys():
if line["Item"].keys()[counter] == "123464":
collection.update_one({"User": "2145", "Item": {"123464": {"process1":[]}}}, {"$set": {"Item.123464.process2": [
{"timestamp": "21354879546213", "process1_value": 0,
"state": {"B": 0.1, "F": 0.2,
"E": 0.3}}], "Item.123464.process2": [
{"timestamp": "11354879546213", "process2_value": 0, "P": 0.8}]}})
else:
collection.update_one({"User": "2145"},{"$set": {"Item.123464.process1": [{"timestamp": "21354879546213", "process1_value": 0.4, "state": {"B": 0.1, "F": 0.2, "E": 0.3}}], "Item.123464.process2": [{"timestamp": "11354879546213", "process2_value": 0.2, "P": 0.8}]}})
counter = counter + 1
In the first if stetement if the count it is equally to zero I am creating the json file with that specific user, while if it is already there then i need to do the same for Sla and then I need to update the db with new temp_json1 and temp_json2. How can I update a subdocument into my initial document. Actually I want to perform a check to see if there is in the db a user with the specific id (otherwise I want to add him) then if the current item_id does not exist to add the item in the user document (as I did in my code already). Finally, if the item does exist then I want just to add temp_json1 and temp_json2 in the already created subdocument. How can I do so?
What you desire is subdocument querying (querying documents by their nested contents).
You can control this query by using the $elemMatch feature, to specify what matches your query by the contents of your Item subdocument's 123456 subdocument's process1 array.
The Mongo Shell format of the query is the following (for python driver just use the query part):
db.your_collection.find({
"User": "2145",
"Item.123456.process1": {$elemMatch: {$eq: "12345"} }
});
So if your collection is populated with the following 2 documents:
{ "_id" : ObjectId("aaa"), "User" : "2145", "Item" : { "123456" : { "process1" : [ ], "process2" : [ ] } } }
{ "_id" : ObjectId("bbb"), "User" : "2145", "Item" : { "123456" : { "process1" : [ "12345" ], "process2" : [ ] } } }
This query will only return the 2nd document, but the omit the first because its process1 array contents don't contain a "12345".
Hope this helps!