find time difference between previous and new data in collection using pymongo - python

I have list of dictionary data which is inserted in mongodb using pymongo
data =[{
"cam_name" : "cam1",
"stats" : [
{
"total" : 10,
"red" : 5,
"yellow" : 0,
"green" : 5,
"time_stamp" : datetime(2020,6,20,17,52,4,992000),
"image_path" : "image/19-06-2020/cam1/19-06-2020_17-52-16.jpg"
},
{
"total" : 10,
"red" : 5,
"yellow" : 0,
"green" : 5,
"time_stamp" : datetime(2020,6,20,17,52,27,992000),
"image_path" : "image/19-06-2020/cam1/19-06-2020_17-52-25.jpg"
},
{
"total" : 10,
"red" : 5,
"yellow" : 0,
"green" : 5,
"time_stamp" : datetime(2020,6,20,17,52,1,992000),
"image_path" : "image/19-06-2020/cam1/19-06-2020_17-52-25.jpg"
}]
},
{
"cam_name": "cam2",
"stats": [
{
"total": 10,
"red": 5,
"yellow": 0,
"green": 5,
"time_stamp": datetime(2020, 6, 20, 17, 52, 6, 992000),
"image_path": "image/19-06-2020/cam1/19-06-2020_17-52-16.jpg"
},
{
"total": 10,
"red": 5,
"yellow": 0,
"green": 5,
"time_stamp": datetime(2020, 6, 20, 17, 52, 59, 992000),
"image_path": "image/19-06-2020/cam1/19-06-2020_17-52-25.jpg"
},
{
"total": 10,
"red": 5,
"yellow": 0,
"green": 5,
"time_stamp": datetime(2020,6, 20, 17, 52, 4, 992000),
"image_path": "image/19-06-2020/cam1/19-06-2020_17-52-25.jpg"
}]
}
]
And after entering the data I am trying to find the difference between time between each element in list of dictionaries available for key ['stats']['time_stamp']
But I am not able to find out by using the approach of finding difference between the elements available in the list
from pymongo import MongoClient
from datetime import datetime
import json
import pandas as pd
from datetime import timedelta
myclient = MongoClient('localhost', 27017)
master_data = myclient['data_set']
cam_db = master_data['cam_table']
prev = None
for i,c in enumerate(data):
for k in c['stats']:
#print(k['time_stamp'])
if prev==None:
prev = k['time_stamp']
#print(prev)
else:
prev = k['time_stamp'] - prev
print(prev)
output:
0:00:23
2020-06-20 17:52:08.992000
-1 day, 23:59:56
2020-06-20 17:52:31.992000
0:00:00
after that, I am not able to find the perfect approach to use for find out the time difference
Note: I want to check whether time difference between every data available is less than 20 seconds or not
Suggestions will be really helpful

Related

python and Json - editing file

So, I want to change my info in json file from python, but I am having trouble.
my json file is just info that I want to edit later:
[
{
"codigo": 10,
"Nom_articulo": "jabon",
"valor": 2500,
"cantidad": 6,
"subtotal": 0,
"descuento": 0
},
{
"codigo": 20,
"Nom_articulo": "Crema",
"valor": 9800,
"cantidad": 4,
"subtotal": 0,
"descuento": 0
},
{
"codigo": 30,
"Nom_articulo": "Cepillo",
"valor": 6000,
"cantidad": 7,
"subtotal": 0,
"descuento": 0
},
{
"codigo": 40,
"Nom_articulo": "Servilletas",
"valor": 3000,
"cantidad": 2,
"subtotal": 0,
"descuento": 0
},
{
"codigo": 50,
"Nom_articulo": "Desodorante",
"valor": 5000,
"cantidad": 6,
"subtotal": 0,
"descuento": 0
}
]
I want to change the value of "subtotal" in all my dictionaries.
so basically what I did was:
for i in range(len(archivo_r)):
precio= archivo_r[i]["valor"]
cantidad=archivo_r[i]["cantidad"]
subtotal=precio*cantidad
print(archivo_r[i]["codigo"], " - " ,archivo_r[i]["Nom_articulo"], " = ", str(subtotal))
#almacenar mis subtotales en el archivo json
print("sbtotal" ,archivo_r[i]["subtotal"])
archivo_r[i]["subtotal"]=subtotal
#archivo_r[i]["subtotal"].append(subtotal)
#print(archivo_r)
write_json(**XXXXX**)
This part of the code:
archivo_r[i]["subtotal"]=subtotal does exactly what I need, but (and this could be very silly, but I am a little lost here) I do not know how to use that to re-write my json file. I mean, I have the function to write it.
def write_json(info, nombre_archivo="productos.json"):
with open(nombre_archivo, "w") as p:
json.dump(info, p)
I need to pass the information in write_json(**XXXXX**), but have been trying to storage my archivo_r[i]["subtotal"]=subtotal in a variable to pass it and other things, but nothing work. I know I am doing wrong but not sure how to solve it.
Once you're done processing the data, simply pass archivo_r to your write_json() function and you should be fine.
As an aside, you can iterate directly over the JSON objects like so:
for section in archivo_r:
precio = section["valor"]
...
You can then replace all instances of archivo_r[i] with section, or whatever you want to call the variable.

Bucketing for histogram in MongoDB

Syntax issue on line #25. Can someone please help me to spot the mistake not sure if the problem is in the code before this line.
File SyntaxError: invalid syntax, line 25
} #25
^
The line with the syntax is highlighted by #25. Thank you in advance :)
import pandas as pd
def length_vs_references(articles):
res = {"1-5" : 0, "6-10" : 0, "11-15" : 0, "16-20" : 0, "21-25" : 0, "25-30" : 0, ">30" :0}
n = {"1-5" : 0, "6-10" : 0, "11-15" : 0, "16-20" : 0, "21-25" : 0, "25-30" : 0, ">30" :0}
cursor = articles.aggregate([
{'$match': {'$and' : [{'references': {'$exists': False}
}, {'$ne':['$page_end', '']}, {'$ne':['$page_start', '']} ]}},
{'$project': {'len_refernces': {"$size": '$references'},
'pages': {'$subtract': [{"$toInt": 'page_end'},
{"$toInt" : 'page_start'}]}}},
{'$bucket' : {
'$groupBy': '$pages',
'boundaries': [ 0, 6, 11, 16, 21, 26, 31, 1000000],
'default': 'Other',
{
'output' : {"average": {"$avg" : '$len_references'}},
}
} #25
}
])
return cursor
print(length_vs_references(articles))
'default': 'Other',
{
'output' : {"average": {"$avg" : '$len_references'}},
}
That is the problem area. You have a sub-dictionary without a key name.
To illustrate the problem more simply, here is an equivalent dictionary:
mydict = {
'some_key': 5,
'other_key': 10,
'yet_another_key': 100,
3,
'final_key': 1000
}
The 3 is an error because it's just a value without a key name. Your code has a sub-dictionary instead of an integer, but it's the same kind of error.

Determining Turn Order

I wanted to try it like this, but I don't want to nest a bunch of for loops or if statements. For some reason I'm not seeing what the actual comparison should be to take the speeds list and populate turn order. Basically just comparing what each individual minion's speed is against the others using the sort function in it's own method and returning a list. Then I want to compare the list values against the key values stored in the minion_collection. I'm not sure if I'm setting myself up for a world of hurt here or not. Still somewhat new to python so I don't know many of the utilities.
minion_collection = {
'minion1' : {
"name" : "zombie", "str" : 10, "def" : 5, "hp" : 25, "maxhp" : 25, "speed" : 15, "spdbar": 0},
'minion2' : {
"name": 'cleric', "str": 4, "def": 5, "hp": 25,"maxhp": 25, "speed": 20, "spdbar": 0},
'minion3' : {
"name" : "professor", "str" : 10, "def" : 5, "hp" : 25, "maxhp" : 25, "speed" : 15, "spdbar": 0},
'minion4' : {
"name": 'hunter', "str": 4, "def": 5, "hp": 25,"maxhp": 25, "speed": 30, "spdbar": 0}
}
def initiative(speeds):
for x in minion_collection:
minion_collection[x]['spdbar'] = 100
order = sort(speeds, reverse = True)
return order
Where I start to determine the turn order
turn = 1
speeds = (fighter1['speed'], fighter2['speed'], pcfighter1['speed'], pcfighter2['speed'])
order = initiative(speeds)
for speed in order:
if fighter1['speed'] == speed:
first = fighter1
second = '???'
third = '???'
last = '???'
In this case you can use sorted()
turn = 1
speeds = (fighter1['speed'], fighter2['speed'], pcfighter1['speed'], pcfighter2['speed'])
order = initiative(speeds)
for speed in sorted(order):
if fighter1['speed'] == speed:
first = fighter1
second = '???'
third = '???'
last = '???'

How to find a document by using specific datetime?

This code runs perfectly:
dt = datetime.datetime(2018, 7, 15, 13, 24, 1, 962)
for doc in db.wing_model.find({'sampling_time': {'$gte': dt}}):
print doc
But when I try to find a document with specific datetime like below, it returns nothing.
dt = datetime.datetime(2018, 7, 15, 13, 24, 1, 962)
for doc in db.wing_model.find({'sampling_time': dt}):
print doc
Here's my document looks like:
{
"_id" : ObjectId("5b4ae88100285b236134e08e"),
"value" : [
{
"MQ3_ALCOHOL" : 0,
"MQ5_H2" : 0.39,
"MQ138_PROPANE" : 4.4,
"MQ135_ALCOHOL" : 1.25
}
],
"enose_id" : "node1",
"sampling_time" : ISODate("2018-07-15T13:24:01.962Z")
}

Printing parsed JSON in Python

Assuming this is the .JSON file I have to parse:
{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}
How would I go about printing the total values like:
1000 - 0
1001 - 6
1002 - 5
1003 - 4
1004 - 12
1005 - 0
I have already parsed the values, but I'm unsure of how to actually print them. I've already spent awhile on this and couldn't find a solution so any help is appreciated. Here is my code thus far:
import requests
import json
src = requests.get('https://hastebin.com/raw/nenowimite').json()
stats = src['item']['allInventory']['total']
print(stats)
This can be done through a for loop as follows:
for key in stats.keys():
print(key, '-', stats[key])
Using full Python 3.6 you can do (similarly than Ecir's answer)
for key, value in stats.items():
printf(f'{key} - {value}')
but being clearer about what is the key and the value and using the f-string interpolation.
You are almost there:
for item in stats.items():
print '%d - %d' % item
What this does is that stats is already a dict. Looking at the documentation, there is the items method which returns "a copy of the dictionary’s list of (key, value) pairs". And each pair is formatted as two numbers, i.e. '%d - %d'.
You can try:
>>> import json
>>> data= """{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}"""
>>> data = json.loads(data)
>>> print data["item"]["allInventory"]["total"]
{'1005': 0, '1004': 12, '1003': 3, '1002': 5, '1001': 6, '1000': 0}

Categories

Resources