I have a JSON file with the following structure.
I want to identify the channel with the minimum fee and to print its details (channel_id, node1, node2 and the total fee)
The total fee of a channel is the sum of fees of both nodes:
total_fee = fee_node1 + fee_node2
{
"edges": [
{
"channel_id": "1",
"node1": "Alice",
"node2": "Bob",
"node1_policy": {
"fee": "1000"
},
"node2_policy": {
"fee": "1000"
}
},
{
"channel_id": "2",
"node1": "Caleb",
"node2": "Daniel",
"node1_policy": {
"fee": "500",
},
"node2_policy": {
"fee": "3000",
}
},
{
"channel_id": "3",
"node1": "Elen",
"node2": "Fatih",
"node1_policy": {
"fee": "2000"
},
"node2_policy": {
"fee": "5000"
}
}
}
What is the best method to accomplish this task?
It is quite simple to do that in a few lines of code. I should mention though that the format of your JSON file is invalid, because of the redundant commas right after the fee keys. Consider removing them in order to be valid:
import json
with open('channels.json', 'r') as inFile:
jsonData = json.loads(inFile.read())
minimum = min([elem for elem in jsonData['edges']], key=lambda e: int(e['node1_policy']['fee']) + int(e['node2_policy']['fee']))
print(minimum)
Output:
{'channel_id': '1', 'node1': 'Alice', 'node2': 'Bob', 'node1_policy': {'fee': '1000'}, 'node2_policy': {'fee': '1000'}}
Related
I have the below function that returns a query that I would then use for my API call:
def creator_rating_query(self, creator_ids, platform, ages, gender):
data = [
{
"query": {
"included": {
"creators": creator_ids
}
},
"filter": {
"audience": {
"genders": gender,
"ages": ages
},
"period": {
"since": "2021-01",
"until": "2021-01"
},
"platform": platform
}
}
]
return data
creator_ids are a fixed list of 200 string values, for ex:
creators_id = ['cdkfsd1','kdsfdd','dskfdfie']
however platform, ages and gender needs to be looped through for each value. I placed them in a list of a list but if there is a better way to have them arranged I'm open to suggestions!
gender=[['f'],['m']]
platform=[['Facebook'],['YouTube']]
ages=[['13_17'],['18_24'],['25_34'],['35_44'],['45_54'],['55_plus']]
so the first query data returned would get each gender, each platform and each age for ex:
[
{
"query": {
"included": {
"creators": ['cdkfsd1','kdsfdd','dskfdfie']
}
},
"filter": {
"audience": {
"genders": "f",
"ages": ['13_17']
},
"period": {
"since": "2021-01",
"until": "2021-01"
},
"platform":"Facebook"
}
}
]
and another would be:
[
{
"query": {
"included": {
"creators": ['cdkfsd1','kdsfdd','dskfdfie']
}
},
"filter": {
"audience": {
"genders": "f",
"ages": ['18_24']
},
"period": {
"since": "2021-01",
"until": "2021-01"
},
"platform":"Facebook"
}
}
]
etc, so the same would be built for all ages, platforms, and genders
Probably you want to find product of three lists:
from itertools import product
result = []
for gender, platform, age in product(gender, platform, ages):
result.append({
"query": {
"included": {
"creators": ['cdkfsd1', 'kdsfdd', 'dskfdfie']
}
},
"filter": {
"audience": {
"genders": gender,
"ages": age
},
"period": {
"since": "2021-01",
"until": "2021-01"
},
"platform": platform
}
})
print(result)
Or you can explicitly iterate over lists:
for age in ages:
for platform in platforms:
for gender in genders:
print(age, platform, gender)
I have a streaming data coming from a source and I was able to capture few important variables like ID, Timestamp and Vital signs. What I am trying to do is create a json file object and unload it into a file system. This is one of the examples:
for k in streaming:
id = k[1] ----------> 1
timestamp = k[2] ------> 1652304692
vsigns = k[3] -------> Nested dictionary
This is how vsigns looks like
"vsigns":
{
"ECG":
{
"VPB_FREQ": "0",
"STI": "+0.00"
},
"HR":
{
"HR_EKG": "87",
"HR_PPG_1": "87",
"HR_PULSED": "87"
},
"NIBP":
{
"NIBPS": "119",
"NIBPD": "88",
"NIBPM": "95"
}
}
And I want a json structure in the following format:
[{
"id": "1",
"ECG":
{
"timestamp": 1652304692,
"VPB_FREQ": "0",
"STI": "+0.00",
}
},
{
"id": "1",
"HR":
{
"timestamp": 1652304692,
"HR_EKG": "87",
"HR_PPG_1": "87",
"HR_PULSED": "87"
}
},
{
"id": "1",
"NIBP":
{
"timestamp": 1652304692,
"NIBPS": "119",
"NIBPD": "88",
"NIBPM": "95"
},
}]
I tried an approach but doesn't give me what I want. How do I get this right.
for k in streaming:
id = k[1]
timestamp = k[2]
vsigns = k[3]
vlist = []
for k, v in vsigns.items():
vdict = {"id": id,
k:{"timestamp":timestamp,
"vsigns": v}}
vlist.append(vdict)
print(vlist)
Output:
[{
"id": "1",
"ECG":
{
"timestamp": 1652951054.0,
"vsigns":
{
"VPB_FREQ": "0"
}
}
},
{
"id": "1",
"HR":
{
"timestamp": 1652951054.0,
"vsigns":
{
"HR_EKG": "126",
"HR_PPG_1": "127",
"HR_PULSED": "127"
}
}
},
{
"id": "1",
"NIBP":
{
"timestamp": 1652951054.0,
"vsigns":
{
"NIBPS": "95",
"NIBPD": "46",
"NIBPM": "62"
}
}
}
}]
The following piece of code worked for me:
for k in streaming:
id = k[1]
timestamp = k[2]
vsigns = k[3]
vlist = []
for k, v in vsigns.items():
v['timestamp'] = epoch_time
vdict = {"id": id,
k:v}
vlist.append(vdict)
I want to merge many JSON files with the same nested structure, using jsonmerge, but have been unsuccessful so far. For example, I want to merge base and head:
base = {
"data": [
{
"author_id": "id1",
"id": "1"
},
{
"author_id": "id2",
"id": "2"
}
],
"includes": {
"users": [
{
"id": "user1",
"name": "user1"
},
{
"id": "user2",
"name": "user2"
}
]
}
}
head = {
"data": [
{
"author_id": "id3",
"id": "3"
},
{
"author_id": "id4",
"id": "4"
}
],
"includes": {
"users": [
{
"id": "user3",
"name": "user3"
},
{
"id": "user4",
"name": "user4"
}
]
}
}
The resulting JSON should be:
final_result = {
"data": [
{
"author_id": "id1",
"id": "1"
},
{
"author_id": "id2",
"id": "2"
},
{
"author_id": "id3",
"id": "3"
},
{
"author_id": "id4",
"id": "4"
}
],
"includes": {
"users": [
{
"id": "user1",
"name": "user1"
},
{
"id": "user2",
"name": "user2"
},
{
"id": "user3",
"name": "user3"
},
{
"id": "user4",
"name": "user4"
}
]
}
}
However, I've only managed to merge correctly the data fields, while for users it doesn't seem to work. This is my code:
from jsonmerge import merge
from jsonmerge import Merger
schema = { "properties": {
"data": {
"mergeStrategy": "append"
},
"includes": {
"users": {
"mergeStrategy": "append"
}
}
}
}
merger = Merger(schema)
result = merger.merge(base, head)
The end result is:
{'data': [{'author_id': 'id1', 'id': '1'},
{'author_id': 'id2', 'id': '2'},
{'author_id': 'id3', 'id': '3'},
{'author_id': 'id4', 'id': '4'}],
'includes': {'users': [{'id': 'user3', 'name': 'user3'},
{'id': 'user4', 'name': 'user4'}]}}
The issue is with the definition of the schema, but I do not know if it is possible to do it like that with jsonmerge. Any help is appreciated!
Thank you!
It is based on jsonschema. So when you have an object within an object (e.g. "users" within "includes") then you'll need to tell jsonschema it is dealing with another object like so:
schema = {
"properties": {
"data": {
"mergeStrategy": "append"
},
"includes": {
"type": "object",
"properties": {
"users": {
"mergeStrategy": "append"
}
}
}
}
}
Note that this also happens for your top-level objects, hence you have "properties" argument on the highest level.
I want to use an api and would need to put my dataframe in a dictionary format first.
The dataframe df that looks like this:
OrigC OrigZ OrigN Weigh DestC DestZ DestN
0 PL 97 TP 59 DE 63 SN
Exepected output of the first row:
{"section":[
{"location":
{
"zipCode":
{"OrigC": "PL",
"OrigZ":"97"},
"location": {"id": "1"},
"OrigN": "TP"
},
"carriageParameter":
{"road":
{"truckLoad": "Auto"}
},
"load":
{"Weigh": "59",
"unit": "ton",
"showEmissionsAtResponse": "true"
}
},
{"location":
{
"zipCode":
{"DestC": "DE",
"DestZ":"63"},
"location": {"id": "2"},
"DestN": "SN"
},
"carriageParameter":
{"road":
{"truckLoad":"Auto"}
},
"unload":
{"WEIGHTTONS":"59",
"unit": "ton",
"showEmissionsAtResponse": "true"
}
}]}
Note that there is static information in the dictionary that doesn't require any change.
How can this be done in Python?
You can use iterrows.
dic = {}
dic['section'] = []
for ix, row in df.iterrows():
in_dict = {
'location': {
'zip_code': {
'OrigC': row['OrigC'],
'OrigZ': row['OrigZ'],
},
'location': {'id': ix+1}, # I am guessing here.
'OrigN': 'TP',
},
'CarriageParameter': {
'road': {
'truckLoad': 'Auto'}
},
'load': {
'Weigh': str(row['Weigh']),
}
}
dic['section'].append(in_dict)
Note that this is not the entire entry, but I think it is clear enough to illustrate the idea.
All,
I am trying to change the way some json looks by going through and formatting it in the following way:
1. flatten all of the fields lists
2. Then remove the fields lists and replace them with the name : flatten list
Example:
{
"name": "",
"fields": [{
"name": "keys",
"fields": [{
"node-name": "0/0/CPU0"
},
{
"interface-name": "TenGigE0/0/0/47"
},
{
"device-id": "ASR9K-H1902.corp.cisco.com"
}
]
},
{
"name": "content",
"fields": [{
"name": "lldp-neighbor",
"fields": [{
"receiving-interface-name": "TenGigE0/0/0/47"
},
{
"receiving-parent-interface-name": "Bundle-Ether403"
},
{
"device-id": "ASR9K-H1902.corp.cisco.com"
},
{
"chassis-id": "78ba.f975.a64f"
},
{
"port-id-detail": "Te0/1/0/4/0"
},
{
"header-version": 0
},
{
"hold-time": 120
},
{
"enabled-capabilities": "R"
},
{
"platform": ""
}
]
}]
}
]
}
Would turn into:
{
"": [{
"keys": [{
"node-name": "0/0/CPU0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
}]
},
{
"content": [{
"lldp-neighbor": [{
"receiving-interface-name": "TenGigE0/0/0/47",
"receiving-parent-interface-name": "Bundle-Ether403",
"device-id": "ASR9K-H1902.corp.cisco.com",
"chassis-id": "78ba.f975.a64f",
"port-id-detail": "Te0/1/0/4/0",
"header-version": 0,
"hold-time": 120,
"enabled-capabilities": "R",
"platform": ""
}]
}]
}
]
}
I have tried the following to get the list flattened:
def _flatten_fields(self, fields_list):
c = {}
for b in [d for d in fields_list if bool(d)]:
c.update(b)
return c
This seems to work but I can't figure out a way to get into the sub levels using recursion, I am saving all flatten lists and names into a new dictionary, is there a way to do it by just manipulating the original dictionary?
This worked on the example you provided:
import json
def flatten(data):
result = dict()
if isinstance(data, dict):
if 'name' in data:
name = data['name']
result[name] = flatten(data['fields'])
else:
key = data.keys()[0]
value = data.values()[0]
result[key] = value
else:
for entry in data:
result.update(flatten(entry))
return result
print json.dumps(flatten(data), indent=4)
Output
{
"": {
"keys": {
"node-name": "0/0/CPU0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
},
"content": {
"lldp-neighbor": {
"receiving-interface-name": "TenGigE0/0/0/47",
"receiving-parent-interface-name": "Bundle-Ether403",
"header-version": 0,
"port-id-detail": "Te0/1/0/4/0",
"chassis-id": "78ba.f975.a64f",
"platform": "",
"device-id": "ASR9K-H1902.corp.cisco.com",
"hold-time": 120,
"enabled-capabilities": "R"
}
}
}
}
It doesn't have the extra list layers shown in your expected output, but I don't think you want those.
This worked on the example you provided:
def flatten_fields(fields_list):
c = {}
for item in fields_list:
for key in item:
if key == "fields":
c[item["name"]] = flatten_fields(item["fields"])
elif key != "name":
c[key] = item[key]
break
return [c]
But it works on a list of dictionaries, so you should call it like flatten_fields([data])[0].
The output is:
{
"": [{
"keys": [{
"node-name": "0/0/CP0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
}],
"content": [{
"lldp-neighbor": [{
"chassis-id": "78ba.f975.a64f",
"receiving-parent-interface-name": "Bndle-Ether403",
"enabled-capabilities": "R",
"device-id": "ASR9K-H1902.corp.cisco.com",
"hold-time": 120,
"receiving-interface-name": "TenGigE0/0/0/47",
"platform": "",
"header-version": 0,
"port-id-detail": "Te0/1/0/4/0"
}]
}]
}]
}