I have a nested JSON and I need "Max" & "Remaining" percentage values from it.
This is sample formula I am thinking of 100-(Remaining/Max)*100=(Value)
Sample JSON:
{
"ConcurrentAsyncGetReportInstances":
{
"Max": 5,
"Remaining": 3
},
"DailyApiRequests":
{
"Max":15000,"Remaining":14108
}
}
This is the JSON output.
I need to add the % value to the key
Sample output:
{
"ConcurrentAsyncGetReportInstances":40,(%value) 100-(5/3)*100
"DailyApiRequests": 5.95(%value) 100-(14108/15000)*100
}
Workarounds:
Tried to do make it a flat JSON and worked but didn't helped me
Worked on converting JSON into CSV and tried some but it was hard
Can someone suggest the best to do this? If possible provide some examples. Some help would also be appreciated.
Note: I am using Python 2.7
There is now a Python package for this called flatten_json. An introduction is provided here.
An example from that page--
In your shell:
> pip install flatten_json
In your Python console:
from flatten_json import flatten
input_dict = {
"a": 1,
"b": 2,
"c": [{"d": [2, 3, 4], "e": [{"f": 1, "g": 2}]}]
}
print(flatten(input_dict))
Results:
{'a': 1,
'b': 2,
'c_0_d_0': 2,
'c_0_d_1': 3,
'c_0_d_2': 4,
'c_0_e_0_f': 1,
'c_0_e_0_g': 2}
I've tested this in both Python 3.6 and 2.7.
Firstly receive your json and convert it to dictionary
import json
input_dict = json.loads(<your received son string>)
Then work on the input dict like below through recursive calls:
input_dict = {
"ConcurrentAsyncGetReportInstances":
{
"Max": 200,"Remaining":200
},
"DailyApiRequests":
{
"Max": 15000, "Remaining": 14108,
"Ant Migration Tool": {"Max": 0, "Remaining": 0},
"Chatter Desktop": {"Max": 0, "Remaining": 0},
"Chatter Mobile for BlackBerry":
{"Max": 0, "Remaining": 0},
"Chemical Equipment And Processing":
{"Max": 0,"Remaining": 0}
}
}
def flattenjson(input_dict, odict):
for ky in input_dict.keys():
if isinstance(input_dict[ky], dict):
if set(['Max', 'Remaining']).issubset(input_dict[ky].keys()):
if input_dict[ky]["Max"] != 0:
odict[ky] = 100-(float(input_dict[ky]["Remaining"])/input_dict[ky]["Max"])*100
else:
odict[ky] = 0
for iky in input_dict[ky].keys():
if isinstance(input_dict[ky][iky], dict):
tmp = {iky : input_dict[ky][iky]}
odict = flattenjson(tmp, odict)
return odict
odict = flattenjson(input_dict, dict())
print json.dumps(odict)
flattenjson helps you recursively work on to get your desired output for all Max and Remaining entries
You can retrieve nested values using the json library like so:
import json
sample_json = '{"ConcurrentAsyncGetReportInstances":{"Max": 5,"Remaining": 3},"DailyApiRequests": {"Max":15000,"Remaining":14108}}'
jason = json.loads(sample_json)
cagri_max = jason['ConcurrentAsyncGetReportInstances']['Max']
cagri_rem = jason['ConcurrentAsyncGetReportInstances']['Remaining']
You don't need to flatten the data structure. Just reference that pieces of it you want—so, for example, I think the following does essentially what you want:
import json
json_data = {
"ConcurrentAsyncGetReportInstances": {
"Max": 5,
"Remaining": 3
},
"DailyApiRequests": {
"Max": 15000,
"Remaining": 14108
}
}
def percentage_values(remaining, maximum):
return 100 - (float(remaining)/float(maximum)) * 100
# build output dictionary
json_out = {}
for key in json_data:
value = percentage_values(json_data[key]["Remaining"], json_data[key]["Max"])
json_out.update([(key, value)])
print(json.dumps(json_out, indent=4))
The resulting output showing the contents of json_out is:
{
"ConcurrentAsyncGetReportInstances": 40.0,
"DailyApiRequests": 5.9466666666666725
}
There are more succinct ways to write this in Python, but they all would do what is done above in a very simple manner.
Related
This question already has answers here:
How to prettyprint a JSON file?
(15 answers)
Closed 2 years ago.
I have a json data like
{"Player1": {"inventory": {"tas-count": 5, "kiriktas-count": 0, "odun-count": 0}}}
But it seems too complex. I want to edit, change it like
{
"Player1": {
"inventory": {
"tas-count": 5,
"kiriktas-count": 0,
"odun-count": 0,
}
}
}
I looked for it but there is nothing on Stackoverflow and also things like "\n" are not working. I heard that in other languages, there are libraries for making a clear json data. Might there are some like this in Python.
Here's an example:
>>> import json
>>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4))
{
"4": 5,
"6": 7
}
You can check out pretty printing here: https://docs.python.org/3/library/json.html
You can try:
import json
data = {"Player1": {"inventory": {"tas-count": 5, "kiriktas-count": 0, "odun-count": 0}}}
print(json.dumps(data, indent=4, sort_keys=True))
Output:
{
"Player1": {
"inventory": {
"kiriktas-count": 0,
"odun-count": 0,
"tas-count": 5
}
}
}
I need to sort my JSON based on value in ascending/descending order in PYTHON
This is my JSON:
{
"efg": 1,
"mnp": 4,
"xyz": 3
}
expected output is :
{
"mnp": 4,
"xyz": 3,
"efg": 1,
}
The above is just a sample JSON, Actual JSON is much bigger
And how to reverse sort it based on value
{
"efg": 1,
"xyz": 3,
"mnp": 4
}
Please help
-Ashish
import json
from collections import OrderedDict
json_str = """
{
"efg": 1,
"mnp": 4,
"xyz": 3
}
"""
json_dict = json.loads(json_str)
dict_sorted = OrderedDict(sorted(json_dict.items(), key=lambda x: x[1]))
str_sorted = json.dumps(dict_sorted) # '{"efg": 1, "xyz": 3, "mnp": 4}'
Is there any way to pull the key from JSON if the only thing I know is the value? (In groovy or python)
An example:
I know the "_number" value and I need a key.
So let's say, known _number is 2 and as an output, I should get dsf34f43f34f34f
{
"id": "8e37ecadf4908f79d58080e6ddbc",
"project": "some_project",
"branch": "master",
"current_revision": "3rtgfgdfg2fdsf",
"revisions": {
"43g5g534534rf34f43f": {
"_number": 3,
"created": "2019-04-16 09:03:07.459000000",
"uploader": {
"_account_id": 4
},
"description": "Rebase"
},
"dsf34f43f34f34f": {
"_number": 2,
"created": "2019-04-02 10:54:14.682000000",
"uploader": {
"_account_id": 2
},
"description": "Rebase"
}
}
}
With Groovy:
def json = new groovy.json.JsonSlurper().parse("x.json" as File)
println(json.revisions.findResult{ it.value._number==2 ? it.key : null })
// => dsf34f43f34f34f
Python 3: (assuming that data is saved in data.json):
import json
with open('data.json') as f:
json_data = json.load(f)
for rev, revdata in json_data['revisions'].items():
if revdata['_number'] == 2:
print(rev)
Prints all revs where _number equals 2.
using dict-comprehension:
print({k for k,v in d['revisions'].items() if v.get('_number') == 2})
OUTPUT:
{'dsf34f43f34f34f'}
Assuming this is the .JSON file I have to parse:
{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}
How would I go about printing the total values like:
1000 - 0
1001 - 6
1002 - 5
1003 - 4
1004 - 12
1005 - 0
I have already parsed the values, but I'm unsure of how to actually print them. I've already spent awhile on this and couldn't find a solution so any help is appreciated. Here is my code thus far:
import requests
import json
src = requests.get('https://hastebin.com/raw/nenowimite').json()
stats = src['item']['allInventory']['total']
print(stats)
This can be done through a for loop as follows:
for key in stats.keys():
print(key, '-', stats[key])
Using full Python 3.6 you can do (similarly than Ecir's answer)
for key, value in stats.items():
printf(f'{key} - {value}')
but being clearer about what is the key and the value and using the f-string interpolation.
You are almost there:
for item in stats.items():
print '%d - %d' % item
What this does is that stats is already a dict. Looking at the documentation, there is the items method which returns "a copy of the dictionary’s list of (key, value) pairs". And each pair is formatted as two numbers, i.e. '%d - %d'.
You can try:
>>> import json
>>> data= """{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}"""
>>> data = json.loads(data)
>>> print data["item"]["allInventory"]["total"]
{'1005': 0, '1004': 12, '1003': 3, '1002': 5, '1001': 6, '1000': 0}
As a java developer I need some tips how to solve this problem in python 2. My skills in python are in a beginning state. But now the question:
We provide a service for devices, which are reporting some technical statistics in a format, which we can not change. The server runs with python.
The main report are coming as dictionaries and we need to save the the json way. The converting from dict to json is not the problem, but converting the flat and with points separated keys need to be converted.
Perhaps an example can show what I want to say. This is the format from the devices, name it source:
{
'Device.DeviceInfo.SoftwareVersion': 'ote-2.2.1',
'Device.GatewayInfo.ProductClass': 'OEM-TX23',
'Device.GatewayInfo.SerialNumber': 'A223142D1CC7',
'Device.Ethernet.Interface.1.MaxBitRate': 1000,
'Device.HomePlug.Interface.1.AssociatedDevice.1.RxPhyRate': 522,
'Device.HomePlug.Interface.1.AssociatedDevice.1.TxPhyRate': 706,
'Device.HomePlug.Interface.1.AssociatedDevice.1.Active': 1,
'Device.HomePlug.Interface.1.AssociatedDevice.1.MACAddress': 'af:49:79:e4:64:fc',
'Device.HomePlug.Interface.1.AssociatedDevice.2.RxPhyRate': 544,
'Device.HomePlug.Interface.1.AssociatedDevice.2.TxPhyRate': 0,
'Device.HomePlug.Interface.1.AssociatedDevice.2.Active': 1,
'Device.HomePlug.Interface.1.AssociatedDevice.2.MACAddress': 'af:49:79:e4:64:dd',
'Device.Ethernet.Interface.2.MaxBitRate': 1000,
'Device.HomePlug.Interface.2.AssociatedDevice.1.RxPhyRate': 671,
'Device.HomePlug.Interface.2.AssociatedDevice.1.TxPhyRate': 607,
'Device.HomePlug.Interface.2.AssociatedDevice.1.Active': 1,
'Device.HomePlug.Interface.2.AssociatedDevice.1.MACAddress': 'bf:49:79:e4:64:fc',
'Device.HomePlug.Interface.2.AssociatedDevice.2.RxPhyRate': 340,
'Device.HomePlug.Interface.2.AssociatedDevice.2.TxPhyRate': 0,
'Device.HomePlug.Interface.2.AssociatedDevice.2.Active': 1,
'Device.HomePlug.Interface.2.AssociatedDevice.2.MACAddress': 'bf:49:79:e4:64:dd'
}
The Integer values within the source represents the index of the interfaces and AssociatedDevices for this interfaces. So the part behind an integer should be a list of multiple dictionaries. The integer value should not be included within the result.
We need the following nested structure before we can persist it to database, especially mysql docstore. And again, the conversion from nested dict to json is not the problem.
Here is the format we need:
{
'Device': {
'GatewayInfo': {
'SerialNumber': 'A223142D1CC7',
'ProductClass': 'OEM-TX23'
},
'DeviceInfo': {
'SoftwareVersion': 'ote-2.2.1'
},
'Ethernet': {
'Interface': [{
'MaxBitRate': 1000
}, {
'MaxBitRate': 1000
}]
},
'HomePlug': {
'Interface': [{
'AssociatedDevice': [{
'RxPhyRate': 522,
'TxPhyRate': 706,
'Active': 1,
'MACAddress': 'af:49:79:e4:64:fc',
}, {
'RxPhyRate': 544,
'TxPhyRate': 0,
'Active': 1,
'MACAddress': 'af:49:79:e4:64:dd',
}]
}, {
'AssociatedDevice': [{
'RxPhyRate': 671,
'TxPhyRate': 607,
'Active': 1,
'MACAddress': 'bf:49:79:e4:64:fc',
}, {
'RxPhyRate': 340,
'TxPhyRate': 0,
'Active': 1,
'MACAddress': 'bf:49:79:e4:64:dd',
}]
}]
}
}
}
UPDATE:
The first answer is partially correct, except that the parts after the integers should converted to a list containing the rest as dictionary.
You may iterate over your original dict to recursively add keys and add value to the final item as:
new_dict = {}
for key, value in my_dict.items():
k_list = key.split('.')
temp_dict = new_dict
for k in k_list[:-1]:
if k not in temp_dict:
temp_dict[k] = {}
temp_dict = temp_dict[k]
temp_dict[k_list[-1]] = value
where my_dict is your original dict object as mentioned in question.
Final value hold by new_dict will be:
{
"Device":{
"GatewayInfo":{
"SerialNumber":"A223142D1CC7",
"ProductClass":"OEM-TX23"
},
"DeviceInfo":{
"SoftwareVersion":"ote-2.2.1"
},
"HomePlug":{
"Interface":{
"1":{
"AssociatedDevice":{
"1":{
"RxPhyRate":522,
"Active":1,
"TxPhyRate":706,
"MACAddress":"af:49:79:e4:64:fc"
},
"2":{
"Active":1,
"MACAddress":"af:49:79:e4:64:dd",
"RxPhyRate":544,
"TxPhyRate":0
}
}
},
"2":{
"AssociatedDevice":{
"1":{
"RxPhyRate":671,
"Active":1,
"TxPhyRate":607,
"MACAddress":"bf:49:79:e4:64:fc"
},
"2":{
"RxPhyRate":340,
"MACAddress":"bf:49:79:e4:64:dd",
"TxPhyRate":0,
"Active":1
}
}
}
}
},
"Ethernet":{
"Interface":{
"1":{
"MaxBitRate":1000
},
"2":{
"MaxBitRate":1000
}
}
}
}
}
This should work. Just pass the unconverted dict to convert and it will return the converted dict
def convert(data):
to_convert = set()
new_dict = {}
for key, value in data.items():
path_stack = []
k_list = key.split('.')
temp_dict = new_dict
for k in k_list[:-1]:
path_stack.append(k)
if k.isnumeric():
to_convert.add(tuple(path_stack))
if k not in temp_dict:
temp_dict[k] = {}
temp_dict = temp_dict[k]
temp_dict[k_list[-1]] = value
for path in sorted(to_convert, key=len, reverse=True):
current_level = new_dict
for k in path[:-2]:
current_level = current_level[k]
if isinstance(current_level[path[-2]], dict):
new_level = [current_level[path[-2]][i] for i in sorted(current_level[path[-2]].keys())]
else:
new_level = current_level[path[-2]]
current_level[path[-2]] = new_level
return new_dict
If you want to dive into the python you may be interested in module dotteddict.
It's a little tricky but very intersting "pythonic" code. At a moment it doesn't convert numeric keys into list but some concepts from it is definitely worth a time to spend.