Develop Reference

TypeError: string indices must be integers while parsing a json

I'm parsing data from the fitbit API (https://dev.fitbit.com/build/reference/web-api/heart-rate/)
and keep getting TypeError: string indices must be integers. I understand, based off of reading other questions, that this could be due to dealing with lists of dictionaries. Right now, I'm parsing my json like this:
fat_burn = "{0:.2f}".format(activity_request.json()['activities-heart'][0]['value']['heartRateZones'][1]['minutes'])
My json looks like this:
{
"activities-heart": [
{
"dateTime": "2015-08-04",
"value": {
"customHeartRateZones": [],
"heartRateZones": [
{
"caloriesOut": 740.15264,
"max": 94,
"min": 30,
"minutes": 593,
"name": "Out of Range"
},
{
"caloriesOut": 249.66204,
"max": 132,
"min": 94,
"minutes": 46,
"name": "Fat Burn"
},
{
"caloriesOut": 0,
"max": 160,
"min": 132,
"minutes": 0,
"name": "Cardio"
},
{
"caloriesOut": 0,
"max": 220,
"min": 160,
"minutes": 0,
"name": "Peak"
}
],
"restingHeartRate": 68
}
}
]
}
I had an error that looked like TypeError: list indices must be integers or slices, not str when I was parsing the json like:
fat_burn = "{0:.2f}".format(activity_request.json()['activities-heart']['value']['heartRateZones'][1]['minutes'])
but managed to resolve it by adding a [0].
Why did I need to index the first entry to solve that error, and what do I need to do to resolve the error I'm having now?

activities-heart is a key who's value is a list. Lists are sequential and are indexed with integers (eg, 0, 1, 2, ...).
In your expression, you are trying perform a look up on the list using a string (value). This is where your TypeError is coming from.
Other elements in the data such as this..
{
"caloriesOut": 740.15264,
"max": 94,
"min": 30,
"minutes": 593,
"name": "Out of Range"
}
Are dictionaries which allow you to use any hashable values for the key. This is why you're able to use a string for the key.
https://www.geeksforgeeks.org/difference-between-list-and-dictionary-in-python

See, let's store your data in a variable a, then,
When you are doing this, a['activities-heart']:
You will get an output like this:
[{'dateTime': '2015-08-04',
'value': {'customHeartRateZones': [],
'heartRateZones': [{'caloriesOut': 740.15264,
'max': 94,
'min': 30,
'minutes': 593,
'name': 'Out of Range'},
{'caloriesOut': 249.66204,
'max': 132,
'min': 94,
'minutes': 46,
'name': 'Fat Burn'},
{'caloriesOut': 0, 'max': 160, 'min': 132, 'minutes': 0, 'name': 'Cardio'},
{'caloriesOut': 0, 'max': 220, 'min': 160, 'minutes': 0, 'name': 'Peak'}],
'restingHeartRate': 68}}]
This is a list and at 0 index is your value, therefore,
a['activities-heart'][0] will give you a dictionary.
Therefore you can perform like this:
a['activities-heart'][0]['value']['heartRateZones'][1]['minutes']
Output:
46

Determining Turn Order

I wanted to try it like this, but I don't want to nest a bunch of for loops or if statements. For some reason I'm not seeing what the actual comparison should be to take the speeds list and populate turn order. Basically just comparing what each individual minion's speed is against the others using the sort function in it's own method and returning a list. Then I want to compare the list values against the key values stored in the minion_collection. I'm not sure if I'm setting myself up for a world of hurt here or not. Still somewhat new to python so I don't know many of the utilities.
minion_collection = {
'minion1' : {
"name" : "zombie", "str" : 10, "def" : 5, "hp" : 25, "maxhp" : 25, "speed" : 15, "spdbar": 0},
'minion2' : {
"name": 'cleric', "str": 4, "def": 5, "hp": 25,"maxhp": 25, "speed": 20, "spdbar": 0},
'minion3' : {
"name" : "professor", "str" : 10, "def" : 5, "hp" : 25, "maxhp" : 25, "speed" : 15, "spdbar": 0},
'minion4' : {
"name": 'hunter', "str": 4, "def": 5, "hp": 25,"maxhp": 25, "speed": 30, "spdbar": 0}
}
def initiative(speeds):
for x in minion_collection:
minion_collection[x]['spdbar'] = 100
order = sort(speeds, reverse = True)
return order
Where I start to determine the turn order
turn = 1
speeds = (fighter1['speed'], fighter2['speed'], pcfighter1['speed'], pcfighter2['speed'])
order = initiative(speeds)
for speed in order:
if fighter1['speed'] == speed:
first = fighter1
second = '???'
third = '???'
last = '???'

In this case you can use sorted()
turn = 1
speeds = (fighter1['speed'], fighter2['speed'], pcfighter1['speed'], pcfighter2['speed'])
order = initiative(speeds)
for speed in sorted(order):
if fighter1['speed'] == speed:
first = fighter1
second = '???'
third = '???'
last = '???'

Printing parsed JSON in Python

Assuming this is the .JSON file I have to parse:
{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}
How would I go about printing the total values like:
1000 - 0
1001 - 6
1002 - 5
1003 - 4
1004 - 12
1005 - 0
I have already parsed the values, but I'm unsure of how to actually print them. I've already spent awhile on this and couldn't find a solution so any help is appreciated. Here is my code thus far:
import requests
import json
src = requests.get('https://hastebin.com/raw/nenowimite').json()
stats = src['item']['allInventory']['total']
print(stats)

This can be done through a for loop as follows:
for key in stats.keys():
print(key, '-', stats[key])

Using full Python 3.6 you can do (similarly than Ecir's answer)
for key, value in stats.items():
printf(f'{key} - {value}')
but being clearer about what is the key and the value and using the f-string interpolation.

You are almost there:
for item in stats.items():
print '%d - %d' % item
What this does is that stats is already a dict. Looking at the documentation, there is the items method which returns "a copy of the dictionary’s list of (key, value) pairs". And each pair is formatted as two numbers, i.e. '%d - %d'.

You can try:
>>> import json
>>> data= """{
"item": {
"allInventory": {
"onHand": 64,
"total": {
"1000": 0,
"1001": 6,
"1002": 5,
"1003": 3,
"1004": 12,
"1005": 0
}
}
},
"image": {
"tag": "/828402de-6cc8-493e-8abd-935a48a3d766_1.285a6f66ecf3ee434100921a3911ce6c.jpeg?odnHeight=450&odnWidth=450&odnBg=FFFFFF"
}
}"""
>>> data = json.loads(data)
>>> print data["item"]["allInventory"]["total"]
{'1005': 0, '1004': 12, '1003': 3, '1002': 5, '1001': 6, '1000': 0}

How to get a flat JSON from a nested one?

I have a nested JSON and I need "Max" & "Remaining" percentage values from it.
This is sample formula I am thinking of 100-(Remaining/Max)*100=(Value)
Sample JSON:
{
"ConcurrentAsyncGetReportInstances":
{
"Max": 5,
"Remaining": 3
},
"DailyApiRequests":
{
"Max":15000,"Remaining":14108
}
}
This is the JSON output.
I need to add the % value to the key
Sample output:
{
"ConcurrentAsyncGetReportInstances":40,(%value) 100-(5/3)*100
"DailyApiRequests": 5.95(%value) 100-(14108/15000)*100
}
Workarounds:
Tried to do make it a flat JSON and worked but didn't helped me
Worked on converting JSON into CSV and tried some but it was hard
Can someone suggest the best to do this? If possible provide some examples. Some help would also be appreciated.
Note: I am using Python 2.7

There is now a Python package for this called flatten_json. An introduction is provided here.
An example from that page--
In your shell:
> pip install flatten_json
In your Python console:
from flatten_json import flatten
input_dict = {
"a": 1,
"b": 2,
"c": [{"d": [2, 3, 4], "e": [{"f": 1, "g": 2}]}]
}
print(flatten(input_dict))
Results:
{'a': 1,
'b': 2,
'c_0_d_0': 2,
'c_0_d_1': 3,
'c_0_d_2': 4,
'c_0_e_0_f': 1,
'c_0_e_0_g': 2}
I've tested this in both Python 3.6 and 2.7.

Firstly receive your json and convert it to dictionary
import json
input_dict = json.loads(<your received son string>)
Then work on the input dict like below through recursive calls:
input_dict = {
"ConcurrentAsyncGetReportInstances":
{
"Max": 200,"Remaining":200
},
"DailyApiRequests":
{
"Max": 15000, "Remaining": 14108,
"Ant Migration Tool": {"Max": 0, "Remaining": 0},
"Chatter Desktop": {"Max": 0, "Remaining": 0},
"Chatter Mobile for BlackBerry":
{"Max": 0, "Remaining": 0},
"Chemical Equipment And Processing":
{"Max": 0,"Remaining": 0}
}
}
def flattenjson(input_dict, odict):
for ky in input_dict.keys():
if isinstance(input_dict[ky], dict):
if set(['Max', 'Remaining']).issubset(input_dict[ky].keys()):
if input_dict[ky]["Max"] != 0:
odict[ky] = 100-(float(input_dict[ky]["Remaining"])/input_dict[ky]["Max"])*100
else:
odict[ky] = 0
for iky in input_dict[ky].keys():
if isinstance(input_dict[ky][iky], dict):
tmp = {iky : input_dict[ky][iky]}
odict = flattenjson(tmp, odict)
return odict
odict = flattenjson(input_dict, dict())
print json.dumps(odict)
flattenjson helps you recursively work on to get your desired output for all Max and Remaining entries

You can retrieve nested values using the json library like so:
import json
sample_json = '{"ConcurrentAsyncGetReportInstances":{"Max": 5,"Remaining": 3},"DailyApiRequests": {"Max":15000,"Remaining":14108}}'
jason = json.loads(sample_json)
cagri_max = jason['ConcurrentAsyncGetReportInstances']['Max']
cagri_rem = jason['ConcurrentAsyncGetReportInstances']['Remaining']

You don't need to flatten the data structure. Just reference that pieces of it you want—so, for example, I think the following does essentially what you want:
import json
json_data = {
"ConcurrentAsyncGetReportInstances": {
"Max": 5,
"Remaining": 3
},
"DailyApiRequests": {
"Max": 15000,
"Remaining": 14108
}
}
def percentage_values(remaining, maximum):
return 100 - (float(remaining)/float(maximum)) * 100
# build output dictionary
json_out = {}
for key in json_data:
value = percentage_values(json_data[key]["Remaining"], json_data[key]["Max"])
json_out.update([(key, value)])
print(json.dumps(json_out, indent=4))
The resulting output showing the contents of json_out is:
{
"ConcurrentAsyncGetReportInstances": 40.0,
"DailyApiRequests": 5.9466666666666725
}
There are more succinct ways to write this in Python, but they all would do what is done above in a very simple manner.

Navigating JSON in Python?

ANSWER: The square brackets inside data["data"]["items"] indicates a list of dictionaries. I had thought that the brackets indicated that all of those dictionaries were a list inside a list of one item (i.e. [[item1, item2]]) so I had to call a 0 index like so: data["data"]["items"][0]. That was incorrect, the [0] is unecessary, which I really should have known, but didn't realize.
I am using GAE, so you can think of the self.response.write statement as a print.
For some reason, although the user_xid prints correctly, the interpreter is giving me the error when I try to parse json_data["data"]["items"][0] in my for loop, it appears to be evaluating x as a string, because I am getting an error telling me that string indicies must be integers.
I didn't realize json_data["data"]["items"][0] would yield a string. I'm somewhat stymied. Would it have anything to do with using json_data["data"]["items"][0] as opposed to json_data["data"]["items"]?
I have also tried json.loads(json_data["data"]["items"][0]) to no avail. Any input is appreciated.
I have the following JSON, which I have broken out a bit to make it easier to read:
{"meta":
{"user_xid": "REDACTED", "message": "OK", "code": 200, "time": 1436273692},
"data":
{"items": [
{"time_completed": 1436193938, "xid": "4tLz2M3x8_E9mTPIQsQUdrj-83spikha", "title": "for 7h 40m", "time_created": 1436165067, "time_updated": 1436194828, "details":
{"body": 0, "sound": 14348, "tz": "America/Vancouver", "awakenings": 1, "light": 13258, "mind": 0, "asleep_time": 1436165760, "awake": 1265, "rem": 0, "duration": 28871, "smart_alarm_fire": 1436193000, "quality": 81, "awake_time": 1436193600, "sunrise": 1436184840, "sunset": 1436156340},
"date": 20150706, "shared": true, "sub_type": 0},
{"time_completed": 1436135400, "xid": "4tLz2M3x8_Hk71lnuSLbeq4ascwJg-J8", "title": "for 25m", "time_created": 1436133600, "time_updated": 1436135590, "details":
{"body": 0, "sound": 0, "tz": "America/Vancouver", "awakenings": 0, "light": 1500, "mind": 0, "asleep_time": 1436133600, "awake": 300, "rem": 0, "duration": 1800, "smart_alarm_fire": 0, "quality": 2, "awake_time": 1436134800, "sunrise": 0, "sunset": 0},
"date": 20150705, "shared": true, "sub_type": 2},
{"time_completed": 1436133497, "xid": "4tLz2M3x8_GIYf6G9_ZKH9xrKpuEzMjm", "title": "for 23m", "time_created": 1436131387, "time_updated": 1436135461, "details":
{"body": 0, "sound": 1391, "tz": "America/Vancouver", "awakenings": 0, "light": 0, "mind": 0, "asleep_time": 1436132100, "awake": 719, "rem": 0, "duration": 2110, "smart_alarm_fire": 1436133445, "quality": 4, "awake_time": 1436133300, "sunrise": 0, "sunset": 0},
"date": 20150705, "shared": true, "sub_type": 1},
{"time_completed": 1436117080, "xid": "4tLz2M3x8_EUZyqCoXYOJO7Hfe92eSqg", "title": "for 26m","time_created": 1436115061, "time_updated": 1436121619, "details":
{"body": 0, "sound": 480, "tz": "America/Vancouver", "awakenings": 0, "light": 1119, "mind": 0, "asleep_time": 1436115479, "awake": 420, "rem": 0, "duration": 2019, "smart_alarm_fire": 1436117069, "quality": 2, "awake_time": 1436116800, "sunrise": 1436098380, "sunset": 1436156280},
"date": 20150705, "shared": true, "sub_type": 1},
],
"links":
{"next": "REDACTED"},
"size": 10}
}
I am using the following in Python:
json_data = json.loads(user_data)
self.response.write('<br><br>x_id: ' + json_data["meta"]["user_xid"])
i = 0
for x in json_data["data"]["items"][0]:
duration = x['details']['duration']
date = x["date"]
self.response.write('<br><br>sleep number ' + str(i) + ' | duration: ' + str(duration) + ' | date: ' + str(date))
i += 1
I am using GAE, so you can think of the self.response.write statement as a print.
For some reason, although the user_xid prints correctly, the interpreter is giving me the error when I try to parse json_data["data"]["items"][0] in my for loop, it appears to be evaluating x as a string, because I am getting an error telling me that string indicies must be integers.
I didn't realize json_data["data"]["items"][0] would yield a string. I'm somewhat stymied. Would it have anything to do with using json_data["data"]["items"][0] as opposed to json_data["data"]["items"]?
I have also tried json.loads(json_data["data"]["items"][0]) to no avail. Any input is appreciated.
Thanks!

By doing some basic debugging and data examination you will find that the use of the [0] is incorrect
I quickly loaded that data and tried it
In [11]: for i in x['data']['items'][0]:
....: print i
....:
time_completed
xid
details
title
date
shared
sub_type
time_created
time_updated
And this is effectively what you got, but not what you expected.
So what is going on? If you you do the following
x['data']['items'][0]
Out[28]:
{'date': 20150706,
'details': {'asleep_time': 1436165760,
'awake': 1265,
'awake_time': 1436193600,
'awakenings': 1,
'body': 0,
'duration': 28871,
'light': 13258,
'mind': 0,
'quality': 81,
'rem': 0,
'smart_alarm_fire': 1436193000,
'sound': 14348,
'sunrise': 1436184840,
'sunset': 1436156340,
'tz': 'America/Vancouver'},
'shared': True,
'sub_type': 0,
'time_completed': 1436193938,
'time_created': 1436165067,
'time_updated': 1436194828,
'title': 'for 7h 40m',
'xid': '4tLz2M3x8_E9mTPIQsQUdrj-83spikha'}
The result is as you expect.
The answer is because you are trying to iterate over x['data']['items'][0]
When you iterate over a dictionary you will get the keys not the values by default. Hence you are getting the unexpected string.
Where as the following code will give you what you want.
In [30]: for i in x['data']['items']:
print i['details']['duration']
....:
28871
1800
2110
2019
This iterates over each item (a dictioanary) in the list of items.
I would suggest the downvote is due to a large posting of code, and no real evidence of basic debugging.