Develop Reference

How do you parsing nested JSON data for specific information?

I'm using the national weather service API and when you use a specific URL you get JSON data back. My program so far grabs everything including 155 hours of weather data.
Simply put I'm trying to parse the data and grab the weather for the
latest hour but everything is in a nested data structure.
My code, JSON data, and more information are below. Any help is appreciated.
import requests
import json
def get_current_weather(): #This method returns json data from the api
url = 'https://api.weather.gov/gridpoints/*office*/*any number,*any number*/forecast/hourly'
response = requests.get(url)
full_data = response.json()
return full_data
def main(): #Prints the information grabbed from the API
print(get_current_weather())
if __name__ == "__main__":
main()
In the JSON response, I get there are 3 layers before you get to the 'shortForecast' data that I'm trying to get. The first nest is 'properties, everything before it is irrelevant to my program. The second nest is 'periods' and each period is a new hour, 0 being the latest. Lastly, I just need to grab the 'shortForcast' in the first period or periods[0].
{
"#context": [
"https://geojson.org/geojson-ld/geojson-context.jsonld",
{
"#version": "1.1",
"wx": "https://api.weather.gov/ontology#",
"geo": "http://www.opengis.net/ont/geosparql#",
"unit": "http://codes.wmo.int/common/unit/",
"#vocab": "https://api.weather.gov/ontology#"
}
],
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
*data I'm not gonna add*
]
]
},
"properties": {
"updated": "2021-02-11T05:57:24+00:00",
"units": "us",
"forecastGenerator": "HourlyForecastGenerator",
"generatedAt": "2021-02-11T07:12:58+00:00",
"updateTime": "2021-02-11T05:57:24+00:00",
"validTimes": "2021-02-10T23:00:00+00:00/P7DT14H",
"elevation": {
"value": ,
"unitCode": "unit:m"
},
"periods": [
{
"number": 1,
"name": "",
"startTime": "2021-02-11T02:00:00-05:00",
"endTime": "2021-02-11T03:00:00-05:00",
"isDaytime": false,
"temperature": 18,
"temperatureUnit": "F",
"temperatureTrend": null,
"windSpeed": "10 mph",
"windDirection": "N",
"icon": "https://api.weather.gov/icons/land/night/snow,40?size=small",
"shortForecast": "Chance Light Snow",
"detailedForecast": ""
},
{
"number": 2,
"name": "",
"startTime": "2021-02-11T03:00:00-05:00",
"endTime": "2021-02-11T04:00:00-05:00",
"isDaytime": false,
"temperature": 17,
"temperatureUnit": "F",
"temperatureTrend": null,
"windSpeed": "12 mph",
"windDirection": "N",
"icon": "https://api.weather.gov/icons/land/night/snow,40?size=small",
"shortForecast": "Chance Light Snow",
"detailedForecast": ""
},
OK, so I didn't want to edit everything again so this is the new get_current_weather method. I was able to get to 'periods but after that I'm still stumped. This is the new method.
def get_current_weather():
url = 'https://api.weather.gov/gridpoints/ILN/82,83/forecast/hourly'
response = requests.get(url)
full_data = response.json()
return full_data['properties'].get('periods')

For the dictionary object, you can access the nested elements by using indexing multiple times.
So, for your dictionary object, you can use the following to get the value for the key shortForecast for the first element in the list of dictionaries under key periods under the key properties in the main dictionary:
full_data['properties']['periods'][0]['shortForecast']

Parsing list of dictionaries in a dictionary to retrieve a specific key's value from each dictionary

I got a JSON response and converted it to a python dictionary using json.loads(). So the dictionary looks like this:
{u'body': u'[{"id":"1","entity":"zone","status":"PROCESSING","url":null,"createdOn":"2019-10-11T05:49:11Z"},{"id":"2","entity":"floor","status":"FAILED","url":null,"createdOn":"2019-10-11T05:49:15Z"},{"id":"3","entityType":"apartment","status":"SUCCESS","url":null,"createdOn":"2019-10-11T05:49:18Z"}]',u'isBase64Encoded': False, u'statusCode': 200}
I named this as testStatusList. I want to retrieve the value of "status" key of every dictionary inside "body". I was able to retrieve the "body" by giving body = testStatusList['body']. Now, the dictionary looks like:
[
{
"id": "1",
"entityType": "zone",
"status": "PROCESSING",
"url": null,
"createdOn": "2019-03-07T12:47:10Z"
},
{
"id": "2",
"entityType": "floor",
"status": "FAILED",
"url": null,
"createdOn": "2019-08-19T16:46:13Z"
},
{
"id": "3",
"entityType": "apartment",
"status": "SUCCESS",
"url": null,
"createdOn": "2019-08-19T16:46:13Z"
}
]
I tried out this solution [Parsing a dictionary to retrieve a key in Python 3.6
testStatusList= json.loads(status_response['Payload'].read())
body = testStatusList['body']
status =[]
for b in body:
for k,v in b.items():
if k == 'status':
status.append(v)
but I keep getting AttributeError: 'unicode' object has no attribute 'items'. Is there a different method to get items for unicode objects?
So I basically want to retrieve all the statuses i.e., PROCESSING, FAILED AND SUCCESS so that I can put an 'if' condition to display appropriate messages when something failed for that particular "id". I am very unsure about my approach as I am totally new to Python. Any help would be much appreciated thanks!

body is still a (unicode) string in your top blob. Use json.loads again on that string:
body = """[
{
"id": "1",
"entityType": "zone",
"status": "PROCESSING",
"url": null,
"createdOn": "2019-03-07T12:47:10Z"
},
{
"id": "2",
"entityType": "floor",
"status": "FAILED",
"url": null,
"createdOn": "2019-08-19T16:46:13Z"
},
{
"id": "3",
"entityType": "apartment",
"status": "SUCCESS",
"url": null,
"createdOn": "2019-08-19T16:46:13Z"
}
]"""
import json
body = json.loads(body)
status =[]
for b in body:
for k,v in b.items():
if k == 'status':
status.append(v)
print(status)
Result:
['PROCESSING', 'FAILED', 'SUCCESS']

JSON or Python dict / list decoding problem

I have been using the Python script below to try and retrieve and extract some data from Flightradar24, it would appear that it extracts the data in JSON format and will print the data out ok fully using json.dumps, but when I attempt to select the data I want (the status text in this case) using get it gives the following error:
'list' object has no attribute 'get'
Is the Data in JSON or a List ? I'm totally confused now.
I'm fairly new to working with data in JSON format, any help would be appreciated!
Script:
import flightradar24
import json
flight_id = 'BA458'
fr = flightradar24.Api()
flight = fr.get_flight(flight_id)
y = flight.get("data")
print (json.dumps(flight, indent=4))
X= (flight.get('result').get('response').get('data').get('status').get('text'))
print (X)
Sample of output data:
{
"result": {
"request": {
"callback": null,
"device": null,
"fetchBy": "flight",
"filterBy": null,
"format": "json",
"limit": 25,
"page": 1,
"pk": null,
"query": "BA458",
"timestamp": null,
"token": null
},
"response": {
"item": {
"current": 16,
"total": null,
"limit": 25
},
"page": {
"current": 1,
"total": null
},
"timestamp": 1546241512,
"data": [
{
"identification": {
"id": null,
"row": 4852575431,
"number": {
"default": "BA458",
"alternative": null
},
"callsign": null,
"codeshare": null
},
"status": {
"live": false,
"text": "Scheduled",
"icon": null,
"estimated": null,
"ambiguous": false,
"generic": {
"status": {
"text": "scheduled",
"type": "departure",
"color": "gray",
"diverted": null
},

You can use print(type(variable_name)) to see what type it is. The .get(key[,default]) is not supported on lists - it is supported for dict's.
X = (flight.get('result').get('response').get('data').get('status').get('text'))
# ^^^^^^^^ does not work, data is a list of dicts
as data is a list of dicts:
"data": [ # <<<<<< this is a list
{
"identification": {
"id": null,
"row": 4852575431,
"number": {
"default": "BA458",
"alternative": null
},
"callsign": null,
"codeshare": null
},
"status": {
This should work:
X = (flight.get('result').get('response').get('data')[0].get('status').get('text')

The issue, as pointed out by #PatrickArtner, is your data is actually a list rather than a dictionary. As an aside, you may find your code more readable if you were to use a helper function to apply dict.get repeatedly on a nested dictionary:
from functools import reduce
def ng(dataDict, mapList):
"""Nested Getter: Iterate nested dictionary"""
return reduce(dict.get, mapList, dataDict)
X = ng(ng(flight, ['result', 'response', 'data'])[0], ['status'[, 'text']])

Getting an element of a json?

I have a json and I want to get a element of it by python. the json look like this:
{"$id":"1","Data":
[{"$id":"2","ArticleId":39511,"Title":"a","Order":39,"TotalCount":0},
{"$id":"3","ArticleId":345921,"Title":"a","Order":21,"TotalCount":0},
{"$id":"4","ArticleId":43289,"Title":"a","Order":18,"TotalCount":0},
{"$id":"5","ArticleId":39540,"Title":"a","Order":16,"TotalCount":0},
{"$id":"6","ArticleId":963167,"Title":"a","Order":15,"TotalCount":0},
{"$id":"7","ArticleId":519284,"Title":"a","Order":14,"TotalCount":0},
{"$id":"8","ArticleId":109699,"Title":"a","Order":13,"TotalCount":0},
{"$id":"9","ArticleId":43466,"Title":"a","Order":12,"TotalCount":0},
{"$id":"10","ArticleId":39510,"Title":"a","Order":10,"TotalCount":0},
{"$id":"11","ArticleId":43307,"Title":"a","Order":9,"TotalCount":0},
{"$id":"12","ArticleId":715048,"Title":"a","Order":9,"TotalCount":0},
{"$id":"13","ArticleId":133952,"Title":"a","Order":8,"TotalCount":0},
{"$id":"14","ArticleId":459712,"Title":"a","Order":8,"TotalCount":0},
{"$id":"15","ArticleId":11503,"Title":"a","Order":7,"TotalCount":0}],"TotalCount":
14,"Id":"0","IsSuccess":true,"Message":"OK","StatusCode":200,"ValidationMessages":
null,"Exception":null}
In python I use:
mydata = json.loads(response.text)
now I want to know how to use mydata to get the strings. I mean "a"

You can do it with:
mydata = json.loads(response_text)
titles = [dct['Title'] for dct in mydata['Data']]
A good way to help figure things like this out is to first do something like this:
print(json.dumps(mydata, indent=4))
Which, in this case, results in something like this being displayed:
{
"Exception": null,
"$id": "1",
"ValidationMessages": null,
"IsSuccess": true,
"TotalCount": 14,
"Message": "OK",
"Data": [
{
"$id": "2",
"TotalCount": 0,
"ArticleId": 39511,
"Order": 39,
"Title": "a"
},
{
"$id": "3",
"TotalCount": 0,
"ArticleId": 345921,
"Order": 21,
"Title": "a"
},
..., etc,
{
"$id": "15",
"TotalCount": 0,
"ArticleId": 11503,
"Order": 7,
"Title": "a"
}
],
"Id": "0",
"StatusCode": 200
}
From that, it usually becomes relatively easy to determine what you need to do to access any part or parts of the object.

Getting Deeper Level JSON Values in Python

I have a Python script that make an API call to retrieve data from Zendesk. (Using Python 3.x) The JSON object has a structure like this:
{
"id": 35436,
"url": "https://company.zendesk.com/api/v2/tickets/35436.json",
"external_id": "ahg35h3jh",
"created_at": "2009-07-20T22:55:29Z",
"updated_at": "2011-05-05T10:38:52Z",
"type": "incident",
"subject": "Help, my printer is on fire!",
"raw_subject": "{{dc.printer_on_fire}}",
"description": "The fire is very colorful.",
"priority": "high",
"status": "open",
"recipient": "support#company.com",
"requester_id": 20978392,
"submitter_id": 76872,
"assignee_id": 235323,
"organization_id": 509974,
"group_id": 98738,
"collaborator_ids": [35334, 234],
"forum_topic_id": 72648221,
"problem_id": 9873764,
"has_incidents": false,
"due_at": null,
"tags": ["enterprise", "other_tag"],
"via": {
"channel": "web"
},
"custom_fields": [
{
"id": 27642,
"value": "745"
},
{
"id": 27648,
"value": "yes"
}
],
"satisfaction_rating": {
"id": 1234,
"score": "good",
"comment": "Great support!"
},
"sharing_agreement_ids": [84432]
}
Where I am running into issues is in the "custom_fields" section specifically. I have a particular custom field inside of each ticket I need the value for, and I only want that particular value.
To spare you too many specifics of the Python code, I am reading through each value below for each ticket and adding it to an output variable before writing that output variable to a .csv. Here is the particular place the breakage is occuring:
output += str(ticket['custom_fields'][id:23825198]).replace(',', '')+','
All the replace nonsense is to make sure that since it is going into a comma delimited file, any commas inside of the values are removed. Anyway, here is the error I am getting:
output += str(ticket['custom_fields'][id:int(23825198)]).replace(',', '')+','
TypeError: slice indices must be integers or None or have an __index__ method
As you can see I have tried a couple different variations of this to try and resolve the issue, and have yet to find a fix. I could use some help!
Thanks...

Are you using json.loads()? If so you can then get the keys, and do an if statement against the keys. An example on how to get the keys and their respective values is shown below.
import json
some_json = """{
"id": 35436,
"url": "https://company.zendesk.com/api/v2/tickets/35436.json",
"external_id": "ahg35h3jh",
"created_at": "2009-07-20T22:55:29Z",
"updated_at": "2011-05-05T10:38:52Z",
"type": "incident",
"subject": "Help, my printer is on fire!",
"raw_subject": "{{dc.printer_on_fire}}",
"description": "The fire is very colorful.",
"priority": "high",
"status": "open",
"recipient": "support#company.com",
"requester_id": 20978392,
"submitter_id": 76872,
"assignee_id": 235323,
"organization_id": 509974,
"group_id": 98738,
"collaborator_ids": [35334, 234],
"forum_topic_id": 72648221,
"problem_id": 9873764,
"has_incidents": false,
"due_at": null,
"tags": ["enterprise", "other_tag"],
"via": {
"channel": "web"
},
"custom_fields": [
{
"sid": 27642,
"value": "745"
},
{
"id": 27648,
"value": "yes"
}
],
"satisfaction_rating": {
"id": 1234,
"score": "good",
"comment": "Great support!"
},
"sharing_agreement_ids": [84432]
}"""
# load the json object
zenJSONObj = json.loads(some_json)
# Shows a list of all custom fields
print("All the custom field data")
print(zenJSONObj['custom_fields'])
print("----")
# Tells you all the keys in the custom_fields
print("How keys and the values")
for custom_field in zenJSONObj['custom_fields']:
print("----")
for key in custom_field.keys():
print("key:",key," value: ",custom_field[key])
You can then modify the JSON object by doing something like
print(zenJSONObj['custom_fields'][0])
zenJSONObj['custom_fields'][0]['value'] = 'something new'
print(zenJSONObj['custom_fields'][0])
Then re-encode it using the following:
newJSONObject = json.dumps(zenJSONObj, sort_keys=True, indent=4)
I hope this is of some help.