Develop Reference

how do I access this json data in python?

hi I'm pretty new at coding and I was trying to create a program in python that reads and save in another file the data inside a json file (not everything, just what I want). I googled how to parse data but there's something I don't understand.
that's a part of the json file:
`
{
"profileRevision": 548789,
"profileId": "campaign",
"profileChangesBaseRevision": 548789,
"profileChanges": [
{
"changeType": "fullProfileUpdate",
"profile": {
"_id": "2da4f079f8984cc48e84fc99dace495d",
"created": "2018-03-29T11:02:15.190Z",
"updated": "2022-10-31T17:34:43.284Z",
"rvn": 548789,
"wipeNumber": 9,
"accountId": "63881e614ef543b2932c70fed1196f34",
"profileId": "campaign",
"version": "refund_teddy_perks_september_2022",
"items": {
"8ec8f13f-6bf6-4933-a7db-43767a055e66": {
"templateId": "Quest:heroquest_loadout_constructor_2",
"attributes": {
"quest_state": "Claimed",
"creation_time": "min",
"last_state_change_time": "2019-05-18T16:09:12.750Z",
"completion_complete_pve03_diff26_loadout_constructor": 300,
"level": -1,
"item_seen": true,
"sent_new_notification": true,
"quest_rarity": "uncommon",
"xp_reward_scalar": 1
},
"quantity": 1
},
"6940c71b-c74b-4581-9f1e-c0a87e246884": {
"templateId": "Worker:workerbasic_sr_t01",
"attributes": {
"gender": "2",
"personality": "Homebase.Worker.Personality.IsDreamer",
"level": 1,
"item_seen": true,
"squad_slot_idx": -1,
"portrait": "WorkerPortrait:IconDef-WorkerPortrait-Dreamer-F02",
"building_slot_used": -1,
"set_bonus": "Homebase.Worker.SetBonus.IsMeleeDamageLow"
}
}
}
]
}
`
I can access profileChanges. I wrote this to create another json file with only the profileChanges things:
`
myjsonfile= open("file.json",'r')
jsondata=myjsonfile.read()
obj=json.loads(jsondata)
ciso=obj['profileChanges']
for i in ciso:
print(i)
with open("file2", "w") as outfile:
json.dump( ciso, outfile, indent=1)
the issue I have is that I can't access "profile" (inside profileChanges) in the same way by parsing the new file and I have no idea on how to do it

Access to JSON or dict element is realized by list indexes, please look at below example:
a = [
{
"friends": [
{
"id": 0,
"name": "Reba May"
}
],
"greeting": "Hello, Doris Gallagher! You have 2 unread messages.",
"favoriteFruit": "strawberry"
},
]
b = a['friends']['id] # b = 0

I've added a couple of closing braces to make your snippet valid json:
s = '''{
"profileRevision": 548789,
"profileId": "campaign",
"profileChangesBaseRevision": 548789,
"profileChanges": [
{
"changeType": "fullProfileUpdate",
"profile": {
"_id": "2da4f079f8984cc48e84fc99dace495d",
"created": "2018-03-29T11:02:15.190Z",
"updated": "2022-10-31T17:34:43.284Z",
"rvn": 548789,
"wipeNumber": 9,
"accountId": "63881e614ef543b2932c70fed1196f34",
"profileId": "campaign",
"version": "refund_teddy_perks_september_2022",
"items": {
"8ec8f13f-6bf6-4933-a7db-43767a055e66": {
"templateId": "Quest:heroquest_loadout_constructor_2",
"attributes": {
"quest_state": "Claimed",
"creation_time": "min",
"last_state_change_time": "2019-05-18T16:09:12.750Z",
"completion_complete_pve03_diff26_loadout_constructor": 300,
"level": -1,
"item_seen": true,
"sent_new_notification": true,
"quest_rarity": "uncommon",
"xp_reward_scalar": 1
},
"quantity": 1
},
"6940c71b-c74b-4581-9f1e-c0a87e246884": {
"templateId": "Worker:workerbasic_sr_t01",
"attributes": {
"gender": "2",
"personality": "Homebase.Worker.Personality.IsDreamer",
"level": 1,
"item_seen": true,
"squad_slot_idx": -1,
"portrait": "WorkerPortrait:IconDef-WorkerPortrait-Dreamer-F02",
"building_slot_used": -1,
"set_bonus": "Homebase.Worker.SetBonus.IsMeleeDamageLow"
}
}
}
}
}
]
}
'''
d = json.loads(s)
print(d['profileChanges'][0]['profile']['version'])
This prints refund_teddy_perks_september_2022
Explanation:
d is a dict
d['profileChanges'] is a list of dicts
d['profileChanges'][0] is the first dict in the list
d['profileChanges'][0]['profile'] is a dict
d['profileChanges'][0]['profile']['version'] is the value of version key in the profile dict in the first entry of the profileChanges list.

Type error : list indices must be integers or slices, not dict

I am trying to read json data and writing to csv file but not working. How to approach for it to get my expected output
Code :
import json
import csv
import ast
# Using json loads to load data from varaible which holds json data
jdata = json.loads(data)
# Fetching all headers from json and converting into list
jkey = list(jdata[0].keys())
# get length
glen_count = len(data)
# Looping through the keys to write data to csv file
for r in range(glen_count):
for hd in jkey:
with open("/content/sample_data/abc.csv",'w') as wr:
csv_wr = csv.DictWriter(wr,fieldnames=jkey)
csv_wr.writeheader()
csv_wr.writerows(jdata[jdata[r][hd]]) # error : Type error : list indices must be integers or slices, not dict
My variable data contain json
data = '''
[
{
"_id": {
"$oid": "5fc7897969b8ab14e1cad8f4"
},
"m": "24:6F:28:55:7C:55",
"t": "o",
"pId": {
"$oid": "5fc7894ff7fa2b262ecb6c46"
},
"pVal": {
"title": "Switch",
"val": 1,
"status": null,
"portType": "AC",
"portIcon": "AC",
"_id": "5fc7894ff7fa2b262ecb6c46",
"no": "0",
"cat": "s",
"device": {
"ports": [
"5fc7894ff7fa2b262ecb6c46"
],
"_id": "5fc7894ff7fa2b262ecb6c45",
"macId": "24:6F:28:55:7C:55",
"deviceId": "LTAC21220050009",
"dType": "5ee38d3a4afdf74cf8b07dfb",
"createdAt": "2020-12-02T12:32:15.235Z",
"updatedAt": "2020-12-02T12:32:36.773Z",
"__v": 0,
"assignedTo": "5fc62223f7fa2b262ecb6c2d"
},
"mac": "24:6F:28:55:7C:55",
"__v": 0,
"createdAt": "2020-12-02T12:32:15.237Z",
"updatedAt": "2020-12-02T12:32:57.776Z"
},
"dId": {
"$oid": "5fc7894ff7fa2b262ecb6c45"
},
"createdAt": {
"$date": "2020-12-02T12:32:57.781Z"
},
"updatedAt": {
"$date": "2020-12-02T12:32:57.781Z"
},
"__v": 0
},{
"_id": {
"$oid": "5fc7897b69b8ab14e1cad8f5"
},
"m": "24:6F:28:55:7C:55",
"t": "o",
"pId": {
"$oid": "5fc7894ff7fa2b262ecb6c46"
},
"pVal": {
"title": "Switch",
"val": 0,
"status": null,
"portType": "AC",
"portIcon": "AC",
"_id": "5fc7894ff7fa2b262ecb6c46",
"no": "0",
"cat": "s",
"device": {
"ports": [
"5fc7894ff7fa2b262ecb6c46"
],
"_id": "5fc7894ff7fa2b262ecb6c45",
"macId": "24:6F:28:55:7C:55",
"deviceId": "LTAC21220050009",
"dType": "5ee38d3a4afdf74cf8b07dfb",
"createdAt": "2020-12-02T12:32:15.235Z",
"updatedAt": "2020-12-02T12:32:36.773Z",
"__v": 0,
"assignedTo": "5fc62223f7fa2b262ecb6c2d"
},
"mac": "24:6F:28:55:7C:55",
"__v": 0,
"createdAt": "2020-12-02T12:32:15.237Z",
"updatedAt": "2020-12-02T12:32:59.951Z"
},
"dId": {
"$oid": "5fc7894ff7fa2b262ecb6c45"
},
"createdAt": {
"$date": "2020-12-02T12:32:59.955Z"
},
"updatedAt": {
"$date": "2020-12-02T12:32:59.955Z"
},
"__v": 0
}
]
Expected output is need to write to csv file : column : id , val , createdAt , updatedAt
id , val , createdAt , updatedAt
5fc7897969b8ab14e1cad8f4 , 0 , 2020-12-02T12:32:57.781Z , 2020-12-02T12:32:57.781Z
5fc7897b69b8ab14e1cad8f5 , 0 , 2020-12-02T12:32:59.955Z , 2020-12-02T12:32:59.955Z

Okay, a few things:
jkey = list(jdata[0].keys()) will give you jkeys = ['_id', 'm', 't', 'pId', 'pVal', 'dId', 'createdAt', 'updatedAt', '__v'], which is more than your desired keys. If your goal is to select only id , val , createdAt , updatedAt, jkeys must reflect that as jkeys = ['id' , 'val' , 'createdAt' , 'updatedAt']. It's the value you're passing to the fieldnames, after all. Those are the header names, and it identifies the order in which the values will be written.
glen_count = len(data) is likely a typo and you meant to write jdata, data is the string containing the json data, so you would be counting the length of this string.
The error you're going through is because the return of the expression jdata[jdata[r][hd]] is {'$oid': '5fc7897969b8ab14e1cad8f4'}, a dict. You are using this one as a index for the list jdata.
About DictWriter and writerows: You pass to its constructor specific fieldnames, in your case, it's jkeys = ['id' , 'val' , 'createdAt' , 'updatedAt']. This is your desired header. You must then either use a list of dictionaries with writerows or a single dictionary with writerow. Writerows will have you write all instances at once, and writerow is a single instance at a time. See CSV File Reading and Writing as a reference.
Careful with looping through with open("/content/sample_data/abc.csv",'w') as wr:, you'll be rewriting your own file! See Reading and Writing Files to learn about the writing modes!
I will post the following as a solution, using writerows to separate the logic of writing to a file and the logic of setting up the rows. Hopefully those are the headers desired. If not, it's easy to change according to your own needs.
import json
import csv
import ast
jdata = json.loads(data)
# Only desired headers
jkeys = ['id' , 'val' , 'createdAt' , 'updatedAt']
# setting up rows
list_rows = []
for el in jdata:
dict_jdata = {}
dict_jdata['id'] = el['_id']['$oid']
dict_jdata['val'] = str(el['pVal']['val'])
dict_jdata['createdAt'] = el['createdAt']['$date']
dict_jdata['updatedAt'] = el['updatedAt']["$date"]
list_rows.append(dict_jdata)
# Writing to a file
with open("/content/sample_data/abc2.csv",'w') as wr:
csv_wr = csv.DictWriter(wr,fieldnames=jkeys)
csv_wr.writeheader()
csv_wr.writerows(list_rows)
Hopefully this helped :)

How can I fix this regex to match an object in Json and replace it as a list of Object

I have tried the following but i am failing to match the object in Json
:\s*(\{[^\"]*\})
I want to know the way to replace the object type in Json as list of object.
Here is the sample of Json:
{
"resourceType": "ChargeItem",
"id": "example",
"text": {
"status": "generated",
"session": "Done"
},
"identifier": [
{
"system": "http://myHospital.org/ChargeItems",
"value": "654321"
}
],
"definitionUri": [
"http://www.kbv.de/tools/ebm/html/01520_2904360860826220813632.html"
],
"status": "billable",
"code": {
"coding": [
{
"code": "01510",
"display": "Zusatzpauschale für Beobachtung nach diagnostischer Koronarangiografie"
}
]
}
}
I need i want to convert to this form:
{
"resourceType": "ChargeItem",
"id": "example",
"text": [{
"status": "generated",
"session": "Done"
}],
"identifier": [
{
"system": "http://myHospital.org/ChargeItems",
"value": "654321"
}
],
"definitionUri": [
"http://www.kbv.de/tools/ebm/html/01520_2904360860826220813632.html"
],
"status": "billable",
"code": [{
"coding": [
{
"code": "01510",
"display": "Zusatzpauschale für Beobachtung nach diagnostischer Koronarangiografie"
}
]
}]
}

This appears to be a few simple transformations:
First, change
"text": {
to
"text": [{
Second, change
},
"identifier": [
to
}],
"identifier": [
Third, change
"code": {
to
"code": [{
And finally, change
}
}
<EOF>
to
}]
}
<EOF>
However, it might not be as straightforward as it appears, i.e. what if the identifer section isn't always present, or doesn't immediately follow the text section?
Regular expressions are a poor choice for doing this work. It would be much better to read the json file into a native Python data structure, apply your desired changes, then save the json back to the file.

Solution using a multiline regexp search
>>> import re
>>> blocks = re.compile(r'(?ms)(.*)("text": )([{][^{}]+[}])(,.*"status": "billable"[^"]+)("code": )([{][^"]+"coding":[^]]+\]\s+\})')
>>> m = blocks.search(s)
>>> result = ""
>>> for i in range(1,len(m.groups()) + 1):
... if i not in (3,6):
... result += m.group(i)
... else:
... result += "[" + m.group(i) + "]"
...
>>> result += "\n}"

JSON or Python dict / list decoding problem

I have been using the Python script below to try and retrieve and extract some data from Flightradar24, it would appear that it extracts the data in JSON format and will print the data out ok fully using json.dumps, but when I attempt to select the data I want (the status text in this case) using get it gives the following error:
'list' object has no attribute 'get'
Is the Data in JSON or a List ? I'm totally confused now.
I'm fairly new to working with data in JSON format, any help would be appreciated!
Script:
import flightradar24
import json
flight_id = 'BA458'
fr = flightradar24.Api()
flight = fr.get_flight(flight_id)
y = flight.get("data")
print (json.dumps(flight, indent=4))
X= (flight.get('result').get('response').get('data').get('status').get('text'))
print (X)
Sample of output data:
{
"result": {
"request": {
"callback": null,
"device": null,
"fetchBy": "flight",
"filterBy": null,
"format": "json",
"limit": 25,
"page": 1,
"pk": null,
"query": "BA458",
"timestamp": null,
"token": null
},
"response": {
"item": {
"current": 16,
"total": null,
"limit": 25
},
"page": {
"current": 1,
"total": null
},
"timestamp": 1546241512,
"data": [
{
"identification": {
"id": null,
"row": 4852575431,
"number": {
"default": "BA458",
"alternative": null
},
"callsign": null,
"codeshare": null
},
"status": {
"live": false,
"text": "Scheduled",
"icon": null,
"estimated": null,
"ambiguous": false,
"generic": {
"status": {
"text": "scheduled",
"type": "departure",
"color": "gray",
"diverted": null
},

You can use print(type(variable_name)) to see what type it is. The .get(key[,default]) is not supported on lists - it is supported for dict's.
X = (flight.get('result').get('response').get('data').get('status').get('text'))
# ^^^^^^^^ does not work, data is a list of dicts
as data is a list of dicts:
"data": [ # <<<<<< this is a list
{
"identification": {
"id": null,
"row": 4852575431,
"number": {
"default": "BA458",
"alternative": null
},
"callsign": null,
"codeshare": null
},
"status": {
This should work:
X = (flight.get('result').get('response').get('data')[0].get('status').get('text')

The issue, as pointed out by #PatrickArtner, is your data is actually a list rather than a dictionary. As an aside, you may find your code more readable if you were to use a helper function to apply dict.get repeatedly on a nested dictionary:
from functools import reduce
def ng(dataDict, mapList):
"""Nested Getter: Iterate nested dictionary"""
return reduce(dict.get, mapList, dataDict)
X = ng(ng(flight, ['result', 'response', 'data'])[0], ['status'[, 'text']])

Getting Deeper Level JSON Values in Python

I have a Python script that make an API call to retrieve data from Zendesk. (Using Python 3.x) The JSON object has a structure like this:
{
"id": 35436,
"url": "https://company.zendesk.com/api/v2/tickets/35436.json",
"external_id": "ahg35h3jh",
"created_at": "2009-07-20T22:55:29Z",
"updated_at": "2011-05-05T10:38:52Z",
"type": "incident",
"subject": "Help, my printer is on fire!",
"raw_subject": "{{dc.printer_on_fire}}",
"description": "The fire is very colorful.",
"priority": "high",
"status": "open",
"recipient": "support#company.com",
"requester_id": 20978392,
"submitter_id": 76872,
"assignee_id": 235323,
"organization_id": 509974,
"group_id": 98738,
"collaborator_ids": [35334, 234],
"forum_topic_id": 72648221,
"problem_id": 9873764,
"has_incidents": false,
"due_at": null,
"tags": ["enterprise", "other_tag"],
"via": {
"channel": "web"
},
"custom_fields": [
{
"id": 27642,
"value": "745"
},
{
"id": 27648,
"value": "yes"
}
],
"satisfaction_rating": {
"id": 1234,
"score": "good",
"comment": "Great support!"
},
"sharing_agreement_ids": [84432]
}
Where I am running into issues is in the "custom_fields" section specifically. I have a particular custom field inside of each ticket I need the value for, and I only want that particular value.
To spare you too many specifics of the Python code, I am reading through each value below for each ticket and adding it to an output variable before writing that output variable to a .csv. Here is the particular place the breakage is occuring:
output += str(ticket['custom_fields'][id:23825198]).replace(',', '')+','
All the replace nonsense is to make sure that since it is going into a comma delimited file, any commas inside of the values are removed. Anyway, here is the error I am getting:
output += str(ticket['custom_fields'][id:int(23825198)]).replace(',', '')+','
TypeError: slice indices must be integers or None or have an __index__ method
As you can see I have tried a couple different variations of this to try and resolve the issue, and have yet to find a fix. I could use some help!
Thanks...

Are you using json.loads()? If so you can then get the keys, and do an if statement against the keys. An example on how to get the keys and their respective values is shown below.
import json
some_json = """{
"id": 35436,
"url": "https://company.zendesk.com/api/v2/tickets/35436.json",
"external_id": "ahg35h3jh",
"created_at": "2009-07-20T22:55:29Z",
"updated_at": "2011-05-05T10:38:52Z",
"type": "incident",
"subject": "Help, my printer is on fire!",
"raw_subject": "{{dc.printer_on_fire}}",
"description": "The fire is very colorful.",
"priority": "high",
"status": "open",
"recipient": "support#company.com",
"requester_id": 20978392,
"submitter_id": 76872,
"assignee_id": 235323,
"organization_id": 509974,
"group_id": 98738,
"collaborator_ids": [35334, 234],
"forum_topic_id": 72648221,
"problem_id": 9873764,
"has_incidents": false,
"due_at": null,
"tags": ["enterprise", "other_tag"],
"via": {
"channel": "web"
},
"custom_fields": [
{
"sid": 27642,
"value": "745"
},
{
"id": 27648,
"value": "yes"
}
],
"satisfaction_rating": {
"id": 1234,
"score": "good",
"comment": "Great support!"
},
"sharing_agreement_ids": [84432]
}"""
# load the json object
zenJSONObj = json.loads(some_json)
# Shows a list of all custom fields
print("All the custom field data")
print(zenJSONObj['custom_fields'])
print("----")
# Tells you all the keys in the custom_fields
print("How keys and the values")
for custom_field in zenJSONObj['custom_fields']:
print("----")
for key in custom_field.keys():
print("key:",key," value: ",custom_field[key])
You can then modify the JSON object by doing something like
print(zenJSONObj['custom_fields'][0])
zenJSONObj['custom_fields'][0]['value'] = 'something new'
print(zenJSONObj['custom_fields'][0])
Then re-encode it using the following:
newJSONObject = json.dumps(zenJSONObj, sort_keys=True, indent=4)
I hope this is of some help.