Develop Reference

"TypeError: string indices must be integers" after reading from a JSON file

I am trying to duplicate a simple code in my reading material where I want to extract data from a JSON file and plot dots at the capitals of countries on a map.
Regarding my issue,
Traceback (most recent call last):
File "C:\Users\serta\Desktop\python\db\capitals.py", line 14, in <module>
lons.append(cp_dicts['geometries']['coordinates'][0])
TypeError: string indices must be integers
[Finished in 188ms]
I read similar posts here and I think I understand the "why" of the issue,I double checked my []'s and the depths of the nests but I cannot seem to fix it myself.
I am pretty sure I am looking at an integer where I target with my code (lons.append line) but I am still getting the "TypeError: string indices must be integers".
Here is the code:
import json
from plotly.graph_objs import Scattergeo, Layout
from plotly import offline
#Explore the structure of the data.
filename = 'data/capitals.topo.json'
with open (filename) as f:
all_cp_data = json.load(f)
all_cp_dicts=all_cp_data
lons,lats,hover_texts =[],[],[]
for cp_dicts in all_cp_dicts['objects']['capitals']:
lons.append(cp_dicts['geometries']['coordinates'][0])
lats.append(cp_dicts['geometries']['coordinates'][1])
hover_texts.append(cp_dicts['properties']['capital'])
#Map the earthquakes.
data = [{
'type':'scattergeo',
'lon':lons,
'lat':lats,
'text': hover_texts,
'marker':{
'size': [5],
#'color': mags,
#'colorscale': 'plasma',
#'reversescale':True,
#'colorbar':{'title':'Magnitude'},
},
}]
my_layout = Layout(title="Capital Cities")
fig ={'data':data, 'layout':my_layout}
offline.plot(fig, filename ='capital_cities.html')
Here is also the capitals.topo.json I am using:
{
"type": "Topology",
"objects": {
"capitals": {
"type": "GeometryCollection",
"geometries": [
{
"type": "Point",
"coordinates": [
90.24,
23.43
],
"id": "BD",
"properties": {
"country": "Bangladesh",
"city": "Dhaka",
"tld": "bd",
"iso3": "BGD",
"iso2": "BD"
}
},

On line 14, this is not valid, given the input data:
lons.append(cp_dicts['geometries']['coordinates'][0]
You need to update the loop along these lines:
for geometry in all_cp_dicts['objects']['capitals']['geometries']:
lons.append(geometry['coordinates'][0])
lats.append(geometry['coordinates'][1])
hover_texts.append(geometry['properties'].get('city', ""))
Note that for some of the locations, the 'cities' key is missing in the json. So you need to handle that when populating the hover_texts list, as shown.
Also, the 'data' variable was not working with Scattergeo. Below is a suggested revision to the syntax
data = Scattergeo(
lat = lats,
lon = lons,
text = hover_texts,
marker = dict(
size = 5
)
)

How to parse the json data using python?

I've been messing around with JSON for some time. I want to get the values of "box" and "text" in this format using python can someone help me out how to resolve this example output:[92,197,162,215,AUTHORS,...!]
{ "form": [ { "box": [ 92,162,197,215], "text": "AUTHORS", "label": "question", "words": [ { "box": [ 92,197,162,215 ],"text": "AUTHORS"} ], "linking": [[0,13]],"id": 0 },
import os
import json
# Directory name consisting of json
file = open('033.json')
data = json.load(file)
result = []
for value in data['form']:
my_dict=[]
my_dict=value.get('box')
print(my_dict)
result.append(my_dict)

Probably like this:
collector = []
for obj in form:
collector.append({"box": obj["box"], "text": obj["text"]})
print(collector)

Okay, few issues with your code -
Why is your list named my_dict? A name should indicate what the object is/ what it contains. Your name does the opposite and if someone works with that code in the future then it will most likely confuse them.
Why are you initializing a list before doing this value.get('box')?
As for the solution, it is a short piece of code that would require 2 lines of code.
result = []
for form_dict in data['form']:
result.append(tuple(form_dict[key]
for key in ('box', 'text') if key in form_dict))
That piece of code would result in this: [([92, 162, 197, 215], 'AUTHORS')] based on the data you provided.
This is assuming that there can be more items in the data['form'] list, otherwise the for loop is not needed.

json dumps not including attribute name

I am writing a python lambda function that reads in a json file from s3 and then will take one of the nodes and send it to another lambda function. Here is my code:
The json snippet I want
"jobstreams": [
{
"jobname": "team-summary",
"bucket": "aaa-bbb",
"key": "team-summary.json"
}
step 1 – convert JSON to python objects for processing
note: these I got from another Stack Overflow guru - thanks!!
def _json_object_hook(d): return namedtuple('X', d.keys())(*d.values())
def json2obj(data): return json.loads(data, object_hook=_json_object_hook)
routes = json2obj(jsonText)
step 2 - I then traverse the python objects and find the json I need and dump it
for jobstream in jobstreams:
x = json.dumps(jobstream, ensure_ascii=False)
Howeever, when I print it out, I only have the values not the attributes. Why is that?
print(json.dumps(jobstream, ensure_ascii=False))
yields
["team-summary", "aaa-bbb", "team-summary.json"]

I'm assuming your full json file looks somewhat like what I have in my example
import json
js = {"jobstreams": [
{
"jobname": "team-summary",
"bucket": "aaa-bbb",
"key": "team-summary.json"
},
{
"jobname": "team-2222",
"bucket": "aaa-2222",
"key": "team-222.json"
}
]}
def extract_by_jobname(jobname):
for d in js['jobstreams']:
if d['jobname'] == jobname:
return d
json.dumps(extract_by_jobname("team-summary"))
# '{"jobname": "team-summary", "bucket": "aaa-bbb", "key": "team-summary.json"}'

I ended up creating a new Dictionary from the list that the json.dumps gave me.
["team-summary", "aaa-bbb", "team-summary.json"]
once i had the new dictionary (that is flat), then i converted that to json.... probably not the most efficient approach but i have other fish to fry. THANKS to all for your help!

python: I need to append a file with json but it does not output a valid json

I am working on a file which prints json encoded messages
{
"Status": "Non_Malicious",
"alert_level": "Low",
"alert_count": 1,
"alert": "",
"hosts_alert": ""
}
NOTE the file is appended every message as above
def jsonconvert(new_line, set_alert_level, set_status, hosts_alert, ip_address_alert, alert_count, title):
data = {}
data['alert_description'] = new_line
data['alert_level'] = set_alert_level
data['Status'] = set_status
data['hosts_alert'] = hosts_alert
data['alert'] = title
json_data = json.dumps(data)
file_write = open("json_file", 'a')
file_write.write(json_data+','+'\n')
file_write.close()
But in order to make it valid, i need to append it as such bellow : which im not sure how to get it done.
{
"isoc": [{
"Status": "Non_Malicious",
"alert_level": "Low",
"alert_count": 1,
"alert": "",
"hosts_alert": ""
}, {
"Status": "Non_Malicious",
"alert_level": "Low",
"alert_count": 1,
"alert": "",
"hosts_alert": ""
}]
}

You can't make valid JSON files by just appending. This is not just a problem with Python.
That's because, as you noted, every open bracket must be closed, and there should not be trailing commas.
Here are some possible solutions that come to my mind:
Use another file format that does not suffer for this limitation, like CSV or YAML or even plain text
Don't append to a file, but use a database instead
Keep all the contents in memory and dump them all at once in a single JSON file, overwriting it every time
At every append operation, delete the closing brackets, append your data then re-add them again. This solution is a ugly hack, so I don't advice it, but it works.

Python Objects and Lists within Dictionary

I'm pretty new to Python, so just working my way through understanding the data sets.
I'm having a little trouble producing the JSON output that is required for the API I am working with.
I am using
import json
json.load(data_file)
Working with Python dictionary and then doing
json.dump(dict, json_data)
My data needs to look like the following when it is output.
{
"event":{
"id":10006,
"event_name":"My Event Name",
},
"sub event":[
],
"attendees":[
{
"id":11201,
"first_name":"Jeff",
"last_name":"Smith",
},
{
"id":10002,
"first_name":"Victoria",
"last_name":"Baker",
},
]
}
I have been able to create the arrays in python and dump to json, but I am having difficulty creating the event "object" in the dictionary. I am using the below:
attendees = ['attendees']
attendeesdict = {}
attendeesdict['first_name'] = "Jeff"
attendees.append(attendeesdict.copy())
Can anyone help me add the "event" object properly?

In general, going from JSON to dictionary is almost no work because the two are very similar, if not identical:
attendees = [
{
"first_name": "Jeff"
# Add other fields which you need here
},
{
"first_name": "Victoria"
}
]
In this instance, attendees is a list of dictionaries. For the event:
event = {
"id": 10006,
"event_name": "My Event Name"
}
Putting it all together:
data = {
"event": event,
"sub event": [],
"attendees": attendees
}
Now, you can convert it to a JSON object, ready to send to your API:
json_object = json.dumps(data)

Assuming you have built all the values elsewhere and now you're just putting them together:
result = {'event':event_dict, 'sub event':subevent_list, 'attendees':attendees_list}

If you want just to statically create a nested dict, you can use a single literal. If you paste the JSON above into python code, you would get a valid dict literal.

Construct your dicts and add like below
{
"event":"add your dict"
"sub event":["add your dict"],
"attendees":["add your dict"]
}