A GeoJson with point features contains two attributes: City and Rating.
City as identifier is never changing, but Rating will be updated on a regular basis.
The new Rating is stored in a dictionary as vales ("dnew").
My for loop is not working well. Please see the code below, where "#here is the problem" marks the problem which I cannot solve.
import json
dnew = {"Budapest": "fair", "New York": "very good", "Rome": "awesome"}
data = {
"type": "FeatureCollection",
"name": "Cities",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "City": "New York", "Rating": "good" }, "geometry": { "type": "Point", "coordinates": [ -73.991836734693834, 40.736734693877537 ] } },
{ "type": "Feature", "properties": { "City": "Rome", "Rating": "fair" }, "geometry": { "type": "Point", "coordinates": [ 12.494557823129199, 41.903401360544223 ] } },
{ "type": "Feature", "properties": { "City": "Budapest", "Rating": "awesome" }, "geometry": { "type": "Point", "coordinates": [ 19.091836734693832, 47.494557823129256 ] } }
]
}
#at this point, keys of two dictionaies are compared. If they are the same, the value of the old dict is updated/replaced by the value of the new dict
for key in data["features"]:
citykey = (key["properties"]["City"])
ratingvalue = (key["properties"]["Rating"])
#print(citykey + "| " + ratingvalue)
for keynew in dnew:
citynew = (keynew)
ratingnew = dnew[keynew]
#print(citynew + " | " + ratingnew)
print(citykey + "==" + citynew)
if citykey == citynew:
#
#here is the problem
#
data["features"]["properties"]["Rating"] = ratingnew
print(True)
else:
print(False)
Error Message:
TypeError: list indices must be integers or slices, not str
Thank you!
It misses a number index after "features" as it's a list not a dictionary.
data["features"][0]["properties"]["Rating"]
You're missing the benefits of dictionaries by looping over all the keys in the dnew dictionary for each element in the data['features'] list.
E.Coms noted the problem you're having but that only checks the first item in the list, (data["features"][0])
Perhaps the following will solve your problem.
for key in data["features"]:
citykey = (key["properties"]["City"])
ratingvalue = (key["properties"]["Rating"])
#print(citykey + "| " + ratingvalue)
if citykey in dnew:
key["properties"]["Rating"] = dnew[citykey]
Related
I am new in python and I am trying to list comprehsion my list dictionaries.
I have a serialized response in dictionaries inside list like :-
[
{
"data": {
"id": 61,
"title": "First"
},
"type": "like"
},
{
"data": {
"id": 62,
"title": "Seven"
},
"type": "like"
},
{
"data": {
"id": 103,
"title": "Third",
},
"type": "dislike"
},
{
"data": {
"id": 7,
"title": "Fifth",
},
"type": "dislike"
}
]
Multiple dictionaries with same type key are inside the list and I am trying to merge dictionaries into one list which have same keys.
I am trying to get like :-
[
{
"like": [
{
"id": 61,
"title": "First"},
{
"id": 62,
"title": "Second"
}
],
},
{
"dislike": [
{
"id": 103,
"title": "Third"
},
{
"id": 7,
"title": "Fifth"
}
],
},
]
I have tried using set() and union()
def comprehsion_method(list_dict):
converted_list = {
k : [d.get(k) for d in list_dict if k in d]
for k in set().union(*list_dict)
}
return converted_list
but This method merged all the data keys into one and all the type keys into one like :-
{
"data": [
{
"id":61,
"title": "First"
},
{
"id":62,
"title": "Second"
},
{
"id":103,
"title": "Third"
},
{
"id":7,
"title": "Seven"
},
],
"type": [
"like",
"like",
"dislike",
"dislike"
]
}
I have many times but it is still not working. Any help would be much Appreicated.
With comprehensions, especially complicated ones, it is best to start by writing out the explicit loop. In this case, you need something like:
new = {"like": [], "dislike": []}
for item in data:
if item["type"] == "like":
new["like"].append(item["data"])
else:
new["dislike"].append(item["data"])
print(new)
At this stage it is apparent that this function is trying to aggregate to a series of lists - something which comprehensions aren't designed to do. While it might be possible to convert it into a list comprehension, it will likely be relatively complex and less readable than the above code - so in this case I would leave it as is.
Hey I'm new to python and working on a uni assignment and need help. So I have a query of cities and some properties to go with them and i need to print out a list of the bordering cities and the cities more south and west of the city of Guelph. Apparently it can be done in 3 lines but I just want to figure out how to get my code to work for now.
ontario = {
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"name": "Guelph",
"border town?": "false"
},
"geometry": {
"type": "Point",
"coordinates": [-80.255126953125, 43.545561990655855]
}
}
]
}
citiesInOntario = ontario["features"]
for city in citiesInOntario:
if(city["properties"]["border town?"] == "true" and \
city["geometry"]["coordinates"][0] < -80.255 and \
city["geometry"]["coordinates"] < 43.545):
print(city["properties"]["name"])
As people have pointed out in the comments, your conditions for the coordinates don't match
city["geometry"]["coordinates"][0] < -80.255 and \
city["geometry"]["coordinates"] < 43.545):
In the first one you're getting the first coordinate value from the list city["geometry"]["coordinates"] and comparing it, but in the second condition you're comparing the entire list to a number. You just need to reference the second element of the list with city["geometry"]["coordinates"][1].
city["geometry"]["coordinates"][0] < -80.255 and \
city["geometry"]["coordinates"][1] < 43.545)
Also consider changing your "true" and "false" strings to Python's True and False:
ontario = {
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"name": "Guelph",
"borders_town": False
},
"geometry": {
"type": "Point",
"coordinates": [-80.255126953125, 43.545561990655855]
}
}
]
}
so that your new if condition is now:
if(city["properties"]["borders_town"] and ...
I just need contexts to be an Array ie., 'contexts' :[{}] instead of 'contexts':{}
Below is my python code which helps in converting python data-frame to required JSON format
This is the sample df for one row
name type aim context
xxx xxx specs 67646546 United States of America
data = {'entities':[]}
for key,grp in df.groupby('name'):
for idx, row in grp.iterrows():
temp_dict_alpha = {'name':key,'type':row['type'],'data' :{'contexts':{'attributes':{},'context':{'dcountry':row['dcountry']}}}}
attr_row = row[~row.index.isin(['name','type'])]
for idx2,row2 in attr_row.iteritems():
dict_temp = {}
dict_temp[idx2] = {'values':[]}
dict_temp[idx2]['values'].append({'value':row2,'source':'internal','locale':'en_Us'})
temp_dict_alpha['data']['contexts']['attributes'].update(dict_temp)
data['entities'].append(temp_dict_alpha)
print(json.dumps(data, indent = 4))
Desired output:
{
"entities": [{
"name": "XXX XXX",
"type": "specs",
"data": {
"contexts": [{
"attributes": {
"aim": {
"values": [{
"value": 67646546,
"source": "internal",
"locale": "en_Us"
}
]
}
},
"context": {
"country": "United States of America"
}
}
]
}
}
]
}
However I am getting below output
{
"entities": [{
"name": "XXX XXX",
"type": "specs",
"data": {
"contexts": {
"attributes": {
"aim": {
"values": [{
"value": 67646546,
"source": "internal",
"locale": "en_Us"
}
]
}
},
"context": {
"country": "United States of America"
}
}
}
}
]
}
Can any one please suggest ways for solving this problem using Python.
I think this does it:
import pandas as pd
import json
df = pd.DataFrame([['xxx xxx','specs','67646546','United States of America']],
columns = ['name', 'type', 'aim', 'context' ])
data = {'entities':[]}
for key,grp in df.groupby('name'):
for idx, row in grp.iterrows():
temp_dict_alpha = {'name':key,'type':row['type'],'data' :{'contexts':[{'attributes':{},'context':{'country':row['context']}}]}}
attr_row = row[~row.index.isin(['name','type'])]
for idx2,row2 in attr_row.iteritems():
if idx2 != 'aim':
continue
dict_temp = {}
dict_temp[idx2] = {'values':[]}
dict_temp[idx2]['values'].append({'value':row2,'source':'internal','locale':'en_Us'})
temp_dict_alpha['data']['contexts'][0]['attributes'].update(dict_temp)
data['entities'].append(temp_dict_alpha)
print(json.dumps(data, indent = 4))
Output:
{
"entities": [
{
"name": "xxx xxx",
"type": "specs",
"data": {
"contexts": [
{
"attributes": {
"aim": {
"values": [
{
"value": "67646546",
"source": "internal",
"locale": "en_Us"
}
]
}
},
"context": {
"country": "United States of America"
}
}
]
}
}
]
}
The problem is here in the following code
temp_dict_alpha = {'name':key,'type':row['type'],'data' :{'contexts':{'attributes':{},'context':{'dcountry':row['dcountry']}}}}
As you can see , you are already creating a contexts dict and assigning values to it. What you could do is something like this
contextObj = {'attributes':{},'context':{'dcountry':row['dcountry']}}
contextList = []
for idx, row in grp.iterrows():
temp_dict_alpha = {'name':key,'type':row['type'],'data' :{'contexts':{'attributes':{},'context':{'dcountry':row['dcountry']}}}}
attr_row = row[~row.index.isin(['name','type'])]
for idx2,row2 in attr_row.iteritems():
dict_temp = {}
dict_temp[idx2] = {'values':[]}
dict_temp[idx2]['values'].append({'value':row2,'source':'internal','locale':'en_Us'})
contextObj['attributes'].update(dict_temp)
contextList.append(contextObj)
Please Note - This code will have logical errors and might not run ( as it is difficult for me , to understand the logic behind it). But here is what you need to do .
You need to create a list of objects, which is not what you are doing. You are trying to manipulate an object and when its JSON dumped , you are getting an object back instead of a list. What you need is a list. You create context object for each and every iteration and keep on appending them to the local list contextList that we created earlier.
Once when the for loop terminates, you can update your original object by using the contextList and you will have a list of objects instead of and object which you are having now.
I have a big json/dictionary with different levels of nested json arrays, I would like to flatten it, and also capture the relationship of the structure,
Part of my json looks like:
{
"name": "root",
"type": "all",
"children": [
{
"name": "properties",
"type": "feature",
"children": [
{
"name": "print",
"type": "feature",
"children": [
{
"name": "graphic print",
"type": "feature",
"inherits": true
},
{
"name": "striped print",
"type": "feature",
"inherits": true,
"children": [
{
"name": "pinstriped",
"type": "feature",
"inherits": true
},
{
"name": "light stripe",
"type": "feature",
"inherits": true
},
{
"name": "wide stripe",
"type": "feature",
"inherits": true
}
]
}
]
}
]
},
{
"name": "colours",
"type": "colour",
"children": [
{
"name": "main colours",
"type": "colour",
"children": [
{
"name": "black",
"type": "colour",
"children": [
{
"name": "light black",
"type": "colour",
"inherits": true
},
{
"name": "blue black",
"type": "colour",
"inherits": true
}
]
},
{
"name": "red",
"type": "colour",
"children": [
{
"name": "bright red",
"type": "colour",
"inherits": true
},
{
"name": "light red",
"type": "colour"
}
]
}
]
}
]
},
{
"name": "genders",
"type": "gender",
"children": [
{
"name": "female",
"type": "gender"
},
{
"name": "male",
"type": "gender"
}
]
}
]
}
The depth of nests is not all the same. I
- want all the nodes (values of "name")
- also want all its parents if the node has "Inherit" key of True value.
Something like:
But if there are better ideas on how to store this data, will be happy to accept as well!
Many Thanks!
I think this should do your need
def parse_dict_of_dict(_dict, _parent = '', ret_dict={}):
_name, _children, _inherit = _dict["name"], _dict.get('children', None), _dict.get('inherits', False)
if _children is not None:
if isinstance(_children, list):
for _child in _children:
parse_dict_of_dict(_child, _name+ ', ' + _parent if _inherit else _name , ret_dict)
ret_dict[ _name] = _parent.strip(' ').strip(',') if _inherit else None
return ret_dict
Can you elaborate more on your output?
OR you can use this function to flatten a nested JSON to a simple JSON.
def parse_dict_of_dict(_dict, _str = ''):
ret_dict = {}
for k, v in _dict.iteritems():
if isinstance(v, dict):
ret_dict.update(parse_dict_of_dict(v, _str= _str+k+'_'))
elif isinstance(v, list):
for index, item in enumerate(v):
if isinstance(item, dict):
ret_dict.update(parse_dict_of_dict(item, _str=_str+k+'_%d_'%(index)))
else:
ret_dict.update({k+'_%d'%(index): item})
else:
try:
ret_dict[_str + k] = str(v)
except Exception as e:
ret_dict[_str + k] = unicode.encode(v, errors='ignore')
return ret_dict
i have a json with structure:
{
" features": [
{
"geometry": {
"type": "Polygon",
"coordinates": []
},
"type": "Feature",
"properties": {
"ADMIN_LEVE": "REGION",
"POPULTION": 4363916,
"GEO_CENTER": "7.923209152686669, 45.06052300898206",
"ID": "01",
"NAME": "PIEMONTE"
}
}
]
}
and i need to rename the field "ADMIN_LEVE".
I have write a code using pandas to rename but doesn't work
df = pd.DataFrame(data)
df.rename(columns={'ADMIN_LEVE':'ADMIN_LEVEL'}, inplace=True)
I can i do it?
i have also tryied with replace but doesn't work
json_data=open(path + ".json").read()
data = json.loads(json_data)
for d in data:
d.replace('"ADMIN_LEVE"', '"ADMIN_LEVEL"')
Thanks
Source JSON (as string):
In [325]: print(s)
{
" features": [
{
"geometry": {
"type": "Polygon",
"coordinates": []
},
"type": "Feature",
"properties": {
"ADMIN_LEVE": "REGION",
"POPULTION": 4363916,
"GEO_CENTER": "7.923209152686669, 45.06052300898206",
"ID": "01",
"NAME": "PIEMONTE"
}
}
]
}
Replaced:
In [327]: s = s.replace('"ADMIN_LEVE":', '"ADMIN_LEVEL":')
In [328]: print(s)
{
" features": [
{
"geometry": {
"type": "Polygon",
"coordinates": []
},
"type": "Feature",
"properties": {
"ADMIN_LEVEL": "REGION",
"POPULTION": 4363916,
"GEO_CENTER": "7.923209152686669, 45.06052300898206",
"ID": "01",
"NAME": "PIEMONTE"
}
}
]
}
UPDATE: helper function:
def replace_in_json_file(filename, from_str, to_str):
with open(filename) as f:
data = f.read()
with open(filename, 'w') as f:
f.write(data.replace(from_str, to_str))