I have a python script which grabs data from a site in json:
channels_json = json.loads(url)
the site returns data as follows:
[ { '1': 'http://ht.co/bbda24210d7bgfbbbbcdfc2a023f' },
{ '2': 'http://ht.co/bbd10d7937932965369c248f7ccdfc2a023f' },
{ '3': 'http://ht.co/d3a01f6e5e74eb2cb5840556d80a52adf2871d' },
{ '4': 'http://ht.co/56d3a01f6e5e72cb5840556d80a52adf2871d' },
{ '5': 'http://ht.co/9ed0bb4cc447b99c9ce609916ccf931f16a' },
{ '6': 'http://ht.co/9ed0bb4cc44bb99c9ce609916ccf931f16a' },
....]
The problem is that Python is making this into a list rather than a dictionary. So I can't reference '4' like this:
print (channels_json["4"])
and get the response:
http://ht.co/56d3a01f6e5e72cb5840556d80a52adf2871d
Instead Python spits out:
TypeError: list indices must be integers, not str
If I run this code:
for c in channels_json:
print c
Python prints out each set of coupled data like this:
{u'1': u'http://ht.co/bbda24210d7bgfbbbbcdfc2a023f' },
{ u'2': u'http://ht.co/bbd10d7937932965369c248f7ccdfc2a023f' },
{ u'3': u'http://ht.co/d3a01f6e5e74eb2cb5840556d80a52adf2871d' },
{ u'4': u'http://ht.co/56d3a01f6e5e72cb5840556d80a52adf2871d' },
{ u'5': u'http://ht.co/9ed0bb4cc447b99c9ce609916ccf931f16a' },
{ u'6': u'http://ht.co/9ed0bb4cc44bb99c9ce609916ccf931f16a' },
How can I get the above into a dictionary so I can reference value '6' as a string and get back
http://ht.co/9ed0bb4cc44bb99c9ce609916ccf931f16a
You can create the dictionary you want by iterating over the dictionaries in the list returned from json.load() as shown below:
#json_data = json.loads(url)
json_data = [{ '1': 'http://ht.co/bbda24210d7bgfbbbbcdfc2a023f' },
{ '2': 'http://ht.co/bbd10d7937932965369c248f7ccdfc2a023f' },
{ '3': 'http://ht.co/d3a01f6e5e74eb2cb5840556d80a52adf2871d' },
{ '4': 'http://ht.co/56d3a01f6e5e72cb5840556d80a52adf2871d' },
{ '5': 'http://ht.co/9ed0bb4cc447b99c9ce609916ccf931f16a' },
{ '6': 'http://ht.co/9ed0bb4cc44bb99c9ce609916ccf931f16a' },]
# Convert to single dictionary.
channels_json = dict(d.popitem() for d in json_data)
print(json.dumps(channels_json, indent=4)) # Pretty-print result.
Output:
{
"1": "http://ht.co/bbda24210d7bgfbbbbcdfc2a023f",
"2": "http://ht.co/bbd10d7937932965369c248f7ccdfc2a023f",
"3": "http://ht.co/d3a01f6e5e74eb2cb5840556d80a52adf2871d",
"4": "http://ht.co/56d3a01f6e5e72cb5840556d80a52adf2871d",
"5": "http://ht.co/9ed0bb4cc447b99c9ce609916ccf931f16a",
"6": "http://ht.co/9ed0bb4cc44bb99c9ce609916ccf931f16a"
}
dd = {}
for d in c:
for key, value in d.items():
dd[key] = value
You can just iterate through the array and build your dictionary
channels_json = {}
channels_array = json.loads(url)
for d in channels_array:
key = list(d.keys())[0]
val = d[key]
channels_json[key] = val
Now you should be able to reference your dictionary channels_json
More pythonic way of doing this.
channels = {}
for i in json.loads(url): channels.update(i)
or
channels = {}
[channels.update(i) for i in json.loads(url)]
In both cases the dictionary gets updated with your list of separate dictionaries.
Related
I was trying to to get all the types inside a colours->numbers but it doesn't work because the colours e.g. green,blue inside an integer so i could not go through a loop to get them.
Here is the code that I was trying to do:
x=0
for colour in colours['rootdata']:
print(colour[x][type])
x+1
but is shows 'string indices must be integers'
I'm able to get a single value with for loop like this :(but this not what i want)
colour_red = JsonResult['rootdata']['colours']['0']['type']
print (colour_red )
This is the simple json sample that I'm using
{
"rootdata": {
"colours": {
"0": {
"type": "red"
},
"1": {
"type": "green"
},
"2": {
"type": "blue"
}
}
}
}
Try this:
my_dict = {"rootdata":
{
"colours": {"0": {"type": "red"}, "1": {"type": "green"}, "2": {"type": "blue"}}
}
}
types = []
types_dict = {}
for k, v in my_dict["rootdata"]["colours"].items():
types.append(v["type"])
types_dict[k] = v["type"]
print(types)
# R e s u l t : ['red', 'green', 'blue']
print(types_dict)
# R e s u l t : {'0': 'red', '1': 'green', '2': 'blue'}
Regards...
You do not need to use a counter. You can obtain the colours by using the following code:
for colour in colours['rootdata']['colours'].values():
print(colour['type'])
This code is works even if the numbers are not in sequential order in the json, and the order of the output does not really matter.
Is there any way to pull the key from JSON if the only thing I know is the value? (In groovy or python)
An example:
I know the "_number" value and I need a key.
So let's say, known _number is 2 and as an output, I should get dsf34f43f34f34f
{
"id": "8e37ecadf4908f79d58080e6ddbc",
"project": "some_project",
"branch": "master",
"current_revision": "3rtgfgdfg2fdsf",
"revisions": {
"43g5g534534rf34f43f": {
"_number": 3,
"created": "2019-04-16 09:03:07.459000000",
"uploader": {
"_account_id": 4
},
"description": "Rebase"
},
"dsf34f43f34f34f": {
"_number": 2,
"created": "2019-04-02 10:54:14.682000000",
"uploader": {
"_account_id": 2
},
"description": "Rebase"
}
}
}
With Groovy:
def json = new groovy.json.JsonSlurper().parse("x.json" as File)
println(json.revisions.findResult{ it.value._number==2 ? it.key : null })
// => dsf34f43f34f34f
Python 3: (assuming that data is saved in data.json):
import json
with open('data.json') as f:
json_data = json.load(f)
for rev, revdata in json_data['revisions'].items():
if revdata['_number'] == 2:
print(rev)
Prints all revs where _number equals 2.
using dict-comprehension:
print({k for k,v in d['revisions'].items() if v.get('_number') == 2})
OUTPUT:
{'dsf34f43f34f34f'}
I'm comparing json files between two different API endpoints to see which json records need an update, which need a create and what needs a delete. So, by comparing the two json files, I want to end up with three json files, one for each operation.
The json at both endpoints is structured like this (but they use different keys for same sets of values; different problem):
{
"records": [{
"id": "id-value-here",
"c": {
"d": "eee"
},
"f": {
"l": "last",
"f": "first"
},
"g": ["100", "89", "9831", "09112", "800"]
}, {
…
}]
}
So the json is represented as a list of dictionaries (with further nested lists and dictionaries).
If a given json endpoint (j1) id value ("id":) exists in the other endpoint json (j2), then that record should be added to j_update.
So far I have something like this, but I can see that .values() doesn't work because it's trying to operate on the list instead of on all the listed dictionaries(?):
j_update = {r for r in j1['records'] if r['id'] in
j2.values()}
This doesn't return an error, but it creates an empty set using test json files.
Seems like this should be simple, but tripping over the nesting I think of dictionaries in a list representing the json. Do I need to flatten j2, or is there a simpler dictionary method python has to achieve this?
====edit j1 and j2====
have same structure, use different keys; toy data
j1
{
"records": [{
"field_5": 2329309841,
"field_12": {
"email": "cmix#etest.com"
},
"field_20": {
"last": "Mixalona",
"first": "Clara"
},
"field_28": ["9002329309999", "9002329309112"],
"field_44": ["1002329309832"]
}, {
"field_5": 2329309831,
"field_12": {
"email": "mherbitz345#test.com"
},
"field_20": {
"last": "Herbitz",
"first": "Michael"
},
"field_28": ["9002329309831", "9002329309112", "8002329309999"],
"field_44": ["1002329309832"]
}, {
"field_5": 2329309855,
"field_12": {
"email": "nkatamaran#test.com"
},
"field_20": {
"first": "Noriss",
"last": "Katamaran"
},
"field_28": ["9002329309111", "8002329309112"],
"field_44": ["1002329309877"]
}]
}
j2
{
"records": [{
"id": 2329309831,
"email": {
"email": "mherbitz345#test.com"
},
"name_primary": {
"last": "Herbitz",
"first": "Michael"
},
"assign": ["8003329309831", "8007329309789"],
"hr_id": ["1002329309877"]
}, {
"id": 2329309884,
"email": {
"email": "yinleeshu#test.com"
},
"name_primary": {
"last": "Lee Shu",
"first": "Yin"
},
"assign": ["8002329309111", "9003329309831", "9002329309111", "8002329309999", "8002329309112"],
"hr_id": ["1002329309832"]
}, {
"id": 23293098338,
"email": {
"email": "amlouis#test.com"
},
"name_primary": {
"last": "Maxwell Louis",
"first": "Albert"
},
"assign": ["8002329309111", "8007329309789", "9003329309831", "8002329309999", "8002329309112"],
"hr_id": ["1002329309877"]
}]
}
If you read the json it will output a dict. You are looking for a particular key in the list of the values.
if 'records' in j2:
r = j2['records'][0].get('id', []) # defaults if id does not exist
It it prettier to do a recursive search but i dunno how you data is organized to quickly come up with a solution.
To give an idea for recursive search consider this example
def resursiveSearch(dictionary, target):
if target in dictionary:
return dictionary[target]
for key, value in dictionary.items():
if isinstance(value, dict):
target = resursiveSearch(value, target)
if target:
return target
a = {'test' : 'b', 'test1' : dict(x = dict(z = 3), y = 2)}
print(resursiveSearch(a, 'z'))
You tried:
j_update = {r for r in j1['records'] if r['id'] in j2.values()}
Aside from the r['id'/'field_5] problem, you have:
>>> list(j2.values())
[[{'id': 2329309831, ...}, ...]]
The id are buried inside a list and a dict, thus the test r['id'] in j2.values() always return False.
The basic solution is fairly simple.
First, create a set of j2 ids:
>>> present_in_j2 = set(record["id"] for record in j2["records"])
Then, rebuild the json structure of j1 but without the j1 field_5 that are not present in j2:
>>> {"records":[record for record in j1["records"] if record["field_5"] in present_in_j2]}
{'records': [{'field_5': 2329309831, 'field_12': {'email': 'mherbitz345#test.com'}, 'field_20': {'last': 'Herbitz', 'first': 'Michael'}, 'field_28': ['9002329309831', '9002329309112', '8002329309999'], 'field_44': ['1002329309832']}]}
It works, but it's not totally satisfying because of the weird keys of j1. Let's try to convert j1 to a more friendly format:
def map_keys(json_value, conversion_table):
"""Map the keys of a json value
This is a recursive DFS"""
def map_keys_aux(json_value):
"""Capture the conversion table"""
if isinstance(json_value, list):
return [map_keys_aux(v) for v in json_value]
elif isinstance(json_value, dict):
return {conversion_table.get(k, k):map_keys_aux(v) for k,v in json_value.items()}
else:
return json_value
return map_keys_aux(json_value)
The function focuses on dictionary keys: conversion_table.get(k, k) is conversion_table[k] if the key is present in the conversion table, or the key itself otherwise.
>>> j1toj2 = {"field_5":"id", "field_12":"email", "field_20":"name_primary", "field_28":"assign", "field_44":"hr_id"}
>>> mapped_j1 = map_keys(j1, j1toj2)
Now, the code is cleaner and the output may be more useful for a PUT:
>>> d1 = {record["id"]:record for record in mapped_j1["records"]}
>>> present_in_j2 = set(record["id"] for record in j2["records"])
>>> {"records":[record for record in mapped_j1["records"] if record["id"] in present_in_j2]}
{'records': [{'id': 2329309831, 'email': {'email': 'mherbitz345#test.com'}, 'name_primary': {'last': 'Herbitz', 'first': 'Michael'}, 'assign': ['9002329309831', '9002329309112', '8002329309999'], 'hr_id': ['1002329309832']}]}
I have the following list of dicts in python.
[
{
"US": {
"Intial0": 12.515
},
{
"GE": {
"Intial0": 11.861
}
},
{
"US": {
"Final0": 81.159
}
},
{
"GE": {
"Final0": 12.9835
}
}
]
I want the final list of dicts as
[{"US": {"Initial0":12.515, "Final0": 81.159}}, {"GE": {"Initial0": 11.861, "Final0": 12.9835}}]
I am struggling with this from quite some time . Any help?
You could use Python's defaultdict as follows:
from collections import defaultdict
lod = [
{"US": {"Intial0": 12.515}},
{"GE": {"Intial0": 11.861}},
{"US": {"Final0": 81.159}},
{"GE": {"Final0": 12.9835}}]
output = defaultdict(dict)
for d in lod:
output[d.keys()[0]].update(d.values()[0])
print output
For the data given, this would display the following:
defaultdict(<type 'dict'>, {'GE': {'Intial0': 11.861, 'Final0': 12.9835}, 'US': {'Intial0': 12.515, 'Final0': 81.159}})
Or you could convert it back to a standard Python dictionary with print dict(output) giving:
{'GE': {'Intial0': 11.861, 'Final0': 12.9835}, 'US': {'Intial0': 12.515, 'Final0': 81.159}}
list1=[{"US": {"Intial0": 12.515}},{"GE": {"Intial0": 11.861}},{"US": {"Final0": 81.159}},{"GE": {"Final0": 12.9835}}]
dict_US={}
dict_GE={}
for dict_x in list1:
if dict_x.keys()==['US']:
dict_US.update(dict_x["US"])
if dict_x.keys()==['GE']:
dict_GE.update(dict_x["GE"])
list2=[{"US":dict_US},{"GE":dict_GE}]
print list2
From my current query:
response_data = {}
response_data["medium"] = list(BuildingStructure.objects.filter(geom__intersects = getgeom_medium).values('brgy_locat').annotate(countmedium = Count('brgy_locat')))
response_data["high"] = list(BuildingStructure.objects.filter(geom__intersects = getgeom).values('brgy_locat').annotate(counthigh = Count('brgy_locat')))
response_data["low"] = list(BuildingStructure.objects.filter(geom__intersects = getgeom_low).values('brgy_locat').annotate(countlow = Count('brgy_locat')))
result = {}
for category in response_data.values():
for element in category:
key = element.pop('brgy_locat')
if key not in result: result[key] = {
"loc": key
}
result[key].update(element)
return HttpResponse(json.dumps(result), content_type = 'application/json')
The JSON output is this:
{
"Mabini": {
"counthigh": 3,
"loc": "Mabini",
"countmedium": 2,
"countlow": 25
},
"Barangay 12": {
"loc": "Barangay 12",
"countlow": 29
},
"Katugasan": {
"loc": "Katugasan",
"countlow": 3
}
}
But I wanted to have this output instead:
{
"0": {
"counthigh": 3,
"loc": "Mabini",
"countmedium": 2,
"countlow": 25
},
"1": {
"loc": "Barangay 12",
"countlow": 29
},
"2": {
"loc": "Katugasan",
"countlow": 3
}
}
I'm new to Django and Python, I tried looping and used a variable to be the index, but the output changes and I can't figure it out. Any help is appreciated.
Add another simple transformation of the result:
json_result = {str(i): v for i, v in enumerate(result.values())}
return HttpResponse(json.dumps(json_result), content_type='application/json')
UPDATE: The order of the keys in the dict is not guaranteed. But if you want to get the exact output of the JSON then you can use the collections.OrderedDict of the python 2.7:
from collections import OrderedDict
json_result = OrderedDict((str(i), v) for i, v in enumerate(result.values()))
Your output is changing because you are using dictionary to store the keys. Dictionaries don't keep the order of their elements (they are unordered), its better for you to use python lists instead when you want keep the order of the elements.
E.g.
results = []
#...
results.append(...)