I have a list of JSON objects, already sorted (by time let's say). Each JSON object has type and status. For example:
[
{'type': 'A', 'status': 'ok'},
{'type': 'B', 'status': 'ok'},
{'type': 'A', 'status': 'fail'}
]
I'd like to convert it to:
{
'A': ['ok', 'fail'],
'B': ['ok']
}
So of course it's an easy task but I'm looking for the Pythonic way doing that, so it will spare me a few lines of code
I don't know if there is a one-liner but you can make use of setdefault or defaultdict to achieve the desired result as:
data = [
{'type': 'A', 'status': 'ok'},
{'type': 'B', 'status': 'ok'},
{'type': 'A', 'status': 'fail'}
]
Using setdefault():
res = {}
for elt in data:
res.setdefault(elt['type'], []).append(elt['status'])
Output:
{'A': ['ok', 'fail'], 'B': ['ok']}
Using defaultdict:
from collections import defaultdict
res = defaultdict(list)
for elt in data:
res[elt['type']].append(elt['status'])
Output:
defaultdict(<class 'list'>, {'A': ['ok', 'fail'], 'B': ['ok']})
Since your new structure depends on older (already processed) keys also, I doubt you can use dictionary comprehension (which operates on per-key basis) to achieve the required final state.
I would prefer to use defaultdict where each default key is a list.
from collections import defaultdict
new_data = defaultdict(list)
for item in data:
new_data[item['type']].append(item['status'])
print(new_data)
which gives output:
defaultdict(<class 'list'>, {'A': ['ok', 'fail'], 'B': ['ok']})
A one-liner with multiple list comprehensions and sorted(set(…)) to remove the duplicate keys:
original = [
{"type": "A", "status": "ok"},
{"type": "B", "status": "ok"},
{"type": "A", "status": "fail"},
]
result = {
key: [x["status"] for x in original if x["type"] == key]
for key in sorted(set([x["type"] for x in original]))
}
Output:
{'A': ['ok', 'fail'], 'B': ['ok']}
Related
I am working on xbrl document parsing. I got to a point where I have a large dic structured like this....
sample of a dictionary I'm working on
Since it's bit challenging to describe the pattern of what I'm trying to achieve I just put an example of what I'd like it to be...
sample of what I'm trying to achieve
Since I'm fairly new to programing, I'm hustling for days with this. Trying different approaches with loops, list and dic comprehension starting from here...
for k in storage_gaap:
if 'context_ref' in storage_gaap[k]:
for _k in storage_gaap[k]['context_ref']:
storage_gaap[k]['context_ref']={_k}```
storage_gaap being the master dictionary. Sorry for attaching pictures, but it's just much clearer to see the dictionary
I'd really appreciate any and ever help
Here's a solution using zip and dictionary comprehension to do what you're trying to do using toy data in a similar structure.
import itertools
import pprint
# Sample data similar to provided screenshots
data = {
'a': {
'id': 'a',
'vals': ['a1', 'a2', 'a3'],
'val_num': [1, 2, 3]
},
'b': {
'id': 'b',
'vals': ['b1', 'b2', 'b3'],
'val_num': [4, 5, 6]
}
}
# Takes a tuple of keys, and a list of tuples of values, and transforms them into a list of dicts
# i.e ('id', 'val'), [('a', 1), ('b', 2) => [{'id': 'a', 'val': 1}, {'id': 'b', 'val': 2}]
def get_list_of_dict(keys, list_of_tuples):
list_of_dict = [dict(zip(keys, values)) for values in list_of_tuples]
return list_of_dict
def process_dict(key, values):
# Transform the dict with lists of values into a list of dicts
list_of_dicts = get_list_of_dict(('id', 'val', 'val_num'), zip(itertools.repeat(key, len(values['vals'])), values['vals'], values['val_num']))
# Dictionary comprehension to group them based on the 'val' property of each dict
return {d['val']: {k:v for k,v in d.items() if k != 'val'} for d in list_of_dicts}
# Reorganize to put dict under a 'context_values' key
processed = {k: {'context_values': process_dict(k, v)} for k,v in data.items()}
# {'a': {'context_values': {'a1': {'id': 'a', 'val_num': 1},
# 'a2': {'id': 'a', 'val_num': 2},
# 'a3': {'id': 'a', 'val_num': 3}}},
# 'b': {'context_values': {'b1': {'id': 'b', 'val_num': 4},
# 'b2': {'id': 'b', 'val_num': 5},
# 'b3': {'id': 'b', 'val_num': 6}}}}
pprint.pprint(processed)
Ok, Here is the updated solution from my case. Catch for me was the was the zip function since it only iterates over the smallest list passed. Solution was the itertools.cycle method Here is the code:
data = {'us-gaap_WeightedAverageNumberOfDilutedSharesOutstanding': {'context_ref': ['D20210801-20220731',
'D20200801-20210731',
'D20190801-20200731',
'D20210801-20220731',
'D20200801-20210731',
'D20190801-20200731'],
'decimals': ['-5',
'-5',
'-5',
'-5',
'-5',
'-5'],
'id': ['us-gaap:WeightedAverageNumberOfDilutedSharesOutstanding'],
'master_id': ['us-gaap_WeightedAverageNumberOfDilutedSharesOutstanding'],
'unit_ref': ['shares',
'shares',
'shares',
'shares',
'shares',
'shares'],
'value': ['98500000',
'96400000',
'96900000',
'98500000',
'96400000',
'96900000']},
def get_list_of_dict(keys, list_of_tuples):
list_of_dict = [dict(zip(keys, values)) for values in list_of_tuples]
return list_of_dict
def process_dict(k, values):
list_of_dicts = get_list_of_dict(('context_ref', 'decimals', 'id','master_id','unit_ref','value'),
zip((values['context_ref']),values['decimals'],itertools.cycle(values['id']),
itertools.cycle(values['master_id']),values['unit_ref'], values['value']))
return {d['context_ref']: {k:v for k,v in d.items()if k != 'context_ref'} for d in list_of_dicts}
processed = {k: {'context_values': process_dict(k, v)} for k,v in data.items()}
pprint.pprint(processed)
I have created the following list of dictionaries:
[{'A': {'*96': 'Active'}},
{'A': {'*1(ABCD-EFG-SW01-P1)g': 'Active'}},
{'A': {'*65(Interoute-10G-to-AMS)gG': 'Active'}},
{'B': {'*9': 'Active'}},
{'B': {'*10': 'Disabled'}}]
And I would like to turn that into something like this:
{
'A': {
'*96': 'Active',
'*1(ABCD-EFG-SW01-P1)g': 'Active',
'*65(int-10F-to-ABC)gG': 'Active'
}
'B': {
'*9': 'Active',
'*10': 'Disabled'
}
}
I've tried a lot of things but somehow can't seem te figure it out.
Note that I am using Python3.
You can unpack the dictionaries as follows:
raw_dicts = [
{'A': {'*96': 'Active'}},
{'A': {'*1(ABCD-EFG-SW01-P1)g': 'Active'}},
{'A': {'*65(Interoute-10G-to-AMS)gG': 'Active'}},
{'B': {'*9': 'Active'}},
{'B': {'*10': 'Disabled'}}
]
dicts = {}
for raw_dict in raw_dicts:
for key, val in raw_dict.items():
if key in dicts:
dicts[key].update(val)
else:
dicts[key] = val
quite similar but a bit shorter:
d={}
for i in arr: # arr - your list
for k,v in i.items():
d.setdefault(k,dict()).update(v)
>>> d
'''
{'A': {'*96': 'Active',
'*1(ABCD-EFG-SW01-P1)g': 'Active',
'*65(Interoute-10G-to-AMS)gG': 'Active'},
'B': {'*9': 'Active',
'*10': 'Disabled'}}
I'm trying to convert a dictionary into a list, which contains many dictionaries. for example:
input:
{
'first-name':'foo',
'last-name':'bar',
'gender':'unknown',
'age':99
}
output:
[{'first-name': 'foo'}, {'last-name': 'bar'}, {'gender': 'unknown'}, {'age':99}]
I am already using this code to do such:
def convert(info: dict):
new_list = list
for item in info:
new_parameter = {item:info[item]}
new_list.append(new_parameter)
return new_list
but I was wondering if there is a built-in function to do this? or a more readable piece of code? it's almost 10 lines! and kind of confusing to understand!
Using list comprehension
items() method is used to return the list with all dictionary keys with values.
Ex.
dict1 = {
'first-name':'foo',
'last-name':'bar',
'gender':'unknown',
'age':99
}
new_list = [{k:v} for k,v in dict1.items() ]
print(new_list)
O/P:
[{'first-name': 'foo'}, {'last-name': 'bar'}, {'gender': 'unknown'}, {'age': 99}]
Another solution suggested by #josoler
new_list = list(map(dict, zip(dict1.items())))
Dict = {
'first-name':'foo',
'last-name':'bar',
'gender':'unknown',
'age':99
}
list = [(key, value) for key, value in Dict.items()]
list >> [('first-name':'foo'), ('last-name':'bar'), ('gender':'unknown'),('age':99)]
This is the simplest way to Convert dictionary to list of tuples
Reference : Python | Convert dictionary to list of tuples
>>> d={
... 'first-name':'foo',
... 'last-name':'bar',
... 'gender':'unknown',
... 'age':99
... }
>>> [{k: v} for (k, v) in d.items()]
[{'first-name': 'foo'}, {'last-name': 'bar'}, {'gender': 'unknown'}, {'age': 99}]
An alternative method:
>>> [dict([i]) for i in d.items()]
[{'gender': 'unknown'}, {'age': 99}, {'first-name': 'foo'}, {'last-name': 'bar'}]
k={
'first-name':'foo',
'last-name':'bar',
'gender':'unknown',
'age':99
}
s =list(map(lambda x:{x[0]:x[1]},k.items() ))
output
[{'first-name': 'foo'},
{'last-name': 'bar'},
{'gender': 'unknown'},
{'age': 99}]
This question already has answers here:
Merge two dicts by same key [duplicate]
(5 answers)
Closed 3 years ago.
How can I add multiple dictionaries in lists having name, value pairs such as:
dict1 = [{'name':'A','value':6}, {'name':'B', 'value':5}]
dict2 = [{'name':'A', 'value':10}, {'name':'C', 'value':12}]
Giving output as:
dict3 = [{'name':'A', 'value':16}, {'name':'B', 'value':5}, {'name':'C', 'value':12}]
Names in dictionary are not fixed and therefore random value in name field are possible.
Try this if you want to sum all values for unique 'name' in each dictionaries :
names = set([k['name'] for k in dict1+dict2])
dict3 = []
for name in names:
temp_val = []
for dict_ in dict1+dict2:
if dict_['name'] == name:
temp_val.append(dict_['value'])
dict3.append({'name': name, 'value' : sum(temp_val)})
OUTPUT :
[{'name': 'A', 'value': 16}, {'name': 'B', 'value': 5}, {'name': 'C', 'value': 12}]
Using collections.defaultdict:
res = defaultdict(int)
for d in dict1+dict2:
res[d['name']] += d['value']
[{'name': k, 'value':v} for k,v in res.items()]
Output:
[{'name': 'B', 'value': 5},
{'name': 'C', 'value': 12},
{'name': 'A', 'value': 16}]
What you have is a list of dictionaries with one key each, you can make that data structure much cleaner like so
dict1 = {'A' :6 , 'B' :5 }
dict2 = { 'A':10 , 'C' : 12 }
After that you can use Counter to simplify the addition
from collections import Counter
dict1 = {'A' :6 , 'B' :5 }
dict2 = { 'A':10 , 'C' : 12 }
#Add the two counters and get the result
dict3 = Counter(dict1) + Counter(dict2)
print(dict(dict3))
The output will be
{'A': 16, 'B': 5, 'C': 12}
I am trying to sort on multiple parameters of list of dictionaries and lists. I checked How do I sort a list of dictionaries by values of the dictionary in Python? .
I have a complex dictionary:
dict1 =
{"outer_list" : [
#1st dict
{ "id" : 1,
"name" : "xyz",
"nested_list" : [{"id" : "5","key":"val"},{"id" : "4","key":"val"}]
},
#2nd dict
{
"outer_id" : 11,
"name" : "abc",
"nested_list" : [{"id" : "12","key":"val"},{"id" : "8","key" : "val"}]
}
] # outer_list ends
} #dict1 ends
I want to sort on the key name and nested_list[id] and expected output :
[{'outer_id': 11, 'name': 'abc', 'nested_list': [{'id': '8', 'key': 'val'}, {'id': '12', 'key': 'val'}]}, {'nested_list': [{'id': 4, 'key': 'val'}, {'id': 5, 'key': 'val'}], 'id': 1, 'name': 'xyz'}]
My attempt :
def sort_cluster(data):
for items in data:
item=items['outer_list']
newlist = sorted(item, key=itemgetter('name'))
print newlist
if __name__ == "__main__":
list1=[]
list1.append(dict1)
sort_cluster(list1)
It sorts on name properly, next if i follow the same procedure for sorting on "newlist" for nested_list[id] , it's not working.
Sort all the nested_lists, then sort the top-level list. You can do these in either order. For the keys, I prefer to use lambdas rather than operator.itemgetter here since we need to convert the result of one of the things to an int (I assume, based on your expected output), and you'd have to wrap the operator.itemgetter in a lambda anyway to do that.
def do_thing(dct):
lst = dct["outer_list"]
# Sort the nested_lists, assuming you want to sort by the numeric value of the "id" value
for obj in lst:
obj["nested_list"].sort(key=lambda d: int(d["id"]))
# Sort the outer_list
lst.sort(key=lambda d: d["name"])
return lst
Then:
>>> do_thing(dict1)
[{'name': 'abc', 'outer_id': 11, 'nested_list': [{'key': 'val', 'id': '8'}, {'key': 'val', 'id': '12'}]},
{'name': 'xyz', 'nested_list': [{'key': 'val', 'id': '4'}, {'key': 'val', 'id': '5'}], 'id': 1}]