I am trying to update a key while retaining its values within a nested dictionaries.
While I have found a method to do so, I had to create new dictionaries in order to cater for it. As such, wondering if there anyone could provide me with a better insight on the approach I have taken?
init_dict = {
'pageA' : {
0 : {
'menuA' : [
'a01',
'a02'
]
}
},
'pageB' : {
1 : {
'menuB' : [
'b10'
]
}
}
}
changed = {'pageB' : 0, 'pageA' : 1}
condense_dict = {}
for k, v in init_dict.items():
for i in v.keys():
condense_dict[k] = init_dict[k][i]
new_dict = {}
for i in condense_dict:
new_dict[i] = {}
new_dict[i][changed.get(i)] = condense_dict.get(i)
My expected output is as follows:
{
'pageA' : {
1 : {
'menuA' : [
'a01',
'a02'
]
}
},
'pageB' : {
0 : {
'menuB' : [
'b10'
]
}
}
}
You can pop the presumably only key from the sub-dict and assign it to the new key for each entry in changed:
for k, v in changed.items():
init_dict[k][v] = init_dict[k].pop(next(iter(init_dict[k])))
init_dict becomes:
{'pageA': {1: {'menuA': ['a01', 'a02']}}, 'pageB': {0: {'menuB': ['b10']}}}
Using the .pop() method this can be done similar to this (although I'm sure you could rewrite it better)
init_dict = {
'pageA': {
0: {
'menuA' : [
'a01',
'a02'
]
}
},
'pageB': {
1: {
'menuB': [
'b10'
]
}
}
}
print(init_dict)
thing = init_dict.pop('pageA')
sub_thing = thing.pop(0)
redone = {1: sub_thing}
init_dict.update({'pageA': redone})
print(init_dict)
{'pageA': {0: {'menuA': ['a01', 'a02']}}, 'pageB': {1: {'menuB': ['b10']}}}
{'pageA': {1: {'menuA': ['a01', 'a02']}}, 'pageB': {1: {'menuB': ['b10']}}}
You can see it's the same data as we start with, but we changed 0 to 1
Here I use .pop() and change it inplace. With the same init_dict as you:
change_to = {1: 0, 0: 1}
for k, v in init_dict.items():
for old_key in v.keys():
if old_key in change_to:
v[change_to[old_key]] = v.pop(old_key)
Related
Given a list of dictionaries:
data = {
"data": [
{
"categoryOptionCombo": {
"id": "A"
},
"dataElement": {
"id": "123"
}
},
{
"categoryOptionCombo": {
"id": "B"
},
"dataElement": {
"id": "123"
}
},
{
"categoryOptionCombo": {
"id": "C"
},
"dataElement": {
"id": "456"
}
}
]
}
I would like to display the dataElement where the count of distinct categoryOptionCombo is larger than 1.
e.g. the result of the function would be an iterable of IDs:
[123]
because the dataElement with id 123 has two different categoryOptionCombos.
tracker = {}
for d in data['data']:
data_element = d['dataElement']['id']
coc = d['categoryOptionCombo']['id']
if data_element not in tracker:
tracker[data_element] = set()
tracker[data_element].add(coc)
too_many = [key for key,value in tracker.items() if len(value) > 1]
How can I iterate the list of dictionaries preferably with a comprehension? This solution above is not pythonic.
One approach:
import collections
counts = collections.defaultdict(set)
for d in data["data"]:
counts[d["dataElement"]["id"]].add(d["categoryOptionCombo"]["id"])
res = [k for k, v in counts.items() if len(v) > 1]
print(res)
Output
['123']
This approach creates a dictionary mapping dataElements to the different types of categoryOptionCombo:
defaultdict(<class 'set'>, {'123': {'B', 'A'}, '456': {'C'}})
Almost a one-liner:
counts = collections.Counter( d['dataElement']['id'] for d in data['data'] )
print( counts )
Output:
Counter({'123': 2, '456': 1})
No need for sets, you can just remember each data element's first coc or mark it as having 'multiple'.
tracker = {}
for d in data['data']:
data_element = d['dataElement']['id']
coc = d['categoryOptionCombo']['id']
if tracker.setdefault(data_element, coc) != coc:
tracker[data_element] = 'multiple'
too_many = [key for key,value in tracker.items() if value == 'multiple']
(If the string 'multiple' can be a coc id, then use multiple = object() and compare with is).
I want to sort all lists within a deeply nested dictionary. It is basically a JSON object which deep nesting of dictionaries within lists and then lists within dictionaries. All I want to do is, parse through all dictionary keys to all leaf nodes and sort all the lists that i encounter on the way. Basically, any list directly available or deep down within that given dictionary object should get sorted and the same dictionary with all sorted lists should be returned.
I tried doing recursion on the dict object to pass any dict object encountered to the recursion method and sorting the lists when encountered. But they fail to produce results when there is a dict inside a list and then another list inside that dict object.
Sample JSON below:
my_json = {
a: {
b: {
c: [
{
d: [
{ f: 'some_string' }
]
},
{
e: {
g: [
h: 'another string'
]
}
}
]
}
}
z: [
b: {
c: [
{
d: [
{ f: 'some_string1' }
]
},
{
e: {
g: [
h: 'another string1'
]
}
}
]
},
x: {
c: [
{
d: [
{ f: 'some_string2' }
]
},
{
e: {
g: [
h: 'another string2'
]
}
}
]
}
]
}
def gen_dict_extract(input_dict):
result_obj = input_dict;
if hasattr(var, 'iteritems'):
for k, v in var.iteritems():
if isinstance(v, dict):
for result in gen_dict_extract(v):
yield result
elif isinstance(v, list):
v.sort();
for d in v:
for result in gen_dict_extract(d):
yield result
The output expectation is just to have all lists sorted irrespective of where they lie. I am even okay with sorting every item in the dictionary but list sorting is what I require.
Taking a smaller example here to explain the output:
old_json = {
'x': [
{
'z': {
'y': ['agsd', 'xef', 'sdsd', 'erer']
}
},
{
's': {
'f': 'ererer',
'd': [5, 6, 2, 3, 1]
}
}
]
}
new_json = {
'x': [
{
's': {
'f': 'ererer',
'd': [1, 2, 3, 5, 6]
}
},
{
'z': {
'y': ['agsd', 'erer', 'sdsd','xef']
}
}
]
}
Something like above.
If you want the output to be a different dictionary (i.e. not sorting the original), the function should be written like this:
def sortedDeep(d):
if isinstance(d,list):
return sorted( sortedDeep(v) for v in d )
if isinstance(d,dict):
return { k: sortedDeep(d[k]) for k in sorted(d)}
return d
This way you can use sortedDeep() the same way you would use the buil-in sorted() function:
new_json = sortedDeep(old_json)
[EDIT] Improved version that will also sort lists of dictionaries (or list of lists) based on the smallest key/value of the embedded object:
def sortedDeep(d):
def makeTuple(v): return (*v,) if isinstance(v,(list,dict)) else (v,)
if isinstance(d,list):
return sorted( map(sortedDeep,d) ,key=makeTuple )
if isinstance(d,dict):
return { k: sortedDeep(d[k]) for k in sorted(d)}
return d
I believe the code snippet here will do the job for sorting nested dictionaries.
def nested_sort(d:dict):
for v in d.values():
if isinstance(v,dict):
nested_sort(v)
elif isinstance(v,list):
v.sort()
However, I cannot test the code because the example you gave is not in legal JSON format or a legal python dictionary.
I have a nested dictionary in which it has the ordering that I had wanted.
But as soon as I tried to iterate the items within then, trying to increment a key, the ordering of the objects are already sorted in alphabetical mode in the first for loop, is throwing off the increment.
given = {
"testC": { "0": { "ccc": ["c100"] } },
"testA": { "1": { "aaa": ["a100", "a200"] } },
"testB": { "2": { "bbb": [] }}
}
index_order = 0
for k1, k2 in given.items(): # the ordering is change here, where testA will reordered
for page_index_order in given[k1].keys():
var = given[k1][page_index_order]
if page_index_order != index_order:
index_order += 1
else:
index_order = page_index_order
given[k1][index_order] = var
given[k1].pop(page_index_order)
pprint(given)
'''
{'testA': {1: {'aaa': ['a100', 'a200']}},
'testB': {3: {'bbb': []}},
'testC': {2: {'ccc': ['c100']}}}
'''
I was expecting my results to be (same ordering but note the index value where I expected testC to be 1 but it return me 2 instead):
{'testC': {1: {'ccc': ['c100']}},
'testA': {2: {'aaa': ['a100', 'a200']}},
'testB': {3: {'bbb': []}}}
I tried using OrderedDict(given) in the first for loop, but it returns a sorted dictionary.
given = {
"testC": { "0": { "ccc": ["c100"] } },
"testA": { "1": { "aaa": ["a100", "a200"] } },
"testB": { "2": { "bbb": [] }}
}
given_keys = ["testC", "testA", "testB"] # decide order
sorted_given = OrderedDict([(k, given[k]) for k in given_keys])
Your dict would be properly ordered if you either use Python 3.7+ or collections.OrderedDict, but if you use a regular dict in Python 3.7+ the problem would be that pprint always prints dicts with sorted keys, with no option not to. Use print instead to output the dict keys in their actual order:
given = {
"testC": { "0": { "ccc": ["c100"] } },
"testA": { "1": { "aaa": ["a100", "a200"] } },
"testB": { "2": { "bbb": [] }}
}
index_order = 0
for k1, k2 in given.items():
for page_index_order in list(given[k1]):
var = given[k1][page_index_order]
if page_index_order != index_order:
index_order += 1
else:
index_order = page_index_order
given[k1][index_order] = var
given[k1].pop(page_index_order)
print(given)
This outputs (in Python 3.7):
{'testC': {1: {'ccc': ['c100']}}, 'testA': {2: {'aaa': ['a100', 'a200']}}, 'testB': {3: {'bbb': []}}}
Or with collections.OrderedDict:
given = OrderedDict((
("testC", { "0": { "ccc": ["c100"] }}),
("testA", { "1": { "aaa": ["a100", "a200"] }} ),
("testB", { "2": { "bbb": [] }})
))
index_order = 0
for k1, k2 in given.items(): # the ordering is change here, where testA will reordered
for page_index_order in list(given[k1]):
var = given[k1][page_index_order]
if page_index_order != index_order:
index_order += 1
else:
index_order = page_index_order
given[k1][index_order] = var
given[k1].pop(page_index_order)
print(given)
This outputs:
OrderedDict([('testC', {1: {'ccc': ['c100']}}), ('testA', {2: {'aaa': ['a100', 'a200']}}), ('testB', {3: {'bbb': []}})])
This question already has answers here:
How to sum dict elements
(11 answers)
Closed 4 years ago.
I have the following list of dictionaries:
allChannelTraffic = [ { "Web" : 4, "android" : 3 }, { "Web" : 1 }, { "Web" : 1 }, { "Web" : 1 },]
I would like to know the easiest way to get an output like this from the above list:
[{'Web':7,'android':3}]
Where I want to get the sum of the values of all the keys. I heard that using list comprehension in python, we can easily do operations. Can someone please tell me how to implement this using list comprehension?
You can use Counter with sum:
from collections import Counter
allChannelTraffic = [{"Web": 4, "android": 3}, {"Web": 1}, {"Web": 1}, {"Web": 1}, ]
result = sum(map(Counter, allChannelTraffic), Counter())
print(result)
Output
Counter({'Web': 7, 'android': 3})
List comprehensions are not really useful here.
Generator expressions instead allows us to do the following:
allChannelTraffic = [ { "Web" : 4, "android" : 3 }, { "Web" : 1 }, { "Web" : 1 }, { "Web" : 1 },]
keys = set(k for d in allChannelTraffic for k in d.keys())
totals = {key: sum(d.get(key, 0) for d in allChannelTraffic) for key in keys}
print(totals)
# {'Web': 7, 'android': 3}
That last one {key: sum([...]) for key in keys} is a dictionary comprehension by the way.
And I could have used a set comprehension instead of the set() in line 2:
{k ... for k in d.keys()} == set(k ... for k in d.keys())
But I'd rather not do that, since the set() is much clearer for the reader.
In general though the Counter or Defaultdict approach for your problem might be more understandable for unexperienced pythonistas....
You can use collections.defaultdict to sum up the values for each key
import collections
totals = collections.defaultdict(int)
for sub in allChannelTraffic:
for key, value in sub.items():
totals[key] += value
Output
>>> totals
defaultdict(<class 'int'>, {'android': 3, 'Web': 7})
This isn't a list comprehension, but you can always use Counter.update() here:
from collections import Counter
allChannelTraffic = [ { "Web" : 4, "android" : 3 }, { "Web" : 1 }, { "Web" : 1 }, { "Web" : 1 }]
counts = Counter()
for d in allChannelTraffic:
counts.update(d)
print(counts)
# Counter({'Web': 7, 'android': 3})
A non-library approach would look something like this:
allChannelTraffic = [ { "Web" : 4, "android" : 3 }, { "Web" : 1 }, { "Web" : 1 }, { "Web" : 1 }]
counts = {}
for d in allChannelTraffic:
for k in d:
counts[k] = counts.get(k, 0) + d[k]
print(counts)
# Counter({'Web': 7, 'android': 3})
allChannelTraffic = [ { "Web" :4,"android" : 3 }, { "Web" : 1 }, { "Web" : 1 },{ "Web" : 1 },]
allChannelTraffic = [{"web": sum([item[1].get("Web",0) for item in enumerate(allChannelTraffic)]), "android":sum([item[1].get("android",0) for item in enumerate(allChannelTraffic)])}]
I have a dictionary like this
a = [{'CohortList': [{'DriverValue': 0.08559936}, {'DriverValue': 0.08184596527051588}],
'_id': {'DriverName': 'Yield', 'MonthsOnBooks': 50, 'SegmentName': 'LTV110-Super Prime'}},
{'CohortList': [{'DriverValue': 2406.04329}, {'DriverValue': 2336.0058100690103}, ],
'_id': {'DriverName': 'ADB', 'MonthsOnBooks': 15, 'SegmentName': 'LTV110-Super Prime'}},
{'CohortList': [{'DriverValue': 2406.04329}, {'DriverValue': 2336.0058100690103}, ],
'_id': {'DriverName': 'ADB', 'MonthsOnBooks': 16, 'SegmentName': 'LTV110-Prime'}}]
I want to construct a list of dictionary with values as lists from the above dict set like this
{
"LTV110-Prime": [
{
"ADB": [
{
"16": 1500
}
]
},
{
"Yield": []
}
],
"LTV110-Super Prime": [
{
"ADB": [
{
"15": 1500
}
]
},
{
"Yield": [
{
"50": 0.09
}
]
}
]
}
Essentially, I want to group ADB and Yield for each segments with their values.
This is what I have done so far to achieve this target. The values for ADB are mean of DriverValue from CohortList list. I have used statistics.mean to find out the mean of the mapped values.
sg_wrap = defaultdict(dict)
for p in pp_data:
mapped = map(lambda d: d.get('DriverValue', 0), p['CohortList'])
dic = {p['_id']['MonthsOnBooks']: statistics.mean(mapped)}
print(p)
print(sg_wrap)
I am not able to append the Drivers to the inner dict. Please help.
Since you are wrapping everything into lists, you do not need a defaultdict(dcit) but a defaultdict(list).
The following seems to work:
result = defaultdict(list)
for entry in a:
id_ = entry["_id"]
name, months, segment = id_["DriverName"], id_["MonthsOnBooks"], id_["SegmentName"]
values = [x["DriverValue"] for x in entry["CohortList"]]
d = {name: [{months: statistics.mean(values)}]}
result[segment].append(d)
Result is
{'LTV110-Prime': [{'ADB': [{16: 2371.0245500345054}]}],
'LTV110-Super Prime': [{'Yield': [{50: 0.08372266263525793}]},
{'ADB': [{15: 2371.0245500345054}]}]}