I have 2 nested dictionaries variable that have the similar keys, each defining different values :
data1 = {
"2010":{
'A':2,
'B':3,
'C':5
},
"2011":{
'A':1,
'B':2,
'C':3
},
"2012":{
'A':1,
'B':2,
'C':4
}
}
data2 = {
"2010":{
'A':4,
'B':4,
'C':5
},
"2011":{
'A':1,
'B':1,
'C':3
},
"2012":{
'A':3,
'B':2,
'C':4
}
}
In my case, i need to sum both dictionaries values based on the same keys, so the answer will be like this:
data3 = {
"2010":{
'A':6,
'B':7,
'C':10
},
"2011":{
'A':2,
'B':3,
'C':6
},
"2012":{
'A':4,
'B':4,
'C':8
}
}
How can i do that?
Given the structure of the two dictionaries is the same, you can use dictionary comprehension for that:
data3 = {key:{key2:val1+data2[key][key2] for key2,val1 in subdic.items()} for key,subdic in data1.items()}
In the repl:
>>> {key:{key2:val1+data2[key][key2] for key2,val1 in subdic.items()} for key,subdic in data1.items()}
{'2010': {'B': 7, 'C': 10, 'A': 6}, '2012': {'B': 4, 'C': 8, 'A': 4}, '2011': {'B': 3, 'C': 6, 'A': 2}}
The comprehension works as follows: in the outerloop, we iterate over the key,subdic of data1. So in your case, key is a year and subdic is the dictionary (of data1) for that year.
Now for each of these years, we iterate over the items of the subdic and here key2 is 'A', 'B' and 'C'. val1 is the value that we find in data1 for these keys. We get the other value by querying data2[key][key2]. We sum these up and construct new dictionaries for that.
I hope this helps:
data1 = { "2010":{ 'A':2, 'B':3, 'C':5 }, "2011":{ 'A':1, 'B':2, 'C':3 }, "2012":{ 'A':1, 'B':2, 'C':4 } }
data2 = { "2010":{ 'A':4, 'B':4, 'C':5 }, "2011":{ 'A':1, 'B':1, 'C':3 }, "2012":{ 'A':3, 'B':2, 'C':4 } }
data3 = {}
for data in [data1,data2]:
for year in data.keys():
for x,y in data[year].items():
if not year in data3.keys():
data3[year] = {x:y}
else:
if not x in data3[year].keys():
data3[year].update({x:y})
else:
data3[year].update({x:data3[year][x] + y})
print data3
This works for arbitrary lengths of the inner and outer dictionaries.
Another solution :)
You can also use zip to get both data1 and data2 in the same for loop, and then use collections.Counter to add the value of each dicts.
from collections import Counter
>> {k1: Counter(v1) + Counter(v2) for (k1, v1), (k2, v2) in zip(sorted(data1.items()), sorted(data2.items()))}
{'2011': Counter({'C': 6, 'B': 3, 'A': 2}), '2010': Counter({'C': 10, 'B': 7, 'A': 6}), '2012': Counter({'C': 8, 'A': 4, 'B': 4})}
You will ended with Counter dict but since it is a subclass of dict you can still use the same method as a regular dict.
If you add dict() to Max Chrétiens' nice short solution from above, you will end up with regular dictionaries:
data3 = {k1: dict(Counter(v1) + Counter(v2)) for (k1, v1), (k2, v2) in
zip(data1.items(), data2.items())}
This will, however, only work correctly if both dictionaries share exactly the same keys as already discussed above. Willem Van Onsem's solution will not work if there are any keys not shared by both dictionaries either (it will result in an error, whereas Max Chrétiens' solution will in this case merge items incorrectly). Now you mentioned you are using JSON data which always contains the same structure with similar keys, so this should not constitute a problem and Max Chrétien's solution should work nicely.
In case you do want to make sure only keys shared by both dictionaries (and their subdictionaries) are used, the following will work. Notice how I added 'X': 111111 as a key value pair to the 2012 subdictionary and "1999": { 'Z': 999999 } as an entire subdictionary.
def sum_two_nested_dicts(d1, d2):
dicts = [d1, d2]
d_sum = {}
for topkey in dicts[0]:
if topkey in dicts[1]:
d_sum[topkey] = {}
for key in dicts[0][topkey]:
if key in dicts[1][topkey]:
new_val = sum([d[topkey][key] for d in dicts])
d_sum[topkey][key] = new_val
return d_sum
data1 = {
"2010": {
'A': 2,
'B': 3,
'C': 5
},
"2011": {
'A': 1,
'B': 2,
'C': 3
},
"2012": {
'A': 1,
'B': 2,
'C': 4,
'X': 111111
},
"1999": {
'Z': 999999
}
}
data2 = {
"2010": {
'A': 4,
'B': 4,
'C': 5
},
"2011": {
'A': 1,
'B': 1,
'C': 3
},
"2012": {
'A': 3,
'B': 2,
'C': 4
}
}
data3 = sum_two_nested_dicts(data1, data2)
print(data3)
# different order of arguments
data4 = sum_two_nested_dicts(data2, data1)
print(data4)
# {'2010': {'C': 10, 'A': 6, 'B': 7}, '2012': {'C': 8, 'A': 4, 'B': 4}, '2011': {'C': 6, 'A': 2, 'B': 3}}
# {'2010': {'C': 10, 'A': 6, 'B': 7}, '2012': {'C': 8, 'A': 4, 'B': 4}, '2011': {'C': 6, 'A': 2, 'B': 3}}
I realize this is far from as concise and elegant as can be, but as I already wrote it anyways, I post it here in case someone is trying to achieve this particular functionality.
Long and bloated version which retains unshared keys/values, just because I already wrote it...
def sum_nested_dicts(dic1, dic2):
# create list of both dictionaries
dicts = [dic1, dic2]
# create a set of all unique keys from both dictionaries
topkeys = set(sum([list(dic.keys()) for dic in dicts], []))
# this is the merged dictionary to be returned
d_sum = {}
for topkey in topkeys:
# if topkey is shared by both dictionaries
if topkey in dic1 and topkey in dic2:
d_sum[topkey] = {}
keys = set(sum([list(dic[topkey].keys()) for dic in
dicts], []))
for key in keys:
# if key is shared by both subdictionaries
if key in dic1[topkey] and key in dic2[topkey]:
new_val = sum([d[topkey][key] for d in dicts])
d_sum[topkey][key] = new_val
# if key is only contained in one subdictionary
elif key in dic1[topkey]:
d_sum[topkey][key] = dic1[topkey][key]
elif key in dic2[topkey]:
d_sum[topkey][key] = dic2[topkey][key]
# if topkey is only contained in one dictionary
elif topkey in dic1:
d_sum[topkey] = dic1[topkey]
elif topkey in dic2:
d_sum[topkey] = dic2[topkey]
return d_sum
See Crystal's solution for what seems to be the most concise and functional solution posted thus far.
Related
This question already has answers here:
Sum values of similar keys inside two nested dictionary in python
(2 answers)
Closed 1 year ago.
I have a list of dictionary:
data = [
{"2010" : {'A' : 2,'B' : 3,'C' : 5,}},
{"2011" : {'A' : 1,'B' : 2,}},
{"2010" : {'A' : 1,'B' : 2,}}
]
I'd like sum the values where the key is same. So the result I expected should be like this:
res =
{"2010" : {'A' : 3, 'B' : 5, 'C' : 5},
"2011" : {'A' : 1, 'B' : 2}}
How can I do this easily?
So you have a list of dictionaries as input and you want to create an output dictionary which contains the sums:
from collections import defaultdict
data = [{'2010': {'A': 2, 'B': 3, 'C': 5}}, {'2011': {'A': 1, 'B': 2}}, {'2010': {'A': 1, 'B': 2}}]
def sum_values(data):
out = {}
for i in data:
for k in i.keys():
if k not in out:
out[k] = defaultdict(int)
for k1,v1 in i[k].items():
out[k][k1] += v1
return out
sum_values(data)
{'2010': defaultdict(<class 'int'>, {'A': 3, 'B': 5, 'C': 5}), '2011': defaultdict(<class 'int'>, {'A': 1, 'B': 2})}
Try this
data = [
{ "2010":{'A':2,'B':3,'C':5,}},
{ "2011":{'A':1,'B':2,}},
{"2010":{'A':1,'B':2,}}
]
res = {}
for d in data:
# print(d.items())
items = list(d.items())[0] # convert to list because dict items isn't subscriptable
year, value = items
if year not in res:
res[year] = value
res is equal to
{
'2010': {'A': 2, 'B': 3, 'C': 5},
'2011': {'A': 1, 'B': 2}
}
Say I have 2 dict with same structure (this entire dict is the structure for any one of the dicts):
{
'0': {
'A': a,
'B': b,
},
'1': {
'C': c,
'D': d,
}
}
#Sample input and output:
#dict1
{
'0': {
'A': 0,
'B': 1,
},
'1': {
'C': 2,
'D': 3,
}
}
#dict2
{
'0': {
'A': 5,
'B': 5,
},
'1': {
'C': 5,
'D': 5,
},
'3': { 'E': 5 } #this will be ignored when combining
}
#merged output with addition:
{
'0': {
'A': 5,
'B': 6,
},
'1': {
'C': 7,
'D': 8,
}
}
Ideally, everything about both dicts are the same except for the values a,b,c,d. For any subsequent dicts that have parts in their structure that are different from the first dict, those parts are ignored when merging. Like how the 3 AND E key in dict2 was ignored in the merge.
How can I combine both dicts into one dict that maintains the same structure, but with merged values? I would like to make this generic, so this 'merge' operation could be addition, subtraction, etc. And the number of dicts to merge can also change (not just 2 at a time).
Thanks, I hope to learn more about Python from your solutions
is you want that type of code
marged={}
for i in a:
for j in a[i]:
marged.setdefault(j,a[i].get(j))
print(marged)
Output:
{'A': 'a', 'B': 'b', 'K': 'k', 'L': 'l', 'C': 'c', 'D': 'd'}
Input:
a={
'0': {
'A': "a",
'B': "b",
'K':"k",
'L':"l"
},
'1': {
'C': "c",
'D': "d"
}
}
You can simply use UPDATE method.
if a is a dictionary and you want to add another dictionary say b.
then simply use
a.update(b)
This is to combine all the dicts, i.e, to merge.
d = {
'0': {
'A': a,
'B': b,
},
'1': {
'C': c,
'D': d,
},
}
def merge(d):
merged_dict = {}
for k in d:
for key in d[k]:
if not merged_dict.get(key, False):
merged_dict[key] = d[k][key]
return merged_dict
md = merge(d)
This should just merge the different dicts into one.
For supporting arbitrary dict structure and nesting, you will have to use a recursive function. Pass the operator as a parameter and use e.g. reduce to calculate the value for all the passed dicts (one or more). This assumes only dicts of dicts, but could also handle e.g. lists with another block if isinstance(first, list) and a corresponding list comprehension.
import functools, operator
def merge(op, first, *more):
if isinstance(first, dict):
return {key: merge(op, first[key], *(d[key] for d in more if key in d))
for key in first}for key in first}
else:
return functools.reduce(op, more, first)
A = {'0': {'A': 1, 'B': 2}, '1': {'C': 3, 'D': 4}}
B = {'0': {'A': 5, 'B': 6}, '1': {'C': 7, 'D': 8}}
print(merge(operator.add, A, B))
# {'0': {'A': 6, 'B': 8}, '1': {'C': 10, 'D': 12}}
Also works with more than two dicts, different operator, or if the more dicts have more or fewer keys than first:
C = {'0': {'A': 9, 'B': 10}, '1': {'C': 11, 'D': 12}}
D = {'0': {'A': 13, 'X': 14}, '2': {'C': 15, 'D': 16}}
print(merge(lambda x, y: f"{x}-{y}", A, B, C, D))
# {'0': {'A': '1-5-9-13', 'B': '2-6-10'}, '1': {'C': '3-7-11', 'D': '4-8-12'}}
It still assumes that values corresponding to the same keys in first and more have the same type, though.
I am stuck on convert jsons from regular (key-value) to nested.
for example:
j = { 'a': 5,
'b': 3,
'c.c1': 2,
'c.c2':5,
'd.dd.d1': 0,
'd.dd.d2':9
}
and I need to get:
new_j = {
'a': 5,
'b': 3,
'c':
{'c1': 2,
'c2':5}
'd':
{'dd' :
{'d1': 0,
'd2': 9}
},
}
there are easy method to do this?
How about this?
j_new = {}
for key,value in j.items():
keys = key.split('.')
level = j_new
for key in keys[:-1]:
if key not in level:
level[key]={}
level = level[key]
level[keys[-1]]=value
print(j_new)
Which returns:
{'a': 5, 'b': 3, 'c': {'c1': 2, 'c2': 5}, 'd': {'dd': {'d1': 0, 'd2': 9}}}
Try this:
d = {}
def setValue(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
for k,v in j.items():
setValue(d, k.split('.'), v)
Output:
{
"a": 5,
"c": {
"c2": 5,
"c1": 2
},
"b": 3,
"d": {
"dd": {
"d2": 9,
"d1": 0
}
}
}
I have multiples list of dictionaries like this:
list_of_dictionaries_1 = [{
'a':1,
'b':2
}, {
'a':3,
'b':4
}]
list_of_dictionaries_2 = [{
'c':1,
'd':2
}, {
'c':3,
'd':4
}]
And I want to add each element into a new dictionary.
new_dictionary = {
data: [{
'a':1,
'b':2
}, {
'a':3,
'b':4
}, {
'c':1,
'd':2
}, {
'c':3,
'd':4
}]
}
So I made this for each list of dictionaries:
for dictionary_ in list_of_dictionaries_1:
new_dictionary['data'] = dictionary_
But this just return the last element in the list of dictionaries.
new_dictionary = {
data:[{
'c':3,
'd':4
}]
}
How can I add all de dictionaries in the new dictionary?
If I understood correctly, you could do it like this:
new_dictionary = {'data': []}
for elem in list_of_dictionaries_1 + list_of_dictionaries_2:
new_dictionary['data'].append(elem)
print(new_dictionary)
Output:
{'data': [{'a': 1, 'b': 2}, {'a': 3, 'b': 4}, {'c': 1, 'd': 2}, {'c': 3, 'd': 4}]}
You can use itertools.chain to merge the two lists:
from itertools import chain
new_dictionary = {'data': list(chain(list_of_dictionaries_1, list_of_dictionaries_2))}
new_dictionary becomes:
{'data': [{'a': 1, 'b': 2}, {'a': 3, 'b': 4}, {'c': 1, 'd': 2}, {'c': 3, 'd': 4}]}
Your dictionary structure looks inconsistent. But however, you can do the below to achieve what you are trying for.
list_of_dictionaries_1 = [{'a':1, 'b':2 }, {'a':3, 'b':4}]
list_of_dictionaries_2 = [{'c':1, 'd':2 }, {'c':3, 'd':4 }]
list_of_dictionaries_1.extend(list_of_dictionaries_2)
print(list_of_dictionaries_1)
Output:
[{'a': 1, 'b': 2}, {'a': 3, 'b': 4}, {'c': 1, 'd': 2}, {'c': 3, 'd': 4}]
input = { "a" : { "x": 1, "y": 2 },
"b" : { "x": 3, "z": 4 } }
output = {'y': {'a': 2},
'x': {'a': 1, 'b': 3},
'z': {'b': 4} }
I need to make a new dictionary given the input and it should come out as the output. So far I have written:
def pivot_nested_dict(nested_dict):
5 stateDict={}
6 for i in nested_dict:
7 dict2=nested_dict[i]
8 for j in dict2:
9 stateDict[j]= {i: dict2[j]}
10 return stateDict
11
12 print pivot_nested_dict(input)
I almost have the answer, output={'y': {'a': 2}, 'x': {'b': 3}, 'z': {'b': 4}}.
So, the x key is overwriting the 'a': 1. How do I keep both the a and the b?
What you need to this is check if the key is already in stateDict, if it is, then add a new element to the nested dictionary:
def pivot_nested_dict(nested_dict):
stateDict = {}
for i in nested_dict:
dict2 = nested_dict[i]
for j in dict2:
if j not in stateDict:
stateDict[j] = {i: dict2[j]}
else:
stateDict[j][i] = dict2[j]
return stateDict
print pivot_nested_dict(inp)
Short of duplicating the dictionary, there is no way to retain the original value on a dictionary once you change the value that corresponds to the given key.
For example, the below overwriting will always happen no matter how complex the data structure.
x = {'a':1}
x['a'] = 2
print x
>>> {'a': 2}
You can duplicate it, but the original value will need to be managed manually.
x = {'a':1}
x,oldx = x,x.copy()
oldx['a'] = 2
print x, oldx
>>> {'a': 2} {'a': 1}
You could take advantage of the fact that values can be any type of structure including arrays or dictionaries, to store each successive change of value.
I ve got this little script yo get your job done.
NOTE: Provided the input is correct.
code:
output = {}
for d in input.items():
for item in d:
if type(item) == dict:
for i in item.keys():
d = {k:item[i]}
output[i] = {k:item[i]} if not i in output.keys() else dict(output[i].items() + d.items())
else: k=item
input:
input = { "a" : { "x": 1, "y": 2 ,'z':15},"b" : { "x": 3, "z": 4 } }
output:
{'y': {'a': 2}, 'x': {'a': 1, 'b': 3}, 'z': {'a': 15, 'b': 4}}
Hope this helps :)
I have this one
input = { "a" : { "x": 1, "y": 2 },
"b" : { "x": 3, "z": 4 } }
output = {}
for k,v in zip(input.keys(), input.values()):
for kk, vv in zip(v.keys(), v.values()):
if kk in output.keys():
output[kk][k] = vv
else:
output[kk] = {k : vv}