I have two dictionaries, and what I'm trying to do is a bit odd. Basically, I want to merge them. That's simple enough. But they're hierarchies of of dictionaries, and I want to merge them in such a way that if an item in a dictionary is itself a dictionary and exists in both, I want to merge those dictionaries as well. If it's not a dictionary, I want the values from the second dictionary to overwrite the values from the first one. Something sort of like this:
a = {0: {0: "a"},
1: [0, 1, 2]}
b = {0: {1: "b"},
1: [3, 4, 5]}
Merge(a, b)
#output:
{0: {0: "a",
1: "b"},
1: [3, 4, 5]}
Does that make sense? Because the key "0" contained a dictionary in both a and b, it merged those dictionaries as well. But in the case of the second key, it was a list so it just overwrote it.
So I suppose I'll be looking at some kind of recursive function? Not quite sure how to approach this one.
Thanks!
Edit: I forgot to mention one pretty crucial detail:
I need a function that works in both 2.6.2 and 2.7.3.
Assuming you might have nested dictionaries (based on your thinking in terms of recursion), something like this should work,
from copy import deepcopy
def merge(a, b):
if isinstance(b, dict) and isinstance(a, dict):
a_and_b = a.viewkeys() & b.viewkeys()
every_key = a.viewkeys() | b.viewkeys()
return {k: merge(a[k], b[k]) if k in a_and_b else
deepcopy(a[k] if k in a else b[k]) for k in every_key}
return deepcopy(b)
The return value of merge(a, b) is conceptually like creating a (deep) copy of a and running a recursive version of a.update(b).
Using some nested examples,
a = {0: {0: 'a'},
1: [0, 1, 2],
2: [9, 9],
3: {'a': {1: 1, 2: 2}, 'b': [0, 1]}}
b = {0: {1: 'b'},
1: [3, 4, 5],
2: {22: 22, 33: 33},
3: {'a': {2: 22, 3: 33}, 'b': [99, 88]}}
merge(a, b) produces,
{0: {0: 'a', 1: 'b'},
1: [3, 4, 5],
2: {22: 22, 33: 33},
3: {'a': {1: 1, 2: 22, 3: 33}, 'b': [99, 88]}}
EDIT: Python 2.6 version
def merge(a, b):
if isinstance(b, dict) and isinstance(a, dict):
a_and_b = set(a).intersection(b)
every_key = set(a).union(b)
return dict((k, merge(a[k], b[k]) if k in a_and_b else
deepcopy(a[k] if k in a else b[k])) for k in every_key)
return deepcopy(b)
Hmm.. as long as you're not arbitrarily-nested, you don't need recursion.
from itertools import chain
{k:(v if not isinstance(v,dict) else dict(chain(a[k].items(), v.items()))) for k,v in b.items()}
Out[10]: {0: {0: 'a', 1: 'b'}, 1: [3, 4, 5]}
(I'm using python 3 here, feel free to replace .items with .iteritems in python 2)
Since this is a bit verbose, there's always the sneaky way to merge two dicts:
{k:(v if not isinstance(v,dict) else dict(a[k], **v)) for k,v in b.items()}
Out[11]: {0: {0: 'a', 1: 'b'}, 1: [3, 4, 5]}
You may or may not want to use this syntax - though it is compact, it kind of abuses cPython implementation details.
Needed something similar and implemented a more straightforward recursive solution. In-place updates dict 'd'.
from Collections import MutableMapping
def merge(d, v):
"""
Merge two dictionaries.
Merge dict-like `v` into dict-like `d`. In case keys between them are the same, merge
their sub-dictionaries where possible. Otherwise, values in `v` overwrite `d`.
"""
for key in v:
if key in d and isinstance(d[key], MutableMapping) and isinstance(v[key], MutableMapping):
d[key] = merge(d[key], v[key])
else:
d[key] = v[key]
return d
Example 1:
a = {0: {0: "a"},
1: [0, 1, 2]}
b = {0: {1: "b"},
1: [3, 4, 5]}
>>> merge(a, b)
{0: {0: 'a', 1: 'b'}, 1: [3, 4, 5]}
Example 2:
a = {0: {0: 'a'},
1: [0, 1, 2],
2: [9, 9],
3: {'a': {1: 1, 2: 2}, 'b': [0, 1]}}
b = {0: {1: 'b'},
1: [3, 4, 5],
2: {22: 22, 33: 33},
3: {'a': {2: 22, 3: 33}, 'b': [99, 88]}}
>>> merge(a, b)
{0: {0: 'a', 1: 'b'},
1: [3, 4, 5],
2: {22: 22, 33: 33},
3: {'a': {1: 1, 2: 22, 3: 33}, 'b': [99, 88]}}
Related
This question already has answers here:
How do I initialize a dictionary of empty lists in Python?
(7 answers)
Closed last month.
here's my code:
import copy
teamdict = {1:{},2:{},3:{},4:{},5:{}}
teamlst = []
matchlst = [1,2,3,4,5]
lst = [[1,5,0],[1,0,3],[2,3,0],[2,0,1],[3,0,6],[3,0,1],[4,0,1],[4,0,5],[5,0,8]]
for i in matchlst:
for j in lst:
if i == j[0]:
teamlst.append(j[2])
elif len(teamlst)>0 and i != j:
continue
else:
pass
teamlst = set(teamlst)
teamlst = list(teamlst)
teamlst.sort()
team_dictionary = dict.fromkeys(teamlst, [])
teamdict[i] = team_dictionary
teamlst.clear()
print(teamdict)
dic = copy.deepcopy(teamdict)
for i in lst:
dic[i[0]][i[2]].append(i[1])
print(dic)
this is what i got:
{1: {0: [5, 0], 3: [5, 0]}, 2: {0: [3, 0], 1: [3, 0]}, 3: {1: [0, 0], 6: [0, 0]}, 4: {1: [0, 0], 5: [0, 0]}, 5: {8: [0]}}
but what i expect is:
{1: {0: [5], 3: [0]}, 2: {0: [3], 1: [0]}, 3: {1: [0], 6: [0]}, 4: {1: [0], 5: [0]}, 5: {8: [0]}}
what goes wrong?
seems like this append method add the element to all the keys in the nested dictionary
I assume you have created your dictionary with multiple references to the same list.
If I execute your code I get what you want.
>>> lst = [[1,5,0],[1,0,3],[2,3,0],[2,0,1],[3,0,6],[3,0,1],[4,0,1],[4,0,5],[5,0,8]]
>>> dic = {1: {0: [], 3: []}, 2: {0: [], 1: []}, 3: {1: [], 6: []}, 4: {1: [], 5: []}, 5: {8: []}}
>>> for i in lst:
... dic[i[0]][i[2]].append(i[1])
...
>>> print(dic)
{1: {0: [5], 3: [0]}, 2: {0: [3], 1: [0]}, 3: {1: [0], 6: [0]}, 4: {1: [0], 5: [0]}, 5: {8: [0]}}
I have a large list, which I separated into small sized lists which have elements of occurrences of 1s and 0s, randomly.
Also, the first two lists are made with different parameters from the last two.
Example:
list_of_lists[0] =[1,0,1,1,1,0,1,1,1,0]
list_of_lists[1] =[0,0,0,0,0,0,0,0,0,0]
list_of_lists[2] =[1,1,1,1,1,1,1,1,1,1]
list_of_lists[3] =[0,0,1,1,1,1,1,1,1,0]
I would like to count the occurrences of 1s and 0s in each list, and append them into a dictionary to plot the occurrences.
My trial is as follows:
counts_each = dict()
for i in range(4): #all 4 lists
for k in list_of_lists[i]: #elements of the lists
counts_each[k] = counts_each.get(k, 0) + 1
print(counts_each)
which calculates the general occurrences of the 1s and 0s for the al lists:
{0: 16, 1: 24}
If I do:
list_counts = []
for i in range(4):
counts_each = dict()
for k in list_of_lists[i]:
counts_each[k] = counts_each.get(k, 0) + 1
list_counts.append(counts_each)
print(list_counts)
It does not accumulate all of the counts:
[{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{0: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{1: 10},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7},
{0: 3, 1: 7}]
I would be glad to have some insights of what I am doing wrong.
Thank you.
You can let the collections module do all the counting work for you.
from collections import Counter
list_of_lists = [[] for _ in range(4)]
list_of_lists[0] =[1,0,1,1,1,0,1,1,1,0]
list_of_lists[1] =[0,0,0,0,0,0,0,0,0,0]
list_of_lists[2] =[1,1,1,1,1,1,1,1,1,1]
list_of_lists[3] =[0,0,1,1,1,1,1,1,1,0]
counters = [Counter(l) for l in list_of_lists]
print(*counters, sep="\n")
OUTPUT
Counter({1: 7, 0: 3})
Counter({0: 10})
Counter({1: 10})
Counter({1: 7, 0: 3})
You could use a Dict Comprehension, given your nested list:
list_of_lists = [[1,0,1,1,1,0,1,1,1,0], [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1], [0,0,1,1,1,1,1,1,1,0]]
use it in this way:
{ idx: {0: lst.count(0), 1: lst.count(1)} for idx, lst in enumerate(list_of_lists) }
#=> {0: {0: 3, 1: 7}, 1: {0: 10, 1: 0}, 2: {0: 0, 1: 10}, 3: {0: 3, 1: 7}}
In the above case I used the index as a key, but you could just use a list comprehension to get a list of dictionaries:
[ {0: lst.count(0), 1: lst.count(1)} for lst in list_of_lists ]
#=> [{0: 3, 1: 7}, {0: 10, 1: 0}, {0: 0, 1: 10}, {0: 3, 1: 7}]
Chris Doyle's answer is excellent, but perhaps your goal is to understand the problem with your solution, specifically.
You have not included your expected output. If I am correct that your issue with your current solution is the repetition of the counts, and you want an output like this:
[{1: 7, 0: 3}, {0: 10}, {1: 10}, {0: 3, 1: 7}]
Then the issue appears to be with the indenting of the line list_counts.append(counts_each). You are doing this each time through the k loop (looping through the items in the list) when I think you want to do it only after finishing the count for a given list:
list_counts = []
for i in range(4):
counts_each = dict()
for k in list_of_lists[i]:
counts_each[k] = counts_each.get(k, 0) + 1
list_counts.append(counts_each)
print(list_counts)
I have 2 dicts like:
x = {2: {'a':1, 'b':1}}
y = {3: {'a':1}, 2: {'a':2, 'c':2}}
The merging result I want should be:
z = {3: {'a':1}, 2: {'a':3, 'b':1, 'c':2}}
I tried dict(x.items() + y.items()) and Counter, but only get
{2: {'a': 2, 'c': 2}, 3: {'a': 1}}
as a result.
How can I merge dict whose value is also a dict itself?
I have explained that the difference between How to merge two Python dictionaries in a single expression?
is that my dict's value is also a dict.
What about this:
For Python2.x:
from collections import Counter
y = {2: {'a': 1, 'b': 1}, 1: {'a': 1, 'b': 1}}
x = {3: {'a': 1}, 2: {'a': 2, 'c': 2}}
t=x.copy()
print(dict({k: (dict(Counter(t.pop(k, None)) + Counter(v))) for k, v in y.items()},**t))
Result:
{1: {'a': 1, 'b': 1}, 2: {'a': 3, 'c': 2, 'b': 1}, 3: {'a': 1}}
For Python3.5+:
{**{k: (dict(Counter(t.pop(k, None)) + Counter(v))) for k, v in y.items()},**t}
Here you have a possible solution. Although it doesn't use any library, it is quite complex.
Also it will work with any two dictionaries, independently on which is the largest.
{key: {ikey: x.get(key, {}).get(ikey, 0) + y.get(key, {}).get(ikey, 0)
for ikey in x.get(key, {}).keys()+y.get(key, {}).keys()}
for key in x.keys()+y.keys()}
Output:
{2: {'a': 3, 'b': 1, 'c': 2}, 3: {'a': 1}}
Suppose for any same depth for a certain key, in two dicts the values are of same type (both dict or both number, i don't know how to define merging a number into a dict).
def merge(d1, d2):
for i in d2:
if i not in d1:
continue
if isinstance(d1[i], dict) and isinstance(d2[i], dict):
merge(d1[i], d2[i])
else:
d2[i] += d1[i]
d1.pop(i)
for j in d1:
d2[j] = d1[j]
It's straightforward but maybe not elegant.
Use collections.Counter to "merge" the items you want to merge (only the common keys), and leave the others intact, using a double ternary expression in a dict comprehension, iterating on the union of the keys:
import collections
x = {2: {'a':1, 'b':1}}
y = {3: {'a':1}, 2: {'a':2, 'c':2}}
# pre-compute common keys, one-liners are not the ultimate goal in life!
common_keys = set(x) & set(y)
# now generate the merged dict
result = {k:collections.Counter(x[k])+collections.Counter(y[k]) if k in common_keys else x[k] if k in x else y[k] for k in set(x)|set(y)}
print(result)
result:
{2: Counter({'a': 3, 'c': 2, 'b': 1}), 3: {'a': 1}}
Note: we could have avoided computing the common keys beforehand by replacing:
if k in common_keys
by
if k in x and k in y
This works:
x = {2: {'a':1, 'b':1}}
y = {3: {'a':1}, 2: {'a':2, 'c':2}}
def key_merge (dict1, dict2):
## function which checks for any matching keys, and merges them
if len(dict1.keys()) == 0 or len(dict2.keys()) == 0:
return {}
else:
for key in dict1.keys():
if key in dict2.keys():
return {key:{ k: dict1[key].get(k, 0) + dict2[key].get(k, 0) for k in set(dict1[key])|set(dict2[key]) }}
z = {**x, **y, **key_merge(x, y)}
Again, up to you if it's elegant!
I don't know what your definition of 'elegantly' is but assuming you mean readable then perhaps this way will suit you.
from collections import Counter
def counter_dict(in_dict):
"""
Create a dict of Counters from a dict of dicts.
"""
return dict((k, Counter(v)) for (k, v) in in_dict.items())
def merge_counter_dicts(a, b):
"""
Create a dict of Counters from two dicts of Counters.
Where keys exist in both counts are summed.
"""
out_dict = a.copy()
for k in b.keys():
out_dict[k] = out_dict.setdefault(k, Counter()) + b[k]
return out_dict
x = {2: {'a': 1, 'b': 1}}
y = {3: {'a': 1}, 2: {'a': 2, 'c': 2}}
xc = counter_dict(x)
xy = counter_dict(y)
print merge_counter_dicts(xc, xy)
# What about something like this:
# merge_nested_dicts.py
# tested under Python3.6
# assuming dict_02 overwrites dict_01
# one liner functional style
def deep_merge(dict_01, dict_02):
return {k: {**dict_01.get(k), **dict_02.get(k)} if k in dict_01 and
isinstance(dict_01.get(k), dict) and
isinstance(dict_02.get(k), dict) else v
for k, v in {**dict_01, **dict_02}.items()}
if __name__ == '__main__':
y = {2: {'a': 1, 'b': 1}, 1: {'a': 1, 'b': 1}}
x = {3: {'a': 1}, 2: {'a': 2, 'c': 2}}
print(x)
print(y)
print(deep_merge(x, y))
'''
{3: {'a': 1}, 2: {'a': 2, 'c': 2}}
{2: {'a': 1, 'b': 1}, 1: {'a': 1, 'b': 1}}
{3: {'a': 1}, 2: {'a': 1, 'c': 2, 'b': 1}, 1: {'a': 1, 'b': 1}}
'''
q = "cats dogs"
d = {'cats': {1: 1, 2: 3, 3: 1, 4: 1}, 'nuts': {3: 1}, 'egg': {5: 1, 6: 2}, 'dogs': {1: 8, 2: 2, 3: 4}, 'idea': {4: 1}, 'frog': {2: 1, 4: 1, 5: 1}}
newList = []
for word in q:
for x,y in d.values():
newList.append(d[word].x())
What I'm attempting to do here is get each first part of the number pairs and put into the list when it corresponds to a word in q. So I'm expecting newList to be [1,2,3,4,1,2,3].
I am getting ValueError: too many values to unpack (expected 2). How should I be doing this?
The values in the dictionary 'd' are actually of type dictionary themselves, not tuples. I think you may want to reorganize this a bit.
for word in q.split(" "):
if word in d:
child_dict = d[word]
for key in child_dict:
newList.append(key)
Is this what you're looking for?
q = "cats dogs"
d = {'cats': {1: 1, 2: 3, 3: 1, 4: 1}, 'nuts': {3: 1}, 'egg': {5: 1, 6: 2}, 'dogs': {1: 8, 2: 2, 3: 4}, 'idea': {4: 1}, 'frog': {2: 1, 4: 1, 5: 1}}
newList = []
for word in q.split(' '):
newList.extend(list(d[word].keys()))
print(newList)
It prints out [1, 2, 3, 4, 1, 2, 3]
q = "cats dogs"
d = {'cats': {1: 1, 2: 3, 3: 1, 4: 1}, 'nuts': {3: 1}, 'egg': {5: 1, 6: 2}, 'dogs': {1: 8, 2: 2, 3: 4}, 'idea': {4: 1}, 'frog': {2: 1, 4: 1, 5: 1}}
newList = []
for word in q.split(' '):
if word in d:
newList.extend([i for i in d[word]])
print(newList)
I have a python list which contains dictionaries and I want to make a new list which contain dictionaries with unique keys and associated list values like below:
Input:
[{1: 2}, {2: 2}, {1: 3}, {2: 1}, {1: 3}]
Output:
[{1:[2,3,3]},{2:[2,1]}]
Thanks in advance.
How about:
input = [{1: 2}, {2: 2}, {1: 3}, {2: 1}, {1: 3}]
r = {}
for d in input:
# (assumes just one key/value per dict)
((x, y),) = d.items()
r.setdefault(x, []).append(y)
print [ {k: v} for (k, v) in r.items() ]
Result:
[{1: [2, 3, 3]}, {2: [2, 1]}]
[update]
just curious : Can you explain whats going on in ((x, y),) = d.items() and r.setdefault(x, []).append(y) ? – damned
First the ((x, y),) = d.items():
at this point, d will be an element from input, like {1: 2}
d.items() will be something analogous to [(1, 2)]
in order to unpack 1 and 2 into x and y, we need the extra , (otherwise the interpreter will think the outer parenthesis are doing grouping instead of defining a single element tuple)
The r.setdefault(x, []).append(y) is analogous to:
if not r.has_key(x):
r[x] = []
r[x].append(y)
Trick is to use dict.setdefault to start off a list and append to it:
input = [{1: 2}, {2: 2}, {1: 3}, {2: 1}, {1: 3}]
output = {}
for d in input:
for k,v in d.items():
output.setdefault(k, []).append(v)
# output contains {1: [2, 3, 3], 2: [2, 1]}
output=[{k:v} for k,v in output.items()]
# output contains [{1: [2, 3, 3]}, {2: [2, 1]}]
What setdefault does is return either the existing list keyed by 'k', or if that key does not exist in the dictionary, it creates a new entry for that key with the second argument and returns that. Either way it returns the list whether it was pre-existing or new, so that you can then append to it.
>>> lis=[{1: 2}, {2: 2}, {1: 3}, {2: 1}, {1: 3}]
>>> new_lis=[{}]
>>> for x in lis:
for y in x:
if y in new_lis[0]:
new_lis[0][y].append(x[y])
new_lis[0][y].sort()
else :new_lis[0][y]=[x[y]]
>>> new_lis
[{1: [2, 3, 3], 2: [1, 2]}]
>>> data = [{1: 2}, {2: 2}, {1: 3}, {2: 1}, {1: 3}]
...
... from itertools import groupby
...
... keyFn = lambda x: x.keys()[0]
...
... [{k: [i.values()[0] for i in g]} for k, g in groupby(sorted(data, key=keyFn), keyFn)]
0: [{1: [2, 3, 3]}, {2: [2, 1]}]
output = []
for b in a:
added = 0
i = 0
for c in output:
if b.keys()[0] in c.keys():
output[i][b.keys()[0]].append(b[b.keys()[0]])
added = 1
i += 1
if not added:
output.append({b.keys()[0]: [b[b.keys()[0]]]})