I have a dictionary of dictionaries:
d = {"a": {"x":1, "y":2, "z":3}, "b": {"x":2, "y":3, "z":4}, "c": {"x":3, "y":4, "z":5}}
And I want to convert it to:
new_d = {"x":[1, 2, 3], "y": [2, 3, 4], "z": [3, 4, 5]}
The requirement is that new_d[key][i] and new_d[another_key][i] should be in the same sub-dictionary of d.
So I created new_d = {} and then:
for key in d.values()[0].keys():
new_d[key] = [d.values()[i][key] for i in range(len(d.values()))]
This gives me what I expected, but I am just wondering if there are some built-in functions for this operation or there are better ways to do it.
There is no built-in function for this operation, no. I'd just loop over values directly:
new_d = {}
for sub in d.itervalues(): # Python 3: use d.values()
for key, value in sub.iteritems(): # Python 3: use d.items()
new_d.setdefault(key, []).append(value)
This avoids creating a new list for the dict.values() call each time.
Note that dictionaries have no order. The values in the resulting lists are going to fit your criteria however; they'll be added in the same order for each of the keys in new_d:
>>> d = {"a": {"x":1, "y":2, "z":3}, "b": {"x":2, "y":3, "z":4}, "c": {"x":3, "y":4, "z":5}}
>>> new_d = {}
>>> for sub in d.values():
... for key, value in sub.items():
... new_d.setdefault(key, []).append(value)
...
>>> new_d
{'x': [1, 2, 3], 'y': [2, 3, 4], 'z': [3, 4, 5]}
List Comprehension Method
If you like dictionary and list comprehensions ...
d1 = {"a": {"x": 1, "y": 2, "z": 3},
"b": {"x": 2, "y": 3, "z": 4},
"c": {"x": 3, "y": 4, "z": 5}}
dl1 = {kl: [v for di in d1.values() for k, v in di.items() if k == kl]
for di in d1.values() for kl in di.keys()}
print(dl1)
And yields the results hoped for ...
{'x': [1, 2, 3], 'y': [2, 3, 4], 'z': [3, 4, 5]}
Related
how to merge two dictonaries in python with union of values if the keys do exist in both. Each dictionary has values as list.
I have three dictionaries:
d1 = {"KEY1": [1, 2, 3]}
d2 = {"KEY1": [2, 3, 4]}
d3 = {"KEY2": [1, 2, 3]}
how could I merge then so if:
merge(d1,d2) --> {"KEY1": [1, 2, 3, 4]}
merge(d1,d3) --> {"KEY1": [1, 2, 3],"KEY2": [1, 2, 3]}
For such a specific kind of dictionary merging, you have to create a custom merge function. Something like this is probably a good start:
def merge(d1, d2):
merged = {}
for k in d1:
if k in d2:
merged[k] = sorted(list(set(d1[k] + d2[k])))
else:
merged[k] = d1[k]
for k in d2:
if k not in merged:
merged[k] = d2[k]
return merged
d1 = {'KEY1': [1, 2, 3]}
d2 = {'KEY1': [2, 3, 4]}
d3 = {'KEY2': [1, 2, 3]}
print(merge(d1, d2))
print(merge(d1, d3))
Output:
{'KEY1': [1, 2, 3, 4]}
{'KEY1': [1, 2, 3], 'KEY2': [1, 2, 3]}
define a merge function (here as lambda function):
merge = lambda x,y: {v:list(set(x.get(v, []) + y.get(v, []))) for v in x.keys() | y.keys()}
output:
merge(d1,d3)
{'KEY2': [1, 2, 3], 'KEY1': [1, 2, 3]}
merge(d1,d2)
{'KEY1': [1, 2, 3, 4]}
I got the expected output by using **dictionary which treats the elements of this iterable as positional arguments to any function call.
def merge(a,b):
try:
return {**{**a,**b},**{k: set(set(v).union(b[k])) for k, v in a.items()}}
except:
return {**a,**b}
print(merge(d1,d2))
print(merge(d1,d3))
Output:
I would like to merge two dictionaries, but if they have the same key, I would only merge non-duplicate values.
The following code works, but I have a question if it's possible to rewrite this when trying to get a union by using | or (**dict1, **dict2)? When I tried using |, my output would be from this dict_merge({ 'A': [1, 2, 3] }, { 'A': [2, 3, 4] }) to this {'A': [2, 3, 4]}
def dict_merge(dict1, dict2):
for key in dict2.keys():
if key in dict1.keys():
d3 = dict1[key] + dict2[key]
d3 = set(d3)
dict1[key] = list(d3)
else:
dict1[key] = dict2[key]
return dict1
dict_merge({ 'A': [1, 2, 3] }, { 'B': [2, 4, 5, 6]})
Output
{ 'A': [1, 2, 3], 'B': [2, 4, 5, 6] }
Giving your two dictionaries names, let's get the union of their keys.
>>> d1 = { 'A': [1, 2, 3] }
>>> d2 = { 'A': [2, 3, 4] }
>>> d1.keys() | d2.keys()
{'A'}
Assuming the lists are really sets based on your code, we can now iterate over the union of the keys in a dictionary comprehension, and union those two sets and turning them back into a list.
>>> {k: list(set(d1.get(k, [])) | set(d2.get(k, []))) for k in d1.keys() | d2.keys()}
{'A': [1, 2, 3, 4]}
If we incorporate some more interesting dictionaries and repeat the same dictionary comprehension:
>>> d1 = {'A': [1,2,3], 'B': [4,5,6]}
>>> d2 = {'B': [5,6,7,8], 'C': [9,10]}
>>> {k: list(set(d1.get(k, [])) | set(d2.get(k, []))) for k in d1.keys() | d2.keys()}
{'C': [9, 10], 'A': [1, 2, 3], 'B': [4, 5, 6, 7, 8]}
Is that the solution you're wanting?
In [61]: my_dict = {}
...: d1 = {'A': [1, 2, 3], 'C': 123}
...: d2 = {'B': [2, 3, 4], 'A': [1, 2, 3]}
...: for i in set(d1).symmetric_difference(set(d2)):
...: my_dict[i] = d1[i] if i in d1 else d2[i]
...: print(my_dict)
Output :
{'B': [2, 3, 4], 'C': 123}
So I have a list with several dictionaries, they all have the same keys. Some dictionaries are the same but one value is different. How could I merge them into 1 dictionary having that different values as array?
Let me give you an example:
let's say I have this dictionaries
[{'a':1, 'b':2,'c':3},{'a':1, 'b':2,'c':4},{'a':1, 'b':3,'c':3},{'a':1, 'b':3,'c':4}]
My desired output would be this:
[{'a':1, 'b':2,'c':[3,4]},{'a':1, 'b':3,'c':[3,4]}]
I've tried using for and if nested, but it's too expensive and nasty, and I'm sure there must be a better way. Could you give me a hand?
How could I do that for any kind of dictionary assuming that the amount of keys is the same on the dictionaries and knowing the name of the key to be merged as array (c in this case)
thanks!
Use a collections.defaultdict to group the c values by a and b tuple keys:
from collections import defaultdict
lst = [
{"a": 1, "b": 2, "c": 3},
{"a": 1, "b": 2, "c": 4},
{"a": 1, "b": 3, "c": 3},
{"a": 1, "b": 3, "c": 4},
]
d = defaultdict(list)
for x in lst:
d[x["a"], x["b"]].append(x["c"])
result = [{"a": a, "b": b, "c": c} for (a, b), c in d.items()]
print(result)
Could also use itertools.groupby if lst is already ordered by a and b:
from itertools import groupby
from operator import itemgetter
lst = [
{"a": 1, "b": 2, "c": 3},
{"a": 1, "b": 2, "c": 4},
{"a": 1, "b": 3, "c": 3},
{"a": 1, "b": 3, "c": 4},
]
result = [
{"a": a, "b": b, "c": [x["c"] for x in g]}
for (a, b), g in groupby(lst, key=itemgetter("a", "b"))
]
print(result)
Or if lst is not ordered by a and b, we can sort by those two keys as well:
result = [
{"a": a, "b": b, "c": [x["c"] for x in g]}
for (a, b), g in groupby(
sorted(lst, key=itemgetter("a", "b")), key=itemgetter("a", "b")
)
]
print(result)
Output:
[{'a': 1, 'b': 2, 'c': [3, 4]}, {'a': 1, 'b': 3, 'c': [3, 4]}]
Update
For a more generic solution for any amount of keys:
def merge_lst_dicts(lst, keys, merge_key):
groups = defaultdict(list)
for item in lst:
key = tuple(item.get(k) for k in keys)
groups[key].append(item.get(merge_key))
return [
{**dict(zip(keys, group_key)), **{merge_key: merged_values}}
for group_key, merged_values in groups.items()
]
print(merge_lst_dicts(lst, ["a", "b"], "c"))
# [{'a': 1, 'b': 2, 'c': [3, 4]}, {'a': 1, 'b': 3, 'c': [3, 4]}]
You could use a temp dict to solve this problem -
>>>python3
Python 3.6.9 (default, Nov 7 2019, 10:44:02)
>>> di=[{'a':1, 'b':2,'c':3},{'a':1, 'b':2,'c':4},{'a':1, 'b':3,'c':3},{'a':1, 'b':3,'c':4}]
>>> from collections import defaultdict as dd
>>> dt=dd(list) #default dict of list
>>> for d in di: #create temp dict with 'a','b' as tuple and append 'c'
... dt[d['a'],d['b']].append(d['c'])
>>> for k,v in dt.items(): #Create final output from temp
... ol.append({'a':k[0],'b':k[1], 'c':v})
...
>>> ol #output
[{'a': 1, 'b': 2, 'c': [3, 4]}, {'a': 1, 'b': 3, 'c': [3, 4]}]
If the number of keys in input dict is large, the process to extract
tuple for temp_dict can be automated -
if the keys the define condition for merging are known than it can be simply a constant tuple eg.
keys=('a','b') #in this case, merging happens over these keys
If this is not known at until runtime, then we can get these keys using zip function and set difference, eg.
>>> di
[{'a': 1, 'b': 2, 'c': 3}, {'a': 1, 'b': 2, 'c': 4}, {'a': 1, 'b': 3, 'c': 3}, {'a': 1, 'b': 3, 'c': 4}]
>>> key_to_ignore_for_merge='c'
>>> keys=tuple(set(list(zip(*zip(*di)))[0])-set(key_to_ignore_for_merge))
>>> keys
('a', 'b')
At this point, we can use map to extract tuple for keys only-
>>> dt=dd(list)
>>> for d in di:
... dt[tuple(map(d.get,keys))].append(d[key_to_ignore_for_merge])
>>> dt
defaultdict(<class 'list'>, {(1, 2): [3, 4], (1, 3): [3, 4]})
Now, to recreate the dictionary from default_dict and keys will require some zip magic again!
>>> for k,v in dt.items():
... dtt=dict(tuple(zip(keys, k)))
... dtt[key_to_ignore_for_merge]=v
... ol.append(dtt)
...
>>> ol
[{'a': 1, 'b': 2, 'c': [3, 4]}, {'a': 1, 'b': 3, 'c': [3, 4]}]
This solution assumes that you only know the keys that can be different (eg. 'c') and rest is all runtime.
I would like to add two dictionaries and not update first one with second. The value will be a single variable or a list.
What I would like to achieve:
Input:
x = {"a": [1] }
y = {"a": [2, 3], "b": [2] }
Output:
z = {"a": [1,2,3], "b": [2] }
It would be great if it could also accept a single int instead of only list as mentioned earlier:
Input:
x = {"a": 1 }
y = {"a": [2, 3], "b": [2] }
Output:
z = {"a": [1,2,3], "b": [2] }
So looking for
z = append_dicts(x, y)
This is what I came up with.
def merge_two_dicts(x, y):
"""Given two dicts, append them into a new dict."""
for key in x:
if isinstance(x[key], list):
y[key].extend(x[key])
else:
y[key].append(x[key])
return y
A one-liner (works on Python2.7)
dict([(k, x.get(k,[])+y.get(k,[])) for k in set(x)|set(y)])
and a not very pretty one that can digest non-lists
dict([(k, [a for b in [x.get(k,[])]+[y.get(k,[])] for a in (b if isinstance(b, list) else [b])]) for k in set(x)|set(y)])
This can be a solution for you:
def append_dicts(x, y):
z = dict(y)
for k, v in x.items():
if isinstance(v, list):
z[k].extend(v)
if isinstance(v, int):
z[k].append(v)
return z
x = {"a": [1] }
y = {"a": [2, 3], "b": [2] }
print(append_dicts(x, y))
# {'a': [2, 3, 1], 'b': [2]}
x = {"a": 1 }
y = {"a": [2, 3], "b": [2] }
print(append_dicts(x, y))
# {'a': [2, 3, 1], 'b': [2]}
Accumulating functions: one which accepts duplicates and one which does not...
#!/usr/bin/python
from collections import defaultdict
from sets import Set
def combineUnique(input_dict_list):
accumulating_dict = defaultdict(Set)
for d in input_dict_list:
for key,values in d.items():
for value in values:
accumulating_dict[key].add(value)
return accumulating_dict
def combineWithDuplicates(input_dict_list):
accumulating_dict = defaultdict(list)
for d in input_dict_list:
for key,value in d.items():
accumulating_dict[key].extend(value)
return accumulating_dict
if __name__=='__main__':
x = {"a": [1, 2] }
y = {"a": [2, 3], "b": [2] }
print "With duplicates..."
print combineWithDuplicates([x,y])
print "Unique..."
print combineUnique([x,y])
Returns...
With duplicates...
defaultdict(<type 'list'>, {'a': [1, 2, 2, 3], 'b': [2]})
Unique...
defaultdict(<class 'sets.Set'>, {'a': Set([1, 2, 3]), 'b': Set([2])})
I am trying to combine two dictionaries to yield a result like this:
a = {"cat": 3, "dog": 4, "rabbit": 19, "horse": 3, "shoe": 2}
b = {"cat": 2, "rabbit": 1, "fish": 9, "horse": 5}
ab = {"cat": 5, "dog": 4, "rabbit": 20, "horse": 8, "shoe": 2, "fish": 9}
So that if they have the same keys, the values will be added, if one key is present in one dictionary but not the other, it will add it to the new dictionary with its corresponding value.
These two dictionaries are also both nested in separate dictionaries as well such that:
x = {'a': {"cat": 3, "dog": 4, "rabbit": 19, "horse": 3, "shoe": 2}, 'c': blah, 'e': fart}
y = {'a': {"cat": 2, "rabbit": 1, "fish": 9, "horse": 5}, 'c': help, 'e': me}
The keys are the same in both main dictionaries.
I have been trying to combine the two dictionaries:
def newdict(x,y):
merged= [x,y]
newdict = {}
for i in merged:
for k,v in i.items():
new.setdefault(k,[]).append(v)
All this gives me is a dictionary with values belonging to the same keys in a list. I can't figure out how to iterate through the two lists for a key and add the values together to create one joint dictionary. Can anyone help me?
End result should be something like:
xy = {'a' = {"cat": 5, "dog": 4, "rabbit": 20, "horse": 8, "shoe": 2, "fish": 9}, 'c': blah, 'e': me}
The 'c' and 'e' keys I will have to iterate through and perform a different calculation based on the results from 'a'.
I hope I explained my problem clearly enough.
My attempt would be:
a = {"cat": 3, "dog": 4, "rabbit": 19, "horse": 3, "shoe": 2}
b = {"cat": 2, "rabbit": 1, "fish": 9, "horse": 5}
def newdict(x, y):
ret = {}
for key in x.keys():
if isinstance(x[key], dict):
ret[key] = newdict(x[key], y.get(key, {}))
continue
ret[key] = x[key] + y.get(key, 0)
for key in y.keys():
if isinstance(y[key], dict):
ret[key] = newdict(y[key], x.get(key, {}))
continue
ret[key] = y[key] + x.get(key, 0)
return ret
ab = newdict(a, b)
print ab
> {'horse': 8, 'fish': 9, 'dog': 4, 'cat': 5, 'shoe': 2, 'rabbit': 20}
Explanation:
The newdict function first iterates through the first dictionary (x). For every key in x, it creates a new entry in the new dictionary, setting the value to the sum of x[key] and y[key]. The dict.get function supplies an optional second argument that it returns when key isn't in dict.
If x[key] is a dict, it sets ret[key] to a merged dictionary of x[key] and y[key].
It then does the same for y and returns.
Note: This doesn't work for functions. Try figuring something out yourself there.
Using collections.Counter and isinstance:
>>> from collections import Counter
>>> from itertools import chain
>>> x = {'e': 'fart', 'a': {'dog': 4, 'rabbit': 19, 'shoe': 2, 'cat': 3, 'horse': 3}, 'c': 'blah'}
>>> y = {'e': 'me', 'a': {'rabbit': 1, 'fish': 9, 'cat': 2, 'horse': 5}, 'c': 'help'}
>>> c = {}
>>> for k, v in chain(x.items(), y.items()):
if isinstance(v, dict):
c[k] = c.get(k, Counter()) + Counter(v)
...
>>> c
{'a': Counter({'rabbit': 20, 'fish': 9, 'horse': 8, 'cat': 5, 'dog': 4, 'shoe': 2})}
Now based on the value of 'a' you can calculate the values for keys 'a' and 'e', but this time use: if not isinstance(v, dict)
Update: Solution using no imports:
>>> c = {}
>>> for d in (x, y):
for k, v in d.items():
if isinstance(v, dict):
keys = (set(c[k]) if k in c else set()).union(set(v)) #Common keys
c[k] = { k1: v.get(k1, 0) + c.get(k, {}).get(k1, 0) for k1 in keys}
...
>>> c
{'a': {'dog': 4, 'rabbit': 20, 'shoe': 2, 'fish': 9, 'horse': 8, 'cat': 5}}
To do it easily, you can use collections.Counter:
>>> from collections import Counter
>>> a = {"cat": 3, "dog": 4, "rabbit": 19, "horse": 3, "shoe": 2}
>>> b = {"cat": 2, "rabbit": 1, "fish": 9, "horse": 5}
>>> Counter(a) + Counter(b)
Counter({'rabbit': 20, 'fish': 9, 'horse': 8, 'cat': 5, 'dog': 4, 'shoe': 2})
So, in your case, it would be something like:
newdict['a'] = Counter(x['a']) + Counter(y['a'])
If you for some reason don't want it to be a Counter, you just pass the result to dict().
Edit:
If you're not allowed imports, you'll have to do the addition manually, but this should be simple enough.
Since this sounds like homework, I'll give you a few hints instead of a full answer:
create a collection of all keys, or loop over each dict(you can use a set to make sure the keys are unique, but duplicates shouldn't be a problem, since they'll be overwritten)
for each key, add the sum of values in the old dicts to the new dict(you can use dict.get() to get a 0 if the key is not present)
def newDict(a,b):
newD={}
for key in a:
newD[key]=a[key]
for key in b:
newD[key]=newD.get(key,0)+b[key]
return newD
My naive solution is:
a = {'a':'b'}
b = {'c':'d'}
c = {'e':'f'}
def Merge(n):
m = {}
for i in range(len(n)):
m.update({i+1:n[i]})
return m
print(Merge([a,b,c]))