Related
I want to merge list of dictionaries in python. The number of dictionaries contained inside the list is not fixed and the nested dictionaries are being merged on both same and different keys. The dictionaries within the list do not contain nested dictionary. The values from same keys can be stored in a list.
My code is:
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3} ...... ]
output = {}
for i in list_of_dict:
for k,v in i.items():
if k in output:
output[k].append(v)
else:
output[k] = [v]
Is there a shorter and faster way of implementing this?
I am actually trying to implement the most fast way of doing this because the list of dictionary is very large and then there are lots of rows with such data.
One way using collections.defaultdict:
from collections import defaultdict
res = defaultdict(list)
for d in list_of_dict:
for k, v in d.items():
res[k].append(v)
Output:
defaultdict(list,
{'a': [1, 3, 3, 3],
'b': [2, 5],
'c': [3],
'k': [5, 5],
'j': [5],
'd': [4]})
items() is a dictionary method, but list_of_dict is a list. You need a nested loop so you can loop over the dictionaries and then loop over the items of each dictionary.
ou = {}
for d in list_of_dict:
for key, value in d.items():
output.setdefault(key, []).append(value)
another shorten version can be,
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3}]
output = {
k: [d[k] for d in list_of_dict if k in d]
for k in set().union(*list_of_dict)
}
print(output)
{'d': [4], 'k': [5, 5], 'a': [1, 3, 3, 3], 'j': [5], 'c': [3], 'b': [2, 5]}
Python 3.9+ you can use the merge operator for this.
def merge_dicts(dicts):
result = dict()
for _dict in dicts:
result |= _dict
return result
One of the shortest way would be to
prepare a list/set of all the keys from all the dictionaries
and call that key on all the dictionary in the list.
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3}]
# prepare a list/set of all the keys from all the dictionaries
# method 1: use sum
all_keys = sum([[a for a in x.keys()] for x in list_of_dict], [])
# method 2: use itertools
import itertools
all_keys = list(itertools.chain.from_iterable(list_of_dict))
# method 3: use union of the set
all_keys = set().union(*list_of_dict)
print(all_keys)
# ['a', 'b', 'c', 'a', 'b', 'k', 'j', 'a', 'k', 'd', 'a']
# convert the list to set to remove duplicates
all_keys = set(all_keys)
print(all_keys)
# {'a', 'k', 'c', 'd', 'b', 'j'}
# now merge the dictionary
merged = {k: [d.get(k) for d in list_of_dict if k in d] for k in all_keys}
print(merged)
# {'a': [1, 3, 3, 3], 'k': [5, 5], 'c': [3], 'd': [4], 'b': [2, 5], 'j': [5]}
In short:
all_keys = set().union(*list_of_dict)
merged = {k: [d.get(k) for d in list_of_dict if k in d] for k in all_keys}
print(merged)
# {'a': [1, 3, 3, 3], 'k': [5, 5], 'c': [3], 'd': [4], 'b': [2, 5], 'j': [5]}
I want to merge list of dictionaries in python. The number of dictionaries contained inside the list is not fixed and the nested dictionaries are being merged on both same and different keys. The dictionaries within the list do not contain nested dictionary. The values from same keys can be stored in a list.
My code is:
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3} ...... ]
output = {}
for i in list_of_dict:
for k,v in i.items():
if k in output:
output[k].append(v)
else:
output[k] = [v]
Is there a shorter and faster way of implementing this?
I am actually trying to implement the most fast way of doing this because the list of dictionary is very large and then there are lots of rows with such data.
One way using collections.defaultdict:
from collections import defaultdict
res = defaultdict(list)
for d in list_of_dict:
for k, v in d.items():
res[k].append(v)
Output:
defaultdict(list,
{'a': [1, 3, 3, 3],
'b': [2, 5],
'c': [3],
'k': [5, 5],
'j': [5],
'd': [4]})
items() is a dictionary method, but list_of_dict is a list. You need a nested loop so you can loop over the dictionaries and then loop over the items of each dictionary.
ou = {}
for d in list_of_dict:
for key, value in d.items():
output.setdefault(key, []).append(value)
another shorten version can be,
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3}]
output = {
k: [d[k] for d in list_of_dict if k in d]
for k in set().union(*list_of_dict)
}
print(output)
{'d': [4], 'k': [5, 5], 'a': [1, 3, 3, 3], 'j': [5], 'c': [3], 'b': [2, 5]}
Python 3.9+ you can use the merge operator for this.
def merge_dicts(dicts):
result = dict()
for _dict in dicts:
result |= _dict
return result
One of the shortest way would be to
prepare a list/set of all the keys from all the dictionaries
and call that key on all the dictionary in the list.
list_of_dict = [{'a': 1, 'b': 2, 'c': 3}, {'a': 3, 'b': 5}, {'k': 5, 'j': 5}, {'a': 3, 'k': 5, 'd': 4}, {'a': 3}]
# prepare a list/set of all the keys from all the dictionaries
# method 1: use sum
all_keys = sum([[a for a in x.keys()] for x in list_of_dict], [])
# method 2: use itertools
import itertools
all_keys = list(itertools.chain.from_iterable(list_of_dict))
# method 3: use union of the set
all_keys = set().union(*list_of_dict)
print(all_keys)
# ['a', 'b', 'c', 'a', 'b', 'k', 'j', 'a', 'k', 'd', 'a']
# convert the list to set to remove duplicates
all_keys = set(all_keys)
print(all_keys)
# {'a', 'k', 'c', 'd', 'b', 'j'}
# now merge the dictionary
merged = {k: [d.get(k) for d in list_of_dict if k in d] for k in all_keys}
print(merged)
# {'a': [1, 3, 3, 3], 'k': [5, 5], 'c': [3], 'd': [4], 'b': [2, 5], 'j': [5]}
In short:
all_keys = set().union(*list_of_dict)
merged = {k: [d.get(k) for d in list_of_dict if k in d] for k in all_keys}
print(merged)
# {'a': [1, 3, 3, 3], 'k': [5, 5], 'c': [3], 'd': [4], 'b': [2, 5], 'j': [5]}
This question already has answers here:
Return a default value if a dictionary key is not available
(15 answers)
Closed 1 year ago.
I want to take the elements of a list one by one and search them in 4 different dictionaries in python. then I want to create a new dictionary and put the elements of that list as keys and the values I found from those 4 dictionaries as values?
For example:
list = ['a', 'b', 'c', 'd', 'e']
dict1 = {'a': 10, 'b': 2, 'c': 45}
dict2 = {'a': 15, 'b': 55}
dict3 = {'a': 79, 'b': 6, 'c': 3}
dict4 = {'d': 600, 'e': 30}
The result I want:
newlist = {'a': [10, 15, 79, 0],
'b': [2, 55, 6, 0],
'c': [45, 0, 3, 0],
'd': [0, 0, 0, 600],
'e': [0, 0, 0, 30]}
This dict comprehension will result in what you're looking for:
dicts = dict1, dict2, dict3, dict4
{k: [d.get(k, 0) for d in dicts] for k in list1}
Something like this?
from collections import defaultdict
list1 = ['a', 'b', 'c', 'd', 'e']
dict1 = {'a': 10, 'b': 2, 'c': 45}
dict2 = {'a':11, 'b':20, 'z':100}
def collect_values(list1, dictionaries):
result = defaultdict(list)
for key in list1:
for d in dictionaries:
result[key].append(d.get(key, 0))
return result
print(collect_values(list1, [dict1, dict2]))
Which would return
defaultdict(<class 'list'>, {'a': [10, 11], 'b': [2, 20], 'c': [45, 0], 'd': [0, 0], 'e': [0, 0]})
EDIT
If appending zeros each time the key is not found in the dictionary is undesired behavior, the function below can be used:
def collect_values(list1, dictionaries):
result = defaultdict(list)
for key in list1:
for d in dictionaries:
if key in d.keys():
result[key].append(d[key])
elif not result[key]:
result[key].append(0)
return result
Which returns
defaultdict(<class 'list'>, {'a': [10, 11], 'b': [2, 20], 'c': [45], 'd': [0], 'e': [0]})
I have a dictionary that looks something like this:
letters_by_number = {
1: ['a', 'b', 'c', 'd'],
2: ['b', 'd'],
3: ['a', 'c'],
4: ['a', 'd'],
5: ['b', 'c']
}
I want to reverse it to look something like this:
numbers_by_letter = {
'a': [1, 3, 4],
'b': [1, 2, 5],
'c': [1, 3, 5],
'd': [1, 2, 4]
}
I know that I could do this by looping through (key, value) through letters_by_number, looping through value (which is a list), and adding (val, key) to a list in the dictionary. This is cumbersome and I feel like there must be a more "pythonic" way to do this. Any suggestions?
This is well-suited for collections.defaultdict:
>>> from collections import defaultdict
>>> numbers_by_letter = defaultdict(list)
>>> for k, seq in letters_by_number.items():
... for letter in seq:
... numbers_by_letter[letter].append(k)
...
>>> dict(numbers_by_letter)
{'a': [1, 3, 4], 'b': [1, 2, 5], 'c': [1, 3, 5], 'd': [1, 2, 4]}
Note that you don't really need the final dict() call (a defaultdict will already give you the behavior you probably want), but I included it here because the result from your question is type dict.
Use setdefault:
letters_by_number = {
1: ['a', 'b', 'c', 'd'],
2: ['b', 'd'],
3: ['a', 'c'],
4: ['a', 'd'],
5: ['b', 'c']
}
inv = {}
for k, vs in letters_by_number.items():
for v in vs:
inv.setdefault(v, []).append(k)
print(inv)
Output
{'a': [1, 3, 4], 'b': [1, 2, 5], 'c': [1, 3, 5], 'd': [1, 2, 4]}
A (trivial) subclass of dict would make this very easy:
class ListDict(dict):
def __missing__(self, key):
value = self[key] = []
return value
letters_by_number = {
1: ['a', 'b', 'c', 'd'],
2: ['b', 'd'],
3: ['a', 'c'],
4: ['a', 'd'],
5: ['b', 'c']
}
numbers_by_letter = ListDict()
for key, values in letters_by_number.items():
for value in values:
numbers_by_letter[value].append(key)
from pprint import pprint
pprint(numbers_by_letter, width=40)
Output:
{'a': [1, 3, 4],
'b': [1, 2, 5],
'c': [1, 3, 5],
'd': [1, 2, 4]}
Here's a solution using a dict comprehension, without adding list elements in a loop. Build a set of keys by joining all the lists together, then build each list using a list comprehension. To be more efficient, I've first built another dictionary containing sets instead of lists, so that k in v is an O(1) operation.
from itertools import chain
def invert_dict_of_lists(d):
d = { i: set(v) for i, v in d.items() }
return {
k: [ i for i, v in d.items() if k in v ]
for k in set(chain.from_iterable(d.values()))
}
Strictly, dictionaries in modern versions of Python 3 retain the order that keys are inserted in. This produces a result where the keys are in the order they appear in the lists; not alphabetical order like in your example. If you do want the keys in sorted order, change for k in set(...) to for k in sorted(set(...)).
I need to take a list and use a dictionary to catalogue where a particular item occurs in a list, as an example:
L = ['a', 'b', 'c', 'b', 'c', 'a', 'e']
the dictionary needs to contain the following:
D = {'a': 0, 5 , 'b': 1, 3 , 'c': 2, 4 , 'e': 6}
However if I use what I wrote:
for i in range(len(word_list)):
if D.has_key('word_list[i]') == False:
D['word_list[i]'] = i
else:
D[word_list[i]] += i
Then I get a KeyError for a certain word and I don't understand why I should be getting an error.
if D.has_key('word_list[i]') == False:
Uh, what?
At the very least, you should drop the quotes:
if D.has_key(word_list[i]) == False:
But you're also misusing a number of Python structures:
Why are summing up the indices?
Why are you comparing to False?
Shouldn't you be using setdefault
Like this:
for i in range(len(word_list)):
D.setdefault(word_list[i], []).append(i)
I modified you solution a bit to work
word_list = ['a', 'b', 'c', 'b', 'c', 'a', 'e']
dict = {'a': [], 'b': [], 'c': [], 'e': []}
for i in range(len(word_list)):
if word_list[i] not in dict:
dict[word_list[i]] = [i]
else:
dict[word_list[i]].append(i)
Result
{'a': [0, 5], 'c': [2, 4], 'b': [1, 3], 'e': [6]}
I think this would be the shortest solution for your problem:
>>> from collections import defaultdict
>>> D = defaultdict(list)
>>> for i,el in enumerate(L):
D[el].append(i)
>>> D
defaultdict(<type 'list'>, {'a': [0, 5], 'c': [2, 4], 'b': [1, 3], 'e': [6]})
If you want to stick with dict, correcting your code I would came up with:
>>> D = {}
>>> for i,el in enumerate(L):
if el not in D:
D[el] = [i] #crate a new list
else:
D[el].append(i) #appending to the existing list
>>> D
{'a': [0, 5], 'c': [2, 4], 'b': [1, 3], 'e': [6]}
Also, there is a setdefault method in dict which can be used:
>>> D = {}
>>> for i,el in enumerate(L):
D.setdefault(el,[]).append(i)
>>> D
{'a': [0, 5], 'c': [2, 4], 'b': [1, 3], 'e': [6]}
But I prefer to use defaultdict from collections.