Can I use list comprehension syntax to create a dictionary?
For example, by iterating over pairs of keys and values:
d = {... for k, v in zip(keys, values)}
Use a dict comprehension (Python 2.7 and later):
{key: value for (key, value) in iterable}
Alternatively for simpler cases or earlier version of Python, use the dict constructor, e.g.:
pairs = [('a', 1), ('b', 2)]
dict(pairs) #=> {'a': 1, 'b': 2}
dict([(k, v+1) for k, v in pairs]) #=> {'a': 2, 'b': 3}
Given separate arrays of keys and values, use the dict constructor with zip:
keys = ['a', 'b']
values = [1, 2]
dict(zip(keys, values)) #=> {'a': 1, 'b': 2}
2) "zip'ped" from two separate iterables of keys/vals
dict(zip(list_of_keys, list_of_values))
In Python 3 and Python 2.7+, dictionary comprehensions look like the below:
d = {k:v for k, v in iterable}
For Python 2.6 or earlier, see fortran's answer.
In fact, you don't even need to iterate over the iterable if it already comprehends some kind of mapping, the dict constructor doing it graciously for you:
>>> ts = [(1, 2), (3, 4), (5, 6)]
>>> dict(ts)
{1: 2, 3: 4, 5: 6}
>>> gen = ((i, i+1) for i in range(1, 6, 2))
>>> gen
<generator object <genexpr> at 0xb7201c5c>
>>> dict(gen)
{1: 2, 3: 4, 5: 6}
Create a dictionary with list comprehension in Python
I like the Python list comprehension syntax.
Can it be used to create dictionaries too? For example, by iterating
over pairs of keys and values:
mydict = {(k,v) for (k,v) in blah blah blah}
You're looking for the phrase "dict comprehension" - it's actually:
mydict = {k: v for k, v in iterable}
Assuming blah blah blah is an iterable of two-tuples - you're so close. Let's create some "blahs" like that:
blahs = [('blah0', 'blah'), ('blah1', 'blah'), ('blah2', 'blah'), ('blah3', 'blah')]
Dict comprehension syntax:
Now the syntax here is the mapping part. What makes this a dict comprehension instead of a set comprehension (which is what your pseudo-code approximates) is the colon, : like below:
mydict = {k: v for k, v in blahs}
And we see that it worked, and should retain insertion order as-of Python 3.7:
>>> mydict
{'blah0': 'blah', 'blah1': 'blah', 'blah2': 'blah', 'blah3': 'blah'}
In Python 2 and up to 3.6, order was not guaranteed:
>>> mydict
{'blah0': 'blah', 'blah1': 'blah', 'blah3': 'blah', 'blah2': 'blah'}
Adding a Filter:
All comprehensions feature a mapping component and a filtering component that you can provide with arbitrary expressions.
So you can add a filter part to the end:
>>> mydict = {k: v for k, v in blahs if not int(k[-1]) % 2}
>>> mydict
{'blah0': 'blah', 'blah2': 'blah'}
Here we are just testing for if the last character is divisible by 2 to filter out data before mapping the keys and values.
In Python 2.7, it goes like:
>>> list1, list2 = ['a', 'b', 'c'], [1,2,3]
>>> dict( zip( list1, list2))
{'a': 1, 'c': 3, 'b': 2}
Zip them!
Python version >= 2.7, do the below:
d = {i: True for i in [1,2,3]}
Python version < 2.7(RIP, 3 July 2010 - 31 December 2019), do the below:
d = dict((i,True) for i in [1,2,3])
To add onto #fortran's answer, if you want to iterate over a list of keys key_list as well as a list of values value_list:
d = dict((key, value) for (key, value) in zip(key_list, value_list))
or
d = {(key, value) for (key, value) in zip(key_list, value_list)}
Just to throw in another example. Imagine you have the following list:
nums = [4,2,2,1,3]
and you want to turn it into a dict where the key is the index and value is the element in the list. You can do so with the following line of code:
{index:nums[index] for index in range(0,len(nums))}
Here is another example of dictionary creation using dict comprehension:
What i am tring to do here is to create a alphabet dictionary where each pair; is the english letter and its corresponding position in english alphabet
>>> import string
>>> dict1 = {value: (int(key) + 1) for key, value in
enumerate(list(string.ascii_lowercase))}
>>> dict1
{'a': 1, 'c': 3, 'b': 2, 'e': 5, 'd': 4, 'g': 7, 'f': 6, 'i': 9, 'h': 8,
'k': 11, 'j': 10, 'm': 13, 'l': 12, 'o': 15, 'n': 14, 'q': 17, 'p': 16, 's':
19, 'r': 18, 'u': 21, 't': 20, 'w': 23, 'v': 22, 'y': 25, 'x': 24, 'z': 26}
>>>
Notice the use of enumerate here to get a list of alphabets and their indexes in the list and swapping the alphabets and indices to generate the key value pair for dictionary
Hope it gives a good idea of dictionary comp to you and encourages you to use it more often to make your code compact
This code will create dictionary using list comprehension for multiple lists with different values that can be used for pd.DataFrame()
#Multiple lists
model=['A', 'B', 'C', 'D']
launched=[1983,1984,1984,1984]
discontinued=[1986, 1985, 1984, 1986]
#Dictionary with list comprehension
keys=['model','launched','discontinued']
vals=[model, launched,discontinued]
data = {key:vals[n] for n, key in enumerate(keys)}
#Convert dict to dataframe
df=pd.DataFrame(data)
display(df)
enumerate will pass n to vals to match each key with its list
Try this,
def get_dic_from_two_lists(keys, values):
return { keys[i] : values[i] for i in range(len(keys)) }
Assume we have two lists country and capital
country = ['India', 'Pakistan', 'China']
capital = ['New Delhi', 'Islamabad', 'Beijing']
Then create dictionary from the two lists:
print get_dic_from_two_lists(country, capital)
The output is like this,
{'Pakistan': 'Islamabad', 'China': 'Beijing', 'India': 'New Delhi'}
Adding to #Ekhtiar answer, if you want to make look up dict from list, you can use this:
names = ['a', 'b', 'd', 'f', 'c']
names_to_id = {v:k for k, v in enumerate(names)}
# {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'f': 4}
Or in rare case that you want to filter duplicate, use set first (best in list of number):
names = ['a', 'b', 'd', 'f', 'd', 'c']
sorted_list = list(set(names))
sorted_list.sort()
names_to_id = {v:k for k, v in enumerate(sorted_list)}
# {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'f': 4}
names = [1,2,5,5,6,2,1]
names_to_id = {v:k for k, v in enumerate(set(names))}
# {1: 0, 2: 1, 5: 2, 6: 3}
>>> {k: v**3 for (k, v) in zip(string.ascii_lowercase, range(26))}
Python supports dict comprehensions, which allow you to express the creation of dictionaries at runtime using a similarly concise syntax.
A dictionary comprehension takes the form {key: value for (key, value) in iterable}. This syntax was introduced in Python 3 and backported as far as Python 2.7, so you should be able to use it regardless of which version of Python you have installed.
A canonical example is taking two lists and creating a dictionary where the item at each position in the first list becomes a key and the item at the corresponding position in the second list becomes the value.
The zip function used inside this comprehension returns an iterator of tuples, where each element in the tuple is taken from the same position in each of the input iterables. In the example above, the returned iterator contains the tuples (“a”, 1), (“b”, 2), etc.
Output:
{'i': 512, 'e': 64, 'o': 2744, 'h': 343, 'l': 1331, 's': 5832, 'b': 1, 'w': 10648, 'c': 8, 'x': 12167, 'y': 13824, 't': 6859, 'p': 3375, 'd': 27, 'j': 729, 'a': 0, 'z': 15625, 'f': 125, 'q': 4096, 'u': 8000, 'n': 2197, 'm': 1728, 'r': 4913, 'k': 1000, 'g': 216, 'v': 9261}
Yes, it's possible. In python, Comprehension can be used in List, Set, Dictionary, etc.
You can write it this way
mydict = {k:v for (k,v) in blah}
Another detailed example of Dictionary Comprehension with the Conditional Statement and Loop:
parents = [father, mother]
parents = {parent:1 - P["mutation"] if parent in two_genes else 0.5 if parent in one_gene else P["mutation"] for parent in parents}
You can create a new dict for each pair and merge it with the previous dict:
reduce(lambda p, q: {**p, **{q[0]: q[1]}}, bla bla bla, {})
Obviously this approaches requires reduce from functools.
Assuming blah blah blah is a two-tuples list:
Let's see two methods:
# method 1
>>> lst = [('a', 2), ('b', 4), ('c', 6)]
>>> dict(lst)
{'a': 2, 'b': 4, 'c': 6}
# method 2
>>> lst = [('a', 2), ('b', 4), ('c', 6)]
>>> d = {k:v for k, v in lst}
>>> d
{'a': 2, 'b': 4, 'c': 6}
this approach uses iteration over the given date using a for loop.
Syntax: {key: value for (key, value) in data}
Eg:
# create a list comprehension with country and code:
Country_code = [('China', 86), ('USA', 1),
('Ghana', 233), ('Uk', 44)]
# use iterable method to show results
{key: value for (key, value) in Country_code}
Related
I have a complex dictionary:
l = {10: [{'a':1, 'T':'y'}, {'a':2, 'T':'n'}], 20: [{'a':3,'T':'n'}]}
When I'm trying to iterate over the dictionary I'm not getting a dictionary with a list for values that are a dictionary I'm getting a tuple like so:
for m in l.items():
print(m)
(10, [{'a': 1, 'T': 'y'}, {'a': 2, 'T': 'n'}])
(20, [{'a': 3, 'T': 'n'}])
But when I just print l I get my original dictionary:
In [7]: l
Out[7]: {10: [{'a': 1, 'T': 'y'}, {'a': 2, 'T': 'n'}], 20: [{'a': 3, 'T': 'n'}]}
How do I iterate over the dictionary? I still need the keys and to process each dictionary in the value list.
There are two questions here. First, you ask why this is turned into a "tuple" - the answer to that question is because that is what the .items() method on dictionaries returns - a tuple of each key/value pair.
Knowing this, you can then decide how to use this information. You can choose to expand the tuple into the two parts during iteration
for k, v in l.items():
# Now k has the value of the key and v is the value
# So you can either use the value directly
print(v[0]);
# or access using the key
value = l[k];
print(value[0]);
# Both yield the same value
With a dictionary you can add another variable while iterating over it.
for key, value in l.items():
print(key,value)
I often rely on pprint when processing a nested object to know at a glance what structure that I am dealing with.
from pprint import pprint
l = {10: [{'a':1, 'T':'y'}, {'a':2, 'T':'n'}], 20: [{'a':3,'T':'n'}]}
pprint(l, indent=4, width=40)
Output:
{ 10: [ {'T': 'y', 'a': 1},
{'T': 'n', 'a': 2}],
20: [{'T': 'n', 'a': 3}]}
Others have already answered with implementations.
Thanks for all the help. I did discuss figure out how to process this. Here is the implementation I came up with:
for m in l.items():
k,v = m
print(f"key: {k}, val: {v}")
for n in v:
print(f"key: {n['a']}, val: {n['T']}")
Thanks for everyones help!
Given the two dictionaries below, I'm trying to create a list of new dictionaries combining the items from one (com), which will repeat for each member, with only the values of the second (e), entered one at a time, under key 'n'. E.g. first list member would be:
{'n': 330, 'b': 2, 'a': 1}
If I use update() within the list comprehension to add the key-pair values from the first dictionary to the result I get a list with two None members.
I've tried different ways to write this, e.g. using map() and on both python 2 and 3; so I ask the experts.
>>> com
{'b': 2, 'a': 1}
>>> e
{'p': 330, 'r': 220}
>>> [n for rt in e.values() for n in [{'n':rt}]]
[{'n': 330}, {'n': 220}]
>>> [n.update(com) for rt in e.values() for n in [{'n':rt}]]
[None, None]
To clarify the problem statement:
Given a fixed map com and a map containing a series of elements (I would find the name 'nvals' more mnemonic), return a list of maps, where each element of the list has one of the values from nvals as the value of key 'n', and all the elements of com.
So the following code returns the required result:
>>> com = {'b': 2, 'a': 1}
>>> nvals = {'p': 330, 'r': 220}
>>> l = []
>>> for n in nvals.values():
... d = dict(com) # Make sure to clone the dictionary.
... d["n"] = n
... l.append(d)
...
>>> l
[{'n': 330, 'b': 2, 'a': 1}, {'n': 220, 'b': 2, 'a': 1}]
>>>
The trouble is that is not a list comprehension
>>> [dict(list(com.items()) + [('n', n)]) for n in nvals.values()]
[{'n': 330, 'b': 2, 'a': 1}, {'n': 220, 'b': 2, 'a': 1}]
>>>
Seems to meet the requirements.
I have to say, I think I find the loop easier to read.
Try this:
com = {'b': 2, 'a': 1}
e = {'p': 330, 'r': 220}
res = [{i:j for i,j in com.items()+[("n",v)]} for v in e.values()]
Or this:
res = map(lambda x: dict([("n",x)]+com.items()), e.values())
Or maybe:
res = [dict(zip(["n"]+com.keys(),[v]+com.values())) for v in e.values()]
Don't you just love python?
I know how to remove an entry, 'key' from my dictionary d, safely. You do:
if d.has_key('key'):
del d['key']
However, I need to remove multiple entries from a dictionary safely. I was thinking of defining the entries in a tuple as I will need to do this more than once.
entities_to_remove = ('a', 'b', 'c')
for x in entities_to_remove:
if x in d:
del d[x]
However, I was wondering if there is a smarter way to do this?
Using dict.pop:
d = {'some': 'data'}
entries_to_remove = ('any', 'iterable')
for k in entries_to_remove:
d.pop(k, None)
Using Dict Comprehensions
final_dict = {key: value for key, value in d if key not in [key1, key2]}
where key1 and key2 are to be removed.
In the example below, keys "b" and "c" are to be removed & it's kept in a keys list.
>>> a
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
>>> keys = ["b", "c"]
>>> print {key: a[key] for key in a if key not in keys}
{'a': 1, 'd': 4}
>>>
Why not like this:
entries = ('a', 'b', 'c')
the_dict = {'b': 'foo'}
def entries_to_remove(entries, the_dict):
for key in entries:
if key in the_dict:
del the_dict[key]
A more compact version was provided by mattbornski using dict.pop()
a solution is using map and filter functions
python 2
d={"a":1,"b":2,"c":3}
l=("a","b","d")
map(d.__delitem__, filter(d.__contains__,l))
print(d)
python 3
d={"a":1,"b":2,"c":3}
l=("a","b","d")
list(map(d.__delitem__, filter(d.__contains__,l)))
print(d)
you get:
{'c': 3}
If you also need to retrieve the values for the keys you are removing, this would be a pretty good way to do it:
values_removed = [d.pop(k, None) for k in entities_to_remove]
You could of course still do this just for the removal of the keys from d, but you would be unnecessarily creating the list of values with the list comprehension. It is also a little unclear to use a list comprehension just for the function's side effect.
Found a solution with pop and map
d = {'a': 'valueA', 'b': 'valueB', 'c': 'valueC', 'd': 'valueD'}
keys = ['a', 'b', 'c']
list(map(d.pop, keys))
print(d)
The output of this:
{'d': 'valueD'}
I have answered this question so late just because I think it will help in the future if anyone searches the same. And this might help.
Update
The above code will throw an error if a key does not exist in the dict.
DICTIONARY = {'a': 'valueA', 'b': 'valueB', 'c': 'valueC', 'd': 'valueD'}
keys = ['a', 'l', 'c']
def remove_key(key):
DICTIONARY.pop(key, None)
list(map(remove_key, keys))
print(DICTIONARY)
output:
DICTIONARY = {'b': 'valueB', 'd': 'valueD'}
Some timing tests for cpython 3 shows that a simple for loop is the fastest way, and it's quite readable. Adding in a function doesn't cause much overhead either:
timeit results (10k iterations):
all(x.pop(v) for v in r) # 0.85
all(map(x.pop, r)) # 0.60
list(map(x.pop, r)) # 0.70
all(map(x.__delitem__, r)) # 0.44
del_all(x, r) # 0.40
<inline for loop>(x, r) # 0.35
def del_all(mapping, to_remove):
"""Remove list of elements from mapping."""
for key in to_remove:
del mapping[key]
For small iterations, doing that 'inline' was a bit faster, because of the overhead of the function call. But del_all is lint-safe, reusable, and faster than all the python comprehension and mapping constructs.
I have no problem with any of the existing answers, but I was surprised to not find this solution:
keys_to_remove = ['a', 'b', 'c']
my_dict = {k: v for k, v in zip("a b c d e f g".split(' '), [0, 1, 2, 3, 4, 5, 6])}
for k in keys_to_remove:
try:
del my_dict[k]
except KeyError:
pass
assert my_dict == {'d': 3, 'e': 4, 'f': 5, 'g': 6}
Note: I stumbled across this question coming from here. And my answer is related to this answer.
I have tested the performance of three methods:
# Method 1: `del`
for key in remove_keys:
if key in d:
del d[key]
# Method 2: `pop()`
for key in remove_keys:
d.pop(key, None)
# Method 3: comprehension
{key: v for key, v in d.items() if key not in remove_keys}
Here are the results of 1M iterations:
del: 2.03s 2.0 ns/iter (100%)
pop(): 2.38s 2.4 ns/iter (117%)
comprehension: 4.11s 4.1 ns/iter (202%)
So both del and pop() are the fastest. Comprehensions are 2x slower.
But anyway, we speak nanoseconds here :) Dicts in Python are ridiculously fast.
Why not:
entriestoremove = (2,5,1)
for e in entriestoremove:
if d.has_key(e):
del d[e]
I don't know what you mean by "smarter way". Surely there are other ways, maybe with dictionary comprehensions:
entriestoremove = (2,5,1)
newdict = {x for x in d if x not in entriestoremove}
inline
import functools
#: not key(c) in d
d = {"a": "avalue", "b": "bvalue", "d": "dvalue"}
entitiesToREmove = ('a', 'b', 'c')
#: python2
map(lambda x: functools.partial(d.pop, x, None)(), entitiesToREmove)
#: python3
list(map(lambda x: functools.partial(d.pop, x, None)(), entitiesToREmove))
print(d)
# output: {'d': 'dvalue'}
I think using the fact that the keys can be treated as a set is the nicest way if you're on python 3:
def remove_keys(d, keys):
to_remove = set(keys)
filtered_keys = d.keys() - to_remove
filtered_values = map(d.get, filtered_keys)
return dict(zip(filtered_keys, filtered_values))
Example:
>>> remove_keys({'k1': 1, 'k3': 3}, ['k1', 'k2'])
{'k3': 3}
It would be nice to have full support for set methods for dictionaries (and not the unholy mess we're getting with Python 3.9) so that you could simply "remove" a set of keys. However, as long as that's not the case, and you have a large dictionary with potentially a large number of keys to remove, you might want to know about the performance. So, I've created some code that creates something large enough for meaningful comparisons: a 100,000 x 1000 matrix, so 10,000,00 items in total.
from itertools import product
from time import perf_counter
# make a complete worksheet 100000 * 1000
start = perf_counter()
prod = product(range(1, 100000), range(1, 1000))
cells = {(x,y):x for x,y in prod}
print(len(cells))
print(f"Create time {perf_counter()-start:.2f}s")
clock = perf_counter()
# remove everything above row 50,000
keys = product(range(50000, 100000), range(1, 100))
# for x,y in keys:
# del cells[x, y]
for n in map(cells.pop, keys):
pass
print(len(cells))
stop = perf_counter()
print(f"Removal time {stop-clock:.2f}s")
10 million items or more is not unusual in some settings. Comparing the two methods on my local machine I see a slight improvement when using map and pop, presumably because of fewer function calls, but both take around 2.5s on my machine. But this pales in comparison to the time required to create the dictionary in the first place (55s), or including checks within the loop. If this is likely then its best to create a set that is a intersection of the dictionary keys and your filter:
keys = cells.keys() & keys
In summary: del is already heavily optimised, so don't worry about using it.
Another map() way to remove list of keys from dictionary
and avoid raising KeyError exception
dic = {
'key1': 1,
'key2': 2,
'key3': 3,
'key4': 4,
'key5': 5,
}
keys_to_remove = ['key_not_exist', 'key1', 'key2', 'key3']
k = list(map(dic.pop, keys_to_remove, keys_to_remove))
print('k=', k)
print('dic after = \n', dic)
**this will produce output**
k= ['key_not_exist', 1, 2, 3]
dic after = {'key4': 4, 'key5': 5}
Duplicate keys_to_remove is artificial, it needs to supply defaults values for dict.pop() function.
You can add here any array with len_ = len(key_to_remove)
For example
dic = {
'key1': 1,
'key2': 2,
'key3': 3,
'key4': 4,
'key5': 5,
}
keys_to_remove = ['key_not_exist', 'key1', 'key2', 'key3']
k = list(map(dic.pop, keys_to_remove, np.zeros(len(keys_to_remove))))
print('k=', k)
print('dic after = ', dic)
** will produce output **
k= [0.0, 1, 2, 3]
dic after = {'key4': 4, 'key5': 5}
def delete_keys_from_dict(dictionary, keys):
"""
Deletes the unwanted keys in the dictionary
:param dictionary: dict
:param keys: list of keys
:return: dict (modified)
"""
from collections.abc import MutableMapping
keys_set = set(keys)
modified_dict = {}
for key, value in dictionary.items():
if key not in keys_set:
if isinstance(value, list):
modified_dict[key] = list()
for x in value:
if isinstance(x, MutableMapping):
modified_dict[key].append(delete_keys_from_dict(x, keys_set))
else:
modified_dict[key].append(x)
elif isinstance(value, MutableMapping):
modified_dict[key] = delete_keys_from_dict(value, keys_set)
else:
modified_dict[key] = value
return modified_dict
_d = {'a': 1245, 'b': 1234325, 'c': {'a': 1245, 'b': 1234325}, 'd': 98765,
'e': [{'a': 1245, 'b': 1234325},
{'a': 1245, 'b': 1234325},
{'t': 767}]}
_output = delete_keys_from_dict(_d, ['a', 'b'])
_expected = {'c': {}, 'd': 98765, 'e': [{}, {}, {'t': 767}]}
print(_expected)
print(_output)
I'm late to this discussion but for anyone else. A solution may be to create a list of keys as such.
k = ['a','b','c','d']
Then use pop() in a list comprehension, or for loop, to iterate over the keys and pop one at a time as such.
new_dictionary = [dictionary.pop(x, 'n/a') for x in k]
The 'n/a' is in case the key does not exist, a default value needs to be returned.
If I have these two lists:
la = [1, 2, 3]
lb = [4, 5, 6]
I can iterate over them as follows:
for i in range(min(len(la), len(lb))):
print la[i], lb[i]
Or more pythonically
for a, b in zip(la, lb):
print a, b
What if I have two dictionaries?
da = {'a': 1, 'b': 2, 'c': 3}
db = {'a': 4, 'b': 5, 'c': 6}
Again, I can iterate manually:
for key in set(da.keys()) & set(db.keys()):
print key, da[key], db[key]
Is there some builtin method that allows me to iterate as follows?
for key, value_a, value_b in common_entries(da, db):
print key, value_a, value_b
There is no built-in function or method that can do this. However, you could easily define your own.
def common_entries(*dcts):
if not dcts:
return
for i in set(dcts[0]).intersection(*dcts[1:]):
yield (i,) + tuple(d[i] for d in dcts)
This builds on the "manual method" you provide, but, like zip, can be used for any number of dictionaries.
>>> da = {'a': 1, 'b': 2, 'c': 3}
>>> db = {'a': 4, 'b': 5, 'c': 6}
>>> list(common_entries(da, db))
[('c', 3, 6), ('b', 2, 5), ('a', 1, 4)]
When only one dictionary is provided as an argument, it essentially returns dct.items().
>>> list(common_entries(da))
[('c', 3), ('b', 2), ('a', 1)]
With no dictionaries, it returns an empty generator (just like zip())
>>> list(common_entries())
[]
The object returned by dict.keys() (called a dictionary key view) acts like a set object, so you can just take the intersection of the keys:
da = {'a': 1, 'b': 2, 'c': 3, 'e': 7}
db = {'a': 4, 'b': 5, 'c': 6, 'd': 9}
common_keys = da.keys() & db.keys()
for k in common_keys:
print(k, da[k], db[k])
On Python 2 you'll need to convert the keys to sets yourself:
common_keys = set(da) & set(db)
for k in common_keys:
print k, da[k], db[k]
Dictionary key views are already set-like in Python 3. You can remove set():
for key in da.keys() & db.keys():
print(key, da[key], db[key])
In Python 2:
for key in da.viewkeys() & db.viewkeys():
print key, da[key], db[key]
In case if someone is looking for generalized solution:
import operator
from functools import reduce
def zip_mappings(*mappings):
keys_sets = map(set, mappings)
common_keys = reduce(set.intersection, keys_sets)
for key in common_keys:
yield (key,) + tuple(map(operator.itemgetter(key), mappings))
or if you like to separate key from values and use syntax like
for key, (values, ...) in zip_mappings(...):
...
we can replace last line with
yield key, tuple(map(operator.itemgetter(key), mappings))
Tests
from collections import Counter
counter = Counter('abra')
other_counter = Counter('kadabra')
last_counter = Counter('abbreviation')
for (character,
frequency, other_frequency, last_frequency) in zip_mappings(counter,
other_counter,
last_counter):
print('character "{}" has next frequencies: {}, {}, {}'
.format(character,
frequency,
other_frequency,
last_frequency))
gives us
character "a" has next frequencies: 2, 3, 2
character "r" has next frequencies: 1, 1, 1
character "b" has next frequencies: 1, 1, 2
(tested on Python 2.7.12 & Python 3.5.2)
Python3: How about the following?
da = {'A': 1, 'b': 2, 'c': 3}
db = {'B': 4, 'b': 5, 'c': 6}
for key, (value_a, value_b) in {k:(da[k],db[k]) for k in set(da)&set(db)}.items():
print(key, value_a, value_b)
The above snippet prints values of common keys ('b' and 'c') and discards the keys which don't match ('A' and 'B').
In order to include all keys into the output we could use a slightly modified comprehension: {k:(da.get(k),db.get(k)) for k in set(da)|set(db)}.
For example I have two dicts:
Dict A: {'a': 1, 'b': 2, 'c': 3}
Dict B: {'b': 3, 'c': 4, 'd': 5}
I need a pythonic way of 'combining' two dicts such that the result is:
{'a': 1, 'b': 5, 'c': 7, 'd': 5}
That is to say: if a key appears in both dicts, add their values, if it appears in only one dict, keep its value.
Use collections.Counter:
>>> from collections import Counter
>>> A = Counter({'a':1, 'b':2, 'c':3})
>>> B = Counter({'b':3, 'c':4, 'd':5})
>>> A + B
Counter({'c': 7, 'b': 5, 'd': 5, 'a': 1})
Counters are basically a subclass of dict, so you can still do everything else with them you'd normally do with that type, such as iterate over their keys and values.
A more generic solution, which works for non-numeric values as well:
a = {'a': 'foo', 'b':'bar', 'c': 'baz'}
b = {'a': 'spam', 'c':'ham', 'x': 'blah'}
r = dict(a.items() + b.items() +
[(k, a[k] + b[k]) for k in set(b) & set(a)])
or even more generic:
def combine_dicts(a, b, op=operator.add):
return dict(a.items() + b.items() +
[(k, op(a[k], b[k])) for k in set(b) & set(a)])
For example:
>>> a = {'a': 2, 'b':3, 'c':4}
>>> b = {'a': 5, 'c':6, 'x':7}
>>> import operator
>>> print combine_dicts(a, b, operator.mul)
{'a': 10, 'x': 7, 'c': 24, 'b': 3}
>>> A = {'a':1, 'b':2, 'c':3}
>>> B = {'b':3, 'c':4, 'd':5}
>>> c = {x: A.get(x, 0) + B.get(x, 0) for x in set(A).union(B)}
>>> print(c)
{'a': 1, 'c': 7, 'b': 5, 'd': 5}
Intro:
There are the (probably) best solutions. But you have to know it and remember it and sometimes you have to hope that your Python version isn't too old or whatever the issue could be.
Then there are the most 'hacky' solutions. They are great and short but sometimes are hard to understand, to read and to remember.
There is, though, an alternative which is to to try to reinvent the wheel.
- Why reinventing the wheel?
- Generally because it's a really good way to learn (and sometimes just because the already-existing tool doesn't do exactly what you would like and/or the way you would like it) and the easiest way if you don't know or don't remember the perfect tool for your problem.
So, I propose to reinvent the wheel of the Counter class from the collections module (partially at least):
class MyDict(dict):
def __add__(self, oth):
r = self.copy()
try:
for key, val in oth.items():
if key in r:
r[key] += val # You can custom it here
else:
r[key] = val
except AttributeError: # In case oth isn't a dict
return NotImplemented # The convention when a case isn't handled
return r
a = MyDict({'a':1, 'b':2, 'c':3})
b = MyDict({'b':3, 'c':4, 'd':5})
print(a+b) # Output {'a':1, 'b': 5, 'c': 7, 'd': 5}
There would probably others way to implement that and there are already tools to do that but it's always nice to visualize how things would basically works.
Definitely summing the Counter()s is the most pythonic way to go in such cases but only if it results in a positive value. Here is an example and as you can see there is no c in result after negating the c's value in B dictionary.
In [1]: from collections import Counter
In [2]: A = Counter({'a':1, 'b':2, 'c':3})
In [3]: B = Counter({'b':3, 'c':-4, 'd':5})
In [4]: A + B
Out[4]: Counter({'d': 5, 'b': 5, 'a': 1})
That's because Counters were primarily designed to work with positive integers to represent running counts (negative count is meaningless). But to help with those use cases,python documents the minimum range and type restrictions as follows:
The Counter class itself is a dictionary
subclass with no restrictions on its keys and values. The values are
intended to be numbers representing counts, but you could store
anything in the value field.
The most_common() method requires only
that the values be orderable.
For in-place operations such as c[key]
+= 1, the value type need only support addition and subtraction. So fractions, floats, and decimals would work and negative values are
supported. The same is also true for update() and subtract() which
allow negative and zero values for both inputs and outputs.
The multiset methods are designed only for use cases with positive values.
The inputs may be negative or zero, but only outputs with positive
values are created. There are no type restrictions, but the value type
needs to support addition, subtraction, and comparison.
The elements() method requires integer counts. It ignores zero and negative counts.
So for getting around that problem after summing your Counter you can use Counter.update in order to get the desire output. It works like dict.update() but adds counts instead of replacing them.
In [24]: A.update(B)
In [25]: A
Out[25]: Counter({'d': 5, 'b': 5, 'a': 1, 'c': -1})
myDict = {}
for k in itertools.chain(A.keys(), B.keys()):
myDict[k] = A.get(k, 0)+B.get(k, 0)
The one with no extra imports!
Their is a pythonic standard called EAFP(Easier to Ask for Forgiveness than Permission). Below code is based on that python standard.
# The A and B dictionaries
A = {'a': 1, 'b': 2, 'c': 3}
B = {'b': 3, 'c': 4, 'd': 5}
# The final dictionary. Will contain the final outputs.
newdict = {}
# Make sure every key of A and B get into the final dictionary 'newdict'.
newdict.update(A)
newdict.update(B)
# Iterate through each key of A.
for i in A.keys():
# If same key exist on B, its values from A and B will add together and
# get included in the final dictionary 'newdict'.
try:
addition = A[i] + B[i]
newdict[i] = addition
# If current key does not exist in dictionary B, it will give a KeyError,
# catch it and continue looping.
except KeyError:
continue
EDIT: thanks to jerzyk for his improvement suggestions.
import itertools
import collections
dictA = {'a':1, 'b':2, 'c':3}
dictB = {'b':3, 'c':4, 'd':5}
new_dict = collections.defaultdict(int)
# use dict.items() instead of dict.iteritems() for Python3
for k, v in itertools.chain(dictA.iteritems(), dictB.iteritems()):
new_dict[k] += v
print dict(new_dict)
# OUTPUT
{'a': 1, 'c': 7, 'b': 5, 'd': 5}
OR
Alternative you can use Counter as #Martijn has mentioned above.
For a more generic and extensible way check mergedict. It uses singledispatch and can merge values based on its types.
Example:
from mergedict import MergeDict
class SumDict(MergeDict):
#MergeDict.dispatch(int)
def merge_int(this, other):
return this + other
d2 = SumDict({'a': 1, 'b': 'one'})
d2.merge({'a':2, 'b': 'two'})
assert d2 == {'a': 3, 'b': 'two'}
From python 3.5: merging and summing
Thanks to #tokeinizer_fsj that told me in a comment that I didn't get completely the meaning of the question (I thought that add meant just adding keys that eventually where different in the two dictinaries and, instead, i meant that the common key values should be summed). So I added that loop before the merging, so that the second dictionary contains the sum of the common keys. The last dictionary will be the one whose values will last in the new dictionary that is the result of the merging of the two, so I thing the problem is solved. The solution is valid from python 3.5 and following versions.
a = {
"a": 1,
"b": 2,
"c": 3
}
b = {
"a": 2,
"b": 3,
"d": 5
}
# Python 3.5
for key in b:
if key in a:
b[key] = b[key] + a[key]
c = {**a, **b}
print(c)
>>> c
{'a': 3, 'b': 5, 'c': 3, 'd': 5}
Reusable code
a = {'a': 1, 'b': 2, 'c': 3}
b = {'b': 3, 'c': 4, 'd': 5}
def mergsum(a, b):
for k in b:
if k in a:
b[k] = b[k] + a[k]
c = {**a, **b}
return c
print(mergsum(a, b))
Additionally, please note a.update( b ) is 2x faster than a + b
from collections import Counter
a = Counter({'menu': 20, 'good': 15, 'happy': 10, 'bar': 5})
b = Counter({'menu': 1, 'good': 1, 'bar': 3})
%timeit a + b;
## 100000 loops, best of 3: 8.62 µs per loop
## The slowest run took 4.04 times longer than the fastest. This could mean that an intermediate result is being cached.
%timeit a.update(b)
## 100000 loops, best of 3: 4.51 µs per loop
One line solution is to use dictionary comprehension.
C = { k: A.get(k,0) + B.get(k,0) for k in list(B.keys()) + list(A.keys()) }
def merge_with(f, xs, ys):
xs = a_copy_of(xs) # dict(xs), maybe generalizable?
for (y, v) in ys.iteritems():
xs[y] = v if y not in xs else f(xs[x], v)
merge_with((lambda x, y: x + y), A, B)
You could easily generalize this:
def merge_dicts(f, *dicts):
result = {}
for d in dicts:
for (k, v) in d.iteritems():
result[k] = v if k not in result else f(result[k], v)
Then it can take any number of dicts.
This is a simple solution for merging two dictionaries where += can be applied to the values, it has to iterate over a dictionary only once
a = {'a':1, 'b':2, 'c':3}
dicts = [{'b':3, 'c':4, 'd':5},
{'c':9, 'a':9, 'd':9}]
def merge_dicts(merged,mergedfrom):
for k,v in mergedfrom.items():
if k in merged:
merged[k] += v
else:
merged[k] = v
return merged
for dct in dicts:
a = merge_dicts(a,dct)
print (a)
#{'c': 16, 'b': 5, 'd': 14, 'a': 10}
Here's yet another option using dictionary comprehensions combined with the behavior of dict():
dict3 = dict(dict1, **{ k: v + dict1.get(k, 0) for k, v in dict2.items() })
# {'a': 4, 'b': 2, 'c': 7, 'g': 1}
From https://docs.python.org/3/library/stdtypes.html#dict:
https://docs.python.org/3/library/stdtypes.html#dict
and also
If keyword arguments are given, the keyword arguments and their values are added to the dictionary created from the positional argument.
The dict comprehension
**{ k: v + dict1.get(v, 0), v in dict2.items() }
handles adding dict1[1] to v. We don't need an explicit if here because the default value for our dict1.get can be set to 0 instead.
This solution is easy to use, it is used as a normal dictionary, but you can use the sum function.
class SumDict(dict):
def __add__(self, y):
return {x: self.get(x, 0) + y.get(x, 0) for x in set(self).union(y)}
A = SumDict({'a': 1, 'c': 2})
B = SumDict({'b': 3, 'c': 4}) # Also works: B = {'b': 3, 'c': 4}
print(A + B) # OUTPUT {'a': 1, 'b': 3, 'c': 6}
The above solutions are great for the scenario where you have a small number of Counters. If you have a big list of them though, something like this is much nicer:
from collections import Counter
A = Counter({'a':1, 'b':2, 'c':3})
B = Counter({'b':3, 'c':4, 'd':5})
C = Counter({'a': 5, 'e':3})
list_of_counts = [A, B, C]
total = sum(list_of_counts, Counter())
print(total)
# Counter({'c': 7, 'a': 6, 'b': 5, 'd': 5, 'e': 3})
The above solution is essentially summing the Counters by:
total = Counter()
for count in list_of_counts:
total += count
print(total)
# Counter({'c': 7, 'a': 6, 'b': 5, 'd': 5, 'e': 3})
This does the same thing but I think it always helps to see what it is effectively doing underneath.
What about:
def dict_merge_and_sum( d1, d2 ):
ret = d1
ret.update({ k:v + d2[k] for k,v in d1.items() if k in d2 })
ret.update({ k:v for k,v in d2.items() if k not in d1 })
return ret
A = {'a': 1, 'b': 2, 'c': 3}
B = {'b': 3, 'c': 4, 'd': 5}
print( dict_merge_and_sum( A, B ) )
Output:
{'d': 5, 'a': 1, 'c': 7, 'b': 5}
More conventional way to combine two dict. Using modules and tools are good but understanding the logic behind it will help in case you don't remember the tools.
Program to combine two dictionary adding values for common keys.
def combine_dict(d1,d2):
for key,value in d1.items():
if key in d2:
d2[key] += value
else:
d2[key] = value
return d2
combine_dict({'a':1, 'b':2, 'c':3},{'b':3, 'c':4, 'd':5})
output == {'b': 5, 'c': 7, 'd': 5, 'a': 1}
Here's a very general solution. You can deal with any number of dict + keys that are only in some dict + easily use any aggregation function you want:
def aggregate_dicts(dicts, operation=sum):
"""Aggregate a sequence of dictionaries using `operation`."""
all_keys = set().union(*[el.keys() for el in dicts])
return {k: operation([dic.get(k, None) for dic in dicts]) for k in all_keys}
example:
dicts_same_keys = [{'x': 0, 'y': 1}, {'x': 1, 'y': 2}, {'x': 2, 'y': 3}]
aggregate_dicts(dicts_same_keys, operation=sum)
#{'x': 3, 'y': 6}
example non-identical keys and generic aggregation:
dicts_diff_keys = [{'x': 0, 'y': 1}, {'x': 1, 'y': 2}, {'x': 2, 'y': 3, 'c': 4}]
def mean_no_none(l):
l_no_none = [el for el in l if el is not None]
return sum(l_no_none) / len(l_no_none)
aggregate_dicts(dicts_diff_keys, operation=mean_no_none)
# {'x': 1.0, 'c': 4.0, 'y': 2.0}
dict1 = {'a':1, 'b':2, 'c':3}
dict2 = {'a':3, 'g':1, 'c':4}
dict3 = {} # will store new values
for x in dict1:
if x in dict2: #sum values with same key
dict3[x] = dict1[x] +dict2[x]
else: #add the values from x to dict1
dict3[x] = dict1[x]
#search for new values not in a
for x in dict2:
if x not in dict1:
dict3[x] = dict2[x]
print(dict3) # {'a': 4, 'b': 2, 'c': 7, 'g': 1}
Merging three dicts a,b,c in a single line without any other modules or libs
If we have the three dicts
a = {"a":9}
b = {"b":7}
c = {'b': 2, 'd': 90}
Merge all with a single line and return a dict object using
c = dict(a.items() + b.items() + c.items())
Returning
{'a': 9, 'b': 2, 'd': 90}