I want to get all combinations of the values in a dictionary as multiple dictionaries (each containing every key of the original but only one value of the original values). Say I want to parametrize a function call with:
kwargs = {'a': [1, 2, 3], 'b': [1, 2, 3]}
How do I get a list of all the combinations like this:
combinations = [{'a': 1, 'b': 1}, {'a': 1, 'b': 2}, {'a': 1, 'b': 3},
{'a': 2, 'b': 1}, {'a': 2, 'b': 2}, {'a': 2, 'b': 3},
{'a': 3, 'b': 1}, {'a': 3, 'b': 2}, {'a': 3, 'b': 3}]
There can be an arbitary amount of keys in the original kwargs and each value is garantueed to be an iterable but the number of values is not fixed.
If possible: the final combinations should be a generator (not a list).
You can flatten the kwargs to something like this
>>> kwargs = {'a': [1, 2, 3], 'b': [1, 2, 3]}
>>> flat = [[(k, v) for v in vs] for k, vs in kwargs.items()]
>>> flat
[[('b', 1), ('b', 2), ('b', 3)], [('a', 1), ('a', 2), ('a', 3)]]
Then, you can use itertools.product like this
>>> from itertools import product
>>> [dict(items) for items in product(*flat)]
[{'a': 1, 'b': 1},
{'a': 2, 'b': 1},
{'a': 3, 'b': 1},
{'a': 1, 'b': 2},
{'a': 2, 'b': 2},
{'a': 3, 'b': 2},
{'a': 1, 'b': 3},
{'a': 2, 'b': 3},
{'a': 3, 'b': 3}]
itertools.product actually returns an iterator. So you can get the values on demand and build your dictionaries. Or you can use map, which also returns an iterator.
>>> for item in map(dict, product(*flat)):
... print(item)
...
...
{'b': 1, 'a': 1}
{'b': 1, 'a': 2}
{'b': 1, 'a': 3}
{'b': 2, 'a': 1}
{'b': 2, 'a': 2}
{'b': 2, 'a': 3}
{'b': 3, 'a': 1}
{'b': 3, 'a': 2}
{'b': 3, 'a': 3}
Just another way, building the value tuples first and then combining with keys afterwards (pretty much the opposite of #thefourtheye's way :-).
>>> combinations = (dict(zip(kwargs, vs)) for vs in product(*kwargs.values()))
>>> for c in combinations:
print(c)
{'a': 1, 'b': 1}
{'a': 1, 'b': 2}
{'a': 1, 'b': 3}
{'a': 2, 'b': 1}
{'a': 2, 'b': 2}
{'a': 2, 'b': 3}
{'a': 3, 'b': 1}
{'a': 3, 'b': 2}
{'a': 3, 'b': 3}
Related
I have a dict I'm trying to loop through which stores integer values:
myDict = {"a":1,"b":2,"c":3}
And I'm trying to loop through every combination of these integers from 1 to their value. For example:
{"a":1,"b":1,"c":1}
{"a":1,"b":1,"c":2}
{"a":1,"b":1,"c":3}
{"a":1,"b":2,"c":1}
{"a":1,"b":2,"c":2}
And so on...
Ending at their max values at the start while going through all the combinations.
Is there an elegant way to this that applies to any size dictionary? :
myDict = {"a":1,"b":2,"c":3,"d":5}
I'm currently doing this with nested if statements that edit a clone dictionary, so any solution setting dict values would work :) :
copyDict["c"] = 2
Yes, this will do it, for any size input (that fits in memory, of course):
import itertools
myDict = {"a":1,"b":2,"c":3,"d":5}
keys = list(myDict.keys())
ranges = tuple(range(1,k+1) for k in myDict.values())
outs = [dict(zip(keys,s)) for s in itertools.product(*ranges)]
print(outs)
You can actually replace *ranges with the *(range(1,k+1) for k in myDict.values()) and reduce it by one line, but it's not easier to read.
Another way with itertools.product:
>>> [dict(zip(myDict.keys(), p)) for p in itertools.product(myDict.values(), repeat=len(myDict))]
[{'a': 1, 'b': 1, 'c': 1},
{'a': 1, 'b': 1, 'c': 2},
{'a': 1, 'b': 1, 'c': 3},
{'a': 1, 'b': 2, 'c': 1},
{'a': 1, 'b': 2, 'c': 2},
{'a': 1, 'b': 2, 'c': 3},
{'a': 1, 'b': 3, 'c': 1},
{'a': 1, 'b': 3, 'c': 2},
{'a': 1, 'b': 3, 'c': 3},
{'a': 2, 'b': 1, 'c': 1},
{'a': 2, 'b': 1, 'c': 2},
{'a': 2, 'b': 1, 'c': 3},
{'a': 2, 'b': 2, 'c': 1},
{'a': 2, 'b': 2, 'c': 2},
{'a': 2, 'b': 2, 'c': 3},
{'a': 2, 'b': 3, 'c': 1},
{'a': 2, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 3},
{'a': 3, 'b': 1, 'c': 1},
{'a': 3, 'b': 1, 'c': 2},
{'a': 3, 'b': 1, 'c': 3},
{'a': 3, 'b': 2, 'c': 1},
{'a': 3, 'b': 2, 'c': 2},
{'a': 3, 'b': 2, 'c': 3},
{'a': 3, 'b': 3, 'c': 1},
{'a': 3, 'b': 3, 'c': 2},
{'a': 3, 'b': 3, 'c': 3}]
My question is somewhat similar to this question: https://codereview.stackexchange.com/questions/175079/removing-key-value-pairs-in-list-of-dicts. Essentially, I have a list of dictionaries, and I want to remove duplicates from the list based on the unique combination of two (or more) keys within each dictionary.
Suppose I have the following list of dictionaries:
some_list_of_dicts = [
{'a': 1, 'b': 1, 'c': 1, 'd': 2, 'e': 4},
{'a': 1, 'b': 1, 'c': 1, 'd': 5, 'e': 1},
{'a': 1, 'b': 1, 'c': 1, 'd': 7, 'e': 8},
{'a': 1, 'b': 1, 'c': 1, 'd': 9, 'e': 6},
{'a': 1, 'b': 1, 'c': 2, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 3, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 4, 'd': 2, 'e': 3}
]
And let's suppose the combination of a, b, and c have to be unique; any other values can be whatever they want, but the combination of these three must be unique to this list. I would want to take whichever unique combo of a, b, and c came first, keep that, and discard everything else where that combination is the same.
The new list, after running it through some remove_duplicates function would look like this:
new_list = [
{'a': 1, 'b': 1, 'c': 1, 'd': 2, 'e': 4},
{'a': 1, 'b': 1, 'c': 2, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 3, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 4, 'd': 2, 'e': 3}
]
I've only managed to come up with this:
def remove_duplicates(old_list):
uniqueness_check_list = []
new_list = []
for item in old_list:
# The unique combination is 'a', 'b', and 'c'
uniqueness_check = "{}{}{}".format(
item["a"], item["b"], item["c"]
)
if uniqueness_check not in uniqueness_check_list:
new_list.append(item)
uniqueness_check_list.append(uniqueness_check)
return new_list
But this doesn't feel very Pythonic. It also has the problem that I've hardcoded in the function which keys have to be unique; it would be better if I could specify that as an argument to the function itself, but again, not sure what's the most elegant way to do this.
You can use a dict comprehension to construct a dict from the list of dicts in the reversed order so that the values of the first of any unique combinations would take precedence. Use operator.itemgetter to get the unique keys as a tuple. Reverse again in the end for the original order:
from operator import itemgetter
list({itemgetter('a', 'b', 'c')(d): d for d in reversed(some_list_of_dicts)}.values())[::-1]
This returns:
[{'a': 1, 'b': 1, 'c': 1, 'd': 2, 'e': 4},
{'a': 1, 'b': 1, 'c': 2, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 3, 'd': 2, 'e': 3},
{'a': 1, 'b': 1, 'c': 4, 'd': 2, 'e': 3}]
With the help of a function to keep track of duplicates, you can use some list comprehension:
def remove_duplicates(old_list, cols=('a', 'b', 'c')):
duplicates = set()
def is_duplicate(item):
duplicate = item in duplicates
duplicates.add(item)
return duplicate
return [x for x in old_list if not is_duplicate(tuple([x[col] for col in cols]))]
To use:
>>> remove_duplicates(some_list_of_dicts)
[
{'a': 1, 'c': 1, 'b': 1, 'e': 4, 'd': 2},
{'a': 1, 'c': 2, 'b': 1, 'e': 3, 'd': 2},
{'a': 1, 'c': 3, 'b': 1, 'e': 3, 'd': 2},
{'a': 1, 'c': 4, 'b': 1, 'e': 3, 'd': 2}
]
You can also provide different columns to key on:
>>> remove_duplicates(some_list_of_dicts, cols=('a', 'd'))
[
{'a': 1, 'c': 1, 'b': 1, 'e': 4, 'd': 2},
{'a': 1, 'c': 1, 'b': 1, 'e': 1, 'd': 5},
{'a': 1, 'c': 1, 'b': 1, 'e': 8, 'd': 7},
{'a': 1, 'c': 1, 'b': 1, 'e': 6, 'd': 9}
]
I am trying to update values for a dict() key dynamically with a for loop.
def update_dict():
f = []
for i, j in enumerate(list_range):
test_dict.update({'a': i})
j['c'] = test_dict
print(j)
f.append(j)
print(f)
test_dict = dict({'a': 1})
list_range = [{'b': i} for i in range(0, 5)]
update_dict()
Even print(j) gives iterating value (0,1,2,3,4), somehow the last dict is getting overwritten all over the list and giving wrong output (4,4,4,4,4).
Expected Output,
[{'b': 0, 'c': {'a': 0}}, {'b': 1, 'c': {'a': 1}}, {'b': 2, 'c': {'a': 2}}, {'b': 3, 'c': {'a': 3}}, {'b': 4, 'c': {'a': 4}}]
Output obtained,
[{'b': 0, 'c': {'a': 4}}, {'b': 1, 'c': {'a': 4}}, {'b': 2, 'c': {'a': 4}}, {'b': 3, 'c': {'a': 4}}, {'b': 4, 'c': {'a': 4}}]
I need to understand how the dictionaries are getting overwritten and what could be the best solution to avoid this?
Thanks in advance!
P.S. : please avoid suggesting list or dict comprehension method as bare answer as i am aware of them and the only purpose of this question is to understand the wrong behavior of dict().
The reason of such behaviour is that all references in list points to the same dict. Line j['c'] = test_dict doesn't create copy of dictionary, but just make j['c'] refer to test_dict. To get expected result you need change this line to:
j['c'] = test_dict.copy(). It will make deep copy of test_dict and assign it to j['c'].
You try to add values to same dictionary every time in the loop and as loop progresses, you keep replacing the values.
You need to define dictionary in every iteration to create separate references of the dictionary:
def update_dict():
f = []
for i, j in enumerate(list_range):
test_dict = {'a': i}
j['c'] = test_dict
f.append(j)
print(f)
list_range = [{'b': i} for i in range(0, 5)]
update_dict()
# [{'b': 0, 'c': {'a': 0}},
# {'b': 1, 'c': {'a': 1}},
# {'b': 2, 'c': {'a': 2}},
# {'b': 3, 'c': {'a': 3}},
# {'b': 4, 'c': {'a': 4}}]
A simpler solution could be to iterate through list_range and create c using the values from b
lista = [{'b': i } for i in range(0, 5)]
for i in lista:
i['c'] = {'a': i['b']}
# [{'b': 0, 'c': {'a': 0}}, {'b': 1, 'c': {'a': 1}}, {'b': 2, 'c': {'a': 2}}, {'b': 3, 'c': {'a': 3}}, {'b': 4, 'c': {'a': 4}}]
def update_dict():
f = []
for i, j in enumerate(list_range):
j['c'] = {'a': i}
print(j)
f.append(j)
return f
list_range = [{'b': i} for i in range(0, 5)]
print(update_dict())
#output
{'b': 0, 'c': {'a': 0}}
{'b': 1, 'c': {'a': 1}}
{'b': 2, 'c': {'a': 2}}
{'b': 3, 'c': {'a': 3}}
{'b': 4, 'c': {'a': 4}}
This question already has answers here:
How do I merge two dictionaries in a single expression in Python?
(43 answers)
Closed 5 years ago.
I have two lists of dictionaries which I am trying to get the product of:
from itertools import product
list1 = [{'A': 1, 'B': 1}, {'A': 2, 'B': 2}, {'A': 2, 'B': 1}, {'A': 1, 'B': 2}]
list2 = [{'C': 1, 'D': 1}, {'C': 1, 'D': 2}]
for p in product(list1, list2):
print p
and this gives me the output:
({'A': 1, 'B': 1}, {'C': 1, 'D': 1})
({'A': 1, 'B': 1}, {'C': 1, 'D': 2})
({'A': 2, 'B': 2}, {'C': 1, 'D': 1})
({'A': 2, 'B': 2}, {'C': 1, 'D': 2})
({'A': 2, 'B': 1}, {'C': 1, 'D': 1})
({'A': 2, 'B': 1}, {'C': 1, 'D': 2})
({'A': 1, 'B': 2}, {'C': 1, 'D': 1})
({'A': 1, 'B': 2}, {'C': 1, 'D': 2})
How would I flatten these so the output is a single dict rather than a tuple of dicts?:
{'A': 1, 'B': 1, 'C': 1, 'D': 1}
{'A': 1, 'B': 1, 'C': 1, 'D': 2}
{'A': 2, 'B': 2, 'C': 1, 'D': 1}
{'A': 2, 'B': 2, 'C': 1, 'D': 2}
{'A': 2, 'B': 1, 'C': 1, 'D': 1}
{'A': 2, 'B': 1, 'C': 1, 'D': 2}
{'A': 1, 'B': 2, 'C': 1, 'D': 1}
{'A': 1, 'B': 2, 'C': 1, 'D': 2}
Looks like you want to merge the dictionaries
for p1, p2 in product(list1, list2):
merged = {**p1, **p2}
print(merged)
In earlier versions of Python, you can't merge with this expression. Use p1.update(p2) instead.
I have the following dict:
my_dict = {'A': [1, 2], 'B': [1, 4]}
And I want to end up with a list of dicts like this:
[
{'A': 1, 'B': 1},
{'A': 1, 'B': 4},
{'A': 2, 'B': 1},
{'A': 2, 'B': 4}
]
So, I'm after the product of dict's lists, expressed as a list of dicts using the same keys as the incoming dict.
The closest I got was:
my_dict = {'A': [1, 2], 'B': [1, 4]}
it = []
for k in my_dict.keys():
current = my_dict.pop(k)
for i in current:
it.append({k2: i2 for k2, i2 in my_dict.iteritems()})
it[-1].update({k: i})
Which, apart from looking hideous, doesn't give me what I want:
[
{'A': 1, 'B': [1, 4]},
{'A': 2, 'B': [1, 4]},
{'B': 1},
{'B': 4}
]
If anyone feels like solving a riddle, I'd love to see how you'd approach it.
You can use itertools.product for this, i.e calculate cartesian product of the value and then simply zip each of the them with the keys from the dictionary. Note that ordering of a dict's keys() and corresponding values() remains same if it is not modified in-between hence ordering won't be an issue here:
>>> from itertools import product
>>> my_dict = {'A': [1, 2], 'B': [1, 4]}
>>> keys = list(my_dict)
>>> [dict(zip(keys, p)) for p in product(*my_dict.values())]
[{'A': 1, 'B': 1}, {'A': 1, 'B': 4}, {'A': 2, 'B': 1}, {'A': 2, 'B': 4}]
you can use itertools.product function within a list comprehension :
>>> from itertools import product
>>> [dict(i) for i in product(*[[(i,k) for k in j] for i,j in my_dict.items()])]
[{'A': 1, 'B': 1}, {'A': 1, 'B': 4}, {'A': 2, 'B': 1}, {'A': 2, 'B': 4}]
You can get the pairs contain your key and values with the following list comprehension :
[(i,k) for k in j] for i,j in my_dict.items()]
[[('A', 1), ('A', 2)], [('B', 1), ('B', 4)]]
Then you can use product to calculate the product of the preceding lists and then convert them to dictionary with dict function.
With itertools:
>>> from itertools import product
>>> my_dict = {'A': [1, 2], 'B': [1, 4]}
>>> keys, items = zip(*my_dict.items())
>>> [dict(zip(keys, x)) for x in product(*items)]
[{'A': 1, 'B': 1}, {'A': 1, 'B': 4}, {'A': 2, 'B': 1}, {'A': 2, 'B': 4}]
Try this:
from itertools import product
def dict_product(values, first, second):
return [
{first: first_value, second: second_value}
for first_value, second_value in product(values[first], values[second])
]
This is the result:
>>> dict_product({'A': [1, 2], 'B': [1, 4]}, 'A', 'B')
[{'A': 1, 'B': 1}, {'A': 1, 'B': 4}, {'A': 2, 'B': 1}, {'A': 2, 'B': 4}]