Develop Reference

Removing keys from a list of dicts

I have a list:
[{'key':'key1', 'version':'1'}, {'key':'key2', 'version':'2'}, {'key':'key3', 'version':'3'}]
I want to remove all other keys from the dictionaries in the list and only to have 'key' and its value so the list would look like this:
[{'key': 'key1'}, {'key': 'key2'}, {'key': 'key3'}]
How can I extract it?

Here is an easy solution. This uses the .pop() method to remove the key that you do not want. -
lst = [{'key':'key1','version':'1'},{'key':'key2','version':'2'},{'key':'key3','version':'3'}]
for dicts in lst:
dicts.pop('version')
print(lst)
Result:
[{'key': 'key1'}, {'key': 'key2'}, {'key': 'key3'}]
This removes all version keys. Or you could use this method to remove everything except desired key:
new_l = []
for d in lst:
for key in d:
if key == 'key':
new_l.append({key: d[key]})
print(new_l)
Or even simpler, you can use a list and dictionary comprehension to solve this. I recommend you to do this because it is more readable and efficient.:
new_l = [{key: d[key] for key in d if key == 'key'} for d in lst]
print(new_l)

You can try this
in just one line!
yourList = [{'key':'key1','version':'1'},{'key':'key2','version':'2'},{'key':'key3','version':'3'}]
resultList = [{'key':dic['key']} for dic in yourList if 'key' in dic]
print(resultList)

Try this:
lst = [{'key':'key1','version':'1'},{'key':'key2','version':'2'},{'key':'key3','version':'3'}]
[item.pop('version') for item in lst]
Or, removing all possible keys that does not match 'key':
lst = [{'key':'key1','version':'1'},{'key':'key2','version':'2'},{'key':'key3','version':'3'}]
for item in lst:
for key in list(item.keys()):
item.pop(key) if key!='key' else None

You can try this:
l1 = [{'key':'key1','version':'1'},{'key':'key2','version':'2'},{'key':'key3','version':'3'}]
li2 = [{'key': j[x]} for j in l1 for x in j if x=='key']
print(li2)

Create dict from list of lists [duplicate]

I want to combine these:
keys = ['name', 'age', 'food']
values = ['Monty', 42, 'spam']
Into a single dictionary:
{'name': 'Monty', 'age': 42, 'food': 'spam'}

Like this:
keys = ['a', 'b', 'c']
values = [1, 2, 3]
dictionary = dict(zip(keys, values))
print(dictionary) # {'a': 1, 'b': 2, 'c': 3}
Voila :-) The pairwise dict constructor and zip function are awesomely useful.

Imagine that you have:
keys = ('name', 'age', 'food')
values = ('Monty', 42, 'spam')
What is the simplest way to produce the following dictionary ?
dict = {'name' : 'Monty', 'age' : 42, 'food' : 'spam'}
Most performant, dict constructor with zip
new_dict = dict(zip(keys, values))
In Python 3, zip now returns a lazy iterator, and this is now the most performant approach.
dict(zip(keys, values)) does require the one-time global lookup each for dict and zip, but it doesn't form any unnecessary intermediate data-structures or have to deal with local lookups in function application.
Runner-up, dict comprehension:
A close runner-up to using the dict constructor is to use the native syntax of a dict comprehension (not a list comprehension, as others have mistakenly put it):
new_dict = {k: v for k, v in zip(keys, values)}
Choose this when you need to map or filter based on the keys or value.
In Python 2, zip returns a list, to avoid creating an unnecessary list, use izip instead (aliased to zip can reduce code changes when you move to Python 3).
from itertools import izip as zip
So that is still (2.7):
new_dict = {k: v for k, v in zip(keys, values)}
Python 2, ideal for <= 2.6
izip from itertools becomes zip in Python 3. izip is better than zip for Python 2 (because it avoids the unnecessary list creation), and ideal for 2.6 or below:
from itertools import izip
new_dict = dict(izip(keys, values))
Result for all cases:
In all cases:
>>> new_dict
{'age': 42, 'name': 'Monty', 'food': 'spam'}
Explanation:
If we look at the help on dict we see that it takes a variety of forms of arguments:
>>> help(dict)
class dict(object)
| dict() -> new empty dictionary
| dict(mapping) -> new dictionary initialized from a mapping object's
| (key, value) pairs
| dict(iterable) -> new dictionary initialized as if via:
| d = {}
| for k, v in iterable:
| d[k] = v
| dict(**kwargs) -> new dictionary initialized with the name=value pairs
| in the keyword argument list. For example: dict(one=1, two=2)
The optimal approach is to use an iterable while avoiding creating unnecessary data structures. In Python 2, zip creates an unnecessary list:
>>> zip(keys, values)
[('name', 'Monty'), ('age', 42), ('food', 'spam')]
In Python 3, the equivalent would be:
>>> list(zip(keys, values))
[('name', 'Monty'), ('age', 42), ('food', 'spam')]
and Python 3's zip merely creates an iterable object:
>>> zip(keys, values)
<zip object at 0x7f0e2ad029c8>
Since we want to avoid creating unnecessary data structures, we usually want to avoid Python 2's zip (since it creates an unnecessary list).
Less performant alternatives:
This is a generator expression being passed to the dict constructor:
generator_expression = ((k, v) for k, v in zip(keys, values))
dict(generator_expression)
or equivalently:
dict((k, v) for k, v in zip(keys, values))
And this is a list comprehension being passed to the dict constructor:
dict([(k, v) for k, v in zip(keys, values)])
In the first two cases, an extra layer of non-operative (thus unnecessary) computation is placed over the zip iterable, and in the case of the list comprehension, an extra list is unnecessarily created. I would expect all of them to be less performant, and certainly not more-so.
Performance review:
In 64 bit Python 3.8.2 provided by Nix, on Ubuntu 16.04, ordered from fastest to slowest:
>>> min(timeit.repeat(lambda: dict(zip(keys, values))))
0.6695233230129816
>>> min(timeit.repeat(lambda: {k: v for k, v in zip(keys, values)}))
0.6941362579818815
>>> min(timeit.repeat(lambda: {keys[i]: values[i] for i in range(len(keys))}))
0.8782548159942962
>>>
>>> min(timeit.repeat(lambda: dict([(k, v) for k, v in zip(keys, values)])))
1.077607496001292
>>> min(timeit.repeat(lambda: dict((k, v) for k, v in zip(keys, values))))
1.1840861019445583
dict(zip(keys, values)) wins even with small sets of keys and values, but for larger sets, the differences in performance will become greater.
A commenter said:
min seems like a bad way to compare performance. Surely mean and/or max would be much more useful indicators for real usage.
We use min because these algorithms are deterministic. We want to know the performance of the algorithms under the best conditions possible.
If the operating system hangs for any reason, it has nothing to do with what we're trying to compare, so we need to exclude those kinds of results from our analysis.
If we used mean, those kinds of events would skew our results greatly, and if we used max we will only get the most extreme result - the one most likely affected by such an event.
A commenter also says:
In python 3.6.8, using mean values, the dict comprehension is indeed still faster, by about 30% for these small lists. For larger lists (10k random numbers), the dict call is about 10% faster.
I presume we mean dict(zip(... with 10k random numbers. That does sound like a fairly unusual use case. It does makes sense that the most direct calls would dominate in large datasets, and I wouldn't be surprised if OS hangs are dominating given how long it would take to run that test, further skewing your numbers. And if you use mean or max I would consider your results meaningless.
Let's use a more realistic size on our top examples:
import numpy
import timeit
l1 = list(numpy.random.random(100))
l2 = list(numpy.random.random(100))
And we see here that dict(zip(... does indeed run faster for larger datasets by about 20%.
>>> min(timeit.repeat(lambda: {k: v for k, v in zip(l1, l2)}))
9.698965263989521
>>> min(timeit.repeat(lambda: dict(zip(l1, l2))))
7.9965161079890095

Try this:
>>> import itertools
>>> keys = ('name', 'age', 'food')
>>> values = ('Monty', 42, 'spam')
>>> adict = dict(itertools.izip(keys,values))
>>> adict
{'food': 'spam', 'age': 42, 'name': 'Monty'}
In Python 2, it's also more economical in memory consumption compared to zip.

keys = ('name', 'age', 'food')
values = ('Monty', 42, 'spam')
out = dict(zip(keys, values))
Output:
{'food': 'spam', 'age': 42, 'name': 'Monty'}

You can also use dictionary comprehensions in Python ≥ 2.7:
>>> keys = ('name', 'age', 'food')
>>> values = ('Monty', 42, 'spam')
>>> {k: v for k, v in zip(keys, values)}
{'food': 'spam', 'age': 42, 'name': 'Monty'}

A more natural way is to use dictionary comprehension
keys = ('name', 'age', 'food')
values = ('Monty', 42, 'spam')
dict = {keys[i]: values[i] for i in range(len(keys))}

If you need to transform keys or values before creating a dictionary then a generator expression could be used. Example:
>>> adict = dict((str(k), v) for k, v in zip(['a', 1, 'b'], [2, 'c', 3]))
Take a look Code Like a Pythonista: Idiomatic Python.

with Python 3.x, goes for dict comprehensions
keys = ('name', 'age', 'food')
values = ('Monty', 42, 'spam')
dic = {k:v for k,v in zip(keys, values)}
print(dic)
More on dict comprehensions here, an example is there:
>>> print {i : chr(65+i) for i in range(4)}
{0 : 'A', 1 : 'B', 2 : 'C', 3 : 'D'}

For those who need simple code and aren’t familiar with zip:
List1 = ['This', 'is', 'a', 'list']
List2 = ['Put', 'this', 'into', 'dictionary']
This can be done by one line of code:
d = {List1[n]: List2[n] for n in range(len(List1))}

you can use this below code:
dict(zip(['name', 'age', 'food'], ['Monty', 42, 'spam']))
But make sure that length of the lists will be same.if length is not same.then zip function turncate the longer one.

2018-04-18
The best solution is still:
In [92]: keys = ('name', 'age', 'food')
...: values = ('Monty', 42, 'spam')
...:
In [93]: dt = dict(zip(keys, values))
In [94]: dt
Out[94]: {'age': 42, 'food': 'spam', 'name': 'Monty'}
Tranpose it:
lst = [('name', 'Monty'), ('age', 42), ('food', 'spam')]
keys, values = zip(*lst)
In [101]: keys
Out[101]: ('name', 'age', 'food')
In [102]: values
Out[102]: ('Monty', 42, 'spam')

Here is also an example of adding a list value in you dictionary
list1 = ["Name", "Surname", "Age"]
list2 = [["Cyd", "JEDD", "JESS"], ["DEY", "AUDIJE", "PONGARON"], [21, 32, 47]]
dic = dict(zip(list1, list2))
print(dic)
always make sure the your "Key"(list1) is always in the first parameter.
{'Name': ['Cyd', 'JEDD', 'JESS'], 'Surname': ['DEY', 'AUDIJE', 'PONGARON'], 'Age': [21, 32, 47]}

I had this doubt while I was trying to solve a graph-related problem. The issue I had was I needed to define an empty adjacency list and wanted to initialize all the nodes with an empty list, that's when I thought how about I check if it is fast enough, I mean if it will be worth doing a zip operation rather than simple assignment key-value pair. After all most of the times, the time factor is an important ice breaker. So I performed timeit operation for both approaches.
import timeit
def dictionary_creation(n_nodes):
dummy_dict = dict()
for node in range(n_nodes):
dummy_dict[node] = []
return dummy_dict
def dictionary_creation_1(n_nodes):
keys = list(range(n_nodes))
values = [[] for i in range(n_nodes)]
graph = dict(zip(keys, values))
return graph
def wrapper(func, *args, **kwargs):
def wrapped():
return func(*args, **kwargs)
return wrapped
iteration = wrapper(dictionary_creation, n_nodes)
shorthand = wrapper(dictionary_creation_1, n_nodes)
for trail in range(1, 8):
print(f'Itertion: {timeit.timeit(iteration, number=trails)}\nShorthand: {timeit.timeit(shorthand, number=trails)}')
For n_nodes = 10,000,000
I get,
Iteration: 2.825081646999024
Shorthand: 3.535717916001886
Iteration: 5.051560923002398
Shorthand: 6.255070794999483
Iteration: 6.52859034499852
Shorthand: 8.221581164998497
Iteration: 8.683652416999394
Shorthand: 12.599181543999293
Iteration: 11.587241565001023
Shorthand: 15.27298851100204
Iteration: 14.816342867001367
Shorthand: 17.162912737003353
Iteration: 16.645022411001264
Shorthand: 19.976680120998935
You can clearly see after a certain point, iteration approach at n_th step overtakes the time taken by shorthand approach at n-1_th step.

It can be done by the following way.
keys = ['name', 'age', 'food']
values = ['Monty', 42, 'spam']
dict = {}
for i in range(len(keys)):
dict[keys[i]] = values[i]
print(dict)
{'name': 'Monty', 'age': 42, 'food': 'spam'}

All answers sum up:
l = [1, 5, 8, 9]
ll = [3, 7, 10, 11]
zip:
dict(zip(l,ll)) # {1: 3, 5: 7, 8: 10, 9: 11}
#if you want to play with key or value #recommended
{k:v*10 for k, v in zip(l, ll)} #{1: 30, 5: 70, 8: 100, 9: 110}
counter:
d = {}
c=0
for k in l:
d[k] = ll[c] #setting up keys from the second list values
c += 1
print(d)
{1: 3, 5: 7, 8: 10, 9: 11}
enumerate:
d = {}
for i,k in enumerate(l):
d[k] = ll[i]
print(d)
{1: 3, 5: 7, 8: 10, 9: 11}

Solution as dictionary comprehension with enumerate:
dict = {item : values[index] for index, item in enumerate(keys)}
Solution as for loop with enumerate:
dict = {}
for index, item in enumerate(keys):
dict[item] = values[index]

If you are working with more than 1 set of values and wish to have a list of dicts you can use this:
def as_dict_list(data: list, columns: list):
return [dict((zip(columns, row))) for row in data]
Real-life example would be a list of tuples from a db query paired to a tuple of columns from the same query. Other answers only provided for 1 to 1.

keys = ['name', 'age', 'food']
values = ['Monty', 42, 'spam']
dic = {}
c = 0
for i in keys:
dic[i] = values[c]
c += 1
print(dic)
{'name': 'Monty', 'age': 42, 'food': 'spam'}

import pprint
p = ['A', 'B', 'C']
q = [5, 2, 7]
r = ["M", "F", "M"]
s = ['Sovabazaar','Shyambazaar','Bagbazaar','Hatkhola']
def makeDictUsingAlternateLists1(**rest):
print("*rest.keys() : ",*rest.keys())
print("rest.keys() : ",rest.keys())
print("*rest.values() : ",*rest.values())
print("**rest.keys() : ",rest.keys())
print("**rest.values() : ",rest.values())
[print(a) for a in zip(*rest.values())]
[ print(dict(zip(rest.keys(),a))) for a in zip(*rest.values())]
print("...")
finalRes= [ dict( zip( rest.keys(),a)) for a in zip(*rest.values())]
return finalRes
l = makeDictUsingAlternateLists1(p=p,q=q,r=r,s=s)
pprint.pprint(l)
"""
*rest.keys() : p q r s
rest.keys() : dict_keys(['p', 'q', 'r', 's'])
*rest.values() : ['A', 'B', 'C'] [5, 2, 7] ['M', 'F', 'M'] ['Sovabazaar', 'Shyambazaar', 'Bagbazaar', 'Hatkhola']
**rest.keys() : dict_keys(['p', 'q', 'r', 's'])
**rest.values() : dict_values([['A', 'B', 'C'], [5, 2, 7], ['M', 'F', 'M'], ['Sovabazaar', 'Shyambazaar', 'Bagbazaar', 'Hatkhola']])
('A', 5, 'M', 'Sovabazaar')
('B', 2, 'F', 'Shyambazaar')
('C', 7, 'M', 'Bagbazaar')
{'p': 'A', 'q': 5, 'r': 'M', 's': 'Sovabazaar'}
{'p': 'B', 'q': 2, 'r': 'F', 's': 'Shyambazaar'}
{'p': 'C', 'q': 7, 'r': 'M', 's': 'Bagbazaar'}
...
[{'p': 'A', 'q': 5, 'r': 'M', 's': 'Sovabazaar'},
{'p': 'B', 'q': 2, 'r': 'F', 's': 'Shyambazaar'},
{'p': 'C', 'q': 7, 'r': 'M', 's': 'Bagbazaar'}]
"""

method without zip function
l1 = [1,2,3,4,5]
l2 = ['a','b','c','d','e']
d1 = {}
for l1_ in l1:
for l2_ in l2:
d1[l1_] = l2_
l2.remove(l2_)
break
print (d1)
{1: 'd', 2: 'b', 3: 'e', 4: 'a', 5: 'c'}

Although there are multiple ways of doing this but i think most fundamental way of approaching it; creating a loop and dictionary and store values into that dictionary. In the recursive approach the idea is still same it but instead of using a loop, the function called itself until it reaches to the end. Of course there are other approaches like using dict(zip(key, value)) and etc. These aren't the most effective solutions.
y = [1,2,3,4]
x = ["a","b","c","d"]
# This below is a brute force method
obj = {}
for i in range(len(y)):
obj[y[i]] = x[i]
print(obj)
# Recursive approach
obj = {}
def map_two_lists(a,b,j=0):
if j < len(a):
obj[b[j]] = a[j]
j +=1
map_two_lists(a, b, j)
return obj
res = map_two_lists(x,y)
print(res)
Both the results should print
{1: 'a', 2: 'b', 3: 'c', 4: 'd'}

Return dictionary keys based on list integer value

I have the following:
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
I am trying to create a new dictionary which will include the dictionary value (from the original dictionary) that matches the list entry to the dictionary key (in the original dictionary)
for k in my_dict.keys():
if k in my_list:
new_dict.append(k)
print(new_dict)
should return
7777777: '500'
But I'm returning an empty set. I'm not sure what I'm doing wrong here

A dictionary comprehension would provide you what you need.
You need to make sure the types agree (int vs. str)
Unless the list is significantly longer than the dict, it will be much more efficient to iterate over the list and check that key is in the dict than the other way around.
E.g.:
In []:
new_dict = {k: my_dict[k] for k in map(int, my_list) if k in my_dict}
print(new_dict)
Out[]:
{7777777: '500'}

Try like this :
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
new_dict = {k:my_dict[k] for k in my_dict.keys() if str(k) in my_list}
print(new_dict)
# {7777777: '500'}
Update:
You can also do this with project function from funcy library.
from funcy import project
new_dict = project(my_dict, map(int, my_list))
print(new_dict)

You have to int() the iterator.
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
for l_i in my_list:
if l_i in my_dict:
print(my_dict[int(l_i)])

Your dictionary keys are of the type 'Int', while your list items are strings. You need to convert one into the other.
for example:
new_dict = {}
for k in my_dict.keys():
if str(k) in my_list:
new_dict[k] = my_dict[k]
Note that you cannot use .append() to add key-value pairs to a dictionary.

Python - Filter list of dictionaries based on if key value contains all items in another list

I have a list of dictionaries that looks like this, but with around 500 items:
listOfDicts = [{'ID': 1, 'abc': {'123': 'foo'}}, ... {'ID': 7, 'abc': {'123':'foo','456': 'bar'}}]
sampleFilterList = ['123', '456']
I am trying to filter the listOfDicts for all the results where all the values in the sampleFilterList are in the key 'abc'
The result should be a list:
[{'ID': 7, 'abc': {'123':'foo','456': 'bar'}}, ...]
I tried [i for i in listOfDicts if a for a in sampleFilterList in i['abc']], but I am getting an UnboundLocalError: local variable 'a' referenced before assignment

First, convert your second list to a set for more efficient comparisons:
sampleFilterSet = set(sampleFilterList)
Now, compare the 'abc' keys for each list item to the aforesaid set:
[item for item in listOfDicts if not (sampleFilterSet - item['abc'].keys())]
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]
This is the fastest solution. A more Pythonic (but somewhat slower) solution is to use filter():
list(filter(lambda item: not (sampleFilterSet - item['abc'].keys()), listOfDicts))
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

You need to move the in condition test before the for keyword in the list comprehension and also use get will be more safe, which returns a default value instead of throwing an error, if you are not sure if all the dictionaries in the list have the keyword abc:
listOfDicts = [{'ID': 1, 'abc': {'123': 'foo'}}, {'ID': 7, 'abc': {'123':'foo','456': 'bar'}}] ​
sampleFilterList = ['123', '456']
[d for d in listOfDicts if all(s in d.get('abc', {}) for s in sampleFilterList)]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]
Or if use a set as of #DYZ, you can use issubset:
filterSet = set(sampleFilterList)
[d for d in listOfDicts if filterSet.issubset(d.get('abc', {}))]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

Here is a working version with nested list comprehensions. Your problem is that the a for a in... is a list comprehension, and needs to be used in constructing a new list.
[i for i in listOfDicts if [a for a in sampleFilterList if a in i['abc']] == sampleFilterList]

You could try the following one-liner:
passed_the_filter = [[dictionary_entry for dictionary_entry in list_of_dicts if filter_test in dictionary_entry['abc']] for filter_test in filter]
It is a nested list comprehension that iterates through both the filter and the dictionary list. It checks if the filter is a key in the dictionary entries' "abc" value. Your problem was that you used the wrong list comprehension syntax.
N.B. You may want to note that you might not be sure that an element has a "abc" key!
Thank you for reading this.

for i in zip(listOfDicts):
a = i[0]['abc']
print (a)
or:
for i in zip(listOfDicts):
if 'abc' in i[0]:
a = i
print (a)
This is an elegant way to do it, I hope it will be useful.

List of tuples, first tuple as key, grouping by key into into a dictionary [duplicate]

This question already has answers here:
Creating a dict from list of key, value tuples while maintaining duplicate keys
(3 answers)
Closed 8 years ago.
I have a list of tuples. The first part is an identifier that may or may not be repeated. I want to process this list into a dictionary, keyed by identifier. The problem, I've been unable to think around overwriting by key:
def response_items(self):
ri = self.response_items_listing#(gets the list)
response_items = {}
for k, g in groupby(ri, itemgetter(0)):
x = list(g)
l = [(xx[1],xx[2]) for xx in x]
response_items[k] = l
return response_items
e.g. A list like:
[('123', 'abc', 'def'),('123', 'efg', 'hij'),('456', 'klm','nop')]
will come back as
{123:('efg', 'hij'), 456:('klm', 'nop')}
but I need:
{123:[('abc', 'def'),('efg', 'hij')], 456:('klm', 'nop')}
I need to put in a step to merge/aggregate by key but I am not seeing it exactly atm.
What is a better or more efficient solution?

A simple approach would be
from collections import defaultdict
ri = [('123', 'abc', 'def'),('123', 'efg', 'hij'),('456', 'klm','nop')]
response_items = defaultdict(list)
for r in ri:
response_items[r[0]].append(r[1:])
print response_items
which gives
defaultdict(<type 'list'>, {'123': [('abc', 'def'), ('efg', 'hij')],
'456': [('klm', 'nop')]})
If you want
defaultdict(<type 'list'>, {'123': ['abc', 'def', 'efg', 'hij'],
'456': ['klm', 'nop']})
as output, you can use response_items[r[0]].extend(r[1:]).

you can use setdefault():
In [79]: dic={}
In [80]: for x in lis:
dic.setdefault(x[0],[]).append(x[1:])
....:
....:
In [82]: dic
Out[82]: {'123': [('abc', 'def'), ('efg', 'hij')], '456': [('klm', 'nop')]}

If there's some reason to be using itertools.groupby, then you can avoid using a defaultdict or setdefault methodology - [in fact, if you want to go down those routes, then you don't really need the groupby!] - by:
mydict = {}
for k, g in groupby(some_list, itemgetter(0)):
mydict[k] = [el[1:] for el in g]