Develop Reference

Convert all keys in dictionary to lower case with dictionary comprehension

I wrote a function to convert all keys in a dictionary to lowercase:
def lower_dict_keys(some_dict):
"""Convert all keys to lowercase"""
result = {}
for key, value in some_dict.items():
if type(key) == str:
result[key.lower()] = value
else:
result[key] = value
return result
So far so good. Then I thought, hm, this would be more elegant in a dictionary comprehension, so this is what I came up with:
def lower_dict_keys2(some_dict):
return {key.lower(): value for key, value in some_dict.items() if type(key) == str}
This only works if all keys are strings. I there are numeric keys, they get dropped:
d1 = {'A':'foo', 'b':'bar', 1:'zip'}
print(lower_dict_keys(d1))
>>>{'a': 'foo', 'b': 'bar', 1: 'zip'}
print(lower_dict_keys2(d1))
>>>{'a': 'foo', 'b': 'bar'}
So my question is: is it possible to write this function as a dictionary comprehension?

The correct translation of your code to a comprehension is:
{key.lower() if type(key) == str else key: value for key, value in some_dict.items()}
using a conditional expression to determine the key for each key-value pair.
Using if after the for loop filters on elements; those elements that don't pass the if test are not included in the output.
You also want to avoid using type(...) is, use isinstance() instead; subclasses of str will support the lower() method too:
{key.lower() if isinstance(key, str) else key: value for key, value in some_dict.items()}

If you expect your keys to ordinarily be strings, a more appropriate solution would be to use a try / except clause.
Here we catch AttributeError, which will occur only for non-string elements.
def lower_dict_keys(some_dict):
"""Convert all keys to lowercase"""
result = {}
for key, value in some_dict.items():
try:
result[key.lower()] = value
except AttributeError:
result[key] = value
return result
d1 = {'A':'foo', 'b':'bar', 1:'zip'}
lower_dict_keys(d1)
# {'a': 'foo', 'b': 'bar', 1: 'zip'}
Unfortunately, this will not work with a dictionary comprehension, since statements are not permitted in comprehensions and PEP-463 was rejected.

Merging values from 2 dictionaries (Python)

(I'm new to Python!)
Trying to figure out this homework question:
The function will takes a​s input​ two dictionaries, each mapping strings to integers. The function will r​eturn​ a dictionary that maps strings from the two input dictionaries to the sum of the integers in the two input dictionaries.
my idea was this:
def ​add(​dicA,dicB):
dicA = {}
dicB = {}
newdictionary = dicA.update(dicB)
however, that brings back None.
In the professor's example:
print(add({'alice':10, 'Bob':3, 'Carlie':1}, {'alice':5, 'Bob':100, 'Carlie':1}))
the output is:
{'alice':15, 'Bob':103, 'Carlie':2}
My issue really is that I don't understand how to add up the values from each dictionaries. I know that the '+' is not supported with dictionaries. I'm not looking for anyone to do my homework for me, but any suggestions would be very much appreciated!

From the documentation:
update([other])
Update the dictionary with the key/value pairs from other, overwriting existing keys. Return None.
You don't want to replace key/value pairs, you want to add the values for similar keys. Go through each dictionary and add each value to the relevant key:
def ​add(​dicA,dicB):
result = {}
for d in dicA, dicB:
for key in d:
result[key] = result.get(key, 0) + d[key]
return result
result.get(key, 0) will retrieve the value of an existing key or produce 0 if key is not yet present.

First of all, a.update(b) updates a in place, and returns None.
Secondly, a.update(b) wouldn't help you to sum the keys; it would just produce a dictionary with the resulting dictionary having all the key, value pairs from b:
>>> a = {'alice':10, 'Bob':3, 'Carlie':1}
>>> b = {'alice':5, 'Bob':100, 'Carlie':1}
>>> a.update(b)
>>> a
{'alice': 5, 'Carlie': 1, 'Bob': 100}
It'd be easiest to use collections.Counter to achieve the desired result. As a plus, it does support addition with +:
from collections import Counter
def add(dicA, dicB):
return dict(Counter(dicA) + Counter(dicB))
This produces the intended result:
>>> print(add({'alice':10, 'Bob':3, 'Carlie':1}, {'alice':5, 'Bob':100, 'Carlie':1}))
{'alice': 15, 'Carlie': 2, 'Bob': 103}

The following is not meant to be the most elegant solution, but to get a feeling on how to deal with dicts.
dictA = {'Alice':10, 'Bob':3, 'Carlie':1}
dictB = {'Alice':5, 'Bob':100, 'Carlie':1}
# how to iterate through a dictionary
for k,v in dictA.iteritems():
print k,v
# make a new dict to keep tally
newdict={}
for d in [dictA,dictB]: # go through a list that has your dictionaries
print d
for k,v in d.iteritems(): # go through each dictionary item
if not k in newdict.keys():
newdict[k]=v
else:
newdict[k]+=v
print newdict
Output:
Bob 3
Alice 10
Carlie 1
{'Bob': 3, 'Alice': 10, 'Carlie': 1}
{'Bob': 100, 'Alice': 5, 'Carlie': 1}
{'Bob': 103, 'Alice': 15, 'Carlie': 2}

def ​add(​dicA,dicB):
You define a function that takes two arguments, dicA and dicB.
dicA = {}
dicB = {}
Then you assign an empty dictionary to both those variables, overwriting the dictionaries you passed to the function.
newdictionary = dicA.update(dicB)
Then you update dicA with the values from dicB, and assign the result to newdictionary. dict.update always returns None though.
And finally, you don’t return anything from the function, so it does not give you any results.
In order to combine those dictionaries, you actually need to use the values that were passed to it. Since dict.update mutates the dictionary it is called on, this would change one of those passed dictionaries, which we generally do not want to do. So instead, we use an empty dictionary, and then copy the values from both dictionaries into it:
def add (dicA, dicB):
newDictionary = {}
newDictionary.update(dicA)
newDictionary.update(dicB)
return newDictionary
If you want the values to sum up automatically, then use a Counter instead of a normal dictionary:
from collections import Counter
def add (dicA, dicB):
newDictionary = Counter()
newDictionary.update(dicA)
newDictionary.update(dicB)
return newDictionary

I suspect your professor wants to achieve this using more simple methods. But you can achieve this very easily using collections.Counter.
from collections import Counter
def add(a, b):
return dict(Counter(a) + Counter(b))
Your professor probably wants something like this:
def add(a, b):
new_dict = copy of a
for each key/value pair in b
if key in new_dict
add value to value already present in new_dict
else
insert key/value pair into new_dict
return new_dict

You can try this:
def add(dict1, dict2):
return dict([(key,dict1[key]+dict2[key]) for key in dict1.keys()])

I personally like using a dictionary's get method for this kind of merge:
def add(a, b):
result = {}
for dictionary in (a, b):
for key, value in dictionary.items():
result[key] = result.get(key, 0) + value
return result

Neat way of popping key, value PAIR from dictionary?

pop is a great little function that, when used on dictionaries (given a known key) removes the item with that key from the dictionary and also returns the corresponding value. But what if I want the key as well?
Obviously, in simple cases I could probably just do something like this:
pair = (key, some_dict.pop(key))
But if, say, I wanted to pop the key-value pair with the lowest value, following the above idea I would have to do this...
pair = (min(some_dict, key=some.get), some_dict.pop(min(some_dict, key=some_dict.get)))
... which is hideous as I have to do the operation twice (obviously I could store the output from min in a variable, but I'm still not completely happy with that). So my question is: Is there an elegant way to do this? Am I missing an obvious trick here?

You can define yourself dictionary object using python ABCs which provides the infrastructure for defining abstract base classes. And then overload the pop attribute of python dictionary objects based on your need:
from collections import Mapping
class MyDict(Mapping):
def __init__(self, *args, **kwargs):
self.update(dict(*args, **kwargs))
def __setitem__(self, key, item):
self.__dict__[key] = item
def __getitem__(self, key):
return self.__dict__[key]
def __delitem__(self, key):
del self.__dict__[key]
def pop(self, k, d=None):
return k,self.__dict__.pop(k, d)
def update(self, *args, **kwargs):
return self.__dict__.update(*args, **kwargs)
def __iter__(self):
return iter(self.__dict__)
def __len__(self):
return len(self.__dict__)
def __repr__(self):
return repr(self.__dict__)
Demo:
d=MyDict()
d['a']=1
d['b']=5
d['c']=8
print d
{'a': 1, 'c': 8, 'b': 5}
print d.pop(min(d, key=d.get))
('a', 1)
print d
{'c': 8, 'b': 5}
Note : As #chepner suggested in comment as a better choice you can override popitem, which already returns a key/value pair.

A heap supports the pop-min operation you describe. You'll need to create a heap from your dictionary first, though.
import heapq
# Must be two steps; heapify modifies its argument in-place.
# Reversing the key and the value because the value will actually be
# the "key" in the heap. (Or rather, tuples are compared
# lexicographically, so put the value in the first position.)
heap = [(v, k) for k, v in some_dict.items()]
heapq.heapify(heap)
# Get the smallest item from the heap
value, key = heapq.heappop(heap)

here is a simpler implementation
class CustomDict(dict):
def pop_item(self, key):
popped = {key:self[key]} #save "snapshot" of the value of key before popping
self.pop(key)
return popped
a = CustomDict()
b = {"hello":"wassup", "lol":"meh"}
a.update(b)
print(a.pop_item("lol"))
print(a)
So here we create a custom dict that pops the item you want and gives out the key-value pair

Find lowest value in a list of dictionaries in python

I want to find and return the minimal value of an id in a string, for example:
find_min_id([{"nonid": "-222", "id": 0}, {"id": -101}])
-101
find_min_id([{’id’: 63, 'id': 42}])
42
So far I have this:
def find_min_id(list):
return min(list)
but that gives:
{'id': -101}
and I only want the value of the lowest id.

Use the key parameter of min:
def find_min_id(l):
return min(l, key=lambda d: d.get("id", float('inf')))["id"]
This actually finds the min id, and does it without creating a new list.
The only thing is, the elements in your list might not had an 'id' key. For that reason I had to use .get("id", float('inf')). Thus the function will return inf if there is no id key, which might not be desirable. What min() does when given an empty list is it throws an exception, so we'd probably like to do the same if none of the dicts we pass it have an 'id' key. In that case the min of a generator appoach might indeed be better:
def find_min_id(l):
return min(d["id"] for d in l if "id" in d)
The other approach would be to check for inf as the result of min, but this is more cumbersome:
import math
def find_min_id(l):
res = min(l, key=lambda d: d.get("id", float('inf')))["id"]
if math.isinf(res):
raise ValueError("No dict has an 'id' key")
return res

Another approach, but works where there is no id in the dictionary, and when there is no dictionary with an id at all:
def find_min_id(lst):
ids = [d['id'] for d in lst if 'id' in d]
return min(ids) if ids else None
Without exceptions, and without running min for artificially extended list (i.e. the answer which puts maximum floats where an entry isn't an id-entry).

Following code return None if no id key is in list.
>>> data = [{"nonid": "-222", "id": 0}, {"id": -101}, {"nonid":-200}]
>>> min(filter(lambda x: x is not None, map(lambda x: x.get('id', None),
... data)) or [None])
-101
Here filter(lambda x: x is not None, ...) filters out dictionaries without id,
map(lambda x: x.get('id', None), data) gets all id's from data, and or [None] part treats case when no id key will be found in data.

You are trying to find the 'lowest' dictionaries. What we want is, to find the lowest 'id' value in the list.
def find_min_id(lst):
return min([d[key] for key in d for d in lst if key=="id"])
Also, avoid using list as a variable name, it overrides the built-in function list().
A little demo:
>>> def find_min_id(lst):
return min([d[key] for key in d for d in lst if key=="id"])
>>> find_min_id(lst)
-101
Hope this helps!

>>> ids = [{"nonid": "-222", "id": 0}, {"id": -101}]
>>> min([val for obj in ids for key, val in obj.items() if key == 'id'])
-101
>>> ids = [{'id': 63}, { 'id': 42}]
>>> min([val for obj in ids for key, val in obj.items() if key == 'id'])
42
Try the above.
You can make this into a function definition:
def find_lowest(ids):
return min([val for obj in ids for key, val in obj.items() if key == 'id'])
Working example.
Let me explain what I'm doing. Firstly, the min function takes in an iterable object as an argument. So, let me demonstrate:
>>> min([1,2,3,4,6,1,0])
0
So, what this means is this, we are essentially taking the minimum value of the list that we get from this, [val for obj in ids for key, val in obj.items() if key == 'id'].
Now, you might be wondering, well whats happening in there? It might be a little intimidating at first, but thats a list comprehension. Whats that you say? Well, in simple terms its a concise we in which we make a list:
Let me start with be first part, and no its not the beginning of the statement:
for obj in ids
What we are doing here, is iterating over all the dictionary objects in in side of ids. Now, we use that object here:
key, val in obj.items() if key == 'id'
Since object, is a dict, we use the items function to get a generator that gives a tuple of key, value pairs. In an object like this: {'id': 100}, the id would be they key and 100 would be the value. So, we are going over all the items in the dictionary object, and if the key happens to be id, then we append it to the list:
[val
Thats what the first part does. The first part of the list comprehension appends something to the final list, and that is val.
UPDATE:
If for some reason, the list does not containt anything with id as a key, then it will throw a ValueError as min does not accept an empty list, so to remedy this, we can check:
def find_lowest(ids):
_ret = [val for obj in ids for key, val in obj.items() if key == 'id']
if _ret:
return min(_ret)
else:
return None

list is a built-in type in Python. don't use it as an identifier
def find_min_id(my_list)
id_list = []
for record in my_list:
if 'id' in record:
id_list.append(record['id'])
return min(id_list)

How does the performance of dictionary key lookups compare in Python?

How does:
dict = {}
if key not in dict:
dict[key] = foo
Compare to:
try:
dict[key]
except KeyError:
dict[key] = foo
ie, is the look up of a key in anyway faster than the linear search through dict.keys(), that I assume the first form will do?

Just to clarify one point: if key not in d doesn't do a linear search through d's keys. It uses the dict's hash table to quickly find the key.

You're looking for the setdefault method:
>>> r = {}
>>> r.setdefault('a', 'b')
'b'
>>> r
{'a': 'b'}
>>> r.setdefault('a', 'e')
'b'
>>> r
{'a': 'b'}

The answer depends on how often the key is already in the dict (BTW, has anyone mentioned to you how bad an idea it is to hide a builtin such as dict behind a variable?)
if key not in dct:
dct[key] = foo
If the key is in the dictionary this does one dictionary lookup. If the key is in the dictionary it looks up the dictionary twice.
try:
dct[key]
except KeyError:
dct[key] = foo
This may be slightly faster for the case where the key is in the dictionary, but throwing an exception has quite a big overhead, so it is almost always not the best option.
dct.setdefault(key, foo)
This one is slightly tricky: it always involves two dictionary lookups: the first one is to find the setdefault method in the dict class, the second is to look for key in the dct object. Also if foo is an expression it will be evaluated every time whereas the earlier options only evaluate it when they have to.
Also look at collections.defaultdict. That is the most appropriate solution for a large class of situations like this.

Try: my_dict.setdefault(key, default). It's slightly slower than the other options, though.
If key is in the dictionary, return its value. If not, insert key with a value of default and return default. default defaults to None.
#!/usr/bin/env python
example_dict = dict(zip(range(10), range(10)))
def kn(key, d):
if key not in d:
d[key] = 'foo'
def te(key, d):
try:
d[key]
except KeyError:
d[key] = 'foo'
def sd(key, d):
d.setdefault(key, 'foo')
if __name__ == '__main__':
from timeit import Timer
t = Timer("kn(2, example_dict)", "from __main__ import kn, example_dict")
print t.timeit()
t = Timer("te(2, example_dict)", "from __main__ import te, example_dict")
print t.timeit()
t = Timer("sd(2, example_dict)", "from __main__ import sd, example_dict")
print t.timeit()
# kn: 0.249855041504
# te: 0.244259119034
# sd: 0.375113964081

my_dict.get(key, foo) returns foo if key isn't in my_dict. The default value is None, so my_dict.get(key) will return None if key isn't in my_dict. The first of your options is better if you want to just add key to your dictionary. Don't worry about speed here. If you find that populating your dictionary is a hot spot in your program, then think about it. But it isn't. So don't.