Develop Reference

Dictionary sized change due to iteration of dict

I am attempting to remove key-value pairs from a dict when a sub-dictionary matches values from another dictionary.
Example set-up:
e = {'a':{'aa':'yes'}, 'b':{'ac':'no'}, 'a':{'aa':'yes'}}
f = {'a':{'aa':'yes'}, 'e':{'ab':'no'}, 'a':{'aa':'yes'}}
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') == v.get('aa'):
e.pop(keys)
RuntimeError: dictionary changed size during iteration
Expected result:
#from
e = {'a':{'aa':'yes'}, 'b':{'ac':'no'}, 'a':{'aa':'yes'}}
#to
e = {'b':{'ac':'no'}}

With single dict comprehension:
e = {k:v for k, v in e.items() if v.items() != f.get(k, {}).items()}
{'b': {'ac': 'no'}}
dict.get(key[, default]) allows you to set the needed(or preferred) default value returned for the key in dict

In general, you should not add or remove items from iterables that you are currently iterating over.

As you've been told, you can't modify the length of a thing while you're iterating it. There are a few options here, such as:
Saving a list of what you want to remove, then doing it later:
to_remove = []
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') == v.get('aa'):
to_remove.append(keys)
for tr in to_remove:
e.pop(tr)
Cloning the object, so that what you're iterating does not change but the original object can be modified. This is even more memory expensive than the previous however.
for keys, values in dict(e).items(): # or list(e.items())
for k, v in f.items():
if values.get('aa') == v.get('aa'):
e.pop(keys)
You could also, in your case, simply create a new object:
g = {}
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') != v.get('aa'):
g[keys] = values

how to combine the common key and join the values in the dictionary python

I have one list which contain a few dictionaries.
[{u'TEXT242.txt': u'work'},{u'TEXT242.txt': u'go to work'},{u'TEXT1007.txt': u'report'},{u'TEXT797.txt': u'study'}]
how to combine the dictionary when it has the same key. for example:
u'work', u'go to work'are under one key:'TEXT242.txt', so that i can remove the duplicated key.
[{u'TEXT242.txt': [u'work', u'go to work']},{u'TEXT1007.txt': u'report'},{u'TEXT797.txt': u'study'}]

The setdefault method of dictionaries is handy here... it can create an empty list when a dictionary key doesn't exist, so that you can always append the value.
dictlist = [{u'TEXT242.txt': u'work'},{u'TEXT242.txt': u'go to work'},{u'TEXT1007.txt': u'report'},{u'TEXT797.txt': u'study'}]
newdict = {}
for d in dictlist:
for k in d:
newdict.setdefault(k, []).append(d[k])

from collections import defaultdict
before = [{u'TEXT242.txt': u'work'},{u'TEXT242.txt': u'go to work'},{u'TEXT1007.txt': u'report'},{u'TEXT797.txt': u'study'}]
after = defaultdict(list)
for i in before:
for k, v in i.items():
after[k].append(v)
out:
defaultdict(list,
{'TEXT1007.txt': ['report'],
'TEXT242.txt': ['work', 'go to work'],
'TEXT797.txt': ['study']})
This technique is simpler and faster
than an equivalent technique using dict.setdefault()

operations with elements in list of list in python

This is my list:
volume = [['1.986', '3000'], ['1.987', '2000'], ['1.986', '700'],['1.987', '4000']]
How can I get the sum of volume[1] when volume[0] is the same price?
results = [['1.986', '3700'], ['1.987', '6000']]

Dictionaries would be a good data structure to use here. The default dict holds unique strings as the keys and assumes empty values are 0 because I set it to be based off of int.
from collections import defaultdict
d = defaultdict(int)
for v in volume:
d[v[0]] += int(v[1])
print d
If you need a list afterwards you can use a list comprehension:
list_version = [[key, value] for key,value in d]

How to reorder a python ordered dict based on array?

Say I have an Ordered Dict with the following items:
mydict = {'Rust': {'definition':'rusts definition'}, 'Iron': {'definition:'iron definition'}, 'Pyrite': {'definition':'pyrite definition'}}
If I have an array:
myorder = ['Pyrite', 'Rust', 'Iron']
How can I reorder the Ordered Dict such that the items in mydict are ordered based on myorder?

Try this:
mydict = {'Rust': {'definition':'rusts definition'},
'Iron': {'definition':'iron definition'},
'Pyrite': {'definition':'pyrite definition'}}
myorder = ['Pyrite', 'Rust', 'Iron']
from collections import OrderedDict
ordered = OrderedDict()
for k in myorder:
ordered[k] = mydict[k]
Or even shorter:
ordered = OrderedDict((k, mydict[k]) for k in myorder)
Using the above snippet, ordered will contain the same keys/values as mydict, but they'll be inserted in the same order specified by myorder. That's the advantage of OrderedDict: when iterating over it, it'll preserve the insertion order.
There's no way to sort the existing dictionary in-place (well, you could extract all the key-value pairs, eliminate them and add them again in the correct order, but that's not the idea, is it?), it's necessary to create a new one ... or simply iterate over the existing dictionary in the specified order:
for k in myorder:
x = mydict[k] # do something with x

If you would like to use them in that order, you can do this, for example.
for key in myorder:
value = mydict[key]
print value
Outputs:
{'definition': 'pyrite definition'}
{'definition': 'rusts definition'}
{'definition': 'iron definiti

Refactoring with python dictionary comprehension

I have 2 dictionary which contain the same keys but the value pairs are different. Let's make dictA and dictB represent the two dictionaries in question.
dictA = {'key1':'Joe', 'key2':'Bob'}
dictB = {'key1':'Smith', 'key2':'Johnson'}
Currently, I am creating a new dictionary based the common occurring keys through a nested if statement. In doing so, the values that share a key are contained within a list, in the new dictionary. See this done below:
dictAB = {} # Create a new dictionary
# Create a list container for dictionary values
for key in dictA.keys():
dictAB[key] = []
# Iterate through keys in both dictionaries
# Find matching keys and append the respective values to the list container
for key, value in dictA.iteritems():
for key2, value2 in dictB.iteritems():
if key == key2:
dictAB[key].append(value)
dictAB[key].append(value2)
else:
pass
How can this be made into a more clean structure using python dictionary comprehension?

Use sets or key views (python 2.7):
dictAB = {k: [dictA[k], dictB[k]] for k in dictA.viewkeys() & dictB.viewkeys()}
Before 2.7:
dictAB = dict((k, [dictA[k], dictB[k]]) for k in set(dictA) & set(dictB))
In python 3, you can use the .keys method for such operations directly, as they are implemented as views:
dictAB = {k: [dictA[k], dictB[k]] for k in dictA.keys() & dictB.keys()}
Demo (python 2.7):
>>> dictA = {'key1':'Joe', 'key2':'Bob'}
>>> dictB = {'key1':'Smith', 'key2':'Johnson'}
>>> dictAB = {k: [dictA[k], dictB[k]] for k in dictA.viewkeys() & dictB.viewkeys()}
>>> print dictAB
{'key2': ['Bob', 'Johnson'], 'key1': ['Joe', 'Smith']}
The & operator on either two sets or on a dict view creates the intersection of both sets; all keys that are present in both sets.
By using an intersection of the keys, this code will work even if either dictA or dictB has keys that do not appear in the other dictionary. If you are absolutely sure the keys will always match, you could just iterate over either dict directly without the intersection:
dictAB = {k: [dictA[k], dictB[k]] for k in dictA}

dictAB = { key: [dictA[key],dictB[key]] for key in dictA if key in dictB }