Develop Reference

RuntimeError: dictionary changed size during iteration using Python3? [duplicate]

I have a dictionary of lists in which some of the values are empty:
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
At the end of creating these lists, I want to remove these empty lists before returning my dictionary. I tried doing it like this:
for i in d:
if not d[i]:
d.pop(i)
but I got a RuntimeError. I am aware that you cannot add/remove elements in a dictionary while iterating through it...what would be a way around this then?
See Modifying a Python dict while iterating over it for citations that this can cause problems, and why.

In Python 3.x and 2.x you can use use list to force a copy of the keys to be made:
for i in list(d):
In Python 2.x calling keys made a copy of the keys that you could iterate over while modifying the dict:
for i in d.keys():
But note that in Python 3.x this second method doesn't help with your error because keys returns an a view object instead of copying the keys into a list.

You only need to use copy:
This way you iterate over the original dictionary fields and on the fly can change the desired dict d.
It works on each Python version, so it's more clear.
In [1]: d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
In [2]: for i in d.copy():
...: if not d[i]:
...: d.pop(i)
...:
In [3]: d
Out[3]: {'a': [1], 'b': [1, 2]}
(BTW - Generally to iterate over copy of your data structure, instead of using .copy for dictionaries or slicing [:] for lists, you can use import copy -> copy.copy (for shallow copy which is equivalent to copy that is supported by dictionaries or slicing [:] that is supported by lists) or copy.deepcopy on your data structure.)

Just use dictionary comprehension to copy the relevant items into a new dict:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.items() if v}
>>> d
{'a': [1], 'b': [1, 2]}
For this in Python 2:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.iteritems() if v}
>>> d
{'a': [1], 'b': [1, 2]}

This worked for me:
d = {1: 'a', 2: '', 3: 'b', 4: '', 5: '', 6: 'c'}
for key, value in list(d.items()):
if value == '':
del d[key]
print(d)
# {1: 'a', 3: 'b', 6: 'c'}
Casting the dictionary items to list creates a list of its items, so you can iterate over it and avoid the RuntimeError.

I would try to avoid inserting empty lists in the first place, but, would generally use:
d = {k: v for k,v in d.iteritems() if v} # re-bind to non-empty
If prior to 2.7:
d = dict( (k, v) for k,v in d.iteritems() if v )
or just:
empty_key_vals = list(k for k in k,v in d.iteritems() if v)
for k in empty_key_vals:
del[k]

To avoid "dictionary changed size during iteration error".
For example: "when you try to delete some key",
Just use 'list' with '.items()'. Here is a simple example:
my_dict = {
'k1':1,
'k2':2,
'k3':3,
'k4':4
}
print(my_dict)
for key, val in list(my_dict.items()):
if val == 2 or val == 4:
my_dict.pop(key)
print(my_dict)
Output:
{'k1': 1, 'k2': 2, 'k3': 3, 'k4': 4}
{'k1': 1, 'k3': 3}
This is just an example. Change it based on your case/requirements.

For Python 3:
{k:v for k,v in d.items() if v}

You cannot iterate through a dictionary while it’s changing during a for loop. Make a casting to list and iterate over that list. It works for me.
for key in list(d):
if not d[key]:
d.pop(key)

Python 3 does not allow deletion while iterating (using the for loop above) a dictionary. There are various alternatives to do it; one simple way is to change the line
for i in x.keys():
with
for i in list(x)

The reason for the runtime error is that you cannot iterate through a data structure while its structure is changing during iteration.
One way to achieve what you are looking for is to use a list to append the keys you want to remove and then use the pop function on dictionary to remove the identified key while iterating through the list.
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
pop_list = []
for i in d:
if not d[i]:
pop_list.append(i)
for x in pop_list:
d.pop(x)
print (d)

For situations like this, I like to make a deep copy and loop through that copy while modifying the original dict.
If the lookup field is within a list, you can enumerate in the for loop of the list and then specify the position as the index to access the field in the original dict.

Nested null values
Let's say we have a dictionary with nested keys, some of which are null values:
dicti = {
"k0_l0":{
"k0_l1": {
"k0_l2": {
"k0_0":None,
"k1_1":1,
"k2_2":2.2
}
},
"k1_l1":None,
"k2_l1":"not none",
"k3_l1":[]
},
"k1_l0":"l0"
}
Then we can remove the null values using this function:
def pop_nested_nulls(dicti):
for k in list(dicti):
if isinstance(dicti[k], dict):
dicti[k] = pop_nested_nulls(dicti[k])
elif not dicti[k]:
dicti.pop(k)
return dicti
Output for pop_nested_nulls(dicti)
{'k0_l0': {'k0_l1': {'k0_l2': {'k1_1': 1,
'k2_2': 2.2}},
'k2_l1': 'not '
'none'},
'k1_l0': 'l0'}

The Python "RuntimeError: dictionary changed size during iteration" occurs when we change the size of a dictionary when iterating over it.
To solve the error, use the copy() method to create a shallow copy of the dictionary that you can iterate over, e.g., my_dict.copy().
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in my_dict.copy():
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
You can also convert the keys of the dictionary to a list and iterate over the list of keys.
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in list(my_dict.keys()):
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}

If the values in the dictionary were unique too, then I used this solution:
keyToBeDeleted = None
for k, v in mydict.items():
if(v == match):
keyToBeDeleted = k
break
mydict.pop(keyToBeDeleted, None)

Python convert dict of lists to dict of sets?

I have:
myDict = {'a': [1,2,3], 'b':[4,5,6], 'c':[7,8,9]}
I want:
myDict = {'a': set([1,2,3]), 'b':set([4,5,6]), 'c':set([7,8,9])}
Is there a one-liner I can use to do this rather than looping through it and converting the type of the values?

You'll have to loop anyway:
{key: set(value) for key, value in yourData.items()}
If you're using Python 3.6+, you can also do this:
dict(zip(myDict.keys(), map(set, myDict.values())))

This can be done with map by mapping the values to type set
myDict = dict(map(lambda x: (x[0], set(x[1])), myDict.items()))
Or with either version of dictionary comprehension as well
myDict = {k: set(v) for k, v in myDict.items()}
myDict = {k: set(myDict[k]) for k in myDict}

You can use comprehension for it:
Basically, loop through the key-value pairs and create set out of each value for the corresponding key.
>>> myDict = {'a': [1,2,3], 'b':[4,5,6], 'c':[7,8,9]}
>>> myDict = {k: set(v) for k, v in myDict.items()}
>>> myDict
{'a': {1, 2, 3}, 'b': {4, 5, 6}, 'c': {8, 9, 7}}

You can't do it without looping anyway, but you can have the looping done in one line, with the following code:
myDict = {k:set(v) for k, v in myDict.items()}
This is basically traversing each item in your dictionary and converting the lists to sets and combining the key(str):value(set) pairs to a new dictionary and assigning it back to myDict variable.

How to iterate through a nested dict?

I have a nested python dictionary data structure. I want to read its keys and values without using collection module. The data structure is like bellow.
d = {'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'baz': 3, 'quux': 4}}
I was trying to read the keys in the dictionary using the bellow way but getting error.
Code
for key, value in d:
print(Key)
Error
ValueError: too many values to unpack (expected 2)
So can anyone please explain the reason behind the error and how to iterate through the dictionary.

keys() method returns a view object that displays a list of all the keys in the dictionary
Iterate nested dictionary:
d = {'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'baz': 3, 'quux': 4}}
for i in d.keys():
print i
for j in d[i].keys():
print j
OR
for i in d:
print i
for j in d[i]:
print j
output:
dict1
foo
bar
dict2
baz
quux
where i iterate main dictionary key and j iterate the nested dictionary key.

The following will work with multiple levels of nested-dictionary:
def get_all_keys(d):
for key, value in d.items():
yield key
if isinstance(value, dict):
yield from get_all_keys(value)
d = {'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'dict3': {'baz': 3, 'quux': 4}}}
for x in get_all_keys(d):
print(x)
This will give you:
dict1
foo
bar
dict2
dict3
baz
quux

As the requested output, the code goes like this
d = {'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'baz': 3, 'quux': 4}}
for k1,v1 in d.iteritems(): # the basic way
temp = ""
temp+=k1
for k2,v2 in v1.iteritems():
temp = temp+" "+str(k2)+" "+str(v2)
print temp
In place of iteritems() you can use items() as well, but iteritems() is much more efficient and returns an iterator.
Hope this helps :)

To get keys and values you need dict.items():
for key, value in d.items():
print(key)
If you want just the keys:
for key in d:
print(key)

you can iterate all keys and values of nested dictionary as following:
d = {'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'baz': 3, 'quux': 4}}
for i in d:
for j, k in d[i].items():
print(j,"->", k)
Your output looks like this -
foo -> 1
bar -> 2
baz -> 3
quux -> 4

Iterating through a dictionary only gives you the keys.
You told python to expect a bunch of tuples, and it tried to unpack something that wasn't a tuple (your code is set up to expect each iterated item to be of the form (key,value), which was not the case (you were simply getting key on each iteration).
You also tried to print Key, which is not the same as key, which would have led to a NameError.
for key in d:
print(key)
should work.

if given dictionary pattern has monotone format and keys are known
dict_ = {'0': {'foo': 1, 'bar': 2}, '1': {'foo': 3, 'bar': 4}}
for key, val in dict_.items():
if isinstance(val, dict):
print(val.get('foo'))
print(val.get('bar'))
in this case we can skip nested loop

You could use benedict (a dict subclass) and the traverse utility method:
Installation: pip install python-benedict
from benedict import benedict
d = benedict({'dict1': {'foo': 1, 'bar': 2}, 'dict2': {'baz': 3, 'quux': 4}})
def traverse_item(dct, key, value):
print('key: {} - value: {}'.format(key, value))
d.traverse(traverse_item)
Documentation: https://github.com/fabiocaccamo/python-benedict
Note: I am the author of this project.

Create N new dictionaries based on N keys from an original dictionary

I am new at python and here. I need to create N new dictionaries based on N keys from an original dictionary.
Lets say I have an OriginalDict {a:1, b:2, c:3, d:4, e:5, f:6} I need to create 6 new dictionaries having as keys (new keys are same) the value of the original. Something like that:
Dict1 {Name:a,...}, Dict2 {Name:b,...}, Dict3 {Name:c,...}.....
Dict6 {Name:f...}
This is my code:
d = {}
for key in OriginalDict:
d['Name'] = key
I got a new dictionary but only for the last key.
print d
{Name:f}
I guess cos' last value in a dictionary overrides the previous one if the keys are the same
please advise... :)

If, for example, we put all those dicts in a list, we can use this comprehensions
dicts = [{'Name': k} for k in OriginalDict]
Let's try it out
>>> OriginalDict = {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 6}
>>> dicts = [{'Name': k} for k in OriginalDict]
>>> dicts
[{'Name': 'd'}, {'Name': 'c'}, {'Name': 'a'}, {'Name': 'b'}, {'Name': 'e'}, {'Name': 'f'}]
The statement 6 new dictionaries having as keys (new keys are same) the value of the original seems to contradict your example, at least to me.
In such case we can do
dicts = [{v: k} for k, v in OriginalDict.items()]
Let's try it out:
>>> dicts = [{v: k} for k, v in OriginalDict.items()]
>>> dicts
[{4: 'd'}, {3: 'c'}, {1: 'a'}, {2: 'b'}, {5: 'e'}, {6: 'f'}]

In python 3.x:
for key in OriginalDict.keys():
d = {}
d ['Name'] = key
This will give you a new Dictionary for every key of the Original one. Now, you could save them inside a list or inside another dictionary like:
New_Dicts = []
for key in OriginalDict.keys():
d = {}
d ['Name'] = key
New_Dicts.append(d)
Or,
New_Dicts = {}
for i,key in enumerate(OriginalDict.keys()):
d = {}
d ['Name'] = key
New_Dicts[i] = d

I think you want to create a function which is a generator, then call that function passing in the dictionary and then yielding your new ones:
orig = {'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6}
def make_sub_dictionaries(big_dictionary):
for key, val in big_dictionary.iteritems():
yield {'name': key, 'val': val }
# if you want them one by one, call next
first = make_sub_dictionaries(orig).next()
print first
# if you want all of them
for d in make_sub_dictionaries(orig):
print str(d)
# do whatever stuff you need to do

One method is as follows:
OriginalDict = {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 6}
newDicts = [{'Name':v} for k,v in enumerate(OriginalDict.values())]
which will give you
>>> newDicts
[{'Name': 1}, {'Name': 3}, {'Name': 2}, {'Name': 5}, {'Name': 4}, {'Name': 6}]

First of all, this won't keep the order. You need to use OriginalDict to preserve the order.
my_dict={'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6}
j=0
for key, value in my_dict.iteritems():
j +=1
exec("Dict%s = %s" % (j,{key:value}))
Now when you type.
>>> print Dict1
{'a':1}
>>> print Dict2
{'c':3}

getting dict key using dict value in python

My question is: How can I get a dictionary key using a dictionary value?
d={'dict2': {1: 'one', 2: 'two'}, 'dict1': {3: 'three', 4: 'four'}}
I want to get dict2 the key of the key of two.
Thanks.

Here's a recursive solution that can handle arbitrarily nested dictionaries:
>>> import collections
>>> def dict_find_recursive(d, target):
... if not isinstance(d, collections.Mapping):
... return d == target
... else:
... for k in d:
... if dict_find_recursive(d[k], target) != False:
... return k
... return False
It's not as efficient in the long run as a "reverse dictionary," but if you aren't doing such reverse searches frequently, it probably doesn't matter. (Note that you have to explicitly compare the result of dict_find_recursive(d[k], target) to False because otherwise falsy keys like '' cause the search to fail. In fact, even this version fails if False is used as a key; a fully general solution would use a unique sentinel object() to indicate falseness.)
A few usage examples:
>>> d = {'dict1': {3: 'three', 4: 'four'}, 'dict2': {1: 'one', 2: 'two'}}
>>> dict_find_recursive(d, 'two')
'dict2'
>>> dict_find_recursive(d, 'five')
False
>>> d = {'dict1': {3: 'three', 4: 'four'}, 'dict2': {1: 'one', 2: 'two'},
'dict3': {1: {1:'five'}, 2: 'six'}}
>>> dict_find_recursive(d, 'five')
'dict3'
>>> dict_find_recursive(d, 'six')
'dict3'
If you want to reverse an arbitrarily nested set of dictionaries, recursive generators are your friend:
>>> def dict_flatten(d):
... if not isinstance(d, collections.Mapping):
... yield d
... else:
... for value in d:
... for item in dict_flatten(d[value]):
... yield item
...
>>> list(dict_flatten(d))
['three', 'four', 'five', 'six', 'one', 'two']
The above simply lists all the values in the dictionary that aren't mappings. You can then map each of those values to a key like so:
>>> def reverse_nested_dict(d):
... for k in d:
... if not isinstance(d[k], collections.Mapping):
... yield (d[k], k)
... else:
... for item in dict_flatten(d[k]):
... yield (item, k)
...
This generates a iterable of tuples, so no information is lost:
>>> for tup in reverse_nested_dict(d):
... print tup
...
('three', 'dict1')
('four', 'dict1')
('five', 'dict3')
('six', 'dict3')
('one', 'dict2')
('two', 'dict2')
If you know that all your non-mapping values are hashable -- and if you know they are unique, or if you don't care about collisions -- then just pass the resulting tuples to dict():
>>> dict(reverse_nested_dict(d))
{'six': 'dict3', 'three': 'dict1', 'two': 'dict2', 'four': 'dict1',
'five': 'dict3', 'one': 'dict2'}

If you don't want to reverse the dictionary, here's another possible solution:
def get_key_from_value(my_dict, v):
for key,value in my_dict.items():
if value == v:
return key
return None
>>> d = {1: 'one', 2: 'two'}
>>> get_key_from_value(d,'two')
2

The following will create a reverse dictionary for the two-level example:
d={'dict2': {1: 'one', 2: 'two'}, 'dict1': {3: 'three', 4: 'four'}}
r = {}
for d1 in d:
for d2 in d[d1]:
r[d[d1][d2]] = d1
The result:
>>> r
{'four': 'dict1', 'three': 'dict1', 'two': 'dict2', 'one': 'dict2'}

I don't know about the best solution, but one possibility is reversing the dictionary (so that values becomes keys) and then just doing a normal key lookup. This will reverse a dictionary:
forward_dict = { 'key1': 'val1', 'key2': 'val2'}
reverse_dict = dict([(v,k) for k,v in forward_dict.items()])
So given "val1", I can just do:
reverse_dict["val1"]
to find the corresponding key. There are obvious problems with this solution -- for example, if your values aren't unique, you're going to lose some information.

Write code to reverse the dictionary (i.e. create a new dictionary that maps the values of the old one to the keys of the old one).
Since you seem to be dealing with nested dictionaries, this will obviously be trickier. Figure out the least you need to get your problem solved and code that up (i.e. don't create a solution that will work arbitrary depths of nesting if your problem only deals with dicts in dicts which in turn don't have any dicts)

To handle the nested dictionaries I would do just as senderle's answer states.
However if in the future it does not contain nested dictionaries, be very careful doing a simple reversal. By design the dictionary keys are unique, but the values do not have this requirement.
If you have values that are the same for multiple keys, when reversing the dictionary you will lose all but one of them. And because dictionaries are not sorted, you could lose different data arbitrarily.
Example of reversal working:
>>> d={'dict1': 1, 'dict2': 2, 'dict3': 3, 'dict4': 4}
>>> rd = dict([(v,k) for k,v in d.items()])
>>> print d
{'dict4': 4, 'dict1': 1, 'dict3': 3, 'dict2': 2}
>>> print rd
{1: 'dict1', 2: 'dict2', 3: 'dict3', 4: 'dict4'}
Example of reversal failure: Note that dict4 is lost
>>> d={'dict1': 1, 'dict2': 4, 'dict3': 3, 'dict4': 4}
>>> rd = dict([(v,k) for k,v in d.items()])
>>> print d
{'dict4': 4, 'dict1': 1, 'dict3': 3, 'dict2': 4}
>>> print rd
{1: 'dict1', 3: 'dict3', 4: 'dict2'}

here's an example nobody thinks about: (could be used similarly)
raw_dict = { 'key1': 'val1', 'key2': 'val2', 'key3': 'val1' }
new_dict = {}
for k,v in raw_dict.items():
try: new_dict[v].append(k)
except: new_dict[v] = [k]
result:
>>> new_dict
{'val2': ['key2'], 'val1': ['key3', 'key1']}
maybe not the best of methods, but it works for what I need it for.