I have obj like this
{hello: 'world', "foo.0.bar": v1, "foo.0.name": v2, "foo.1.bar": v3}
It should be expand to
{ hello: 'world', foo: [{'bar': v1, 'name': v2}, {bar: v3}]}
I wrote code below, splite by '.', remove old key, append new key if contains '.', but it said RuntimeError: dictionary changed size during iteration
def expand(obj):
for k in obj.keys():
expandField(obj, k, v)
def expandField(obj, f, v):
parts = f.split('.')
if(len(parts) == 1):
return
del obj[f]
for i in xrange(0, len(parts) - 1):
f = parts[i]
currobj = obj.get(f)
if (currobj == None):
nextf = parts[i + 1]
currobj = obj[f] = re.match(r'\d+', nextf) and [] or {}
obj = currobj
obj[len(parts) - 1] = v
for k, v in obj.iteritems():
RuntimeError: dictionary changed size during iteration
Like the message says: you changed the number of entries in obj inside of expandField() while in the middle of looping over this entries in expand.
You might try instead creating a new dictionary of the form you wish, or somehow recording the changes you want to make, and then making them AFTER the loop is done.
You might want to copy your keys in a list and iterate over your dict using the latter, eg:
def expand(obj):
keys = list(obj.keys()) # freeze keys iterator into a list
for k in keys:
expandField(obj, k, v)
I let you analyse if the resulting behavior suits your expected results.
Edited as per comments, thank you !
I had a similar issue with wanting to change the dictionary's structure (remove/add) dicts within other dicts.
For my situation I created a deepcopy of the dict. With a deepcopy of my dict, I was able to iterate through and remove keys as needed.Deepcopy - PythonDoc
A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.
Hope this helps!
For those experiencing
RuntimeError: dictionary changed size during iteration
also make sure you're not iterating through a defaultdict when trying to access a non-existent key! I caught myself doing that inside the for loop, which caused the defaultdict to create a default value for this key, causing the aforementioned error.
The solution is to convert your defaultdict to dict before looping through it, i.e.
d = defaultdict(int)
d_new = dict(d)
or make sure you're not adding/removing any keys while iterating through it.
Rewriting this part
def expand(obj):
for k in obj.keys():
expandField(obj, k, v)
to the following
def expand(obj):
keys = obj.keys()
for k in keys:
if k in obj:
expandField(obj, k, v)
shall make it work.
Related
I am attempting to remove key-value pairs from a dict when a sub-dictionary matches values from another dictionary.
Example set-up:
e = {'a':{'aa':'yes'}, 'b':{'ac':'no'}, 'a':{'aa':'yes'}}
f = {'a':{'aa':'yes'}, 'e':{'ab':'no'}, 'a':{'aa':'yes'}}
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') == v.get('aa'):
e.pop(keys)
RuntimeError: dictionary changed size during iteration
Expected result:
#from
e = {'a':{'aa':'yes'}, 'b':{'ac':'no'}, 'a':{'aa':'yes'}}
#to
e = {'b':{'ac':'no'}}
With single dict comprehension:
e = {k:v for k, v in e.items() if v.items() != f.get(k, {}).items()}
{'b': {'ac': 'no'}}
dict.get(key[, default]) allows you to set the needed(or preferred) default value returned for the key in dict
In general, you should not add or remove items from iterables that you are currently iterating over.
As you've been told, you can't modify the length of a thing while you're iterating it. There are a few options here, such as:
Saving a list of what you want to remove, then doing it later:
to_remove = []
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') == v.get('aa'):
to_remove.append(keys)
for tr in to_remove:
e.pop(tr)
Cloning the object, so that what you're iterating does not change but the original object can be modified. This is even more memory expensive than the previous however.
for keys, values in dict(e).items(): # or list(e.items())
for k, v in f.items():
if values.get('aa') == v.get('aa'):
e.pop(keys)
You could also, in your case, simply create a new object:
g = {}
for keys, values in e.items():
for k, v in f.items():
if values.get('aa') != v.get('aa'):
g[keys] = values
I have a dictionary where each key has several lists of data as its values like this
myDict = {'data1' : ['data_d','dataD']['data_e','dataE']['data_f','dataF']}
I want to be able to input one of the values in the list and then be given the key. This is so I can get the other value in the list now that I have the key.
I've tried
dataKey = (list(myDict.keys())[list(myDict.values()).index(dataD)])
but that didn't work
I've also tried
for k, v in myDict.items():
if 'dataD' in v:
print k
but that didn't work as well.
Side question, in the questions that I've looked through, I see people using the variable k and v a lot even without the OP mentioning them, so I am wondering if k and v are already set variable in dictionaries?
Your second attempt was almost right, but a nested for loop is needed to traverse the list-of-lists:
myDict = {'data1' : [['data_d','dataD'], ['data_e','dataE'], ['data_f','dataF']]}
for key, value in myDict.items():
for sublist in value:
if 'dataD' in sublist:
print(key) # -> data1
Using variables named k, and v with dictionaries is purely optional and aren't special properties—other than being very short abbreviations for "key" and "value".
Note that if only one match is ever expected to occur, the code could be made more efficient by stopping the search after one is found. Here's one way of doing that:
target = 'dataD'
try:
for key, value in myDict.items():
for sublist in value:
if target in sublist:
print(key) # -> data1
raise StopIteration # found, so quit searching
except StopIteration:
pass # found
else:
print('{} not found'.format(target))
if they are all going to be lists then you can do something like this (if i am understanding correctly)
myDict = {
'data1': [['data_d','dataD'], ['data_e','dataE'], ['data_f','dataF']],
}
def find(dic, item):
for k, v in dic.items():
for ls in v:
if item in ls:
return k
return False
print(find(myDict, "data_d"))
# OUT [data1]
I had the following dictionary:
ref_range = range(0,100)
aas = list("ACDEFGHIKLMNPQRSTVWXY*")
new_dict = {}
new_dict = new_dict.fromkeys(ref_range,{k:0 for k in aas})
Then I added a 1 to a specific key
new_dict[30]['G'] += 1
>>>new_dict[30]['G']
1
but
>>>new_dict[31]['G']
1
What is going on here? I only incremented the nested key 30, 'G' by one.
Note: If I generate the dictionary this way:
new_dict = {}
for i in ref_range:
new_dict[i] = {a:0 for a in aas}
Everything behaves fine. I think this is a similar question here, but I wanted to know a bit about why this happening rather than how to solve it.
fromkeys(S, v) sets all of the keys in S to the same value v. Meaning that all of the keys in your dictionary new_dict refer to the same dictionary object, not to their own copies of that dictionary.
To set each to a different dict object you cannot use fromkeys. You need to just set each key to a new dict in a loop.
Besides what you have you could also do
{i: {a: 0 for a in aas} for i in ref_range}
So, I've been searching endlessly for something similiar to Lua's "Generic For Loop" in Python.
I've been working on a simple text based game in Python, and I've been working with dictionaries a lot.
Here is something I'm looking for (in Lua):
Dictionary = {
"Red" = "There is some red paint on the walls.",
"Green" = "There is a little bit of green paint on the floor.",
}
for i, v in pairs(Dictionary) do
print(i, v)
end
What this will do is, go through the dictionary, then print out the INDEX and the VALUE. How would I do something like this in Python?
I know there is this:
for i in Dictionary:
print(i)
But that just prints the INDEX. I would like to access both the INDEX and the VALUE. Something like:
for i, v in Dictionary:
print(i, v)
Any help is appreciated.
You're looking for items. Iterating over a dict just gives you the keys, so you'd have to do:
for key in my_dict:
x = my_dict[key]
What you want is this:
for key, value in my_dict.items():
# do something
two ways:
for i, v in Dictionary.items():
print(i, v) #outputs pairs as key value
for tup in Dictionary.items(): #same thing
print(tup) # outputs pairs as (key,value)
or
for key in Dictionary:
print(key,Dictionary[key])
EDIT RESPONSE TO COMMENT:
>>> d = {1:1,2:2,3:3,4:4}
>>> for item in d.items(): print(item)
(1, 1)
(2, 2)
(3, 3)
(4, 4)
>>> for key,val in d.items(): print(key,val)
1 1
2 2
3 3
4 4
this is because in the first loop, item is a tuple and the __repr__ for a tuple has the brackets and commas as part of it where as the second loop splits the tuple into two seperate variables. print then automatically adds a space delimiter in between each parameter passed in the print function.
As explained by Two-Bit Alchemist:
In case it's not entirely clear still, in the tup formulation you'd access the key and value as tup[0] and tup[1], respectively. for key, val in my_dict.items(): ... and for tup in my_dict.items(): key, val = tup is the same setup. The point is you can use tuple unpacking just fine inline in a for loop.
The items method (or in Py2, viewitems or iteritems to avoid making a whole new list containing copies of the dict key/value pairs) is the way to go:
for k, v in Dictionary.items(): # For performance, use .viewitems() on Py2.7, .items() on Py3.x
print(k, v)
I have a dictionary in python like:
dict = {'dog':['milo','otis','laurel','hardy'],
'cat':['bob','joe'],
'milo':['otis','laurel','hardy','dog'],
'hardy':['dog'],'bob':['joe','cat']}
...and I want to identify if a key exists elsewhere in a dictionary (in some other list of values). There are other questions I could find that want to know if an item simply exists in the dictionary, but this is not my question. The same goes for items in each list of values, to identify items that do not exist in OTHER keys and their associated values in the dictionary.
In the above example, the idea is that dogs and cats are not equal, their keys/values have nothing in common with those that come from cats. Ideally, a second dictionary would be created that collects all of those associated with each unique cluster:
unique.dict = {'cluster1':['dog','milo','otis','laurel','hardy'],
'cluster2':['cat','bob','joe'] }
This is a follow up question to In Python, count unique key/value pairs in a dictionary
It appears that the relationship is symmetric, but your data is not (e.g. there is no key 'otis'). The first part involves making it symmetric, so it won't matter where we start.
(If your data actually is symmetric, then skip that part.)
Python 2.7
from collections import defaultdict
data = {'dog':['milo','otis','laurel','hardy'],'cat':['bob','joe'],'milo':['otis','laurel','hardy','dog'],'hardy':['dog'],'bob':['joe','cat']}
# create symmetric version of data
d = defaultdict(list)
for key, values in data.iteritems():
for value in values:
d[key].append(value)
d[value].append(key)
visited = set()
def connected(key):
result = []
def connected(key):
if key not in visited:
visited.add(key)
result.append(key)
map(connected, d[key])
connected(key)
return result
print [connected(key) for key in d if key not in visited]
Python 3.3
from collections import defaultdict
data = {'dog':['milo','otis','laurel','hardy'],'cat':['bob','joe'],'milo':['otis','laurel','hardy','dog'],'hardy':['dog'],'bob':['joe','cat']}
# create symmetric version of data
d = defaultdict(list)
for key, values in data.items():
for value in values:
d[key].append(value)
d[value].append(key)
visited = set()
def connected(key):
visited.add(key)
yield key
for value in d[key]:
if key not in visited:
yield from connected(value)
print([list(connected(key)) for key in d if key not in visited])
Result
[['otis', 'milo', 'laurel', 'dog', 'hardy'], ['cat', 'bob', 'joe']]
Performance
O(n), where n is the total number of keys and values in data (in your case, 17 if I count correctly).
I'm taking "in some other list of values" literally, to mean that a key existing in its own set of values is OK. If not, that would make things slightly simpler, but you should be able to adjust the code yourself, so I won't write it both ways.
If you insist on using this data structure, you have to do it by brute force:
def does_key_exist_in_other_value(d, key):
for k, v in d.items():
if k != key and key in v:
return True
You could of course condense that into a one-liner with a genexpr and any:
return any(key in v for k, v in d.items() if k != key)
But a smarter thing to do would be to use a better data structure. At the very least use sets instead of lists as your values (which wouldn't simplify your code, but would make it a lot faster—if you have K keys and V total elements across your values, it would run in O(K) instead of O(KV).
But really, if you want to look things up, build a dict to look things up in:
inv_d = defaultdict(set)
for key, value in d.items():
for v in value:
inv_d[v].add(key)
And now, your code is just:
def does_key_exist_in_other_value(inv_d, key):
return inv_d[key] != {key}