Related
I have a nested dictionary where every element can be of any type including a list or
dictionary.
I'm looking for a method to update any key at any depth with a particular value.
(So the replacement occurs if the target value is not a list or dictionary)
e.g
{
'a': 1,
'b': 2,
'c': [{'a': 2, 'b': 3}],
'd': [{'d_d': {'a': 1, 'b': 2}}],
'e': {'a': 4},
}
would become
{
'a': 'xx',
'b': 2,
'c': [{'a': 'xx', 'b': 3}],
'd': [{'d_d': {'a': 'xx', 'b': 2}}],
'e': {'a': 'xx'},
}
where the function takes a dictionary, key and new value like so
update_nested(dict, key='a', value='xx')
Let's look at which parts you need and how to implement them:
iterate over a dictionary
there are three different methods that help you to iterate over a dictionary:
dict.keys()
iterating over all keys in the dict. e.g.
for key in {"Hello": 10, "World", 20}.keys():
print(k)
# output: Hello\nWorld
dict.items()
iterating over all (key, value) tuples in the dict. for k, v in d.items()
dict.values()
iterating over all values in the dict. for v in d.values()
Handle nested dictionaries
When you have nested structures a good concept would recursion
In short: You call the same function in itself with different parameters.
putting both concepts together.
Iterate over the array
If the key is the same as the one you are searching for: Replace the value
If the value is a dict: Call the function again with the value as dict parameter
If the value is a list: Iterate over all items and check if they are dicts. If so handle them like above
Possible final code:
def update_nested(in_dict, key, value):
for k, v in in_dict.items():
if key == k:
in_dict[k] = value
elif isinstance(v, dict):
update_nested(v, key, value)
elif isinstance(v, list):
for o in v:
if isinstance(o, dict):
update_nested(o, key, value)
Extra note:
You should never use a builtin name/type as a variable name. In your case dict. This will override the builtin type and can lead to unexpected behavior.
You can use a recursive function that checks on instances of dict and list:
def nested_update(obj, key, value):
if isinstance(obj, dict):
for k, v in obj.items():
if isinstance(v, (dict, list)):
nested_update(v, key, value)
elif k == key:
obj[k] = value
elif isinstance(obj, list):
for item in obj:
nested_update(item, key, value)
I had a similar problem to update imported JSON values in varying nested dictionaries.
Using #Uli Sotschok's solution, this is my working code:
def changejsonimage(key, value, dictionary):
for k,v in dictionary.items():
if isinstance(v, str) and 'images' in v: #if string and has 'images' in the value
dictionary[k] = 'InlineImage(tpl,"' + v + '")' #change the value
elif isinstance(v, dict): #any value that is a dictionary, loop back into the fn
changejsonimage(key,value,v)
This function updates at any depth and remove all None values so keys can be deleted.
def deep_update(main_dict, update_dict):
main_dict.update([(k, deep_update(main_dict[k], v) if isinstance(main_dict.get(k), dict) and isinstance(v, dict) else v) for k, v in update_dict.items()])
return dict((k, v) for k, v in main_dict.items() if v is not None)
Okay, not sure how to explain this problem but here goes.
This is my dictionary myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
I'm trying to list out the '1003', '1040'.
I've tried using this method:
for i in range(len(a['12/2019'])):
print(a['12/2019'][i].keys())
It kind of works however it returns me
dict_keys(['1003'])
dict_keys(['1040'])
Is there anyway I can just get 1030 and 1040 to be isolated from the ... dict_keys([''])?
You can iterate over the values of the dictionary, and if the value is a list, iterate over the dictionaries in the list, and append all the keys to the result
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}],
'11/2019': '1005', '10/2019': 1234,
'09/2019': [{'1006' : 2}, {'1042' : 3}],
'08/2019': (1,2)}
keys=[]
#Iterate over values
for value in myDictionary.values():
#If value is a list
if isinstance(value, list):
#Iterate over the list
for d in value:
#Add the keys to the result
keys.extend(list(d.keys()))
print(keys)
The output will be
['1003', '1040', '1006', '1042']
You can also do this via a list-comprehension, but it's not really readable, so I would avoid against it
[key for value in myDictionary.values() if isinstance(value, list) for d in value for key in d.keys()]
Iterate over the sub-dicts in the list, grabbing the keys:
for sub_dict in a['12/2019']:
for key, val in sub_dict.items():
print(key)
You can use a list-comprehension:
[k for v in myDictionary.values() for x in v for k in x]
In code:
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
print([k for v in myDictionary.values() for x in v for k in x])
# ['1003', '1040']
You could do this by doing something like this
data = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
# Data is a dictionary of where each value corresponding
# to the key is a list of dictionaries.
for k, v in data.items():
# v is a list of dictionaries
all_keys = []
for value in v:
all_keys.extend(list(value.keys()))
print (all_keys)
The result is ['1003', '1040']
This may be more than you need, but if the data you shared for the example is just a small subset of what you're really looking at, here's a recursive function that can go as deep as you want it to.
Since you seem to want to skip the keys in the outer most level, I put in a spot to allow you to skip over the keys at a given depth. You shouldn't set the level argument manually since the code will just update that for itself.
def list_keys(element, ignore_levels=[], level=1):
"""
Recursivly list keys found in an element.
element: item to inspect.
ignore_levels: list of numeric levels to ignore.
level: recursion depth level.
"""
keys = []
# add keys if we're not ignoring this level and set iterable to values
if isinstance(element, dict):
if level not in ignore_levels:
keys.extend(element.keys())
iterable = element.values()
# if we hve a list or tuple, we can iterate over it
elif isinstance(element, (list, tuple)):
iterable = element
# if we have a single value, it's not a key and there's nothing to iterate
else:
iterable = []
# Iterate over elements and append any keys found
for i in iterable:
subkeys = list_keys(i, ignore_levels, level + 1)
if subkeys:
keys.extend(subkeys)
return keys
d = {'12/2019': [{'1003': 2}, {'1040': 3}]}
print(list_keys(d, ignore_levels=[1]))
# ['1003', '1040']
I have a dict as below
{"low":[18,12,9],"medium":[6,3],"high":[2,1],"final":[0]}
and I want to search for a number in this dict and get its respective 'key'
Eg: for 12, i need to return 'low'. slly for 2, return 'high'
You can use a dictionary comprehension for this.
dict = {"low":[18,12,9],"medium":[6,3],"high":[2,1],"final":[0]}
key = {k:v for k, v in dict.items() if 12 in v}
Output
In[1]: key.popitem()[0]
Out[1] : 12
This is a job for next.
my_d = {"low":[18,12,9],"medium":[6,3],"high":[2,1],"final":[0]}
target = 12
res = next((k for k, v in my_d.items() if target in v), 'N\A')
print(res) # low
Note that if your target value exists in more than one keys, this code will return one of them at random1. If that might by the case and depending on the problem you are working on it may be wiser to get all matching keys instead. To do that, use:
res = [k for k, v in my_d.items() if target in v]
1Actually more like in an uncontrolled fashion.
def getKey(number):
for key, value in d.iteritems():
if number in value:
return key
dict = {"low":[18,12,9],"medium":[6,3],"high":[2,1],"final":[0]}
def search_key(val):
for key, value in dict.iteritems():
for i in value:
if i == val:
print "the key is:"+key
return key
#pass any value for which you want to get the key
search_key(9)
I need to write a function which is doing following work
Find a string as value in a dictionary of dictionaries and return its key
(1st key if found in main dictionary, 2nd key if found in sub dictionary).
Source Code
Here is the function which I try to implement, but it works incorrect as I can't find any answer of how to convert list into dictionary as in this case the following error occurs
for v, k in l:
ValueError: need more than 1 value to unpack
def GetKeyFromDictByValue(self, dictionary, value_to_find):
""""""
key_list = [k for (k, v) in dictionary.items() if v == value_to_find]
if key_list.__len__() is not 0:
return key_list[0]
else:
l = [s for s in dictionary.values() if ":" in str(s)]
d = defaultdict(list)
for v, k in l:
d[k].append(v)
print d
dict = {'a': {'a1': 'a2'}, "aa": "aa1", 'aaa': {'aaa1': 'aaa2'}}
print GetKeyFromDictByValue(dict, "a2")
I must do this on Python 2.5
You created a list of only the dictionary values, but then try to loop over it as if it already contains both keys and values of those dictionaries. Perhaps you wanted to loop over each matched dictionary?
l = [v for v in dictionary.values() if ":" in str(v)]
d = defaultdict(list)
for subdict in l:
for k, v in subdict.items():
I'd instead flatten the structure:
def flatten(dictionary):
for key, value in dictionary.iteritems():
if isinstance(value, dict):
# recurse
for res in flatten(value):
yield res
else:
yield key, value
then just search:
def GetKeyFromDictByValue(self, dictionary, value_to_find):
for key, value in flatten(dictionary):
if value == value_to_find:
return key
Demo:
>>> sample = {'a': {'a1': 'a2'}, "aa": "aa1", 'aaa': {'aaa1': 'aaa2'}}
>>> GetKeyFromDictByValue(None, sample, "a2")
'a1'
Let's say I have a pretty complex dictionary.
{'fruit':'orange','colors':{'dark':4,'light':5}}
Anyway, my objective is to scan every key in this complex multi-level dictionary. Then, append "abc" to the end of each key.
So that it will be:
{'fruitabc':'orange','colorsabc':{'darkabc':4,'lightabc':5}}
How would you do that?
Keys cannot be changed. You will need to add a new key with the modified value then remove the old one, or create a new dict with a dict comprehension or the like.
For example like this:
def appendabc(somedict):
return dict(map(lambda (key, value): (str(key)+"abc", value), somedict.items()))
def transform(multilevelDict):
new = appendabc(multilevelDict)
for key, value in new.items():
if isinstance(value, dict):
new[key] = transform(value)
return new
print transform({1:2, "bam":4, 33:{3:4, 5:7}})
This will append "abc" to each key in the dictionary and any value that is a dictionary.
EDIT: There's also a really cool Python 3 version, check it out:
def transform(multilevelDict):
return {str(key)+"abc" : (transform(value) if isinstance(value, dict) else value) for key, value in multilevelDict.items()}
print(transform({1:2, "bam":4, 33:{3:4, 5:7}}))
I use the following utility function that I wrote that takes a target dict and another dict containing the translation and switches all the keys according to it:
def rename_keys(d, keys):
return dict([(keys.get(k), v) for k, v in d.items()])
So with the initial data:
data = { 'a' : 1, 'b' : 2, 'c' : 3 }
translation = { 'a' : 'aaa', 'b' : 'bbb', 'c' : 'ccc' }
We get the following:
>>> data
{'a': 1, 'c': 3, 'b': 2}
>>> rename_keys(data, translation)
{'aaa': 1, 'bbb': 2, 'ccc': 3}
>>> mydict={'fruit':'orange','colors':{'dark':4,'light':5}}
>>> def f(mydict):
... return dict((k+"abc",f(v) if hasattr(v,'keys') else v) for k,v in mydict.items())
...
>>> f(mydict)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
My understanding is that you can't change the keys, and that you would need to make a new set of keys and assign their values to the ones the original keys were pointing to.
I'd do something like:
def change_keys(d):
if type(d) is dict:
return dict([(k+'abc', change_keys(v)) for k, v in d.items()])
else:
return d
new_dict = change_keys(old_dict)
here's a tight little function:
def keys_swap(orig_key, new_key, d):
d[new_key] = d.pop(orig_key)
for your particular problem:
def append_to_dict_keys(appendage, d):
#note that you need to iterate through the fixed list of keys, because
#otherwise we will be iterating through a never ending key list!
for each in d.keys():
if type(d[each]) is dict:
append_to_dict_keys(appendage, d[each])
keys_swap(each, str(each) + appendage, d)
append_to_dict_keys('abc', d)
#! /usr/bin/env python
d = {'fruit':'orange', 'colors':{'dark':4,'light':5}}
def add_abc(d):
newd = dict()
for k,v in d.iteritems():
if isinstance(v, dict):
v = add_abc(v)
newd[k + "abc"] = v
return newd
d = add_abc(d)
print d
Something like that
def applytoallkeys( dic, func ):
def yielder():
for k,v in dic.iteritems():
if isinstance( v, dict):
yield func(k), applytoallkeys( v, func )
else:
yield func(k), v
return dict(yielder())
def appendword( s ):
def appender( x ):
return x+s
return appender
d = {'fruit':'orange','colors':{'dark':4,'light':5}}
print applytoallkeys( d, appendword('asd') )
I kinda like functional style, you can read just the last line and see what it does ;-)
You could do this with recursion:
import collections
in_dict={'fruit':'orange','colors':{'dark':4,'light':5}}
def transform_dict(d):
out_dict={}
for k,v in d.iteritems():
k=k+'abc'
if isinstance(v,collections.MutableMapping):
v=transform_dict(v)
out_dict[k]=v
return out_dict
out_dict=transform_dict(in_dict)
print(out_dict)
# {'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
you should also consider that there is the possibility of nested dicts in nested lists, which will not be covered by the above solutions. This function ads a prefix and/or a postfix to every key within the dict.
def transformDict(multilevelDict, prefix="", postfix=""):
"""adds a prefix and/or postfix to every key name in a dict"""
new_dict = multilevelDict
if prefix != "" or postfix != "":
new_key = "%s#key#%s" % (prefix, postfix)
new_dict = dict(map(lambda (key, value): (new_key.replace('#key#', str(key)), value), new_dict.items()))
for key, value in new_dict.items():
if isinstance(value, dict):
new_dict[key] = transformDict(value, prefix, postfix)
elif isinstance(value, list):
for index, item in enumerate(value):
if isinstance(item, dict):
new_dict[key][index] = transformDict(item, prefix, postfix)
return new_dict
for k in theDict: theDict[k+'abc']=theDict.pop(k)
I use this for converting docopt POSIX-compliant command-line keys to PEP8 keys
(e.g. "--option" --> "option", "" --> "option2", "FILENAME" --> "filename")
arguments = docopt.docopt(__doc__) # dictionary
for key in arguments.keys():
if re.match('.*[-<>].*', key) or key != key.lower():
value = arguments.pop(key)
newkey = key.lower().translate(None, '-<>')
arguments[newkey] = value
Hi I'm a new user but finding an answer for same question, I can't get anything fully functional to my problem, I make this little piece of cake with a full nested replace of keys, you can send list with dict or dict.
Finally your dicts can have list with dict or more dict nested and it is all replaced with your new key needs.
To indicate who key want replace with a new key use "to" parameter sending a dict.
See at end my little example.
P/D: Sorry my bad english. =)
def re_map(value, to):
"""
Transform dictionary keys to map retrieved on to parameters.
to parameter should have as key a key name to replace an as value key name
to new dictionary.
this method is full recursive to process all levels of
#param value: list with dictionary or dictionary
#param to: dictionary with re-map keys
#type to: dict
#return: list or dict transformed
"""
if not isinstance(value, dict):
if not isinstance(value, list):
raise ValueError(
"Only dict or list with dict inside accepted for value argument.") # #IgnorePep8
if not isinstance(to, dict):
raise ValueError("Only dict accepted for to argument.")
def _re_map(value, to):
if isinstance(value, dict):
# Re map dictionary key.
# If key of original dictionary is not in "to" dictionary use same
# key otherwise use re mapped key on new dictionary with already
# value.
return {
to.get(key) or key: _re_map(dict_value, to)
for key, dict_value in value.items()
}
elif isinstance(value, list):
# if value is a list iterate it a call _re_map again to parse
# values on it.
return [_re_map(item, to) for item in value]
else:
# if not dict or list only return value.
# it can be string, integer or others.
return value
result = _re_map(value, to)
return result
if __name__ == "__main__":
# Sample test of re_map method.
# -----------------------------------------
to = {"$id": "id"}
x = []
for i in range(100):
x.append({
"$id": "first-dict",
"list_nested": [{
"$id": "list-dict-nested",
"list_dic_nested": [{
"$id": "list-dict-list-dict-nested"
}]
}],
"dict_nested": {
"$id": "non-nested"
}
})
result = re_map(x, to)
print(str(result))
A functional (and flexible) solution: this allows an arbitrary transform to be applied to keys (recursively for embedded dicts):
def remap_keys(d, keymap_f):
"""returns a new dict by recursively remapping all of d's keys using keymap_f"""
return dict([(keymap_f(k), remap_keys(v, keymap_f) if isinstance(v, dict) else v)
for k,v in d.items()])
Let's try it out; first we define our key transformation function, then apply it to the example:
def transform_key(key):
"""whatever transformation you'd like to apply to keys"""
return key + "abc"
remap_keys({'fruit':'orange','colors':{'dark':4,'light':5}}, transform_key)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
(note: if you're still on Python 2.x, you'll need to replace d.items() on the last line with d.iteritems() -- thanks to #Rudy for reminding me to update this post for Python 3).
Based on #AndiDog's python 3 version and similar to #sxc731's version but with a flag for whether to apply it recursively:
def transform_keys(dictionary, key_fn, recursive=True):
"""
Applies function to keys and returns as a new dictionary.
Example of key_fn:
lambda k: k + "abc"
"""
return {key_fn(key): (transform_keys(value, key_fn=key_fn, recursive=recursive)
if recursive and isinstance(value, dict) else value)
for key, value in dictionary.items()}