Related
for k, v in d.iteritems():
if type(v) is dict:
for t, c in v.iteritems():
print "{0} : {1}".format(t, c)
I'm trying to loop through a dictionary and print out all key value pairs where the value is not a nested dictionary. If the value is a dictionary I want to go into it and print out its key value pairs...etc. Any help?
EDIT
How about this? It still only prints one thing.
def printDict(d):
for k, v in d.iteritems():
if type(v) is dict:
printDict(v)
else:
print "{0} : {1}".format(k, v)
Full Test Case
Dictionary:
{u'xml': {u'config': {u'portstatus': {u'status': u'good'}, u'target': u'1'},
u'port': u'11'}}
Result:
xml : {u'config': {u'portstatus': {u'status': u'good'}, u'target': u'1'}, u'port': u'11'}
As said by Niklas, you need recursion, i.e. you want to define a function to print your dict, and if the value is a dict, you want to call your print function using this new dict.
Something like :
def myprint(d):
for k, v in d.items():
if isinstance(v, dict):
myprint(v)
else:
print("{0} : {1}".format(k, v))
There are potential problems if you write your own recursive implementation or the iterative equivalent with stack. See this example:
dic = {}
dic["key1"] = {}
dic["key1"]["key1.1"] = "value1"
dic["key2"] = {}
dic["key2"]["key2.1"] = "value2"
dic["key2"]["key2.2"] = dic["key1"]
dic["key2"]["key2.3"] = dic
In the normal sense, nested dictionary will be a n-nary tree like data structure. But the definition doesn't exclude the possibility of a cross edge or even a back edge (thus no longer a tree). For instance, here key2.2 holds to the dictionary from key1, key2.3 points to the entire dictionary(back edge/cycle). When there is a back edge(cycle), the stack/recursion will run infinitely.
root<-------back edge
/ \ |
_key1 __key2__ |
/ / \ \ |
|->key1.1 key2.1 key2.2 key2.3
| / | |
| value1 value2 |
| |
cross edge----------|
If you print this dictionary with this implementation from Scharron
def myprint(d):
for k, v in d.items():
if isinstance(v, dict):
myprint(v)
else:
print "{0} : {1}".format(k, v)
You would see this error:
> RuntimeError: maximum recursion depth exceeded while calling a Python object
The same goes with the implementation from senderle.
Similarly, you get an infinite loop with this implementation from Fred Foo:
def myprint(d):
stack = list(d.items())
while stack:
k, v = stack.pop()
if isinstance(v, dict):
stack.extend(v.items())
else:
print("%s: %s" % (k, v))
However, Python actually detects cycles in nested dictionary:
print dic
{'key2': {'key2.1': 'value2', 'key2.3': {...},
'key2.2': {'key1.1': 'value1'}}, 'key1': {'key1.1': 'value1'}}
"{...}" is where a cycle is detected.
As requested by Moondra this is a way to avoid cycles (DFS):
def myprint(d):
stack = list(d.items())
visited = set()
while stack:
k, v = stack.pop()
if isinstance(v, dict):
if k not in visited:
stack.extend(v.items())
else:
print("%s: %s" % (k, v))
visited.add(k)
Since a dict is iterable, you can apply the classic nested container iterable formula to this problem with only a couple of minor changes. Here's a Python 2 version (see below for 3):
import collections
def nested_dict_iter(nested):
for key, value in nested.iteritems():
if isinstance(value, collections.Mapping):
for inner_key, inner_value in nested_dict_iter(value):
yield inner_key, inner_value
else:
yield key, value
Test:
list(nested_dict_iter({'a':{'b':{'c':1, 'd':2},
'e':{'f':3, 'g':4}},
'h':{'i':5, 'j':6}}))
# output: [('g', 4), ('f', 3), ('c', 1), ('d', 2), ('i', 5), ('j', 6)]
In Python 2, It might be possible to create a custom Mapping that qualifies as a Mapping but doesn't contain iteritems, in which case this will fail. The docs don't indicate that iteritems is required for a Mapping; on the other hand, the source gives Mapping types an iteritems method. So for custom Mappings, inherit from collections.Mapping explicitly just in case.
In Python 3, there are a number of improvements to be made. As of Python 3.3, abstract base classes live in collections.abc. They remain in collections too for backwards compatibility, but it's nicer having our abstract base classes together in one namespace. So this imports abc from collections. Python 3.3 also adds yield from, which is designed for just these sorts of situations. This is not empty syntactic sugar; it may lead to faster code and more sensible interactions with coroutines.
from collections import abc
def nested_dict_iter(nested):
for key, value in nested.items():
if isinstance(value, abc.Mapping):
yield from nested_dict_iter(value)
else:
yield key, value
Alternative iterative solution:
def myprint(d):
stack = d.items()
while stack:
k, v = stack.pop()
if isinstance(v, dict):
stack.extend(v.iteritems())
else:
print("%s: %s" % (k, v))
Slightly different version I wrote that keeps track of the keys along the way to get there
def print_dict(v, prefix=''):
if isinstance(v, dict):
for k, v2 in v.items():
p2 = "{}['{}']".format(prefix, k)
print_dict(v2, p2)
elif isinstance(v, list):
for i, v2 in enumerate(v):
p2 = "{}[{}]".format(prefix, i)
print_dict(v2, p2)
else:
print('{} = {}'.format(prefix, repr(v)))
On your data, it'll print
data['xml']['config']['portstatus']['status'] = u'good'
data['xml']['config']['target'] = u'1'
data['xml']['port'] = u'11'
It's also easy to modify it to track the prefix as a tuple of keys rather than a string if you need it that way.
Here is pythonic way to do it. This function will allow you to loop through key-value pair in all the levels. It does not save the whole thing to the memory but rather walks through the dict as you loop through it
def recursive_items(dictionary):
for key, value in dictionary.items():
if type(value) is dict:
yield (key, value)
yield from recursive_items(value)
else:
yield (key, value)
a = {'a': {1: {1: 2, 3: 4}, 2: {5: 6}}}
for key, value in recursive_items(a):
print(key, value)
Prints
a {1: {1: 2, 3: 4}, 2: {5: 6}}
1 {1: 2, 3: 4}
1 2
3 4
2 {5: 6}
5 6
A alternative solution to work with lists based on Scharron's solution
def myprint(d):
my_list = d.iteritems() if isinstance(d, dict) else enumerate(d)
for k, v in my_list:
if isinstance(v, dict) or isinstance(v, list):
myprint(v)
else:
print u"{0} : {1}".format(k, v)
I am using the following code to print all the values of a nested dictionary, taking into account where the value could be a list containing dictionaries. This was useful to me when parsing a JSON file into a dictionary and needing to quickly check whether any of its values are None.
d = {
"user": 10,
"time": "2017-03-15T14:02:49.301000",
"metadata": [
{"foo": "bar"},
"some_string"
]
}
def print_nested(d):
if isinstance(d, dict):
for k, v in d.items():
print_nested(v)
elif hasattr(d, '__iter__') and not isinstance(d, str):
for item in d:
print_nested(item)
elif isinstance(d, str):
print(d)
else:
print(d)
print_nested(d)
Output:
10
2017-03-15T14:02:49.301000
bar
some_string
Your question already has been answered well, but I recommend using isinstance(d, collections.Mapping) instead of isinstance(d, dict). It works for dict(), collections.OrderedDict(), and collections.UserDict().
The generally correct version is:
def myprint(d):
for k, v in d.items():
if isinstance(v, collections.Mapping):
myprint(v)
else:
print("{0} : {1}".format(k, v))
Iterative solution as an alternative:
def traverse_nested_dict(d):
iters = [d.iteritems()]
while iters:
it = iters.pop()
try:
k, v = it.next()
except StopIteration:
continue
iters.append(it)
if isinstance(v, dict):
iters.append(v.iteritems())
else:
yield k, v
d = {"a": 1, "b": 2, "c": {"d": 3, "e": {"f": 4}}}
for k, v in traverse_nested_dict(d):
print k, v
Here's a modified version of Fred Foo's answer for Python 2. In the original response, only the deepest level of nesting is output. If you output the keys as lists, you can keep the keys for all levels, although to reference them you need to reference a list of lists.
Here's the function:
def NestIter(nested):
for key, value in nested.iteritems():
if isinstance(value, collections.Mapping):
for inner_key, inner_value in NestIter(value):
yield [key, inner_key], inner_value
else:
yield [key],value
To reference the keys:
for keys, vals in mynested:
print(mynested[keys[0]][keys[1][0]][keys[1][1][0]])
for a three-level dictionary.
You need to know the number of levels before to access multiple keys and the number of levels should be constant (it may be possible to add a small bit of script to check the number of nesting levels when iterating through values, but I haven't yet looked at this).
I find this approach a bit more flexible, here you just providing generator function that emits key, value pairs and can be easily extended to also iterate over lists.
def traverse(value, key=None):
if isinstance(value, dict):
for k, v in value.items():
yield from traverse(v, k)
else:
yield key, value
Then you can write your own myprint function, then would print those key value pairs.
def myprint(d):
for k, v in traverse(d):
print(f"{k} : {v}")
A test:
myprint({
'xml': {
'config': {
'portstatus': {
'status': 'good',
},
'target': '1',
},
'port': '11',
},
})
Output:
status : good
target : 1
port : 11
I tested this on Python 3.6.
Nested dictionaries looping using isinstance() and yield function.
**isinstance is afunction that returns the given input and reference is true or false as in below case dict is true so it go for iteration.
**Yield is used to return from a function without destroying the states of its local variable and when the function is called, the execution starts from the last yield statement. Any function that contains a yield keyword is termed a generator.
students= {'emp1': {'name': 'Bob', 'job': 'Mgr'},
'emp2': {'name': 'Kim', 'job': 'Dev','emp3': {'namee': 'Saam', 'j0ob': 'Deev'}},
'emp4': {'name': 'Sam', 'job': 'Dev'}}
def nested_dict_pairs_iterator(dict_obj):
for key, value in dict_obj.items():
# Check if value is of dict type
if isinstance(value, dict):
# If value is dict then iterate over all its values
for pair in nested_dict_pairs_iterator(value):
yield (key, *pair)
else:
# If value is not dict type then yield the value
yield (key, value)
for pair in nested_dict_pairs_iterator(students):
print(pair)
For a ready-made solution install ndicts
pip install ndicts
Import a NestedDict in your script
from ndicts.ndicts import NestedDict
Initialize
dictionary = {
u'xml': {
u'config': {
u'portstatus': {u'status': u'good'},
u'target': u'1'
},
u'port': u'11'
}
}
nd = NestedDict(dictionary)
Iterate
for key, value in nd.items():
print(key, value)
While the original solution from #Scharron is beautiful and simple, it cannot handle the list very well:
def myprint(d):
for k, v in d.items():
if isinstance(v, dict):
myprint(v)
else:
print("{0} : {1}".format(k, v))
So this code can be slightly modified like this to handle list in elements:
def myprint(d):
for k, v in d.items():
if isinstance(v, dict):
myprint(v)
elif isinstance(v, list):
for i in v:
myprint(i)
else:
print("{0} : {1}".format(k, v))
These answers work for only 2 levels of sub-dictionaries. For more try this:
nested_dict = {'dictA': {'key_1': 'value_1', 'key_1A': 'value_1A','key_1Asub1': {'Asub1': 'Asub1_val', 'sub_subA1': {'sub_subA1_key':'sub_subA1_val'}}},
'dictB': {'key_2': 'value_2'},
1: {'key_3': 'value_3', 'key_3A': 'value_3A'}}
def print_dict(dictionary):
dictionary_array = [dictionary]
for sub_dictionary in dictionary_array:
if type(sub_dictionary) is dict:
for key, value in sub_dictionary.items():
print("key=", key)
print("value", value)
if type(value) is dict:
dictionary_array.append(value)
print_dict(nested_dict)
You can print recursively with a dictionary comprehension:
def print_key_pairs(d):
{k: print_key_pairs(v) if isinstance(v, dict) else print(f'{k}: {v}') for k, v in d.items()}
For your test case this is the output:
>>> print_key_pairs({u'xml': {u'config': {u'portstatus': {u'status': u'good'}, u'target': u'1'}, u'port': u'11'}})
status: good
target: 1
port: 11
Returns a tuple of each key and value and the key contains the full path
from typing import Mapping, Tuple, Iterator
def traverse_dict(nested: Mapping, parent_key="", keys_to_not_traverse_further=tuple()) -> Iterator[Tuple[str, str]]:
"""Each key is joined with it's parent using dot as a separator.
Once a `parent_key` matches `keys_to_not_traverse_further`
it will no longer find its child dicts.
"""
for key, value in nested.items():
if isinstance(value, abc.Mapping) and key not in keys_to_not_traverse_further:
yield from traverse_dict(value, f"{parent_key}.{key}", keys_to_not_traverse_further)
else:
yield f"{parent_key}.{key}", value
Let's test it
my_dict = {
"isbn": "123-456-222",
"author": {"lastname": "Doe", "firstname": "Jane"},
"editor": {"lastname": "Smith", "firstname": "Jane"},
"title": "The Ultimate Database Study Guide",
"category": ["Non-Fiction", "Technology"],
"first": {
"second": {"third": {"fourth": {"blah": "yadda"}}},
"fifth": {"sixth": "seventh"},
},
}
for k, v in traverse_dict(my_dict):
print(k, v)
Returns
.isbn 123-456-222
.author.lastname Doe
.author.firstname Jane
.editor.lastname Smith
.editor.firstname Jane
.title The Ultimate Database Study Guide
.category ['Non-Fiction', 'Technology']
.first.second.third.fourth.blah yadda
.first.fifth.sixth seventh
If you don't care about some child dicts e.g names in this case then
use the keys_to_not_traverse_further
for k, v in traverse_dict(my_dict, parent_key="", keys_to_not_traverse_further=("author","editor")):
print(k, v)
Returns
.isbn 123-456-222
.author {'lastname': 'Doe', 'firstname': 'Jane'}
.editor {'lastname': 'Smith', 'firstname': 'Jane'}
.title The Ultimate Database Study Guide
.category ['Non-Fiction', 'Technology']
.first.second.third.fourth.blah yadda
.first.fifth.sixth seventh
I have a Python object with multiple layers of dicts and lists that contain keys that I need to get the values from. I found an answer using recursive generators that will allow me to pull the value of one key, but not multiple keys. Here's the code:
with open('data.json') as f:
json_data = json.load(f)
def find_key(obj, key):
if isinstance(obj, dict):
yield from iter_dict(obj, key, [])
elif isinstance(obj, list):
yield from iter_list(obj, key, [])
def iter_dict(d, key, indices):
for k, v in d.items():
if k == key:
yield indices + [k], v
if isinstance(v, dict):
yield from iter_dict(v, key, indices + [k])
elif isinstance(v, list):
yield from iter_list(v, key, indices + [k])
def iter_list(seq, key, indices):
for k, v in enumerate(seq):
if isinstance(v, dict):
yield from iter_dict(v, key, indices + [k])
elif isinstance(v, list):
yield from iter_list(v, key, indices + [k])
for c in find_key(json_data, 'customer_count'):
print(c)
Result:
(['calendar', 'weeks', 0, 'days', 1, 'availabilities', 0, 'customer_count'], 14)
(['calendar', 'weeks', 0, 'days', 2, 'availabilities', 0, 'customer_count'], 7)
Another post has an example to extract multiple keys, but doesn't recurse through the entire object:
[...]
keys = ("customer_count", "utc_start_at", "non_resource_bookable_capacity")
for k in keys:
keypath, val = next(find_key(json_data, k))
print("{!r}: {!r}".format(k, val))
Result:
'customer_count': 14
'utc_start_at': '2018-09-29T16:45:00+0000'
'non_resource_bookable_capacity': 18
How do I iterate through the entire object and extract the three keys shown above?
My desired result would look something like this:
'customer_count': 14
'utc_start_at': '2018-09-29T16:45:00+0000'
'non_resource_bookable_capacity': 18
'customer_count': 7
'utc_start_at': '2018-09-29T16:45:00+0000'
'non_resource_bookable_capacity': 25
sample json
The example function below searches a dict (including all nested dicts) for key / value pairs matching a list of keys you would like to find. This function recursively loops through the dict and any nested dicts and lists it contains to build a list of all possible dicts to be checked for matching keys.
def find_key_value_pairs(q, keys, dicts=None):
if not dicts:
dicts = [q]
q = [q]
data = q.pop(0)
if isinstance(data, dict):
data = data.values()
for d in data:
dtype = type(d)
if dtype is dict or dtype is list:
q.append(d)
if dtype is dict:
dicts.append(d)
if q:
return find_key_value_pairs(q, keys, dicts)
return [(k, v) for d in dicts for k, v in d.items() if k in keys]
Example below uses json.loads to convert an example dataset similar to your json to a dict before passing it to the function.
import json
json_data = """
{"results_count": 2, "results": [{"utc_start_at": "2018-09-29T16:45:00+0000", "counts": {"customer_count": "14", "other_count": "41"}, "capacity": {"non-resource": {"non_resource_bookable_capacity": "18", "other_non_resource_capacity": "1"}, "resource_capacity": "10"}}, {"utc_start_at": "2018-10-29T15:15:00+0000", "counts": {"customer_count": "7", "other_count": "41"}, "capacity": {"non-resource": {"non_resource_bookable_capacity": "25", "other_non_resource_capacity": "1"}, "resource_capacity": "10"}}]}
"""
data = json.loads(json_data) # json_data is a placeholder for your json
keys = ['results_count', 'customer_count', 'utc_start_at', 'non_resource_bookable_capacity']
results = find_key_value_pairs(data, keys)
for k, v in results:
print(f'{k}: {v}')
# results_count: 2
# utc_start_at: 2018-09-29T16:45:00+0000
# utc_start_at: 2018-10-29T15:15:00+0000
# customer_count: 14
# customer_count: 7
# non_resource_bookable_capacity: 18
# non_resource_bookable_capacity: 25
In Python 2.7, how does one dynamically access and print out the keys and values of a nested dictionary? Here's a nonsensical example: https://jsoneditoronline.org/?id=da7a486dc2e24bf8b94add9f04c71b4d
Normally, I would do something like:
import json
json_sample = 'sample_dict.json'
json_file = open(json_sample, 'r')
json_data = json.load(json_file)
items = json_data['sample_dict']
for item in items:
dict_id = item['dict_id']
person = item['person']['person_id']
family = item['family']['members']
print dict_id
print person
print family
I can hard code it like this and it'll give me desirable results, but how would I access each of the keys and values dynamically so that:
The first row just prints the keys (dict_id, person['person_id'], person['name'], family['members']['father'])
The second row prints the values respectively (5, 15, "Martin", "Jose")
The end result should be in a CSV file.
You can use a recursive visitor/generator which returns all the path/value pairs of the leaves:
def visit_dict(d, path=[]):
for k, v in d.items():
if not isinstance(v, dict):
yield path + [k], v
else:
yield from visit_dict(v, path + [k])
(replace the yield from ... with the appropriate equivalent if using Python < 3.4)
Getting the keys:
>>> ','.join('/'.join(k) for k, v in visit_dict(json_data['sample_dict'][0]))
'dict_id,person/person_id,person/name,person/age,family/person_id,family/members/father,family/members/mother,family/members/son,family/family_id,items_id,furniture/type,furniture/color,furniture/size,furniture/purchases'
and the values:
>>> ','.join(str(v) for k, v in visit_dict(json_data['sample_dict'][0]))
'5,15,Martin,18,20,Jose,Maddie,Jerry,2,None,Chair,Brown,Large,[]'
I am receiving data in batches from an API in JSON format. I wish to store only the values, in a list.
The raw data looks like this and will always look like this, i.e: all {...} will look like the first example:
data = content.get('data')
>>> [{'a':1, 'b':{'c':2, 'd':3}, 'e':4}, {...}, {...}, ...]
The nested dictionary is making this harder; I need this unpacked as well.
Here is what I have, which works but it feels so bad:
unpacked = []
data = content.get('data')
for d in data:
item = []
for k, v in d.items():
if k == 'b':
for val in v.values():
item.append(val)
else:
item.append(v)
unpacked.append(item)
Output:
>>> [[1,2,3,4], [...], [...], ...]
How can I improve this?
You could use a recursive function and some type tests:
data = [{'a':1, 'b':{'c':2, 'd':3}, 'e':4}, {'f':5,'g':6}]
def extract_nested_values(it):
if isinstance(it, list):
for sub_it in it:
yield from extract_nested_values(sub_it)
elif isinstance(it, dict):
for value in it.values():
yield from extract_nested_values(value)
else:
yield it
print(list(extract_nested_values(data)))
# [1, 2, 3, 4, 5, 6]
Note that it outputs a flat generator, not a list of lists.
Assuming your dictionaries do not contain inner lists, you could define a simple routine to unpack a nested dictionary, and iterate through each item in data using a loop.
def unpack(data):
for k, v in data.items():
if isinstance(v, dict):
yield from unpack(v)
else:
yield v
Note that this function is as simple as it is thanks to the magic of yield from. Now, let's call it with some data.
data = [{'a':1, 'b':{'c':2, 'd':3}, 'e':4}, {'f':5,'g':6}] # Data "borrowed" from Kaushik NP
result = [list(unpack(x)) for x in data]
print(result)
[[2, 3, 1, 4], [5, 6]]
Note the lack of order in your result, because of the arbitrary order of dictionaries.
For completeness, based on the excellent answer of Eric Duminil, here is a function that returns the maximum depth of a nested dict or list:
def depth(it, count=0):
"""Depth of a nested dict.
# Arguments
it: a nested dict or list.
count: a constant value used in internal calculations.
# Returns
Numeric value.
"""
if isinstance(it, list):
if any(isinstance(v, list) or isinstance(v, dict) for v in it):
for v in it:
if isinstance(v, list) or isinstance(v, dict):
return depth(v, count + 1)
else:
return count
elif isinstance(it, dict):
if any(isinstance(v, list) or isinstance(v, dict) for v in it.values()):
for v in it.values():
if isinstance(v, list) or isinstance(v, dict):
return depth(v, count + 1)
else:
return count
else:
return count
In the Python tradition, it is zero-based.
Other answers (especially #COLDSPEED's) have already covered the situation, but here is a slightly different code based on the old adage it's better to ask forgiveness than permission , which I tend to prefer to type checking:
def unpack(data):
try:
for value in data.values():
yield from unpack(value)
except AttributeError:
yield data
data = [{'a':1, 'b':{'c':2, 'd':3}, 'e':4}]
unpacked = [list(unpack(item)) for item in data]
Doing recursively :
def traverse(d):
for key,val in d.items():
if isinstance(val, dict):
traverse(val)
else:
l.append(val)
out=[]
for d in data:
l=[]
traverse(d)
out.append(l)
print(out)
#driver values :
IN : data = [{'a':1, 'b':{'c':2, 'd':3}, 'e':4}, {'f':5,'g':6}]
OUT : out = [[1, 2, 3, 4], [5, 6]]
EDIT : A better way to do this is using yield so as not to have to rely on global variables as in the first method.
def traverse(d):
for key,val in d.items():
if isinstance(val, dict):
yield from traverse(val)
else:
yield val
out = [list(traverse(d)) for d in data]
Let's say I have a pretty complex dictionary.
{'fruit':'orange','colors':{'dark':4,'light':5}}
Anyway, my objective is to scan every key in this complex multi-level dictionary. Then, append "abc" to the end of each key.
So that it will be:
{'fruitabc':'orange','colorsabc':{'darkabc':4,'lightabc':5}}
How would you do that?
Keys cannot be changed. You will need to add a new key with the modified value then remove the old one, or create a new dict with a dict comprehension or the like.
For example like this:
def appendabc(somedict):
return dict(map(lambda (key, value): (str(key)+"abc", value), somedict.items()))
def transform(multilevelDict):
new = appendabc(multilevelDict)
for key, value in new.items():
if isinstance(value, dict):
new[key] = transform(value)
return new
print transform({1:2, "bam":4, 33:{3:4, 5:7}})
This will append "abc" to each key in the dictionary and any value that is a dictionary.
EDIT: There's also a really cool Python 3 version, check it out:
def transform(multilevelDict):
return {str(key)+"abc" : (transform(value) if isinstance(value, dict) else value) for key, value in multilevelDict.items()}
print(transform({1:2, "bam":4, 33:{3:4, 5:7}}))
I use the following utility function that I wrote that takes a target dict and another dict containing the translation and switches all the keys according to it:
def rename_keys(d, keys):
return dict([(keys.get(k), v) for k, v in d.items()])
So with the initial data:
data = { 'a' : 1, 'b' : 2, 'c' : 3 }
translation = { 'a' : 'aaa', 'b' : 'bbb', 'c' : 'ccc' }
We get the following:
>>> data
{'a': 1, 'c': 3, 'b': 2}
>>> rename_keys(data, translation)
{'aaa': 1, 'bbb': 2, 'ccc': 3}
>>> mydict={'fruit':'orange','colors':{'dark':4,'light':5}}
>>> def f(mydict):
... return dict((k+"abc",f(v) if hasattr(v,'keys') else v) for k,v in mydict.items())
...
>>> f(mydict)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
My understanding is that you can't change the keys, and that you would need to make a new set of keys and assign their values to the ones the original keys were pointing to.
I'd do something like:
def change_keys(d):
if type(d) is dict:
return dict([(k+'abc', change_keys(v)) for k, v in d.items()])
else:
return d
new_dict = change_keys(old_dict)
here's a tight little function:
def keys_swap(orig_key, new_key, d):
d[new_key] = d.pop(orig_key)
for your particular problem:
def append_to_dict_keys(appendage, d):
#note that you need to iterate through the fixed list of keys, because
#otherwise we will be iterating through a never ending key list!
for each in d.keys():
if type(d[each]) is dict:
append_to_dict_keys(appendage, d[each])
keys_swap(each, str(each) + appendage, d)
append_to_dict_keys('abc', d)
#! /usr/bin/env python
d = {'fruit':'orange', 'colors':{'dark':4,'light':5}}
def add_abc(d):
newd = dict()
for k,v in d.iteritems():
if isinstance(v, dict):
v = add_abc(v)
newd[k + "abc"] = v
return newd
d = add_abc(d)
print d
Something like that
def applytoallkeys( dic, func ):
def yielder():
for k,v in dic.iteritems():
if isinstance( v, dict):
yield func(k), applytoallkeys( v, func )
else:
yield func(k), v
return dict(yielder())
def appendword( s ):
def appender( x ):
return x+s
return appender
d = {'fruit':'orange','colors':{'dark':4,'light':5}}
print applytoallkeys( d, appendword('asd') )
I kinda like functional style, you can read just the last line and see what it does ;-)
You could do this with recursion:
import collections
in_dict={'fruit':'orange','colors':{'dark':4,'light':5}}
def transform_dict(d):
out_dict={}
for k,v in d.iteritems():
k=k+'abc'
if isinstance(v,collections.MutableMapping):
v=transform_dict(v)
out_dict[k]=v
return out_dict
out_dict=transform_dict(in_dict)
print(out_dict)
# {'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
you should also consider that there is the possibility of nested dicts in nested lists, which will not be covered by the above solutions. This function ads a prefix and/or a postfix to every key within the dict.
def transformDict(multilevelDict, prefix="", postfix=""):
"""adds a prefix and/or postfix to every key name in a dict"""
new_dict = multilevelDict
if prefix != "" or postfix != "":
new_key = "%s#key#%s" % (prefix, postfix)
new_dict = dict(map(lambda (key, value): (new_key.replace('#key#', str(key)), value), new_dict.items()))
for key, value in new_dict.items():
if isinstance(value, dict):
new_dict[key] = transformDict(value, prefix, postfix)
elif isinstance(value, list):
for index, item in enumerate(value):
if isinstance(item, dict):
new_dict[key][index] = transformDict(item, prefix, postfix)
return new_dict
for k in theDict: theDict[k+'abc']=theDict.pop(k)
I use this for converting docopt POSIX-compliant command-line keys to PEP8 keys
(e.g. "--option" --> "option", "" --> "option2", "FILENAME" --> "filename")
arguments = docopt.docopt(__doc__) # dictionary
for key in arguments.keys():
if re.match('.*[-<>].*', key) or key != key.lower():
value = arguments.pop(key)
newkey = key.lower().translate(None, '-<>')
arguments[newkey] = value
Hi I'm a new user but finding an answer for same question, I can't get anything fully functional to my problem, I make this little piece of cake with a full nested replace of keys, you can send list with dict or dict.
Finally your dicts can have list with dict or more dict nested and it is all replaced with your new key needs.
To indicate who key want replace with a new key use "to" parameter sending a dict.
See at end my little example.
P/D: Sorry my bad english. =)
def re_map(value, to):
"""
Transform dictionary keys to map retrieved on to parameters.
to parameter should have as key a key name to replace an as value key name
to new dictionary.
this method is full recursive to process all levels of
#param value: list with dictionary or dictionary
#param to: dictionary with re-map keys
#type to: dict
#return: list or dict transformed
"""
if not isinstance(value, dict):
if not isinstance(value, list):
raise ValueError(
"Only dict or list with dict inside accepted for value argument.") # #IgnorePep8
if not isinstance(to, dict):
raise ValueError("Only dict accepted for to argument.")
def _re_map(value, to):
if isinstance(value, dict):
# Re map dictionary key.
# If key of original dictionary is not in "to" dictionary use same
# key otherwise use re mapped key on new dictionary with already
# value.
return {
to.get(key) or key: _re_map(dict_value, to)
for key, dict_value in value.items()
}
elif isinstance(value, list):
# if value is a list iterate it a call _re_map again to parse
# values on it.
return [_re_map(item, to) for item in value]
else:
# if not dict or list only return value.
# it can be string, integer or others.
return value
result = _re_map(value, to)
return result
if __name__ == "__main__":
# Sample test of re_map method.
# -----------------------------------------
to = {"$id": "id"}
x = []
for i in range(100):
x.append({
"$id": "first-dict",
"list_nested": [{
"$id": "list-dict-nested",
"list_dic_nested": [{
"$id": "list-dict-list-dict-nested"
}]
}],
"dict_nested": {
"$id": "non-nested"
}
})
result = re_map(x, to)
print(str(result))
A functional (and flexible) solution: this allows an arbitrary transform to be applied to keys (recursively for embedded dicts):
def remap_keys(d, keymap_f):
"""returns a new dict by recursively remapping all of d's keys using keymap_f"""
return dict([(keymap_f(k), remap_keys(v, keymap_f) if isinstance(v, dict) else v)
for k,v in d.items()])
Let's try it out; first we define our key transformation function, then apply it to the example:
def transform_key(key):
"""whatever transformation you'd like to apply to keys"""
return key + "abc"
remap_keys({'fruit':'orange','colors':{'dark':4,'light':5}}, transform_key)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
(note: if you're still on Python 2.x, you'll need to replace d.items() on the last line with d.iteritems() -- thanks to #Rudy for reminding me to update this post for Python 3).
Based on #AndiDog's python 3 version and similar to #sxc731's version but with a flag for whether to apply it recursively:
def transform_keys(dictionary, key_fn, recursive=True):
"""
Applies function to keys and returns as a new dictionary.
Example of key_fn:
lambda k: k + "abc"
"""
return {key_fn(key): (transform_keys(value, key_fn=key_fn, recursive=recursive)
if recursive and isinstance(value, dict) else value)
for key, value in dictionary.items()}