I have two dictionaries that may be formatted like the following:
d1 = { "root" : { "leaf" : [diff1, diff2] }}
d2 = { "root" : { "leaf": [diff3], "node": { "leaf" : [diff1] } } }
I would like to be able to combine them into such a result:
d3 = {'root': {'leaf': ['diff1', 'diff2', 'diff 3'], 'node': {'leaf': ['diff 1']}}}
The level of recursion is undefined for any given input, but every "leaf node" will be an array, which I would like to combine if one exists for a leaf node in each dictionary. NOTE: The keys are not actually "node" and "leaf", i simply used these keys to help illustrate the example.
This code seems to completely overwrite my arrays with the second one:
d3 = {**d1, **d2} yielding
result = {'root': {'leaf': ['diff 3'], 'node': {'leaf': ['diff 1']}}}
Untested, but something like this should work:
def dictmerge(d1, d2):
if d1 is None:
return d2
if d2 is None:
return d1
if isinstance(d1, list) and isinstance(d2, list):
return d1+d2
if isinstance(d1, dict) and isinstance(d2, dict):
return {k: dictmerge(d1.get(k), d2.get(k))
for k in set(d1.keys()) | set(d2.keys())}
raise ValueError, "inputs have incompatible structure"
You can do this with a recursive function. Note this will only merge lists, not tuples or any other kind of sequence:
def merge_dicts(d1, d2):
out_dict = {}
for key in (set(d1.keys()) | set(d2.keys())):
if key in d1 and key in d2:
if isinstance(d1[key], dict) and isinstance(d2[key], dict):
out_dict[key] = merge_dicts(d1[key], d2[key])
elif isinstance(d1[key], list) and isinstance(d2[key], list):
out_dict[key] = d1[key] + d2[key]
elif d1[key] == d2[key]:
out_dict[key] = d1[key]
else:
raise ValueError(
'Cannot merge conflicting values {} and {}'.format(
d1[key], d2[key]))
elif key in d1:
out_dict[key] = d1[key]
else:
out_dict[key] = d2[key]
return out_dict
d1 = { "root" : { "leaf" : ['diff1', 'diff2'] }}
d2 = { "root" : { "leaf": ['diff3'], "node": { "leaf" : ['diff1'] } } }
print(merge_dicts(d1, d2))
You haven't defined what to do if you call merge_dicts({'key': 'value'}, {'key': 'different_value'}). I have it raising a ValueError, since I would think you can't merge these two dicts (which value would you take?) but you can change that if you want to pick the value from one of the dicts as default, for example.
Related
I have a large nested dictionary with an unknown depth and i would like to know how i can find the keys which led to the value. For example...
{'furniture':{'chair':{'sofa':{'cushion':{}}}}}
Ideally what i am looking for is a function to determine the path to the value that i have entered. I have tried researching online and this is what i tried...
def route(d,key):
if key in d: return d[key]
for k,v in d.items():
if isinstance(v,dict):
item = route(v, key)
if item is not None:
return item
This returns the items inside the key. I am looking to be able to extract the path which leads to that item. For example, route(dictionary,'sofa') then i would be able to get an expected output as such or something similar...
{'sofa':{'chair':'furniture'}}
What are some of the ways that i can achieve this ? Thanks for your help
You can do this recursively and return a list of keys that lead you to your target key:
def route(d, key):
if key in d: return [key]
for k, v in d.items():
if type(v) == dict:
found = route(v, key)
if found: return [k] + found
return []
If we run this on the following dictionary:
data = {
'furniture': {
'chair': {
'sofa': {
'cushion': {}
}
}
},
'electronics': {
'tv': {
'samsung43': 800,
'tcl54': 200
}
}
}
print(route(data, 'cushion'))
print(route(data, 'tcl54'))
print(route(data, 'hello'))
we get the following output:
['furniture', 'chair', 'sofa', 'cushion']
['electronics', 'tv', 'tcl54']
[]
I am trying to update values in a nested dictionary, without over-writting previous entries when the key already exists.
For example, I have a dictionary:
myDict = {}
myDict["myKey"] = { "nestedDictKey1" : aValue }
giving,
print myDict
>> { "myKey" : { "nestedDictKey1" : aValue }}
Now, I want to add another entry , under "myKey"
myDict["myKey"] = { "nestedDictKey2" : anotherValue }}
This will return:
print myDict
>> { "myKey" : { "nestedDictKey2" : anotherValue }}
But I want:
print myDict
>> { "myKey" : { "nestedDictKey1" : aValue ,
"nestedDictKey2" : anotherValue }}
Is there a way to update or append "myKey" with new values, without overwriting the previous ones?
This is a very nice general solution to dealing with nested dicts:
import collections
def makehash():
return collections.defaultdict(makehash)
That allows nested keys to be set at any level:
myDict = makehash()
myDict["myKey"]["nestedDictKey1"] = aValue
myDict["myKey"]["nestedDictKey2"] = anotherValue
myDict["myKey"]["nestedDictKey3"]["furtherNestedDictKey"] = aThirdValue
For a single level of nesting, defaultdict can be used directly:
from collections import defaultdict
myDict = defaultdict(dict)
myDict["myKey"]["nestedDictKey1"] = aValue
myDict["myKey"]["nestedDictKey2"] = anotherValue
And here's a way using only dict:
try:
myDict["myKey"]["nestedDictKey2"] = anotherValue
except KeyError:
myDict["myKey"] = {"nestedDictKey2": anotherValue}
You can use collections.defaultdict for this, and just set the key-value pairs within the nested dictionary.
from collections import defaultdict
my_dict = defaultdict(dict)
my_dict['myKey']['nestedDictKey1'] = a_value
my_dict['myKey']['nestedDictKey2'] = another_value
Alternatively, you can also write those last 2 lines as
my_dict['myKey'].update({"nestedDictKey1" : a_value })
my_dict['myKey'].update({"nestedDictKey2" : another_value })
You can write a generator to update key in nested dictionary, like this.
def update_key(key, value, dictionary):
for k, v in dictionary.items():
if k == key:
dictionary[key]=value
elif isinstance(v, dict):
for result in update_key(key, value, v):
yield result
elif isinstance(v, list):
for d in v:
if isinstance(d, dict):
for result in update_key(key, value, d):
yield result
list(update_key('Any level key', 'Any value', DICTIONARY))
from ndicts.ndicts import NestedDict
nd = NestedDict()
nd["myKey", "nestedDictKey1"] = 0
nd["myKey", "nestedDictKey2"] = 1
>>> nd
NestedDict({'myKey': {'nestedDictKey1': 0, 'nestedDictKey2': 1}})
>>> nd.to_dict()
{'myKey': {'nestedDictKey1': 0, 'nestedDictKey2': 1}}
To install ndicts pip install ndicts
You could treat the nested dict as immutable:
myDict["myKey"] = dict(myDict["myKey"], **{ "nestedDictKey2" : anotherValue })
myDict["myKey"]["nestedDictKey2"] = anotherValue
myDict["myKey"] returns the nested dictionary to which we can add another key like we do for any dictionary :)
Example:
>>> d = {'myKey' : {'k1' : 'v1'}}
>>> d['myKey']['k2'] = 'v2'
>>> d
{'myKey': {'k2': 'v2', 'k1': 'v1'}}
I wrote myself a function to tackle this issue
def updateDict2keys(myDict,mykey1,mykey2,myitems):
"""
updates a dictionary by appending values at given keys (generating key2 if not already existing)
input: dictionary, key1, key2 and items to append
output: dictionary orgnanized as {mykey1:{mykey2:myitems}}
"""
myDict.setdefault(mykey1, {})[mykey2] = myitems
return myDict
I am trying to update the keys of a JSON object which looks like this:
results =
{
'Game':12345,
'stats':[
{
'detail':[
{
'goals':4,
'refs':{
'number':0
I am currently manually updating each key as follow
##update Game to newValue
results['newValue'] = results['Game']
del results['Game']
## update nested key "goals" to "goals_against"
results['stats'][0]['detail'][0]['goals_against'] = results['stats'][0]['detail'][0]['goals']
del results['stats'][0]['detail'][0]['goals']
there has to be a better way to do as I am finding myself having to update multiple keys on results. For example, I also want to update the "number" key to "assis_ref".
I know how to update a key if is the json file is "simple": ie if i could do this:
result['stats']['details']['refs']
however, 'stats' and 'details' require [0] next to it which i assume is the index of the element i am trying to go next.
I wrote a recursive function to handle transforming keys in json objects. Especially useful for deeply nested json!
def walk_json(obj, key_transform):
"""
Recurse over a json object modifying the keys according to the `key_transform` function.
Returns a new json object with the modified keys.
"""
assert isinstance(obj, dict), "walk_json expects obj to be of type dict"
def _walk_json(obj, new):
if isinstance(obj, dict):
if isinstance(new, dict):
for key, value in obj.items():
new_key = key_transform(key)
if isinstance(value, dict):
new[new_key] = {}
_walk_json(value, new=new[new_key])
elif isinstance(value, list):
new[new_key] = []
for item in value:
_walk_json(item, new=new[new_key])
else: # take value as is
new[new_key] = value
elif isinstance(new, list):
new.append(_walk_json(obj, new={}))
else: # take object as is
new.append(obj)
return new
return _walk_json(obj, new={})
Here's how it's used for an overly simple json object:
def my_key_transform(x):
return x.upper()
my_obj = {"a": 1, "b": 2, "c": 3}
result = walk_json(my_obj, key_transform=my_key_transform)
result
{"A": 1, "B": 2, "C": 3}
It can be painful navigating and modify deeply nested objects derived from JSON objects. In Functions that help to understand json(dict) structure I posted code that allows you to navigate such objects. Please read the explanation in that answer. In this answer, I'll show how you can use that code to modify the dictionary keys in such objects.
Briefly, find_key is a recursive generator that will find all the keys with a given name. You can use the next function to get the first (or only) matching name. Or call find_key in a for loop if you need to work with multiple keys that have the same name.
Each value yielded by find_key is a list of the dict keys and list indices need to reach the desired key.
from json import dumps
def find_key(obj, key):
if isinstance(obj, dict):
yield from iter_dict(obj, key, [])
elif isinstance(obj, list):
yield from iter_list(obj, key, [])
def iter_dict(d, key, indices):
for k, v in d.items():
if k == key:
yield indices + [k], v
if isinstance(v, dict):
yield from iter_dict(v, key, indices + [k])
elif isinstance(v, list):
yield from iter_list(v, key, indices + [k])
def iter_list(seq, key, indices):
for k, v in enumerate(seq):
if isinstance(v, dict):
yield from iter_dict(v, key, indices + [k])
elif isinstance(v, list):
yield from iter_list(v, key, indices + [k])
results = {
"Game": 12345,
"stats": [
{
"detail": [
{
"goals": 4,
"refs": {
"number": 0
}
}
]
}
]
}
# Change oldkey to newkey
oldkey, newkey = 'goals', 'goals_against'
# Find the first occurrence of the oldkey
seq, val = next(find_key(results, oldkey))
print('seq:', seq, 'val:', val)
# Get the object that contains the oldkey
obj = results
for k in seq[:-1]:
obj = obj[k]
# Change the key
obj[newkey] = obj.pop(oldkey)
print(dumps(results, indent=4))
output
seq: ['stats', 0, 'detail', 0, 'goals'] val: 4
{
"Game": 12345,
"stats": [
{
"detail": [
{
"refs": {
"number": 0
},
"goals_against": 4
}
]
}
]
}
I have two dictionary like :
A = {
"Name": ["AAAAA"],
"designation": ["DDDDDDD"],
"doj": ["KKKKKK"],
"RRRRRRRR": ["JJJJJJ"]
}
B = {
{
u'IIIIII': u'200',
u'KKKKKK': u'1/1/2015',
u'JJJJJJ': u'legal',
u'HHHHHH': u'John'
}{
u'AAAAA': u'Stacie',
u'DDDDDDD': u'6/8/2014',
u'BBBB': u'2/9/1988',
u'CCCCCCC': u'legal'
}
}
I have to take the value of A and compare to key of B and if the name matches I should get the value of key from B and store. Something like
{
"Name": ["John"],
"designation": ["12/02/2016"],
"doj":["Manager"]
}
I am trying to loop both and then trying to append value to a dict item using the get method but could not achieve the goal.
for key,value in content.iteritems():
#print value
if isinstance(value, list):
vals.append(key)
for i in value:
ii=0
#while(ii<len(z)):
if (z.get(i)== 'true' ) :
vals.append(z.get(i))
This solution is considering the values of A and B provided in the question:
z = {}
for k, v in A.items():
av = B.get(v[0], None)
if av is not None:
z[k] = av
print(z)
# {'Name': ['John'], 'designation': ['12/02/2016'], 'doj': ['Manager']}
or, a one liner:
z = {k: B[v[0]] for k, v in A.items() if v[0] in B}
# {'Name': ['John'], 'designation': ['12/02/2016'], 'doj': ['Manager']}
I am trying to update values in a nested dictionary, without over-writting previous entries when the key already exists.
For example, I have a dictionary:
myDict = {}
myDict["myKey"] = { "nestedDictKey1" : aValue }
giving,
print myDict
>> { "myKey" : { "nestedDictKey1" : aValue }}
Now, I want to add another entry , under "myKey"
myDict["myKey"] = { "nestedDictKey2" : anotherValue }}
This will return:
print myDict
>> { "myKey" : { "nestedDictKey2" : anotherValue }}
But I want:
print myDict
>> { "myKey" : { "nestedDictKey1" : aValue ,
"nestedDictKey2" : anotherValue }}
Is there a way to update or append "myKey" with new values, without overwriting the previous ones?
This is a very nice general solution to dealing with nested dicts:
import collections
def makehash():
return collections.defaultdict(makehash)
That allows nested keys to be set at any level:
myDict = makehash()
myDict["myKey"]["nestedDictKey1"] = aValue
myDict["myKey"]["nestedDictKey2"] = anotherValue
myDict["myKey"]["nestedDictKey3"]["furtherNestedDictKey"] = aThirdValue
For a single level of nesting, defaultdict can be used directly:
from collections import defaultdict
myDict = defaultdict(dict)
myDict["myKey"]["nestedDictKey1"] = aValue
myDict["myKey"]["nestedDictKey2"] = anotherValue
And here's a way using only dict:
try:
myDict["myKey"]["nestedDictKey2"] = anotherValue
except KeyError:
myDict["myKey"] = {"nestedDictKey2": anotherValue}
You can use collections.defaultdict for this, and just set the key-value pairs within the nested dictionary.
from collections import defaultdict
my_dict = defaultdict(dict)
my_dict['myKey']['nestedDictKey1'] = a_value
my_dict['myKey']['nestedDictKey2'] = another_value
Alternatively, you can also write those last 2 lines as
my_dict['myKey'].update({"nestedDictKey1" : a_value })
my_dict['myKey'].update({"nestedDictKey2" : another_value })
You can write a generator to update key in nested dictionary, like this.
def update_key(key, value, dictionary):
for k, v in dictionary.items():
if k == key:
dictionary[key]=value
elif isinstance(v, dict):
for result in update_key(key, value, v):
yield result
elif isinstance(v, list):
for d in v:
if isinstance(d, dict):
for result in update_key(key, value, d):
yield result
list(update_key('Any level key', 'Any value', DICTIONARY))
from ndicts.ndicts import NestedDict
nd = NestedDict()
nd["myKey", "nestedDictKey1"] = 0
nd["myKey", "nestedDictKey2"] = 1
>>> nd
NestedDict({'myKey': {'nestedDictKey1': 0, 'nestedDictKey2': 1}})
>>> nd.to_dict()
{'myKey': {'nestedDictKey1': 0, 'nestedDictKey2': 1}}
To install ndicts pip install ndicts
You could treat the nested dict as immutable:
myDict["myKey"] = dict(myDict["myKey"], **{ "nestedDictKey2" : anotherValue })
myDict["myKey"]["nestedDictKey2"] = anotherValue
myDict["myKey"] returns the nested dictionary to which we can add another key like we do for any dictionary :)
Example:
>>> d = {'myKey' : {'k1' : 'v1'}}
>>> d['myKey']['k2'] = 'v2'
>>> d
{'myKey': {'k2': 'v2', 'k1': 'v1'}}
I wrote myself a function to tackle this issue
def updateDict2keys(myDict,mykey1,mykey2,myitems):
"""
updates a dictionary by appending values at given keys (generating key2 if not already existing)
input: dictionary, key1, key2 and items to append
output: dictionary orgnanized as {mykey1:{mykey2:myitems}}
"""
myDict.setdefault(mykey1, {})[mykey2] = myitems
return myDict