I want to replace keys name of a dictionary by passing a mapping dict with a function that replace also nested keys.
The issue is that I have multiple keys named 'id' in nested dictionary and I want to rename these 'id' with specific names.
Initial dictionnary:
initial_dict = {'id': 1, 'netAmount': 10.2, 'modifiedOn': '2017-01-01',
'statusId': 3, 'approvalStateId': 3, 'approvalState': {'id': 3,'name':'Approved'}}
Mapping dict:
mapping_dict = {'id': 'pr_id', 'netAmount': 'net_amount', 'modifiedOn': 'modified_date',
'statusId': 'status_id', 'approvalStateId': 'approval_id','approvalState':{'id':'approv_id'}}
Desired output of the dictionary:
output_dict = {'pr_id': 1, 'net_amount': 10.2, 'modified_date': '2017-01-01',
'status_id': 3, 'approval_id': 3, 'approvalState': {'approv_id': 3, 'name': 'Approved'}}
What I did is this but it only replace keys of the first level of the dict and if I try to set nested keys in the mapping dict, I get an error.
def map_fields(obj):
new_obj = {}
mapping_dict = {'id': 'pr_id', 'netAmount': 'net_amount', 'modifiedOn': 'modified_date',
'statusId': 'status_id', 'approvalStateId': 'approval_id','approvalState':{'id':'approv_id'}}
for key in obj.keys():
if key in mapping_dict:
new_key = mapping_dict[key]
else:
new_key = key
new_obj[new_key] = obj[key]
return new_obj
Do you have any idea how to do it?
Thanks
You need a recursive function to be able to step up through the nested dictionaries. The key is that when recursing, you need to pass both the child dictionary, and the child mapping dictionary.
Note that the structure of your mapping dict is specific to this problem, and doesn't allow you to change the key of a nested dictionary - you'd need to restructure how you store the mapping to achieve that.
The following should do what you want (print statements added to help follow the logic when it runs):
def map_fields(init_dict, map_dict, res_dict=None):
res_dict = res_dict or {}
for k, v in init_dict.items():
print("Key: ", k)
if isinstance(v, dict):
print("value is a dict - recursing")
v = map_fields(v, map_dict[k])
elif k in map_dict.keys():
print("Remapping:", k, str(map_dict[k]))
k = str(map_dict[k])
res_dict[k] = v
return res_dict
print(map_fields(initial_dict, mapping_dict))
Also if you want the same result but do not want to use a nested mapping dict such as :
mapping_dict = {'id': 'pr_id', 'netAmount': 'net_amount', 'modifiedOn': 'modified_date',
'statusId': 'status_id', 'approvalStateId': 'approval_id', 'id':'approv_id'}
(ie 'id'/'approv_id' is not nested in 'approvalState')
You can use the slightly different function:
def map_fields(init_dict, map_dict, res_dict=None):
res_dict = res_dict or {}
for k, v in init_dict.items():
if isinstance(v, dict):
v = map_fields(v, map_dict)
if k in map_dict.keys():
k = str(map_dict[k])
res_dict[k] = v
return res_dict
This can be especially useful if the key 'approvalState' was also to map (to something like 'resultState', ie if your result needed to be:
output_dict = {'pr_id': 1, 'net_amount': 10.2, 'modified_date': '2017-01-01',
'status_id': 3, 'approval_id': 3, 'resultState': {'approv_id': 3, 'name': 'Approved'}}
In which case, you would just have had to add the key/value pair 'approvalState'/'resultState' in your mapping dict.
Related
The following dictionary is given:
dict_nested = {"A":{"C":100, "D":{"E":100, "F":100}}, "B":200}
The result should look like this:
dict_result = {"C":100, "E":100, "F":100, "B":200}
the result should be 1 Dictionary which only contain the key-value pairs, which its values are from type Integer and not dict.
the order should be maintained (i dont mean the alphabetical order of the keys)
Like Sembei said, recursion is needed for this case:
dict_nested = {"A":{"C":100, "D":{"E":100, "F":100}}, "B":200}
def extract_dict(d1, d2):
for k, v in d1.items():
if isinstance(v, dict):
extract_dict(v, d2)
elif isinstance(v, int):
d2.update({k: v})
dict_result = {}
extract_dict(dict_nested, dict_result)
print(dict_result)
Output:
{'C': 100, 'E': 100, 'F': 100, 'B': 200}
I have two dictionaries with similar values, but different keys. I'd like to update the second dictionary db with the key value of the first dictionary user.
What would be the optimal way to do that without overcomplicating logic?
For instance, I have the following dictionaries:
user = {'school': ['books', 'pencils', 'sheets', 'notebooks']}
db = {'education': ['books', 'pencils', 'sheets', 'notebooks'],
'actors': ['student', 'teacher', 'principal']}
user['school'] and db['education'] have the same list value. I just want to update the key value in db with the one found in user. The way I thought the implementation is the following:
def get_key(my_dict, val):
for key, value in my_dict.items():
if val == value:
return key
return None
res = {}
for key, value in db.items():
if value in user.values():
key_user = get_key(user, value)
if key_user:
res.update({key_user: value})
else:
res.update({key: value})
print(res)
I am implementing this in Python 3.6.6, so as I understand the order of the dictionary entries is kept.
The desired output would be as follows:
{'school': ['books', 'pencils', 'sheets', 'notebooks'],
'actors': ['student', 'teacher', 'principal']}
As you said the order was important I think the easiest way is to re-create the dictionary, not very efficient for larger datasets though.
We can iterate db to re-create it, the only thing we need to look at is whether to use the key from user or db. If the key exists inside user_values then we can fetch it's index and use that to get key from user_keys, otherwise we use key.
user = {'school': ['books', 'pencils', 'sheets', 'notebooks']}
db = {'education': ['books', 'pencils','sheets','notebooks'],
'actors': ['student','teacher','principal']}
user_keys = tuple(user.keys())
user_values = tuple(user.values())
new_db = {user_keys[user_values.index(value)] if value in user_values else key: value for key, value in db.items()}
>>> {'school': ['books', 'pencils', 'sheets', 'notebooks'],
'actors': ['student', 'teacher', 'principal']}
I like Serial Lazer's answer with a reversed dict, but the rest is a bit clunky IMHO. I would do a plain for-loop instead:
x = {'a': [1, 2, 3]} # "user"
y = {'b': [1, 2, 3], 'c': [4, 5, 6]} # "db"
rev = {tuple(v): k for k, v in x.items()}
result = {}
for k, v in y.items():
try:
k = rev[tuple(v)]
except KeyError:
pass
result[k] = v
print(result) # -> {'a': [1, 2, 3], 'c': [4, 5, 6]}
Or you could use a ChainMap if you're fine with reversing all the dicts, converting their values to tuples, then back.
from collections import ChainMap
c = ChainMap(*[{tuple(v): k for k, v in d.items()} for d in (x, y)])
result = {v: list(k) for k, v in c.items()}
Or just a plain dict:
x_rev, y_rev = [{tuple(v): k for k, v in d.items()} for d in (x, y)]
y_rev.update(x_rev)
result = {v: list(k) for k, v in y_rev.items()}
Might be more efficient if you create a reverse-loopup dict for the user dictionary.
Logic reduces to just 2 lines of code:
reverse_user = {tuple(user[key]):key for key in user}
new_db_dict = {reverse_user[tuple(db[db_key])]:db[db_key] for db_key in db if tuple(db[db_key]) in reverse_user}
new_db_dict_excl = {key:db[key] for key in db if tuple(db[key]) not in reverse_user}
new_db_dict.update(new_db_dict_excl)
We can make it a bit simpler when we realize that we can update the entire db with user. What we need to figure out is which keys to delete before we update it, while also being careful not to modify db when we are iterating it.
user = {'school': ['books', 'pencils', 'sheets', 'notebooks']}
db = {'education': ['books', 'pencils','sheets','notebooks'],
'actors': ['student','teacher','principal']}
for user_value in user.values():
for del_key in [db_key for db_key, db_value in db.items() if user_value == db_value]:
del db[del_key]
db.update(user)
>>> {'actors': ['student', 'teacher', 'principal'],
'school': ['books', 'pencils', 'sheets', 'notebooks']}
Okay, not sure how to explain this problem but here goes.
This is my dictionary myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
I'm trying to list out the '1003', '1040'.
I've tried using this method:
for i in range(len(a['12/2019'])):
print(a['12/2019'][i].keys())
It kind of works however it returns me
dict_keys(['1003'])
dict_keys(['1040'])
Is there anyway I can just get 1030 and 1040 to be isolated from the ... dict_keys([''])?
You can iterate over the values of the dictionary, and if the value is a list, iterate over the dictionaries in the list, and append all the keys to the result
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}],
'11/2019': '1005', '10/2019': 1234,
'09/2019': [{'1006' : 2}, {'1042' : 3}],
'08/2019': (1,2)}
keys=[]
#Iterate over values
for value in myDictionary.values():
#If value is a list
if isinstance(value, list):
#Iterate over the list
for d in value:
#Add the keys to the result
keys.extend(list(d.keys()))
print(keys)
The output will be
['1003', '1040', '1006', '1042']
You can also do this via a list-comprehension, but it's not really readable, so I would avoid against it
[key for value in myDictionary.values() if isinstance(value, list) for d in value for key in d.keys()]
Iterate over the sub-dicts in the list, grabbing the keys:
for sub_dict in a['12/2019']:
for key, val in sub_dict.items():
print(key)
You can use a list-comprehension:
[k for v in myDictionary.values() for x in v for k in x]
In code:
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
print([k for v in myDictionary.values() for x in v for k in x])
# ['1003', '1040']
You could do this by doing something like this
data = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
# Data is a dictionary of where each value corresponding
# to the key is a list of dictionaries.
for k, v in data.items():
# v is a list of dictionaries
all_keys = []
for value in v:
all_keys.extend(list(value.keys()))
print (all_keys)
The result is ['1003', '1040']
This may be more than you need, but if the data you shared for the example is just a small subset of what you're really looking at, here's a recursive function that can go as deep as you want it to.
Since you seem to want to skip the keys in the outer most level, I put in a spot to allow you to skip over the keys at a given depth. You shouldn't set the level argument manually since the code will just update that for itself.
def list_keys(element, ignore_levels=[], level=1):
"""
Recursivly list keys found in an element.
element: item to inspect.
ignore_levels: list of numeric levels to ignore.
level: recursion depth level.
"""
keys = []
# add keys if we're not ignoring this level and set iterable to values
if isinstance(element, dict):
if level not in ignore_levels:
keys.extend(element.keys())
iterable = element.values()
# if we hve a list or tuple, we can iterate over it
elif isinstance(element, (list, tuple)):
iterable = element
# if we have a single value, it's not a key and there's nothing to iterate
else:
iterable = []
# Iterate over elements and append any keys found
for i in iterable:
subkeys = list_keys(i, ignore_levels, level + 1)
if subkeys:
keys.extend(subkeys)
return keys
d = {'12/2019': [{'1003': 2}, {'1040': 3}]}
print(list_keys(d, ignore_levels=[1]))
# ['1003', '1040']
I have predefined dictionaries and I need a function to form one dictionary out of several input dicts where the keys are the names of the input dictionaries.
For example, I have the following:
double_left = {
'left': 6
}
double_right = {
'right': 6
}
The output should be:
>>> output_dicts
{'double_left': {'left': 6}, 'double_right': {'right': 6}}
I already tried:
How do you create nested dict in Python?
but could not derive the answer.
Suggestions ?
If you want to do it dynamically without hardcoding the names of the top-level items, you can get a list of variables by names using globals() and/or locals(). For example:
output = {}
for name, val in globals().items():
# Skipping variables that aren't dicts, and special
# variables starting with '_'
if isinstance(val, dict) and not name.startswith('_'):
output[name] = val
for name, val in locals().items():
if isinstance(val, dict) and not name.startswith('_'):
output[name] = val
Or more compact:
output = {name:val for name, val in (list(locals().items()) +
list(globals().items()))
if isinstance(val, dict) and not name.startswith('_')}
Just make a new dictionary.
output_dict = {"double_left": double_left, "double_right": double_right}
print(output_dict)
Just like your current dicts have 'key': 'value' pairs, with your values being ints (6), you can create a new dict with the keys being the name of the dict and the values being the nested dict.
output_dicts = {'double_left': double_left, 'double_right': double_right}
>>> output_dicts
{'double_left': {'left': 6}, 'double_right': {'right': 6}}
You can also use dict_name[new_key] = new_value.
For example:
output_dicts['double_left'] = double_left
output_dicts['double_right'] = double_right
Try this
Newdict = dict(double_left = { 'left': 6},double_right =
{'right': 6})
I'm trying to programmatically set a value in a dictionary, potentially nested, given a list of indices and a value.
So for example, let's say my list of indices is:
['person', 'address', 'city']
and the value is
'New York'
I want as a result a dictionary object like:
{ 'Person': { 'address': { 'city': 'New York' } }
Basically, the list represents a 'path' into a nested dictionary.
I think I can construct the dictionary itself, but where I'm stumbling is how to set the value. Obviously if I was just writing code for this manually it would be:
dict['Person']['address']['city'] = 'New York'
But how do I index into the dictionary and set the value like that programmatically if I just have a list of the indices and the value?
Python
Something like this could help:
def nested_set(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
And you can use it like this:
>>> d = {}
>>> nested_set(d, ['person', 'address', 'city'], 'New York')
>>> d
{'person': {'address': {'city': 'New York'}}}
I took the freedom to extend the code from the answer of Bakuriu. Therefore upvotes on this are optional, as his code is in and of itself a witty solution, which I wouldn't have thought of.
def nested_set(dic, keys, value, create_missing=True):
d = dic
for key in keys[:-1]:
if key in d:
d = d[key]
elif create_missing:
d = d.setdefault(key, {})
else:
return dic
if keys[-1] in d or create_missing:
d[keys[-1]] = value
return dic
When setting create_missing to True, you're making sure to only set already existing values:
# Trying to set a value of a nonexistent key DOES NOT create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, False))
>>> {'A': {'B': 1}}
# Trying to set a value of an existent key DOES create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, True))
>>> {'A': {'B': 1, '8': 2}}
# Set the value of an existing key
print(nested_set({"A": {"B": 1}}, ["A", "B"], 2))
>>> {'A': {'B': 2}}
Here's another option:
from collections import defaultdict
recursivedict = lambda: defaultdict(recursivedict)
mydict = recursivedict()
I originally got this from here: Set nested dict value and create intermediate keys.
It is quite clever and elegant if you ask me.
First off, you probably want to look at setdefault.
As a function I'd write it as
def get_leaf_dict(dct, key_list):
res=dct
for key in key_list:
res=res.setdefault(key, {})
return res
This would be used as:
get_leaf_dict( dict, ['Person', 'address', 'city']) = 'New York'
This could be cleaned up with error handling and such. Also using *args rather than a single key-list argument might be nice; but the idea is that
you can iterate over the keys, pulling up the appropriate dictionary at each level.
Here is my simple solution: just write
terms = ['person', 'address', 'city']
result = nested_dict(3, str)
result[terms] = 'New York' # as easy as it can be
You can even do:
terms = ['John', 'Tinkoff', '1094535332'] # account in Tinkoff Bank
result = nested_dict(3, float)
result[terms] += 2375.30
Now the backstage:
from collections import defaultdict
class nesteddict(defaultdict):
def __getitem__(self, key):
if isinstance(key, list):
d = self
for i in key:
d = defaultdict.__getitem__(d, i)
return d
else:
return defaultdict.__getitem__(self, key)
def __setitem__(self, key, value):
if isinstance(key, list):
d = self[key[:-1]]
defaultdict.__setitem__(d, key[-1], value)
else:
defaultdict.__setitem__(self, key, value)
def nested_dict(n, type):
if n == 1:
return nesteddict(type)
else:
return nesteddict(lambda: nested_dict(n-1, type))
The dotty_dict library for Python 3 can do this. See documentation, Dotty Dict for more clarity.
from dotty_dict import dotty
dot = dotty()
string = '.'.join(['person', 'address', 'city'])
dot[string] = 'New York'
print(dot)
Output:
{'person': {'address': {'city': 'New York'}}}
Use these pair of methods
def gattr(d, *attrs):
"""
This method receives a dict and list of attributes to return the innermost value of the give dict
"""
try:
for at in attrs:
d = d[at]
return d
except:
return None
def sattr(d, *attrs):
"""
Adds "val" to dict in the hierarchy mentioned via *attrs
For ex:
sattr(animals, "cat", "leg","fingers", 4) is equivalent to animals["cat"]["leg"]["fingers"]=4
This method creates necessary objects until it reaches the final depth
This behaviour is also known as autovivification and plenty of implementation are around
This implementation addresses the corner case of replacing existing primitives
https://gist.github.com/hrldcpr/2012250#gistcomment-1779319
"""
for attr in attrs[:-2]:
# If such key is not found or the value is primitive supply an empty dict
if d.get(attr) is None or isinstance(d.get(attr), dict):
d[attr] = {}
d = d[attr]
d[attrs[-2]] = attrs[-1]
Here's a variant of Bakuriu's answer that doesn't rely on a separate function:
keys = ['Person', 'address', 'city']
value = 'New York'
nested_dict = {}
# Build nested dictionary up until 2nd to last key
# (Effectively nested_dict['Person']['address'] = {})
sub_dict = nested_dict
for key_ind, key in enumerate(keys[:-1]):
if not key_ind:
# Point to newly added piece of dictionary
sub_dict = nested_dict.setdefault(key, {})
else:
# Point to newly added piece of sub-dictionary
# that is also added to original dictionary
sub_dict = sub_dict.setdefault(key, {})
# Add value to last key of nested structure of keys
# (Effectively nested_dict['Person']['address']['city'] = value)
sub_dict[keys[-1]] = value
print(nested_dict)
>>> {'Person': {'address': {'city': 'New York'}}}
This is a pretty good use case for a recursive function. So you can do something like this:
def parse(l: list, v: str) -> dict:
copy = dict()
k, *s = l
if len(s) > 0:
copy[k] = parse(s, v)
else:
copy[k] = v
return copy
This effectively pops off the first value of the passed list l as a key for the dict copy that we initialize, then runs the remaining list through the same function, creating a new key under that key until there's nothing left in the list, whereupon it assigns the last value to the v param.
This is much easier in Perl:
my %hash;
$hash{"aaa"}{"bbb"}{"ccc"}=1; # auto creates each of the intermediate levels
# of the hash (aka: dict or associated array)