Related
I had trouble coming up with an appropriate title for this, so apologies there.
I have an existing dictionary di_end which already has an order to its keys. I also have some objects which have a property containing the keys for where in di_end the user-entered value will go
Note: the methods setProperty() and property() are from the pyqt library where setProperty() creates a custom property for an object where the first argument is the name of the property and the second argument is the value for that named property and property() just returns the values for whatever name is passed into it as an argument.
Something like this:
a.setProperty('keys', [key1, key2, key3])
b.setProperty('keys', [key4, key5, key6, key7, key8])
c.setProperty('keys', [key9])
objects_list = [a, b, c]
I want to be able to use the keys stored in the object properties to load the value that the user enters into a field to a dictionary
I'd like to iterate the process such that these parts
di_end[a.property(['keys'])[0]][a.property(['keys'])[1]][a.property(['keys'])[2]] = a.value
di_end[b.property(['keys'])[0]][b.property(['keys'])[1]][b.property(['keys'])[2]]\
[b.property(['keys'])[3]][b.property(['keys'])[4]] = b.value
di_end[c.property(['keys'])[0]] = c.value
or
a_li, b_li, c_li = a.property(['keys']), b.property(['keys']), c.property(['keys'])
di_end[a_li[0]][a_li[1]][a_li[2]] = a.value
di_end[b_li[0]][b_li[1]][b_li[2]][b_li[3]][b_li[4]] = b.value
di_end[c_li[0]] = c.value
do not need to be manually typed out and could be performed procedurally. I think I could do this if it was the same amount of keys but I'm not sure how to do it with differing amounts of keys. If they were all the same amounts I'd just do this
a.setProperty('keys', [key1, key2, key3, key4])
b.setProperty('keys', [key5, key6, key7, key8])
c.setProperty('keys', [key9, key10, key11, key12])
objects_list = [a, b, c]
a_li, b_li, c_li = a.property(['keys']), b.property(['keys']), c.property(['keys']) # assuming all are 4 entires each
for count, item in enumerate([a_li, b_li, c_li]):
di_end[item[0]][item[1]][item[2]][item[3]] = objects_list[count].value
but since there are different amounts of keys for each entry, I'm not sure how to accomplish this.
Edit: Added a note about setProperty() and property()
It's pretty hard to tell what your code is doing here, mostly due to the setProperty and property objects, which seem awkward and not-pythonic.
That being said, this looks like yet another use case for the excellent glom library (pip install glom).
from glom import assign, Path
assign(data, Path(*keys), value)
Note that in your case
data = di_end
keys = a.property['keys']
value = a.value
and an example:
>>> data = {'a': {'b': {'c': 5}}}
>>> keys = ['a', 'b', 'c']
>>> value = 100
>>> assign(data, Path(*keys), value)
{'a': {'b': {'c': 100}}}
>>> data
{'a': {'b': {'c': 100}}}
Note that this will raise a KeyError if the path doesn't already exist. For the generate-dict-on-demand (like a infinite nested defaultdict) style, you need to use Assign instead:
glom(data, Assign(Path(*keys), value))
See https://glom.readthedocs.io/en/latest/mutation.html for more details.
It seems you can try use recursion here. A simple example below.
put function creates nested dict and put value. fetch function can extract value from nested dict.
def fetch(data, keys):
if len(keys) == 1:
return data[keys[0]]
return fetch(data[keys.pop()], keys)
def put(data, keys, value):
if len(keys) == 1:
data[keys[0]] = value
return
key = keys.pop()
data[key] = {}
put(data[key], keys, value)
di_empty = {}
keys_values = [
(("a", "b", "c"), "abc"),
(("b", "c", "d", "e"), "bcde")
]
for selected in keys_values:
put(di_empty, list(reversed(selected[0])), selected[1])
di_end = dict()
di_end["a"] = {"b": {"c": "abc"}}
di_end["b"] = {"c":{"d": {"e": "bcde"}}}
keys = [("a", "b", "c"), ("b", "c", "d", "e")]
for selected in keys:
s1 = fetch(di_end, list(reversed(selected)))
s2 = fetch(di_empty, list(reversed(selected)))
assert s1 == s2
print(s1)
print(s2)
Say I have a dictionary that looks like this:
mappings = {"some_key": 3}
or it could look like this:
mappings = {"some_key": [4,5,6]}
Say I have a value 100 and a key of "some_key" in this function:
def add_to_mappings(key, value):
if key in mappings:
mappings[key] = ?
and I either want to add to the list if it exists or create one if it does not. At the end, I want my mappings to look like either:
mappings = {"some_key": [3, 100]}
or
mappings = {"some_key": [4,5,6,100]}
Without defaultdict:
mappings = dict()
def add_to_mappings(key, value):
try:
mappings[key].append(100)
except KeyError:
mappings[key] = [100]
With defaultdict:
from collections import defaultdict
mappings = defaultdict(list)
def add_to_mappings(key, value):
mappings[key].append(value)
Edit: I misunderstood the original requirements, to take an item if it already existed and create a list out of it and the new item, then the first example could be changed to this:
mappings = dict(foo=3)
def add_to_mappings(key, value):
try:
mappings[key].append(100)
except KeyError:
mappings[key] = [100]
except AttributeError:
mappings[key] = [mappings[key], value]
add_to_mappings("foo", 5)
# mappings ==> { "foo": [3, 5] }
You check if something is a list with isinstance(x, list). You can extract existing values from a dictionary and replace the value with simple assignment. So:
def add_to_mappings(d, key, value): # Remember to pass in the dict too!
if key in d:
# The value is present
v = d[k]
if isinstance(v, list):
# Already a list: just append to it
v.append(value)
else:
# Not a list: make a new list
d[k] = [v, value]
else:
# Not present at all: make a new list
d[key] = [value]
Let's say I have a regular "dict-of-dicts" as follows:
d = {}
d['a'] = {}
d['a']['b'] = 3
I can of course access the element using d['a']['b'].
In my case, I have a recursive application, in which I keep the current state as a list of keys. So I would have
my_key = ['a', 'b']
How do I access the value 3, using my_key? The issue, of course, is that my_key can be arbitrarily long (deep).
I realize I can write another traversal function or so, but it seems like there should be a straightforward way of doing so. Any ideas?
You could use reduce to iteratively index each layer of dict with a different key:
>>> from functools import reduce #only necessary in 3.X
>>> d = {}
>>> d['a'] = {} #I'm assuming this is what you meant to type
>>> d['a']['b'] = 3
>>> keys = ("a", "b")
>>> reduce(dict.get, keys, d)
3
Currently dictionary keys can only be hashable types, list (ListType) is not one of them, so if you try to specify a list as a dictionary key:
{}[[]]
you'll get:
TypeError: unhashable type: 'list'`.
You could enhance the current dictionary, allowing to specify a list as a key, and iterate over the list on inner objects. Here's the code (note that it only takes care of the get/read part):
from types import DictType, ListType
class EnhancedDictType(DictType):
def __getitem__(self, key):
if key and isinstance(key, ListType):
new_obj = self
for item in key:
new_obj = new_obj[item]
return new_obj
else:
return super(EnhancedDictType, self).__getitem__(key)
dict = EnhancedDictType
Here's also some test code:
d = dict()
d[1] = dict()
d[1][2] = dict({3: 4})
d[(1, 2, 3)] = 5
print d
print d[1]
print d[1][2]
print d[[1, 2]]
print d[[1 ,2, 3]]
print d[(1, 2, 3)]
I'm trying to programmatically set a value in a dictionary, potentially nested, given a list of indices and a value.
So for example, let's say my list of indices is:
['person', 'address', 'city']
and the value is
'New York'
I want as a result a dictionary object like:
{ 'Person': { 'address': { 'city': 'New York' } }
Basically, the list represents a 'path' into a nested dictionary.
I think I can construct the dictionary itself, but where I'm stumbling is how to set the value. Obviously if I was just writing code for this manually it would be:
dict['Person']['address']['city'] = 'New York'
But how do I index into the dictionary and set the value like that programmatically if I just have a list of the indices and the value?
Python
Something like this could help:
def nested_set(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
And you can use it like this:
>>> d = {}
>>> nested_set(d, ['person', 'address', 'city'], 'New York')
>>> d
{'person': {'address': {'city': 'New York'}}}
I took the freedom to extend the code from the answer of Bakuriu. Therefore upvotes on this are optional, as his code is in and of itself a witty solution, which I wouldn't have thought of.
def nested_set(dic, keys, value, create_missing=True):
d = dic
for key in keys[:-1]:
if key in d:
d = d[key]
elif create_missing:
d = d.setdefault(key, {})
else:
return dic
if keys[-1] in d or create_missing:
d[keys[-1]] = value
return dic
When setting create_missing to True, you're making sure to only set already existing values:
# Trying to set a value of a nonexistent key DOES NOT create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, False))
>>> {'A': {'B': 1}}
# Trying to set a value of an existent key DOES create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, True))
>>> {'A': {'B': 1, '8': 2}}
# Set the value of an existing key
print(nested_set({"A": {"B": 1}}, ["A", "B"], 2))
>>> {'A': {'B': 2}}
Here's another option:
from collections import defaultdict
recursivedict = lambda: defaultdict(recursivedict)
mydict = recursivedict()
I originally got this from here: Set nested dict value and create intermediate keys.
It is quite clever and elegant if you ask me.
First off, you probably want to look at setdefault.
As a function I'd write it as
def get_leaf_dict(dct, key_list):
res=dct
for key in key_list:
res=res.setdefault(key, {})
return res
This would be used as:
get_leaf_dict( dict, ['Person', 'address', 'city']) = 'New York'
This could be cleaned up with error handling and such. Also using *args rather than a single key-list argument might be nice; but the idea is that
you can iterate over the keys, pulling up the appropriate dictionary at each level.
Here is my simple solution: just write
terms = ['person', 'address', 'city']
result = nested_dict(3, str)
result[terms] = 'New York' # as easy as it can be
You can even do:
terms = ['John', 'Tinkoff', '1094535332'] # account in Tinkoff Bank
result = nested_dict(3, float)
result[terms] += 2375.30
Now the backstage:
from collections import defaultdict
class nesteddict(defaultdict):
def __getitem__(self, key):
if isinstance(key, list):
d = self
for i in key:
d = defaultdict.__getitem__(d, i)
return d
else:
return defaultdict.__getitem__(self, key)
def __setitem__(self, key, value):
if isinstance(key, list):
d = self[key[:-1]]
defaultdict.__setitem__(d, key[-1], value)
else:
defaultdict.__setitem__(self, key, value)
def nested_dict(n, type):
if n == 1:
return nesteddict(type)
else:
return nesteddict(lambda: nested_dict(n-1, type))
The dotty_dict library for Python 3 can do this. See documentation, Dotty Dict for more clarity.
from dotty_dict import dotty
dot = dotty()
string = '.'.join(['person', 'address', 'city'])
dot[string] = 'New York'
print(dot)
Output:
{'person': {'address': {'city': 'New York'}}}
Use these pair of methods
def gattr(d, *attrs):
"""
This method receives a dict and list of attributes to return the innermost value of the give dict
"""
try:
for at in attrs:
d = d[at]
return d
except:
return None
def sattr(d, *attrs):
"""
Adds "val" to dict in the hierarchy mentioned via *attrs
For ex:
sattr(animals, "cat", "leg","fingers", 4) is equivalent to animals["cat"]["leg"]["fingers"]=4
This method creates necessary objects until it reaches the final depth
This behaviour is also known as autovivification and plenty of implementation are around
This implementation addresses the corner case of replacing existing primitives
https://gist.github.com/hrldcpr/2012250#gistcomment-1779319
"""
for attr in attrs[:-2]:
# If such key is not found or the value is primitive supply an empty dict
if d.get(attr) is None or isinstance(d.get(attr), dict):
d[attr] = {}
d = d[attr]
d[attrs[-2]] = attrs[-1]
Here's a variant of Bakuriu's answer that doesn't rely on a separate function:
keys = ['Person', 'address', 'city']
value = 'New York'
nested_dict = {}
# Build nested dictionary up until 2nd to last key
# (Effectively nested_dict['Person']['address'] = {})
sub_dict = nested_dict
for key_ind, key in enumerate(keys[:-1]):
if not key_ind:
# Point to newly added piece of dictionary
sub_dict = nested_dict.setdefault(key, {})
else:
# Point to newly added piece of sub-dictionary
# that is also added to original dictionary
sub_dict = sub_dict.setdefault(key, {})
# Add value to last key of nested structure of keys
# (Effectively nested_dict['Person']['address']['city'] = value)
sub_dict[keys[-1]] = value
print(nested_dict)
>>> {'Person': {'address': {'city': 'New York'}}}
This is a pretty good use case for a recursive function. So you can do something like this:
def parse(l: list, v: str) -> dict:
copy = dict()
k, *s = l
if len(s) > 0:
copy[k] = parse(s, v)
else:
copy[k] = v
return copy
This effectively pops off the first value of the passed list l as a key for the dict copy that we initialize, then runs the remaining list through the same function, creating a new key under that key until there's nothing left in the list, whereupon it assigns the last value to the v param.
This is much easier in Perl:
my %hash;
$hash{"aaa"}{"bbb"}{"ccc"}=1; # auto creates each of the intermediate levels
# of the hash (aka: dict or associated array)
Let's say I have a pretty complex dictionary.
{'fruit':'orange','colors':{'dark':4,'light':5}}
Anyway, my objective is to scan every key in this complex multi-level dictionary. Then, append "abc" to the end of each key.
So that it will be:
{'fruitabc':'orange','colorsabc':{'darkabc':4,'lightabc':5}}
How would you do that?
Keys cannot be changed. You will need to add a new key with the modified value then remove the old one, or create a new dict with a dict comprehension or the like.
For example like this:
def appendabc(somedict):
return dict(map(lambda (key, value): (str(key)+"abc", value), somedict.items()))
def transform(multilevelDict):
new = appendabc(multilevelDict)
for key, value in new.items():
if isinstance(value, dict):
new[key] = transform(value)
return new
print transform({1:2, "bam":4, 33:{3:4, 5:7}})
This will append "abc" to each key in the dictionary and any value that is a dictionary.
EDIT: There's also a really cool Python 3 version, check it out:
def transform(multilevelDict):
return {str(key)+"abc" : (transform(value) if isinstance(value, dict) else value) for key, value in multilevelDict.items()}
print(transform({1:2, "bam":4, 33:{3:4, 5:7}}))
I use the following utility function that I wrote that takes a target dict and another dict containing the translation and switches all the keys according to it:
def rename_keys(d, keys):
return dict([(keys.get(k), v) for k, v in d.items()])
So with the initial data:
data = { 'a' : 1, 'b' : 2, 'c' : 3 }
translation = { 'a' : 'aaa', 'b' : 'bbb', 'c' : 'ccc' }
We get the following:
>>> data
{'a': 1, 'c': 3, 'b': 2}
>>> rename_keys(data, translation)
{'aaa': 1, 'bbb': 2, 'ccc': 3}
>>> mydict={'fruit':'orange','colors':{'dark':4,'light':5}}
>>> def f(mydict):
... return dict((k+"abc",f(v) if hasattr(v,'keys') else v) for k,v in mydict.items())
...
>>> f(mydict)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
My understanding is that you can't change the keys, and that you would need to make a new set of keys and assign their values to the ones the original keys were pointing to.
I'd do something like:
def change_keys(d):
if type(d) is dict:
return dict([(k+'abc', change_keys(v)) for k, v in d.items()])
else:
return d
new_dict = change_keys(old_dict)
here's a tight little function:
def keys_swap(orig_key, new_key, d):
d[new_key] = d.pop(orig_key)
for your particular problem:
def append_to_dict_keys(appendage, d):
#note that you need to iterate through the fixed list of keys, because
#otherwise we will be iterating through a never ending key list!
for each in d.keys():
if type(d[each]) is dict:
append_to_dict_keys(appendage, d[each])
keys_swap(each, str(each) + appendage, d)
append_to_dict_keys('abc', d)
#! /usr/bin/env python
d = {'fruit':'orange', 'colors':{'dark':4,'light':5}}
def add_abc(d):
newd = dict()
for k,v in d.iteritems():
if isinstance(v, dict):
v = add_abc(v)
newd[k + "abc"] = v
return newd
d = add_abc(d)
print d
Something like that
def applytoallkeys( dic, func ):
def yielder():
for k,v in dic.iteritems():
if isinstance( v, dict):
yield func(k), applytoallkeys( v, func )
else:
yield func(k), v
return dict(yielder())
def appendword( s ):
def appender( x ):
return x+s
return appender
d = {'fruit':'orange','colors':{'dark':4,'light':5}}
print applytoallkeys( d, appendword('asd') )
I kinda like functional style, you can read just the last line and see what it does ;-)
You could do this with recursion:
import collections
in_dict={'fruit':'orange','colors':{'dark':4,'light':5}}
def transform_dict(d):
out_dict={}
for k,v in d.iteritems():
k=k+'abc'
if isinstance(v,collections.MutableMapping):
v=transform_dict(v)
out_dict[k]=v
return out_dict
out_dict=transform_dict(in_dict)
print(out_dict)
# {'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
you should also consider that there is the possibility of nested dicts in nested lists, which will not be covered by the above solutions. This function ads a prefix and/or a postfix to every key within the dict.
def transformDict(multilevelDict, prefix="", postfix=""):
"""adds a prefix and/or postfix to every key name in a dict"""
new_dict = multilevelDict
if prefix != "" or postfix != "":
new_key = "%s#key#%s" % (prefix, postfix)
new_dict = dict(map(lambda (key, value): (new_key.replace('#key#', str(key)), value), new_dict.items()))
for key, value in new_dict.items():
if isinstance(value, dict):
new_dict[key] = transformDict(value, prefix, postfix)
elif isinstance(value, list):
for index, item in enumerate(value):
if isinstance(item, dict):
new_dict[key][index] = transformDict(item, prefix, postfix)
return new_dict
for k in theDict: theDict[k+'abc']=theDict.pop(k)
I use this for converting docopt POSIX-compliant command-line keys to PEP8 keys
(e.g. "--option" --> "option", "" --> "option2", "FILENAME" --> "filename")
arguments = docopt.docopt(__doc__) # dictionary
for key in arguments.keys():
if re.match('.*[-<>].*', key) or key != key.lower():
value = arguments.pop(key)
newkey = key.lower().translate(None, '-<>')
arguments[newkey] = value
Hi I'm a new user but finding an answer for same question, I can't get anything fully functional to my problem, I make this little piece of cake with a full nested replace of keys, you can send list with dict or dict.
Finally your dicts can have list with dict or more dict nested and it is all replaced with your new key needs.
To indicate who key want replace with a new key use "to" parameter sending a dict.
See at end my little example.
P/D: Sorry my bad english. =)
def re_map(value, to):
"""
Transform dictionary keys to map retrieved on to parameters.
to parameter should have as key a key name to replace an as value key name
to new dictionary.
this method is full recursive to process all levels of
#param value: list with dictionary or dictionary
#param to: dictionary with re-map keys
#type to: dict
#return: list or dict transformed
"""
if not isinstance(value, dict):
if not isinstance(value, list):
raise ValueError(
"Only dict or list with dict inside accepted for value argument.") # #IgnorePep8
if not isinstance(to, dict):
raise ValueError("Only dict accepted for to argument.")
def _re_map(value, to):
if isinstance(value, dict):
# Re map dictionary key.
# If key of original dictionary is not in "to" dictionary use same
# key otherwise use re mapped key on new dictionary with already
# value.
return {
to.get(key) or key: _re_map(dict_value, to)
for key, dict_value in value.items()
}
elif isinstance(value, list):
# if value is a list iterate it a call _re_map again to parse
# values on it.
return [_re_map(item, to) for item in value]
else:
# if not dict or list only return value.
# it can be string, integer or others.
return value
result = _re_map(value, to)
return result
if __name__ == "__main__":
# Sample test of re_map method.
# -----------------------------------------
to = {"$id": "id"}
x = []
for i in range(100):
x.append({
"$id": "first-dict",
"list_nested": [{
"$id": "list-dict-nested",
"list_dic_nested": [{
"$id": "list-dict-list-dict-nested"
}]
}],
"dict_nested": {
"$id": "non-nested"
}
})
result = re_map(x, to)
print(str(result))
A functional (and flexible) solution: this allows an arbitrary transform to be applied to keys (recursively for embedded dicts):
def remap_keys(d, keymap_f):
"""returns a new dict by recursively remapping all of d's keys using keymap_f"""
return dict([(keymap_f(k), remap_keys(v, keymap_f) if isinstance(v, dict) else v)
for k,v in d.items()])
Let's try it out; first we define our key transformation function, then apply it to the example:
def transform_key(key):
"""whatever transformation you'd like to apply to keys"""
return key + "abc"
remap_keys({'fruit':'orange','colors':{'dark':4,'light':5}}, transform_key)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
(note: if you're still on Python 2.x, you'll need to replace d.items() on the last line with d.iteritems() -- thanks to #Rudy for reminding me to update this post for Python 3).
Based on #AndiDog's python 3 version and similar to #sxc731's version but with a flag for whether to apply it recursively:
def transform_keys(dictionary, key_fn, recursive=True):
"""
Applies function to keys and returns as a new dictionary.
Example of key_fn:
lambda k: k + "abc"
"""
return {key_fn(key): (transform_keys(value, key_fn=key_fn, recursive=recursive)
if recursive and isinstance(value, dict) else value)
for key, value in dictionary.items()}