I have a dictionary like this:
dct = {'one': 'value',
'two': ['value1','value2','value1'],
'three':['otherValue1','otherValue2','otherValue1'],
'dontCareAboutThisKey':'debug'}
I need to remove duplicate values from the lists. I wrote a function to do this:
no_dups = {}
for keys in dct:
if isinstance(dct[keys], list) and keys != 'dontCareAboutThisKey':
for value in dct[keys]:
if value not in no_dups.values():
no_dups[keys].append(value)
else:
no_dups[keys] = dct[keys]
I'm checking if value of the current key is a list. If no, it just 'copy' key to no_dups dictionary. If it is a list and not a key that I don't care about (there are no duplicates for sure) - it should check if current value already exists in no_dups.values() and append it to current key. Problem is that I'm getting an error:
KeyError: 'two:'
I know it's because I'm trying to add a value to non existing key but I have no idea how to deal with this and make it work.
I think the best way to deal with adding the key and appending at the same time is with dicts' setdefault() method:
no_dups.setdefault(keys,[]).append(value)
But rather than that, you can do this in a more neat way like this:
#remove duplicates
no_dups = {k:list(set(v)) if isinstance(v, list) and k != 'dontCareAboutThisKey' else v
for k,v in dct.items()} # or dct.iteritems() if using python2.x
That hack will, for key value combinations that pass the if test, convert the list into a set (removing duplicates) and then in a list again. For other key value combinations it will leave it intact.
dct = {'one': 'value',
'two': ['value1','value2','value1'],
'three':['otherValue1','otherValue2','otherValue1'],
'dontCareAboutThisKey':'debug'}
set(dct) returns a set, which is a list without duplicates:
for key, value in dct.items():
if not isinstance(value, basestring):
dct[key] = set(value)
If you need a new dictionary you could do:
new_dct = {}
for key, value in dct.items():
if not isinstance(value, basestring):
new_dct[key] = set(value)
else:
new_dct[key] = value
If You want to remove duplicates, just change You list to set, with set() function:
https://docs.python.org/2/tutorial/datastructures.html#sets
It automatically gives You unique set, then You can always change it back to list.
I need a function to change one item in composite dictionary.
I've tried something like..
def SetItem(keys, value):
item = self.dict
for key in keys:
item = item[key]
item = value
and
SetItem(['key1', 'key2'], 86)
It should be equivalent to self.dict['key1']['key2'] = 86, but this function has no effect.
Almost. You actually want to do something like:
def set_keys(d, keys, value):
item = d
for key in keys[:-1]:
item = item[key]
item[keys[-1]] = value
Or recursively like this:
def set_key(d, keys, value):
if len(keys) == 1:
d[keys[0]] = value
else:
set_key(d[keys[0]], keys[1:], value)
Marcin's right though. You would really want to incorporate something more rigorous, with some error handling for missing keys/missing dicts.
setItem = lambda self,names,value: map((lambda name: setattr(self,name,value)),names)
You don't have a self parameter
Just use the line of working code you have.
If you insist, here's a way:
def setitem(self, keys, value):
reduce(dict.get, # = lambda dictionary, key: dictionary[key]
keys[:-1], self.dictionary)[keys[-1]] = value
Obviously, this will break if the list of keys hits a non-dict value. You'll want to handle that. In fact, an explicit loop would probably be better for that reason, but you get the idea.
An idea involving recursion and EAFP, both of which I always like:
def set_item(d, keys, value):
key = keys.pop(0)
try:
set_item(d[key], keys, value)
# IndexError happens when the pop fails (empty list), KeyError happens when it's not a dict.
# Assume both mean we should finish recursing
except (IndexError, KeyError):
d[key] = value
Example:
>>> d = {'a': {'aa':1, 'ab':2}, 'b':{'ba':1, 'bb':2}}
>>> set_item(d, ['a', 'ab'], 50)
>>> print d
{'a': {'aa': 1, 'ab': 50}, 'b': {'ba': 1, 'bb': 2}}
Edit: As Marcin points out below, this will not work for arbitrarily nested dicts since Python has a recursion limit. It's also not for highly performance-sensitive situations (recursion in Python generally isn't). Nonetheless, outside of these two situations I find this to be somewhat more explicit than something involving reduce or lambda.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Inverse dictionary lookup - Python
Is there a built in way to index a dictionary by value in Python.
e.g. something like:
dict = {'fruit':'apple','colour':'blue','meat':'beef'}
print key where dict[key] == 'apple'
or:
dict = {'fruit':['apple', 'banana'], 'colour':'blue'}
print key where 'apple' in dict[key]
or do I have to manually loop it?
You could use a list comprehension:
my_dict = {'fruit':'apple','colour':'blue','meat':'beef'}
print [key for key, value in my_dict.items() if value == 'apple']
The code above is doing almost exactly what said you want:
print key where dict[key] == 'apple'
The list comprehension is going through all the key, value pairs given by your dictionary's items method, and making a new list of all the keys where the value is 'apple'.
As Niklas pointed out, this does not work when your values could potentially be lists. You have to be careful about just using in in this case since 'apple' in 'pineapple' == True. So, sticking with a list comprehension approach requires some type checking. So, you could use a helper function like:
def equals_or_in(target, value):
"""Returns True if the target string equals the value string or,
is in the value (if the value is not a string).
"""
if isinstance(target, str):
return target == value
else:
return target in value
Then, the list comprehension below would work:
my_dict = {'fruit':['apple', 'banana'], 'colour':'blue'}
print [key for key, value in my_dict.items() if equals_or_in('apple', value)]
You'll have to manually loop it, but if you'll need the lookup repeatedly this is a handy trick:
d1 = {'fruit':'apple','colour':'blue','meat':'beef'}
d1_rev = dict((v, k) for k, v in d1.items())
You can then use the reverse dictionary like this:
>>> d1_rev['blue']
'colour'
>>> d1_rev['beef']
'meat'
Your requirements are more complex than you realize:
You need to handle both list values and plain values
You don't actually need to get back a key, but a list of keys
You could solve this in two steps:
normalize the dict so that every value is a list (every plain value becomes a single-element)
build a reverse dictionary
The following functions will solve this:
from collections import defaultdict
def normalize(d):
return { k:(v if isinstance(v, list) else [v]) for k,v in d.items() }
def build_reverse_dict(d):
res = defaultdict(list)
for k,values in normalize(d).items():
for x in values:
res[x].append(k)
return dict(res)
To be used like this:
>>> build_reverse_dict({'fruit':'apple','colour':'blue','meat':'beef'})
{'blue': ['colour'], 'apple': ['fruit'], 'beef': ['meat']}
>>> build_reverse_dict({'fruit':['apple', 'banana'], 'colour':'blue'})
{'blue': ['colour'], 'apple': ['fruit'], 'banana': ['fruit']}
>>> build_reverse_dict({'a':'duplicate', 'b':['duplicate']})
{'duplicate': ['a', 'b']}
So you just build up the reverse dictionary once and then lookup by value and get back a list of keys.
Let's say I have a pretty complex dictionary.
{'fruit':'orange','colors':{'dark':4,'light':5}}
Anyway, my objective is to scan every key in this complex multi-level dictionary. Then, append "abc" to the end of each key.
So that it will be:
{'fruitabc':'orange','colorsabc':{'darkabc':4,'lightabc':5}}
How would you do that?
Keys cannot be changed. You will need to add a new key with the modified value then remove the old one, or create a new dict with a dict comprehension or the like.
For example like this:
def appendabc(somedict):
return dict(map(lambda (key, value): (str(key)+"abc", value), somedict.items()))
def transform(multilevelDict):
new = appendabc(multilevelDict)
for key, value in new.items():
if isinstance(value, dict):
new[key] = transform(value)
return new
print transform({1:2, "bam":4, 33:{3:4, 5:7}})
This will append "abc" to each key in the dictionary and any value that is a dictionary.
EDIT: There's also a really cool Python 3 version, check it out:
def transform(multilevelDict):
return {str(key)+"abc" : (transform(value) if isinstance(value, dict) else value) for key, value in multilevelDict.items()}
print(transform({1:2, "bam":4, 33:{3:4, 5:7}}))
I use the following utility function that I wrote that takes a target dict and another dict containing the translation and switches all the keys according to it:
def rename_keys(d, keys):
return dict([(keys.get(k), v) for k, v in d.items()])
So with the initial data:
data = { 'a' : 1, 'b' : 2, 'c' : 3 }
translation = { 'a' : 'aaa', 'b' : 'bbb', 'c' : 'ccc' }
We get the following:
>>> data
{'a': 1, 'c': 3, 'b': 2}
>>> rename_keys(data, translation)
{'aaa': 1, 'bbb': 2, 'ccc': 3}
>>> mydict={'fruit':'orange','colors':{'dark':4,'light':5}}
>>> def f(mydict):
... return dict((k+"abc",f(v) if hasattr(v,'keys') else v) for k,v in mydict.items())
...
>>> f(mydict)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
My understanding is that you can't change the keys, and that you would need to make a new set of keys and assign their values to the ones the original keys were pointing to.
I'd do something like:
def change_keys(d):
if type(d) is dict:
return dict([(k+'abc', change_keys(v)) for k, v in d.items()])
else:
return d
new_dict = change_keys(old_dict)
here's a tight little function:
def keys_swap(orig_key, new_key, d):
d[new_key] = d.pop(orig_key)
for your particular problem:
def append_to_dict_keys(appendage, d):
#note that you need to iterate through the fixed list of keys, because
#otherwise we will be iterating through a never ending key list!
for each in d.keys():
if type(d[each]) is dict:
append_to_dict_keys(appendage, d[each])
keys_swap(each, str(each) + appendage, d)
append_to_dict_keys('abc', d)
#! /usr/bin/env python
d = {'fruit':'orange', 'colors':{'dark':4,'light':5}}
def add_abc(d):
newd = dict()
for k,v in d.iteritems():
if isinstance(v, dict):
v = add_abc(v)
newd[k + "abc"] = v
return newd
d = add_abc(d)
print d
Something like that
def applytoallkeys( dic, func ):
def yielder():
for k,v in dic.iteritems():
if isinstance( v, dict):
yield func(k), applytoallkeys( v, func )
else:
yield func(k), v
return dict(yielder())
def appendword( s ):
def appender( x ):
return x+s
return appender
d = {'fruit':'orange','colors':{'dark':4,'light':5}}
print applytoallkeys( d, appendword('asd') )
I kinda like functional style, you can read just the last line and see what it does ;-)
You could do this with recursion:
import collections
in_dict={'fruit':'orange','colors':{'dark':4,'light':5}}
def transform_dict(d):
out_dict={}
for k,v in d.iteritems():
k=k+'abc'
if isinstance(v,collections.MutableMapping):
v=transform_dict(v)
out_dict[k]=v
return out_dict
out_dict=transform_dict(in_dict)
print(out_dict)
# {'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
you should also consider that there is the possibility of nested dicts in nested lists, which will not be covered by the above solutions. This function ads a prefix and/or a postfix to every key within the dict.
def transformDict(multilevelDict, prefix="", postfix=""):
"""adds a prefix and/or postfix to every key name in a dict"""
new_dict = multilevelDict
if prefix != "" or postfix != "":
new_key = "%s#key#%s" % (prefix, postfix)
new_dict = dict(map(lambda (key, value): (new_key.replace('#key#', str(key)), value), new_dict.items()))
for key, value in new_dict.items():
if isinstance(value, dict):
new_dict[key] = transformDict(value, prefix, postfix)
elif isinstance(value, list):
for index, item in enumerate(value):
if isinstance(item, dict):
new_dict[key][index] = transformDict(item, prefix, postfix)
return new_dict
for k in theDict: theDict[k+'abc']=theDict.pop(k)
I use this for converting docopt POSIX-compliant command-line keys to PEP8 keys
(e.g. "--option" --> "option", "" --> "option2", "FILENAME" --> "filename")
arguments = docopt.docopt(__doc__) # dictionary
for key in arguments.keys():
if re.match('.*[-<>].*', key) or key != key.lower():
value = arguments.pop(key)
newkey = key.lower().translate(None, '-<>')
arguments[newkey] = value
Hi I'm a new user but finding an answer for same question, I can't get anything fully functional to my problem, I make this little piece of cake with a full nested replace of keys, you can send list with dict or dict.
Finally your dicts can have list with dict or more dict nested and it is all replaced with your new key needs.
To indicate who key want replace with a new key use "to" parameter sending a dict.
See at end my little example.
P/D: Sorry my bad english. =)
def re_map(value, to):
"""
Transform dictionary keys to map retrieved on to parameters.
to parameter should have as key a key name to replace an as value key name
to new dictionary.
this method is full recursive to process all levels of
#param value: list with dictionary or dictionary
#param to: dictionary with re-map keys
#type to: dict
#return: list or dict transformed
"""
if not isinstance(value, dict):
if not isinstance(value, list):
raise ValueError(
"Only dict or list with dict inside accepted for value argument.") # #IgnorePep8
if not isinstance(to, dict):
raise ValueError("Only dict accepted for to argument.")
def _re_map(value, to):
if isinstance(value, dict):
# Re map dictionary key.
# If key of original dictionary is not in "to" dictionary use same
# key otherwise use re mapped key on new dictionary with already
# value.
return {
to.get(key) or key: _re_map(dict_value, to)
for key, dict_value in value.items()
}
elif isinstance(value, list):
# if value is a list iterate it a call _re_map again to parse
# values on it.
return [_re_map(item, to) for item in value]
else:
# if not dict or list only return value.
# it can be string, integer or others.
return value
result = _re_map(value, to)
return result
if __name__ == "__main__":
# Sample test of re_map method.
# -----------------------------------------
to = {"$id": "id"}
x = []
for i in range(100):
x.append({
"$id": "first-dict",
"list_nested": [{
"$id": "list-dict-nested",
"list_dic_nested": [{
"$id": "list-dict-list-dict-nested"
}]
}],
"dict_nested": {
"$id": "non-nested"
}
})
result = re_map(x, to)
print(str(result))
A functional (and flexible) solution: this allows an arbitrary transform to be applied to keys (recursively for embedded dicts):
def remap_keys(d, keymap_f):
"""returns a new dict by recursively remapping all of d's keys using keymap_f"""
return dict([(keymap_f(k), remap_keys(v, keymap_f) if isinstance(v, dict) else v)
for k,v in d.items()])
Let's try it out; first we define our key transformation function, then apply it to the example:
def transform_key(key):
"""whatever transformation you'd like to apply to keys"""
return key + "abc"
remap_keys({'fruit':'orange','colors':{'dark':4,'light':5}}, transform_key)
{'fruitabc': 'orange', 'colorsabc': {'darkabc': 4, 'lightabc': 5}}
(note: if you're still on Python 2.x, you'll need to replace d.items() on the last line with d.iteritems() -- thanks to #Rudy for reminding me to update this post for Python 3).
Based on #AndiDog's python 3 version and similar to #sxc731's version but with a flag for whether to apply it recursively:
def transform_keys(dictionary, key_fn, recursive=True):
"""
Applies function to keys and returns as a new dictionary.
Example of key_fn:
lambda k: k + "abc"
"""
return {key_fn(key): (transform_keys(value, key_fn=key_fn, recursive=recursive)
if recursive and isinstance(value, dict) else value)
for key, value in dictionary.items()}