I have dictionary that is built as part of the initialization of my object. I know that it will not change during the lifetime of the object. The dictionary maps keys to sets. I want to convert all the values from sets to frozensets, to make sure they do not get changed. Currently I do that like this:
for key in self.my_dict.iterkeys():
self.my_dict[key] = frozenset(self.my_dict[key])
Is there a simpler way to achieve this? I cannot build frozenset right away, because I do not how much items will be in each set until i have built the complete dictionary.
Given, for instance,
>>> d = {'a': set([1, 2]), 'b': set([3, 4])}
>>> d
{'a': set([1, 2]), 'b': set([3, 4])}
You can do the conversion in place as
>>> d.update((k, frozenset(v)) for k, v in d.iteritems())
With the result
>>> d
{'a': frozenset([1, 2]), 'b': frozenset([3, 4])}
If you have to do it in-place, probably this is the simplest way (almost the same as you posted):
for key, value in self.my_dict.iteritems():
self.my_dict[key] = frozenset(value)
This a variant which builds a temporary dict:
self.my_dict = dict(((key, frozenset(value)) \
for key, value in self.my_dict.iteritems()))
In Python 3, you could use a dictionary comprehension:
d = {k: frozenset(v) for k, v in d.items()}
In Python 2, though, I don't know that there's anything shorter -- this at least feels less "redundant":
for k,v in d.iteritems():
d[k] = frozenset(v)
Related
I have a dictionary that looks like this: d = {'a': '[1]', 'b': '[2]'}. I want to covert the string value into a float and have d = {'a': [1], 'b': [2]}, so the value should be a list with a single element of float type. How to do that?
{k:[float(v.replace(']','').replace('[',''))] for k, v in d.items()}
OR
{k:[float(v[1,-1])] for k, v in d.items()}
ast.literal_eval would create a list of int but OP required explicitly a list of float. One could:
import ast
{k: [float(a) for a in ast.literal_eval(v)] for k, v in d.items()}
You are probably looking for python's eval() function:
d = {'a': '[1]', 'b': '[2]'}
for k in d:
d[k] = eval(d[k])
print(d)
And the output:
{'a': [1], 'b': [2]}
EDIT
As stated in the comments, eval can be sometimes dangerous. So here's an alternative:
d = {'a': '[1]', 'b': '[2]'}
for k in d:
d[k] = [float(d[k][1:-1])] # because the list conatins a single element
print(d)
EDIT2
If you want the alternative to work with lists with multiple values:
replace this line
d[k] = [float(d[k][1:-1])]
with
d[k] = list(map(lambda x: float(x.strip()), d[k][1:-1].split(',')))
I will just add a reference to the Python PEP 274 syntactic sugar for this sort of thing, the Dict Comprehension:
>>> d = {'a': '[1]', 'b': '[2]'}
>>> d = {k: eval(v) for k, v in d.items()}
>>> d
{'a': [1], 'b': [2]}
This is slightly more compact, but works the exact same.
You also made the following note:
so the value should be a list with a single element of float type.
The above code could be modified to do this easily enough, but I should mention that, for the most part, you can use Python integers anywhere you would want a float.
>>> d = {'a': '[1]', 'b': '[2]'}
>>> d = {k: [float(eval(v)[0])] for k, v in d.items()}
>>> d
{'a': [1.0], 'b': [2.0]}
EDIT
I can't reply to your comment because of reputation, JoKing, but it's a great reminder. However, the typical use case for this conversion is probably not fetching source code from potentially malicious 3rd parties and executing it locally. So I would say that, in the case of simple data handling where you control the data io and your source code isn't public or published, eval() is absolutely fine.
OP added an additional question in regards to those mentioned dangers of eval():
So instead of '[1]' I have say '[1, 2]'
If you use previous methods to strip off the brackets, you can use .split(",") to separate the list into individuals strings to be converted. Using the dict comprehension is becoming more ugly here (because it contains a list comprehension to account for this new case), but it still works:
>>> d = {'a': '[1, 2]', 'b': '[2]'}
>>> d = {k: [float(i) for i in v[1:-1].split(",")] for k, v in d.items()}
{'a': [1.0, 2.0], 'b': [2.0]}
Is there any built-in function that would do the following?
dictionary = {‘a’:1, ‘b’:2, ‘c’:3}
dictionary.update(c=10)
# what happens
dictionary ---- {‘a’:1, ‘b’:2, ‘c’:10}
# what I want to happen:
dictionary ---- {‘a’:1, ‘b’:2, ‘c’:(3, 10)}
By default if keys are the same, later key would override earlier one.
If the key is already present in dict, the value of the new key: value pair would be added to already existing value in a form of container, like tuple, or list or set.
I can write a helper function to do so but I believe it should be something built-in for this matter.
You can do this
from collections import defaultdict
d = defaultdict(list)
d["a"].append(1)
d["b"].append(2)
d["c"].append(3)
d["c"].append(10)
print(d)
Result
defaultdict(list, {'a': [1], 'b': [2], 'c': [3, 10]})
Your desired solution is not very elegant, so I am going to propose an alternative one.
Tuples are immutable. Let's use lists instead, because we can easily append to them.
The data type of the values should be consistent. Use lists in any case, even for single values.
Let's use a defaultdict such that we don't have to initialize lists manually.
Putting it together:
>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> for v, k in enumerate('abc', 1):
... d[k].append(v)
...
>>> d
defaultdict(<class 'list'>, {'a': [1], 'b': [2], 'c': [3]})
>>> d['c'].append(10)
>>> d
defaultdict(<class 'list'>, {'a': [1], 'b': [2], 'c': [3, 10]})
You could rewrite the update function by creating a new class:
In Python bulitins.py:
def update(self, E=None, **F): # known special case of dict.update
"""
D.update([E, ]**F) -> None. Update D from dict/iterable E and F.
If E is present and has a .keys() method, then does: for k in E: D[k] = E[k]
If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v
In either case, this is followed by: for k in F: D[k] = F[k]
"""
pass
So I write this(Inherit from UserDict, suggested by #timgeb):
from collections import UserDict
class CustomDict(UserDict):
def __init__(self):
super().__init__()
def update(self, E=None, **F) -> None:
if E:
if isinstance(E, dict):
for k in E:
self[k] = E[k]
else:
for k, v in E:
self[k] = v
else:
if isinstance(F, dict):
for key in F:
if isinstance(self[key], list):
self[key].append(F[key])
else:
self[key] = [self[key], F[key]]
dictionary = CustomDict()
dictionary.update({'a': 1, 'b': 2, 'c': 3})
print(dictionary)
dictionary.update(a=3)
print(dictionary)
dictionary.update(a=4)
print(dictionary)
Result:
{'a': 1, 'b': 2, 'c': 3}
{'a': [1, 3], 'b': 2, 'c': 3}
{'a': [1, 3, 4], 'b': 2, 'c': 3}
Maybe there are some logic errors in my code,but welcome to point out.
Perhaps you could use something like:
dictionary = {'a':1, 'b':2, 'c':3}
dictionary.update({'c': 10 if not dictionary.get('c') else tuple([dictionary['c'],] + [10,])})
# {'a': 1, 'b': 2, 'c': (3, 10)}
But it should probably be wrapped into a function to make things clean. The general pattern would be (I suppose, based on your question):
dict = {...}
if 'a' not in dict:
do_this() # just add it to the dict?
else:
do_that() # build a tuple or list?
In your above question you're mixing types -- I'm not sure if you want that, a more pythonic approach might be to have all the values as list and use a defaultdict.
I have a dictionary of lists in which some of the values are empty:
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
At the end of creating these lists, I want to remove these empty lists before returning my dictionary. I tried doing it like this:
for i in d:
if not d[i]:
d.pop(i)
but I got a RuntimeError. I am aware that you cannot add/remove elements in a dictionary while iterating through it...what would be a way around this then?
See Modifying a Python dict while iterating over it for citations that this can cause problems, and why.
In Python 3.x and 2.x you can use use list to force a copy of the keys to be made:
for i in list(d):
In Python 2.x calling keys made a copy of the keys that you could iterate over while modifying the dict:
for i in d.keys():
But note that in Python 3.x this second method doesn't help with your error because keys returns an a view object instead of copying the keys into a list.
You only need to use copy:
This way you iterate over the original dictionary fields and on the fly can change the desired dict d.
It works on each Python version, so it's more clear.
In [1]: d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
In [2]: for i in d.copy():
...: if not d[i]:
...: d.pop(i)
...:
In [3]: d
Out[3]: {'a': [1], 'b': [1, 2]}
(BTW - Generally to iterate over copy of your data structure, instead of using .copy for dictionaries or slicing [:] for lists, you can use import copy -> copy.copy (for shallow copy which is equivalent to copy that is supported by dictionaries or slicing [:] that is supported by lists) or copy.deepcopy on your data structure.)
Just use dictionary comprehension to copy the relevant items into a new dict:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.items() if v}
>>> d
{'a': [1], 'b': [1, 2]}
For this in Python 2:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.iteritems() if v}
>>> d
{'a': [1], 'b': [1, 2]}
This worked for me:
d = {1: 'a', 2: '', 3: 'b', 4: '', 5: '', 6: 'c'}
for key, value in list(d.items()):
if value == '':
del d[key]
print(d)
# {1: 'a', 3: 'b', 6: 'c'}
Casting the dictionary items to list creates a list of its items, so you can iterate over it and avoid the RuntimeError.
I would try to avoid inserting empty lists in the first place, but, would generally use:
d = {k: v for k,v in d.iteritems() if v} # re-bind to non-empty
If prior to 2.7:
d = dict( (k, v) for k,v in d.iteritems() if v )
or just:
empty_key_vals = list(k for k in k,v in d.iteritems() if v)
for k in empty_key_vals:
del[k]
To avoid "dictionary changed size during iteration error".
For example: "when you try to delete some key",
Just use 'list' with '.items()'. Here is a simple example:
my_dict = {
'k1':1,
'k2':2,
'k3':3,
'k4':4
}
print(my_dict)
for key, val in list(my_dict.items()):
if val == 2 or val == 4:
my_dict.pop(key)
print(my_dict)
Output:
{'k1': 1, 'k2': 2, 'k3': 3, 'k4': 4}
{'k1': 1, 'k3': 3}
This is just an example. Change it based on your case/requirements.
For Python 3:
{k:v for k,v in d.items() if v}
You cannot iterate through a dictionary while it’s changing during a for loop. Make a casting to list and iterate over that list. It works for me.
for key in list(d):
if not d[key]:
d.pop(key)
Python 3 does not allow deletion while iterating (using the for loop above) a dictionary. There are various alternatives to do it; one simple way is to change the line
for i in x.keys():
with
for i in list(x)
The reason for the runtime error is that you cannot iterate through a data structure while its structure is changing during iteration.
One way to achieve what you are looking for is to use a list to append the keys you want to remove and then use the pop function on dictionary to remove the identified key while iterating through the list.
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
pop_list = []
for i in d:
if not d[i]:
pop_list.append(i)
for x in pop_list:
d.pop(x)
print (d)
For situations like this, I like to make a deep copy and loop through that copy while modifying the original dict.
If the lookup field is within a list, you can enumerate in the for loop of the list and then specify the position as the index to access the field in the original dict.
Nested null values
Let's say we have a dictionary with nested keys, some of which are null values:
dicti = {
"k0_l0":{
"k0_l1": {
"k0_l2": {
"k0_0":None,
"k1_1":1,
"k2_2":2.2
}
},
"k1_l1":None,
"k2_l1":"not none",
"k3_l1":[]
},
"k1_l0":"l0"
}
Then we can remove the null values using this function:
def pop_nested_nulls(dicti):
for k in list(dicti):
if isinstance(dicti[k], dict):
dicti[k] = pop_nested_nulls(dicti[k])
elif not dicti[k]:
dicti.pop(k)
return dicti
Output for pop_nested_nulls(dicti)
{'k0_l0': {'k0_l1': {'k0_l2': {'k1_1': 1,
'k2_2': 2.2}},
'k2_l1': 'not '
'none'},
'k1_l0': 'l0'}
The Python "RuntimeError: dictionary changed size during iteration" occurs when we change the size of a dictionary when iterating over it.
To solve the error, use the copy() method to create a shallow copy of the dictionary that you can iterate over, e.g., my_dict.copy().
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in my_dict.copy():
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
You can also convert the keys of the dictionary to a list and iterate over the list of keys.
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in list(my_dict.keys()):
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
If the values in the dictionary were unique too, then I used this solution:
keyToBeDeleted = None
for k, v in mydict.items():
if(v == match):
keyToBeDeleted = k
break
mydict.pop(keyToBeDeleted, None)
I have two or more dictionary, I like to merge it as one with retaining multiple values of the same key as list. I would not able to share the original code, so please help me with the following example.
Input:
a= {'a':1, 'b': 2}
b= {'aa':4, 'b': 6}
c= {'aa':3, 'c': 8}
Output:
c= {'a':1,'aa':[3,4],'b': [2,6], 'c': 8}
I suggest you read up on the defaultdict: it lets you provide a factory method that initializes missing keys, i.e. if a key is looked up but not found, it creates a value by calling factory_method(missing_key). See this example, it might make things clearer:
from collections import defaultdict
a = {'a': 1, 'b': 2}
b = {'aa': 4, 'b': 6}
c = {'aa': 3, 'c': 8}
stuff = [a, b, c]
# our factory method is the list-constructor `list`,
# so whenever we look up a value that doesn't exist, a list is created;
# we can always be sure that we have list-values
store = defaultdict(list)
for s in stuff:
for k, v in s.items():
# since we know that our value is always a list, we can safely append
store[k].append(v)
print(store)
This has the "downside" of creating one-element lists for single occurences of values, but maybe you are able to work around that.
Please find below to resolve your issue. I hope this would work for you.
from collections import defaultdict
a = {'a':1, 'b': 2}
b = {'aa':4, 'b': 6}
c={'aa':3, 'c': 8}
dd = defaultdict(list)
for d in (a,b,c):
for key, value in d.items():
dd[key].append(value)
print(dd)
Use defaultdict to automatically create a dictionary entry with an empty list.
To process all source dictionaries in a single loop, use itertools.chain.
The main loop just adds a value from the current item, to the list under
the current key.
As you wrote, for cases when under some key there is only one item,
you have to generate a work dictionary (using dictonary comprehension),
limited to items with value (list) containing only one item.
The value of such item shoud contain only the first (and only) number
from the source list.
Then use this dictionary to update d.
So the whole script can be surprisingly short, as below:
from collections import defaultdict
from itertools import chain
a = {'a':1, 'b': 2}
b = {'aa':4, 'b': 6}
c = {'aa':3, 'c': 8}
d = defaultdict(list)
for k, v in chain(a.items(), b.items(), c.items()):
d[k].append(v)
d.update({ k: v[0] for k, v in d.items() if len(v) == 1 })
As you can see, the actual processing code is contained in only 4 (last) lines.
If you print d, the result is:
defaultdict(list, {'a': 1, 'b': [2, 6], 'aa': [4, 3], 'c': 8})
I have 0 or more dicts in a list:
>>> dicts = [dict(a=3, b=89, d=2), dict(a=3, b=89, c=99), dict(a=3, b=42, c=33)]
I want to create a new dict that contains only keys that are in all the above dicts, and only if the values are all the same:
>>> dict_intersection(*dicts)
{"a": 3}
I feel that there should be an elegant way of writing dict_intersection, but I'm only coming up with inelegant and/or inefficient solutions myself.
>>> dict(set.intersection(*(set(d.iteritems()) for d in dicts)))
{'a': 3}
Note: This solution requires the dictionary values to be hashable, in addition to the keys.
Since the key/value pairs must already be in the first dict, you can iterate over this dict's items.
dict(pair for pair in dicts[0].items()
if all((pair in d.items() for d in dicts[1:])))
Looks less elegant than interjay's answer, but works without the restriction of hashable values.
Edit: Changed the all expression to a generator expression for speed improvement
How's this?
def intersect_two_dicts (d1, d2):
return { k:v for k,v in d1.iteritems() if ((k in d2)and(d1[k]==d2[k])) }
def intersect_dicts (list_of_dicts):
return reduce(intersect_two_dicts, list_of_dicts)
# Tests
dicts = [dict(a=3, b=89, d=2), dict(a=3, b=89, c=99), dict(a=3, b=42, c=33)]
print (intersect_two_dicts(dicts[0], dicts[1]))
print (intersect_dicts(dicts))
Edit(1): I'm not sure which of these is fastest. The set.intersection solutions are certainly most elegant (short one liners!) but I would be interested to see some benchmarking.
Edit(2): Bonus - get any dictionary entries whose (key:value) pairs are common to any two dictionaries:
{k:count for k,count in
collections.Counter(itertools.chain(*[d.iteritems() for d in dicts])).iteritems()
if count > 1}
>>> dicts
[{'a': 3, 'b': 89, 'd': 2}, {'a': 3, 'c': 99, 'b': 89}, {'a': 3, 'c': 33, 'b': 42}]
>>> sets = (set(d.iteritems()) for d in dicts)
>>> dict_intersection = dict(set.intersection(*sets))
>>> dict_intersection
{'a': 3}
A slightly more hands-dirty approach: Take the list of keys for each dictionary, sort each list, and then proceed as if you were merging them (keep an index for each list, advance the one w/ the lowest value). Whenever all of the indices point to the same key, check the values for equality; either way, advance all indices.