I am faced with a situation where I need to iterate nested dicts with same outermost keys, compare inner key value to inner key value of other dict then append. Below will help to understand:
{AD:{(62,'VMX','Barcelona','ES'): ['123','567','666'}} #dict1 many more rows
{AD:{(151,'CMXS','','ES','62'): ['345','6754']}} #dict 2 many more rows
So I need to iterate both dicts, compare tuple key[0] of dict1 == tuple key[4] of dict2, if they match append inner list in dict1 with inner list in dict2.
after this dict1 looks like
{AD:{(62,'VMX','Barcelona','ES'): ['123','567','666','345','6754'}}
Wrote this code but it not working as expected
for k,v in dict1.items():
if len(v)!= 0:
for c,a in dict2.items():
if len(a)!= 0:
for k2,v2 in v.items():
for c2,a2 in a.items():
if str(k2[0]) == c2[4]:
v2.append(a2)
dict1 = {"AD":{(62,'VMX','Barcelona','ES'): ['123','567','666']}} #dict1 many more rows
dict2 = {"AD":{(151,'CMXS','','ES','62'): ['345','6754']}} #dict 2 many more rows
# create a simplified map of dict2
dict3 = {}
for outer_key, outer_value_dict in dict2.items():
for inner_key_tuple, inner_value_list in outer_value_dict.items():
new_key = (outer_key, inner_key_tuple[4]) # e.g. (AD, 62)
dict3.setdefault(new_key, []).extend(inner_value_list)
# now modify dict1 using dict3 instead of dict2
for outer_key, outer_value in dict1.items():
for inner_key_tuple, inner_value_list in outer_value.items():
lookup_key = (outer_key, str(inner_key_tuple[0]))
# search dict3 using the lookup_key.
# if not present, get will return an empty list
list_to_extend = dict3.get(lookup_key, [])
inner_value_list.extend(list_to_extend)
print(dict1)
Output
/tmp $ python test.py.py
{'AD': {(62, 'VMX', 'Barcelona', 'ES'): ['123', '567', '666', '345', '6754']}}
Based on the comments, since you wanted to know how to avoid one of the loops - here's an updated version
for k,v in dict1.items():
a = dict2.get(k)
if a is None or len(v) == 0:
continue
for k2,v2 in v.items():
for c2,a2 in a.items():
if str(k2[0]) == c2[4]:
v2.append(a2)
The for loops are placed correctly as long as the example goes.
What do you mean by unexpectedly?
Do you mean this?
{'AD': {(62, 'VMX', 'Barcelona', 'ES'): ['123', '567', '666', ['345', '6754']]}}
That extra array within the array?
Because if so, the answer would be to switch the .append() with a .extend()
Related
Okay, not sure how to explain this problem but here goes.
This is my dictionary myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
I'm trying to list out the '1003', '1040'.
I've tried using this method:
for i in range(len(a['12/2019'])):
print(a['12/2019'][i].keys())
It kind of works however it returns me
dict_keys(['1003'])
dict_keys(['1040'])
Is there anyway I can just get 1030 and 1040 to be isolated from the ... dict_keys([''])?
You can iterate over the values of the dictionary, and if the value is a list, iterate over the dictionaries in the list, and append all the keys to the result
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}],
'11/2019': '1005', '10/2019': 1234,
'09/2019': [{'1006' : 2}, {'1042' : 3}],
'08/2019': (1,2)}
keys=[]
#Iterate over values
for value in myDictionary.values():
#If value is a list
if isinstance(value, list):
#Iterate over the list
for d in value:
#Add the keys to the result
keys.extend(list(d.keys()))
print(keys)
The output will be
['1003', '1040', '1006', '1042']
You can also do this via a list-comprehension, but it's not really readable, so I would avoid against it
[key for value in myDictionary.values() if isinstance(value, list) for d in value for key in d.keys()]
Iterate over the sub-dicts in the list, grabbing the keys:
for sub_dict in a['12/2019']:
for key, val in sub_dict.items():
print(key)
You can use a list-comprehension:
[k for v in myDictionary.values() for x in v for k in x]
In code:
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
print([k for v in myDictionary.values() for x in v for k in x])
# ['1003', '1040']
You could do this by doing something like this
data = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
# Data is a dictionary of where each value corresponding
# to the key is a list of dictionaries.
for k, v in data.items():
# v is a list of dictionaries
all_keys = []
for value in v:
all_keys.extend(list(value.keys()))
print (all_keys)
The result is ['1003', '1040']
This may be more than you need, but if the data you shared for the example is just a small subset of what you're really looking at, here's a recursive function that can go as deep as you want it to.
Since you seem to want to skip the keys in the outer most level, I put in a spot to allow you to skip over the keys at a given depth. You shouldn't set the level argument manually since the code will just update that for itself.
def list_keys(element, ignore_levels=[], level=1):
"""
Recursivly list keys found in an element.
element: item to inspect.
ignore_levels: list of numeric levels to ignore.
level: recursion depth level.
"""
keys = []
# add keys if we're not ignoring this level and set iterable to values
if isinstance(element, dict):
if level not in ignore_levels:
keys.extend(element.keys())
iterable = element.values()
# if we hve a list or tuple, we can iterate over it
elif isinstance(element, (list, tuple)):
iterable = element
# if we have a single value, it's not a key and there's nothing to iterate
else:
iterable = []
# Iterate over elements and append any keys found
for i in iterable:
subkeys = list_keys(i, ignore_levels, level + 1)
if subkeys:
keys.extend(subkeys)
return keys
d = {'12/2019': [{'1003': 2}, {'1040': 3}]}
print(list_keys(d, ignore_levels=[1]))
# ['1003', '1040']
I have 2 for loops that are working but they're not too pythonic looking.
EDIT: I realized that oversimplified my example. I still need the values corresponding to the specific key.
The first one loops through a list of dictionaries and then through the keys and values of each dictionary and prints the value if the key equals to 'MATCH'.
The second one loops through the same list of dictionaries and a different dictionary. If a value in the first list of dictionaries is equal to the key of the second dictionary then it loops through the list of dictionaries again and prints the value if the key equals to 'MATCH'.
for item in data_dict:
for k, v in item.items():
if k == 'MATCH':
print(v)
for item in data_dict:
for k, v in item.items():
for kx, vx in dictA.items():
if v == kx:
for k2, v2 in item.items():
if k2 == 'MATCH':
print(v2)
for item in data_dict:
for k, v in item.items():
if k == 'MATCH':
print('found')
you're not using the dict properly, just using it as a list of tuples. Just write:
for item in data_dict:
if 'MATCH' in item:
print('found: ',item['MATCH'])
# break here? not if you want multiple matches
same goes for the other loop. But I just reduced complexity from O(n**2) to O(n). You're still performing a linear search in a list of dictionaries, still not satisfactory.
The other way would be: create one dictionary with lists of your elements as the elements, instead of a list of dictionaries with elements.
That way you get all the info with 1 key lookup. Practical example:
Let's say your data looks like this:
list_of_dicts = [
{'MATCH':12, 'foo':14},
{'MATCH':5, 'bar':2},
{'what':0}
]
transform it as a dict of lists with collections.defaultdict(list):
import collections
dict_of_lists = collections.defaultdict(list)
for d in list_of_dicts:
for k,v in d.items():
dict_of_lists[k].append(v)
Let's look at the new dict:
>>> dict_of_lists
{'MATCH': [12, 5],
'bar': [2],
'foo': [14],
'what': [0]}
once this is done, just get all the values matching a key with this:
if 'MATCH' in dict_of_lists:
print(dict_of_lists['MATCH'])
which prints:
[12, 5]
The transformation done at the start must be done only once (or not at all if the data is properly organized from the start) to get super fast & easy data lookup.
I have the following:
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
I am trying to create a new dictionary which will include the dictionary value (from the original dictionary) that matches the list entry to the dictionary key (in the original dictionary)
for k in my_dict.keys():
if k in my_list:
new_dict.append(k)
print(new_dict)
should return
7777777: '500'
But I'm returning an empty set. I'm not sure what I'm doing wrong here
A dictionary comprehension would provide you what you need.
You need to make sure the types agree (int vs. str)
Unless the list is significantly longer than the dict, it will be much more efficient to iterate over the list and check that key is in the dict than the other way around.
E.g.:
In []:
new_dict = {k: my_dict[k] for k in map(int, my_list) if k in my_dict}
print(new_dict)
Out[]:
{7777777: '500'}
Try like this :
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
new_dict = {k:my_dict[k] for k in my_dict.keys() if str(k) in my_list}
print(new_dict)
# {7777777: '500'}
Update:
You can also do this with project function from funcy library.
from funcy import project
new_dict = project(my_dict, map(int, my_list))
print(new_dict)
You have to int() the iterator.
my_list = ["7777777", "888888", "99999"]
my_dict = {21058199: '500', 7777777: '500', 21058199: '500'}
for l_i in my_list:
if l_i in my_dict:
print(my_dict[int(l_i)])
Your dictionary keys are of the type 'Int', while your list items are strings. You need to convert one into the other.
for example:
new_dict = {}
for k in my_dict.keys():
if str(k) in my_list:
new_dict[k] = my_dict[k]
Note that you cannot use .append() to add key-value pairs to a dictionary.
I have two dictionaries. In both dictionaries, the value of each key is a single list. If any element in any list in dictionary 2 is equal to a key of dictionary 1, I want to replace that element with the first element in that dictionary 1 list.
In other words, I have:
dict1 = {'IDa':['newA', 'x'], 'IDb':['newB', 'x']}
dict2 = {1:['IDa', 'IDb']}
and I want:
dict2 = {1:['newA', 'newB']}
I tried:
for ID1, news in dict1.items():
for x, ID2s in dict2.items():
for ID in ID2s:
if ID == ID1:
print ID1, 'match'
ID.replace(ID, news[0])
for k, v in dict2.items():
print k, v
and I got:
IDb match
IDa match
1 ['IDa', IDb']
So it looks like everything up to the replace method is working. Is there a way to make this work? To replace an entire string in a value-list with a string in another value-list?
Thanks a lot for your help.
Try this:
dict1 = {'IDa':['newA', 'x'], 'IDb':['newB', 'x']}
dict2 = {1:['IDa', 'IDb']}
for key in dict2.keys():
dict2[key] = [dict1[x][0] if x in dict1.keys() else x for x in dict2[key]]
print dict2
this will print:
{1: ['newA', 'newB']}
as required.
Explanation
dict.keys() gives us just the keys of a dictionary (i.e. just the left hand side of the colon). When we use for key in dict2.keys(), at present our only key is 1. If the dictionary was larger, it'd loop through all keys.
The following line uses a list comprehension - we know that dict2[key] gives us a list (the right side of the colon), so we loop through every element of the list (for x in dict2[key]) and return the first entry of the corresponding list in dict1 only if we can find the element in the keys of dict1 (dict1[x][0] if x in dict1.keys) and otherwise leave the element untouched ([else x]).
For example, if we changed our dictionaries to be the following:
dict1 = {'IDa':['newA', 'x'], 'IDb':['newB', 'x']}
dict2 = {1:['IDa', 'IDb'], 2:{'IDb', 'IDc'}}
we'd get the output:
{1: ['newA', 'newB'], 2: ['newB', 'IDc']}
because 'IDc' doesn't exist in the keys of dict1.
You could also use dictionary comprehensions, but I am not sure that they are working in Python 2.7, it may be limited to Python 3 :
# Python 3
dict2 = {k: [dict1.get(e, [e])[0] for e in v] for k,v in dict2.items()}
edit: I just checked, this is working in Python 2.7. However, dict2.items() should be replaced by dict2.iteritems() :
# Python 2.7
dict2 = {k: [dict1.get(e, [e])[0] for e in v] for k,v in dict2.iteritems()}
This was a fun one!
dict2[1] = [dict1[val][0] if val in dict1 else val for val in dict2[1]]
Or, here is the same logic without list comprehension:
new_dict = {1: []}
for val in dict2[1]:
if val in dict1:
new_dict[1].append(dict1[val][0])
else:
new_dict[1].append(val)
dict2 = new_dict
I have 2 dictionaries.
dict1={('SAN RAMON', 'CA'): 1, ('UPLAND', 'CA'): 4, ('POUGHKEESIE', 'NY'): 3, ('CATTANOOGA', 'TN'): 1}
dict2={('UPLAND', 'CA'): 5223, ('PORT WASHING', 'WI'): 11174, ('PORT CLINTON', 'OH'): 6135, ('GRAIN VALLEY', 'MO'): 10352, ('GRAND JUNCTI', 'CO'): 49688, ('FAIRFIELD', 'IL'): 5165}
These are just samples, in reality each dict has hundreds of entries. I am trying to merge the two dictionaries and create dict 3 that contains {dict1.values(): dict2.values()} but only if that city appears in both dicts. So, one entry in dict3 would look like
{4:5223} # for 'UPLAND', 'CA' since it appears in both dict1 and dict2
This is just a small step in a larger function I am writing. I was going to try something like :
for item in dict1.keys():
if item not in dict2.keys():
del item
return dict[(dict1.keys())=(dict2.keys())]
I can't figure out how to make sure the number of complaints from dict1 matches the same city it is being referred to in dict2.
Here's what I think you want (demo):
dict3 = dict((dict1[key], dict2[key]) for key in dict1 if key in dict2)
Expanded a little, it looks like this:
dict3 = {}
for key in dict1:
if key in dict2:
dict3[dict1[key]] = dict2[key]
The common keys are:
set(dict1.keys()) & set(dict2.keys())
create dict 3 that contains {dict1.values(): dict2.values()}
This doesn't make sense, dictionaries are key-value pairs... what do you really want? Tip:
dict3 = {}
for k in set(dict1.keys()) & set(dict2.keys()):
dict3[dict1[k]]=dict2[k]
{4: 5223}
The straightforward way would be to check each key in one for membership in the other:
result = {}
for key in dict1:
if key in dict2:
result[dict1[key]] = dict2[key]
You could also try converting them into a set or frozenset and taking their intersection, but it's not clear to me whether that will be faster or not:
keys_in_both = frozenset(dict1) & frozenset(dict2)
result = dict((dict1[key], dict2[key]) for key in keys_in_both)