I have 2 for loops that are working but they're not too pythonic looking.
EDIT: I realized that oversimplified my example. I still need the values corresponding to the specific key.
The first one loops through a list of dictionaries and then through the keys and values of each dictionary and prints the value if the key equals to 'MATCH'.
The second one loops through the same list of dictionaries and a different dictionary. If a value in the first list of dictionaries is equal to the key of the second dictionary then it loops through the list of dictionaries again and prints the value if the key equals to 'MATCH'.
for item in data_dict:
for k, v in item.items():
if k == 'MATCH':
print(v)
for item in data_dict:
for k, v in item.items():
for kx, vx in dictA.items():
if v == kx:
for k2, v2 in item.items():
if k2 == 'MATCH':
print(v2)
for item in data_dict:
for k, v in item.items():
if k == 'MATCH':
print('found')
you're not using the dict properly, just using it as a list of tuples. Just write:
for item in data_dict:
if 'MATCH' in item:
print('found: ',item['MATCH'])
# break here? not if you want multiple matches
same goes for the other loop. But I just reduced complexity from O(n**2) to O(n). You're still performing a linear search in a list of dictionaries, still not satisfactory.
The other way would be: create one dictionary with lists of your elements as the elements, instead of a list of dictionaries with elements.
That way you get all the info with 1 key lookup. Practical example:
Let's say your data looks like this:
list_of_dicts = [
{'MATCH':12, 'foo':14},
{'MATCH':5, 'bar':2},
{'what':0}
]
transform it as a dict of lists with collections.defaultdict(list):
import collections
dict_of_lists = collections.defaultdict(list)
for d in list_of_dicts:
for k,v in d.items():
dict_of_lists[k].append(v)
Let's look at the new dict:
>>> dict_of_lists
{'MATCH': [12, 5],
'bar': [2],
'foo': [14],
'what': [0]}
once this is done, just get all the values matching a key with this:
if 'MATCH' in dict_of_lists:
print(dict_of_lists['MATCH'])
which prints:
[12, 5]
The transformation done at the start must be done only once (or not at all if the data is properly organized from the start) to get super fast & easy data lookup.
Related
I am faced with a situation where I need to iterate nested dicts with same outermost keys, compare inner key value to inner key value of other dict then append. Below will help to understand:
{AD:{(62,'VMX','Barcelona','ES'): ['123','567','666'}} #dict1 many more rows
{AD:{(151,'CMXS','','ES','62'): ['345','6754']}} #dict 2 many more rows
So I need to iterate both dicts, compare tuple key[0] of dict1 == tuple key[4] of dict2, if they match append inner list in dict1 with inner list in dict2.
after this dict1 looks like
{AD:{(62,'VMX','Barcelona','ES'): ['123','567','666','345','6754'}}
Wrote this code but it not working as expected
for k,v in dict1.items():
if len(v)!= 0:
for c,a in dict2.items():
if len(a)!= 0:
for k2,v2 in v.items():
for c2,a2 in a.items():
if str(k2[0]) == c2[4]:
v2.append(a2)
dict1 = {"AD":{(62,'VMX','Barcelona','ES'): ['123','567','666']}} #dict1 many more rows
dict2 = {"AD":{(151,'CMXS','','ES','62'): ['345','6754']}} #dict 2 many more rows
# create a simplified map of dict2
dict3 = {}
for outer_key, outer_value_dict in dict2.items():
for inner_key_tuple, inner_value_list in outer_value_dict.items():
new_key = (outer_key, inner_key_tuple[4]) # e.g. (AD, 62)
dict3.setdefault(new_key, []).extend(inner_value_list)
# now modify dict1 using dict3 instead of dict2
for outer_key, outer_value in dict1.items():
for inner_key_tuple, inner_value_list in outer_value.items():
lookup_key = (outer_key, str(inner_key_tuple[0]))
# search dict3 using the lookup_key.
# if not present, get will return an empty list
list_to_extend = dict3.get(lookup_key, [])
inner_value_list.extend(list_to_extend)
print(dict1)
Output
/tmp $ python test.py.py
{'AD': {(62, 'VMX', 'Barcelona', 'ES'): ['123', '567', '666', '345', '6754']}}
Based on the comments, since you wanted to know how to avoid one of the loops - here's an updated version
for k,v in dict1.items():
a = dict2.get(k)
if a is None or len(v) == 0:
continue
for k2,v2 in v.items():
for c2,a2 in a.items():
if str(k2[0]) == c2[4]:
v2.append(a2)
The for loops are placed correctly as long as the example goes.
What do you mean by unexpectedly?
Do you mean this?
{'AD': {(62, 'VMX', 'Barcelona', 'ES'): ['123', '567', '666', ['345', '6754']]}}
That extra array within the array?
Because if so, the answer would be to switch the .append() with a .extend()
I have a list of pairs.The list contains items of [x,y].I would like to make list or dictionary making the left item the key and right the value.The list maybe contains multiple times the same key. I want to sum the values and keep one time the key.
E.x
pairs[0]=['3106124650', 2.86]
pairs[1]=['3106124650', 8.86]
pairs[2]=['5216154610', 23.77]
I want to keep '3106124650' one time and sum the values.So my new list or dictionary will contain one time this key with value 11.72.
'3106124650',11.72
Here's a way. For large datasets, numpy will probably be faster though.
import collections
result = collections.defaultdict(lambda : 0)
for k,v in pairs:
result[k]+=v
sumdict = dict()
for i, v in pairs:
sumdict[i] = v + sumdict.get(i, 0)
li=[['a',1],['a',2],['b',3],['c',4]]
d={}
for w in li:
d[w[0]]=w[1]+d.get(w[0],0)
Output:{'a': 3, 'b': 3, 'c': 4}
you can try this:
d={}
for entry in pairs:
if entry[0] in d:
d[entry[0]]+=entry[1]
else:
d[entry[0]]=entry[1]
Okay, not sure how to explain this problem but here goes.
This is my dictionary myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
I'm trying to list out the '1003', '1040'.
I've tried using this method:
for i in range(len(a['12/2019'])):
print(a['12/2019'][i].keys())
It kind of works however it returns me
dict_keys(['1003'])
dict_keys(['1040'])
Is there anyway I can just get 1030 and 1040 to be isolated from the ... dict_keys([''])?
You can iterate over the values of the dictionary, and if the value is a list, iterate over the dictionaries in the list, and append all the keys to the result
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}],
'11/2019': '1005', '10/2019': 1234,
'09/2019': [{'1006' : 2}, {'1042' : 3}],
'08/2019': (1,2)}
keys=[]
#Iterate over values
for value in myDictionary.values():
#If value is a list
if isinstance(value, list):
#Iterate over the list
for d in value:
#Add the keys to the result
keys.extend(list(d.keys()))
print(keys)
The output will be
['1003', '1040', '1006', '1042']
You can also do this via a list-comprehension, but it's not really readable, so I would avoid against it
[key for value in myDictionary.values() if isinstance(value, list) for d in value for key in d.keys()]
Iterate over the sub-dicts in the list, grabbing the keys:
for sub_dict in a['12/2019']:
for key, val in sub_dict.items():
print(key)
You can use a list-comprehension:
[k for v in myDictionary.values() for x in v for k in x]
In code:
myDictionary = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
print([k for v in myDictionary.values() for x in v for k in x])
# ['1003', '1040']
You could do this by doing something like this
data = {'12/2019' : [{'1003' : 2}, {'1040' : 3}]}
# Data is a dictionary of where each value corresponding
# to the key is a list of dictionaries.
for k, v in data.items():
# v is a list of dictionaries
all_keys = []
for value in v:
all_keys.extend(list(value.keys()))
print (all_keys)
The result is ['1003', '1040']
This may be more than you need, but if the data you shared for the example is just a small subset of what you're really looking at, here's a recursive function that can go as deep as you want it to.
Since you seem to want to skip the keys in the outer most level, I put in a spot to allow you to skip over the keys at a given depth. You shouldn't set the level argument manually since the code will just update that for itself.
def list_keys(element, ignore_levels=[], level=1):
"""
Recursivly list keys found in an element.
element: item to inspect.
ignore_levels: list of numeric levels to ignore.
level: recursion depth level.
"""
keys = []
# add keys if we're not ignoring this level and set iterable to values
if isinstance(element, dict):
if level not in ignore_levels:
keys.extend(element.keys())
iterable = element.values()
# if we hve a list or tuple, we can iterate over it
elif isinstance(element, (list, tuple)):
iterable = element
# if we have a single value, it's not a key and there's nothing to iterate
else:
iterable = []
# Iterate over elements and append any keys found
for i in iterable:
subkeys = list_keys(i, ignore_levels, level + 1)
if subkeys:
keys.extend(subkeys)
return keys
d = {'12/2019': [{'1003': 2}, {'1040': 3}]}
print(list_keys(d, ignore_levels=[1]))
# ['1003', '1040']
Essentially I want to delete every key in a dictionary if its value doesn't equal the highest value.
Let's say this is the dictionary:
myDict = {"Bob": 1, "Bill": 5, "Barry": 4, "Steve": 5}
I'm able to sort it by value using this:
myDict = sorted(myDict, key=myDict.get, reverse=True)
Now I want to remove any key in the dictionary that doesn't equal the highest value (in this case '5'). To end up with this:
myDict = {"Bill": 5, "Steve": 5}
I've tried using this for loop:
for item, v in myDict:
if v < myDict[0]:
del myDict[v]
But I get this error:
ValueError: too many values to unpack (expected 2)
This is a) my first time posting here, and b) I've only been learning Python for a few months so I'm sorry if I've made any stupid mistakes.
for item, v in myDict just give you keys mydict, and you are collecting that key in item, v that's why,
use myDict.items() or myDict.iteritems().
for item, v in myDict.iteritems():
if v < myDict[0]:
del myDict[v]
To get Highest value of myDict
max(myDict.values())
To delete keys from Dict never change the iterator you are iterating on, it will give you RuntimeError. So copy it in another variable and change previous one as Anand S Kumar suggested.
You should never alter the object you're iterating over, that usually yields unexpected results (internal pointers get shifted and you miss elements in your iteration and such). You best gather the keys you want to delete and then remove the keys in a separate iteration:
keys = [k for k in myDict.keys() if myDict[k] == max(myDict.values())];
for k in keys: del myDict[k];
It might be best to put the max expression in a variable too so it doesn't get evaluated multiple times. Not sure if Python's able to optimize that for you (probably not).
You can use dictionary comprehension to create a new dictionary:
newDict = {k: v for k,v in myDict.items() if v == max(myDict.values())}
The output for newDict:
{'Steve': 5, 'Bill': 5}
I've got a list k with the 0'th element:
k[0]: {'pattern': 0, 'pos': array([ 9.83698, 106.539 , 130.314 ]), 'id': 1922}
(It looks like a dict, but its a list indeed)
when I iterate through the 0'th element of the list k and print out each element I Get:
for i in k:
print i
=>output:
pattern
pos
id
I'd like to access not only the keys but the values as well. How to do this?
I've also tried to convert the list back into a dict using zip and izip, but same resutlts...i.e. only keys are printed, no values...
any help will be appreciated
thx in advance
you can use k.values() to iterate through the values, or k.items() to iterate through (key, value) pairs
for value in k.values():
print value
for key, value in k.items():
print key, value
The fastest way to iterate over the dictionary you created (it is in fact a dictionary) is not to create the lists of keys/values using k[0].keys(), k[0].values and k[0].items() but using k[0].iteritems() which creates a dictionary iterator that returns just the pairs without allocating lists in the memory.
It also runs much faster for big dictionaries (a being the dictionary):
>>> non_iter_timer = timeit.Timer("for k,v in a.items(): k + v", setup="a = {x:x for x in xrange(10000000)}")
>>> non_iter_timer.repeat(3, 10)
[25.612606023166585, 25.100741935717622, 24.840450306339463]
>>> iter_timer = timeit.Timer("for k,v in a.iteritems(): k + v", setup="a = {x:x for x in xrange(10000000)}")
>>> iter_timer.repeat(3, 10)
[9.26259596885518, 9.198298194571748, 9.77466250122282]