Iterate multiple dict with different keys - python

The goal
I am trying to iterate two dicts at the same time knowing that they have some keys in common (for sure), but some of them are not (possibly). What is more the same keys could (rarely, but still) be ordered differently. Another issue is that dicts can have different lenght. In my case the keys are all numerical.
Atempted solutions
Example dicts:
di1 = {1: "a", 2: "b", 3: "c", 5:"e"}
di2 = {1: "a", 2: "b", 4: "d", 5:"e", 6:"f"}
After reading some answers to iterating multiple dicts I tried zip()ing the two dicts:
for i, j in zip( di1, di2 ): print( i, j )
1 1
2 2
3 4
5 5
but this 'cuts' the longer dict, also this iterates over keys of each dict seperately instead of keeping them consistent (always i == j, even if i in di1 and j in di2 would return False)
Given that in my case all keys are numerical I tried the following:
for i in range(max(max(di1), max(di2))+1): print(i)
0
1
2
3
4
5
6
which works (I can pass i as dict key), but:
Doesn't iterate dicts per se, just generates numbers to try to match to given dicts.
Iterates over values even if they are non existent keys in both dicts (i in di1 or i in di2 is False).
This works only if keys are numerical.
Doesn't seem very pythonic.
The quesstion
How do I iterate two (or more) dicts (keys) given that it is enough for the key to exist in at least one of them?
Conditions
Solutions using standard libraries are preferable.
You can assume dict keys are numerical but a more general solution is preferable.
Iteration order is of no importance but additional information on the matter is a bonus.
I'm iterating two dicts.
Both dicts should remain unaltered.
I'm using python 3.6.1

You can iterate over common keys:
for key in di1.keys() & di2.keys():
print(key)
Or union of keys:
for key in di1.keys() | di2.keys():
print(key)
You choose. Use dict.viewkeys() in Python 2.

I would extract keys from both dicts (.keys()), join the lists of the keys, remove duplicates (make it set), then iterate over the dicts using this new set of keys.
keys1 = di1.keys()
keys2 = di2.keys()
keys = keys1 + keys2
keys = set(keys)
for key in keys:
try:
di1[key]
di2[key]
except KeyNotFoundError:
# key is not present in both dicts
pass

You can use itertools.izip_longest, to iterate till the longer collection, where zip iterates till the smaller collection.
>>> di1 = {1: "a", 2: "b", 3: "c", 5:"e"}
>>> di2 = {1: "a", 2: "b", 4: "d", 5:"e", 6:"f"}
>>>
>>> from itertools import izip_longest
>>> for a, b in izip_longest(di1, di2):
... print(di1.get(a), di2.get(b))
...
('a', 'a')
('b', 'b')
('c', 'd')
('e', 'e')
(None, 'f')
The thing to look at here is the use of dict.get(key), because using dict[key] will cause KeyError for unique keys. You can however, add an optional default value as second parameter inside dict.get(key, default_value).
Hope this helps.

Assuming that the values are consistent between the various dictionaries you can use collections.ChainMap to iterate over multiple dictionaries:
from collections import ChainMap
di1 = {1: "a", 2: "b", 3: "c", 5:"e"}
di2 = {1: "a", 2: "b", 4: "d", 5:"e", 6:"f"}
chained_dicts = ChainMap(di1, di2) # add more dicts as required
for key in chained_dicts:
print(key, chained_dicts[key])
Output:
1 a
2 b
3 c
4 d
5 e
6 f
Or more simply:
for key, value in ChainMap(di1, di2).items():
print(key, value)
As mentioned above, this is fine if the values for duplicated keys are the same. Where there is variation the value from the first chained dictionary will be returned.

Related

finding values in dictionary based on their key

I'm trying to find the values of keys based on their 3 first letters. I have three different categories of subjects that i have to get the grade from, stored as value with the subject being key. I have ECO, GEO, and INF. As there are multiple subjects i want to get the values from every key containing either ECO, GEO or INF.
subject={"INFO100":"A"}
(subject.get("INF"))
In this method i don't get the value, i have to use the whole Key. Is there a work-a-round? I want the values seperately so i can calculate their GPA based on their field of study:)
You need to iterate on the pairs, to filter on the key and keep the value
subject = {"INFO100": "A", "INF0200": "B", "ECO1": "C"}
grades_inf = [v for k, v in subject.items() if k.startswith("INF")]
print(grades_inf) # ['A', 'B']
grades_eco = [v for k, v in subject.items() if k.startswith("ECO")]
print(grades_eco) # ['C']
A said in the comments, the purpose of a dictionary is to have unique keys. Indexing is extremely fast as it uses hash tables. By searching for parts of the keys you need to loop and lose the benefit of hashing.
Why don't you store your data in a nested dictionary?
subject={'INF': {"INFO100":"A", "INFO200":"B"},
'OTH': {"OTHER100":"C", "OTHER200":"D"},
}
Then access:
# all subitems
subject['INF']
# given item
subject['INF']['INFO100']
For understanding porpoises, you can create a function that returns a dictionary, like:
def getGradesBySubject(dict, search_subject):
return [grade for subject,grade in dict.iteritems() if subject.startwith(search_subject)]
I'd suggest using a master dict object that contains a mapping of the three-letter subjects like ECO, GEO, to all subject values. For example:
subject = {"INFO100": "A",
"INFO200": "B",
"GEO100": "D",
"ECO101": "B",
"GEO003": "C",
"INFO101": "C"}
master_dict = {}
for k, v in subject.items():
master_dict.setdefault(k[:3], []).append(v)
print(master_dict)
# now you can access it like: master_dict['INF']
Output:
{'INF': ['A', 'B', 'C'], 'GEO': ['D', 'C'], 'ECO': ['B']}
If you want to eliminate duplicate grades for a subject, or just as an alternate approach, I'd also suggest a defaultdict:
from collections import defaultdict
subject = {"INFO100": "A",
"INFO300": "A",
"INFO200": "B",
"GEO100": "D",
"ECO101": "B",
"GEO003": "C",
"GEO102": "D",
"INFO101": "C"}
master_dict = defaultdict(set)
for k, v in subject.items():
master_dict[k[:3]].add(v)
print(master_dict)
defaultdict(<class 'set'>, {'INF': {'B', 'A', 'C'}, 'GEO': {'D', 'C'}, 'ECO': {'B'}})

Iterate adding values to an existing dictionary by using a variety of keys stored as a list

I had trouble coming up with an appropriate title for this, so apologies there.
I have an existing dictionary di_end which already has an order to its keys. I also have some objects which have a property containing the keys for where in di_end the user-entered value will go
Note: the methods setProperty() and property() are from the pyqt library where setProperty() creates a custom property for an object where the first argument is the name of the property and the second argument is the value for that named property and property() just returns the values for whatever name is passed into it as an argument.
Something like this:
a.setProperty('keys', [key1, key2, key3])
b.setProperty('keys', [key4, key5, key6, key7, key8])
c.setProperty('keys', [key9])
objects_list = [a, b, c]
I want to be able to use the keys stored in the object properties to load the value that the user enters into a field to a dictionary
I'd like to iterate the process such that these parts
di_end[a.property(['keys'])[0]][a.property(['keys'])[1]][a.property(['keys'])[2]] = a.value
di_end[b.property(['keys'])[0]][b.property(['keys'])[1]][b.property(['keys'])[2]]\
[b.property(['keys'])[3]][b.property(['keys'])[4]] = b.value
di_end[c.property(['keys'])[0]] = c.value
or
a_li, b_li, c_li = a.property(['keys']), b.property(['keys']), c.property(['keys'])
di_end[a_li[0]][a_li[1]][a_li[2]] = a.value
di_end[b_li[0]][b_li[1]][b_li[2]][b_li[3]][b_li[4]] = b.value
di_end[c_li[0]] = c.value
do not need to be manually typed out and could be performed procedurally. I think I could do this if it was the same amount of keys but I'm not sure how to do it with differing amounts of keys. If they were all the same amounts I'd just do this
a.setProperty('keys', [key1, key2, key3, key4])
b.setProperty('keys', [key5, key6, key7, key8])
c.setProperty('keys', [key9, key10, key11, key12])
objects_list = [a, b, c]
a_li, b_li, c_li = a.property(['keys']), b.property(['keys']), c.property(['keys']) # assuming all are 4 entires each
for count, item in enumerate([a_li, b_li, c_li]):
di_end[item[0]][item[1]][item[2]][item[3]] = objects_list[count].value
but since there are different amounts of keys for each entry, I'm not sure how to accomplish this.
Edit: Added a note about setProperty() and property()
It's pretty hard to tell what your code is doing here, mostly due to the setProperty and property objects, which seem awkward and not-pythonic.
That being said, this looks like yet another use case for the excellent glom library (pip install glom).
from glom import assign, Path
assign(data, Path(*keys), value)
Note that in your case
data = di_end
keys = a.property['keys']
value = a.value
and an example:
>>> data = {'a': {'b': {'c': 5}}}
>>> keys = ['a', 'b', 'c']
>>> value = 100
>>> assign(data, Path(*keys), value)
{'a': {'b': {'c': 100}}}
>>> data
{'a': {'b': {'c': 100}}}
Note that this will raise a KeyError if the path doesn't already exist. For the generate-dict-on-demand (like a infinite nested defaultdict) style, you need to use Assign instead:
glom(data, Assign(Path(*keys), value))
See https://glom.readthedocs.io/en/latest/mutation.html for more details.
It seems you can try use recursion here. A simple example below.
put function creates nested dict and put value. fetch function can extract value from nested dict.
def fetch(data, keys):
if len(keys) == 1:
return data[keys[0]]
return fetch(data[keys.pop()], keys)
def put(data, keys, value):
if len(keys) == 1:
data[keys[0]] = value
return
key = keys.pop()
data[key] = {}
put(data[key], keys, value)
di_empty = {}
keys_values = [
(("a", "b", "c"), "abc"),
(("b", "c", "d", "e"), "bcde")
]
for selected in keys_values:
put(di_empty, list(reversed(selected[0])), selected[1])
di_end = dict()
di_end["a"] = {"b": {"c": "abc"}}
di_end["b"] = {"c":{"d": {"e": "bcde"}}}
keys = [("a", "b", "c"), ("b", "c", "d", "e")]
for selected in keys:
s1 = fetch(di_end, list(reversed(selected)))
s2 = fetch(di_empty, list(reversed(selected)))
assert s1 == s2
print(s1)
print(s2)

How can i take a given dict and return a new one with multiple values for one key? [duplicate]

This question already has answers here:
Reverse / invert a dictionary mapping
(32 answers)
Closed 2 years ago.
I'm new to python and struggling with this question - Create a function which works as follows:
d={1:"x",2:"y",3:"x",4:"z"}
invert_d(d)
{"x":(1,3),"y":(2),"z":(4)}
So invert_d gives a new dict with the values of multiple keys as lists.
I have to say, that I never worked with dicts and started python 1 week ago...so I'm a total newb.
I am new to python.... and I'm struggling with this question:
Edit: I read wrong and fixed the Dict. Sorry guys :(
Also we cant import in the exam
You can use the setdefault method to make the dictionary construction simpler. A very basic way to do it is like this:
d={1:"x",2:"y",3:"x",4:"z"}
def invertDict(d):
result = dict()
for k,v in d.items(): result.setdefault(v,[]).append(k)
return result
print(invertDict(d))
{'x': [1, 3], 'y': [2], 'z': [4]}
If you want a one-liner solution, you can use a dictionary comprehension. Here's one that outputs the list of keys as tuples:
def invertDict(d):
return {v:tuple(inv[v]) for inv in [{}] for k,v in d.items() if [inv.setdefault(v,[]).append(k)] }
print(invertDict(d))
{'x': (1, 3), 'y': (2,), 'z': (4,)}
A dict associates exactly one value with a particular key, so your original dict d would see two different values for the key "x" (first 1 and then 3), and the last one would replace previous ones. If you have access to an interactive Python session, this looks like:
$ python
Python 3.9.1 (default, Dec 13 2020, 11:55:53)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> d = {"x": 1, "y": 2, "x": 3, "z": 5}
>>> d
{'x': 3, 'y': 2, 'z': 5}
(The d by itself there just asks Python to use print to display the current value, as if you had called print(d).)
So at this point you have some options. You could just define a dict to contain the eventual values you want: {"x":(1,3),"y":(2,),"z":(5,)}. Or, if you actually have a source of a stream or list of pairs of values that you want to process into a dict using a function like the one you describe, we can do that, too, but we need to use something like a list of tuples as the input, rather than a dict: l = [("x", 1), ("y", 2), ("x", 3), ("z", 5)]. With that new notion of the input, we could write a function that in some way processes each element of that list and compiles those elements into a particular dict. Here's one way that could look (without any error handling):
from functools import reduce
def add_tuple_to_dict(d, t): # here `d` is the dict and `t` is the tuple
if t[0] in d:
d[t[0]].append(t[1])
else:
d[t[0]] = [t[1],]
return d
def invert_d(list_of_tuples):
return reduce(add_tuple_to_dict, list_of_tuples, {})
Edit: after submitting the above, I now realize you do actually want to invert your dict, which could look like this, instead:
from functools import reduce
def add_tuple_to_dict(d, t): # here `d` is the dict and `t` is the tuple
if t[1] in d:
d[t[1]].append(t[0])
else:
d[t[1]] = [t[0],]
return d
def invert_d(d):
return reduce(add_tuple_to_dict, d.items(), {})
You can use collections.defaultdict to initially create your dictionary having keys with lists as values. Then convert it to a dict having keys with tuples as values:
import collections
d = {1: "x", 2: "y", 3: "x", 4: "z"}
def invert(d):
# first as a dict with lists as values
inv = collections.defaultdict(list)
for k, v in d.items():
# note that the previous VALUE is the key of the inverted dict
inv[v].append(k)
# then convert the existing "list of values" to "tuples of values"
inv = {k: tuple(v) for k, v in inv.items()}
return inv
invert(d)
# {'x': (1, 3), 'y': (2,), 'z': (4,)}
What defaultdict does is when a value in the dict is looked up (get or set), if it doesn't exist, it automatically creates a value with the default factory provided. In this case, list.
So each value when looked up is set to an empty list. In the loop, the new values (keys of the old one), are appended to this list. If it didn't exist before, the new value is appended to the empty list. If it did exist (from a previous iteration) and has a value, then a new value is appended to that list.
Since what you want at the end is tuples of values and not lists, I have a dictionary comprehension which uses the same keys and converts the values to tuples.
You can do a reverse look-up
invert_d = lambda d: { v:[k for k in d if d[k] == v] for v in d.values() }
whereat you can read v as value and k as key.
d={1:"x",2:"y",3:"x",4:"z"}
invert_d(d)
{'x': [1, 3], 'y': [2], 'z': [4]}
Edit
Maybe a brief explanation: This is a Lambda Expressions using two List Comprehensions. One as value and an outer one to wrap it up (actually a dictionary comprehensions but works the same).
This is not really performant but totally sufficient for every day usage.

Compare two dicts and print non-equal values in python

I need to compare dictionary b against a to check whether the keys of b are in a.
If present check the values of a[key]==b[key]. If not equal print the key:value pair of both dictionaries for reference. How can I do that?
a = {'key_1': 1,'key_2': 2, 'key_3': 3}
b = {'key_1': 1,'key_2': 5}
[k for key in b if key in a if b[k]!=a[k]]
I used the above code, but not able to print both the dictionaries keys and value as like
not equal: b[key_2]=5 and a[key_2]=2
I need to compare dictionary b against a to check whether the keys of b are in a. You want to find the intersecting keys and then check their values:
a = {'key_1': 1,'key_2': 2, 'key_3': 3}
b = {'key_1': 1,'key_2': 5}
# find keys common to both
inter = a.keys() & b
diff_vals = [(k, a[k], b[k]) for k in inter if a[k] != b[k]]
# find keys common to both
inter = a.keys() & b
for k,av, bv in diff_vals:
print("key = {}, val_a = {}, val_b = {}".format(k, av, bv))
key = key_2, val_a = 2, val_b = 5
You can use many different set methods on the dict_view objetcs:
# find key/value pairings that are unique to either dict
symmetric = a.items() ^ b.items()
{('key_2', 2), ('key_2', 5), ('key_3', 3)}
# key/values in b that are not in a
difference = b.items() - a.items()
{('key_2', 5)}
# key/values in a that are not in b
difference = a.items() - b.items()
{('key_3', 3), ('key_2', 2)}
# get unique set of all keys from a and b
union = a.keys() | b
{'key_1', 'key_2', 'key_3'}
# get keys common to both dicts
inter = a.keys() & b
{'key_1', 'key_2'}
What you want is likely this:
result = [(k, a[k], b[k]) for k in a if k in b and a[k]!=b[k]]
In other words, "generate a list of tuples composed of the key, the first value and the second, whenever a key in a is also in b and the corresponding values are not equal".
Since the boolean expressions with "and" are failsafe (they evaluate from left to right and stop as soon as a False value is found), you don't have to worry that "b[k]!=a[k]" could raise an exception.
This raises another question: what if the key is in a and not b or vice-versa, e.g. ('car', 2, None) or ('car', None, 2)? Should that also be a valid answer?
I think there is a small error in the code you posted. First, you seem to mix k and key for the same object. Second, you cannot have two if-clauses in a list comprehension, but instead you can combine them with and. Here is how it could look: [k for k in b if k in a and a[k]!=b[k]]
This will produce a list with all the keys for which the values don't match. Given such a key, you can simply use e.g. "a[{k}]={a} and b[{k}]={b}".format(k=k,a=a[k],b=b[k]) to get a human-readable string describing that mismatch. Of couse, if you are only going to create that list of keys to loop over it afterwards (for printing), then there is no need to actually create that list in the first place. Simply iterate directly over the dictionary keys.
This might work
a = {'key_1': 1,'key_2': 2, 'key_3': 3}
b = {'key_1': 1,'key_2': 5}
i=[k for k in b if k in a if b[k]!=a[k]]
if i:
for k in i:
print('not equal:b[',k,']=',b[k],'and a[',k,']=',a[k])
Output
not equal:b[ key_2 ]= 5 and a[ key_2 ]= 2

Duplicates in a dictionary (Python)

I need to write a function that returns true if the dictionary has duplicates in it. So pretty much if anything appears in the dictionary more than once, it will return true.
Here is what I have but I am very far off and not sure what to do.
d = {"a", "b", "c"}
def has_duplicates(d):
seen = set()
d={}
for x in d:
if x in seen:
return True
seen.add(x)
return False
print has_duplicates(d)
If you are looking to find duplication in values of the dictionary:
def has_duplicates(d):
return len(d) != len(set(d.values()))
print has_duplicates({'a': 1, 'b': 1, 'c': 2})
Outputs:
True
def has_duplicates(d):
return False
Dictionaries do not contain duplicate keys, ever. Your function, btw., is equivalent to this definition, so it's correct (just a tad long).
If you want to find duplicate values, that's
len(set(d.values())) != len(d)
assuming the values are hashable.
In your code, d = {"a", "b", "c"}, d is a set, not a dictionary.
Neither dictionary keys nor sets can contain duplicates. If you're looking for duplicate values, check if the set of the values has the same size as the dictionary itself:
def has_duplicate_values(d):
return len(set(d.values())) != len(d)
Python dictionaries already have unique keys.
Are you possibly interested in unique values?
set(d.values())
If so, you can check the length of that set to see if it is smaller than the number of values. This works because sets eliminate duplicates from the input, so if the result is smaller than the input, it means some duplicates were found and eliminated.
Not only is your general proposition that dictionaries can have duplicate keys false, but also your implementation is gravely flawed: d={} means that you have lost sight of your input d arg and are processing an empty dictionary!
The only thing that a dictionary can have duplicates of, is values. A dictionary is a key, value store where the keys are unique. In Python, you can create a dictionary like so:
d1 = {k1: v1, k2: v2, k3: v1}
d2 = [k1, v1, k2, v2, k3, v1]
d1 was created using the normal dictionary notation. d2 was created from a list with an even number of elements. Note that both versions have a duplicate value.
If you had a function that returned the number of unique values in a dictionary then you could say something like:
len(d1) != func(d1)
Fortunately, Python makes it easy to do this using sets. Simply converting d1 into a set is not sufficient. Lets make our keys and values real so you can run some code.
v1 = 1; v2 = 2
k1 = "a"; k2 = "b"; k3 = "c"
d1 = {k1: v1, k2: v2, k3: v1}
print len(d1)
s = set(d1)
print s
You will notice that s has three members too and looks like set(['c', 'b', 'a']). That's because a simple conversion only uses the keys in the dict. You want to use the values like so:
s = set(d1.values())
print s
As you can see there are only two elements because the value 1 occurs two times. One way of looking at a set is that it is a list with no duplicate elements. That's what print sees when it prints out a set as a bracketed list. Another way to look at it is as a dict with no values. Like many data processing activities you need to start by selecting the data that you are interested in, and then manipulating it. Start by selecting the values from the dict, then create a set, then count and compare.
This is not a dictionary, is a set:
d = {"a", "b", "c"}
I don't know what are you trying to accomplish but you can't have dictionaries with same key. If you have:
>>> d = {'a': 0, 'b':1}
>>> d['a'] = 2
>>> print d
{'a': 2, 'b': 1}

Categories

Resources