I have something like this:
class C:
def get_data(): returns a list
d = {several instances of C}
Now, how do I iterate over all elements of all lists? I failed miserably:
[e for e in [v.get_data() for _, v in d.items()]]
The inner loop will produce a list of lists which can not be digested by the outer loop.
You want a flatten:
[e for _, v in d.items() for e in v.get_data()]
Note the order of the loops.
import itertools
class C:
def get_data(self): return ['a','b']
d = {'a' : C(), 'b' : C()}
print [e for e in itertools.chain.from_iterable(v.get_data() for v in d.values())]
Will print
['a', 'b', 'a', 'b']
Just modify your existing expression -
[item for sublist in [v.get_data() for _, v in d.items()] for item in sublist]
#piokuc's answer is the better option, but in Python 3, yield from is a possibility:
class C:
def get_data(self):
return ['a','b']
def extract(dict_):
"""Yield elements from sublists."""
for _, v in dict_.items():
yield from v.get_data()
d = {'a' : C(), 'b' : C()}
[e for e in extract(d)]
# ['a', 'b', 'a', 'b']
Not sure what you mean by "iterate over all elements", but based on the code in your question, the following will create a list of all the elements in both Python 2 & 3 and doesn't require importing anything else:
class C:
def get_data(self): return ['a', 'b']
d = {'a' : C(), 'b' : C()}
print([e for c in d.values() for e in c.get_data()]) # -> ['a', 'b', 'a', 'b']
In Python 2 you might want to use d.itervalues() instead of d.values() because it creates an iterator of the values rather than creating a temporary list of them (which is what Python 3 does automatically when the values() method called).
Related
d = {"a":[1,2,3], "b":4}
I'm trying to create a nested list from d containing all pairs of key-value:
[["a",1],["a",2],["a",3],["b",4]]
Should I be using list comprehensions here, or is there a more pythonic way?
You can use a list comprehension:
d = {"a":[1,2,3], "b":4}
r = [[a, i] for a, b in d.items() for i in (b if isinstance(b, list) else [b])]
Output:
[['a', 1], ['a', 2], ['a', 3], ['b', 4]]
You can't use a list comprehension because it has to map from one item to another. You are intending to create more items than exist in the dict:
d = {"a":[1,2,3], "b":4}
result = []
for k,v in d.items():
if isinstance(v, list):
for item in v:
result.append([k,item])
else:
result.append([k,v])
def di(my_dictionary):
for key in my_dictionary:
if type(my_dictionary[key]) == list:
for item in my_dictionary[key]:
yield [key,item]
else:
yield [key,my_dictionary[key]]
and just use it by
d = {"a":[1,2,3], "b":4}
list(di(d))
This problem is similar to another where I learned How to Create Nested Dictionary in Python with 3 lists
How can I achieve an output such from 3 lists that takes the following output form
a = ['A', 'B', 'C', 'D']
b = [1,2,3,4]
c = ['N1', 'N2', 'N3', N4]
output = [{'A': {'N1':1}}, {'B':{'N2':2}}, {'C':{'N3':3}}, {'D':{'N4':4}}]
The output is different than on the posted link.
If I could describe this I would say it is a dictionary where 'N#' are the keywords for the values 1-4, and in turn those dictionaries have keyword 'A'-'D', which all seem to be dictionaries insided some top level keyword not shown under which they all are the elements of that master keyword.
I used this line of code which provided an output. These are different values than I'm using, but the point is this doesn't have as many curly brackets or the square brackets at the edges
d = {k: {x: y} for k, x, y in zip(a, b, c)}
# output: {'A':{'1' :'9'} , 'B':{'2':'8'}, 'C':{'3':'7'} , 'D':{'4':'6'}}
So I've explained what I think it means but I'm not sure how to proceed. I tried doing something like
d = {w:{k:{x:y}}for w,k,x,y in zip(sound, a, b, c)}
where 'sound' is a list of the same element value but that just prints out the 4th things on the list. Any help to clear this up would be much appreciated. Thanks.
Use a list comprehension. Iterate over the zipped lists and create the nested dictionaries:
a = ['A', 'B', 'C', 'D']
b = [1, 2, 3, 4]
c = ['N1', 'N2', 'N3', 'N4']
>>> [{k1: {k2: v}} for k1, v, k2 in zip(a, b, c)]
[{'A': {'N1': 1}}, {'B': {'N2': 2}}, {'C': {'N3': 3}}, {'D': {'N4': 4}}]
The dict comprehension is good by #mhawke, but let me add a more naive solution that may need to understand how to implement this type of nested Dictionary:
a = ['A', 'B', 'C', 'D']
b = [1, 2, 3, 4]
c = ['N1', 'N2', 'N3', 'N4']
output = [{'A': {'N1':1}}, {'B':{'N2':2}}, {'C':{'N3':3}}, {'D':{'N4':4}}]
def create_nested(a, b, c):
result = []
for i in range(len(a)):
parent = {}
child = {c[i]: b[i]}
parent.setdefault(a[i], child)
result.append(parent)
return result
result = create_nested(a, b, c)
print(result)
assert result == output
Using enumerate
def create_nested(a, b, c):
result = []
for idx, value in enumerate(a):
child = {c[idx]: b[idx]}
parent = {a[idx]: child}
result.append(parent)
return result
Shorter form
for idx, value in enumerate(a):
parent = {a[idx]: {c[idx]: b[idx]}}
result.append(parent)
return result
Enumerate
I am parsing some XML data from Open Street Map into JSON (later to be uploaded into a database). It is large, so I am using iterparse. I have some tags that look like this
<tag 'k'=stringA:stringB:stringC 'v'=value>
which I want to parse into
{stringA: {stringB: {stringC: value} } }
I have done so with an ugly hardcode. However, I would like to create a function that uses recursion to address the same issue in case the 'k' attribute value contains arbitrarily as many ':'.
I created this
def nestify(l):
"""Takes a list l and returns nested dictionaries where each element from
the list is both a key to the inner dictionary and a value to the outer,
except for the last element of the list, one which is only a value.
For best understanding, feed w = [ 'a', 'b', 'c', 'd', 'e', 'f', 'v'] to the
function and look at the output."""
n = len(l)
if n==2:
key = l[0]
value = l[1]
else:
key = l[0]
value = nestify(l[1:])
return {key:value}
which works. (You have to make the keys and value into a list first.) Except not really, because it always makes a new dictionary. I need it to respect the previous data and integrate. For example, if my parser encounters
<tag 'k'=a:b:c 'v'=1 />
and then in the same element
<tag 'k'=a:d:e 'v'=2 />
I need it to make
{a: {'b': {'c' : 1}, 'd' : {'e' : 2}}}
and not
{a: {'b' : {'c' : 1}}}
{a: {'d' : {'e' : 2}}}
I have tried this code:
def smart_nestify(l, record):
n = len(l)
if n==2:
key = l[0]
value = l[1]
else:
key = l[0]
value = smart_nestify(l[1:], key)
if key not in record:
return {key:value}
else:
record[key] = value
return record
but it still writes over and returns only the latest record. Why is that? How can I fix this code?
Here is a "smarter" nestify that merges a list l into the dictionary record:
def smarter_nestify(l, record):
if len(l) == 2:
return {l[0]: l[1]}
key = l.pop(0)
record[key] = smarter_nestify(l, record.get(key, {}))
return record
Each time through the recursion, it pops the first element from the list l.pop(0) and merges the value from the next level of the dictionary record.get(key, {}).
You can call it with two lists like this:
l = ['a', 'b', 'c', 1]
record = smarter_nestify(l, {})
l = ['a', 'd', 'e', 2]
record = smarter_nestify(l, record)
print record
Sometimes it is better to just use iteration rather than recursion. Here’s the iterative answer.
g_values = dict()
def make_nested(l):
"""
l is the array like you have for nestify
e.g., ['a', 'b', 'c', 1 ] or ['a', 'd', 'e', 2]
"""
global g_values
value = l[-1] ; l = l[:-1]
last_dict = None
for x in l[:-1]:
mp_temp = last_dict if last_dict is not None or g_values
last_dict = mp_temp.get(x)
if last_dict is None:
mp_temp[x] = dict()
last_dict = mp_temp[x]
last_dict[l[-1]] = value
While I didn’t test it, you could also try something like this recursively:
def make_nested(values, l):
if len(l == 2):
values[l[0]] = l[1]
return
mp = values.get(l[0])
if mp is None:
values[l[0]] = dict()
make_nested(values[l[0]], l[1:])
else:
make_nested(mp, l[1:])
I am given a question as follows:
B = C, D
A = B
C = E
meaning B is dependent on variable C and D, while A is dependent on B, and so on. Variable E and D is independent.
Input of 'A' should then return:
E, C, D, B, A
listing all dependent variables. To solve this problem, I first started off by defining a dictionary to iterate this condition easily:
letter = {'A' : ['B'], 'B' : ['C', 'D'], 'C' : ['E']}
However, I am now stuck on how I should loop this in order to print all the children efficiently. I believe this is wrong but I think I may be going in the right direction:
def problem(value):
letter = {'A' : ['B'], 'B' : ['C', 'D'], 'C' : ['E']}
for i in letter:
if i != value:
continue
if len(letter[i]) > 1:
for k in letter:
print("".join(letter[k]), k)
print("".join(letter[i]), i)
Please help!
Stolen from http://code.activestate.com/recipes/576570-dependency-resolver/ after some quick googling.
def dep(arg):
'''
Dependency resolver
"arg" is a dependency dictionary in which
the values are the dependencies of their respective keys.
'''
d=dict((k, set(arg[k])) for k in arg)
r=[]
while d:
# values not in keys (items without dep)
t=set(i for v in d.values() for i in v)-set(d.keys())
# and keys without value (items without dep)
t.update(k for k, v in d.items() if not v)
# can be done right away
r.append(t)
# and cleaned up
d=dict(((k, v-t) for k, v in d.items() if v))
return r
if __name__=='__main__':
d=dict(
a=('b','c'),
b=('c','d'),
e=(),
f=('c','e'),
g=('h','f'),
i=('f',)
)
print dep(d)
You have a list inside of another list. Write a nested for loop inside of another for loop and then run that through your test logic. For the outer list iterate over every part of the inner list and then move to the next part of the outer list.
I have a Python list which holds pairs of key/value:
l = [[1, 'A'], [1, 'B'], [2, 'C']]
I want to convert the list into a dictionary, where multiple values per key would be aggregated into a tuple:
{1: ('A', 'B'), 2: ('C',)}
The iterative solution is trivial:
l = [[1, 'A'], [1, 'B'], [2, 'C']]
d = {}
for pair in l:
if pair[0] in d:
d[pair[0]] = d[pair[0]] + tuple(pair[1])
else:
d[pair[0]] = tuple(pair[1])
print(d)
{1: ('A', 'B'), 2: ('C',)}
Is there a more elegant, Pythonic solution for this task?
from collections import defaultdict
d1 = defaultdict(list)
for k, v in l:
d1[k].append(v)
d = dict((k, tuple(v)) for k, v in d1.items())
d contains now {1: ('A', 'B'), 2: ('C',)}
d1 is a temporary defaultdict with lists as values, which will be converted to tuples in the last line. This way you are appending to lists and not recreating tuples in the main loop.
Using lists instead of tuples as dict values:
l = [[1, 'A'], [1, 'B'], [2, 'C']]
d = {}
for key, val in l:
d.setdefault(key, []).append(val)
print(d)
Using a plain dictionary is often preferable over a defaultdict, in particular if you build it just once and then continue to read from it later in your code:
First, the plain dictionary is faster to build and access.
Second, and more importantly, the later read operations will error out if you try to access a key that doesn't exist, instead of silently creating that key. A plain dictionary lets you explicitly state when you want to create a key-value pair, while the defaultdict always implicitly creates them, on any kind of access.
This method is relatively efficient and quite compact:
reduce(lambda x, (k,v): x[k].append(v) or x, l, defaultdict(list))
In Python3 this becomes (making exports explicit):
dict(functools.reduce(lambda x, d: x[d[0]].append(d[1]) or x, l, collections.defaultdict(list)))
Note that reduce has moved to functools and that lambdas no longer accept tuples. This version still works in 2.6 and 2.7.
Are the keys already sorted in the input list? If that's the case, you have a functional solution:
import itertools
lst = [(1, 'A'), (1, 'B'), (2, 'C')]
dct = dict((key, tuple(v for (k, v) in pairs))
for (key, pairs) in itertools.groupby(lst, lambda pair: pair[0]))
print dct
# {1: ('A', 'B'), 2: ('C',)}
I had a list of values created as follows:
performance_data = driver.execute_script('return window.performance.getEntries()')
Then I had to store the data (name and duration) in a dictionary with multiple values:
dictionary = {}
for performance_data in range(3):
driver.get(self.base_url)
performance_data = driver.execute_script('return window.performance.getEntries()')
for result in performance_data:
key=result['name']
val=result['duration']
dictionary.setdefault(key, []).append(val)
print(dictionary)
My data was in a Pandas.DataFrame
myDict = dict()
for idin set(data['id'].values):
temp = data[data['id'] == id]
myDict[id] = temp['IP_addr'].to_list()
myDict
Gave me a Dict of the keys, ID, mappings to >= 1 IP_addr. The first IP_addr is Guaranteed. My code should work even if temp['IP_addr'].to_list() == []
{'fooboo_NaN': ['1.1.1.1', '8.8.8.8']}
My two coins for toss into that amazing discussion)
I've tried to wonder around one line solution with only standad libraries. Excuse me for the two excessive imports. Perhaps below code could solve the issue with satisfying quality (for the python3):
from functools import reduce
from collections import defaultdict
a = [1, 1, 2, 3, 1]
b = ['A', 'B', 'C', 'D', 'E']
c = zip(a, b)
print({**reduce(lambda d,e: d[e[0]].append(e[1]) or d, c, defaultdict(list))})