Related
I have a list of dictionary, and I would like to count the values for keys of "increasing" and "decreasing" respectively. My script returned a key error that might because not all dictionaries have "increasing" and "decreasing" both. But I have no idea how to fix it. As a Python beginner, any help would be appreciated.
list_of_dicts = [{"decreasing": 1}, {"increasing": 4}, {"decreasing": 1}, {"increasing": 3},{"decreasing": 1},
{"increasing": 1}]
values1 = [a_dict["decreasing"] for a_dict in list_of_dicts]
values2 = [a_dict["increasing"] for a_dict in list_of_dicts]
print(values1)
print(values2)
The expected result is:
[1,1,1]
[4,3,1]
You can use sum() + dict.get with default value 0:
values1 = sum(d.get("increasing", 0) for d in list_of_dicts)
values2 = sum(d.get("decreasing", 0) for d in list_of_dicts)
print("Increasing:", values1)
print("Decreasing:", values2)
Prints:
Increasing: 29
Decreasing: 9
EDIT: To get values:
values1, values2 = [], []
for d in list_of_dicts:
if "increasing" in d:
values1.append(d["increasing"])
elif "decreasing" in d:
values2.append(d["decreasing"])
print("Increasing:", values1)
print("Decreasing:", values2)
Prints:
Increasing: [4, 3, 1, 5, 11, 1, 4]
Decreasing: [1, 1, 1, 1, 2, 1, 2]
Add an if in the list comprehension, that'll keep the good ones
values1 = [a_dict["decreasing"] for a_dict in list_of_dicts if "decreasing" in a_dict]
values2 = [a_dict["increasing"] for a_dict in list_of_dicts if "increasing" in a_dict]
print(values1) # [1, 1, 1, 1, 2, 1, 2]
print(values2) # [4, 3, 1, 5, 11, 1, 4]
When working with list comprehensions that are having issues, break them down.
First it can be broken down into:
values1 = list()
for a_dict in list_of_dicts:
if a_dict.get('decreasing'):
values1.append(a_dict.get('decreasing'))
values2 = list()
for a_dict in list_of_dicts:
if a_dict.get('increasing'):
values2.append(a_dict.get('increasing'))
Then bring them back up into the comprehension.
list_of_dicts = [{"decreasing": 1}, {"increasing": 4}, {"decreasing": 1}, {"increasing": 3},{"decreasing": 1},
{"increasing": 1},{"decreasing": 1}, {"increasing": 5}, {"decreasing": 2}, {"increasing": 11},
{"decreasing": 1}, {"increasing": 1},{"decreasing": 2}, {"increasing": 4}]
values1 = [a_dict.get('decreasing') for a_dict in list_of_dicts if a_dict.get('decreasing')]
values2 = [a_dict.get('increasing') for a_dict in list_of_dicts if a_dict.get('increasing')]
values11 = list()
for a_dict in list_of_dicts:
if a_dict.get('decreasing'):
values11.append(a_dict.get('decreasing'))
values22 = list()
for a_dict in list_of_dicts:
if a_dict.get('increasing'):
values22.append(a_dict.get('increasing'))
print(values1)
print(values2)
print(values1 == values11)
print(values2 == values22)
Results in
$ python testing.py
[1, 1, 1, 1, 2, 1, 2]
[4, 3, 1, 5, 11, 1, 4]
True
True
I have the following output from a code I ran on Python:
T1 = [{0: 0}, {15: 3}, {19: 1}, {20: 1}, {0: 0}]
I want to extract the keys and values from each object respectively. For T1, I would thus have:
P1 = [0,15,19,20,0]
D1 = [0, 3, 1,1,0]
What would be the best way to code it?
Thanks in advance,
Sounds like a good case for chain.from_iterable:
>>> from itertools import chain
>>> from operator import methodcaller
>>> T1 = [{0: 0}, {15: 3}, {19: 1}, {20: 1}, {0: 0}]
>>> list(chain.from_iterable(T1))
[0, 15, 19, 20, 0]
>>> list(chain.from_iterable(map(methodcaller('values'), T1)))
[0, 3, 1, 1, 0]
A dictionary when iterated over yields its keys; chain.from_iterable takes a list of such iterables and yields all their keys in a sequence. To do the same with the values, call values() on each item, for which we map a methodcaller here (equivalent to (i.values() for i in T1)).
this should work:
T1 = [{0: 0}, {15: 3}, {19: 1}, {20: 1}, {0: 0}]
P1 = [next(iter(dct)) for dct in T1]
D1 = [next(iter(dct.values())) for dct in T1]
you take the first element (next) of an iterator over the keys (iter(dct)) or an interator over the values (iter(dct.values()).
this will not create any unnecessary lists.
or in one go (note: these return tuples not lists):
P1, D1 = zip(*(next(iter(dct.items())) for dct in T1))
or (using parts of deceze's answer):
from itertools import chain
P1, D1 = zip(*chain.from_iterable(dct.items() for dct in T1))
Use List Comprehensions:
In [148]: P1 = [list(i.keys())[0] for i in T1]
In [149]: D1 = [list(i.values())[0] for i in T1]
In [150]: P1
Out[150]: [0, 15, 19, 20, 0]
In [151]: D1
Out[151]: [0, 3, 1, 1, 0]
If I have a nested dictionary in Python, is there any way to restructure it based on keys?
I'm bad at explaining, so I'll give a little example.
d = {'A':{'a':[1,2,3],'b':[3,4,5],'c':[6,7,8]},
'B':{'a':[7,8,9],'b':[4,3,2],'d':[0,0,0]}}
Re-organize like this
newd = {'a':{'A':[1,2,3],'B':[7,8,9]},
'b':{'A':[3,4,5],'B':[4,3,2]},
'c':{'A':[6,7,8]},
'd':{'B':[0,0,0]}}
Given some function with inputs like
def mysteryfunc(olddict,newkeyorder):
????
mysteryfunc(d,[1,0])
Where the [1,0] list passed means to put the dictionaries 2nd level of keys in the first level and the first level in the 2nd level. Obviously the values need to be associated with their unique key values.
Edit:
Looking for an answer that covers the general case, with arbitrary unknown nested dictionary depth.
Input:
d = {'A':{'a':[1,2,3],'b':[3,4,5],'c':[6,7,8]},
'B':{'a':[7,8,9],'b':[4,3,2],'d':[0,0,0]}}
inner_dict={}
for k,v in d.items():
print(k)
for ka,va in v.items():
val_list=[]
if ka not in inner_dict:
val_dict={}
val_dict[k]=va
inner_dict[ka]=val_dict
else:
val_dict=inner_dict[ka]
val_dict[k]=va
inner_dict[ka]=val_dict
Output:
{'a': {'A': [1, 2, 3], 'B': [7, 8, 9]},
'b': {'A': [3, 4, 5], 'B': [4, 3, 2]},
'c': {'A': [6, 7, 8]},
'd': {'B': [0, 0, 0]}}
you can use 2 for loops, one to iterate over each key, value pair and the second for loop to iterate over the nested dict, at each step form the second for loop iteration you can build your desired output:
from collections import defaultdict
new_dict = defaultdict(dict)
for k0, v0 in d.items():
for k1, v1 in v0.items():
new_dict[k1][k0] = v1
print(dict(new_dict))
output:
{'a': {'A': [1, 2, 3], 'B': [7, 8, 9]},
'b': {'A': [3, 4, 5], 'B': [4, 3, 2]},
'c': {'A': [6, 7, 8]},
'd': {'B': [0, 0, 0]}}
You can use recursion with a generator to handle input of arbitrary depth:
def paths(d, c = []):
for a, b in d.items():
yield from ([((c+[a])[::-1], b)] if not isinstance(b, dict) else paths(b, c+[a]))
from collections import defaultdict
def group(d):
_d = defaultdict(list)
for [a, *b], c in d:
_d[a].append([b, c])
return {a:b[-1][-1] if not b[0][0] else group(b) for a, b in _d.items()}
print(group(list(paths(d))))
Output:
{'a': {'A': [1, 2, 3], 'B': [7, 8, 9]}, 'b': {'A': [3, 4, 5], 'B': [4, 3, 2]}, 'c': {'A': [6, 7, 8]}, 'd': {'B': [0, 0, 0]}}
I have Nested dictionary something like this.
{'A': {'21-26': 2,
'26-31': 7,
'31-36': 3,
'36-41': 2,
'41-46': 0,
'46-51': 0,
'Above 51': 0},
'B': {'21-26': 2,
'26-31': 11,
'31-36': 5,
'36-41': 4,
'41-46': 1,
'46-51': 0,
'Above 51': 3}}
And I want to create list by key from second dictionary.
And i don't want duplicates in my list.
Required Output is
ls = ['21-26','26-31','31-36','36-41','41-46','46-51','Above 51']
Thank you for your time and consideration.
You can use:
>>> list(set(key for val in d.values() for key in val.keys()))
['21-26', '36-41', '31-36', '46-51', 'Above 51', '26-31', '41-46']
Where d is your dictionary.
Simple set comprehension, then convert to list. a is your dict.
list({k for v in a.values() for k in v.keys()})
Output ordering is random, but you can sort how you like.
Can you use pandas? IF so:
import pandas as pd
a = {'A': {'21-26': 2, '26-31': 7, '31-36': 3, '36-41': 2, '41-46': 0, '46-51': 0, 'Above 51': 0}, 'B': {'21-26': 2, '26-31': 11, '31-36': 5, '36-41': 4, '41-46': 1, '46-51': 0, 'Above 51': 3}}
pd.DataFrame(a).index.to_list()
output:
['21-26', '26-31', '31-36', '36-41', '41-46', '46-51', 'Above 51']
You can use chain.from_iterable() to chain inner dictionaries and dict.fromkeys() to remove duplicates:
from itertools import chain
c = chain.from_iterable(dct.values())
result = list(dict.fromkeys(c))
I want to change back and forth between a dictionary of (equal-length) lists:
DL = {'a': [0, 1], 'b': [2, 3]}
and a list of dictionaries:
LD = [{'a': 0, 'b': 2}, {'a': 1, 'b': 3}]
For those of you that enjoy clever/hacky one-liners.
Here is DL to LD:
v = [dict(zip(DL,t)) for t in zip(*DL.values())]
print(v)
and LD to DL:
v = {k: [dic[k] for dic in LD] for k in LD[0]}
print(v)
LD to DL is a little hackier since you are assuming that the keys are the same in each dict. Also, please note that I do not condone the use of such code in any kind of real system.
If you're allowed to use outside packages, Pandas works great for this:
import pandas as pd
pd.DataFrame(DL).to_dict(orient="records")
Which outputs:
[{'a': 0, 'b': 2}, {'a': 1, 'b': 3}]
You can also use orient="list" to get back the original structure
{'a': [0, 1], 'b': [2, 3]}
Perhaps consider using numpy:
import numpy as np
arr = np.array([(0, 2), (1, 3)], dtype=[('a', int), ('b', int)])
print(arr)
# [(0, 2) (1, 3)]
Here we access columns indexed by names, e.g. 'a', or 'b' (sort of like DL):
print(arr['a'])
# [0 1]
Here we access rows by integer index (sort of like LD):
print(arr[0])
# (0, 2)
Each value in the row can be accessed by column name (sort of like LD):
print(arr[0]['b'])
# 2
To go from the list of dictionaries, it is straightforward:
You can use this form:
DL={'a':[0,1],'b':[2,3], 'c':[4,5]}
LD=[{'a':0,'b':2, 'c':4},{'a':1,'b':3, 'c':5}]
nd={}
for d in LD:
for k,v in d.items():
try:
nd[k].append(v)
except KeyError:
nd[k]=[v]
print nd
#{'a': [0, 1], 'c': [4, 5], 'b': [2, 3]}
Or use defaultdict:
nd=cl.defaultdict(list)
for d in LD:
for key,val in d.items():
nd[key].append(val)
print dict(nd.items())
#{'a': [0, 1], 'c': [4, 5], 'b': [2, 3]}
Going the other way is problematic. You need to have some information of the insertion order into the list from keys from the dictionary. Recall that the order of keys in a dict is not necessarily the same as the original insertion order.
For giggles, assume the insertion order is based on sorted keys. You can then do it this way:
nl=[]
nl_index=[]
for k in sorted(DL.keys()):
nl.append({k:[]})
nl_index.append(k)
for key,l in DL.items():
for item in l:
nl[nl_index.index(key)][key].append(item)
print nl
#[{'a': [0, 1]}, {'b': [2, 3]}, {'c': [4, 5]}]
If your question was based on curiosity, there is your answer. If you have a real-world problem, let me suggest you rethink your data structures. Neither of these seems to be a very scalable solution.
Here are the one-line solutions (spread out over multiple lines for readability) that I came up with:
if dl is your original dict of lists:
dl = {"a":[0, 1],"b":[2, 3]}
Then here's how to convert it to a list of dicts:
ld = [{key:value[index] for key,value in dl.items()}
for index in range(max(map(len,dl.values())))]
Which, if you assume that all your lists are the same length, you can simplify and gain a performance increase by going to:
ld = [{key:value[index] for key, value in dl.items()}
for index in range(len(dl.values()[0]))]
Here's how to convert that back into a dict of lists:
dl2 = {key:[item[key] for item in ld]
for key in list(functools.reduce(
lambda x, y: x.union(y),
(set(dicts.keys()) for dicts in ld)
))
}
If you're using Python 2 instead of Python 3, you can just use reduce instead of functools.reduce there.
You can simplify this if you assume that all the dicts in your list will have the same keys:
dl2 = {key:[item[key] for item in ld] for key in ld[0].keys() }
cytoolz.dicttoolz.merge_with
Docs
from cytoolz.dicttoolz import merge_with
merge_with(list, *LD)
{'a': [0, 1], 'b': [2, 3]}
Non-cython version
Docs
from toolz.dicttoolz import merge_with
merge_with(list, *LD)
{'a': [0, 1], 'b': [2, 3]}
The python module of pandas can give you an easy-understanding solution. As a complement to #chiang's answer, the solutions of both D-to-L and L-to-D are as follows:
import pandas as pd
DL = {'a': [0, 1], 'b': [2, 3]}
out1 = pd.DataFrame(DL).to_dict('records')
Output:
[{'a': 0, 'b': 2}, {'a': 1, 'b': 3}]
In the other direction:
LD = [{'a': 0, 'b': 2}, {'a': 1, 'b': 3}]
out2 = pd.DataFrame(LD).to_dict('list')
Output:
{'a': [0, 1], 'b': [2, 3]}
Cleanest way I can think of a summer friday. As a bonus, it supports lists of different lengths (but in this case, DLtoLD(LDtoDL(l)) is no more identity).
From list to dict
Actually less clean than #dwerk's defaultdict version.
def LDtoDL (l) :
result = {}
for d in l :
for k, v in d.items() :
result[k] = result.get(k,[]) + [v] #inefficient
return result
From dict to list
def DLtoLD (d) :
if not d :
return []
#reserve as much *distinct* dicts as the longest sequence
result = [{} for i in range(max (map (len, d.values())))]
#fill each dict, one key at a time
for k, seq in d.items() :
for oneDict, oneValue in zip(result, seq) :
oneDict[k] = oneValue
return result
I needed such a method which works for lists of different lengths (so this is a generalization of the original question). Since I did not find any code here that the way that I expected, here's my code which works for me:
def dict_of_lists_to_list_of_dicts(dict_of_lists: Dict[S, List[T]]) -> List[Dict[S, T]]:
keys = list(dict_of_lists.keys())
list_of_values = [dict_of_lists[key] for key in keys]
product = list(itertools.product(*list_of_values))
return [dict(zip(keys, product_elem)) for product_elem in product]
Examples:
>>> dict_of_lists_to_list_of_dicts({1: [3], 2: [4, 5]})
[{1: 3, 2: 4}, {1: 3, 2: 5}]
>>> dict_of_lists_to_list_of_dicts({1: [3, 4], 2: [5]})
[{1: 3, 2: 5}, {1: 4, 2: 5}]
>>> dict_of_lists_to_list_of_dicts({1: [3, 4], 2: [5, 6]})
[{1: 3, 2: 5}, {1: 3, 2: 6}, {1: 4, 2: 5}, {1: 4, 2: 6}]
>>> dict_of_lists_to_list_of_dicts({1: [3, 4], 2: [5, 6], 7: [8, 9, 10]})
[{1: 3, 2: 5, 7: 8},
{1: 3, 2: 5, 7: 9},
{1: 3, 2: 5, 7: 10},
{1: 3, 2: 6, 7: 8},
{1: 3, 2: 6, 7: 9},
{1: 3, 2: 6, 7: 10},
{1: 4, 2: 5, 7: 8},
{1: 4, 2: 5, 7: 9},
{1: 4, 2: 5, 7: 10},
{1: 4, 2: 6, 7: 8},
{1: 4, 2: 6, 7: 9},
{1: 4, 2: 6, 7: 10}]
Here my small script :
a = {'a': [0, 1], 'b': [2, 3]}
elem = {}
result = []
for i in a['a']: # (1)
for key, value in a.items():
elem[key] = value[i]
result.append(elem)
elem = {}
print result
I'm not sure that is the beautiful way.
(1) You suppose that you have the same length for the lists
Here is a solution without any libraries used:
def dl_to_ld(initial):
finalList = []
neededLen = 0
for key in initial:
if(len(initial[key]) > neededLen):
neededLen = len(initial[key])
for i in range(neededLen):
finalList.append({})
for i in range(len(finalList)):
for key in initial:
try:
finalList[i][key] = initial[key][i]
except:
pass
return finalList
You can call it as follows:
dl = {'a':[0,1],'b':[2,3]}
print(dl_to_ld(dl))
#[{'a': 0, 'b': 2}, {'a': 1, 'b': 3}]
If you don't mind a generator, you can use something like
def f(dl):
l = list((k,v.__iter__()) for k,v in dl.items())
while True:
d = dict((k,i.next()) for k,i in l)
if not d:
break
yield d
It's not as "clean" as it could be for Technical Reasons: My original implementation did yield dict(...), but this ends up being the empty dictionary because (in Python 2.5) a for b in c does not distinguish between a StopIteration exception when iterating over c and a StopIteration exception when evaluating a.
On the other hand, I can't work out what you're actually trying to do; it might be more sensible to design a data structure that meets your requirements instead of trying to shoehorn it in to the existing data structures. (For example, a list of dicts is a poor way to represent the result of a database query.)
List of dicts ⟶ dict of lists
from collections import defaultdict
from typing import TypeVar
K = TypeVar("K")
V = TypeVar("V")
def ld_to_dl(ld: list[dict[K, V]]) -> dict[K, list[V]]:
dl = defaultdict(list)
for d in ld:
for k, v in d.items():
dl[k].append(v)
return dl
defaultdict creates an empty list if one does not exist upon key access.
Dict of lists ⟶ list of dicts
Collecting into "jagged" dictionaries
from typing import TypeVar
K = TypeVar("K")
V = TypeVar("V")
def dl_to_ld(dl: dict[K, list[V]]) -> list[dict[K, V]]:
ld = []
for k, vs in dl.items():
ld += [{} for _ in range(len(vs) - len(ld))]
for i, v in enumerate(vs):
ld[i][k] = v
return ld
This generates a list of dictionaries ld that may be missing items if the lengths of the lists in dl are unequal. It loops over all key-values in dl, and creates empty dictionaries if ld does not have enough.
Collecting into "complete" dictionaries only
(Usually intended only for equal-length lists.)
from typing import TypeVar
K = TypeVar("K")
V = TypeVar("V")
def dl_to_ld(dl: dict[K, list[V]]) -> list[dict[K, V]]:
ld = [dict(zip(dl.keys(), v)) for v in zip(*dl.values())]
return ld
This generates a list of dictionaries ld that have the length of the smallest list in dl.
DL={'a':[0,1,2,3],'b':[2,3,4,5]}
LD=[{'a':0,'b':2},{'a':1,'b':3}]
Empty_list = []
Empty_dict = {}
# to find length of list in values of dictionry
len_list = 0
for i in DL.values():
if len_list < len(i):
len_list = len(i)
for k in range(len_list):
for i,j in DL.items():
Empty_dict[i] = j[k]
Empty_list.append(Empty_dict)
Empty_dict = {}
LD = Empty_list