Replace Nested for loop in list-comprehension [duplicate] - python

This question already has answers here:
Avoiding nested for loops
(3 answers)
Closed 3 months ago.
I want to combine my id list with my status list and I used list comprehension to do it:
# id
id_list = [
1, # UAE1S
2, # UAE2S
3, # UAE3S
]
# status
status_list = [
'okay',
'not okay',
'unknown',
]
result = [
{
'id':id,
'status':status,
}
for id in id_list
for status in status_list
]
print(result)
[{'id': 1, 'status': 'okay'}, {'id': 1, 'status': 'not okay'}, {'id': 1, 'status': 'unknown'}, {'id': 2, 'status': 'okay'}, {'id': 2, 'status': 'not okay'}, {'id': 2, 'status': 'unknown'}, {'id': 3, 'status': 'okay'}, {'id': 3, 'status': 'not okay'}, {'id': 3, 'status': 'unknown'}]
It's outputting the correct list but is there a way to remove the nested for loop?


itertools.product gives the cartesian product,
import itertools
id_list = [1, 2, 3]
status_list = ['okay','not okay','unknown',]
[{'id': item[0], 'status': item[1]} for item in itertools.product(id_list, status_list)]
[{'status': 'okay', 'id': 1}, {'status': 'not okay', 'id': 1}, {'status': 'unknown', 'id': 1}, {'status': 'okay', 'id': 2}, {'status': 'not okay', 'id': 2}, {'status': 'unknown', 'id': 2}, {'status': 'okay', 'id': 3}, {'status': 'not okay', 'id': 3}, {'status': 'unknown', 'id': 3}]

Related

from list of dicts to list of lists of dicts with same values

I have list of dicts:
dict_list = [{'Id': 0, 'UserID': 1, 'Name': 'John'},
{'Id': 1, 'UserID': 2, 'Name': 'Martin'},
{'Id': 2, 'UserID': 1, 'Name': 'Rob'},
{'Id': 3, 'UserID': 1, 'Name': 'Neil'},
{'Id': 4, 'UserID': 2, 'Name': 'Bill'}]
How to make a list of lists of dicts that grouped by key UserID?
So I want to group dicts with the same value of key UserID to lists.
I expect smth like that:
[[{'Id': 0,'UserID': 1, 'Name': 'John'},
{'Id': 2,'UserID': 1, 'Name': 'Rob'},
{'Id': 3,'UserID': 1, 'Name': 'Neil'}],
[{'Id': 1,'UserID': 2, 'Name': 'Martin'},
{'Id': 4,'UserID': 2, 'Name': 'Bill'}]]

First sort the dict_list based on UserID and then use itertools.groupby to group the results based on UserID
>>> from itertools import groupby
>>> key = lambda d: d['UserID']
>>> res = [list(grp) for _,grp in groupby(sorted(dict_list, key=key), key)]
>>>
>>> pprint(res)
[[{'Id': 0, 'Name': 'John', 'UserID': 1},
{'Id': 2, 'Name': 'Rob', 'UserID': 1},
{'Id': 3, 'Name': 'Neil', 'UserID': 1}],
[{'Id': 1, 'Name': 'Martin', 'UserID': 2},
{'Id': 4, 'Name': 'Bill', 'UserID': 2}]]

It's also possible to use list comprehension like this:
dict_list = [{'Id': 0, 'UserID': 1, 'Name': 'John'},
{'Id': 1, 'UserID': 2, 'Name': 'Martin'},
{'Id': 2, 'UserID': 1, 'Name': 'Rob'},
{'Id': 3, 'UserID': 1, 'Name': 'Neil'},
{'Id': 4, 'UserID': 2, 'Name': 'Bill'}]
user_ids=set([x['UserID'] for x in dict_list])
result_list=[]
for user_id in user_ids:
user_id_list = [x for x in dict_list if x['UserID']==user_id]
result_list.append(user_id_list)
print(result_list)

from itertools import groupby
dict_list = [{'Id': 0, 'UserID': 1, 'Name': 'John'},
{'Id': 1, 'UserID': 2, 'Name': 'Martin'},
{'Id': 2, 'UserID': 1, 'Name': 'Rob'},
{'Id': 3, 'UserID': 1, 'Name': 'Neil'},
{'Id': 4, 'UserID': 2, 'Name': 'Bill'}]
res =[list(group) for _,group in groupby(sorted(dict_list, key=lambda f: f['UserID']), lambda f: f['UserID'])]
print(res)

Iterate over two lists of dicts and create list of tuples without loop

I have two lists of dicts: list1 and list2.
print(list1)
[{'name': 'fooa', 'desc': 'bazv', 'city': 1, 'ID': 1},
{'name': 'bard', 'desc': 'besd', 'city': 2, 'ID': 1},
{'name': 'baer', 'desc': 'bees', 'city': 2, 'ID': 1},
{'name': 'aaaa', 'desc': 'bnbb', 'city': 1, 'ID': 2},
{'name': 'cgcc', 'desc': 'dgdd', 'city': 1, 'ID': 2}]
print(list2)
[{'name': 'foo', 'desc': 'baz', 'city': 1, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 1, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 2, 'ID': 1},
{'name': 'aaa', 'desc': 'bbb', 'city': 1, 'ID': 2},
{'name': 'ccc', 'desc': 'ddd', 'city': 1, 'ID': 2}]
I need a list of tuples that will hold two paired dicts (one dict from each list) with the same city and ID.
I did it with double loop:
list_of_tuples = []
for i in list1:
for j in list2:
if i['ID'] == j['ID'] and i['city'] == j['city']:
list_of_tuples.append((i, j))
print(list_of_tuples)
[({'name': 'fooa', 'desc': 'bazv', 'city': 1, 'ID': 1},
{'name': 'foo', 'desc': 'baz', 'city': 1, 'ID': 1}),
({'name': 'fooa', 'desc': 'bazv', 'city': 1, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 1, 'ID': 1}),
({'name': 'bard', 'desc': 'besd', 'city': 2, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 2, 'ID': 1}),
({'name': 'baer', 'desc': 'bees', 'city': 2, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 2, 'ID': 1}),
({'name': 'aaaa', 'desc': 'bnbb', 'city': 1, 'ID': 2},
{'name': 'aaa', 'desc': 'bbb', 'city': 1, 'ID': 2}),
({'name': 'aaaa', 'desc': 'bnbb', 'city': 1, 'ID': 2},
{'name': 'ccc', 'desc': 'ddd', 'city': 1, 'ID': 2}),
({'name': 'cgcc', 'desc': 'dgdd', 'city': 1, 'ID': 2},
{'name': 'aaa', 'desc': 'bbb', 'city': 1, 'ID': 2}),
({'name': 'cgcc', 'desc': 'dgdd', 'city': 1, 'ID': 2},
{'name': 'ccc', 'desc': 'ddd', 'city': 1, 'ID': 2})]
Question: How to do this in a more pythonic way (without loops)?

You can use itertools.product and filter:
from itertools import product
list1 = [{'name': 'fooa', 'desc': 'bazv', 'city': 1, 'ID': 1},
{'name': 'bard', 'desc': 'besd', 'city': 2, 'ID': 1},
{'name': 'baer', 'desc': 'bees', 'city': 2, 'ID': 1},
{'name': 'aaaa', 'desc': 'bnbb', 'city': 1, 'ID': 2},
{'name': 'cgcc', 'desc': 'dgdd', 'city': 1, 'ID': 2}]
list2 = [{'name': 'foo', 'desc': 'baz', 'city': 1, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 1, 'ID': 1},
{'name': 'bar', 'desc': 'bes', 'city': 2, 'ID': 1},
{'name': 'aaa', 'desc': 'bbb', 'city': 1, 'ID': 2},
{'name': 'ccc', 'desc': 'ddd', 'city': 1, 'ID': 2}]
def condition(x):
return x[0]['ID'] == x[1]['ID'] and x[0]['city'] == x[1]['city']
list_of_tuples = list(filter(condition, product(list1, list2)))

This is a problem well suited for pandas. If you convert the lists to DataFrames, matching the records on ID and city is the same as an inner join of the two DataFrames.
import pandas as pd
# convert lists to DataFrames
df1 = pd.DataFrame(list1)
df2 = pd.DataFrame(list2)
# merge the two DataFrames
print(df1.merge(df2, on=["ID", "city"]))
# ID city desc_x name_x desc_y name_y
#0 1 1 bazv fooa baz foo
#1 1 1 bazv fooa bes bar
#2 1 2 besd bard bes bar
#3 1 2 bees baer bes bar
#4 2 1 bnbb aaaa bbb aaa
#5 2 1 bnbb aaaa ddd ccc
#6 2 1 dgdd cgcc bbb aaa
#7 2 1 dgdd cgcc ddd ccc
Now you have the matched records in each row. Since the desc and name columns were present in both (and not used for the merge), they get subscripted with _x and _y to differentiate between the two souce DataFrames.
You just need to reformat it to be in your desired output. You can achieve this using to_dict and a list comprehension:
list_of_tuples = [
(
{"name": r["name_x"], "desc": r["desc_x"], "city": r["city"], "ID": r["ID"]},
{"name": r["name_y"], "desc": r["desc_y"], "city": r["city"], "ID": r["ID"]}
) for r in df1.merge(df2, on=["ID", "city"]).to_dict(orient="records")
]
print(list_of_tuples)
#[({'ID': 1, 'city': 1, 'desc': 'bazv', 'name': 'fooa'},
# {'ID': 1, 'city': 1, 'desc': 'baz', 'name': 'foo'}),
# ({'ID': 1, 'city': 1, 'desc': 'bazv', 'name': 'fooa'},
# {'ID': 1, 'city': 1, 'desc': 'bes', 'name': 'bar'}),
# ({'ID': 1, 'city': 2, 'desc': 'besd', 'name': 'bard'},
# {'ID': 1, 'city': 2, 'desc': 'bes', 'name': 'bar'}),
# ({'ID': 1, 'city': 2, 'desc': 'bees', 'name': 'baer'},
# {'ID': 1, 'city': 2, 'desc': 'bes', 'name': 'bar'}),
# ({'ID': 2, 'city': 1, 'desc': 'bnbb', 'name': 'aaaa'},
# {'ID': 2, 'city': 1, 'desc': 'bbb', 'name': 'aaa'}),
# ({'ID': 2, 'city': 1, 'desc': 'bnbb', 'name': 'aaaa'},
# {'ID': 2, 'city': 1, 'desc': 'ddd', 'name': 'ccc'}),
# ({'ID': 2, 'city': 1, 'desc': 'dgdd', 'name': 'cgcc'},
# {'ID': 2, 'city': 1, 'desc': 'bbb', 'name': 'aaa'}),
# ({'ID': 2, 'city': 1, 'desc': 'dgdd', 'name': 'cgcc'},
# {'ID': 2, 'city': 1, 'desc': 'ddd', 'name': 'ccc'})]

Having nested loops is not "not pythonic". However, you can achieve the same result with a list comprehension. I don't think it's more readable though:
[(i, j) for j in list2 for i in list1 if i['ID'] == j['ID'] and i['city'] == j['city']]

Sort a list of dict with a key from another list of dict

In the following example, I would like to sort the animals by the alphabetical order of their category, which is stored in an order dictionnary.
category = [{'uid': 0, 'name': 'mammals'},
{'uid': 1, 'name': 'birds'},
{'uid': 2, 'name': 'fish'},
{'uid': 3, 'name': 'reptiles'},
{'uid': 4, 'name': 'invertebrates'},
{'uid': 5, 'name': 'amphibians'}]
animals = [{'name': 'horse', 'category': 0},
{'name': 'whale', 'category': 2},
{'name': 'mollusk', 'category': 4},
{'name': 'tuna ', 'category': 2},
{'name': 'worms', 'category': 4},
{'name': 'frog', 'category': 5},
{'name': 'dog', 'category': 0},
{'name': 'salamander', 'category': 5},
{'name': 'horse', 'category': 0},
{'name': 'octopus', 'category': 4},
{'name': 'alligator', 'category': 3},
{'name': 'monkey', 'category': 0},
{'name': 'kangaroos', 'category': 0},
{'name': 'salmon', 'category': 2}]
sorted_animals = sorted(animals, key=lambda k: (k['category'])
How could I achieve this?
Thanks.

You are now sorting on the category id. All you need to do is map that id to a lookup for a given category name.
Create a dictionary for the categories first so you can directly map the numeric id to the associated name from the category list, then use that mapping when sorting:
catuid_to_name = {c['uid']: c['name'] for c in category}
sorted_animals = sorted(animals, key=lambda k: catuid_to_name[k['category']])
Demo:
>>> from pprint import pprint
>>> category = [{'uid': 0, 'name': 'mammals'},
... {'uid': 1, 'name': 'birds'},
... {'uid': 2, 'name': 'fish'},
... {'uid': 3, 'name': 'reptiles'},
... {'uid': 4, 'name': 'invertebrates'},
... {'uid': 5, 'name': 'amphibians'}]
>>> animals = [{'name': 'horse', 'category': 0},
... {'name': 'whale', 'category': 2},
... {'name': 'mollusk', 'category': 4},
... {'name': 'tuna ', 'category': 2},
... {'name': 'worms', 'category': 4},
... {'name': 'frog', 'category': 5},
... {'name': 'dog', 'category': 0},
... {'name': 'salamander', 'category': 5},
... {'name': 'horse', 'category': 0},
... {'name': 'octopus', 'category': 4},
... {'name': 'alligator', 'category': 3},
... {'name': 'monkey', 'category': 0},
... {'name': 'kangaroos', 'category': 0},
... {'name': 'salmon', 'category': 2}]
>>> catuid_to_name = {c['uid']: c['name'] for c in category}
>>> pprint(catuid_to_name)
{0: 'mammals',
1: 'birds',
2: 'fish',
3: 'reptiles',
4: 'invertebrates',
5: 'amphibians'}
>>> sorted_animals = sorted(animals, key=lambda k: catuid_to_name[k['category']])
>>> pprint(sorted_animals)
[{'category': 5, 'name': 'frog'},
{'category': 5, 'name': 'salamander'},
{'category': 2, 'name': 'whale'},
{'category': 2, 'name': 'tuna '},
{'category': 2, 'name': 'salmon'},
{'category': 4, 'name': 'mollusk'},
{'category': 4, 'name': 'worms'},
{'category': 4, 'name': 'octopus'},
{'category': 0, 'name': 'horse'},
{'category': 0, 'name': 'dog'},
{'category': 0, 'name': 'horse'},
{'category': 0, 'name': 'monkey'},
{'category': 0, 'name': 'kangaroos'},
{'category': 3, 'name': 'alligator'}]
Note that within each category, the dictionaries have been left in relative input order. You could return a tuple of values from the sorting key to further apply a sorting order within each category, e.g.:
sorted_animals = sorted(
animals,
key=lambda k: (catuid_to_name[k['category']], k['name'])
)
would sort by animal name within each category, producing:
>>> pprint(sorted(animals, key=lambda k: (catuid_to_name[k['category']], k['name'])))
[{'category': 5, 'name': 'frog'},
{'category': 5, 'name': 'salamander'},
{'category': 2, 'name': 'salmon'},
{'category': 2, 'name': 'tuna '},
{'category': 2, 'name': 'whale'},
{'category': 4, 'name': 'mollusk'},
{'category': 4, 'name': 'octopus'},
{'category': 4, 'name': 'worms'},
{'category': 0, 'name': 'dog'},
{'category': 0, 'name': 'horse'},
{'category': 0, 'name': 'horse'},
{'category': 0, 'name': 'kangaroos'},
{'category': 0, 'name': 'monkey'},
{'category': 3, 'name': 'alligator'}]

imo your category structure is far too complicated - at least as long as the uid is nothing but the index, you could simply use a list for that:
category = [c['name'] for c in category]
# ['mammals', 'birds', 'fish', 'reptiles', 'invertebrates', 'amphibians']
sorted_animals = sorted(animals, key=lambda k: category[k['category']])
#[{'name': 'frog', 'category': 5}, {'name': 'salamander', 'category': 5}, {'name': 'whale', 'category': 2}, {'name': 'tuna ', 'category': 2}, {'name': 'salmon', 'category': 2}, {'name': 'mollusk', 'category': 4}, {'name': 'worms', 'category': 4}, {'name': 'octopus', 'category': 4}, {'name': 'horse', 'category': 0}, {'name': 'dog', 'category': 0}, {'name': 'horse', 'category': 0}, {'name': 'monkey', 'category': 0}, {'name': 'kangaroos', 'category': 0}, {'name': 'alligator', 'category': 3}]

Merge a list of separate JSON data into one piece of JSON data

In Python, I am trying to turn a list of separate JSON data:
[[{'id': 1, 'name': 'pencil', 'description': '2b or not 2b, that is the question'}], [{'id': 2, 'name': 'oil pastel', 'description': None}], [{'id': 3, 'name': 'gouache', 'description': None}], [{'id': 4, 'name': 'paper', 'description': None}]]
into one piece of JSON data:
{'id': 1, 'name': 'pencil', 'description': '2b or not 2b, that is the question'}, {'id': 2, 'name': 'oil pastel', 'description': None}, {'id': 3, 'name': 'gouache', 'description': None}, {'id': 4, 'name': 'paper', 'description': None}, {'id': 5, 'name': 'coloured pencil', 'description': None}
Been struggling with it for a few hours. Does anyone have any ideas?

Use simple list-comprehension
[y for x in list_of_lists for y in x]
Output:
[{'description': '2b or not 2b, that is the question', 'id': 1, 'name': 'pencil'}, {'description': None, 'id': 2, 'name': 'oil pastel'}, {'description': None, 'id': 3, 'name': 'gouache'}, {'description': None, 'id': 4, 'name': 'paper'}]

Use itertools.chain
>>> list(itertools.chain.from_iterable(j))
Or a list comprehension
>>> [x[0] for x in j] # Assuming there is only one item in each list
Both outputs
[{'id': 1,
'name': 'pencil',
'description': '2b or not 2b, that is the question'},
{'id': 2, 'name': 'oil pastel', 'description': None},
{'id': 3, 'name': 'gouache', 'description': None},
{'id': 4, 'name': 'paper', 'description': None}]

Using functools with operator
j = [[{'id': 1, 'name': 'pencil', 'description': '2b or not 2b, that is the question'}], [{'id': 2, 'name': 'oil pastel', 'description': None}], [{'id': 3, 'name': 'gouache', 'description': None}], [{'id': 4, 'name': 'paper', 'description': None}]]
import functools
import operator
functools.reduce(operator.iadd,j,[])
Output:
[{'id': 1,
'name': 'pencil',
'description': '2b or not 2b, that is the question'},
{'id': 2, 'name': 'oil pastel', 'description': None},
{'id': 3, 'name': 'gouache', 'description': None},
{'id': 4, 'name': 'paper', 'description': None}]

What Is a Pythonic Way to Build a Dict of Dictionary-Lists by Attribute?

I'm looking for pythonic way to convert list of tuples which looks like this:
res = [{type: 1, name: 'Nick'}, {type: 2, name: 'Helma'}, ...]
To dict like this:
{1: [{type: 1, name: 'Nick'}, ...], 2: [{type: 2, name: 'Helma'}, ...]}
Now i do this with code like this (based on this question):
d = defaultdict(list)
for v in res:
d[v["type"]].append(v)
Is this a Pythonic way to build dict of lists of objects by attribute?

I agree with the commentators that here, list comprehension will lack, well, comprehension.
Having said that, here's how it can go:
import itertools
a = [{'type': 1, 'name': 'Nick'}, {'type': 2, 'name': 'Helma'}, {'type': 1, 'name': 'Moshe'}]
by_type = lambda a: a['type']
>>> dict([(k, list(g)) for (k, g) in itertools.groupby(sorted(a, key=by_type), key=by_type)])
{1: [{'name': 'Nick', 'type': 1}, {'name': 'Moshe', 'type': 1}], ...}
The code first sorts by 'type', then uses itertools.groupby to group by the exact same critera.
I stopped understanding this code 15 seconds after I finished writing it :-)

You could do it with a dictionary comprehension, which wouldn't be as illegible or incomprehensible as the comments suggest (IMHO):
# A collection of name and type dictionaries
res = [{'type': 1, 'name': 'Nick'},
{'type': 2, 'name': 'Helma'},
{'type': 3, 'name': 'Steve'},
{'type': 1, 'name': 'Billy'},
{'type': 3, 'name': 'George'},
{'type': 4, 'name': 'Sylvie'},
{'type': 2, 'name': 'Wilfred'},
{'type': 1, 'name': 'Jim'}]
# Creating a dictionary by type
res_new = {
item['type']: [each for each in res
if each['type'] == item['type']]
for item in res
}
>>>res_new
{1: [{'name': 'Nick', 'type': 1},
{'name': 'Billy', 'type': 1},
{'name': 'Jim', 'type': 1}],
2: [{'name': 'Helma', 'type': 2},
{'name': 'Wilfred', 'type': 2}],
3: [{'name': 'Steve', 'type': 3},
{'name': 'George', 'type': 3}],
4: [{'name': 'Sylvie', 'type': 4}]}
Unless I missed something, this should give you the result you're looking for.

Categories

Resources