I have a list of dictionaries that I am wanting to convert to a nested list with the first element of that list(lst[0]) containing the dictionary keys and the rest of the elements of the list containing values for each dictionary.
[{'id': '123',
'name': 'bob',
'city': 'LA'},
{'id': '321',
'name': 'sally',
'city': 'manhattan'},
{'id': '125',
'name': 'fred',
'city': 'miami'}]
My expected output result is:
[['id','name','city'], ['123','bob','LA'],['321','sally','manhattan'],['125','fred','miami']]
What would be a way to go about this? Any help would be greatly appreciated.
you can use:
d = [{'id': '123',
'name': 'bob',
'city': 'LA'},
{'id': '321',
'name': 'sally',
'city': 'manhattan'},
{'id': '125',
'name': 'fred',
'city': 'miami'}]
[[k for k in d[0].keys()], *[list(i.values()) for i in d ]]
output:
[['id', 'name', 'city'],
['123', 'bob', 'LA'],
['321', 'sally', 'manhattan'],
['125', 'fred', 'miami']]
first, you get a list with your keys then get a list with the values for every inner dict
>>> d = [{'id': '123',
'name': 'bob',
'city': 'LA'},
{'id': '321',
'name': 'sally',
'city': 'manhattan'},
{'id': '125',
'name': 'fred',
'city': 'miami'}]
>>> [list(x[0].keys())]+[list(i.values()) for i in d]
[['id', 'name', 'city'], ['123', 'bob', 'LA'], ['321', 'sally', 'manhattan'], ['125', 'fred', 'miami']]
Serious suggestion: To avoid the possibility of some dicts having a different iteration order, base the order off the first entry and use operator.itemgetter to get a consistent order from all entries efficiently:
import operator
d = [{'id': '123',
'name': 'bob',
'city': 'LA'},
{'id': '321',
'name': 'sally',
'city': 'manhattan'},
{'id': '125',
'name': 'fred',
'city': 'miami'}]
keys = list(d[0])
keygetter = operator.itemgetter(*keys)
result = [keys, *[list(keygetter(x)) for x in d]] # [keys, *map(list, map(keygetter, d))] might be a titch faster
If a list of tuples is acceptable, this is simpler/faster:
keys = tuple(d[0])
keygetter = operator.itemgetter(*keys)
result = [keys, *map(keygetter, d)]
Unserious suggestion: Let csv do it for you!
import csv
import io
dicts = [{'id': '123',
'name': 'bob',
'city': 'LA'},
{'id': '321',
'name': 'sally',
'city': 'manhattan'},
{'id': '125',
'name': 'fred',
'city': 'miami'}]
with io.StringIO() as sio:
writer = csv.DictWriter(sio, dicts[0].keys())
writer.writeheader()
writer.writerows(dicts)
sio.seek(0)
result = list(csv.reader(sio))
Try it online!
This can be done with for loop and enumerate() built-in method.
listOfDicts = [
{"id": "123", "name": "bob", "city": "LA"},
{"id": "321", "name": "sally", "city": "manhattan"},
{"id": "125", "name": "fred", "city": "miami"},
]
results = []
for index, dic in enumerate(listOfDicts, start = 0):
if index == 0:
results.append(list(dic.keys()))
results.append(list(dic.values()))
else:
results.append(list(dic.values()))
print(results)
output:
[['id', 'name', 'city'], ['123', 'bob', 'LA'], ['321', 'sally', 'manhattan'], ['125', 'fred', 'miami']]
Related
Input data:
s = {'111': {'name': 'john', 'exp': '1'}, '222': {'name': 'mia', 'exp': '1'}}
Code:
import jmespath
jmespath.search("(*)[?name=='john']", s)
Output:
[{'name': 'john', 'exp': '1'}]
Output I want:
[{'111': {'name': 'john', 'exp': '1'}}]
Convert the dictionary to the list
l1 = [{'key': k, 'value': v} for k, v in s.items()]
gives
[{'key': '111', 'value': {'name': 'john', 'exp': '1'}}, {'key': '222', 'value': {'name': 'mia', 'exp': '1'}}]
Select the values where the attribute name is john
l2 = jmespath.search('[?value.name == `john`]', l1)
gives
[{'key': '111', 'value': {'name': 'john', 'exp': '1'}}]
Convert the list back to the dictionary
s2 = dict([[i['key'], i['value']] for i in l2])
gives the expected result
{'111': {'name': 'john', 'exp': '1'}}
Example of complete code for testing
#!/usr/bin/python3
import jmespath
s = {'111': {'name': 'john', 'exp': '1'},
'222': {'name': 'mia', 'exp': '1'}}
# '333': {'name': 'john', 'exp': '1'}}
l1 = [{'key': k, 'value': v} for k, v in s.items()]
print(l1)
l2 = jmespath.search('[?value.name == `john`]', l1)
print(l2)
s2 = dict([[i['key'], i['value']] for i in l2])
print(s2)
Since you cannot preserve keys in JMESPath when doing an object projection, and that you will have to resort to a loop to have a JSON structure that will allow you to have your desired output see the other answer, the best will probably be to let JMESPath aside for your use case and achieve it with a list comprehension:
Given:
s = {
'111': {'name': 'john', 'exp': '1'},
'222': {'name': 'mia', 'exp': '1'},
}
print([
{key: value}
for key, value in s.items()
if value['name'] == 'john'
])
This yields the expect:
[{'111': {'name': 'john', 'exp': '1'}}]
My two dictionaries look like this
list1 = [{'time': '2020', 'name': 'one', 'address': '15423'},{'time': '2021', 'name': 'two', 'address': '8548305'}]
list2 = [{'to': '15423', 'value': '5'}, {'to': '8548305', 'value': '90'}, {'to': '123', 'value': '5'}]
I want my final list of dictionaries to look like this. Also, I don't want to consider dictionary if it doesn't have a match from another list of dictionaries
list3 = [{'time': '2020', 'name': 'one', 'address': '15423', 'value': '5'}, {'time': '2021', 'name': 'two', 'address': '8548305', 'value': '90'}]
Here is what I tried
[lst1.update(lst2) for lst1, lst2 in zip(list1, list2)]
But I don't know how can I group by address.
Appreciate your help
You are on the right track. You can do something like:
[{**lst1 , **lst2} for lst1 in list1 for lst2 in list2 if lst1["address"] == lst2["address"]]
Output
[{'address': '15423', 'name': 'one', 'time': '2020', 'value': '5'},
{'address': '8548305', 'name': 'two', 'time': '2021', 'value': '90'}]
Note that, list comprehension might not be the best idea, since it lacks break and continue commands. Maybe it's more efficient to something like:
for index,lst1 in enumerate(list1):
for lst2 in list2:
if lst2["address"] == lst1["address"]:
list1[index]["value"] = lst2["value"]
break
list1
Output
[{'address': '15423', 'name': 'one', 'time': '2020', 'value': '5'},
{'address': '8548305', 'name': 'two', 'time': '2021', 'value': '90'}]
dictss = [{'country': 'USA'},
{'country': 'USA'},
{'name': 'stuart',
'country': 'USA'},
{'name': 'tom',
'country': 'USA'}]
If key name is not there then have to remove the dicts which does have the key name
expected output:
[{'name': 'stuart',
'country': 'USA'},
{'name': 'tom',
'country': 'USA'}]
Simply use a list comprehension with a filter of if 'name' in d.keys():
dictss = [d for d in dictss if 'name' in d.keys()]
print(dictss)
Output:
[{'name': 'stuart', 'country': 'USA'}, {'name': 'tom', 'country': 'USA'}]
You could just consider the dictionaries that have name as a key using a list comprehension
>>> res = [d for d in dictss if 'name' in d]
>>> res
[{'name': 'stuart', 'country': 'USA'}, {'name': 'tom', 'country': 'USA'}]
Try it:
[i for i in dictss if "name" in i]
Assuming a list of dictionaries with unequal length, what's the best way to make them equal length i.e. for the missing key-value, add key but with value set to empty string or null:
lst = [
{'id': '123', 'name': 'john'},
{'id': '121', 'name': 'jane'},
{'id': '121'},
{'name': 'mary'}
]
to become:
lst = [
{'id': '123', 'name': 'john'},
{'id': '121', 'name': 'jane'},
{'id': '121', 'name': ''},
{'id': '', 'name': 'mary'}
]
The only way I can think of is converting to pandas dataframe then back to dict:
pd.DataFrame(lst).to_dict(orient='records')
Finding all the keys requires a full initial pass of the data:
>>> set().union(*lst)
{'id', 'name'}
Now iterate the dicts and set default for each key:
keys = set().union(*lst)
for d in lst:
for k in keys:
d.setdefault(k, '')
You could use colleections.ChainMap to get all the keys:
>>> lst = [
... {'id': '123', 'name': 'john'},
... {'id': '121', 'name': 'jane'},
... {'id': '121'},
... {'name': 'mary'}
... ]
>>>
>>> from collections import ChainMap
>>>
>>> for k in ChainMap(*lst):
... for d in lst:
... _ = d.setdefault(k, '')
...
>>> lst
[{'id': '123', 'name': 'john'}, {'id': '121', 'name': 'jane'}, {'id': '121', 'name': ''}, {'name': 'mary', 'id': ''}]
Try using this snippet
lst = [
{'id': '123', 'name': 'john'},
{'id': '121', 'name': 'jane'},
{'id': '121'},
{'name': 'mary'}
]
for data in lst:
if "name" not in data:
data["name"] = ""
if "id" not in data:
data["id"] = ""
print(lst)
Here's one way (Python 3.5+).
>>> all_keys = set(key for d in lst for key in d)
>>> [{**dict.fromkeys(all_keys, ''), **d} for d in lst]
[{'id': '123', 'name': 'john'}, {'id': '121', 'name': 'jane'}, {'id': '121', 'name': ''}, {'id': '', 'name': 'mary'}]
(Note that the unpacking order is critical here, you must unpack d after the dictionary with the default values in order to override the default values.)
I have 2 lists with the following data structure:
array1:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
array2:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
I want to have an array3 with union of the above 2 lists
array3:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
How can I do that in Python?
They are lists, not arrays, but here is a solution:
a1 = [{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
a2 = [{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
a3 = a1 + [i for i in a2 if i not in a1]
the value of a3 will be
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'},
{'student': {'name': 'rst'}, 'address': 'add_rst'}]
If you want your resulting list to be sorted (by student name, for example), simply use sort method:
a3.sort(key=lambda x: x["student"]["name"])
and the resulting a3 would be:
[{'student': {'name': 'abc'}, 'address': 'add_abc'},
{'student': {'name': 'rst'}, 'address': 'add_rst'},
{'student': {'name': 'xyz'}, 'address': 'add_xyz'}]
Normally you would simply use set(a1 + a2) but the problem here is that you have dicts inside your lists and they are unhashable, therefore they cannot be set items.