How to convert a list of dictionaries to a list?
Here is what I have:
{
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]}
Here is what I want it to look like:
({'6953':'https://address-jbr.ofp.ae','6967':'https://plots.ae','6970':'https://dubai-creek-harbour.ofp.ae'})
This is indeed very straightforward:
data = {
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]
}
Then:
data_list = [{x["ID"]: x["VALUE"]} for x in data["sources"]]
Which is the same as:
data_list = []
for x in data["sources"]:
data_list.append({
x["ID"]: x["VALUE"]
})
EDIT: You said convert to a "list" in the question and that confused me. Then this is what you want:
data_dict = {x["ID"]: x["VALUE"] for x in data["sources"]}
Which is the same as:
data_dict = {}
for x in data["sources"]:
data_dict[x["ID"]] = x["VALUE"]
P.S. Seems like you're asking for answers to your course assignments or something here, which is not what this place is for.
A solution using pandas
import pandas as pd
data = {
"sources": [
{"ID": "6953", "VALUE": "https://address-jbr.ofp.ae"},
{"ID": "6967", "VALUE": "https://plots.ae"},
{"ID": "6970", "VALUE": "https://dubai-creek-harbour.ofp.ae"},
]
}
a = pd.DataFrame.from_dict(data["sources"])
print(a.set_index("ID").T.to_dict(orient="records"))
outputs to:
[{'6953': 'https://address-jbr.ofp.ae', '6967': 'https://plots.ae', '6970': 'https://dubai-creek-harbour.ofp.ae'}]
this should work.
Dict = {
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]}
# Store all the keys here
value_LIST = []
for item_of_list in Dict["sources"]:
for key, value in item_of_list.items():
value_LIST.append(value)
Related
I am new in python and I am trying to list comprehsion my list dictionaries.
I have a serialized response in dictionaries inside list like :-
[
{
"data": {
"id": 61,
"title": "First"
},
"type": "like"
},
{
"data": {
"id": 62,
"title": "Seven"
},
"type": "like"
},
{
"data": {
"id": 103,
"title": "Third",
},
"type": "dislike"
},
{
"data": {
"id": 7,
"title": "Fifth",
},
"type": "dislike"
}
]
Multiple dictionaries with same type key are inside the list and I am trying to merge dictionaries into one list which have same keys.
I am trying to get like :-
[
{
"like": [
{
"id": 61,
"title": "First"},
{
"id": 62,
"title": "Second"
}
],
},
{
"dislike": [
{
"id": 103,
"title": "Third"
},
{
"id": 7,
"title": "Fifth"
}
],
},
]
I have tried using set() and union()
def comprehsion_method(list_dict):
converted_list = {
k : [d.get(k) for d in list_dict if k in d]
for k in set().union(*list_dict)
}
return converted_list
but This method merged all the data keys into one and all the type keys into one like :-
{
"data": [
{
"id":61,
"title": "First"
},
{
"id":62,
"title": "Second"
},
{
"id":103,
"title": "Third"
},
{
"id":7,
"title": "Seven"
},
],
"type": [
"like",
"like",
"dislike",
"dislike"
]
}
I have many times but it is still not working. Any help would be much Appreicated.
With comprehensions, especially complicated ones, it is best to start by writing out the explicit loop. In this case, you need something like:
new = {"like": [], "dislike": []}
for item in data:
if item["type"] == "like":
new["like"].append(item["data"])
else:
new["dislike"].append(item["data"])
print(new)
At this stage it is apparent that this function is trying to aggregate to a series of lists - something which comprehensions aren't designed to do. While it might be possible to convert it into a list comprehension, it will likely be relatively complex and less readable than the above code - so in this case I would leave it as is.
I have a Python List like this:
myList = [
{
"key": 1,
"date": "2020-01-02"
},
{
"key": 2,
"date": "2020-02-02"
},
{
"key": 3,
"date": "2020-01-03"
},
{
"key": 4,
"date": "2020-01-02"
},
{
"key": 5,
"date": "2020-02-02"
},
]
Now I want to split the array based on the property "date". I want my list to look like this
myList = [
[
{
"key": 1,
"date": "2020-01-02"
},
{
"key": 4,
"date": "2020-01-02"
},
],
[
{
"key": 2,
"date": "2020-02-02"
},
{
"key": 5,
"date": "2020-02-02"
},
],
[
{
"key": 3,
"date": "2020-01-03"
},
]
]
So I want a new array for each specific date in the current list. Can someone help me to achieve that?
d={}
for i in range(len(myList)):
d.setdefault(myList[i]['date'], []).append(i)
myList = [ [myList[i] for i in v] for k,v in d.items() ] # replace the original `myList` following PO behavior.
Logic:
You want to group the data based on 'date' that means you need a dictionary data structure. The rest are just implementation details.
I have a JSON file and that is a nested JSON. I would like to remove duplicates based on two keys.
JSON example:
"books": [
{
"id": "1",
"story": {
"title": "Lonely lion"
},
"description": [
{
"release": false,
"author": [
{
"name": "John",
"main": 1
},
{
"name": "Jeroge",
"main": 0
},
{
"name": "Peter",
"main": 0
}
]
}
]
},
{
"id": "2",
"story": {
"title": "Lonely lion"
},
"description": [
{
"release": false,
"author": [
{
"name": "Jeroge",
"main": 1
},
{
"name": "Peter",
"main": 0
},
{
"name": "John",
"main": 0
}
]
}
]
},
{
"id": "3",
"story": {
"title": "Lonely lion"
},
"description": [
{
"release": false,
"author": [
{
"name": "John",
"main": 1
},
{
"name": "Jeroge",
"main": 0
}
]
}
]
}
]
Here I try to match the title and author name. For example, for id 1 and id 2 are duplicates( as the title is same and author names are also same(the author sequence doesn't matter and no need to consider the main attributes). So, in the output JSON only id:1 or id:2 will remain with id:3. In the final output I need two file.
Output_JSON:
"books": [
{
"id": "1",
"story": {
"title": "Lonely lion"
},
"description": [
{
"release": false,
"author": [
{
"name": "John",
"main": 1
},
{
"name": "Jeroge",
"main": 0
},
{
"name": "Peter",
"main": 0
}
]
}
]
},
{
"id": "3",
"story": {
"title": "Lonely lion"
},
"description": [
{
"release": false,
"author": [
{
"name": "John",
"main": 1
},
{
"name": "Jeroge",
"main": 0
}
]
}
]
}
]
duplicatedID.csv:
1-2
The following method I tried but it is not giving correct results:
list= []
duplicate_Id = []
for data in (json_data['books'])[:]:
elements= []
id = data['id']
title = data['story']['title']
elements.append(title)
for i in (data['description'][0]['author']):
name = (i['name'])
elements.append(name)
if not list:
list.append(elements)
else:
for j in list:
if set(elements) == set(j):
duplicate_Id.append(id)
elements = []
The general idea is to:
Get the groups identified by some function that collects duplicates.
Then return the first entry of each group, ensuring no duplicates.
Define the key function as the sorted list of authors and. As the list of authors is by definition the unique key, but may appear in any order.
import json
from itertools import groupby
j = json.load(books)
def transform(books):
groups = [list(group) for _, group in groupby(books, key=getAuthors)]
return [group[0] for group in groups]
def getAuthors(book):
authors = book['description'][0]['author']
return sorted([author['name'] for author in authors])
print(transform(j['books']))
If we wanted to get the duplicates, then we do the same computation, but return any sublist with length > 1 as this is by our definition duplicated data.
def transform(books):
groups = [list(group) for _, group in groupby(books, key=getAuthors)]
return [group for group in groups if len(group) > 1]
Where j['books'] is the JSON you gave enclosed in an object.
I am try to use python to extract value, but I found a weird result.
Following is part of my json variable
"weatherElement": [
{
"elementName": "ELEV",
"elementValue": {
"value": "20.0"
}
},
{
"elementName": "TEMP",
"elementValue": {
"value": "25.0"
}
},
{
"elementName": "D_TNT",
"elementValue": {
"value": "2019-11-22T02:10:00+08:00"
}
}
],
and following code is correct for getting value 25.0 which is temperature
for unit in data['records']['location']:
# print(type(unit)) <---- output <dict>
if unit['stationId'] == 'C0V490':
for wea_unit in unit['weatherElement']: # unit['weatherElement'] is list
if wea_unit['elementName'] == 'TEMP':
print(type(wea_unit['elementValue'])) # is str
return wea_unit['elementValue']
My question is why type(wea_unit['elementValue']) is str?
I think it should be dict and I should use wea_unit['elementValue']['value'] to get '25.0', but it is wrong. Are there anyone know what mistake I made? Thanks!
edit:
following is example code which can run directly
import json
def parse_json_to_dataframe(data):
for unit in data['records']['location']:
# print(type(unit))
if unit['stationId'] == 'C0A560':
for wea_unit in unit['weatherElement']: # unit['weatherElement'] is list
if wea_unit['elementName'] == 'TEMP':
print(type(wea_unit['elementValue']))
return wea_unit['elementValue']
v = {"success":"true",
"result": {"resource_id": "O-A0001-001",
"fields": [{"id": "lat", "type": "Double"},
{"id": "lon", "type": "Double"},
{"id": "locationName", "type": "String"},
{"id": "stationId", "type": "String"},
{"id": "description", "type": "String"},
{"id": "elementName", "type": "String"},
{"id": "elementValue", "type": "Double"},
{"id": "parameterName", "type": "String"},
{"id": "parameterValue", "type": "String"}]}, # result end
"records": {"location": [{"lat": "24.778333",
"lon": "121.494583",
"locationName": "福山",
"stationId": "C0A560",
"time": {"obsTime": "2019-11-22 22:00:00"},
"weatherElement": [
{"elementName": "ELEV", "elementValue": "405.0"},
{"elementName": "WDIR", "elementValue": "0"},
{"elementName": "WDSD", "elementValue": "0.0"},
{"elementName": "TEMP", "elementValue": "19.6"}],
"parameter": [
{"parameterName": "CITY_SN", "parameterValue": "06"},
{"parameterName": "TOWN_SN", "parameterValue": "061"}]}]}}
temp = parse_json_to_dataframe(v)
print(temp)
All,
I am trying to change the way some json looks by going through and formatting it in the following way:
1. flatten all of the fields lists
2. Then remove the fields lists and replace them with the name : flatten list
Example:
{
"name": "",
"fields": [{
"name": "keys",
"fields": [{
"node-name": "0/0/CPU0"
},
{
"interface-name": "TenGigE0/0/0/47"
},
{
"device-id": "ASR9K-H1902.corp.cisco.com"
}
]
},
{
"name": "content",
"fields": [{
"name": "lldp-neighbor",
"fields": [{
"receiving-interface-name": "TenGigE0/0/0/47"
},
{
"receiving-parent-interface-name": "Bundle-Ether403"
},
{
"device-id": "ASR9K-H1902.corp.cisco.com"
},
{
"chassis-id": "78ba.f975.a64f"
},
{
"port-id-detail": "Te0/1/0/4/0"
},
{
"header-version": 0
},
{
"hold-time": 120
},
{
"enabled-capabilities": "R"
},
{
"platform": ""
}
]
}]
}
]
}
Would turn into:
{
"": [{
"keys": [{
"node-name": "0/0/CPU0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
}]
},
{
"content": [{
"lldp-neighbor": [{
"receiving-interface-name": "TenGigE0/0/0/47",
"receiving-parent-interface-name": "Bundle-Ether403",
"device-id": "ASR9K-H1902.corp.cisco.com",
"chassis-id": "78ba.f975.a64f",
"port-id-detail": "Te0/1/0/4/0",
"header-version": 0,
"hold-time": 120,
"enabled-capabilities": "R",
"platform": ""
}]
}]
}
]
}
I have tried the following to get the list flattened:
def _flatten_fields(self, fields_list):
c = {}
for b in [d for d in fields_list if bool(d)]:
c.update(b)
return c
This seems to work but I can't figure out a way to get into the sub levels using recursion, I am saving all flatten lists and names into a new dictionary, is there a way to do it by just manipulating the original dictionary?
This worked on the example you provided:
import json
def flatten(data):
result = dict()
if isinstance(data, dict):
if 'name' in data:
name = data['name']
result[name] = flatten(data['fields'])
else:
key = data.keys()[0]
value = data.values()[0]
result[key] = value
else:
for entry in data:
result.update(flatten(entry))
return result
print json.dumps(flatten(data), indent=4)
Output
{
"": {
"keys": {
"node-name": "0/0/CPU0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
},
"content": {
"lldp-neighbor": {
"receiving-interface-name": "TenGigE0/0/0/47",
"receiving-parent-interface-name": "Bundle-Ether403",
"header-version": 0,
"port-id-detail": "Te0/1/0/4/0",
"chassis-id": "78ba.f975.a64f",
"platform": "",
"device-id": "ASR9K-H1902.corp.cisco.com",
"hold-time": 120,
"enabled-capabilities": "R"
}
}
}
}
It doesn't have the extra list layers shown in your expected output, but I don't think you want those.
This worked on the example you provided:
def flatten_fields(fields_list):
c = {}
for item in fields_list:
for key in item:
if key == "fields":
c[item["name"]] = flatten_fields(item["fields"])
elif key != "name":
c[key] = item[key]
break
return [c]
But it works on a list of dictionaries, so you should call it like flatten_fields([data])[0].
The output is:
{
"": [{
"keys": [{
"node-name": "0/0/CP0",
"interface-name": "TenGigE0/0/0/47",
"device-id": "ASR9K-H1902.corp.cisco.com"
}],
"content": [{
"lldp-neighbor": [{
"chassis-id": "78ba.f975.a64f",
"receiving-parent-interface-name": "Bndle-Ether403",
"enabled-capabilities": "R",
"device-id": "ASR9K-H1902.corp.cisco.com",
"hold-time": 120,
"receiving-interface-name": "TenGigE0/0/0/47",
"platform": "",
"header-version": 0,
"port-id-detail": "Te0/1/0/4/0"
}]
}]
}]
}