I've been trying to get this to work.
There are two different JSON lists that contain different keys.
list1
[
{
"name":"John",
"measurement": "5.11"
},
{
"name":"Kate",
"measurement": "5.6"
}
]
list2
[
{
"name":"John",
"characteristics": {
"height": [
"6.0"
]
}
},
{
"name":"Mike",
"characteristics": {
"height": [
"5.10"
]
}
}
]
code
for k in list2:
if v['name'] in [key['name'] for key in list1:
list1.append(k['measurement'])
The output I get is,
[{'name': 'John', 'characteristics': ['height': '6.0', 'age': 30}, '5.11']
Expected output
[{'name': 'John', 'characteristics': ['height': '5.11', 'age': 30}]
The loops go over the keys and if the key['name'] are equals in both lists,
then it proceeds into happening that specific values from the given key.
The only problem is that it is not working correctly for me. I just want to replace that value from height in characteristics with the one in measurement.
[EDIT]:
I made changes to the json. It should be correct now. Basically, height is an array.
JSON you posted is invalid so I fixed it little bit. Following code should work:
list_1 = [
{
"name": "John",
"measurement": "5.11"
},
{
"name": "Kate",
"measurement": "5.6"
}
]
list_2 = [
{
"name": "John",
"characteristics": {
"height": "6.0",
"age": 30
}
},
{
"name": "Mike",
"characteristics": {
"height": "5.10",
"age": 25
}
}
]
result = []
for list_1_item in list_1:
single_obj = {}
for list_2_item in list_2:
if list_2_item['name'] == list_1_item['name']:
single_obj['name'] = list_1_item['name']
single_obj['characteristics'] = list_2_item['characteristics']
result.append(single_obj)
print(result)
This gives us following result:
[{'name': 'John', 'characteristics': {'height': '6.0', 'age': 30}}]
Related
dict_example = {
"abc": [
{
"name": "bcd",
"gender": "male",
"options": {
"emp_id": "a10734",
"address": "cfg",
"dept": "IT",
},
}
]
}
I have above dictionary and I need to add below values to options programmatically.
"desgn":"Engineer",
"project" : "xyz",
I need output in the below formart
dict_example = {
"abc": [
{
"name": "bcd",
"gender": "male",
"options": {
"emp_id": "a10734",
"address": "cfg",
"dept": "IT",
"desgn":"Engineer",
"project" : "xyz",
},
}
]
}
Can anyone help me with the above problem it will be great!
You can simply use the keys and indices to reach the item and add or change it:
dict_example['abc'][0]['options']['design'] = 'Engineer'
dict_example['abc'][0]['options']['project'] = 'xyz'
print(dict_example)
The output will be:
{'abc': [{'name': 'bcd', 'gender': 'male', 'options': {'emp_id': 'a10734', 'address': 'cfg', 'dept': 'IT', 'design': 'Engineer', 'project': 'xyz'}}]}
for an input json
[{
"Name": "John",
"Age": "23",
"Des": "SE"
},
{
"Name": "Rai",
"Age": "33",
"Des": "SSE"
},
{
"Name": "James",
"Age": "42",
"Des": "SE"
}
]
I want to filter out the json data where only "Des":"SE" is true
required output
[{
"Name": "John",
"Age": "23"
},
{
"Name": "James",
"Age": "42"
}
]
A list comprehension should do it:
out = [{'Name':d['Name'], 'Age':d['Age']} for d in lst if d['Des']=='SE']
Another way:
out = [d for d in lst if d.pop('Des')=='SE']
Output:
[{'Name': 'John', 'Age': '23'}, {'Name': 'James', 'Age': '42'}]
To make it more dynamic if each json has more elements:
import json
input_str = '[{"Name": "John", "Age": "23", "Des": "SE"}, {"Name": "Rai", "Age": "33", "Des": "SSE"}, {"Name": "James", "Age": "42", "Des": "SE"}]'
input_list = json.loads(input_str)
# If you already converted to a list of dicts, then you don't need the above
# Using pop here removes the key you are using to filter
output = [each for each in input_list if each.pop("Des") == "SE"]
using the json module, you can load a file using loads or a string using load. From there, it acts as a normal python list of dictionaries which you can iterate over and check the keys of. From there, you simply create a new list of dictionaries that match your desired pattern and remove the key you are no longer using. Example:
import json
jsonString = """[{
"Name": "John",
"Age": "23",
"Des": "SE"
},
{
"Name": "Rai",
"Age": "33",
"Des": "SSE"
},
{
"Name": "James",
"Age": "42",
"Des": "SE"
}
]"""
jsonList = json.loads(jsonString)
filteredList = []
def CheckDes(dataDict: dict):
if dataDict['Des'] == 'SE':
dataDict.pop('Des')
filteredList.append(dataDict)
print(jsonList)
"""
[
{
'Name': 'John',
'Age': '23',
'Des': 'SE'
},
{
'Name': 'Rai',
'Age': '33',
'Des': 'SSE'
},
{
'Name': 'James',
'Age': '42',
'Des': 'SE'
}
]"""
[CheckDes(dataDict) for dataDict in jsonList]
print(filteredList)
"""[
{
'Name': 'John',
'Age': '23'
},
{
'Name': 'James',
'Age': '42'
}
]
"""
This is primary flattened element, aka input data:
['a-ab-aba-abaa-abaaa', 'a-ab-aba-abab', 'a-ac-aca-acaa', 'a-ac-aca-acab']
This is the target data what I need, aka output data:
[
{
"title": "a",
"children": [
{
"title": "ab",
"children": [
{
"title": "aba",
"children": [
{
"title": "abaa",
"children": [
{
"title": "abaaa"
}
]
},
{
"title": "abab"
}
]
}
]
},
{
"title": "ac",
"children": [
{
"title": "aca",
"children": [
{
"title": "acaa"
},
{
"title": "acab"
}
]
}
]
}
]
}
]
I thought I can use deep-for-loop iteration to generate this json data, but it's so difficult, because num of level will bigger than 10. so I think for-loop can't do in this process, is there any algrithm or use a packaged code to implement a function to achieve this target?
I'm so grateful if you share your mindset, god bless you!
Here is a recursive solution using itertools. I dont know if this is efficient enough for your purpose, but it works. It works by transforming your list of strings into a list of lists, then dividing that into lists with the same first key, and then building the dict and repeating with the first key removed.
from itertools import groupby
from pprint import pprint
data = ['a-ab-aba-abaa-abaaa', 'a-ab-aba-abab', 'a-ac-aca-acaa', 'a-ac-aca-acab']
components = [x.split("-") for x in data]
def build_dict(component_list):
key = lambda x: x[0]
component_list = sorted(component_list, key=key)
# divide into lists with the same fist key
sublists = groupby(component_list, key)
result = []
for name, values in sublists:
value = {}
value["title"] = name
value["children"] = build_dict([x[1:] for x in values if x[1:]])
result.append(value)
return result
pprint(build_dict(components))
Output:
[{'children': [{'children': [{'children': [{'children': [{'children': [],
'title': 'abaaa'}],
'title': 'abaa'},
{'children': [], 'title': 'abab'}],
'title': 'aba'}],
'title': 'ab'},
{'children': [{'children': [{'children': [], 'title': 'acaa'},
{'children': [], 'title': 'acab'}],
'title': 'aca'}],
'title': 'ac'}],
'title': 'a'}]
To convert this dict to json you can use json.dumps from the json module. I hope my explanaition is clear.
Here is a start:
def populate_levels(dct, levels):
if levels:
if levels[0] not in dct:
dct[levels[0]] = {}
populate_levels(dct[levels[0]], levels[1:])
def create_final(dct):
final = []
for title in dct:
final.append({"title": title, "children": create_final(dct[title])})
return final
data = ['a-ab-aba-abaa-abaaa', 'a-ab-aba-abab', 'a-ac-aca-acaa', 'a-ac-aca-acab']
template = {}
for item in data:
populate_levels(template, item.split('-'))
final = create_final(template)
I couldn't see a clean way of doing it all at once so I created this in-between template dictionary. Right now if a 'node' has no children its corresponding dict will contain 'children': []
you can change this behavior in the create_final function if you like.
You can use collections.defaultdict:
from collections import defaultdict
def get_struct(d):
_d = defaultdict(list)
for a, *b in d:
_d[a].append(b)
return [{'title':a, 'children':get_struct(filter(None, b))} for a, b in _d.items()]
data = ['a-ab-aba-abaa-abaaa', 'a-ab-aba-abab', 'a-ac-aca-acaa', 'a-ac-aca-acab']
import json
print(json.dumps(get_struct([i.split('-') for i in data]), indent=4))
Output:
[
{
"title": "a",
"children": [
{
"title": "ab",
"children": [
{
"title": "aba",
"children": [
{
"title": "abaa",
"children": [
{
"title": "abaaa",
"children": []
}
]
},
{
"title": "abab",
"children": []
}
]
}
]
},
{
"title": "ac",
"children": [
{
"title": "aca",
"children": [
{
"title": "acaa",
"children": []
},
{
"title": "acab",
"children": []
}
]
}
]
}
]
}
]
enter image description hereI am trying to convert a very complex JSON into CSV, and now I have stuck somewhere in the middle. My JSON file is nested with a combination of many lists and dictionaries(dictionaries also have sub dictionaries)
while I am iterating through the complete JSON, I am getting two dictionaries from a for a loop. Now my problem is when I am looping through this set, to append keys(Zip1) and values(value) into my default dictionary which is set as null in the beginning, due to the limitation of dictionaries, I am able to extract only one value i.e. Zip1, 34567
{'type': 'Zip1', 'value': '12345'}
{'type': 'Zip1', 'value': '34567'}
fin_data={}
dict1 is the outcome of some for loop of my code and holds the value as
{'type': 'Zip1', 'value': '12345'}
{'type': 'Zip1', 'value': '34567'}
for key,value in dict1.items():
for data in value:
print(data)
fin_data.update(key:data['value'])
Is there any way, I can iterate through sets of the dictionaries of the dict1?
so that at the first iteration, I will copy data into CSV, and then in the second iteration, the other values to my CSV
The output I am getting is :
{Zip1:34567}
Actual Output is required as both values
Sample of my json, on which i am working, Need to extract data from all of the value attributes:
{
"updatedTime": 1562215101843,
"attributes": {
"ActiveFlag": [
{
"value": "Y"
}
],
"CountryCode": [
{
"value": "United States"
}
],
"LastName": [
{
"value": "Giers"
}
],
"MatchFirstNames": [
{
"value": "Morgan"
}
],
"Address": [
{
"value": {
"Zip": [
{
"value": {
"Zip5": [
{
"type": "Zip1",
"value": "12345"
}
]
}
}
],
"Country": [
{
"value": "United States"
}
]
}
},
{
"value": {
"City": [
{
"value": "Tempe"
}
],
"Zip": [
{
"value": {
"Zip5": [
{
"type": "Zip1",
"value": "85287"
}
]
}
}
]
}
}
]
}
}
Expected Result :
updatedTime, ActiveFlag, CountryCode, LastName, MatchFirstNames, Address_Zip_Zip5, Address_City, Address_Country
1562215101843,Y,United States,Giers,Morgan,12345,,United States
1562215101843,Y,United States,Giers,Morgan,85287,Tempe,
At the first step for each person accumulate all the zip codes in one line/list/record say space separated or in an array.
Then after everything extracted split row into several
How can I flatten a List of dictionary with nested dictonaries, say I have the following dict:
data = [
{ 'Name':'xyx',
'Age':22,
'EmpDetails':{'Salary':100,'Job':'Intern','Location':'TER'}
},
{ 'Name':'abc',
'Age':23,
'EmpDetails':{'JoinDate':'20110912','Salary':200,'Job':'Intern2','Location':'TER2'}
},
{'Name':'efg',
'Age':24,
'EmpDetails':{'JoinDate':'20110912','enddate':'20120912','Salary':300,'Job':'Intern3','Location':'TER3'}
}
]
i would need the EmpDetails Node removed and move its values one level up, like below
data = [
{ 'Name':'xyx','Age':22,'Salary':100,'Job':'Intern','Location':'TER'},
{ 'Name':'abc','Age':23,'JoinDate':'20110912','Salary':200,'Job':'Intern2','Location':'TER2'},
{'Name':'efg','Age':24,'JoinDate':'20110912','enddate':'20120912','Salary':300,'Job':'Intern3','Location':'TER3'}
]
i am this now using below, is there any faster way of doing this?
newlist = []
for d in data:
empdict ={}
for key, val in d.items():
if(key!='EmpDetails'):
empdict[key] = val
if(key=='EmpDetails'):
for key2, val2 in val.items():
empdict[key2] = val2
newlist.append(empdict)
This is one approach using dict.update and .pop
Ex:
data = [
{ 'Name':'xyx',
'Age':22,
'EmpDetails':{'Salary':100,'Job':'Intern','Location':'TER'}
},
{ 'Name':'abc',
'Age':23,
'EmpDetails':{'JoinDate':'20110912','Salary':200,'Job':'Intern2','Location':'TER2'}
},
{'Name':'efg',
'Age':24,
'EmpDetails':{'JoinDate':'20110912','enddate':'20120912','Salary':300,'Job':'Intern3','Location':'TER3'}
}
]
for i in data:
i.update(i.pop("EmpDetails"))
print(data)
Output:
[{'Age': 22, 'Job': 'Intern', 'Location': 'TER', 'Name': 'xyx', 'Salary': 100},
{'Age': 23,
'Job': 'Intern2',
'JoinDate': '20110912',
'Location': 'TER2',
'Name': 'abc',
'Salary': 200},
{'Age': 24,
'Job': 'Intern3',
'JoinDate': '20110912',
'Location': 'TER3',
'Name': 'efg',
'Salary': 300,
'enddate': '20120912'}]
One line method, maybe a little tricky.
data = [
{
"Name": "xyx",
"Age": 22,
"EmpDetails": {"Salary": 100, "Job": "Intern", "Location": "TER"},
},
{
"Name": "abc",
"Age": 23,
"EmpDetails": {
"JoinDate": "20110912",
"Salary": 200,
"Job": "Intern2",
"Location": "TER2",
},
},
{
"Name": "efg",
"Age": 24,
"EmpDetails": {
"JoinDate": "20110912",
"enddate": "20120912",
"Salary": 300,
"Job": "Intern3",
"Location": "TER3",
},
},
]
# only python3.5+
res = [{**item.pop("EmpDetails", {}), **item} for item in data]
I'd prefer using json_normalize() method from pandas library since it would be an elegant solution and had no effect on readability of your code.
Examples can bee seen here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.io.json.json_normalize.html