I have a config.json file, which contains an array of organisations:
config.json
{
"organisations": [
{ "displayName" : "org1", "bucketName" : "org1_bucket" },
{ "displayName" : "org2", "bucketName" : "org2_bucket" },
{ "displayName" : "org3", "bucketName" : "org3_bucket" }
]
}
How can I get an array of all organisation names?
This is what I have tried:
from python_json_config import ConfigBuilder
def read_config():
builder = ConfigBuilder()
org_array = builder.parse_config('config.json')
# return all firstNames in org_array
import json
def read_config():
display_names = []
with open('yourfilename.json', 'r', encoding="utf-8") as file:
orgs = json.load(file)
display_names = [ o["displayName"] for o in orgs["organizations"] ]
return display_names
Also, we don't have any way to know what happens with ConfigBuilder or builder.parse_config since we don't have access to that code, so sorry to not take into account your example
a = {
"organisations": [
{ "displayName" : "org1", "bucketName" : "org1_bucket" },
{ "displayName" : "org2", "bucketName" : "org2_bucket" },
{ "displayName" : "org3", "bucketName" : "org3_bucket" }
]
}
print([i["displayName"] for i in a["organisations"]])
Output:
['org1', 'org2', 'org3']
Use list comprehension, it's very easy. In order to read a json file.
import json
data = json.load(open("config.json"))
Use lambda with map to get array of only organizations names
>>> list(map(lambda i:i['displayName'],x['organisations']))
>>> ['org1', 'org2', 'org3']
If you want to read json data from file into dictionary you can achieve this as following.
import json
with open('config.json') as json_file:
data = json.load(json_file)
org_array = list(map(lambda i:i['displayName'],data['organisations']))
Related
Below is the json with array of elements. How to get all the name values in a array? Is there a simplar way of doing it without for loop.
import json
from unicodedata import name
# Define json variable
jsondata = """[
{
"name":"Pen",
"unit_price":5
},
{
"name":"Eraser",
"unit_price":3
},
{
"name":"Pencil",
"unit_price":10
},
{
"name":"White paper",
"unit_price":15
}
]"""
# load the json data
items = json.loads(jsondata)
namelist = []
for keyval in items:
namelist.append((keyval['name']))
print(namelist)
names = [it['name'] for it in items]
I have a nested json file which I got from json.
I am trying to convert it in to csv through python code.
I tried all the possible way to convert it to csv but couldn't succeed.
I also followed previous question and solution but didn't work for me.
My json format is
{
"d1" : ("value1"),
"d2" : (value2-int),
"d3" : [
{
"sub-d1" : sub-value1(int),
"sub-d2" : sub-value2(int),
"sub-d3" : sub-value3(int),
"sub-d4" : [
{
"sub-sub-d1" : "sub-sub-value3",
"sub-sub-d2" : sub-value3(int)
},
{
"sub-sub-d1" : sub-sub-value3(int),
"sub-sub-d2" : "sub-sub-value3"}
]
],
"sub-d5" : "sub-value4",
"sub-d6" : "sub-value5"
}
],
"d4" : "value3",
"d5" : "value4",
"d6" : "value5,
"d7" : "value6"
}
{ another entry with same pattern..and so on}
Some of the value and sub value has integers and str + int.
What I tried
import json
import csv
import requests
with open('./data/inverter.json', 'r') as myfile:
json_data = myfile.read()
def get_leaves(item, key=None):
if isinstance(item, dict):
leaves = {}
for i in item.keys():
leaves.update(get_leaves(item[i], i))
return leaves
elif isinstance(item, list):
leaves = {}
for i in item:
leaves.update(get_leaves(i, key))
return leaves
else:
return {key : item}
# First parse all entries to get the complete fieldname list
fieldnames = set()
for entry in json_data:
fieldnames.update(get_leaves(entry).keys())
with open('output.csv', 'w', newline='') as f_output:
csv_output = csv.DictWriter(f_output, fieldnames=sorted(fieldnames))
csv_output.writeheader()
csv_output.writerows(get_leaves(entry) for entry in json_data)
This one saves all my data in single column with split values.
I tried to use :
https://github.com/vinay20045/json-to-csv.git
but this also didn't work.
I also tried to parse and do simple trick with following code:
with open("./data/inverter.json") as data_file:
data = data_file.read()
#print(data)
data_content = json.loads(data)
print(data_content)
but it throws an error : 'JSONDecodeError: Expecting value: line 2 column 13 (char 15)'
Can any one help me to convert my nested json to csv ?
It would be appreciated.
Thank you
It looks like the NumberInt(234234) issue you describe was a bug in MongoDB: how to export mongodb without any wrapping with NumberInt(...)?
If you cannot fix it by upgrading MongoDB, I can recommend preprocessing the data with regular expressions and parsing it as regular JSON after that.
For the sake of example, let's say you've got "test.json" that looks like this, which is valid except for the NumberInt(...) stuff:
{
"d1" : "value1",
"d2" : NumberInt(1234),
"d3" : [
{
"sub-d1" : 123,
"sub-d2" : 123,
"sub-d3" : 123,
"sub-d4" : [
{
"sub-sub-d1" : "sub-sub-value3",
"sub-sub-d2" : NumberInt(123)
},
{
"sub-sub-d1" : 43242,
"sub-sub-d2" : "sub-sub-value3"
}
]
}
],
"d4" : "value3",
"d5" : "value4",
"d6" : "value5",
"d7" : "value6"
}
You could import this into Python as follows:
import re
import json
with open("test.json") as f:
data = f.read()
# This regular expression finds/replaces the NumberInt bits with just the contents
fixed_data = re.sub(r"NumberInt\((\d+)\)", r"\1", data)
loaded_data = json.loads(fixed_data)
print(json.dumps(loaded_data, indent=4))
Im getting the json data from a file:
"students": [
{
"name" : "ben",
"age" : 15
},
{
"name" : "sam",
"age" : 14
}
]
}
here's my initial code:
def get_names():
students = open('students.json')
data = json.load(students)
I want to get the values of all names
[ben,sam]
you need to extract the names from the students list.
data = {"students": [
{
"name" : "ben",
"age" : 15
},
{
"name" : "sam",
"age" : 14
}
]
}
names = [each_student['name'] for each_student in data['students']]
print(names) #['ben', 'sam']
Try using a list comprehension:
>>> [dct['name'] for dct in data['students']]
['ben', 'sam']
>>>
import json
with open('./students.json', 'r') as students_file:
students_content = json.load(students_file)
print([student['name'] for student in students_content['students']]) # ['ben', 'sam']
JSON's load function from the docs:
Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object...
The JSON file in students.json will look like:
{
"students": [
{
"name" : "ben",
"age" : 15
},
{
"name" : "sam",
"age" : 14
}
]
}
The JSON load function can then be used to deserialize this JSON object in the file to a Python dictionary:
import json
# use with context manager to ensure the file closes properly
with open('students.json', 'rb')as students_fp:
data = json.load(students_fp)
print(type(data)) # dict i.e. a Python dictionary
# list comprehension to take the name of each student
names = [student['name'] for student in data['students']]
Where names now contains the desired:
["ben", "sam"]
I have some JSON file:
{
"cis" : [ {
"ucmdbId" : "835cfedfaabc32a1358b322ff3bae056",
"type" : "running_software",
"properties" : {
"display_label" : "jboss (site1.ru)"
}
}, {
"ucmdbId" : "7ef9f21c132c12b3d8d2af0964cc5970",
"type" : "node",
"properties" : {
"display_label" : "site2.ru"
}
} ],
"relations" : [ {
"ucmdbId" : "80c42edbe32fbb4c25621756ec9e09d2",
"type" : "compound_f",
"properties" : null,
"end1Id" : "23e30baf2320a3274d0aa1e7f56cdaef",
"end2Id" : "15af0ba134327d32a0c5c72450e63fcd"
}, {
"ucmdbId" : "7fe9fb15d4462d1212aeee4aef2f32b4",
"type" : "compound_f",
"properties" : null,
"end1Id" : "23e30baf2320a3274d0aa327f56cdaef",
"end2Id" : "9232dd2621b814da632932e8cd33ffc8"
} ]
}
I only need the cis array. So this is what I need to parse:
[{
"ucmdbId" : "835cfedfaabc32a1358b322ff3bae056",
"type" : "running_software",
"display_label" : "jboss (site1.ru)"
}, {
"ucmdbId" : "7ef9f21c132c12b3d8d2af0964cc5970",
"type" : "node",
"display_label" : "site2.ru"
}]
Python script:
#!/usr/bin/python
import sys
import os
import tablib
import pandas as pd
import json
from pandas.io.json import json_normalize
f = open('/home/nik/test.json', 'rw')
jsonArray = f.read()
f.close
data = json.dumps(json.loads(jsonArray)['cis'])
jsonResult = pd.read_json(data)
array = json.loads(jsonArray)
print jsonArray
jsonResult.to_excel('/home/nik/output.xlsx', sheet_name='Sheet1')
But how can I get key parameters? I try to use:
print data['type'].keys()
print data['type']
But it gives me error:
AttributeError: 'str' object has no attribute 'keys'
How can I get the proper JSON format?
Update. Solution:
Thanks, it works. My complete code to export JSON into xlsx file:
#!/usr/bin/python
import subprocess
import sys
import os
import tablib
import pandas as pd
import json
import glob
import string
path = '/home/nik/json'
for jsonfile in glob.glob(os.path.join(path, '*.json')):
#jsonfile = '/home/nik/test.json'
with open(jsonfile) as data_file:
data = json.load(data_file)
JSON = '[{ \n'
for index, item in enumerate(data['cis']):
ucmdbId = (item['ucmdbId'])
type = (item['type'])
display_label = (item['properties']['display_label'])
Text1 = ' \"ucmdbId\" : \"%s\",' %(ucmdbId)
Text2 = ' \"type\" : \"%s\",' %(type)
Text3 = ' \"display_label\" : \"%s\",' %(display_label)
if index==(len(data['cis'])-1):
End = '}]'
else:
End = '}, {'
JSON += Text3+'\n'+Text2+'\n'+Text1+'\n'+End+'\n'
JSON = JSON.translate({ord(c): None for c in '\/'})
jsonResult = pd.read_json(JSON)
jsonResult = jsonResult.sort_values(by='type')
jsonResult.to_excel(jsonfile+'.xlsx', sheet_name='Object monitoring', index=False)
import json
from pprint import pprint
jsonfile = 'C:\\temp\\temp.json' # path to your json file
with open(jsonfile) as data_file:
data = json.load(data_file)
pprint(data['cis'])
The above will give you just the cis array.
Below is a more granular output
for item in data['cis']:
ucmdbId = (item['ucmdbId'])
type = (item['type'])
display_label = (item['properties']['display_label'])
print(ucmdbId)
print(type)
print(display_label)
If you want it with key labels then use
for item in data['cis']:
ucmdbId = (item['ucmdbId'])
type = (item['type'])
display_label = (item['properties']['display_label'])
print('ucmdbId:{}'.format(ucmdbId))
print('type:{}'.format(type))
print('display_label:{}'.format(display_label))
I am attempting to parse some JSON that I am receiving from a RESTful API, but I am having trouble accessing the data in Python because it appears that there is an empty property name.
A sample of the JSON returned:
{
"extractorData" : {
"url" : "RetreivedDataURL",
"resourceId" : "e38e1a7dd8f23dffbc77baf2d14ee500",
"data" : [ {
"group" : [ {
"CaseNumber" : [ {
"text" : "PO-1994-1350",
"href" : "http://www.referenceURL.net"
} ],
"DateFiled" : [ {
"text" : "03/11/1994"
} ],
"CaseDescription" : [ {
"text" : "Mary v. JONES"
} ],
"FoundParty" : [ {
"text" : "Lastname, MARY BETH (Plaintiff)"
} ]
}, {
"CaseNumber" : [ {
"text" : "NP-1998-2194",
"href" : "http://www.referenceURL.net"
}, {
"text" : "FD-1998-2310",
"href" : "http://www.referenceURL.net"
} ],
"DateFiled" : [ {
"text" : "08/13/1993"
}, {
"text" : "06/02/1998"
} ],
"CaseDescription" : [ {
"text" : "IN RE: NOTARY PUBLIC VS REDACTED"
}, {
"text" : "REDACTED"
} ],
"FoundParty" : [ {
"text" : "Lastname, MARY H (Plaintiff)"
}, {
"text" : "Lastname, MARY BETH (Defendant)"
} ]
} ]
} ]
And the Python code I am attempting to use
import requests
import json
FirstName = raw_input("Please Enter First name: ")
LastName = raw_input("Please Enter Last Name: ")
with requests.Session() as c:
url = ('https://www.requestURL.net/?name={}&lastname={}').format(LastName, FirstName)
page = c.get(url)
data = page.content
theJSON = json.loads(data)
def myprint(d):
stack = d.items()
while stack:
k, v = stack.pop()
if isinstance(v, dict):
stack.extend(v.iteritems())
else:
print("%s: %s" % (k, v))
print myprint(theJSON["extractorData"]["data"]["group"])
I get the error:
TypeError: list indices must be integers, not str
I am new to parsing Python and more than simple python in general so excuse my ignorance. But what leads me to believe that it is an empty property is that when I use a tool to view the JSON visually online, I get empty brackets, Like so:
Any help parsing this data into text would be of great help.
EDIT: Now I am able to reference a certain node with this code:
for d in group:
print group[0]['CaseNumber'][0]["text"]
But now how can I iterate over all the dictionaries listed in the group property to list all the nodes labeled "CaseNumber" because it should exist in every one of them. e.g
print group[0]['CaseNumber'][0]["text"]
then
for d in group:
print group[1]['CaseNumber'][0]["text"]
and so on and so forth. Perhaps incrementing some sort of integer until it reaches the end? I am not quite sure.
If you look at json carefully the data key that you are accessing is actually a list, but data['group'] is trying to access it as if it were a dictionary, which is raising the TypeError.
To minify your json it is something like this
{
"extractorData": {
"url": "string",
"resourceId": "string",
"data": [{
"group": []
}]
}
}
So if you want to access group, you should first retrieve data which is a list.
data = sample['extractorData']['data']
then you can iterate over data and get group within it
for d in data:
group = d['group']
I hope this clarifies things a bit for you.