I have a very basic problem that I can't figure out. I keep getting a "End of file expected.json" when trying to write object data to a json file. I was wondering how I can fix that? I do it by writing in a for loop. Not sure how I can format.
This is the code in question
with open("data.json", "w") as outfile:
for x,y in structures.infrastructures.items():
outfile.write(Sector(x, y["Depended on by"],y["Depends on"], y["Sub sections"]).toJson())
and this is the output
{
"name": "Chemical",
"depended_on": [
"Critical Manufacturing",
"Nuclear Reactors, Waste, and Material Management",
"Commercial",
"Healthcare and Public Health",
"Food and Agriculture",
"Energy"
],
"depends_on": [
"Emergency Services",
"Energy",
"Food and Agriculture",
"Healthcare and Public Health",
"Information Technology",
"Nuclear Reactors, Waste, and Material Management",
"Transportation Systems",
"Water"
],
"sub_sections": [
"Chemical Plants",
"Chemical Refineries",
"Labs"
],
"Status": 0,
"Strain": 0
}{ -> this is where the error is
"name": "Commercial",
"depended_on": [
....
....
etc
This is my toJson method:
def toJson(self):
return json.dumps(self, default=lambda o: o.__dict__, indent=4)
But yeah how can I implement it where my object data is written in JSON format?
A valid json file can only contain one object. Collect your data into a list and write it with a single call, or simulate the format with your code.
You have to use a function to parse the .json
I hope this helps
https://scriptcrunch.com/parse-json-file-using-python/#:~:text=How%20To%20Parse%20JSON%20File%20Content%20Using%20Python,also%20loop%20through%20all%20the%20JSON%20objects.%20
Related
Suppose you are writing a python web client to access an API of an online supermarket. Given below are the API details.
Base URL = http://host1.open.uom.lk:8080
Write a python program to retrieve all the products from the API Server and print the total number of products currently stored in the server.
Hint: the json response will be of the following example format:
{
"message": "success",
"data": [
{
"id": 85,
"productName": "Araliya Basmathi Rice",
"description": "White Basmathi Rice imported from Pakistan. High-quality rice with extra fragrance. Organically grown.",
"category": "Rice",
"brand": "CIC",
"expiredDate": "2023.05.04",
"manufacturedDate": "2022.02.20",
"batchNumber": 324567,
"unitPrice": 1020,
"quantity": 200,
"createdDate": "2022.02.24"
},
{
"id": 86,
"productName": "Araliya Basmathi Rice",
"description": "White Basmathi Rice imported from Pakistan. High-quality rice with extra fragrance. Organically grown.",
"category": "Rice",
"brand": "CIC",
"expiredDate": "2023.05.04",
"manufacturedDate": "2022.02.20",
"batchNumber": 324567,
"unitPrice": 1020,
"quantity": 200,
"createdDate": "2022.02.24"
}
]
}
The Answer For The Above Question Is The Code Below
Thank You.
import requests
import json
BASE_URL = "http://host1.open.uom.lk:8080"
updated_entity = {
"productName":"Araliya Basmathi Rice",
"description":"White Basmathi Rice imported from Pakistan. High-quality rice with extra fragrance. Organically grown.",
"category":"Rice",
"brand":"Araliya",
"expiredDate":"2023.05.04",
"manufacturedDate":"2022.02.20",
"batchNumber":324567,
"unitPrice":1020,
"quantity":200,
"createdDate":"2022.02.24"
}
response = requests.put(f"{BASE_URL}/api/products/", json=updated_entity)
print(response.json())
Great! So it sounds like you were able to fetch the JSON response successfully.
Mind you, response_API is NOT the JSON, it is just a Response object. You need to call .json() or .text() on it to get the result you intend. Check this out: https://www.w3schools.com/python/ref_requests_response.asp
You’ll now need to parse the JSON string into a useable Python data structure.
Here is information for parsing JSON: https://www.w3schools.com/python/gloss_python_json_parse.asp
parsed = json.load(response_API.text())
JSON is parsed into a dictionary. Here you can access the “data” key, which is a list of products. We can simply get the length of a list.
len(parsed[“data”])
I have a pandas.dataframe named 'df' with the following format:
group_name
Positive_Sentiment
Negative_Sentiment
group1
helpful, great support
slow customer service, weak interface, bad management
I would like to convert this dataframe to a JSON file with the following format:
[{
"Group Name": "group1",
"Postive Sentiment": [
"helpful",
"great support"
],
"Negative Sentiment": [
"slow customer service",
"weak interface",
"bad management"
]
}
]
So far I have used this:
import json
b = []
for i in range(len(df)):
x={}
x['Group Name']=df.iloc[i]['group_name']
x['Positive Sentiment']= [df.iloc[i]['Positive_Sentiment']]
x['Negative Sentiment']= [df.iloc[i]['Negative_Sentiment']]
b.append(x)
##Export
with open('AnalysisResults.json', 'w') as f:
json.dump(b, f, indent = 2)
This results in:
[{
"Group Name": "group1",
"Postive Sentiment": [
"helpful,
great support"
],
"Negative Sentiment": [
"slow customer service,
weak interface,
bad UX"
]
}
]
You can see it is quite close. The crucial difference is the double-quotes around the ENTIRE contents of each row (e.g., "helpful, great support") instead of each comma-separated string in the row (e.g., "helpful", "great support"). I would like double-quotes around each string.
You can apply split(",") to your columns:
from io import StringIO
import pandas as pd
import json
inp = StringIO("""group_name Positive_Sentiment Negative_Sentiment
group1 helpful, great support slow customer service, weak interface, bad management
group2 great, good support interface meeeh, bad management""")
df = pd.read_csv(inp, sep="\s{2,}")
def split_and_strip(sentiment):
[x.strip() for x in sentiment.split(",")]
df["Positive_Sentiment"] = df["Positive_Sentiment"].apply(split_and_strip)
df["Negative_Sentiment"] = df["Negative_Sentiment"].apply(split_and_strip)
print(json.dumps(df.to_dict(orient="record"), indent=4))
# to save directly to a file:
with open("your_file.json", "w+") as f:
json.dump(df.to_dict(orient="record"), f, indent=4)
Output:
[
{
"group_name": "group1",
"Positive_Sentiment": [
"helpful",
"great support"
],
"Negative_Sentiment": [
"slow customer service",
"weak interface",
"bad management"
]
},
{
"group_name": "group2",
"Positive_Sentiment": [
"great",
"good support"
],
"Negative_Sentiment": [
"interface meeeh",
"bad management"
]
}
]
I have a large json file that I would like to split according to the key "metadata". One example of record is
{"text": "The primary outcome of the study was hospital mortality; secondary outcomes included ICU mortality and lengths of stay for hospital and ICU. ICU mortality was defined as survival of a patient at ultimate discharge from the ICU and hospital mortality was defined as survival at discharge or transfer from our hospital.", "label": "conclusion", "metadata": "18982114"}
There are many records in the json file where the key "metadata" is "18982114". How can I extract all of these records and store them into a separate json file? Ideally, I'm looking for a solution that includes no loading and looping over the file, otherwise it would be very cumbersome every time I query it. I think by using shell command maybe it's doable, but unfortunately I'm not an expert in shell commands...so I would highly appreciate a non-looping fast query solution, thx!
==========================================================================
here are some samples of the file (contains 5 records):
{"text": "Finally, after an emergency laparotomy, patients who received i.v. vasoactive drugs within the first 24 h on ICU were 3.9 times more likely to die (OR 3.85; 95% CI, 1.64 -9.02; P\u00bc0.002). No significant prognostic factors were determined by the model on day 2.", "label": "conclusion", "metadata": "18982114"}
{"text": "Kinetics ofA TP Binding to Normal and Myopathic", "label": "conclusion", "metadata": "10700033"}
{"text": "Observed rate constants, k0b,, were obtained by fitting the equation I(t)=oe-kobs+C by the method of moments, where I is the observed fluorescence intensity, and I0 is the amplitude of fluorescence change. 38 ", "label": "conclusion", "metadata": "235564322"}
{"text": "The capabilities of modern angiographic platforms have recently improved substantially.", "label": "conclusion", "metadata": "2877272"}
{"text": "Few studies have concentrated specifically on the outcomes after surgery.", "label": "conclusion", "metadata": "18989842"}
The job is to fast retrieve the text for the record with metadata "18982114"
Use json package to convert the json object into a dictionary then use the data stored in the metadata key. here is an working example:
# importing the module
import json
# Opening JSON file
with open('data.json') as json_file:
data = json.load(json_file)
# Print the type of data variable
print("Type:", type(data))
# Print the data of dictionary
print("metadata: ", data['metadata'])
You can try this approach:
import json
with open('data.json') as data_json:
data = json.load(data_json)
MATCH_META_DATA = '18982114'
match_records = []
for part_data in data:
if part_data.get('metadata') == MATCH_META_DATA:
match_records.append(part_data)
Let us imagine we have the following JSON content in example.json:
{
"1":{"text": "Some text 1.", "label": "xxx", "metadata": "18982114"},
"2":{"text": "Some text 2.", "label": "yyy", "metadata": "18982114"},
"3":{"text": "Some text 3.", "label": "zzz", "metadata": "something else"}
}
You can do the following:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import json
# 1. read json content from file
my_json = None
with open("example.json", "r") as file:
my_json = json.load(file)
# 2. filter content
# you can use a list instead of a new dictionary if you don't want to create a new json file
new_json_data = {}
for record_id in my_json:
if my_json[record_id]["metadata"] == str(18982114):
new_json_data[record_id] = my_json[record_id]
# 3. write a new json with filtered data
with open("result.json"), "w") as file:
json.dump(new_json_data, file)
This will output the following result.json file:
{"1": {"text": "Some text 1.", "label": "", "metadata": "18982114"}, "2": {"text": "Some text 2.", "label": "", "metadata": "18982114"}}
I am new to python. I am writing a code, where I need to read the json file, and dump some of it's data to a new json file.
Following is my code:
if vmName=='IGW':
with open(APIJSONPath+NwIntfJSONFileName) as f:
data=json.load(f)
for item in data['Interfaces']:
jdata=item
with open(NwIntfJSONPath+vmName+'.json','w') as c:
json.dump(jdata,c,indent=2)
Following is a small part of my json file data from which this code is supposed to retrieve the interface details(Interface name,IPAddress, PrefixLength, DefaultGateway) of eth0 and eth1:
{
"Interfaces": [{
"Name": "eth0",
"IPConfigurations": [{
"IPAddress": "10.0.3.7",
"PrefixLength": 24,
"DefaultGateway": "10.0.3.1",
"IsPrimary": true
}],
"Description0": "connected to cloudsimple network",
"IsPrimary": true
} ,
{
"Name": "eth1",
"IPConfigurations": [{
"IPAddress": "10.0.3.8",
"PrefixLength": 24,
"DefaultGateway": "10.0.3.1",
"IsPrimary": true
}],
"Description1": "connected to internet"
}
]
}
But the data that is getting dumped the new json file is:
{
"Name": "eth1",
"IPConfigurations": [
{
"PrefixLength": 24,
"IsPrimary": true,
"IPAddress": "10.0.3.8",
"DefaultGateway": "10.0.3.1"
}
],
"Description1": "connected to internet"
}
Only eth1 details is getting dumped, not the eth0. The dumped data is also in an unordered manner.
Can someone please help me figure out, where I am going wrong, and how to fix this two issues in my code? Thanks in advance.
If you need all content of data['Interfaces'] in your output json use the below snippet.
if vmName=='IGW':
with open(APIJSONPath+NwIntfJSONFileName) as f:
data=json.load(f)
with open(NwIntfJSONPath+vmName+'.json','w') as c:
json.dump(data['Interfaces'],c,indent=2)
In your example you are looping through data['Interfaces'] and jdata holds the last value of the list. That is why you are only getting the last element in the output json.
Note: Please see comments of ticked answer for resolution :)
I'm writing some python which accesses an API and the exports the server's response (JSON) to a CSV file. The JSON is nested.
This is the JSON response from the server (this is just a sample of the response):
{
"id":182774,
"website_id":307842,
"engine_provider":"Google",
"engine_name":"United Kingdom",
"keywords":[
{
"id":4464443,
"groups_id":[
44424
],
"name":"SMART E70 Interactive Display",
"positions":{
"2017-03-16":10
}
},
{
"id":4464442,
"groups_id":[
44424
],
"name":"SMART Podium SP518",
"positions":{
"2017-03-16":4
}
},
{
"id":4464441,
"groups_id":[
44424
],
"name":"SMART Board M680",
"positions":{
"2017-03-16":3
}
},
{
"id":4464338,
"groups_id":[
51168
],
"name":"NEC Lamps",
"positions":{
"2017-03-16":4
}
}
]
}
If the JSON is looking wrong, it's probably because I edited it wrong when sampling it for this post.
In Python I try to parse the query response and then write the nested fields to a CSV table like this:
parsedqueryresponse = queryresponse.json()
f = csv.writer(open(csvoutputpath, "wb+"))
f.writerow(["name", "positions", "id"])
for parsedqueryresponse in parsedqueryresponse:
f.writerow([parsedqueryresponse["keywords"]["name"],
parsedqueryresponse["keywords"]["positions"],
parsedqueryresponse["keywords"]["id"]])
When I run the script I get this error:
"line 146, in
f.writerow([parsedqueryresponse["keywords"]["name"],
TypeError: string indices must be integers"
Line 146 is this one (also referenced by the error message):
f.writerow([parsedqueryresponse["keywords"]["name"]
What am I doing wrong here? I tried changing the JSON fields to use ' instead of " but that didn't seem to make things any better...
Please try this,
import csv
import json
parsedqueryresponse = queryresponse.json()
f = csv.writer(open(csvoutputpath, "wb+"))
f.writerow(["name", "positions", "id"])
for entry in parsedqueryresponse["keywords"]:
f.writerow([entry["name"],
entry["positions"],
entry["id"]])
Output:
name,positions,id
SMART E70 Interactive Display,{2017-03-16: 10},4464443
SMART Podium SP518,{2017-03-16: 4},4464442
SMART Board M680,{2017-03-16: 3},4464441
NEC Lamps,{2017-03-16: 4},4464338
Please let me know in terns of any queries.