I have a large json file (2.4 GB). I want to parse it in python. The data looks like the following:
[
{
"host": "a.com",
"ip": "1.2.2.3",
"port": 8
},
{
"host": "b.com",
"ip": "2.5.0.4",
"port": 3
},
{
"host": "c.com",
"ip": "9.17.6.7",
"port": 4
}
]
I run this python script parser.py to load the data for parsing::
import json
from pprint import pprint
with open('mydata.json') as f:
data = json.load(f)
Previously, I made this post about the same code. I am trying to run the code with larger RAM. but I got a different error. Can you please help me identify the source of the problem?
Traceback (most recent call last): File "parser.py", line 6, in
data = json.load(f) File "/usr/lib/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File
"/usr/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1095583 column 749 (char 56649111)
There is a similar problem in this post but I could not use the solution as I read my json array from a file. Not sure how to apply the solution in this case?
Related
This question already has answers here:
Why can't I call read() twice on an open file?
(7 answers)
Closed last year.
with open(LATEST_UPDATE_FULL_PATH) as JSON_FILE_UPDATE:
JSON_DATA_UPDATE = json.load(JSON_FILE_UPDATE)
JSON_DATA_UPDATE = json.load(JSON_FILE_UPDATE)
why does doing "load" twice causes the following below?
Is there a handle on this file? I could not find anything to release JSON_FILE_UPDATE.
Traceback (most recent call last):
File "PhthonScript", line 69, in <module>
JSON_DATA_UPDATE = json.load(JSON_FILE_UPDATE)
File "...\PythonByMiniconda3\lib\json\__init__.py", line 293, in load
return loads(fp.read(),
File "...\PythonByMiniconda3\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "...\PythonByMiniconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "...\PythonByMiniconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
try this:
JSON_DATA_UPDATE = json.loads(JSON_FILE_UPDATE)
once it is loaded, then the file converted to json. cannot convert json to json.
another way, this may work on string to json:
import ast
file = """{
"students": [
{
"name": "Millie Brown",
"active": False,
"rollno": 11
},
{
"name": "Sadie Sink",
"active": True,
"rollno": 10
}
]
}"""
print(type(file))
dict = ast.literal_eval(file)
print(type(dict))
dict['students'][0]['name'] would be "Millie Brown"
dict['students'][1]['active'] would be True
This is my ciao.json that i need to parse some data from it
{
"opcua": [
{
"ip": ciao,
"port": 4840,
"uri": "http://examples.freeopcua.github.io"}
"objects":[
{
object_name: ListaDeiSensori,
variables:[
"Temperature_Sensor",
"Water_Sensor"]
}]
}]
}
To pase the fileds of the json file i am using this python script:
import json
with open('ciao.json') as file:
data = json.load(file)
print(data
But i get this error:
Traceback (most recent call last):
File "getdata.py", line 11, in <module>
data = json.load(file)
File "/usr/lib/python3.6/json/__init__.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 4 column 28 (char 72)
Is there anyone that can help me in solving this problem?
I do not know how to solve it...
I think there are some errors in the json file.
update json, I changed undefined obj to string.
{
"opcua": [
{
"ip": "ciao",
"port": 4840,
"uri": "http://examples.freeopcua.github.io"
},
{
"objects": [
{
"object_name": "ListaDeiSensori",
"variables": [
"Temperature_Sensor",
"Water_Sensor"
]
}
]
}
]
}
I'm new to Python. I'm working on a script to send notifications on overdue Asana tasks. I'm running into issues with converting the Asana API response, which is a JSON with multiple objects that represent tasks, to Python objects. For now, all I want to do is convert the JSON objects into Python objects to validate the response.
This is what the raw JSON response from Asana looks like:
{
"data": [
{
"gid": "1234567891234567",
"due_on": "2021-02-12",
"name": "My First Task",
"permalink_url": "https://app.asana.com/0/123456789/1234567891234567"
},
{
"gid": "1234567891234568",
"due_on": "2021-02-26",
"name": "My Second Task",
"permalink_url": "https://app.asana.com/0/123456789/1234567891234568"
},
{
"gid": "1234567891234569",
"due_on": null,
"name": "My Third Task",
"permalink_url": "https://app.asana.com/0/123456789/1234567891234569"
}
]
}
My Python code looks like the following:
import json
import asana
taskList = []
with open('./work_in_progress.json') as f:
for jsonObj in f:
taskDict = json.loads(jsonObj)
taskList.append(taskDict)
print("Printing each JSON Decoded Object")
for task in taskList:
print(task["gid"], task["name"], task["due_on"], task["permalink"])
However, when I run it, I get the following error:
Traceback (most recent call last):
File "test.py", line 9, in <module>
taskDict = json.loads(jsonObj)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 2 (char 1)
Any help would be appreciated.
Have you tried the following? Replace the with statement with this code:
with open('./work_in_progress.json') as f:
taskDict = json.load(f)
taskList.append(taskDict)
I get the below error when i try to read a JSON file, not sure what needs to be corrected:
C:\Users\prabhjot2600\PycharmProjects\PythonPractise\venv\Scripts\python.exe C:/Users/prabhjot2600/Desktop/Prabh/alien_invasion/test.py
Traceback (most recent call last):
File "C:/Users/prabhjot2600/Desktop/Prabh/alien_invasion/test.py", line 26, in <module>
data = json.loads(people_string)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 11 column 21 (char 192)
My code is simply reading a JSON file and printing its data. I have checked my JSON data also which looks correct and also formatted it to see if there were any errors left but still cant read the JSON
import json
people_string = '''
{
"people": [
{
"name": "Prabhjot Singh",
"phone": "9999596310",
"emails": [
"prabh#noemail.com",
"prabhjot#ranamail.com"
],
"hasLicence": True
},
{
"name": "Sunny Rana",
"phone": "9999988888",
"emails": null,
"hasLicence": False
}
]
}
'''
data = json.loads(people_string)
print(data)
JSON stands for JavaScript Object Notation.
So it is subset of Javascript's syntax.
Javascript uses true and false keyword as literal boolean values.
(Whereas python uses True and False).
Therefore, you should use true and false instead of True and False.
See https://www.json.org/json-en.html for detailed syntax.
My JSON file is
{ "results": [
{ "ID": "63768E9B-1D66-486A-BCDD-D3991EAFBE94",
"Dt": "2013-08-03T13:01:26.901Z",
"Dt_u": "2013-08-03T13:01:26.901Z",
"obj": "enppXhI7TS"
},
{
"ID": "63768E9B-1D66-486A-BCDD-D3991EAFBE94",
"Dt": "2013-08-03T16:17:33.280Z",
"Dt_u": "2013-08-03T16:17:33.280Z",
"obj": "79J5z6y2UR"
},
{
"ID": "F8B1B9FB-7BCD-47DF-89BD-241440BB6270",
"Dt": "2013-08-06T00:23:43.562Z",
"obj": "Xf75BFtx4O",
"gender": 2,
"language": "en"
}]}
There are many more entries in the file
My python code is as follows
import json
from pprint import pprint
json_data=open('data.json','r')
data = json.load(json_data)
jsondata = data["results"]
for item in jsondata:
name = item.get("ID")
json_data.close()
When i try to run the file, it gives me following error
Traceback (most recent call last):
File "E:\test.py", line 7, in <module>
data = json.load(json_data)
File "C:\python27\lib\json\__init__.py", line 290, in load
**kw)
File "C:\python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\python27\lib\json\decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\python27\lib\json\decoder.py", line 381, in raw_decode
obj, end = self.scan_once(s, idx)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc2 in position 2: invalid continuation byte
[Finished in 0.2s with exit code 1]
Can somebody help me and tell how do i resolve it?