Read JSON file in python: ValueError - python

I have a .txt file with 70+k json object obtained by pulling data from twitter and dumping into file using:
with open("followers.txt", 'a') as f:
for follower in limit_handled(tweepy.Cursor(api.followers, screen_name=account_name).pages()):
for user_obj in follower:
json.dump(user_obj._json, f)
f.write("\n")
When I try to read this in python using the code below:
import json
with open('followers.txt') as json_data:
follower_data = json.load(json_data)
I get error:
ValueError: Extra data: line 2 column 1 - line 2801 column 1 (char 1489 - 8679498)
It worked when I read a test file with one json object copied from the original file using the same code above. Once I add a second json object to this file then using the same code above gives the error:
ValueError: Extra data: line 2 column 1 - line 2 column 2376 (char 1489 - 3864)
How do I read the file with more than one json object?


The issue comes when you write your JSON. You must write a single JSON object, so you may also load a single JSON object. Currently, you are writing multiple separate objects, causing the error.
Modify your write code a bit:
json_data = []
with open("followers.txt", 'a') as f:
for follower in limit_handled(tweepy.Cursor(api.followers, screen_name=account_name).pages()):
for user_obj in follower:
json_data.append(user_obj._json)
# outside the loops
json.dump(json_data, f)
Now, when reading, your existing code should work. You'll get a list of dictionaries.


Of course it is best to treat the problem from the root: write a single json and read it, as suggested by COLDSPEED.
However, if you already wrote multiple json objects into a single file, you can try the following code to use the already created file:
import json
follower_data = [] # a list of all objects
with open('followers.txt') as json_data:
for line in json_data:
follower_data.append( json.loads(line) )
Assuming you did not indent your json objects when you wrote them to 'flowers.txt', then each line in the file is a json object that can be parsed independantly.

Related

JSONDecodeError when parsing funky JSON

Recently, I started working with JSON (with Python 3.7 under Debian 9). This is the first (probably of many) data sets in JSON which I've had the pleasure of working with.
I have used the Python built-in JSON module to interpret arbitrary strings and files. I now have a database with ~5570 rows pertaining information regarding to a given list of servers. There are a lot of things in the pipeline, which I have devised a plan for, but I'm stuck on this particular sanitation.
Here's the code I'm using to parse:
#!/usr/local/bin/python3.7
import json
def servers_from_json(file_name):
with open(file_name, 'r') as f:
data = json.loads(f.read())
servers = [{'asn': item['data']['resource'], 'resource': item['data']['allocations'][0]['asn_name']} for item in data]
return servers
servers = servers_from_json('working-things/working-format-for-parse')
print(servers)
My motive
I'm trying to get match each one of these servers to their ASN_NAME (which is a field ripped straight from RIPE's API; thus providing me with information pertaining to the physical dc each server is located at. Then, once that's done I'll write them to an existing SQL table, next to a Boolean.
So, here's where it gets funky. If I run the whole dataset through this I get this error message:
Traceback (most recent call last):
File "./parse-test.py", line 12, in <module>
servers = servers_from_json('2servers.json')
File "./parse-test.py", line 7, in servers_from_json
data = json.loads(f.read())
File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.7/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 38 column 2 (char 1098)
I noticed that the problem with my initial data set was that each JSON object wasn't delimited by ,\n.
Did some cleaning, still no luck.
I then added the first 3(?) objects to a completely clean file and.. success. I can get the script to read and interpret them the way I want.
Here's the data set with the comma delimiter:
http://db.farnworth.site/servers.json
Here's the working data set:
http://db.farnworth.site/working-format.json
Anyone got any ideas?

I am here assuming that | will not be present as part of the data. And separate each of the information chunks using | and then convert it into a list and load each list item using json module. Hope it helps!
You can try:
import json
import re
with open("servers.json", 'r') as f:
data = f.read()
pattern = re.compile(r'\}\{')
data = pattern.sub('}|{', data).split('|')
for item in data:
server_info = json.loads(item)
allocations = server_info['data']['allocations']
for alloc in allocations:
print(alloc['asn_name'])
I could read the output.json like this
import json
import re
with open("output.json", 'r') as f:
data = f.read()
server_info = json.loads(data)
for item in server_info:
allocations = item['data']['allocations']
for alloc in allocations:
print(alloc['asn_name'])

Extracting an object among many, from JSON file in python

My JSON file (5000 in total) looks like this
{"version":1.2,"people":[{"pose_keypoints_2d":[0,0,0,1383.9,318.242,0.892951,1427.93,327.25,0.828307,1445.77,409.619,0.952105,1416.18,397.863,0.090918,1345.49,306.583,0.789117,0,0,0,0,0,0,1377.87,459.777,0.824136,1404.51,471.452,0.745714,0,0,0,0,0,0,1345.6,459.6,0.731992,0,0,0,0,0,0,0,0,0,0,0,0,1421.95,271.437,0.540316,1377.91,268.169,0.306764,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[1516.22,174.003,0.932503,1507.48,224.333,0.864362,1451.52,221.257,0.883005,1413.29,291.958,0.80518,1463.28,291.779,0.867427,1563.37,238.898,0.84872,1569.37,321.275,0.825046,1566.2,394.961,0.864594,1495.57,412.46,0.804823,1466.11,409.65,0.698038,1454.5,527.379,0.146485,0,0,0,1528.09,412.567,0.733411,1516.28,530.372,0.196314,0,0,0,1504.51,156.44,0.878404,1525.11,159.409,0.852905,1492.74,159.449,0.567964,1542.82,171.264,0.637156,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[630.433,247.712,0.887763,621.52,297.744,0.898341,580.278,297.779,0.803383,600.839,380.229,0.766635,624.594,324.31,0.850614,665.583,294.92,0.862791,677.475,377.217,0.792423,644.959,389.091,0.861932,630.352,459.596,0.708987,606.759,459.742,0.639099,612.697,580.392,0.424434,594.964,809.933,0.445884,659.764,456.741,0.742195,653.902,577.364,0.655682,648.036,689.252,0.638049,621.585,241.786,0.917391,642.178,238.881,0.869318,597.994,244.66,0.8782,648.082,238.926,0.0987843,674.549,715.811,0.559119,677.444,712.729,0.545488,642.173,695.197,0.447904,648.119,833.326,0.4606,639.217,836.383,0.40265,583.314,830.54,0.387601],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[0,0,0,1024.74,274.253,0.879164,1071.92,280.108,0.773811,1092.51,365.365,0.638758,0,0,0,977.61,271.255,0.881595,974.632,342.017,0.124485,0,0,0,1018.81,439.052,0.685377,1048.31,441.989,0.674454,1039.42,553.779,0.665407,1018.85,662.676,0.66249,980.631,436.208,0.673822,983.535,547.897,0.543447,974.669,647.981,0.465162,0,0,0,0,0,0,1068.95,224.063,0.535875,1021.84,215.334,0.331974,989.444,665.68,0.120769,962.987,650.998,0.121694,971.792,665.614,0.374614,1036.49,668.732,0.40357,1036.47,671.618,0.45803,1007.05,671.574,0.609372],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[1133.66,156.449,0.871671,1160.14,200.52,0.909641,1119.08,194.716,0.881631,1101.34,250.727,0.741804,1095.47,306.6,0.554203,1189.7,206.51,0.863547,1210.3,294.77,0.795075,1201.36,359.55,0.150867,1142.45,356.582,0.705377,1118.83,350.711,0.660271,1118.81,450.822,0.10509,0,0,0,1166.08,359.521,0.703378,1160.13,439.008,0.0764702,0,0,0,1124.82,150.485,0.869508,1142.39,150.587,0.958235,0,0,0,1166.1,150.462,0.915307,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[1327.76,344.854,0.7714,1230.79,412.576,0.777522,1186.54,430.265,0.710346,1166.08,574.291,0.745479,1286.74,551.003,0.859514,1274.9,400.802,0.615538,1333.89,500.898,0.732315,1330.99,406.568,0.842077,1251.33,692.147,0.54616,1213.11,695.104,0.558347,1195.5,874.656,0.59577,1151.27,1045.3,0.537499,1283.75,689.225,0.499876,1260.23,868.795,0.569042,1213.25,1018.88,0.64621,1310.24,338.859,0.914034,0,0,0,1260.32,342.03,0.923518,0,0,0,1283.71,1048.34,0.615529,1283.79,1045.27,0.588761,1201.34,1030.64,0.34822,1218.98,1054.23,0.162428,0,0,0,1139.5,1051.25,0.387906],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[851.14,230.059,0.836269,907,300.74,0.759377,886.42,294.972,0.670902,880.598,377.227,0.139703,839.35,365.599,0.321698,927.62,306.627,0.803614,921.722,409.668,0.740124,854.059,388.947,0.874953,886.524,486.13,0.63616,883.497,483.296,0.584742,903.968,624.47,0.744024,912.915,739.165,0.769062,895.273,489.101,0.678105,880.593,627.447,0.80521,880.369,789.348,0.833229,0,0,0,859.958,221.314,0.9495,0,0,0,901.129,227.162,0.8659,821.619,812.882,0.762655,839.321,818.779,0.748007,886.466,809.794,0.762739,865.847,745.129,0.178172,0,0,0,927.672,745.158,0.699705],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[1748.81,400.745,0.865472,1769.41,480.169,0.60199,1692.83,453.801,0.777207,1636.84,571.465,0.602594,1622.26,674.491,0.237392,1831.2,512.658,0.44287,1751.68,715.693,0.0641964,0,0,0,1689.96,718.654,0.295779,1645.81,703.916,0.317921,1607.49,854.024,0.214191,1572.18,1001.31,0.419405,1734.12,733.402,0.223622,1613.47,856.961,0.100664,1580.98,1010.01,0.141665,1745.88,377.234,0.802295,1772.39,397.828,0.831415,0,0,0,1819.34,433.244,0.858218,1489.73,1024.81,0.22791,1498.66,1036.55,0.219253,1589.85,1030.66,0.174787,1492.65,1024.84,0.494689,1492.73,1021.76,0.468759,1586.97,1024.84,0.321146],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[0,0,0,547.977,341.978,0.246313,536.227,341.878,0.601369,550.96,450.742,0.778364,615.651,424.273,0.762666,0,0,0,0,0,0,0,0,0,580.284,530.316,0.318203,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[1831.27,265.312,0.718134,1781.17,291.987,0.733738,1748.81,294.957,0.794407,1719.31,388.954,0.679386,1748.76,418.448,0.11695,1822.32,288.843,0.617834,0,0,0,0,0,0,1754.63,441.953,0.0559453,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1825.18,247.9,0.881403,1834.11,247.735,0.108933,1790.05,247.868,0.858578,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,577.393,527.334,0.376305,580.359,671.567,0.278565,618.596,692.074,0.379358,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,624.434,700.97,0.323246],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,580.365,533.218,0.257544,600.889,686.267,0.313003,600.939,809.808,0.41703,0,0,0,0,0,0,0,0,0,0,0,0,650.978,833.455,0.42398,648.014,827.561,0.372851,589.17,830.452,0.229458,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]},{"pose_keypoints_2d":[930.432,177.066,0.270157,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,927.496,174.034,0.370837,0,0,0,903.997,174.152,0.297842,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]}],"part_candidates":[{"0":[1133.66,156.449,0.871671,1516.22,174.003,0.932503,930.432,177.066,0.270157,851.14,230.059,0.836269,630.433,247.712,0.887763,1831.27,265.312,0.718134,1327.76,344.854,0.7714,1748.81,400.745,0.865472],"1":[1160.14,200.52,0.909641,1507.48,224.333,0.864362,1024.74,274.253,0.879164,1781.17,291.987,0.733738,621.52,297.744,0.898341,907,300.74,0.759377,1383.9,318.242,0.892951,547.977,341.978,0.246313,1230.79,412.576,0.777522,1769.41,480.169,0.60199,1869.48,553.935,0.143333],"2":[1119.08,194.716,0.881631,1451.52,221.257,0.883005,1071.92,280.108,0.773811,886.42,294.972,0.670902,1748.81,294.957,0.794407,580.278,297.779,0.803383,1427.93,327.25,0.828307,536.227,341.878,0.601369,1186.54,430.265,0.710346,1692.83,453.801,0.777207,1922.51,533.203,0.0694578],"3":[1101.34,250.727,0.741804,1413.29,291.958,0.80518,1092.51,365.365,0.638758,880.598,377.227,0.139703,600.839,380.229,0.766635,1719.31,388.954,0.679386,1445.77,409.619,0.952105,550.96,450.742,0.778364,1636.84,571.465,0.602594,1166.08,574.291,0.745479],"4":[1463.28,291.779,0.867427,1095.47,306.6,0.554203,624.594,324.31,0.850614,839.35,365.599,0.321698,1416.18,397.863,0.090918,1748.76,418.448,0.11695,615.651,424.273,0.762666,1286.74,551.003,0.859514,1622.26,674.491,0.237392],"5":[1189.7,206.51,0.863547,1563.37,238.898,0.84872,977.61,271.255,0.881595,1822.32,288.843,0.617834,665.583,294.92,0.862791,927.62,306.627,0.803614,1345.49,306.583,0.789117,545.017,338.899,0.173114,1274.9,400.802,0.615538,1831.2,512.658,0.44287],"6":[1210.3,294.77,0.795075,1569.37,321.275,0.825046,974.632,342.017,0.124485,677.475,377.217,0.792423,921.722,409.668,0.740124,1333.89,500.898,0.732315,1751.68,715.693,0.0641964,1734.09,736.304,0.0629545,1704.59,974.704,0.0607181],"7":[1201.36,359.55,0.150867,644.959,389.091,0.861932,854.059,388.947,0.874953,1566.2,394.961,0.864594,1330.99,406.568,0.842077,1704.67,662.737,0.0836309],"8":[1142.45,356.582,0.705377,1495.57,412.46,0.804823,1018.81,439.052,0.685377,1754.63,441.953,0.0559453,630.352,459.596,0.708987,1377.87,459.777,0.824136,886.524,486.13,0.63616,580.284,530.316,0.318203,1251.33,692.147,0.54616,1689.96,718.654,0.295779,1863.57,992.377,0.0562326],"9":[1118.83,350.711,0.660271,1466.11,409.65,0.698038,1048.31,441.989,0.674454,606.759,459.742,0.639099,1404.51,471.452,0.745714,883.497,483.296,0.584742,577.393,527.334,0.376305,1213.11,695.104,0.558347,1645.81,703.916,0.317921],"10":[1118.81,450.822,0.10509,1454.5,527.379,0.146485,1039.42,553.779,0.665407,612.697,580.392,0.424434,903.968,624.47,0.744024,580.359,671.567,0.278565,1607.49,854.024,0.214191,1195.5,874.656,0.59577],"11":[1018.85,662.676,0.66249,618.596,692.074,0.379358,912.915,739.165,0.769062,594.964,809.933,0.445884,1572.18,1001.31,0.419405,1151.27,1045.3,0.537499],"12":[1166.08,359.521,0.703378,1528.09,412.567,0.733411,980.631,436.208,0.673822,659.764,456.741,0.742195,1345.6,459.6,0.731992,895.273,489.101,0.678105,580.365,533.218,0.257544,1283.75,689.225,0.499876,1734.12,733.402,0.223622,1795.89,980.592,0.0555809,1681.09,1074.83,0.0648456],"13":[1160.13,439.008,0.0764702,1516.28,530.372,0.196314,983.535,547.897,0.543447,653.902,577.364,0.655682,880.593,627.447,0.80521,600.889,686.267,0.313003,1613.47,856.961,0.100664,1260.23,868.795,0.569042],"14":[974.669,647.981,0.465162,648.036,689.252,0.638049,880.369,789.348,0.833229,600.939,809.808,0.41703,1580.98,1010.01,0.141665,1213.25,1018.88,0.64621,1663.41,1074.84,0.076733],"15":[1124.82,150.485,0.869508,1504.51,156.44,0.878404,927.496,174.034,0.370837,856.812,221.238,0.0621831,621.585,241.786,0.917391,1825.18,247.9,0.881403,1310.24,338.859,0.914034,1745.88,377.234,0.802295],"16":[1142.39,150.587,0.958235,1525.11,159.409,0.852905,930.546,174.006,0.13775,859.958,221.314,0.9495,642.178,238.881,0.869318,1834.11,247.735,0.108933,1330.78,330.13,0.111218,1772.39,397.828,0.831415],"17":[1492.74,159.449,0.567964,903.997,174.152,0.297842,1068.95,224.063,0.535875,597.994,244.66,0.8782,1790.05,247.868,0.858578,1421.95,271.437,0.540316,1260.32,342.03,0.923518],"18":[1166.1,150.462,0.915307,1542.82,171.264,0.637156,1021.84,215.334,0.331974,901.129,227.162,0.8659,648.082,238.926,0.0987843,1377.91,268.169,0.306764,1819.34,433.244,0.858218],"19":[989.444,665.68,0.120769,674.549,715.811,0.559119,821.619,812.882,0.762655,650.978,833.455,0.42398,1489.73,1024.81,0.22791,1283.71,1048.34,0.615529],"20":[962.987,650.998,0.121694,677.444,712.729,0.545488,839.321,818.779,0.748007,648.014,827.561,0.372851,1498.66,1036.55,0.219253,1283.79,1045.27,0.588761],"21":[971.792,665.614,0.374614,642.173,695.197,0.447904,886.466,809.794,0.762739,589.17,830.452,0.229458,1201.34,1030.64,0.34822,1589.85,1030.66,0.174787,1660.5,1074.84,0.0597874],"22":[1036.49,668.732,0.40357,668.602,715.634,0.124469,865.847,745.129,0.178172,648.119,833.326,0.4606,1492.65,1024.84,0.494689,1224.82,1051.34,0.162429,1218.98,1054.23,0.162428,1168.98,1074.84,0.174282,1198.4,1074.81,0.16778],"23":[1036.47,671.618,0.45803,650.994,715.785,0.112767,871.693,739.271,0.162696,639.217,836.383,0.40265,1492.73,1021.76,0.468759,1166.04,1074.81,0.239277],"24":[1007.05,671.574,0.609372,1424.99,695.172,0.0523207,624.434,700.97,0.323246,927.672,745.158,0.699705,583.314,830.54,0.387601,1586.97,1024.84,0.321146,1139.5,1051.25,0.387906]}]}
I need to extract only one object named "1" with its contents and save these to a new JSON file with a new name say, "New file 1", "New file 2" ....
Or if deleting all contents except "1" and its contents, and leaving the file as it is with object "1".

You can use the json library to read the file contents into a python data structure (in this case, a nested dictionary). You can them dump specific parts of the JSON data out into new files.
import json
with open('input_json.json', 'r') as infile:
data = json.load(infile)
for k, v in data['part_candidates'][0].items():
with open('output_json_{}.json'.format(k), 'w') as outfile:
json.dump(v, outfile)

convert Json to Dictionary, using python library json
import json
data = json.loads(your json)
Than after you are done performing
json.dump(data)
Edit 1
You can access the element in a Dictionary by
sample_data = dict((k,data[k]) for k in ('key1','key2','key99') if k in data)

Converting dictionary as Json and append to a file

Scenario is i need to convert dictionary object as json and write to a file . New Dictionary objects would be sent on every write_to_file() method call and i have to append Json to the file .Following is the code
def write_to_file(self, dict=None):
f = open("/Users/xyz/Desktop/file.json", "w+")
if json.load(f)!= None:
data = json.load(f)
data.update(dict)
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(data))
else:
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(dict)
Getting this error "No JSON object could be decoded" and Json is not written to the file. Can anyone help ?

this looks overcomplex and highly buggy. Opening the file several times, in w+ mode, and reading it twice won't get you nowhere but will create an empty file that json won't be able to read.
I would test if the file exists, if so I'm reading it (else create an empty dict).
this default None argument makes no sense. You have to pass a dictionary or the update method won't work. Well, we can skip the update if the object is "falsy".
don't use dict as a variable name
in the end, overwrite the file with a new version of your data (w+ and r+ should be reserved to fixed size/binary files, not text/json/xml files)
Like this:
def write_to_file(self, new_data=None):
# define filename to avoid copy/paste
filename = "/Users/xyz/Desktop/file.json"
data = {} # in case the file doesn't exist yet
if os.path.exists(filename):
with open(filename) as f:
data = json.load(f)
# update data with new_data if non-None/empty
if new_data:
data.update(new_data)
# write the updated dictionary, create file if
# didn't exist
with open(filename,"w") as f:
json.dump(data,f)

Error loading json in python line by line?

Here is my json file format,
[{
"name": "",
"official_name_en": "Channel Islands",
"official_name_fr": "Îles Anglo-Normandes",
}, and so on......
while loading the above json which is in a file I get this error,
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes:
here is my python code,
import json
data = []
with open('file') as f:
for line in f:
data.append(json.loads(line))

,} is not allowed in JSON (I guess that's the problem according to the data given).

You appear to be processing the entire file one line at a time. Why not simply use .read() to get the entire contents at once, then feed that to json?
with open('file') as f:
contents = f.read()
data = json.loads(contents)
Better yet, why not use json.load() to pass the readable directly and let it handle the slurping?
with open('file') as f:
data = json.load(f)

The problem is in your reading and decoding the file line by line. Any single line in your file (e.g., "[{") is not a valid JSON expression.

Your individual lines are not valid JSON. For instance, the first line '[{' by itself is not a valid JSON. If your entire file is actually valid JSON and you want individual lines, first load the entire JSON and then browse through the python dictionary.
import json
data = json.loads(open('file').read()) # this should be a list
for list_item in data:
print(list_item['name'])

What is the best way to write json objects to a text file and then read them back in?

I am trying to write a service to read twitter feed stream data and then write it to a file. I am writing each JSON structure to a line in the file. With a different service I need to read each line of the file and load the json structure for further operations.
My problem is that I can read the first line then the JSON loader says the rest are not JSON structures. They look fine. Not sure what is going on.
Writting file:
self.output = open(os.path.join(self.outputdir,self.filename,'w')
self.output.write(status + "\n")
Reading File:
with open(file) as f:
line = line.replace("\n","")
tweet = json.loads(line)
print tweet['text']
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Example json file:
JSON File
JSON File

Your json is composed with multiple json objects and empty lines.
You need to load each line as a new json object and ignore empty lines:
>>> with open('streamer.151205-071156.json') as f:
>>> data = [json.loads(l) for l in f if len(l) > 1]
>>> len(data)
7
>>> print(data[0]['text'])
u'Mnjd \U0001f642\U0001f602 https://t.co/BL5Ezxtt0i'

Categories

Resources