I'm trying to open and print my json file but its returning
JSONDecodeError: Expecting value
This is my code
with open(input("Which json. file would you like to read? "),'r') as f:
weather = json.loads(f.read())
print(weather)
I really don't know whats the problem, all help appreciated!
This works for me:
import json
# Data to be written
dictionary = {
"name": "sathiyajith",
"rollno": 56,
"cgpa": 8.6,
"phonenumber": "9976770500"
}
# Serializing json
json_object = json.dumps(dictionary, indent=4)
with open("sample.json", "w") as outfile:
outfile.write(json_object)
with open(input("Which json. file would you like to read? "),'r', encoding='utf-8') as f:
weather = json.loads(f.read())
print(weather)
I am trying to write line by line the JSON output from my Python request. I already checked some similar issue on StackOverflow in the question: write to file line by line python, without success.
Here is the code:
myfile = open ("data.txt", "a")
for item in pretty_json["geonames"]:
print (item["geonameId"],item["name"])
myfile.write ("%s\n" % item["geonameId"] + "https://www.geonames.org/" + item["name"])
myfile.close()
Here the output from my pretty_json["geonames"]
{
"adminCode1": "FR",
"lng": "7.2612",
"geonameId": 2661847,
"toponymName": "Aeschlenberg",
"countryId": "2658434",
"fcl": "P",
"population": 0,
"countryCode": "CH",
"name": "Aeschlenberg",
"fclName": "city, village,...",
"adminCodes1": {
"ISO3166_2": "FR"
},
"countryName": "Switzerland",
"fcodeName": "populated place",
"adminName1": "Fribourg",
"lat": "46.78663",
"fcode": "PPL"
}
Then, as output saved on my data.txt, I'm having :
11048419
https://www.geonames.org/Aïre2661847
https://www.geonames.org/Aeschlenberg2661880
https://www.geonames.org/Aarberg6295535
The expected result should be something like:
Aïre , https://www.geonames.org/11048419
Aeschlenberg , https://www.geonames.org/2661847
Aarberg , https://www.geonames.org/2661880
Writing the output in CSV could be a solution?
Regards.
Using the csv module.
Ex:
import csv
with open("data.txt", "a") as myfile:
writer = csv.writer(myfile) #Create Writer Object
for item in pretty_json["geonames"]: #Iterate list
writer.writerow([item["name"], "https://www.geonames.org/{}".format(item["geonameId"])]) #Write row.
If I understand correctly, you want the same screen output to your file. That's easy. If you are on python 3 just add to your print function:
print (item["geonameId"],item["name"], file=myfile)
Just compose a proper printing format for the needed items:
...
for item in pretty_json["geonames"]:
print("{}, https://www.geonames.org/{}".format(item["name"], item["geonameId"]))
Sample output:
Aeschlenberg, https://www.geonames.org/2661847
N files of with dictionaries-of-lists, saved as a.json, b.json...
{
"ELEC.GEN.OOG-AK-99.A": [
["2013", null],
["2012", 2.65844],
["2011", 2.7383]
],
"ELEC.GEN.AOR-AK-99.A": [
["2015", 217.30239],
["2014", 214.46868],
["2013", 197.32097]
],
"ELEC.GEN.HYC-AK-99.A": [
["2015", 1542.29841],
["2014", 1538.738],
["2013", 1345.665]
]}
I am unclear how to save them all to one large dictionary/json file, like so:
{
"a":
{
"ELEC.GEN.OOG-AK-99.A": [
["2013", null],
["2012", 2.65844],
["2011", 2.7383]
],
"ELEC.GEN.AOR-AK-99.A": [
["2015", 217.30239],
["2014", 214.46868],
["2013", 197.32097]
],
"ELEC.GEN.HYC-AK-99.A": [
["2015", 1542.29841],
["2014", 1538.738],
["2001", 1345.665]
]},
"b": {...},
...
}
This is data I requested that will be used in a javascript graph, and it is theoretically possible to preprocess it even more when streaming the requested data from its source, as well as maybe possible to work around the fact there are so many data files I need to request to get my graph working, but both those options seem very difficult.
I don't understand the best way to parse json-that-is-meant-for-javascript in python.
====
I have tried:
from collections import defaultdict
# load into memory
data = defaultdict(dict)
filelist = ["a.json", "b.json", ...]
for fn in filelist:
with open(fn, 'rb') as f:
# this brings up TypeError
data[fn] = json.loads(f)
# write
out = "out.json"
with open(out, 'wb') as f:
json.dump(data, f)
===
For json.loads() I get TypeError: expected string or buffer. For json.load() it works!
Loading from string:
>>> with open("a.json", "r") as f:
... json.loads(f.read())
...
{u'Player2': 4, u'Player3': 10, u'Player1': 3}
>>>
Loading from file object:
>>> with open("a.json", "r") as f:
... json.load(f)
...
{u'Player2': 4, u'Player3': 10, u'Player1': 3}
>>>
you are using json.loads instead of json.load to load a file, you also need to open it for reading for string instead of bytes, so change this:
with open(fn, 'rb') as f:
data[fn] = json.loads(f)
to this:
with open(f, 'r') as f: #only r instead of rb
data[fn] = json.load(f) #load instead of loads
And again further down when writing open for w instead of wb
I'm trying to convert a JSON file to CSV format (in memory), so that I can pass it to another Transformer in Mulesoft. Here is a snippet of the JSON:
[
{
"observationid": 1,
"fkey_observation": 1,
"value": 1,
"participantid": null,
"uom": "ppb",
"finishtime": 1008585047000,
"starttime": 1008581447000,
"observedproperty": "NO2",
"measuretime": 1008581567000,
"measurementid": 1,
"longitude": 3.1415,
"identifier": "Test-1",
"latitude": 10
},
{
"observationid": 1,
"fkey_observation": 1,
"value": 12,
"participantid": null,
"uom": "ppb",
"finishtime": 1008585047000,
"starttime": 1008581447000,
"observedproperty": "SO2",
"measuretime": 1008582047000,
"measurementid": 2,
"longitude": 5,
"identifier": "Test-1",
"latitude": 11
}
]
Essentially, this should create a CSV (in memory) with 2 rows, that looks like this:
1,1,1,N,ppb,1008585047000,1008581447000,NO2,1008581567000,1,3.1415,Test-1,10
1,1,12,N,ppb,1008585047000,1008581447000,SO2,1008582047000,2,5,Test-1,11
Currently, the output comes out like this, which is wrong:
[1 1 1 None u'ppb' 1008585047000L 1008581447000L u'NO2' 1008581567000L 1 3.1415 u'Test-1' 10]
[1 1 12 None u'ppb' 1008585047000L 1008581447000L u'SO2' 1008582047000L 2 5 u'Test-1' 11]
I believe the 'u' bit refers to Unicode, but I don't know how to change the encoding.
Any help would be greatly appreciated!
Here is the Python code I have so far:
import json
import cStringIO
f = open('test.json')
data = json.load(f)
f.close()
output = cStringIO.StringIO()
for item in data:
output.write(str([item['observationid'], item['fkey_observation'], item['value'], item['participantid'], item['uom'], item['finishtime'], item['starttime'], item['observedproperty'], item['measuretime'],item['measurementid'], item['longitude'], item['identifier'], item['latitude']]) + '\n')
contents = output.getvalue()
print contents`
EDIT
Hi guys, slight change of plan.
Essentially, I have a String object, but it actually is structured like a JSON file:
"[{observationid=1, fkey_observation=1, value=1, participantid=null, uom=ppb, finishtime=2001-12-17 10:30:47.0, starttime=2001-12-17 09:30:47.0, observedproperty=NO2, measuretime=2001-12-17 09:32:47.0, measurementid=1, longitude=3.1415, identifier=CITISENSE-Test-00000001, latitude=10}, {observationid=1, fkey_observation=1, value=12, participantid=null, uom=ppb, finishtime=2001-12-17 10:30:47.0, starttime=2001-12-17 09:30:47.0, observedproperty=SO2, measuretime=2001-12-17 09:40:47.0, measurementid=2, longitude=5, identifier=CITISENSE-Test-00000001, latitude=11}, {observationid=1, fkey_observation=1, value=7000, participantid=null, uom=ppb, finishtime=2001-12-17 10:30:47.0, starttime=2001-12-17 09:30:47.0, observedproperty=NO2, measuretime=2001-12-17 09:52:47.0, measurementid=3, longitude=6, identifier=CITISENSE-Test-00000001, latitude=9}, {observationid=2, fkey_observation=2, value=5, participantid=null, uom=ppb, finishtime=2001-12-18 10:30:47.0, starttime=2001-12-18 09:30:47.0, observedproperty=SO2, measuretime=2001-12-18 09:32:47.0, measurementid=4, longitude=7, identifier=CITISENSE-Test-00000001, latitude=8}, {observationid=2, fkey_observation=2, value=6, participantid=null, uom=ppb, finishtime=2001-12-18 10:30:47.0, starttime=2001-12-18 09:30:47.0, observedproperty=PM10, measuretime=2001-12-18 09:34:47.0, measurementid=5, longitude=8, identifier=CITISENSE-Test-00000001, latitude=10}, {observationid=3, fkey_observation=3, value=10000, participantid=null, uom=ppb, finishtime=2001-12-19 10:30:47.0, starttime=2001-12-19 09:30:47.0, observedproperty=SO2, measuretime=2001-12-19 09:38:47.0, measurementid=6, longitude=9, identifier=CITISENSE-Test-00000001, latitude=11.2}]"
How do I go about converting this to CSV? I can't use the json module as it is not a JSON file.
Here is my approach: use csv.DictWriter to handle converting from a dictionary to a row of CSV data:
import csv
import json
from cStringIO import StringIO
with open('test.json') as f:
my_data = json.load(f)
headers = [
'observationid', 'fkey_observation', 'value',
'participantid', 'uom', 'finishtime', 'starttime',
'observedproperty', 'measuretime', 'measurementid',
'longitude', 'identifier', 'latitude']
buffer = StringIO()
writer = csv.DictWriter(buffer, headers)
for row in my_data:
writer.writerow(row)
print buffer.getvalue()
Here's a little snippet I wrote up, I think it should handle your scenario and give you a list of lists. Ereli is onto something with that module though, it might make your life easier. But in the meantime maybe this will help.
import json
myFile = open('myJson.json','r+')
myData = json.load(myFile)
myFile.close()
myList = []
for x in range(0,len(myData)):
myList.append([])
for key in myData[x].keys():
value = myData[x][key]
if isinstance(value,(str,unicode)):
value = value.encode('ascii','ignore')
myList[x].append(value)
print myList
You should probably consider using something like csvwriter. it will handle the escaping and delimiter setting for you.
See example for python3:
import csv
with open('output.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',')
for line in data:
writer.writerow(line)
it can also be used with cStringIO.