Opening file sometimes results in json.decoder.JSONDecodeError - python

I have a file that is updated with new content and saved every hour:
with open('data.txt','w') as outfile:
json.dump(data,outfile)
I then read this file at any given time:
with open('data.txt') as json_file:
data = json.load(json_file)
The problem I'm having is sometimes this file is in the process of being updated with new content when trying to read the file which results in a json.decoder.JSONDecodeError. How can I avoid this error? Maybe a try except case that waits for the file to be readable?

Easiest way, would be to catch JSONDecodeError.
try:
with open('data.txt') as json_file:
data = json.load(json_file)
except JSONDecodeError:
time.sleep(10)
with open('data.txt') as json_file:
data = json.load(json_file)
Or you can try answer from https://stackoverflow.com/a/11115521/4916849.

Related

save json file within a loop, python

in jupyter notebook, I ran this code in a cell:
for i in range(10):
with open('data.json', 'w') as f:
json.dump({"counter":i}, f)
time.sleep(10000)
easy so far, but after executing the cell there won't be any update on the actual data.json file during each iteration, it will get updated up until the end of the program. in other words, the data.json as a file object stays open till the end of the code.
how can I update the file on the disk in a loop?
The json module doesn't work that way AFAIK. You'll have to load the json data into a dictionary/list then make your changes, then write the file again:
# funciton to read json files
def read_json(path):
with open(path, 'r') as file:
return json.load(file)
# function to write json files
def write_json(path, data, indent=4):
with open(path, 'w') as file:
json.dump(data, file, indent=indent)
# read some json data
json_data = read_json('./my_json_file.json')
# ... do some stuff to the data
# write the data back to the file
write_json('./my_json_file.json', json_data)

python - json keep returning JSONDecodeError when reading from file

I want to write data to a json file. If it does not exists, I want to create that file, and write data to it. I wrote code for it, but I'm getting json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0).
Here's part of my code:
data = {"foo": "1"}
with open("file.json", "w+") as f:
try:
file_data = json.load(f)
json.dump(data, f)
print("got data from file:", file_data)
except json.decoder.JSONDecodeError:
json.dump(data, f)
print("wrote")
I put print statements so that I can "track" what's going on, but If I try to run this code multiple times, I keep getting wrote message.
Thanks in advance for help!
The problem is that you open the file for write/read therefore once you open the file it will be emptied.
Then you want to load the content with json.load and it obviously fails because the file is not a valid JSON anymore.
So I'd suggest to open the file for reading and writing separately:
import json
with open("file.json") as json_file:
file_data = json.load(json_file)
print("got data from file:", file_data)
data = {"foo": "1"}
with open("file.json", "w") as json_file:
json.dump(data, json_file)
Hope it helps!

Incrementally Append to JSON File in a For Loop

Is there a way to append single JSON objects to a json file while in a for loop in python. I would prefer not store all my data in one giant json object and dump it all at once, as I am planning on performing millions of API requests. I would like to make a single API request, dump the result into a JSON file and then move to the next API request and dump that into the same JSON file.
The below code overwrites the JSON file, I am looking for something that appends.
for url in urls:
r = sesh.get(url)
data = r.json()
with open('data.json', 'w') as outfile:
json.dump(data, outfile)
Such that:
with open('data.json') as outfile:
data = json.load(data, outfile)
type(data)
>> dict
r.json looks something like this:
{'attribute1':1, 'attribute2':10}
Update
Well since I don't have access to your API I just placed some sample responses, in the format you supplied, inside an array.
import json
urls = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];
json_arr = []
for url in urls:
data = json.loads(url)
json_arr.append(data)
with open('data.json', 'w') as outfile:
json.dump(json_arr, outfile)
Basically we keep an array and append each API response to that array. Then, we can write the accumulative JSON to a file. Also if you want to update the same JSON file on different executions of the code, you can just read the existing output file into an array, in the beginning of the code, and then carry on with my example.
Change write mode to append
Try changing this:
with open('data.json', 'w') as outfile:
To this:
with open('data.json', 'a') as outfile:
The previous answer is surprisingly close to what you need to do.
So I will build upon it.
import json
json_arr = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];
with open('data.json', 'w') as outfile:
outfile.write('[')
for element in json_arr:
with open('data.json', 'w') as outfile:
json.dump(element, outfile)
outfile.write(',')
with open('data.json', 'a') as outfile:
outfile.write(']')

Try to read a json file with python

I have a json file that is a synonime dicitonnary in French (I say French because I had an error message with ascii encoding... due to the accents 'é',etc). I want to read this file with python to get a synonime when I input a word.
Well, I can't even read my file...
That's my code:
data=[]
with open('sortieDES.json', encoding='utf-8') as data_file:
data = json.loads(data_file.read())
print(data)
So I have a list quite ugly, but my question is: how can I use the file like a dictionary ? I want to input data['Académie']and have the list of the synonime... Here an example of the json file:
{"Académie française":{
"synonymes":["Institut","Quai Conti","les Quarante"]
}
You only need to call json.load on the File object (you gave it the name data_file):
data=[]
with open('sortieDES.json', encoding='utf-8') as data_file:
data = json.load(data_file)
print(data)
Instead of
json.load(line)
you have to use
json.loads(line)
Your s is missing in loads(...)

i get error after simple try to store json

In python 3,3
import json
peinaw = {"hi":4,"pordi":6}
json_data = open('data.json')
json.dump(peinaw, json_data)
json_data.close()
i get
File "C:\Python33\lib\json\__init__.py", line 179, in dump
fp.write(chunk)
io.UnsupportedOperation: not writable
tried same thing in 2,7 and it works.I s there a different way in 3,3?
>>> import json
>>> peinaw = {"hi":4,"pordi":6}
>>> with open('data.json', 'w') as json_data: # 'w' to open for writing
json.dump(peinaw, json_data)
I used a with statement here, where the file is automatically .close()d at the end of the with block.
You are not opening the file for writing. The file is opened in a read mode. to verify do this:
json_data = open('data.json')
print (json_data) # should work with 2.x and 3.x
To solve the probem, just open the file in write mode.
json_data = open('data.json', 'w')
Also, you should use the with statement when woking with files.
with open('data.json', 'w') as json_data:
json.dump(peinaw, json_data)
You need to open file for writing, use 'w' mode parameter:
json_data = open('data.json', 'w')

Categories

Resources