The following code throws
TypeError: Object of type 'datetime' is not JSON serializable
which I know how to resolve. However my real question is how to cleanly structure the code to avoid a partial file if any exception occurs in json.dump.
import datetime
import json
def save(data):
with open('data.txt', 'w') as outfile:
json.dump(data, outfile)
data = dict(sometime=datetime.datetime.now())
save(data)
The above code throws an exception and results in a partial file like:
{"sometime":
Should I dumps to a string first in a try/except? If so are there any memory implications to be aware of? Or delete the file in an except block?
Use a try/except block like:
Code:
def save_json(data, filename):
try:
with open(filename, 'w') as outfile:
json.dump(data, outfile)
except:
try:
os.unlink(filename)
except FileNotFoundError:
pass
and if you want to preserve the exception:
def save_json(data, filename):
try:
with open(filename, 'w') as outfile:
json.dump(data, outfile)
except:
if os.path.exists(filename):
os.unlink(filename)
raise
Test Code:
import datetime
import json
import os
data = dict(sometime=datetime.datetime.now())
save_json(data, 'data.txt')
That depends on whether your JSON data is under your control or from unknown source. If it’s from somewhere you can’t predict, use try...except... block. Otherwise, fix your program to make it always available to serialize.
Related
Running the following code will raise an io.UnsupportedOperation error, because the file is open in "write" mode -
with open("hi.txt", "w") as f:
print(f.read())
The output is -
io.UnsupportedOperation: not readable
So, we can try to cover this up by doing this -
try:
with open("hi.txt", "w") as f:
print(f.read())
except io.UnsupportedOperation:
print("Please give the file read permission")
Output -
NameError: name 'io' is not defined
Even removing "io." spits out the same error -
try:
with open("hi.txt", "w") as f:
print(f.read())
except UnsupportedOperation:
print("Please give the file read permission")
Output -
NameError: name 'UnsupportedOperation' is not defined
Why isn't it working? Isn't "io.UnsupportedOperation" an error?
The io.UnsupportedError is found in the module io. Therefore, before we can use it, we need to import io
import io
then when we are testing for the error in the try except clause we can use io.UnsupportedError.
This gives us:
import io
try:
with open("hi.txt", "w") as f:
print(f.read())
except io.UnsupportedOperation as e:
print(e)
or if you are only using the io module for checking this specific error.
from io import UnsupportedError
try:
with open("hi.txt", "w") as f:
print(f.read())
except UnsupportedOperation as e:
print(e)
Goal: Merge JSON files into one big file
Background: I am using the code below taken from here Issue with merging multiple JSON files in Python
import json
import glob
result = []
for f in glob.glob("/Users/EER/Desktop/JSON_Combo/*.json"):
with open(f, "rb") as infile:
result.append(json.load(infile))
with open("merged_file.json", "wb") as outfile:
json.dump(result, outfile)
However, I get the following error:
JSONDecodeError: Extra data: line 2 column 1 (char 5733)
I checked Python json.loads shows ValueError: Extra data and JSONDecodeError: Extra data: line 1 column 228 (char 227) and ValueError: Extra Data error when importing json file using python but they are a bit different. A potential reason for the error seems to be that my .json files are a list of strings but I am not sure
Question: Any thoughts on how to fix this error?
There is an invalid JSON file in your files, found out which one caused it by catching the error with try except
import json
import glob
result = []
for f in glob.glob("/Users/EER/Desktop/JSON_Combo/*.json"):
with open(f, "rb") as infile:
try:
result.append(json.load(infile))
except ValueError:
print(f)
with open("merged_file.json", "wb") as outfile:
json.dump(result, outfile)
I have a function that opens a file and returns an opened file object.
def read_any():
try:
opened = gzip.open(fname, 'r')
except IOError:
opened = open(fname, 'r')
return opened
When I attempt to run this function on some non-zipped file except condition does not get triggered and the function crashes with the message: IOError: Not a gzipped file.
Ok, now I try and do the same with with statement:
def read_any2():
try:
with gzip.open(fname, 'r') as f:
return f.read()
except IOError:
with open(fname, 'r') as f:
return f.read()
Now, if I try to run the same file the function works as intended.
Can you explain why doesn't except condition get triggered?
To see what's going on, test it in a REPL:
>>> import gzip
>>> f = gzip.open('some_nongzipped_file', 'r')
You will see that this doesn't raise an error. Once you, however, read from the object:
>>> f.read()
... (snip)
OSError: Not a gzipped file
, it raises the error.
In short: Simply creating the file object doesn't read anything from the file yet, and thus doesn't know if it should fail or not.
Since in the first example you just return the file object, when you try to read from it later it will raise the exception there (outside your raise-except block). In your second example you return f.read() which reads and therefore raises the exception. It has nothing to do with the with block, as you can see if you remove it:
def read_any_mod():
try:
opened = gzip.open(fname, 'r')
return opened.read()
except IOError:
opened = open(fname, 'r')
return opened.read()
Using Python 3 on a windows machine:
I have a function to take a list of lists and open it as a csv file using my default application (excel). Despite closing the file after writing, I get a 'locked for editing' message when excel starts.
def opencsv(data):
"""saves a list of lists as a csv and opens"""
import tempfile
import os
import csv
handle, fn = tempfile.mkstemp(suffix='.csv')
with open(fn,"w", encoding='utf8',errors='surrogateescape',\
newline='') as f:
writer=csv.writer(f)
for row in data:
try:
writer.writerow(row)
except Exception as e:
print ('Error in writing row:',e)
f.close()
url = 'file://' + fn.replace(os.path.sep, '/')
os.startfile(fn)
opencsv([['d1','d2'],['d3','d4','d5']])
How can I fix this?
Answer from swstephe's input:
The issue is that mkstemp opens the file and associates it with an os handle. In my original code I was not closing this file properly. See below for updated code.
def opencsv(data):
"""saves a list of lists as a csv and opens"""
import tempfile
import os
import csv
handle, fn = tempfile.mkstemp(suffix='.csv')
with os.fdopen(handle,"w", encoding='utf8',errors='surrogateescape',\
newline='') as f:
writer=csv.writer(f)
for row in data:
try:
writer.writerow(row)
except Exception as e:
print ('Error in writing row:',e)
print (fn)
os.startfile(fn)
I'm currently making a program that requires a JSON database file. I want the program to check for the file, if it's there then it's perfect, run the rest of the program, but if it doesn't exist create 'Accounts.json' with {} inside the file, instead then run the program.
How would I do this? Whats the most efficient way.
Note: I use this for checking, but how would I create the file:
def startupCheck():
if os.path.isfile(PATH) and os.access(PATH, os.R_OK):
# checks if file exists
print ("File exists and is readable")
else:
print ("Either file is missing or is not readable")
I believe you could simply do:
import io
import json
import os
def startupCheck():
if os.path.isfile(PATH) and os.access(PATH, os.R_OK):
# checks if file exists
print ("File exists and is readable")
else:
print ("Either file is missing or is not readable, creating file...")
with io.open(os.path.join(PATH, 'Accounts.json'), 'w') as db_file:
db_file.write(json.dumps({}))
This is how i did it. I hope it helps.
edit, yeey it looks like a code now :D
import json
import os
def where_json(file_name):
return os.path.exists(file_name)
if where_json('data.json'):
pass
else:
data = {
'user': input('User input: '),
'pass': input('Pass input: ')
}
with open('data.json', 'w') as outfile:
json.dump(data, outfile)
How about wrapping the open file in a try/except? I'm not a professional Python coder, so feel free to weigh in if this is not a kosher approach.
try:
with open('Accounts.json', 'r') as fp:
accounts = json.load(fp)
except IOError:
print('File not found, will create a new one.')
accounts = {}
# do stuff with your data...
with open('Accounts.json', 'w') as fp:
json.dump(accounts, fp, indent=4)
w+ opens with write permissions.
The + creates a new file if one is not found.
filename = 'jsonDB.json'
def openFile():
with open(filename, 'w+') as f:
f.write('{}')
f.close
openFile()