How to write to csv in python with exceptions in the middle - python

I am trying to parse directories and get some values from certain files.
I donot have access to all files.
I want to write a script which will write the results from the files that I can read and simply log the ones I donot have access to.
When I parse directories of which I have access to all the files, then my csv is perfect but if even one exception is thrown(which i handle using try except), my file returns empty.
def read_files_extract_stuff_and_write_to_csv(f, sasfile, writer):
with SAS7BDAT(sasfile) as fsas:
for col in fsas.columns:
if(col.name == 'blah'):
writer.writerow([sasfile, len(fasas.columns)])
f = open(file.csv, 'w')
writer = csv.writer(f)
for sasfile in directory
try:
read_files_extract_stuff_and_write_to_csv(f, sasfile, writer)
except Exception as ex:
print(ex)
f.close()
My main exception is I donot have access to read those files. I just want to simply skip them but when I do, its an empty csv.
Exception :
PermissionError: [Errno 13] Permission denied: '//filepath/filename'

Catch the Exception and continue, for example:
try:
read_files_extract_stuff_and_write_to_csv(f, sasfile, writer)
except PermissionError:
continue

Related

failing to move file from input folder to error folder

I am trying to run a code where i am read files sheets. Issue i am facing is, if there is a file which has no sheet named SheetSum I am not able to move it to error location.
example:
def read_file(data_file):
# data_file = '\rr\ex.xlsx'
sheets = {}
try:
print("Reading file: "+data_file)
sheets['df_1'] = pd.read_excel(open(data_file,'rb'), 'SheetSum')
except Exception as excpt:
print("Exception occurred", exc_info=True)
return sheets
read_file(file)
shutil.move( file, dirpath +'\\processed_files')
Getting an error as:
[WinError 32] The process cannot access the file because it is being used by another process
Updates:
If finally is present, it specifies a cleanup handler. The try
clause is executed, including any except and else clauses. If an
exception occurs in any of the clauses and is not handled, the
exception is temporarily saved. The finally clause is executed. If
there is a saved exception it is re-raised at the end of the finally
clause. If the finally clause raises another exception, the saved
exception is set as the context of the new exception.
..More Here
Accepted Solution worked like charm.
Try closing the file before attempting to move it.
So something like
def read_file(data_file):
# data_file = '\rr\ex.xlsx'
sheets = {}
try:
print("Reading file: "+data_file)
sheets_file = open(data_file,'rb')
sheets['df_1'] = pd.read_excel(sheets_file, 'SheetSum')
except Exception as excpt:
print("Exception occurred", exc_info=True)
finally:
sheets_file.close()
return sheets
read_file(file)
shutil.move( file, dirpath +'\\processed_files')
The problem is the file opens, is handed to the pandas method, and when an exception occurs, your file is never closed.
Adding a finally to your try except block ensures that your file is always closed.

Python - close file and IOError

I'm writing a prog which writes data into a file, and each hour a new file is generated, with a precise title.
When the storage is full, it either stops the application, or deletes the oldest file created in the folder.
fp = open (fileNamePath, 'a')
fp.write('%s' % buffer)
try:
fp.close()
except IOError:
# delete file or close app
But when it comes to the except, it just skips it, and display IOError.
Thanks !
So if I understand it correctly, IOError is not catching exception but it is exploding. Did you try some more generic exception ('except Exception')?
Or you can try workaround with finally instead of except part:
finally:
if not f.closed:
# handle exception here
I looked for solution on Google, and someone suggested to use :
try:
with open(filePath,'a') as fp:
fp.write('%s' % buffer)
except IOError as exc:
# Handling exception
Well I don't get why but it works that way.
Thanks all !

Create new file only if it cannot be opened and read in Python using "with __ as"

I would like to try opening a file to read, and only if this fails, overwrite it or create a new file. I know it can be done using try, except, finally methodology, but I was wondering if there was a more elegant way to do it with the with & as keywords.
try:
f = file.open("file","r")
try:
parse_file(f)
finally:
f.close()
except CustomError as e:
overwrite_file(e)
except FileNotFoundError:
create_file()
except:
handle_errors()

What's the most clean and pythonic way to load data from one of two file paths? Why can't I catch two of the same Exceptions?

I have two potential file-paths where my application can read specific data from. If one fails, I want it to read from the other.
My intuitive attempt to do so was with try...except clauses, having something as below:
# Try the first file path
try:
file = open(possible_path_1)
content = file.read()
# File is not in first location, try the second
except IOError:
file = open(possible_path_2)
content = file.read()
# Could not read from either location, throw custom CriticalException
except IOError:
raise CriticalException("Could not read the file!")
However, this does not seem to work as intuitively expected. The second IOError is never caught. Why is this the case? Is there any "clean" way to read from one file path or the other without having to manually check os.path.exists(filepath) and os.path.isfile(filepath)?
Here's an alternative, but not sure if it is "prettier":
for path in paths:
try:
file = open(path)
content = file.read()
# File is not in first location, try the second
except IOError:
continue
break
else: # for-else gets executed if break never happens
raise CriticalException("Could not read the file!")
Assuming you have all your possible paths in some container, paths
Although honestly, I simply wouldn't use exception handling here, I think this is much clearer (and of course, I would use pathlib not os.path:
from pathlib import Path
for path in map(Path, paths):
if path.exists():
content = path.read_text()
break
else:
raise CriticalException("Could not read the file!")

Raising a FileNotFoundError

I have a function that takes a source file containing times (a csv file), reads it, then sorts the lines in order and writes them in a destination file. However, if the source csv file does not exist, I need to raise a FileNotFoundError. I've raised exceptions before, for example, if a parameter wasn't an integer I had to raise ChangeParameterError by using:
class ChangeParameterError(Exception):
pass
and then raising that in my function.
For my problem, my function is as follows:
def times(src,dst):
s = open(src,'r')
d = open(dst,'w')
lines = s.readlines()
s.close()
lines.sort()
for i in lines:
print((str(i).strip()), file = d)
d.close()
Any help is appreciated!
If the specified file is not found, the FileNotFoundError will be raised automatically by the open call when trying to open the file.
The exception is automatically raised by python. But you may need to wrap your open with a try-except to catch the exception without breaking your code:
try:
s = open(src,'r')
except FileNotFoundError:
print('file not found')

Categories

Resources