Folks,
I've resigned myself to working around this problem, but I wanted to check if Python is really acting as expected.
In the example, "sample.txt" is any multi-line text file that is read and parsed.
try:
file=open('sample.txt','r')
for line in file:
(some action here)
except:
print "Couldn't open file"
file.close()
The actions I observe are that "sample.txt" is opened and the first line handled, the logic then falls through to the "except" clause.
WAD or is this a bug?
If the code in the except block runs it is because an exception was raised. You swallow the exception which makes it hard to know what's going wrong.
Your error message suggests that you are attempting to trap errors raised in the file open. But since your try block surrounds the entire processing of the file, exceptions raised in the processing, as opposed to the file open, will be mis-reported as "Could not open file". If you really must handle the exception then you need to move the for loop to be after the except block.
Personally I'd be inclined to simply ignore the exception and let the default handling of the exception halt execution:
with open('sample.txt','r') as file:
for line in file:
(some action here)
If you must handle the exception then be discerning about the class exception that you handle. For example, handle just IOError since that is what open raises in case of failure.
try:
with open('sample.txt','r') as file:
for line in file:
(some action here)
except IOError:
except IOError as (errno, strerror):
print "I/O error({0}): {1}".format(errno, strerror)
It's not failing on the open line, then. What's the exception?
try:
file=open('sample.txt','r')
for line in file:
(some action here)
except:
print "Exception:"
import traceback; traceback.print_exc()
file.close()
That bare except catches all exceptions, including ones in the (some action here) part. Restructure that as:
try:
inputfile = open('sample.txt', 'r')
except:
print "Couldn't open file"
else:
for line in inputfile: pass
inputfile.close()
Or even better:
with open('sample.txt', 'r') as inputfile:
for line in inputfile: pass
In general, only wrap the bare minimum amount of code possible inside a try block so that you're not accidentally handling exceptions you're not really prepared to deal with.
your code risks raising another assertion trying to close file if for some reason the open() call fails. This is because the file variable won't be set if open() raises an exception, and so your call further down will reference a variable that doesn't exist.
If possible, try using the with statement:
with open('sample.txt', 'r') as file:
try:
for line in file:
(some action)
except:
print "Exception"
import traceback; traceback.print_exc()
This will make sure that file is closed afterwards regardless of what happens inside the with statement.
Try using readlines to generate a list of all lines in the file. And you also shouldnt be catching general errors without at least printing the error code.
try:
file=open('sample.txt','r')
for line in file.readlines():
(some action here)
except Exception, e:
print str(e)
file.close()
The actions I observe are that "sample.txt" is opened and the first line handled, the logic then falls through to the "except" clause.
Yes, this is the case. But this only occurs as long as the file exists.
As for the error falling through to the except clause (assuming the file exists), that implies that there is an issue with the parsing logic you've implemented. We can't be sure what it is since the except catches everything, and unless it's re-raised (may as well not catch it then...), or you print the stack trace from the exception object, you can't tell what is going wrong or where. In general, this is why catching everything is frowned upon; it makes debugging unnecessarily difficult.
I also noticed that you are closing the file at the very end. This is probably another source of errors, since the file only exists within the scope of try. You have two options:
Include file.close() inside of your try block so the file is properly closed after you're done, or
Use the with statement as a context manager to automatically close the file when you're done. #David Heffernan's example is one that's similar to one I would write.
Related
I am using pathlib.Path.rglob to browse through some files in a directory hierarchy. However one of the (sub-sub-...) directory cannot be parsed and an exception is thrown.
import pathlib
for file in pathlib.Path(r'\\path\to\shared\folder').rglob('*.foo'):
pass
# OSError: [WinError 59] An unexpected network error occurred
Catching the exception in the body of the for loop makes no sense as the exeption occurs in rglob. Putting the entire loop within a try block does catch the exception, but then I cannot resume the computation.
Does anybody have a clue about how to do that with pathlib.Path.rglob? A work around is to use good old os.walk but I would be interested in knowing a solution for this supposedly more modern tool.
First of all, try to run filePaths = Path(r'\\path\to\shared\folder').rglob('*.foo') in your python shell to see whether it completes without an error. If so, you can try to catch runtime-exceptions (probably due to unstable network connection) during the iteration.
def try_loop(g):
while True:
try:
yield next(g)
except StopIteration:
break
except OSError as e:
# log error
continue
Then wrap your rglob call in the try_loop function
for file in try_loop(pathlib.Path(r'\\path\to\shared\folder').rglob('*.foo')):
pass
This is inspired by a more general question "Handle an exception thrown in a generator".
Assume I'm going to write a Python script that catches the KeyboardInterrupt exception to be able to get terminated by the user using Ctrl+C safely
However, I can't put all critical actions (like file writes) into the catch block because it relies on local variables and to make sure a subsequent Ctrl+C does not break it anyway.
Would it work and be good practice to use a try-catch block with empty (pass) try part and all the code inside the finally part to define this snippet as "atomic, interrupt-safe code" which may not get interrupted mid-way?
Example:
try:
with open("file.txt", "w") as f:
for i in range(1000000):
# imagine something useful that takes very long instead
data = str(data ** (data ** data))
try:
pass
finally:
# ensure that this code is not interrupted to prevent file corruption:
f.write(data)
except KeyboardInterrupt:
print("User aborted, data created so far saved in file.txt")
exit(0)
In this example I don't care for the currently produced data string, i.e. that creation could be interrupted and no write would be triggered. But once the write started, it must be completed, that's all I want to ensure. Also, what would happen if an exception (or KeyboardInterrupt) happened while performing the write inside the finally clause?
Code in finally can still be interrupted too. Python makes no guarantees about this; all it guarantees is that execution will switch to the finally suite after the try suite completed or if an exception in the try suite was raised. A try can only handle exceptions raised within its scope, not outside of it, and finally is outside of that scope.
As such there is no point in using try on a pass statement. The pass is a no-op, it won't ever be interrupted, but the finally suite can easily be interrupted still.
You'll need to pick a different technique. You could write to a separate file and move that into place on successful completion; the OS guarantees that a file move is atomic, for example. Or record your last successful write position, and truncate the file to that point if a next write is interrupted. Or write markers in your file that signal a successful record, so that reads know what to ignore.
In your case, there is no problem, because file writes are atomic, but if you have some file object implementetion, that is more complex, your try-except is in the wrong place. You have to place exception handling around the write:
try:
f.write(data)
except:
#do some action to restore file integrity
raise
For example, if you write binary data, you could to the following:
filepos = f.tell()
try:
f.write(data)
except:
# remove the already written data
f.seek(filepos)
f.truncate()
raise
I give following code snippet,
As at the end of the code I am getting blank output file
in with context when exception is raised The file is closed and again overridden in next iteration
with open('output', 'w') as f:
try:
for i in range(1, 100):
if i % 2 == 0:
f.write('%d \n' % i)
else:
raise Exception()
except Exception as e:
pass
Is my understanding correct?
If so, Why this behavior is there?As I am handling the exception.
Is it right that with statement will always close files
whenever exception is raised in side block.
What could be possible solution using with statement?
When using a try/except block, the try block is not continued upon completion of the except block.
A possible solution would be to replace the raise Exception() statement - which is currently raising a meaningless exception - with a pass statement instead.
In fact, you should probably do a little reading regarding when to use exceptions.
Python newbie here and am running into some weird behavior in my code.
I am trying to write some data to a file. I print the length of the data to be about 50k before I call the following block of code. The data is a pdf file I got over the internet. And its a valid pdf.
When I call the function F() described below, I get the exception message printed in function F and not in the actual place it fails.
In the code below, in the function write_to_disk() I see the second print and the execution directly jumps to the exception handler in the calling function F(). I cannot figure out why this is happening. On disk I see the file is created but the size is 0.
Can some look at the code below and may be guess what could be happening?
If I am catching exceptions in the write_to_disk() function how is it possible for it to jump out of the function completely?
EDIT: Thanks for kobejohn, turns out the excetion object does not have a errno variable. Getting rid of it made the print appear. But the bigger problem still exists. I see a failure with no way to find out why its failing. How do I get the error message here?
def write_to_disk(self, pathToWrite, pdfFileData):
try:
print 'Here `1.1'
fd = open(pathToWrite, "w+")
print 'Here `2.1'
fd.write(pdfFileData)
print 'Here 3.1'
fd.close()
except Exception as e:
print 'file cannot be opened ' + pathToWrite + e.errno
This function is inturn called by another function F which is like this -
def F(self, url):
pathWrite = get_path_to_use()
pdfData = get_pdf_data(url)
try:
writetodisk(pathToWrite, pdfData)
except Exception as e:
print 'Did I jump directly to here?' + e.errno
Here is the out put of the program. I did not think it will add anything because I see nothing if any use. In fact I get the same output even when running it in pdb.
Here `1.1
Here `2.1
Did I jump directly to here?
Your first exception handler tries to build a string by concatenating another string and an int (e.errno) which causes it (the print statement) to throw an exception itself (which is then caught by the outer exception handler).
It's just bubbling as we mentioned in the comments and Alexander said. Use this code to see how it can work (no errors, but that was just a nasty surprise you got with exceptions).
def f(url):
path_to_write = 'test.dat'
pdf_data = 'asdf'
try:
write_to_disk(path_to_write, pdf_data)
except Exception as e:
print 'Did I jump directly to here?\n' + str(e)
def write_to_disk(path_to_write, pdf_data):
try:
print 'Here `1.1'
with open(path_to_write, "w+") as fd:
print 'Here `2.1'
fd.write(pdf_data)
except Exception as e:
print 'file cannot be opened ' + path_to_write
f('fake_url')
Some areas to make your code safer / more standard:
do as little as possible inside a try block. try to isolate the code that you are worried might raise an error
same rule for the except block. don't do anything strange there.
as someone else mentioned, using a with block is a more standard and readable way to work with a file.
other minor things with function and variable names you can see where I have changed things in the code above. google PEP 8 for more.
I have noticed me writing try-except clauses like the following very much in the past. The main reason for this is to write less code.
class Synchronizer(object):
# ...
def _assert_dir(self, dirname, argname, argnum):
""" *Private*. Raises OSError if the passed string does not point
to an existing directory on the file-system. """
if not os.path.isdir(dirname):
message = 'passed `%s` argument (%d) does not point to a ' \
'directory on the file-system.'
raise OSError(message % (argname, argnum))
def synchronize(self, source_dir, dest_dir, database):
# Ensure the passed directories do exist.
try:
self._assert_dir(source_dir, 'source_dir', 2)
self._assert_dir(dest_dir, 'dest_dir', 3)
except OSError:
raise
# ...
I was doing it this way, because otherwise I would've needed to write
class Synchronizer(object):
# ...
def synchronize(self, source_dir, dest_dir, database):
# Ensure the passed directories do exist.
if not os.path.isdir(source_dir):
message = 'passed `source_dir` argument (2) does not point to a ' \
'directory on the file-system.'
raise OSError(message)
if not os.path.isdir(dest_dir):
message = 'passed `dest_dir` argument (3) does not point to a ' \
'directory on the file-system.'
raise OSError(message)
# ...
I actually like the idea of writing methods doing check-and-raise operations, but I see one big disadvantage: Readability. Especially for editors that do code-folding, the try statement is not very much telling the reader what happens inside of it, while if not os.path.isdir(source_dir) is quite a good hint.
IMHO the try-except clause is required because it would confuse the catcher of the exception (reader of the traceback) where the exception comes from.
What do you think about this design? Is it awful, great or confusing to you? Or do you have any ideas on how to improve the situation?
There are two questions that I ask myself before using try for handling exceptional conditions and if the answer is YES to both, only then I will try to handle the exception.
Q1. Is this truly an exception scenario? I do not want to execute try blocks if the condition occurs 90% of the time. It is better to use if - else in such a case.
Q2. Can I recover from the error? It makes little sense to handle the exception if I cannot recover from it. It's better to propagate it to a higher level which happens automatically without me having to write extra code.
The code posted by you does not do anything to recover if the directory does not exist and it does not appear that you can do much about it. Why not let the error propagate to a higher level? Why do you even need a try block there?
This depends upon your requirement..
If you want to catch some exception, and continue with the code in your method, then you should use the 2nd scenario. Have yout try-except block inside your method.
def function():
try:
raise IOError
except IOError e:
// Handle
//continue with reset of the function
print "This will get printed"
function()
But if you want to handle all the exception at one place, with specific action for specific type, or you just want to halt your function, if one exception is raised, you can better handle them outside your function: -
def function():
raise IOError
// subsequent code Will not execute
print "This will not get printed"
try:
function()
except IOError e:
// Handle IOError
except EOFError e1:
// Handle EOF Error
By using the 2nd way, you are actually increasing the chance of some of your codes not getting executed. In general, your try-except block should be small. They should be separated for handling exception at different points and not all the exceptions should be handled at one place.
As far as I'm concerned, I generally like to minimize my try-except block as much as possible. That way I know where exactly my exception was raised.