I am using pathlib.Path.rglob to browse through some files in a directory hierarchy. However one of the (sub-sub-...) directory cannot be parsed and an exception is thrown.
import pathlib
for file in pathlib.Path(r'\\path\to\shared\folder').rglob('*.foo'):
pass
# OSError: [WinError 59] An unexpected network error occurred
Catching the exception in the body of the for loop makes no sense as the exeption occurs in rglob. Putting the entire loop within a try block does catch the exception, but then I cannot resume the computation.
Does anybody have a clue about how to do that with pathlib.Path.rglob? A work around is to use good old os.walk but I would be interested in knowing a solution for this supposedly more modern tool.
First of all, try to run filePaths = Path(r'\\path\to\shared\folder').rglob('*.foo') in your python shell to see whether it completes without an error. If so, you can try to catch runtime-exceptions (probably due to unstable network connection) during the iteration.
def try_loop(g):
while True:
try:
yield next(g)
except StopIteration:
break
except OSError as e:
# log error
continue
Then wrap your rglob call in the try_loop function
for file in try_loop(pathlib.Path(r'\\path\to\shared\folder').rglob('*.foo')):
pass
This is inspired by a more general question "Handle an exception thrown in a generator".
Related
I am using python unittest and trying to catch the exceptions. I have tried with self.fail but in that case, once there is an exception, it is a failure and it stops running the rest of the code
What can I try so even if one case fails, it still executes the rest of the cases ?
I am trying to avoid printing the exceeptions.
Code currently using:
if 'Anonymous' in elem_welcome.text:
pass
else:
self.fail('Test Failed: Logout Failed'))
Use the try and except block and name the said 'error' on the except block
E.g. Error is 'NameError'
try:
#cases
except NameError:
#other cases
You can read more here Handling Exceptions
You can make every test to be a function and call it from the main block. This way each one will get executed and the failed once will return the fail reason to be filed by main block.
I'm using Pandas and Xlsxwriter to analyze some data, and then periodically create a Excel simple report for another users. The report should always get overwritten by the last one I export. Also, once I export it, it gets automatically opened to review it quickly.
The only issue I have is that I get an [Errno 13] if someone has left that file opened (or even if I'm using it, which I must admit happens).
IOError: [Errno 13] Permission denied: '2320PLFY2018.xlsx'
So i just wanted to add some Error handling to that IOError at the end of the XLSXWRITER part, to let me know about it, and give me a chance to close the file. Started with a simple attempt just to give me an error message. Tried several times but cant make it work. This is one attempt:
try:
writer.save()
os.startfile(company_code_choice + 'PLFY2018.xlsx')
except IOError as e:
print(e.errno)
Also tried without specifying the Error type:
while True:
try:
writer.save()
os.startfile(company_code_choice + 'PLFY2018.xlsx')
break
except:
print "Close the file!"
I'm new to creating code so I apologize if this is an obvious question...
I've been googling for this all morning and read several other questions but couldn't make it work on my code. Thanks in advance-
The best thing that I can think of that may cause this error is that one of the functions that you call inside your try statement is handling the exception inside itself so that your exception handling becomes ineffective.
To make sure:
Try to throw an exception manually from your try block so that you make sure the exception handling works in general.
If the aforementioned assumption is true you can figure out which function is handling the exception inside itself by manually raising an exception once before and once after the call.
To demonstrate what I mean:
while True:
sleep(1)
try:
raise Exception("my exception")
writer.save()
os.startfile(company_code_choice + 'PLFY2018.xlsx')
break
except Exception as e:
print ("Close the file: ", e)
You should definitely capture your raised exception otherwise, something is wrong with your code. Try to simply or check what prevents the excepted behavior if happened. If you could capture your raised exception then the next time:
while True:
sleep(1)
try:
writer.save()
raise Exception("my exception")
os.startfile(company_code_choice + 'PLFY2018.xlsx')
break
except Exception as e:
print ("Close the file: ", e)
So that you either capture your raised exception which means writer.save() does not handle the exception inside itself or you get the same error which means writer.save() handles the exception. By continuing this procedure all over the try block of your code you must be able to figure out which function is making trouble.
That's the best thing I can think of now. I hope it helps. But if I'm right then you could only figure out the source of the error but I don't know how you can achieve the behavior you would like. So maybe you can find a clue on the documentation.
In Python, is there any (proper) way to change the the default exception handling behaviour so that any uncaught exception will terminate/exit the program?
I don't want to wrap the entire program in a generic try-except block:
try:
// write code here
except Exception:
sys.exit(1)
For those asking for more specificity and/or claiming this is already the case, it's my understanding that not all Python exceptions are system-exiting: docs
Edit: It looks like I have forked processes complicating matters so won't be posting any specific details about my own mess.
If you're looking for an answer to the original question, Dmitry's comment is interesting and useful, references the 2nd answer to this question
You can use Specific exception instead of Exception because Exception is a Base class for all exceptions. For more details refer Exception tutorial
You can write your script like this-
try:
# write code here
except OverflowError:
raise SystemExit
except ArithmeticError:
sys.exit()
except IOError:
quit()
Try this different approaches to find what is exactly you are missing.
Edit 1 - Maintain Program Execution
In order to maintain your program execution try this one-
consider error_handler function is raising SystemExit exception then In your main method you need to add below code so you can maintain your program execution.
try:
error_handler()
except SystemExit:
print "sys.exit was called but I'm proceeding anyway (so there!-)."
For example i have a program with this structure:
Domain logic module -> Settings module -> Settings store backend
Next is a part of Settings module.
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise SettingsLoadDataException('Error loading json')
Need I a custom exception SettingsLoadDataException here, or I could just skip catching json.loads errors?
def load_from_json(self, json_str):
self.load_from_dict(json.loads(json_str))
Update.
Also good variant is:
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise ValueError('Error loading json')
That is a problem only you can answer. You could catch all exceptions, or you could let the program crash if it throws an exception you don't handle. If it is vital that the program doesn't crash, catch the exception. However, you should implement a recovery method then. If the Json doesn't load properly, can your program do anything useful without it ? If it can, I would catch the exception, otherwise you could just display an error and terminate.
You should work with exceptions in such a way, that seeing a stack trace explains the problem to you immediately.
I am no Python expert, but won't you loose the piece of information that it was actually ValueError, that caused program crash? You will see only SettingsLoadDataException in a trace without any real reason of it, right?
Also, if you do not rethrow exceptions, you should catch only those, you know how to deal with. It is always better to have your program crash, than to leave it in an unexpected state.
I have noticed me writing try-except clauses like the following very much in the past. The main reason for this is to write less code.
class Synchronizer(object):
# ...
def _assert_dir(self, dirname, argname, argnum):
""" *Private*. Raises OSError if the passed string does not point
to an existing directory on the file-system. """
if not os.path.isdir(dirname):
message = 'passed `%s` argument (%d) does not point to a ' \
'directory on the file-system.'
raise OSError(message % (argname, argnum))
def synchronize(self, source_dir, dest_dir, database):
# Ensure the passed directories do exist.
try:
self._assert_dir(source_dir, 'source_dir', 2)
self._assert_dir(dest_dir, 'dest_dir', 3)
except OSError:
raise
# ...
I was doing it this way, because otherwise I would've needed to write
class Synchronizer(object):
# ...
def synchronize(self, source_dir, dest_dir, database):
# Ensure the passed directories do exist.
if not os.path.isdir(source_dir):
message = 'passed `source_dir` argument (2) does not point to a ' \
'directory on the file-system.'
raise OSError(message)
if not os.path.isdir(dest_dir):
message = 'passed `dest_dir` argument (3) does not point to a ' \
'directory on the file-system.'
raise OSError(message)
# ...
I actually like the idea of writing methods doing check-and-raise operations, but I see one big disadvantage: Readability. Especially for editors that do code-folding, the try statement is not very much telling the reader what happens inside of it, while if not os.path.isdir(source_dir) is quite a good hint.
IMHO the try-except clause is required because it would confuse the catcher of the exception (reader of the traceback) where the exception comes from.
What do you think about this design? Is it awful, great or confusing to you? Or do you have any ideas on how to improve the situation?
There are two questions that I ask myself before using try for handling exceptional conditions and if the answer is YES to both, only then I will try to handle the exception.
Q1. Is this truly an exception scenario? I do not want to execute try blocks if the condition occurs 90% of the time. It is better to use if - else in such a case.
Q2. Can I recover from the error? It makes little sense to handle the exception if I cannot recover from it. It's better to propagate it to a higher level which happens automatically without me having to write extra code.
The code posted by you does not do anything to recover if the directory does not exist and it does not appear that you can do much about it. Why not let the error propagate to a higher level? Why do you even need a try block there?
This depends upon your requirement..
If you want to catch some exception, and continue with the code in your method, then you should use the 2nd scenario. Have yout try-except block inside your method.
def function():
try:
raise IOError
except IOError e:
// Handle
//continue with reset of the function
print "This will get printed"
function()
But if you want to handle all the exception at one place, with specific action for specific type, or you just want to halt your function, if one exception is raised, you can better handle them outside your function: -
def function():
raise IOError
// subsequent code Will not execute
print "This will not get printed"
try:
function()
except IOError e:
// Handle IOError
except EOFError e1:
// Handle EOF Error
By using the 2nd way, you are actually increasing the chance of some of your codes not getting executed. In general, your try-except block should be small. They should be separated for handling exception at different points and not all the exceptions should be handled at one place.
As far as I'm concerned, I generally like to minimize my try-except block as much as possible. That way I know where exactly my exception was raised.