Is this idiomatic/pythonic to do like this or is there a better way? I want all the errors to get in log for in case I don't have access to the console output. Also I want to abort this code path in case the problem arises.
try:
with open(self._file_path, "wb") as out_f:
out_f.write(...)
...
except OSError as e:
log("Saving %s failed: %s" % (self._file_path, str(e)))
raise
EDIT: this question is about handling exceptions in a correct place/with correct idiom. It is not about logging class.
A proven, working scheme is to have a generic except clause at the top level of your application code to make sure any unhandled error will be logged (and re-raised fo course) - and it also gives you an opportunity to try and do some cleanup before crashing)
Once you have this, adding specific "log and re-reraise" exception handlers in your code makes sense if and when you want to capture more contextual informations in your log message, as in your snippet example. This means the exception might end up logged twice but this is hardly and issue .
If you really want to be pythonic (or if you value your error logs), use the stdlib's logging module and it's logger.exception() method that will automagically add the full traceback to the log.
Some (other) benefits of the logging module are the ability to decouple the logging configuration (which should be handled by the app itself, and can be quite fine-grained) from the logging calls (which most often happen at library code level), the compatibility with well-written libs (which already use logging so you just have to configure your loggers to get infos from 3rd-part libs - and this can really save your ass), and the ability to use different logging mechanisms (to stderr, to file, to syslog, via email alerts, whatever, and you're not restricted to a single handler) according to the log source and severity and the deployment environment.
Update:
What would you say about re-raising the same exception (as in example) or re-raising custom exception (MyLibException) instead of original one?
This is a common pattern indeed, but beware of overdoing it - you only want to do this for exceptions that are actually expected and where you really know the cause. Some exception classes can have different causes - cf OSError, 'IOErrorandRuntimeError- so never assume anything about what really caused the exception, either check it with a decently robust condition (for example the.errnofield forIOError`) or let the exception propagate. I once wasted a couple hours trying to understand why some lib complained about a malformed input file when the real reason was a permission issue (which I found out tracing the library code...).
Another possible issue with this pattern is that (in Python2 at least) you will loose the original exception and traceback, so better to log them appropriately before raising your own exception. IIRC Python3 has some mechanism to handle this situation in a cleaner way that let you preserve some of the original exception infos.
Related
I created a big program that does a lot of different stuff. In this program, I added some error management but I would like to add management for critical errors which should start the critical_error_function().
So basically, I've used :
try :
//some fabulous code
except :
critical error(error_type)
But I am here to ask if a better way to do this...
In Python exceptions are the intended way of error handling. Assuming you wrap your whole program in one try-except block, a better way would be to
only try-except-wrap the lines that can generate exceptions instead of your complete program
catch them with a specific exception such as ValueError or even your own custom exception instead of the blank except statement
handle them appropriately. Handling could mean skipping this value, logging the error or calling your critical_error_function.
I use the scrapy library for scraping and it handels all the logging for me. The log file is very big and I was hoping for another way than to read through the whole file. I'm only interested in the exception.
Is it posible to get the raw logging.exception and read from that without creating another log file?
Use the python logging facility which is what Scrapy uses. At the program entry point you need to customize this logging facility, it can be in the __init__.py file for example or anything else before you use any log related calls.
What you need to do is set different handlers for different levels (ERROR, WARNING, ...). Exception level do not exist, you might be mistaking with Error or Critical levels.
This is the "Logging How To", you should check it.
Your solution would look like :
import logging
#this is the logger Scrapy uses throughout the app.
logger = logging.getLogger()
my_exception_handler = logging.FileHandler()
my_exception_handler.setLevel(Logging.ERROR)
logger.addHandler(my_exception_handler)
Finally, if you are really talking about exception and not error or critical levels. You'll need to inherit from your Exception class or overwrite it so you can use the same logging approach I just explained.
What's the point of using raise if it exits the program?
Wouldn't it be just as effective to allow the crash to happen?
If I leave out the try-except block, the function crashes when I divide by zero and displays the reason. Or is there some other use that I don't know about?
def div(x,y):
try:
return(x/y)
except ZeroDivisionError as problem:
raise (problem)
I your case effect would be the same. But you may want to perform some additional logic in case of error (cleanup etc.) and perhaps raise a different (perhaps custom) error instead of original system low-level one, like with a message "Incorrect data, please check your input". And this can be done catching the error and raising a different one.
There is no point (in this case) in using raise. Normally, you'd have some code in there to do "something else" - that could include outputting some more debug information, writing some log data out, retrying the operation with a different set of parameters, etc. etc. etc.
I'm not sure there's much value in your case, where when an exception occurs it just re-raises it - it seems like someone (perhaps) intended to write some sort of handling code there, but just never got around to it.
Some great examples of the use cases for exception handling are in the Python Exception Handling Wiki --> http://wiki.python.org/moin/HandlingExceptions
The reason to re-raise an exception is to allow whatever code is calling you the opportunity to handle it after you have done something to handle it yourself. For example, you have closed a file that you were using (because cleanliness is a virtue) but your code cannot continue.
If you are not going to do anything to handle the exception, then no, there is no reason to write an exception handler for it!
The correct way to re-raise an exception is to simply use raise without any arguments. This way, whoever catches the exception (or the user of the script, if nobody catches it) gets a correct stack trace that tells where the exception was originally raised.
What is my requirement ?
--> I need Exception notifier which will email to some specific configured user, about any sort of exception occurring in plain python app and web.py.
I want something similar to this http://matharvard.ca/posts/2011/jul/31/exception-notification-for-rails-3/
Is there anything same sort present ??
Please reply asap.
Thanks.
You can get what you want by:
Wrapping your code in try..except clause.
Using logging module to log the exceptions with a certain level of severity e.g ERROR.
Setting an SMTPHandler for exceptions of and above certain level.
This way is quite flexible. Your messages can be send to several places (like log files) and you can reconfigure your settings easily.
If you are not using any python heavy weight framework, try: https://github.com/fossilet/exception-notifier , it seems to be similar to the Rails' Exception notification, but quite simple.
If you are using django, seems you can use its built-in feature:https://docs.djangoproject.com/en/dev/howto/error-reporting/ ( and also see this: https://github.com/pinax/django-notification)
If using tornado, try this: https://github.com/kanevski/tornadotoad this is the most similar solution in python comparing to rails. )
You can overwrite the excepthook function from the sys module, and handle any uncought exceptions there.
I have the same requirement with you. I write a simple module to mail uncaught exceptions to the developers, as well as to record them to log files. It is used in our teams' cron scripts written in Python. Hope it can be useful for you too.
Take Django for instance, in manage.py:
try:
import settings
except ImportError:
sys.stderr.write("Error: Can't find the file 'settings.py'...")
Seems legit, but what happens when settings imports non_existant_lib_foo?
Well, you're sent on a goose chase for all the possible things you could have done with PATH, etc.
Of course you can use except ImportError as e: and just print out the actual error message, but what if you want to catch only a specific error and give a really good bit of advice, like above?
You're left to using regexp more or less or, at best, guessing that the "right" module failed importing, and then display the message.
Is there a better way to handle this?
Basically the problem is that your error message is wrong - there are other reasons why import can fail, not just wrong paths. ImportError simply means "you cannot use this module, look at the traceback to find out why", you just jumped to a conclusion.
If you want to display "Can't find the file" then you should look for the file first. imp.find_module does that.
The simplest way to handle cases where you don't want to let the exception propagate, but still provide some indication of the failure, is to always include the details from the original exception in whatever message you print out:
try:
import settings
except ImportError as exc:
sys.stderr.write("Error: failed to import settings module ({})".format(exc))
That way, you get the clear message in the common case (e.g. the user put their settings file in the wrong place, or there is a configuration error somewhere so sys.path is wrong), without completely obscuring any other import errors that arise in the execution of the settings module.
Even better, if using the logging module, is to log the full exception details (including the traceback) as a debug() or info() logging event.
There are other options, such as doing a substring search if 'settings' not in str(exc): raise or separating the location of the module from its execution by using imp.find_module(), but simply including the original error message is enough to prevent the worst hassles.
One thing I've found really frustrating is that when you trap an import error you're really going down a deep rabbit hole right away.
Take the hint.
Do not trap import errors ever. The manage.py example is not a best practice.
Do not do that.
import errors should be very, very rare.