I want to detect errors in a standalone Python script with Sentry+Raven.
I tried to configure it and raven test ... is workging.
Then I place this on top of the script:
from raven import Client
client = Client('http://...#.../1')
client.captureException()
the exception is generated later on this:
import django
django.setup()
from django.conf import settings
And I want to see the actual stack for this error:
ImportError: Could not import settings 'settings' (Is it on sys.path? Is there an import error in the settings file?): No module named 'settings'
But all I see in Sentry is
which is completely useless.
How can I change this to have a normal traceback?
You misunderstand how client.captureException() works, its not a configuration parameter. You use it when you are catching an exception and it will capture the exception type and message:
try:
f = open('oogah-boogah.txt')
except IOError:
client.captureException()
# do something here
To capture any exceptions that could be generated in a block of code, you can use capture_exceptions:
#client.capture_exceptions
def load_django():
import django
django.setup()
from django.conf import settings
Yes you're right, but is there a way to catch an exception not
wrapping a block of code in a try-except. I can see the error in a
terminal, can I see it in Sentry?
There is a default exception handler - and when an exception is not caught, this default handler catches it and then displays the exception. This is what you see in the terminal.
The function that generates this output is sys.excepthook and it will output to stderr by default.
So, in order for you to catch all exception globally, you'll have to create a global exception handler or map your own function to sys.excepthook.
I would strongly recommend against this, though as you don't know what other side effects it may have.
Related
Code for celery task:
import logging
from celery_once import QueueOnce
from celery import shared_task, current_task
from test.set_data import set_data_in_db
logger = logging.getLogger("celery")
#shared_task(base=QueueOnce, once={"graceful": True}, ignore_result=False)
def set_data_task():
try:
logger.info("Set data in db initiated")
set_data_in_db(value=None)
except Exception:
logger.error("data could not be set", exc_info=True)
My unittest case is covering everything which is in the try block. How can I force my unittest to cover except block as well ?
When you call the method set_data_task , there could be only one situation out of two, either method executes normally (try block) or it throw some exception (except block).
If you need to test your except block , you need to configure your method to throw exception so that it could be caught and assert in your test
You will be in need to configure set_data_in_db method to throw exception to be caught
For testing exception, you can read pytest.raises
I am currently debugging a Django project which results in an exception. I would like to enter the ipdb post-mortem debugger. I've tried invoking ipdb as a script (cf. https://docs.python.org/3/library/pdb.html), but this just enters me to the first line of code:
> python -m ipdb manage.py runserver
> /Users/kurtpeek/myproject/manage.py(2)<module>()
1 #!/usr/bin/env python
----> 2 import os
3 import sys
ipdb>
If I press c to continue, I just run into the error, with no possibility to drop into the debugger post-mortem. Presumably I could press n (next) until I get the error, but that would be quite cumbersome.
Is there a way to run python manage.py runserver with post-mortem debugging?
If you know of a line that causes the exception, but don't know how "deep" inside it the exception is caused, you can get a post-mortem debugger for it by catching the exception and calling ipdb.post_mortem() in the exception handler.
For example, change your code from this:
def index(request):
output = function_that_causes_some_exception()
return HttpResponse(output)
To this:
def index(request):
try:
output = function_that_causes_some_exception()
except:
import ipdb
ipdb.post_mortem()
# Let the framework handle the exception as usual:
raise
return HttpResponse(output)
By the way, for server frameworks that could be spewing stuff in the console from other threads I highly recommend wdb, so that you can debug your django app from the comfort of a browser:
def index(request):
try:
output = function_that_causes_some_exception()
except:
import wdb
wdb.post_mortem()
# Let the framework handle the exception as usual:
raise
return HttpResponse(output)
I'm a newbie in Flask and I am trying to display the Built-In Exceptions in python but I can't seem to have them display on my end.
NOTE:
set FLASK_DEBUG = 0
CODE:
def do_something:
try:
doing_something()
except Exception as err:
return f"{err}"
Expectation:
It will display one of the built-in exceptions:
KeyError
IndexError
NameError
Etc.
Reality:
It will return the line of code that didn't worked which is more ambiguous to the end user.
Also:
I have no problem seeing the errors when the debug mode is ON but that's not something that I want to do if I open them in public
Flask supplies you with a function that enables you to register an error handler throughout your entire app; you can do something as shown below:
def handle_exceptions(e):
# Log exception in your logs
# get traceback and sys exception info and log as required
# app.logger.error(getattr(e, 'description', str(e)))
# Print traceback
# return your response using getattr(e, 'code', 500) etc.
# Exception is used to catch all exceptions
app.register_error_handler(Exception, handle_exceptions)
In my honest opinion, this is the way to go. - Following the structure found in werkzeug.exceptions.HTTPException as an example is a solid foundation.
Having a unified exception handler that will standardise your Exception handling, visualisation and logging will make your life a tad better. :)
Try with this:
def do_something:
try:
doing_something()
except Exception as err:
return f"{err.__class__.__name__}: {err}"
I'd like to keep a solid logging system going, but it's also necessary to raise exceptions. This code accomplishes what I'm going for, but it looks clunky and not very Pythonic. What's a better option?
import logging
if not condition_met:
missing = set_one - set_two
logging.error('Missing keys: {}'.format(missing))
raise ValueError('Missing keys: {}'.format(missing))
you could catch the exception and log the error at this time, so if another exception occurs you can log it as well, and propagate the exception upstream.
try:
# some code
if not condition_met:
missing = set_one - set_two
raise ValueError('Missing keys: {}'.format(missing))
except Exception as e: # or ValueError to narrow it down
logging.error(str(e))
raise # propagate the exception again
note than logging an exception without logging the traceback leaves something unfinished, specially if the exception is caught and handled upstream. It's very likely that you're never going to fix that particular error.
You can use logger's exception() function:
from logger import exception
try:
. . .
except Exception as error:
exception(msg="Your message")
so that all of the stack will be logged.
You can read an interesting article about this here.
Another elegant approach is to define custom exceptions for your application that serve the purpose of clarifying lower-level exceptions such as KeyError as well as centralizing error logic. These custom exceptions can be defined on a separate file to make maintenance and updates easier. custom exceptions are derived from a base Error class to inherit global settings which itself is derived from the built-in Exception class.
exceptions.py
from utils import log
class Error(Exception):
"""base class for errors"""
class EnvironmentAttributeError(Error):
"""
Exception raised when environment variables are empty strings
Attributes:
key_attribute -- environment variable
"""
def __init__(self, environment_variable):
self.environment_variable = environment_variable
self.message = f"Environment variable value for key {environment_variable} was not assigned."
self.log = log.logger.error(f"Environment variable value for key {environment_variable} was not assigned.")
super().__init__(self.message)
class EnvironmentKeyError(Error):
"""
Exception raised when the environment variables dict does not have required keys
Attributes:
key_attribute -- environment variable
"""
def __init__(self, vars):
self.environment_variable = vars
self.message = f"Environment variable {vars} was not declared."
self.log = log.logger.error(f"Environment variable {vars} was not declared.")
super().__init__(self.message)
Notice that the exceptions.py file imports a log utility. That way all you need to do elsewhere in your code is raise the right custom code errors and everything gets logged for you. You can then update these errors in a single place for your entire project.
log.py
import logging
# instantiate a logger object writing to connected-apps.log
logging.basicConfig(
format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
datefmt='%Y-%m-%d:%H:%M:%S',
level=logging.DEBUG,
filename='logs/connected_apps.log'
)
# logger object named after module: https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial
logger = logging.getLogger(__name__)
The logger in the log.py file has been formatted in such a way that logs are both descriptive and readable. You can even define different loggers with different formats and levels.
Here is a simple use of the custom exceptions defined above. Environment variables obtained from .env are sent to this validate() function to verify that the right keys and attributes are available. Notice that we just needed to import exceptions and not logs:
environment.py
from utils import exceptions
def validate(env_dict, env_vars):
# check that each environment variable has been declared and assigned
for vars in env_vars:
try:
# check the local dictionary pulled from os.environ
env_dict[vars]
# check that key value length is non-zero
if len(env_dict[vars]) == 0:
raise exceptions.EnvironmentAttributeError(vars)
except KeyError as error:
# raises error if an environment variable has not been declared
raise exceptions.EnvironmentKeyError(vars)
NOTE: This question was based on an assumption that Python emits its error messages via logging. The answers show that the assumption is wrong.
I'm developing a program that is not started from a command line, but by a daemon. Stderr is redirected to null device.
Normally it logs messages to a file, but when some error is preventing a regular start, there is no error message to read, because it was sent to the null device.
To save a little debugging time in such case I tried a little "launcher" which adds a file handler to the root logger as the very first thing.
I have tested it with a deliberate syntax error in the realprog module. It logs the two "start" messages to the file, but the traceback from the syntax error is still printed to stderr. Could you please help?
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler('test.log'))
logger.info("logging start")
def real_start():
# assume e.g. a syntax error in the realprog
import realprog
realprog.main()
if __name__ == '__main__':
logger.info("program start")
real_start()
You could use:
if __name__ == '__main__':
logger.info("program start")
try:
real_start()
except Exception:
# This will log the traceback.
logger.exception("An error ocurred.")
However, you should increase your logger level to, at least, logging.ERROR.
Hope it helps!
You are not passing your exception to your logger, so there is no way for it to write it.
inside your real_start(), put your import statement and your function call in a try, catch and then log the exception.
Lets say for example your realprog.main() divides a number by zero, I want to log the exception, so I do this.
def real_start():
try:
import realprog
realprog.main()
except ZeroDivisionError as e:
logger.info(e, exc_info=True)
If you check your file, you should have the exception inside it.