Develop Reference

Why do I have to import traceback if it already exists?

If I write something in Python and things go awry, I automatically get a traceback.
For example:
#!/usr/bin/env python
print("this will raise a division by zero exception")
print(2/0)
It automatically throws a traceback with the exception
>>> %Run test.py
this will raise a division by zero exception
Traceback (most recent call last):
File "/home/pi/Desktop/test.py", line 4, in <module>
print(2/0)
ZeroDivisionError: division by zero
>>>
Now, let's assume that I don't want the program to crash-and-burn, but I want to handle the exception gracefully - say I need to close files or whatever.
Viz.:
#!/usr/bin/env python
import sys
try:
print("Assume I'm doing something with a resource, like a file")
print(". . . and something goes wrong\n")
print("(This will raise a division by zero exception when I try to \"print(2/0)\")\n")
print(2/0)
except BaseException as e:
print(". . . and I want to handle the exception gracefully:\n")
print("Oops! Something happened! (",e, ")\n")
sys.exit(0)
Which gets me the expected:
>>> %Run test.py
Assume I'm doing something with a resource, like a file
. . . and something goes wrong
(This will raise a division by zero exception when I try to "print(2/0)")
. . . and I want to handle the exception gracefully:
Oops! Something happened! ( division by zero )
─────────────────────────────────────────────────────────────────────────────────────────────────
Python 3.7.3 (/usr/bin/python3)
>>>
However, if I try to handle the exception manually, I am told that in order to get the system provided traceback, (the call stack and location of the error), I have to import traceback.
When I do that, I can get all the information I want.
#!/usr/bin/env python
import sys
import traceback
try:
print("Assume I'm doing something with a resource, like a file")
print(". . . and something goes wrong\n")
print("(This will raise a division by zero exception when I try to \"print(2/0)\")\n")
print(2/0)
except BaseException as e:
print(". . . and I want to handle the exception gracefully:\n")
print("Oops! Something happened! (",e, ")\n")
traceback.print_exc()
sys.exit(0)
And the expected result:
────────────────────────────────────────────────────────────────────────────────────────────────
Python 3.7.3 (/usr/bin/python3)
>>> %Run test.py
Assume I'm doing something with a resource, like a file
. . . and something goes wrong
(This will raise a division by zero exception when I try to "print(2/0)")
. . . and I want to handle the exception gracefully:
Oops! Something happened! ( division by zero )
Traceback (most recent call last):
File "/home/pi/Desktop/test.py", line 9, in <module>
print(2/0)
ZeroDivisionError: division by zero
─────────────────────────────────────────────────────────────────────────────────────────────────
Python 3.7.3 (/usr/bin/python3)
>>>
"Traceback" appears to already exist as the traceback is automatic on unhandled exceptions
Why do I have to import traceback if I use a try/except block, but don't if I don't?

The traceback reflects the state of the Python Interpreter at a given point during the execution of some code. The Python Interpreter obviously has access to its own state, and shares it with the user, by printing that error message, when an exception is thrown and not handled. It also gives the code access to it – via the traceback module, which is why you need to import it.

The default behavior of Python when an exception is raised that isn't caught in a try...except is to print a traceback and exit. If you do catch it, then the default doesn't happen. You have to decide what to do with it. If you decide to print the traceback you now have to explicitly do it by importing the traceback module and using the methods there. However, that's not strictly necessary as you could just inspect the current traceback object (from sys.exc_info()), and write your own formatter.

"Traceback" appears to already exist as the traceback is automatic on unhandled exceptions
What you're seeing is the default behavior of sys.excepthook, which is to print a traceback out to stderr anyway, just before the program exits from an unhandled exception.
See for yourself with:
# example.py
import sys
sys.excepthook = lambda *args: None
print("before")
1/0
print("after")
The script above will exit non-zero, but no traceback will be printed since the hook was replaced with a no-op. And "after" will not get printed, because the process will have already exited. So, it is not quite that a traceback is "automatic" on unhandled exceptions, but rather that is the default configuration.
By calling sys.exit(0) you're intentionally suppressing the ZeroDivisionError exception. The interpreter will exit with a zero return code, and as far as the except hook is concerned you have "handled" the ZeroDivisionError and are opting out of the traceback printing (src).

The traceback exists, and the interpreter can use it (e.g. print it) , but you can only use it if you import it. If you do not use a try/except block you do not have a chance to import it.

How to log a Python 3 exception, but without its stack trace?

When I want to log some specific Exception, but otherwise ignore it, I can do that like so:
import logging
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
logger.error("It failed with:", exc_info=True)
(This is in fact an MRE, as something_that_may_fail hasn't been defined, so the try block will raise NameError with message name 'something_that_may_fail' is not defined. 😉)
This however will also log the stack trace:
It failed with:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
NameError: name 'something_that_may_fail' is not defined
Sometimes that isn't what I want: In some cases, I already know that exception type and exception message (together with my custom log message) will suffice, and don't want to expand the log with stack traces that don't tell me anything new. So I'd want a log entry that simply is
It failed with:
NameError: name 'something_that_may_fail' is not defined
I can achieve that by passing a 3-tuple as the exc_info, with the stack trace component replaced by None:
import logging
import sys
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
exc_type, exc_value, _trace = sys.exc_info()
logger.error("It failed with:", exc_info=(exc_type, exc_value, None))
But I'm not sure how reliable that is. (The documentation doesn't mention how the tuple may or may not deviate from one returned by sys.exc_info().)
Examining the exception myself with
...
except NameError as e:
...
comes with its own problems:
f"{type(e)}" gives string <class 'NameError'> instead of just string NameError
The proper solution to get the fully qualified type name, including packages/modules but without builtin. is rather unwieldy and not something I'd want in exception handling code. See the currently accepted answer to Get fully qualified class name of an object in Python.
Can I rely on the message always being e.args[0]? (I might have uses for other exceptions (with more sub-types) than just NameError, which I've used here only as an example.)
So what is the proper way to log exception type and message without the stack trace? Is there a cleaner way than my make-the-trace-None hack above?

traceback.format_exception_only can be used for that:
import logging
import sys
import traceback
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
exc_type, exc_value, _trace = sys.exc_info()
exc_desc_lines = traceback.format_exception_only(exc_type, exc_value)
exc_desc = ''.join(exc_desc_lines).rstrip()
logger.error(f"It failed with:\n{exc_desc}")
or without sys:
import logging
import traceback
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError as e:
exc_desc_lines = traceback.format_exception_only(type(e), e)
exc_desc = ''.join(exc_desc_lines).rstrip()
logger.error(f"It failed with:\n{exc_desc}")
(Found this by looking how the logging module actually extracts and formats information from exc_info. There traceback.print_exception is being used, so I looked what else is available in the traceback module.)

Python: How to write error in the console in txt file?

I have a python script which every 10 minutes sends me an email with everything written in the console. I am running this with the crontab in my ubuntu 18.04 vps.
Sometimes it doesn't send the mail so I assume that when an error happens execution stops but how can I get the errors to be written in a txt file so I can analyze the error ?

Logging Module
To demonstrate the approach with the logging module, this would be the general approach
import logging
# Create a logging instance
logger = logging.getLogger('my_application')
logger.setLevel(logging.INFO) # you can set this to be DEBUG, INFO, ERROR
# Assign a file-handler to that instance
fh = logging.FileHandler("file_dir.txt")
fh.setLevel(logging.INFO) # again, you can set this differently
# Format your logs (optional)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter) # This will set the format to the file handler
# Add the handler to your logging instance
logger.addHandler(fh)
try:
raise ValueError("Some error occurred")
except ValueError as e:
logger.exception(e) # Will send the errors to the file
And if I cat file_dir.txt
2019-03-14 14:52:50,676 - my_application - ERROR - Some error occurred
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: Some error occurred
Print to File
As I pointed out in the comments, you could accomplish this with print as well (I'm not sure you will be applauded for it)
# Set your stdout pointer to a file handler
with open('my_errors.txt', 'a') as fh:
try:
raise ValueError("Some error occurred")
except ValueError as e:
print(e, file=fh)
cat my_errors.txt
Some error occurred
Note that logging.exception includes the traceback in this case, which is one of the many huge benefits of that module
Edit
In the interest of completeness, the traceback module leverages a similar approach as print, where you can supply a file handle:
import traceback
import sys
with open('error.txt', 'a') as fh:
try:
raise ValueError("Some error occurred")
except ValueError as e:
e_type, e_val, e_tb = sys.exc_info()
traceback.print_exception(e_type, e_val, e_tb, file=fh)
This will include all of the information you want from logging

You can use the logging module as suggested in the comments (possibly superior but outside the scope of my knowledge), or catch the errors with try and except like:
try:
pass
#run the code you currently have
except Exception as e: # catch ALLLLLL errors!!!
print(e) # or more likely you'd want something like "email_to_me(e)"
Although this is generally frowned upon to catch all exceptions, because then should your program fail for whatever reason it will get gobbled up in the except clause so a better approach is to figure out what specific error you are encountering like IndexError and then just catch this specific error like:
try:
pass
#run the code you currently have
except IndexError as e: # catch only indexing errors!!!
print(e) # or more likely you'd want something like "email_to_me(e)"

To be able to debug and not only know the kind of error that happened, you can also get the error stack using traceback module (usually in the starting package of modules):
import traceback
try:
my_function()
except Exception as e:
print(e)
traceback.print_exc()
And then run your code 'my_code.py' in console usig >>
python my_code.py >> my_prints.txt
All the prints of your code will then be written in this .txt file, including the printed error and its stack. This is very interesting in your case or while running code on a docker if you want to detach yourself from it with ctrl+p+q and still know what is printed.

Print Python Exception Type (Raised in Fabric)

I'm using Fabric to automate, including the task of creating a directory. Here is my fabfile.py:
#!/usr/bin/env python
from fabric.api import *
def init():
try:
local('mkdir ./www')
except ##what exception?##:
#print exception name to put in above
Run fab fabfile.py and f I already have ./www created an error is raised, but I don't know what kind, so I don't know how to handle the error yet. Fabric only prints out the following:
mkdir: cannot create directory ‘./www’: File exists
Fatal error: local() encountered an error (return code 1) while executing 'mkdir ./www'
Aborting.
What I want to do is be able to find out the error type so that I can except my errors properly without blanket statements. It would be really helpful if an answer does not just tell me how to handle a mkdir exception, but print (or otherwise find the name to) any exception I may run into down the line (mkdir is just an example).
Thank you!

The issue is that fabric uses subprocess for doing these sorts of things. If you look at the source code for local you can see it doesn't actually raise an exception. It calls suprocess.Popen and uses communicate() to read stdout and stderr. If there is a non-zero return code then it returns a call to either warn or abort. The default is abort. So, to do what you want, try this:
def init():
with settings(warn_only=True):
local('mkdir ./www')
If you look at the source for abort, it looks like this:
10 def abort(msg):
21 from fabric.state import output
22 if output.aborts:
23 sys.stderr.write("\nFatal error: %s\n" % str(msg))
24 sys.stderr.write("\nAborting.\n")
25 sys.exit(1)
So, the exception would be a SystemExit exception. While you could catch this, the proper way to do it is outlined above using settings.

It is nothing to handle with exception, it is from the fabric api
try to set the entire script's warn_only setting to be true with
env.warn_only = True

Normally, when you get an uncaught exception, Python will print the exception type along with the error message:
>>> raise IOError("Error message.")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: Error message.
If that's not happening, you're probably not getting an exception at all.
If you really want to catch an arbitrary exception and print it, you want to catch Exception or BaseException. BaseException will include even things like KeyboardInterrupt, though, so be careful with that.
def init():
try:
local('mkdir ./www')
except BaseException as e:
print "local() threw a", type(e).__name__
raise # Reraise the exception

In general:
try:
some_code()
except Exception, e:
print 'Hit An Exception', e
raise
Will tell you what the exception was but if you are not planning on actually handling some of the exceptions then simply getting rid of the try: except: lines will have exactly the same effect.
Also if you run your code under a debugger then you can look at the exception(s) that you hit in more detail.

def init():
try:
local('mkdir ./www')
except Exception as e:
print e.__class__.__name__
That's all there is to it!
edit: Just re-read your question and realized that my code would only print "Fatal" in your case. It looks like fabric is throwing an error and returning their own error code so you would have to look at the documentation. I don't have any experience with fabric so I'd suggest to look here if you haven't already. Sorry if this isn't helpful!

How do you raise a python exception and include additional data for Sentry?

Sentry can detect additional data associated with an exception such as:
How do you raise such an exception from Python (it's a Django app) with your own additional data fields?.

I log exceptions using the logging library so after debugging the code a bit, I noticed the extra parameter:
import logging
logger = logging.getLogger('my_app_name')
def do_something():
try:
#do some stuff here that might break
except Exception, e:
logger.error(e, exc_info=1, extra={'extra-data': 'blah', })
Passing exc_info=1 is the same as calling logger.exception. However, exception() does not accept kwargs, which are required for using the extra parameter.
These values will show up in the 'Additional Data' section of the Sentry Error dashboard.

wes' answer didn't help me because I want to actually raise an exception, not only log it.
Here's what I did (client is the Raven Sentry client):
client.extra_context({'foo': 'bar'})
raise RuntimeError('Whoops, something went wrong!')

You might try one of these two approaches:
>>> # Raise the exception with the data you want.
>>> raise Exception('extra information')
Traceback (most recent call last):
File "<pyshell#64>", line 1, in <module>
raise Exception('extra information')
Exception: extra information
>>> # Catch an exception and add extra arguments.
>>> try:
raise Exception()
except Exception as error:
error.args += ('extra information',)
raise
Traceback (most recent call last):
File "<pyshell#68>", line 2, in <module>
raise Exception()
Exception: extra information
>>>
You can add as many additional data fields as you want by adding more arguments.

The Sentry handler adds that info in your screenshot when capturing the message for an exception, and takes that information from the traceback, not the exception itself.
You can add extra fields by passing extra keyword arguments to .capture(); the Django client does so for you if you pass in the request object, for example.
Currently, no other data is taken from exceptions. You'd have to expand the exception handling yourself to add such a facility.

None of the existing answers served my exact use case well (which was to add additional context from the django Request object into the sentry data). What ended up working very well for that after some digging was overriding the client using the SENTRY_CLIENT setting.
Here's a complete simple use case:
from raven.contrib.django.raven_compat import DjangoClient
class CustomSentryClient(DjangoClient):
def get_data_from_request(self, request):
result = super(EToolsSentryClient, self).get_data_from_request(request)
if getattr(request, 'custom_field', None):
if 'extra' not in result:
result['extra'] = {}
result['extra']['custom_field'] = request.custom_field
return result
and then in settings.py you would just add
SENTRY_CLIENT = 'myapp.CustomSentryClient'