I am searching for a way to change the printable output of an Exception to a silly message in order to learn more about python internals (and mess with a friend ;), so far without success.
Consider the following code
try:
x # is not defined
except NameError as exc:
print(exc)
The code shall output name 'x' is not defined
I would like the change that output to the name 'x' you suggested is not yet defined, my lord. Improve your coding skills.
So far, I understood that you can't change __builtins__ because they're "baked in" as C code, unless:
You use forbiddenfruit.curse method which adds / changes properties of any object
You manually override the dictionnaries of an object
I've tried both solutions, but without success:
forbiddenfruit solution:
from forbiddenfruit import curse
curse(BaseException, 'repr', lambda self: print("Test message for repr"))
curse(BaseException, 'str', lambda self: print("Test message for str"))
try:
x
except NameError as exc:
print(exc.str()) # Works, shows test message
print(exc.repr()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
Dictionnary overriding solution:
import gc
underlying_dict = gc.get_referents(BaseException.__dict__)[0]
underlying_dict["__repr__"] = lambda self: print("test message for repr")
underlying_dict["__str__"] = lambda self: print("test message for str")
underlying_dict["args"] = 'I am an argument list'
try:
x
except NameError as exc:
print(exc.__str__()) # Works, shows test message
print(exc.__repr__()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
AFAIK, using print(exc) should rely on either __repr__ or __str__, but it seems like the print function uses something else, which I cannot find even when reading all properties of BaseException via print(dir(BaseException)).
Could anyone give me an insight of what print uses in this case please ?
[EDIT]
To add a bit more context:
The problem I'm trying to solve began as a joke to mess with a programmer friend, but now became a challenge for me to understand more of python's internals.
There's no real business problem I'm trying to solve, I just want to get deeper understanding of things in Python.
I'm quite puzzled that print(exc) won't make use of BaseException.__repr__ or __str__ actually.
[/EDIT]
Intro
I'd go with a more critical approach on why you'd even want to do what you want to do.
Python provides you with an ability to handle specific exceptions. That means if you had a business problem, you'd use a particular exception class and provide a custom message for that specific case. Now, remember this paragraph and let's move on, I'll refer to this later.
TL;DR
Now, let's go top-down:
Catching all kinds of errors with except Exception is generally not a good idea if want you catch let's say a variable name error. You'd use except NameError instead. There's really not much you'd add to it that's why it had a default message that perfectly described the issue. So it's assumed you'd use it as it's given. These are called concrete exceptions.
Now, with your specific case notice the alias as exc. By using the alias you can access arguments passed to the exception object, including the default message.
try:
x # is not defined
except NameError as exc:
print(exc.args)
Run that code (I put it in app.py) and you'll see:
$ python app.py
("name 'x' is not defined",)
These args are passed to the exception as a series (list, or in this case immutable list that is a tuple).
This leads to the idea of the possibility of easily passing arguments to exceptions' constructors (__init__). In your case "name 'x' is not defined" was passed as an argument.
You can use this to your advantage to solve your problem without much effort by just providing a custom message, like:
try:
x # is not defined
except NameError as exc:
your_custom_message = "the name 'x' you suggested is not yet defined, my lord. Improve your coding skills"
# Now, you can handle it based on your requirement:
# print(your_custom_message)
# print(NameError(your_custom_message))
# raise NameError(your_custom_message)
# raise NameError(your_custom_message) from exc
The output is now what you wanted to achieve.
$ python app.py
the name 'x' you suggested is not yet defined, my lord. Improve your coding skills
Remember the first paragraph when I said I'd refer to it later? I mentioned providing a custom message for a specific case. If you build your own library when you want to handle name errors to specific variables relevant to your product, you assume your users will use your code that might raise that NameError exception. They will most likely catch it with except Exception as exc or except NameError as exc. And when they do print(exc), they will see your message now.
Summary
I hope that makes sense to you, just provide a custom message and pass it as an argument to NameError or simply just print it. IMO, it's better to learn it right together with why you'd use what you use.
Errors like this are hard-coded into the interpreter (in the case of CPython, anyway, which is most likely what you are using). You will not be able to change the message printed from within Python itself.
The C source code that is executed when the CPython interpreter tries to look up a name can be found here: https://github.com/python/cpython/blob/master/Python/ceval.c#L2602. If you would want to change the error message printed when a name lookup fails, you would need to change this line in the same file:
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
Compiling the modified source code would yield a Python interpreter that prints your custom error message when encountering a name that is not defined.
I'll just explain the behaviour you described:
exc.__repr__()
This will just call your lambda function and return the expected string. Btw you should return the string, not print it in your lambda functions.
print(repr(exc))
Now, this is going a different route in CPython and you can see this in a GDB session, it's something like this:
Python/bltinmodule.c:builtin_repr will call Objects/object.c:PyObject_Repr - this function gets the PyObject *v as the only parameter that it will use to get and call a function that implements the built-in function repr(), BaseException_repr in this case. This function will format the error message based on a value from args structure field:
(gdb) p ((PyBaseExceptionObject *) self)->args
$188 = ("name 'x' is not defined",)
The args value is set in Python/ceval.c:format_exc_check_arg based on a NAME_ERROR_MSG macro set in the same file.
Update: Sun 8 Nov 20:19:26 UTC 2020
test.py:
import sys
import dis
def main():
try:
x
except NameError as exc:
tb = sys.exc_info()[2]
frame, i = tb.tb_frame, tb.tb_lasti
code = frame.f_code
arg = code.co_code[i + 1]
name = code.co_names[arg]
print(name)
if __name__ == '__main__':
main()
Test:
# python test.py
x
Note:
I would also recommend to watch this video from PyCon 2016.
Related
This is a hard question to phrase, but here's a stripped-down version of the situation. I'm using some library code that accepts a callback. It has its own error-handling, and raises an error if anything goes wrong while executing the callback.
class LibraryException(Exception):
pass
def library_function(callback, string):
try:
# (does other stuff here)
callback(string)
except:
raise LibraryException('The library code hit a problem.')
I'm using this code inside an input loop. I know of potential errors that could arise in my callback function, depending on values in the input. If that happens, I'd like to reprompt, after getting helpful feedback from its error message. I imagine it looking something like this:
class MyException(Exception):
pass
def my_callback(string):
raise MyException("Here's some specific info about my code hitting a problem.")
while True:
something = input('Enter something: ')
try:
library_function(my_callback, something)
except MyException as e:
print(e)
continue
Of course, this doesn't work, because MyException will be caught within library_function, which will raise its own (much less informative) Exception and halt the program.
The obvious thing to do would be to validate my input before calling library_function, but that's a circular problem, because parsing is what I'm using the library code for in the first place. (For the curious, it's Lark, but I don't think my question is specific enough to Lark to warrant cluttering it with all the specific details.)
One alternative would be to alter my code to catch any error (or at least the type of error the library generates), and directly print the inner error message:
def my_callback(string):
error_str = "Here's some specific info about my code hitting a problem."
print(error_str)
raise MyException(error_str)
while True:
something = input('Enter something: ')
try:
library_function(my_callback, something)
except LibraryException:
continue
But I see two issues with this. One is that I'm throwing a wide net, potentially catching and ignoring errors other than in the scope I'm aiming at. Beyond that, it just seems... inelegant, and unidiomatic, to print the error message, then throw the exception itself into the void. Plus the command line event loop is only for testing; eventually I plan to embed this in a GUI application, and without printed output, I'll still want to access and display the info about what went wrong.
What's the cleanest and most Pythonic way to achieve something like this?
There seems to be many ways to achieve what you want. Though, which one is more robust - I cannot find a clue about. I'll try to explain all the methods that seemed apparent to me. Perhaps you'll find one of them useful.
I'll be using the example code you provided to demonstrate these methods, here's a fresher on how it looks-
class MyException(Exception):
pass
def my_callback(string):
raise MyException("Here's some specific info about my code hitting a problem.")
def library_function(callback, string):
try:
# (does other stuff here)
callback(string)
except:
raise Exception('The library code hit a problem.')
The simplest approach - traceback.format_exc
import traceback
try:
library_function(my_callback, 'boo!')
except:
# NOTE: Remember to keep the `chain` parameter of `format_exc` set to `True` (default)
tb_info = traceback.format_exc()
This does not require much know-how about exceptions and stack traces themselves, nor does it require you to pass any special frame/traceback/exception to the library function. But look at what this returns (as in, the value of tb_info)-
'''
Traceback (most recent call last):
File "path/to/test.py", line 14, in library_function
callback(string)
File "path/to/test.py", line 9, in my_callback
raise MyException("Here's some specific info about my code hitting a problem.")
MyException: Here's some specific info about my code hitting a problem.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "path/to/test.py", line 19, in <module>
library_function(my_callback, 'boo!')
File "path/to/test.py", line 16, in library_function
raise Exception('The library code hit a problem.')
Exception: The library code hit a problem.
'''
That's a string, the same thing you'd see if you just let the exception happen without catching. Notice the exception chaining here, the exception at the top is the exception that happened prior to the exception at the bottom. You could parse out all the exception names-
import re
exception_list = re.findall(r'^(\w+): (\w+)', tb_info, flags=re.M)
With that, you'll get [('MyException', "Here's some specific info about my code hitting a problem"), ('Exception', 'The library code hit a problem')] in exception_list
Although this is the easiest way out, it's not very context aware. I mean, all you get are class names in string form. Regardless, if that is what suits your needs - I don't particularly see a problem with this.
The "robust" approach - recursing through __context__/__cause__
Python itself keeps track of the exception trace history, the exception currently at hand, the exception that caused this exception and so on. You can read about the intricate details of this concept in PEP 3134
Whether or not you go through the entirety of the PEP, I urge you to at least familiarize yourself with implicitly chained exceptions and explicitly chained exceptions. Perhaps this SO thread will be useful for that.
As a small refresher, raise ... from is for explicitly chaining exceptions. The method you show in your example, is implicit chaining
Now, you need to make a mental note - TracebackException#__cause__ is for explicitly chained exceptions and TracebackException#__context__ is for implicitly chained exceptions. Since your example uses implicit chaining, you can simply follow __context__ backwards and you'll reach MyException. In fact, since this is only one level of nesting, you'll reach it instantly!
import sys
import traceback
try:
library_function(my_callback, 'boo!')
except:
previous_exc = traceback.TracebackException(*sys.exc_info()).__context__
This first constructs the TracebackException from sys.exc_info. sys.exc_info returns a tuple of (exc_type, exc_value, exc_traceback) for the exception at hand (if any). Notice that those 3 values, in that specific order, are exactly what you need to construct TracebackException - so you can simply destructure it using * and pass it to the class constructor.
This returns a TracebackException object about the current exception. The exception that it is implicitly chained from is in __context__, the exception that it is explicitly chained from is in __cause__.
Note that both __cause__ and __context__ will return either a TracebackException object, or None (if you're at the end of the chain). This means, you can call __cause__/__context__ again on the return value and basically keep going till you reach the end of the chain.
Printing a TracebackException object just prints the message of the exception, if you want to get the class itself (the actual class, not a string), you can do .exc_type
print(previous_exc)
# prints "Here's some specific info about my code hitting a problem."
print(previous_exc.exc_type)
# prints <class '__main__.MyException'>
Here's an example of recursing through .__context__ and printing the types of all exceptions in the implicit chain. (You can do the same for .__cause__)
def classes_from_excs(exc: traceback.TracebackException):
print(exc.exc_type)
if not exc.__context__:
# chain exhausted
return
classes_from_excs(exc.__context__)
Let's use it!
try:
library_function(my_callback, 'boo!')
except:
classes_from_excs(traceback.TracebackException(*sys.exc_info()))
That will print-
<class 'Exception'>
<class '__main__.MyException'>
Once again, the point of this is to be context aware. Ideally, printing isn't the thing you'll want to do in a practical environment, you have the class objects themselves on your hands, with all the info!
NOTE: For implicitly chained exceptions, if an exception is explicitly suppressed, it'll be a bad day trying to recover the chain - regardless, you might give __supressed_context__ a shot.
The painful way - walking through traceback.walk_tb
This is probably the closest you can get to the low level stuff of exception handling. If you want to capture entire frames of information instead of just the exception classes and messages and such, you might find walk_tb useful....and a bit painful.
import traceback
try:
library_function(my_callback, 'foo')
except:
tb_gen = traceback.walk_tb(sys.exc_info()[2])
There is....entirely too much to discuss here. .walk_tb takes a traceback object, you may remember from the previous method that the 2nd index of the returned tuple from sys.exec_info is just that. It then returns a generator of tuples of frame object and int (Iterator[Tuple[FrameType, int]]).
These frame objects have all kinds of intricate information. Though, whether or not you'll actually find exactly what you're looking for, is another story. They may be complex, but they aren't exhaustive unless you play around with a lot of frame inspection. Regardless, this is what the frame objects represent.
What you do with the frames is upto you. They can be passed to many functions. You can pass the entire generator to StackSummary.extract to get framesummary objects, you can iterate through each frame to have a look at [0].f_locals (The [0] on Tuple[FrameType, int] returns the actual frame object) and so on.
for tb in tb_gen:
print(tb[0].f_locals)
That will give you a dict of the locals for each frame. Within the first tb from tb_gen, you'll see MyException as part of the locals....among a load of other stuff.
I have a creeping feeling I have overlooked some methods, most probably with inspect. But I hope the above methods will be good enough so that no one has to go through the jumble that is inspect :P
Chase's answer above is phenomenal. For completeness's sake, here's how I implemented their second approach in this situation. First, I made a function that can search the stack for the specified error type. Even though the chaining in my example is implicit, this should be able to follow implicit and/or explicit chaining:
import sys
import traceback
def find_exception_in_trace(exc_type):
"""Return latest exception of exc_type, or None if not present"""
tb = traceback.TracebackException(*sys.exc_info())
prev_exc = tb.__context__ or tb.__cause__
while prev_exc:
if prev_exc.exc_type == exc_type:
return prev_exc
prev_exc = prev_exc.__context__ or prev_exc.__cause__
return None
With that, it's as simple as:
while True:
something = input('Enter something: ')
try:
library_function(my_callback, something)
except LibraryException as exc:
if (my_exc := find_exception_in_trace(MyException)):
print(my_exc)
continue
raise exc
That way I can access my inner exception (and print it for now, although eventually I may do other things with it) and continue. But if my exception wasn't in there, I simply reraise whatever the library raised. Perfect!
I have code that looks like this:
# foo.py
from internallib.a import A
A.some(new_schema)
# a.py
from internallib.b import B
class A:
#staticmethod
def some(schema_name):
B.thing(schema_name, other_parameter)
# b.py
import marshmallow
class B:
#staticmethod
def thing(schema_name, other_parameter):
marshmallow.class_registry.get_class(schema_name)
Marshmallow throws an error that I'd like to catch and print nicely for the user and then exit (instead of just printing a stack trace). The specifics here are that Marshmallow throws a RegistryError when get_class is called and the schema hasn't been registered.
However, this is a completely expected condition - it's ok to TRY to add the schema, and I want to present that error to the user - think of it kind like a 'file not found' error. A stack trace feels way too low level to present.
However, A.some() is called in many places in the code. What's the pythonic way of doing this? My thoughts:
Wrap the marshmallow() in a try-except, print something out and then exit()
Go everywhere I call A.some(new_schema) and wrap that (it's not my preference, but if necessary)
Something else?
Solution
Option 1 - uniform behavior for all A.some() callers: Wrap B.thing() call in A.some() in a try except catching very specific exceptions while using logger to log error in the except block prior to exiting.
Option 2 - varying behavior needed for A.some() callers: If you wanted varying behavior depending on the context of where A.some() is being called, then you would need to refactor to catch the same error from Option 1 and raise your own internal error named FileNotFound, which each caller to A.some() would be able to handle behavior individually.
What does "Error return without exception set" commonly mean? I am trying to call a Python method that should return a list.
for facet in dispPart.FacetedBodies:
#Tag
facet_tag2=thisFctBody.Tag
#calling this method returns an "error return without exception set"
face_list=facet.GetFaces()
This is what I've found on it so far
You shoud tell what third party Python module you are using - this error message implies there is something wrong in that code, not yours. (ok, from your link, it is "NXOpen Python API")
Specifically, the third party module, interfacing with the Python C API returned an incorrect result, reporting an exception should have been raised, but did not tell Python which exception is that.
It is possible that there is something incorrect in your input data that, if fixed, could allow for normal return of the desired call - nonetheless it is buggy as it is.
One thing you might try is to call some other method on your facet object to check if, for example, it is not empty.
I'd like to have something like a custom error code/message database and use it when raising exceptions (in Python 3.4). So I did the following:
class RecipeError(Exception):
# Custom error codes
ERRBADFLAVORMIX = 1
ERRNOINGREDIENTS = ERRBADFLAVORMIX + 1
# Custom messages
ERRMSG = {ERRBADFLAVORMIX: "Bad flavor mix",
ERRNOINGREDIENTS: "No ingredients to mix"}
raise RecipeError(RecipeError.ERRMSG[RecipeError.ERRBADFLAVORMIX])
This works as expected, but the raise statement is just monstrous. Sure, I could have stored the values in a more compact way, but what I really want to know is: Can I just do something like raise RecipeError(code) and leave the work of getting the message to RecipeError?
Sure. Exception classes are just normal classes, so you can define your own __init__ that calls super appropriately:
class RecipeError(BaseException):
# existing stuff
def __init__(self, code):
super().__init__(self, RecipeError.ERRMSG[code])
You might also want to save the code:
class RecipeError(BaseException):
# existing stuff
def __init__(self, code):
msg = RecipeError.ERRMSG[code]
super().__init__(self, msg)
self.code, self.msg = code, msg
Take a look at the information stored in the standard library's exceptions (which are pretty decent in 3.4, although there are still more changes to come…) to see what kinds of things might be useful to stash.
Some side notes:
First, it may be better to use subclasses instead of error codes. For example, if someone wants to write code that catches an ERRBADFLAVORMIX but not an ERRNOINGREDIENTS, they have to do this:
try:
follow_recipe()
except RecipeError as e:
if e != RecipeError.ERRBADFLAVORMIX:
raise
print('Bad flavor, bad!')
Or, if you'd used subclasses:
try:
follow_recipe():
except BadFlavorRecipeError as e:
print('Bad flavor, bad!')
That's exactly why Python no longer has a monolithic OSError with an errno value that you have to switch on, and instead has separate subclasses like FileNotFoundError.
If you do want to use error codes, you might want to consider using an Enum, or maybe one of the fancier enum types on PyPI that make it easier to attach a custom string to each one.
You almost never want to inherit from BaseException, unless you're specifically trying to make sure your exception doesn't get caught.
I have a class (see this previous question if you are interested) which tracks, amongst other things, errors. The class is called in a variety of situations, one of them is during an exception. Even though my class calls sys.exc_clear() as part of the regular course of events, the next time the class is called (even when there is no error, such as when I am just throwing some statistical information in one of the class functions) the sys.exc_info() tuple is still full of the original non-None objects.
I can call sys.exc_clear() and sys.exc_info() is just a bunch of Nones while the thread is executing in that class, but as soon as execution returns to the main program, this ceases to be valid. My reading of the documentation suggests that this is because the execution stack has returned to another frame. This seems to be a situation tangentially mentioned in this question previously.
So, my only option appears to be tacking sys.exc_clear() after each except in my main program. I have tried it in a few places and it works. I can do this, but it seems tedious and ugly. Is another way?
ADDITION:
Imagine the main program as
import tracking
def Important_Function():
try:
something that fails
except:
myTrack.track(level='warning', technical='Failure in Important_Function' ...)
return
def Other_Function():
myTrack.track(level='info', technical='Total=0' ...)
return
myTrack = tracking.Tracking()
myTrack.track(level='debug', parties=['operator'], technical='Started the program.')
Important_Function()
Other_Function()
Then the Tracking code as:
import sys
import inspect
import traceback
... lots of initialization stuff
def track(self, level='info', technical=None, parties=None, clear=True ...):
# What are our errors?
errors = {}
errortype, errorvalue, errortraceback = sys.exc_info()
errortype, errorvalue = sys.exc_info()[:2]
errors['type'] = None
errors['class'] = errortype
errors['value'] = errorvalue
errors['arguments'] = None
errors['traceback'] = None
try:
errors['type'] = str(errortype.__name__)
try:
errors['arguments'] = str(errorvalue.__dict__['args'])
except KeyError:
pass
errors['traceback'] = traceback.format_tb(errortraceback, maxTBlevel)
except:
pass
if clear == True:
sys.exc_clear()
No multi-threading that I'm aware of. If I print sys.exc_info() right after calling sys.exc_clear(), everything has been cleared. But once I return from the track function and then re-enter it, even without errors, sys.exc_info() is back with a tuple full of the previous, old errors.
Please note that last exception information is a per-thread construct. An excerpt from sys.exc_info:
This function returns a tuple of three values that give information about the exception that is currently being handled. The information returned is specific both to the current thread and to the current stack frame.
So, running sys.exc_clear in a thread does not affect other threads.
UPDATE:
Quoting from the documentation:
Warning
Assigning the traceback return value to a local variable in a function that is handling an exception will cause a circular reference. This will prevent anything referenced by a local variable in the same function or by the traceback from being garbage collected. Since most functions don’t need access to the traceback, the best solution is to use something like exctype, value = sys.exc_info()[:2] to extract only the exception type and value. If you do need the traceback, make sure to delete it after use (best done with a try ... finally statement) or to call exc_info() in a function that does not itself handle an exception.
You do assign the traceback to a local variable, and that is why I commented your question with the suggestion to remove the “offending” line.
I believe the error is caused by a misuse of sys.exc_clear() and exceptions. The actual problem is that you're calling it after handling another exception. This is the exception that actually gets cleared, not the one in the you recorded.
A solution to your problem would be to create a different method for tracking exceptions and call it in the except clause and call it only in the except clause -- that way the exception will always be the right one.
Problems I can see in the code above:
You're calling exc.clear_exc() after another exception has been handled, clearing it not the one you want.
You want to call exc.clear_exc() to clear someone else's exceptions - this is wrong and it can break the program (e.g. if a bare raise is called after the call to a fixed version track, it will fail). Whoever is handling the exception likes to use those values, clearing them like this can do no good.
You're expecting that if there was no error, sys.exc_info() will not be set as long as you clear it each time. That is not true -- there might be data for a previous exception there in a completely unrelated call to track. Don't rely on that.
All of those are fixed by using separate methods and never using sys.clear_exc().
Oh, another thing, if those except: clauses without an exceptions are not just examples, it's advisable to handle only exceptions that you know about, and not all of them like that.