Is it fine to raise an exception in __init__ in python? I have this piece of code:
class VersionManager(object):
def __init__(self, path):
self._path = path
if not os.path.exists(path): os.mkdir(path)
myfunction(path)
The second line can potentially result in an exception. In that case the object will not be init'ed properly. Is there a better way to handle situations where code in __init__ might throw an exception?
EDIT
Added a call to a function after os.mkdir
Added a check to see if directory exists
It is perfectly fine to raise an exception in __init__. You would then wrap the object initiation/creation call with try/except and react to the exception.
One potential odd result though is that __del__ is run anyway:
class Demo(object):
def __init__(self, value):
self.value=value
if value==2:
raise ValueError
def __del__(self):
print '__del__', self.value
d=Demo(1) # successfully create an object here
d=22 # new int object labeled 'd'; old 'd' goes out of scope
# '__del__ 1' is printed once a new name is put on old 'd'
# since the object is deleted with no references
Now try with the value 2 that we are testing for:
Demo(2)
Traceback (most recent call last):
File "Untitled 3.py", line 11, in <module>
Demo(2)
File "Untitled 3.py", line 5, in __init__
raise ValueError
ValueError
__del__ 2 # But note that `__del__` is still run.
The creation of the object with value 2 raises a ValueError exception and show that __del__ is still run to clean up the object.
Keep in mind that if you raise an exception during __init__ your object will not get a name. (It will, however, be created and destroyed. Since __del__ is paired with __new__ it still gets called)
ie, just like this does not create x:
>>> x=1/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
>>> x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
Potential sneakier:
>>> x='Old X'
>>> x=1/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero
>>> x
'Old X'
Same thing if you catch an exception of __init__:
try:
o=Demo(2)
except ValueError:
print o # name error -- 'o' never gets bound to the object...
# Worst still -- 'o' is its OLD value!
So don't try to refer to the incomplete object o -- it's gone out of scope by the time you get to except. And the name o is either nothing (i.e., NameError if you try to use it) or its old value.
So wrapping up (thanks to Steve Jessop for the User Defined Exception idea), you can wrap the creation of the object and catch the exception. Just figure out how to react appropriately to the OS error you are looking at.
So:
class ForbiddenTwoException(Exception):
pass
class Demo(object):
def __init__(self, value):
self.value=value
print 'trying to create with val:', value
if value==2:
raise ForbiddenTwoException
def __del__(self):
print '__del__', self.value
try:
o=Demo(2)
except ForbiddenTwoException:
print 'Doh! Cant create Demo with a "2"! Forbidden!!!'
# with your example - react to being unusable to create a directory...
Prints:
trying to create with val: 2
Doh! Cant create Demo with a "2"! Forbidden!!!
__del__ 2
You can wrap the call, as jramirez suggested:
try:
ver = VersionManager(path)
except:
raise
Or you can use a context manager:
class VersionManager(object):
def __init__(self):
#not-so-harmful code
self.path = path
def __enter__(self):
try:
self.path = path
os.mkdir(path)
self.myfunction(path)
except Exception as e:
print e
print "The directory making has failed, the function hasn't been executed."
return self
def __exit__(self, exc_type, exc_value, traceback):
print(exc_type, exc_value, traceback)
And to run it:
with VersionManager(my_path) as myVersionManager:
#do things you want with myVersionManager
This way, you'll catch errors inside the with statement as well.
You can use try/except when initializing the object.
try:
ver = VersionManager(my_path)
except Exception as e:
# raise e or handle error
print e
My favourite is to simply output errors to console and march on:
import sys, os, traceback
class Myclass
def __init__(self, path):
self._path = path
"""Risky Code"""
try:
os.mkdir(path)
except:
traceback.print_exc(file = sys.stdout)
This way an exception will print out more like a warning rather than a real exception.
Related
I am trying to overload new and this is my attempt:-
class String(object):
def __new__(cls, *args, **kargs):
print "in __new__"
return super(String, cls).__new__(cls)
def __init__(self):
print "Initiating instance of String"
raise Exception
def __del__(self):
print "Deleting instance of String"
I read at many places that actually __new__ create the instance and __init__ is just to initialize the instance. I am deliberately throwing exception in __init__ to let it fail. Here call to new returns the instance but init fails, so I am expecting an instance which won't have any attributes. But result surprised me like anything -
st = String()
in __new__
Initiating instance of String
Traceback (most recent call last):
File "<pyshell#89>", line 1, in <module>
st = String()
File "<pyshell#88>", line 7, in __init__
raise Exception
As expected it failed in __init__, Next I tried to print newly created instance 'st' and result surprised me, it deleted instance before printing.
>>> print st
**Deleting instance of String**
Traceback (most recent call last):
File "<pyshell#90>", line 1, in <module>
print st
NameError: name 'st' is not defined
Please help me to understand this strange behavior.
Note - I know when we should overload __new__ and when we shouldn't.
Exception was raised before Python managed to assign the object in memory to the the variable name. Therefore the reference counter of the object in memory is 0, so it got 'garbage collected', and you got name is not defined exception.
Update for the comment:
Python runs GC when it tries to allocate memory for a new object, if the threshold is filled. In your case it's probably the threshold for generation 0 that was reached, because it's where that new 'failed' String object should have been. And the event itself that triggered GC was memory allocation to execute print.
Not sure how possible this is, but here goes:
I'm trying to write an object with some slightly more subtle behavior - which may or may not be a good idea, I haven't determined that yet.
I have this method:
def __getattr__(self, attr):
try:
return self.props[attr].value
except KeyError:
pass #to hide the keyerror exception
msg = "'{}' object has no attribute '{}'"
raise AttributeError(msg.format(self.__dict__['type'], attr))
Now, when I create an instance of this like so:
t = Thing()
t.foo
I get a stacktrace containing my function:
Traceback (most recent call last):
File "attrfun.py", line 23, in <module>
t.foo
File "attrfun.py", line 15, in __getattr__
raise AttributeError(msg.format(self._type, attr))
AttributeError: 'Thing' object has no attribute 'foo'
I don't want that - I want the stack trace to read:
Traceback (most recent call last):
File "attrfun.py", line 23, in <module>
t.foo
AttributeError: 'Thing' object has no attribute 'foo'
Is this possible with a minimal amount of effort, or is there kind of a lot required? I found this answer which indicates that something looks to be possible, though perhaps involved. If there's an easier way, I'd love to hear it! Otherwise I'll just put that idea on the shelf for now.
You cannot tamper with traceback objects (and that's a good thing). You can only control how you process one that you've already got.
The only exceptions are: you can
substitute an exception with another or re-raise it with raise e (i.e make the traceback point to the re-raise statement's location)
raise an exception with an explicit traceback object
remove outer frame(s) from a traceback object by accessing its tb_next property (this reflects a traceback object's onion-like structure)
For your purpose, the way to go appears to be the 1st option: re-raise an exception from a handler one level above your function.
And, I'll say this again, this is harmful for yourself or whoever will be using your module as it deletes valuable diagnostic information. If you're dead set on making your module proprietary with whatever rationale, it's more productive for that goal to make it a C extension.
The traceback object is created during stack unwinding, not directly when you raise the exception, so you can not alter it right in your function. What you could do instead (though it's probably a bad idea) is to alter the top level exception hook so that it hides your function from the traceback.
Suppose you have this code:
class MagicGetattr:
def __getattr__(self, item):
raise AttributeError(f"{item} not found")
orig_excepthook = sys.excepthook
def excepthook(type, value, traceback):
iter_tb = traceback
while iter_tb.tb_next is not None:
if iter_tb.tb_next.tb_frame.f_code is MagicGetattr.__getattr__.__code__:
iter_tb.tb_next = None
break
iter_tb = iter_tb.tb_next
orig_excepthook(type, value, traceback)
sys.excepthook = excepthook
# The next line will raise an error
MagicGetattr().foobar
You will get the following output:
Traceback (most recent call last):
File "test.py", line 49, in <module>
MagicGetattr().foobar
AttributeError: foobar not found
Note that this ignores the __cause__ and __context__ members of the exception, which you would probably want to visit too if you were to implement this in real life.
You can get the current frame and any other level using the inspect module. For instance, here is what I use when I'd like to know where I'm in my code :
from inspect import currentframe
def get_c_frame(level = 0) :
"""
Return caller's frame
"""
return currentframe(level)
...
def locate_error(level = 0) :
"""
Return a string containing the filename, function name and line
number where this function was called.
Output is : ('file name' - 'function name' - 'line number')
"""
fi = get_c_frame(level = level + 2)
return '({} - {} - {})'.format(__file__,
fi.f_code,
fi.f_lineno)
I've seen similar questions to this one but none of them really address the trackback.
If I have a class like so
class Stop_if_no_then():
def __init__(self, value one, operator, value_two, then, line_or_label, line_number):
self._firstvalue = value_one
self._secondvalue = value_two
self._operator = operator
self._gohere = line_or_label
self._then = then
self._line_number = line_number
def execute(self, OtherClass):
"code comparing the first two values and making changes etc"
What I want my execute method to be able to do is if self._then is not equal to the string "THEN" (in allcaps) then I want it to raise a custom error message and terminate the whole program while also not showing a traceback.
If the error is encountered the only thing that should print out would look something like (I'm using 3 as an example, formatting is not a problem) this.
`Syntax Error (Line 3): No -THEN- present in the statement.`
I'm not very picky about it actually being an exception class object, so there's no issue in that aspect. Since I will be using this in a while loop, simple if, elif just repeats the message over and over (because obviously I am not closing the loop). I have seen sys.exit() but that also prints out a giant block of red text, unless I am not using it correctly. I don't want to catch the exception in my loop because there are other classes in the same module in which I need to implement something like this.
You can turn off the traceback by limiting its depth.
Python 2.x
import sys
sys.tracebacklimit = 0
Python 3.x
In Python 3.5.2 and 3.6.1, setting tracebacklimit to 0 does not seem to have the intended effect. This is a known bug. Note that -1 doesn't work either. Setting it to None does however seem to work, at least for now.
In Python 3.6.2 and above you should set tracebacklimit to 0 or -1, as setting it to None does not disable the traceback output.
Python 3.6.1 and bellow results:
>>> import sys
>>> sys.tracebacklimit = 0
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
>>> sys.tracebacklimit = -1
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
>>> sys.tracebacklimit = None
>>> raise Exception
Exception
Python 3.6.2 and above results:
>>> import sys
>>> sys.tracebacklimit = 0
>>> raise Exception
Exception
>>> sys.tracebacklimit = -1
>>> raise Exception
Exception
>>> sys.tracebacklimit = None
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
Nevertheless, for better or worse, if multiple exceptions are raised, they can all still be printed. For example:
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
You can use SystemExit exception:
except Exception as err:
raise SystemExit(err)
https://docs.python.org/3/library/exceptions.html
You can use a try: and then except Exception as inst:
What that will do is give you your error message in a variable named inst and you can print out the arguments on the error with inst.args. Try printing it out and seeing what happens, and is any item in inst.args is the one you are looking for.
EDIT Here is an example I tried with pythons IDLE:
>>> try:
open("epik.sjj")
except Exception as inst:
d = inst
>>> d
FileNotFoundError(2, 'No such file or directory')
>>> d.args
(2, 'No such file or directory')
>>> d.args[1]
'No such file or directory'
>>>
EDIT 2: as for closing the program you can always raise and error or you can use sys.exit()
The cleanest way that I know is to use sys.excepthook.
You implement a three argument function that accepts type, value, and traceback and does whatever you like (say, only prints the value) and assign that function to sys.excepthook.
Here is an example:
import sys
def excepthook(type, value, traceback):
print(value)
sys.excepthook = excepthook
raise ValueError('hello')
This is available in both python 2 and python 3.
If you want to get rid of any traceback for customs exceptions and have line number,
you can do this trick
Python 3
import sys
import inspect
class NoTraceBackWithLineNumber(Exception):
def __init__(self, msg):
try:
ln = sys.exc_info()[-1].tb_lineno
except AttributeError:
ln = inspect.currentframe().f_back.f_lineno
self.args = "{0.__name__} (line {1}): {2}".format(type(self), ln, msg),
sys.exit(self)
class MyNewError(NoTraceBackWithLineNumber):
pass
raise MyNewError("Now TraceBack Is Gone")
Will give this output, and make the raise keyword useless
MyNewError (line 16): Now TraceBack Is Gone
"Exception chaining can be disabled by using from None " - Python docs
>>> try:
... open('database.sqlite')
... except IOError:
... raise RuntimeError from None
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
In general, if you want to catch any exception except SystemExit, and exit with the exception's message without the traceback, define your main function as below:
>>> import sys
>>> def main():
... try:
... # Run your program from here.
... raise RandomException # For testing
... except (Exception, KeyboardInterrupt) as exc:
... sys.exit(exc)
...
>>> main()
name 'RandomException' is not defined
$ echo $?
1
Note that in the case of multiple exceptions being raised, only one message is printed.
This answer is meant to improve upon the one by The-IT.
This is a followup to my question Hang in Python script using SQLAlchemy and multiprocessing. As discussed in that question, pickling exceptions is problematic in Python. This is usually not a issue, but one case when it is, is when errors occur in the python multiprocessing module. Since multiprocessing moves objects around by pickling, if an error occurs inside a multiprocessing process, the entire process may hang, as demonstrated in that question.
One possible approach is to fix all the problematic exceptions, as discussed in that question. This is not easy, since one cannot easily know in advance which exceptions may be called. An alternative approach, which was suggested by lbolla in an answer to the question, is to catch the exception, construct an equivalent harmless exception, and then rethrow.
However, I'm not sure of exactly how to do this. Consider the following code.
class BadExc(Exception):
def __init__(self, message, a):
'''Non-optional param in the constructor.'''
Exception.__init__(self, message)
self.a = a
import sys
try:
try:
#print foo
raise BadExc("bad exception error message", "a")
except Exception, e:
raise Exception(e.__class__.__name__ + ": " +str(e)), None, sys.exc_info()[2]
except Exception, f:
pass
import cPickle
a = cPickle.dumps(f)
l = cPickle.loads(a)
print "raising error"
raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]
This code pickles and unpickles the exception, and then throws it, giving the error
raising error
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
Exception: BadExc: bad exception error message
Credits to Glenn Maynard's answer to "“Inner exception” (with traceback) in Python?".
This has the important stuff, namely the traceback, the error message, and the exception type, so this might be the best one can do. But ideally I'd like something that looks exactly like the original exception, namely
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
__main__.BadExc: bad exception error message
or more generally, with the name of the exception in the front, rather than Exception. Is this possible?
Alternatively, instead of the BadExc class, one can use the print foo statement instead, which gives a NameError. However, this exception does not require special handling.
You can override sys.excepthook to achieve what you want. It at least works for this example, but it's pretty hacky so please test and no promises :-)
import sys
def excepthook_wrapper(type, value, traceback):
if len(value.args) == 2:
name, msg = value.args
value.args = (msg,)
sys.__excepthook__(name, value, traceback)
else:
sys.__excepthook__(type, value, traceback)
sys.excepthook = excepthook_wrapper
(Edit: I'm not really happy with this because now 'normal' Exceptions with two arguments will get handled differently too. Possible solution, 'tag' your special Exceptions by passing "PICKLED" as a first argument and then check for that, instead of checking for the length of the args.)
And then create the Exception with two arguments, the name (__module__.__class__) and the Exception message (str(e)):
try:
try:
#print foo
raise BadExc("bad exception error message", "a")
except Exception, e:
cls = e.__class__
if hasattr(cls, '__module__'):
name = '{0}.{1}'.format(cls.__module__, cls.__name__)
else:
name = cls.__name__
raise Exception(name, str(e)), None, sys.exc_info()[2]
except Exception, f:
pass
Then this:
import cPickle
a = cPickle.dumps(f)
l = cPickle.loads(a)
print "raising error"
raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]
Prints:
raising error
Traceback (most recent call last):
File "test.py", line 18, in <module>
raise BadExc("bad exception error message", "a")
__main__.BadExc: bad exception error message
I want to save the name of the error and the traceback details into a variable. Here's is my attempt.
import sys
try:
try:
print x
except Exception, ex:
raise NameError
except Exception, er:
print "0", sys.exc_info()[0]
print "1", sys.exc_info()[1]
print "2", sys.exc_info()[2]
Output:
0 <type 'exceptions.NameError'>
1
2 <traceback object at 0xbd5fc8>
Desired Output:
0 NameError
1
2 Traceback (most recent call last):
File "exception.py", line 6, in <module>
raise NameError
P.S. I know this can be done easily using the traceback module, but I want to know usage of sys.exc_info()[2] object here.
This is how I do it:
>>> import traceback
>>> try:
... int('k')
... except:
... var = traceback.format_exc()
...
>>> print var
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: invalid literal for int() with base 10: 'k'
You should however take a look at the traceback documentation, as you might find there more suitable methods, depending to how you want to process your variable afterwards...
sys.exc_info() returns a tuple with three values (type, value, traceback).
Here type gets the exception type of the Exception being handled
value is the arguments that are being passed to constructor of exception class
traceback contains the stack information like where the exception occurred etc.
For Example, In the following program
try:
a = 1/0
except Exception,e:
exc_tuple = sys.exc_info()
Now If we print the tuple the values will be this.
exc_tuple[0] value will be "ZeroDivisionError"
exc_tuple[1] value will be "integer division or modulo by zero" (String passed as parameter to the exception class)
exc_tuple[2] value will be "trackback object at (some memory address)"
The above details can also be fetched by simply printing the exception in string format.
print str(e)
Use traceback.extract_stack() if you want convenient access to module and function names and line numbers.
Use ''.join(traceback.format_stack()) if you just want a string that looks like the traceback.print_stack() output.
Notice that even with ''.join() you will get a multi-line string, since the elements of format_stack() contain \n. See output below.
Remember to import traceback.
Here's the output from traceback.extract_stack(). Formatting added for readability.
>>> traceback.extract_stack()
[
('<string>', 1, '<module>', None),
('C:\\Python\\lib\\idlelib\\run.py', 126, 'main', 'ret = method(*args, **kwargs)'),
('C:\\Python\\lib\\idlelib\\run.py', 353, 'runcode', 'exec(code, self.locals)'),
('<pyshell#1>', 1, '<module>', None)
]
Here's the output from ''.join(traceback.format_stack()). Formatting added for readability.
>>> ''.join(traceback.format_stack())
' File "<string>", line 1, in <module>\n
File "C:\\Python\\lib\\idlelib\\run.py", line 126, in main\n
ret = method(*args, **kwargs)\n
File "C:\\Python\\lib\\idlelib\\run.py", line 353, in runcode\n
exec(code, self.locals)\n File "<pyshell#2>", line 1, in <module>\n'
Be careful when you take the exception object or the traceback object out of the exception handler, since this causes circular references and gc.collect() will fail to collect. This appears to be of a particular problem in the ipython/jupyter notebook environment where the traceback object doesn't get cleared at the right time and even an explicit call to gc.collect() in finally section does nothing. And that's a huge problem if you have some huge objects that don't get their memory reclaimed because of that (e.g. CUDA out of memory exceptions that w/o this solution require a complete kernel restart to recover).
In general if you want to save the traceback object, you need to clear it from references to locals(), like so:
import sys, traceback, gc
type, val, tb = None, None, None
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
# some cleanup code
gc.collect()
# and then use the tb:
if tb:
raise type(val).with_traceback(tb)
In the case of jupyter notebook, you have to do that at the very least inside the exception handler:
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
raise type(val).with_traceback(tb)
finally:
# cleanup code in here
gc.collect()
Tested with python 3.7.
p.s. the problem with ipython or jupyter notebook env is that it has %tb magic which saves the traceback and makes it available at any point later. And as a result any locals() in all frames participating in the traceback will not be freed until the notebook exits or another exception will overwrite the previously stored backtrace. This is very problematic. It should not store the traceback w/o cleaning its frames. Fix submitted here.
The object can be used as a parameter in Exception.with_traceback() function:
except Exception as e:
tb = sys.exc_info()
print(e.with_traceback(tb[2]))