I'm writing python 2.6.6 code on windows that looks like this:
try:
dostuff()
except KeyboardInterrupt:
print "Interrupted!"
except:
print "Some other exception?"
finally:
print "cleaning up...."
print "done."
dostuff() is a function that loops forever, reading a line at a time from an input stream and acting on it. I want to be able to stop it and clean up when I hit ctrl-c.
What's happening instead is that the code under except KeyboardInterrupt: isn't running at all. The only thing that gets printed is "cleaning up...", and then a traceback is printed that looks like this:
Traceback (most recent call last):
File "filename.py", line 119, in <module>
print 'cleaning up...'
KeyboardInterrupt
So, exception handling code is NOT running, and the traceback claims that a KeyboardInterrupt occurred during the finally clause, which doesn't make sense because hitting ctrl-c is what caused that part to run in the first place! Even the generic except: clause isn't running.
EDIT: Based on the comments, I replaced the contents of the try: block with sys.stdin.read(). The problem still occurs exactly as described, with the first line of the finally: block running and then printing the same traceback.
EDIT #2:
If I add pretty much anything after the read, the handler works. So, this fails:
try:
sys.stdin.read()
except KeyboardInterrupt:
...
But this works:
try:
sys.stdin.read()
print "Done reading."
except KeyboardInterrupt:
...
Here's what's printed:
Done reading. Interrupted!
cleaning up...
done.
So, for some reason, the "Done reading." line is printed, even though the exception occurred on the previous line. That's not really a problem - obviously I have to be able to handle an exception anywhere inside the "try" block. However, the print doesn't work normally - it doesn't print a newline afterwards like it's supposed to! The "Interruped" is printed on the same line... with a space before it, for some reason...? Anyway, after that the code does what it's supposed to.
It seems to me that this is a bug in handling an interrupt during a blocked system call.
Asynchronous exception handling is unfortunately not reliable (exceptions raised by signal handlers, outside contexts via C API, etc). You can increase your chances of handling the async exception properly if there is some coordination in the code about what piece of code is responsible for catching them (highest possible in the call stack seems appropriate except for very critical functions).
The called function (dostuff) or functions further down the stack may itself have a catch for KeyboardInterrupt or BaseException that you didn't/couldn't account for.
This trivial case worked just fine with python 2.6.6 (x64) interactive + Windows 7 (64bit):
>>> import time
>>> def foo():
... try:
... time.sleep(100)
... except KeyboardInterrupt:
... print "INTERRUPTED!"
...
>>> foo()
INTERRUPTED! #after pressing ctrl+c
EDIT:
Upon further investigation, I tried what I believe is the example that others have used to reproduce the issue. I was lazy so I left out the "finally"
>>> def foo():
... try:
... sys.stdin.read()
... except KeyboardInterrupt:
... print "BLAH"
...
>>> foo()
This returns immediately after hitting CTRL+C. The interesting thing happened when I immediately tried to call foo again:
>>> foo()
Traceback (most recent call last):
File "c:\Python26\lib\encodings\cp437.py", line 14, in decode
def decode(self,input,errors='strict'):
KeyboardInterrupt
The exception was raised immediately without me hitting CTRL+C.
This would seem to make sense - it appears that we are dealing with nuances in how asynchronous exceptions are handled in Python. It can take several bytecode instructions before the async exception is actually popped and then raised within the current execution context. (That's the behavior that I've seen when playing with it in the past)
See the C API: http://docs.python.org/c-api/init.html#PyThreadState_SetAsyncExc
So this somewhat explains why KeyboardInterrupt gets raised in the context of the execution of the finally statement in this example:
>>> def foo():
... try:
... sys.stdin.read()
... except KeyboardInterrupt:
... print "interrupt"
... finally:
... print "FINALLY"
...
>>> foo()
FINALLY
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in foo
KeyboardInterrupt
There could be some crazy mixing of custom signal handlers mixed with the interpreter's standard KeyboardInterrupt/CTRL+C handler that's resulting in this sort of behavior. For example, the read() call sees the signal and bails, but it re-raises the signal after deregistering it's handler. I wouldn't know for sure without inspecting the interpreter codebase.
This is why I generally shy away from making use of async exceptions....
EDIT 2
I think there's a good case for a bug report.
Again more theories...(just based on reading code) See the file object source: http://svn.python.org/view/python/branches/release26-maint/Objects/fileobject.c?revision=81277&view=markup
file_read calls Py_UniversalNewlineFread(). fread can return with an error with errno = EINTR (it performs its own signal handling). In this case Py_UniversalNewlineFread() bails but does not perform any signal checking with PyErr_CheckSignals() so that the handlers can get called synchronously. file_read clears the file error but also does not call PyErr_CheckSignals().
See getline() and getline_via_fgets() for examples of how it's used. The pattern is documented in this bug report for a similar issue: ( http://bugs.python.org/issue1195 ). So it seems that the signal is handled at an indeterminate time by the interpreter.
I guess there's little value in diving any deeper since it's still not clear whether the sys.stdin.read() example is a proper analog of your "dostuff()" function. (there could be multiple bugs at play)
sys.stdin.read() is a system call and so the behavior is going to be different for each system. For windows 7 I think what is happening is that the input is being buffered and so you're getting where sys.stdin.read() is returning everything up to the Ctrl-C and as soon as you access sys.stdin again it'll send the "Ctrl-C".
try the following,
def foo():
try:
print sys.stdin.read()
print sys.stdin.closed
except KeyboardInterrupt:
print "Interrupted!"
This suggests that there is a buffering of stdin going on that is causing another call to stdin to recognize the keyboard input
def foo():
try:
x=0
while 1:
x += 1
print x
except KeyboardInterrupt:
print "Interrupted!"
there doesn't appear to be a problem.
Is dostuff() reading from stdin?
Having similar problem and this is my workaround:
try:
some_blocking_io_here() # CTRL-C to interrupt
except:
try:
print() # any i/o will get the second KeyboardInterrupt here?
except:
real_handler_here()
Here's a guess about what's happening:
pressing Ctrl-C breaks the "print" statement (for whatever reason... bug? Win32 limitation?)
pressing Ctrl-C also throws the first KeyboardInterrupt, in dostuff()
The exception handler runs and tries to print "Interrupted", but the "print" statement is broken and throws another KeyboardInterrupt.
The finally clause runs and tries to print "cleaning up....", but the "print" statement is broken and throws yet another KeyboardInterrupt.
This works for me:
import sys
if __name__ == "__main__":
try:
sys.stdin.read()
print "Here"
except KeyboardInterrupt:
print "Worked"
except:
print "Something else"
finally:
print "Finally"
Try putting a line outside of your dostuff() function or move the loop condition outside of the function. For example:
try:
while True:
dostuff()
except KeyboardInterrupt:
print "Interrupted!"
except:
print "Some other exception?"
finally:
print "cleaning up...."
print "done."
def foo():
try:
x=0
while 1:
x+=1
print (x)
except KeyboardInterrupt:
print ("interrupted!!")
foo()
That works fine.
Related
Does invoking a raise statement in python cause the program to exit with traceback or continue the program from next statement? I want to raise an exception but continue with the remainder program.
Well I need this because I am running the program in a thirdparty system and I want the exception to be thrown yet continue with the program. The concerned code is a threaded function which has to return .
Cant I spawn a new thread just for throwing exception and letting the program continue?
I want to raise an exception but continue with the remainder program.
There's not much sense in that: the program control either continues through the code, or ripples up the call stack to the nearest try block.
Instead you can try some of:
the traceback module (for reading or examining the traceback info you see together with exceptions; you can easily get it as text)
the logging module (for saving diagnostics during program runtime)
Example:
def somewhere():
print 'Oh no! Where am I?'
import traceback
print ''.join(traceback.format_stack()) # or traceback.print_stack(sys.stdout)
print 'Oh, here I am.'
def someplace():
somewhere()
someplace()
Output:
Oh no! Where am I?
File "/home/kos/exc.py", line 10, in <module>
someplace()
File "/home/kos/exc.py", line 8, in someplace
somewhere()
File "/home/kos/exc.py", line 4, in somewhere
print ''.join(traceback.format_stack())
Oh, here I am.
Only an uncaught exception will terminate a program. If you raise an exception that your 3rd-party software is not prepared to catch and handle, the program will terminate. Raising an exception is like a soft abort: you don't know how to handle the error, but you give anyone using your code the opportunity to do so rather than just calling sys.exit().
If you are not prepared for the program to exit, don't raise an exception. Just log the error instead.
I use a list of different python scripts for various functions. To help facilitate this, I've organized all of my reusable functions into custom libraries. However, I found that many of these functions will error out for strange reasons, some known and some unknown. I designed the below function to at least let me see the error message before throwing a giant traceback at me. I have the below command in one library:
FooBar = trace_check(lambda: Foo(bar))
This is the error catching function in a separate library:
def trace_check(func):
try:
return func()
except:
TracebackString = traceback.format_exc() ###This gets the traceback as a string.
type, message, tracebacklocation = sys.exc_info() ###This gets the components, particularly the message.
print "An error has occurred with the following error message:"
print type ###Example is IOError
print message ###The message associated with the error
TracebackPrompt = ask("Do you want to see the entire traceback?") #Returns True/False value
if TracebackPrompt:
print TracebackString
print 'Exiting Python Script' ###This shows me it gets to this point.
sys.exit(0) ###This is the problem
print "Did it work?" ###This statement does not print, so exit worked...
When trace_check runs and I get an error, the sys.exit only quits the function back out to main() instead of ending main. if I use os._exit() instead, the main() function ends correctly, but the program running the script also dies. One command is not strong enough and the other is overkill... what could I do instead to ensure the main() function ends?
Note: I tried putting the meat of my trace_check function into the first library, but the same thing happens with the library call ending but not the main().
tl;dr - Python: main() calls function in library that calls a second function in separate library. Second function has a sys.exit() command that only exits to main() instead of ending main(). os._exit() kills shell and is overkill (requires restarting shell TT^TT). Is there another way to end the main() from a function library?
To directly answer your question, if you want to handle sys.exit() calls from main, then you should catch the SystemExit exception that is raised by sys.exit(). The sample code below illustrates how to do that.
import sys
def func():
sys.exit(1)
def main():
try:
func()
except SystemExit:
print 'Someone sys.exit()d'
return 0
if __name__ == '__main__':
sys.exit(main())
However! You should probably redesign your library. Instead of calling sys.exit() when something unexpected happens, you should raise an Exception. Having a library abruptly exit the interpreter is bad design.
You could try setting this off by throwing an exception:
class ExitFromMain(Exception):
pass
def trace_check(func):
try:
# try stuff
except:
# traceback stuff you had
raise ExitFromMain()
def main():
try:
# Stuff
trace_check()
# More stuff that will not run if the exception is thrown
except ExitFromMain:
print "I hit my excception to flag a quit from this function"
sys.exit(0)
I'm writing a command line utility in Python which, since it is production code, ought to be able to shut down cleanly without dumping a bunch of stuff (error codes, stack traces, etc.) to the screen. This means I need to catch keyboard interrupts.
I've tried using both a try catch block like:
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
print 'Interrupted'
sys.exit(0)
and catching the signal itself (as in this post):
import signal
import sys
def sigint_handler(signal, frame):
print 'Interrupted'
sys.exit(0)
signal.signal(signal.SIGINT, sigint_handler)
Both methods seem to work quite well during normal operation. However, if the interrupt comes during cleanup code at the end of the application, Python seems to always print something to the screen. Catching the interrupt gives
^CInterrupted
Exception KeyboardInterrupt in <bound method MyClass.__del__ of <path.to.MyClass object at 0x802852b90>> ignored
whereas handling the signal gives either
^CInterrupted
Exception SystemExit: 0 in <Finalize object, dead> ignored
or
^CInterrupted
Exception SystemExit: 0 in <bound method MyClass.__del__ of <path.to.MyClass object at 0x802854a90>> ignored
Not only are these errors ugly, they're not very helpful (especially to an end user with no source code)!
The cleanup code for this application is fairly big, so there's a decent chance that this issue will be hit by real users. Is there any way to catch or block this output, or is it just something I'll have to deal with?
Checkout this thread, it has some useful information about exiting and tracebacks.
If you are more interested in just killing the program, try something like this (this will take the legs out from under the cleanup code as well):
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
print('Interrupted')
try:
sys.exit(130)
except SystemExit:
os._exit(130)
[Edited to change the exit code as suggested in comments. 130 is the code typically returned on Linux for a script terminated by Ctrl-C. We may not be on Linux, but the important thing is to return a non-zero value, and 130 is as good as any.]
You could ignore SIGINTs after shutdown starts by calling signal.signal(signal.SIGINT, signal.SIG_IGN) before you start your cleanup code.
Python newbie here and am running into some weird behavior in my code.
I am trying to write some data to a file. I print the length of the data to be about 50k before I call the following block of code. The data is a pdf file I got over the internet. And its a valid pdf.
When I call the function F() described below, I get the exception message printed in function F and not in the actual place it fails.
In the code below, in the function write_to_disk() I see the second print and the execution directly jumps to the exception handler in the calling function F(). I cannot figure out why this is happening. On disk I see the file is created but the size is 0.
Can some look at the code below and may be guess what could be happening?
If I am catching exceptions in the write_to_disk() function how is it possible for it to jump out of the function completely?
EDIT: Thanks for kobejohn, turns out the excetion object does not have a errno variable. Getting rid of it made the print appear. But the bigger problem still exists. I see a failure with no way to find out why its failing. How do I get the error message here?
def write_to_disk(self, pathToWrite, pdfFileData):
try:
print 'Here `1.1'
fd = open(pathToWrite, "w+")
print 'Here `2.1'
fd.write(pdfFileData)
print 'Here 3.1'
fd.close()
except Exception as e:
print 'file cannot be opened ' + pathToWrite + e.errno
This function is inturn called by another function F which is like this -
def F(self, url):
pathWrite = get_path_to_use()
pdfData = get_pdf_data(url)
try:
writetodisk(pathToWrite, pdfData)
except Exception as e:
print 'Did I jump directly to here?' + e.errno
Here is the out put of the program. I did not think it will add anything because I see nothing if any use. In fact I get the same output even when running it in pdb.
Here `1.1
Here `2.1
Did I jump directly to here?
Your first exception handler tries to build a string by concatenating another string and an int (e.errno) which causes it (the print statement) to throw an exception itself (which is then caught by the outer exception handler).
It's just bubbling as we mentioned in the comments and Alexander said. Use this code to see how it can work (no errors, but that was just a nasty surprise you got with exceptions).
def f(url):
path_to_write = 'test.dat'
pdf_data = 'asdf'
try:
write_to_disk(path_to_write, pdf_data)
except Exception as e:
print 'Did I jump directly to here?\n' + str(e)
def write_to_disk(path_to_write, pdf_data):
try:
print 'Here `1.1'
with open(path_to_write, "w+") as fd:
print 'Here `2.1'
fd.write(pdf_data)
except Exception as e:
print 'file cannot be opened ' + path_to_write
f('fake_url')
Some areas to make your code safer / more standard:
do as little as possible inside a try block. try to isolate the code that you are worried might raise an error
same rule for the except block. don't do anything strange there.
as someone else mentioned, using a with block is a more standard and readable way to work with a file.
other minor things with function and variable names you can see where I have changed things in the code above. google PEP 8 for more.
I have a piece of code which is not in a function, say
x = 5
y = 10
if x > 5:
print("stopping")
What can I put after the print statement to stop the code from running further? Sys.exit() works, but raises an error that I don't want in the program. I want it to quietly stop the code as if it had reached the end of the main loop. Thanks.
As JBernardo pointed out, sys.exit() raises an exception. This exception is SystemExit. When it is not handled by the user code, the interpreter exits cleanly (a debugger debugging the program can catch it and keep control of the program, thanks to this mechanism, for instance)—as opposed to os._exit(), which is an unconditional abortion of the program.
This exception is not caught by except Exception:, because SystemExit does not inherit from Exception. However, it is caught by a naked except: clause.
So, if your program sees an exception, you may want to catch fewer exceptions by using except Exception: instead of except:. That said, catching all exceptions is discouraged, because this might hide real problems, so avoid it if you can, by making the except clause (if any) more specific.
My understanding of why this SystemExit exception mechanism is useful is that the user code goes through any finally clause after a sys.exit() found in an except clause: files can be closed cleanly, etc.; then the interpreter catches any SystemExit that was not caught by the user and exits for good (a debugger would instead catch it so as to keep the interpreter running and obtain information about the program that exited).
You can do what you're looking for by doing this:
import os
os._exit(1)
sys.exit() which is equivalent to sys.exit(0) means exit with success. sys.exit(1) or sys.exit("Some message") means exit with failure. Both cases raise a SystemExit exception. In fact when your program exists normally it is exactly like sys.exit(0) has been called.
When I ran across this thread, I was looking for a way to exit the program without an error, without an exception, have the code show as 'PASSED', and continue running other tests files. The solution, for me, was to use the return statement.
.
.
.
if re.match("^[\s\S]*gdm-simple-slave[\s\S]*$", driver.find_element_by_css_selector("BODY").text) == None:
print "Identifiers object gdm-simple-slave not listed in table"
return
else:
driver.find_element_by_xpath("//input[#value='gdm-simple-slave']").click()
.
.
.
That allowed me to run multiple programs while keeping the debugger running...
test_logsIdentifiersApache2EventWindow.py#16::test_LogsIdentifiersApache2EventWi
ndow **PASSED**
test_logsIdentifiersAudispdEventWindow.py#16::test_LogsIdentifiersAudispdEventWi
ndow **PASSED**
test_logsIdentifiersGdmSimpleSlaveEventWindow.py#16::test_LogsIdentifiersGdmSimp
leSlaveEventWindow Identifiers object gdm-simple-slave not listed in table
**PASSED**
test_logsIdentifiersAuditdEventWindow.py#16::test_LogsIdentifiersAuditdEventWind
ow **PASSED**
Use try-except statements.
a = [1, 2, 3, 4, 5]
for x in xrange(0,5):
try:
print a[x+1] #this is a faulty statement for test purposes
except:
exit()
print "This is the end of the program."
Output:
> python test.py
2
3
4
5
No errors printed, despite the error raised.