I'm having problems catching a custom exception when thrown from a signal handler callback when using asyncio.
If I throw ShutdownApp from within do_io() below, I am able to properly catch it in run_app(). However, when the exception is raised from handle_sig(), I can't seem to catch it.
Minimal, Reproducible Example Tested using Python 3.8.5:
import asyncio
from functools import partial
import os
import signal
from signal import Signals
class ShutdownApp(BaseException):
pass
os.environ["PYTHONASYNCIODEBUG"] = "1"
class App:
def __init__(self):
self.loop = asyncio.get_event_loop()
def _add_signal_handler(self, signal, handler):
self.loop.add_signal_handler(signal, handler, signal)
def setup_signals(self) -> None:
self._add_signal_handler(signal.SIGINT, self.handle_sig)
def handle_sig(self, signum):
print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising error for exit")
raise ShutdownApp("Exiting")
async def do_io(self):
print("io start. Press Ctrl+C now.")
await asyncio.sleep(5)
print("io end")
def run_app(self):
print("Starting Program")
try:
self.loop.run_until_complete(self.do_io())
except ShutdownApp as e:
print("ShutdownApp caught:", e)
# TODO: do other shutdown related items
except:
print("Other error")
finally:
self.loop.close()
if __name__ == "__main__":
my_app = App()
my_app.setup_signals()
my_app.run_app()
print("Finished")
The output after pressing CTRL+C (for SIGINT) with asyncio debug mode:
(env_aiohttp) anav#anav-pc:~/Downloads/test$ python test_asyncio_signal.py
Starting Program
io start. Press Ctrl+C now.
^C
pid: 20359, Received signal: SIGINT, raising error for exit
Exception in callback App.handle_sig(<Signals.SIGINT: 2>)
handle: <Handle App.handle_sig(<Signals.SIGINT: 2>) created at /home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/unix_events.py:99>
source_traceback: Object created at (most recent call last):
File "test_asyncio_signal.py", line 50, in <module>
my_app.setup_signals()
File "test_asyncio_signal.py", line 25, in setup_signals
self._add_signal_handler(signal.SIGINT, self.handle_sig)
File "test_asyncio_signal.py", line 22, in _add_signal_handler
self.loop.add_signal_handler(signal, handler, signal)
File "/home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/unix_events.py", line 99, in add_signal_handler
handle = events.Handle(callback, args, self, None)
Traceback (most recent call last):
File "/home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "test_asyncio_signal.py", line 31, in handle_sig
raise ShutdownApp("Exiting")
ShutdownApp: Exiting
io end
Finished
Expected output:
Starting Program
io start. Press Ctrl+C now.
^C
pid: 20359, Received signal: SIGINT, raising error for exit
ShutdownApp caught: Exiting
io end
Finished
Is it possible to raise a custom exception from a signal handler in asyncio? If so, how do I properly catch/except it?
handle_sig is a callback, so it runs directly off the event loop and its exceptions are just reported to the user via a global hook. If you want the exception raised there to be caught elsewhere in the program, you need to use a future to transfer the exception from handle_sig to where you want it noticed.
To catch the exception at top-level, you probably want to introduce another method, let's call it async_main(), that waits for either self.do_io() or the previously-created future to complete:
def __init__(self):
self.loop = asyncio.get_event_loop()
self.done_future = self.loop.create_future()
async def async_main(self):
# wait for do_io or done_future, whatever happens first
io_task = asyncio.create_task(self.do_io())
await asyncio.wait([self.done_future, io_task],
return_when=asyncio.FIRST_COMPLETED)
if self.done_future.done():
io_task.cancel()
await self.done_future # propagate the exception, if raised
else:
self.done_future.cancel()
To raise the exception from inside handle_sig, you just need to set the exception on the future object:
def handle_sig(self, signum):
print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising error for exit")
self.done_future.set_exception(ShutdownApp("Exiting"))
Finally, you modify run_app to pass self.async_main() to run_until_complete, and you're all set:
$ python3 x.py
Starting Program
io start. Press Ctrl+C now.
^C
pid: 2069230, Received signal: SIGINT, raising error for exit
ShutdownApp caught: Exiting
Finished
In closing, note that reliably catching keyboard interrupts is a notoriously tricky undertaking and the above code might not cover all the corner cases.
If I throw ShutdownApp from within do_io() below, I am able to properly catch it in run_app(). However, when the exception is raised from handle_sig(), I can't seem to catch it.
Response to the query given above
Custom Exception Implementation:
class RecipeNotValidError(Exception):
def __init__(self):
self.message = "Your recipe is not valid"
try:
raise RecipeNotValidError
except RecipeNotValidError as e:
print(e.message)
In the method handle_sig, add try and except block. Also, you can customize messages in custom exceptions.
def handle_sig(self, signum):
try:
print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising
error for exit")
raise ShutdownApp("Exiting")
except ShutdownApp as e:
print(e.message)
In response to your second query :
Is it possible to raise a custom exception from a signal handler in asyncio? If so, how do I properly catch/except it?
In-built error handling in Asyncio. For more documentation visit https://docs.python.org/3/library/asyncio-exceptions.html
import asyncio
async def f():
try:
while True: await asyncio.sleep(0)
except asyncio.CancelledError:
print('I was cancelled!')
else:
return 111
If disconnect is called, I want to restart the main method from the connection.py file. Main method does not restart with existing code when disconnect occurs.
connection.py file*
def disconnect(flags):
#do something
raise RuntimeError('You have been disconnected')
main.py file*
import connection
def main():
while True:
try:
#do something to invoke disconnect
except RuntimeError('You have been disconnected'):
main()
main()
Your try-except block is wrong. I ran your code in python3, and it raised a very clear error, "builtins.TypeError: catching classes that do not inherit from BaseException is not allowed" Because of that unhandled exception, you exit main.
Try these:
try:
raise RuntimeError('You have been disconnected')
except RuntimeError:
print ('caught')
try:
raise RuntimeError('You have been disconnected')
except RuntimeError as e:
print (str(e))
Because you already have a while loop in main, there's no need to recursively call main again. Just let the while loop do its job:
def main():
while True:
connect()
try:
raise RuntimeError('You have been disconnected')
except RuntimeError as e:
print (str(e))
If the error is caught, then you will still be inside the while loop.
While trying to implement a DNSRequest, I also needed to do some exception handling and noticed something weird. The following code is able to catch DNS request timeouts
def lambda_handler(event, context):
hostname = "google.de"
dnsIpAddresses = event['dnsIpAddresses']
dnsResolver = dns.resolver.Resolver()
dnsResolver.lifetime = 1.0
result = {}
for dnsIpAddress in dnsIpAddresses:
dnsResolver.nameservers = [dnsIpAddress]
try:
myAnswers = dnsResolver.query(hostname, "A")
print(myAnswers)
result[dnsIpAddress] = "SUCCESS"
except dns.resolver.Timeout:
print("caught Timeout exception")
result[dnsIpAddress] = "FAILURE"
except dns.exception.DNSException:
print("caught DNSException exception")
result[dnsIpAddress] = "FAILURE"
except:
result[dnsIpAddress] = "FAILURE"
print("caught general exception")
return result
Now, if I removed the Timeout block, and assuming that a Timeout would occur, on a DNSException the message
caught DNSException exception
will never be shown.
Now, if I removed the DNSException block, and assuming that a Timeout would occur, the message
caught general exception
will never be shown.
But the Timeout extends the DNSException and the DNSException extends Exception. I had the expectation that at least the general expect block should work.
What am I missing?
In your last block try making the print line come before result[dnsIpAddress] = "FAILURE"
My guess is either there is more code than what is shown here or the line before the print statement causes a different exception.
I cannot understand why sometimes I cannot catch the TimeOutError inside my flash_serial_buffer method.
When running my program I sometimes get a TimeOutError that is not caught and I cannot understand why. I indicate the code of the signal handlers and the methods where the TimeOutError is not caught. How could this be happening?
This is the code for my signal handler definition and callback function.
Basically if the time ends, the signal handler is called and raises a timeout error.
def signal_handler(signum, frame):
print "PUM"
raise TimedOutError("Time out Error")
signal.signal(signal.SIGALRM, signal_handler)
The flush serial buffer blocks if there is no answer to
answer = xbee.wait_read_frame()
The idea is to clean everything in the buffer until there aren’t any more messages. When there are no more messages, it just waits for the SIGALRM to explode and raises the timeout error.
def flush_serial_buffer(xbee):
# Flush coordinators serial buffer if problem happened before
logging.info(" Flashing serial buffer")
try:
signal.alarm(1) # Seconds
while True:
answer = xbee.wait_read_frame()
signal.alarm(1)
logging.error(" Mixed messages in buffer")
except TimedOutError:
signal.alarm(0) # Seconds
logging.error(" No more messages in buffer")
signal.alarm(0) # Supposedly it never leaves without using Except, but...
Is there a case where the TimeOutError might be raised, but not caught by the try: statement?
Here is my error class definition:
class TimedOutError(Exception):
pass
I was able to repeat the error again. I really cannot understand why the try does not catch the error it.
INFO:root: Flashing serial buffer
PUM
Traceback (most recent call last):
File "/home/ls/bin/pycharm-community-4.0.6/helpers/pydev/pydevd.py", line 1458, in trace_dispatch
if self._finishDebuggingSession and not self._terminationEventSent:
File "/home/ls/PiProjects/Deployeth/HW-RPI-API/devices.py", line 42, in signal_handler
raise TimedOutError("Time out Error")
TimedOutError: Time out Error
I would recommend in this case replacing the try and except code with this:
try:
signal.alarm(1) # Seconds
while True:
answer = xbee.wait_read_frame()
signal.alarm(1)
logging.error(" Mixed messages in buffer")
except:
signal.alarm(0) # Seconds
logging.error(" No more messages in buffer")
PS: You don't need to include try (whatever error) in your try and except statements.
I'm attempting to code a try-except loop that refreshes a webpage if it fails to load. Here's what I've done so far:
driver.get("url")
while True:
try:
<operation>
except:
driver.refresh()
I want to set this loop up so that if 5 seconds elapse and the operation is not executed (presumably because the page did not load), it attempts to refresh the page. Is there an exception we can incorporate in except that catches the time delay?
I would recommend reading this post Timeout function if it takes too long to finish.
The gist of it is that you can use signals to interrupt the code and raise an error, which you then catch.
In you example:
def _handle_timeout(signum,frame):
raise TimeoutError("Execution timed out")
driver.get("url")
signal.signal(signal.SIGALRM, _handle_timeout)
while True:
try:
signal.alarm(<timeout value>)
<operation>
signal.alarm(0)
except:
driver.refresh()
You can test this with the following snippet:
import time
import signal
def _handle_timeout(signum,frame):
raise TimeoutError("Execution timed out")
def test(timeout,execution_time):
signal.signal(signal.SIGALRM, _handle_timeout)
try:
signal.alarm(timeout)
time.sleep(execution_time)
signal.alarm(0)
except:
raise
else:
print "Executed successfully"
This will raise an error when execution_time > timeout.
As noted here in Python signal don't work even on Cygwin? the above code will not work on windows machines.