Running this code
import gevent
def f():
while True:
gevent.sleep(1)
if __name__ == '__main__':
tasks = (gevent.spawn(f),)
try:
gevent.wait(tasks)
except KeyboardInterrupt:
print("KeyboardInterrupt trapped")
and then pressing a Ctrl-C, give me this output:
$ python receiver.py
^CKeyboardInterrupt
Tue Aug 8 00:56:04 2017
KeyboardInterrupt trapped
Why?
It seems someone is writing the exit time on output.
How can I prevent that KeyboardInterrupt in the first line and the date in the second?
Those messages are printed by the gevent Hub, which is intercepting the KeyboardInterrupt being raised. Usually you would see a traceback instead of just KeyboardInterrupt and the current date, but because the Hub is special, you get that output.
You have two ways to solve this issue:
Mark KeyboardInterrupt as a non-error:
gevent.get_hub().NOT_ERROR += (KeyboardInterrupt,)
With this trick, the Hub won't print any line when KeyboardInterrupt is caught. This might seem a hack, but it's a short and effective way to stop output pollution.
Register a signal handler for SIGINT:
def handler(signum, frame):
print('SIGINT trapped')
sys.exit(0)
signal.signal(signal.SIGINT, handler)
The default signal handler for SIGINT will raise KeyboardInterrupt, but if you define your own signal handler, you can prevent it and run your cleanup code.
It is important that you exit with an exception from your handler function, otherwise your call to gevent.wait() won't be stopped. The only two exceptions that you can use are SystemExit and GreenletExit (those are the two default exceptions in the NOT_ERROR list above): any other exception will cause gevent to print something on standard error.
Related
I have been using asyncio to run subprocess calls in a separate thread. For this purpose I start an event loop in my main thread as per the recommendation - https://docs.python.org/3/library/asyncio-subprocess.html#subprocess-and-threads .
Now when I use a normal subprocess call in the main thread, I start getting following file descriptor error after few iterations:
Exception ignored when trying to write to the signal wakeup fd:
BlockingIOError: [Errno 11] Resource temporarily unavailable
I have reproduced the problem in a small script and am seeing that the error goes away if I do not start the even loop in the main thread.
import asyncio
import subprocess
import time
def try_error():
for i in range(1,500):
print(i)
try:
subprocess.run(["ls", "-l"], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
except subprocess.CalledProcessError as e:
print(f"Exception raised {e.stderr}\nOutput {e.stdout}")
def definite_error():
w = asyncio.get_child_watcher()
l = asyncio.get_event_loop()
try_error()
if __name__ == "__main__":
definite_error()
I am not sure why this error occurs and how to make it go away. Any help is appreciated. Thanks.
The problem is that the code doesn't actually start the event loop, it just obtains the event loop instance.
Note that the event loop needs to be running, so it's not enough to turn def try_error() into async def and run it with run_until_complete() to fix the problem. You must also add an await to give the event loop a chance to run accumulated tasks, such as draining the signal wakeup fd.
For example, await asyncio.sleep(0) is such a minimal await, which will temporarily yield control to the event loop and allow your code to proceed.
I have an issue where a method call is blocking and not releasing. Unfortunately, the bug as to why isn't exactly solvable, so the workaround at the moment is to build in a timeout.
I've tried to do this by registering a timer and have it raise an exception to break from the blocked call. However, that raises the exception in the timer thread, not the main thread.
It looks like this right now:
from threading import Timer
def timeoutSocket():
raise InterruptedError
socketDeadlockDetector = Timer(DEADLOCK_TIMEOUT, timeoutSocket)
socketDeadlockDetector.start()
# receive and unpack data
try:
packet = server.receive()
except InterruptedError:
print("Interrupted socket receive, continuing")
continue
socketDeadlockDetector.cancel()
server.receive() is the method that is blocking when it shouldn't. However, when I run this, the socketDeadlockDetector thread interrupts itself, without affecting the original thread.
Is there a way to pass this exception up to the parent?
Timer creates a thread to run the function. It doesn't do you any good to raise an exception because that's not the thread needing interruption. When you hit the timeout, you need to cancel whatever is blocking in the other thread. In this case its a socket, so killing the socket should do.
import struct
def timeoutSocket():
# enable linger with timeout 0 to send RESET on close
server.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 0))
server.close()
I cannot understand why sometimes I cannot catch the TimeOutError inside my flash_serial_buffer method.
When running my program I sometimes get a TimeOutError that is not caught and I cannot understand why. I indicate the code of the signal handlers and the methods where the TimeOutError is not caught. How could this be happening?
This is the code for my signal handler definition and callback function.
Basically if the time ends, the signal handler is called and raises a timeout error.
def signal_handler(signum, frame):
print "PUM"
raise TimedOutError("Time out Error")
signal.signal(signal.SIGALRM, signal_handler)
The flush serial buffer blocks if there is no answer to
answer = xbee.wait_read_frame()
The idea is to clean everything in the buffer until there aren’t any more messages. When there are no more messages, it just waits for the SIGALRM to explode and raises the timeout error.
def flush_serial_buffer(xbee):
# Flush coordinators serial buffer if problem happened before
logging.info(" Flashing serial buffer")
try:
signal.alarm(1) # Seconds
while True:
answer = xbee.wait_read_frame()
signal.alarm(1)
logging.error(" Mixed messages in buffer")
except TimedOutError:
signal.alarm(0) # Seconds
logging.error(" No more messages in buffer")
signal.alarm(0) # Supposedly it never leaves without using Except, but...
Is there a case where the TimeOutError might be raised, but not caught by the try: statement?
Here is my error class definition:
class TimedOutError(Exception):
pass
I was able to repeat the error again. I really cannot understand why the try does not catch the error it.
INFO:root: Flashing serial buffer
PUM
Traceback (most recent call last):
File "/home/ls/bin/pycharm-community-4.0.6/helpers/pydev/pydevd.py", line 1458, in trace_dispatch
if self._finishDebuggingSession and not self._terminationEventSent:
File "/home/ls/PiProjects/Deployeth/HW-RPI-API/devices.py", line 42, in signal_handler
raise TimedOutError("Time out Error")
TimedOutError: Time out Error
I would recommend in this case replacing the try and except code with this:
try:
signal.alarm(1) # Seconds
while True:
answer = xbee.wait_read_frame()
signal.alarm(1)
logging.error(" Mixed messages in buffer")
except:
signal.alarm(0) # Seconds
logging.error(" No more messages in buffer")
PS: You don't need to include try (whatever error) in your try and except statements.
I installed the signal in the main method,
But when I pressed ctrl+c during running the process wasn't stopped,
exceptions.SystemExit: 0
^CKilled by user
Unhandled Error
EventTrigger and MemoryInfo are classes inherit from threading
and HttpStreamClient is a class inferits from twosted.reactor
How to kill my process by ctrl+c , thanks
Code
def signal_handler(*args):
print("Killed by user")
# teardown()
sys.exit(0)
def install_signal():
for sig in (SIGABRT, SIGILL, SIGINT, SIGSEGV, SIGTERM):
signal(sig, signal_handler)
def main():
try:
global cgi, config
install_signal()
config = Config().read_file(sys.argv[1])[0]
init_export_folder()
setup_logging()
threads = [
EventTrigger(config),
MemoryInfo(config),
]
for thr in threads:
thr.setDaemon(True)
thr.start()
HttpStreamClient(config).run()
for thr in threads:
thr.join()
except BaseException as e:
traceback.print_exc(file=sys.stdout)
raise e
I think your problem might be the forceful nature that you are terminating the process.
While using twisted you should call reactor.stop() to get the initial run call to stop blocking.
If you change your signal_handler to shutdown the reactor.
def signal_handler(*args):
print("Killed by user")
reactor.stop()
Your threads could still keep the process alive. Thread.join doesn't forcefully stop a thread, which in general is never really a good idea. If EventTrigger or MemoryInfo are still running the thr.join will block. You will need a mechanism to stop threads. Maybe take a look here.
sys.exit() raises a Python exception; I'm pretty sure raising an exception in a signal handler does not do much. Either call reactor.stop() as Alex says or use os._exit(0). Be aware that using os._exit(0) will terminate the process without further ado.
The Python program below starts one thread and then continues to perform actions in the main thread. I wrap the whole main thread in a try-except block so I can tear down all running threads if an exception occured.
When I run the script using Python 2.7.5 and invoke a KeyboardInterrupt at a random point during the programs execution, the exception is triggered but not catched. The program continues to run.
$ python test.py
Running server ...
Searching for servers ...
^CTraceback (most recent call last):
File "test.py", line 50, in <module>
main()
File "test.py", line 40, in main
app_main()
File "test.py", line 35, in app_main
searchservers()
File "test.py", line 26, in searchservers
time.sleep(0.0005)
KeyboardInterrupt
I miss a line in the output which is printed in main() when an exception occured.
Code
import time
import threading
thread_pool = []
running = False
def stop():
global running
running = False
def runserver():
print "Running server ..."
global running
running = True
while running:
time.sleep(0.07)
def searchservers():
print "Searching for servers ..."
for i in xrange(256):
for j in xrange(256):
time.sleep(0.0005)
def app_main():
server = threading.Thread(target=runserver)
thread_pool.append(server)
server.start()
time.sleep(0.1)
searchservers()
stop()
def main():
try:
app_main()
except Exception as exc:
stop()
print "%s occured, joining all threads..." % exc.__class__.__name__
for thread in thread_pool:
thread.join()
raise exc
if __name__ == "__main__":
main()
Why is the KeyboardInterrupt not catched? What is the proper way of catching exceptions in a threaded program and tear down the complete process?
KeyboardInterrupt is a special exception; like MemoryError, GeneratorExit and SystemExit, it does not derive from the base Exception class.
Catching just Exception is thus not enough; you'd normally catch it explicitly:
except (Exception, KeyboardInterrupt) as exc:
However, you are also trying to catch exceptions in threads; threads have their own separate stack; you cannot just go and catch exceptions thrown in those stack in your main thread. You'd have to catch exceptions in that thread:
def runserver():
print "Running server ..."
global running
running = True
try:
while running:
time.sleep(0.07)
except (Exception, KeyboardInterrupt) as exc:
print "Error in the runserver thread"
To handle this in a generic way and 'pass' exceptions to the main thread, you'd need some sort of inter-thread communication. See Catch a thread's exception in the caller thread in Python for a full solution to that.