Alarm signal not firing in infinite loop - python

I'm trying to timeout a function if it runs for more than 3 seconds (for example). I'm using signals and alarms but the alarm never fires. I'd like a timeout mechanism that works for any function. As an example of the problem I'm facing:
import signal
def foobar():
x = 42
while x >= 20:
if x >= 40:
x = 23
return x
def handle_alarm(*args):
print("Alarm raised")
raise TimeoutException("timeout reached")
signal.signal(signal.SIGALRM, handle_alarm)
signal.alarm(3)
try:
print(foobar())
except:
print("Exception Caught")
When run, my program just runs forever and my handler never runs. Any idea why this is the case?
As an aside, if I delete the if statement from foobar, then the alarm does trigger.

On my system, Mac OS X with MacPorts, I tested your code with many version of Python. The only version that exhibits the "bug" you found is 2.7. The timeout works in 2.4, 2.5, 2.6, 3.3, and 3.4.
Now, why is this happening, and what can be done about it?
I think it happens because your foobar() is a tight loop which never "yields" control back to Python's main loop. It just runs as fast as it can, doing no useful work, yet preventing Python from processing the signal.
It will help to understand how signals in *nix are usually handled. Since few library functions are "async signal safe," not much can be done within a C signal handler directly. Python needs to invoke your signal handler which is written in Python, but it can't do that directly in the signal handler that it registers using C. So a typical thing that programs do in their signal handlers is to set some flag to indicate that a signal has been received, and then return. In the main loop, then, that flag is checked (either directly or by using a "pipe" which can be written to in the signal handler and poll()ed or select()ed on).
So I would suppose that the Python main loop is happily executing your foobar() function, and a signal comes in, it sets some internal state to know it needs to handle that signal, and then it waits for for foobar() to end, or failing that, at least for foobar() to invoke some interruptible function, such as sleep() or print().
And indeed, if you add either a sleep (for any amount of time), or a print statement to foobar()'s loop, you will get the timeout you desire in Python 2.7 (as well as the other versions).
It is in general a good idea to put a short sleep in busy loops anyway, to "relax" them, thereby helping scheduling of other work which may need doing. You don't have to sleep on every iteration either--just a tiny sleep every 1000 times through the loop would work fine in this case.

Related

How to make a Python program get into a function and finish with Ctrl+X while running?

My Python program takes a lot of time to complete all the iterations of a for loop. The moment I hit a particular key/key combination on the keyboard while it is running, I want it to go into another method and save the variables into the disk (using pickle which I know) and exit the program safely.
Any idea how I can do this?
Is the KeyboardInterrupt a safe way to this just be wrapping the for loop inside the KeyboardInterrupt exception, catching it and then saving the variables in the except block?
It is only safe if, at every point in your loop, your variables are in a state which allows you to save them and resume later.
To be safe, you could instead catch the KeyboardInterrupt before it happens and set a flag for which you can test. To make this happen, you need to intercept the signal which causes the KeyboardInterrupt, which is SIGINT. In your signal handler, you can then set a flag which you test for in your calculation function. Example:
import signal
import time
interrupted = False
def on_interrupt(signum, stack):
global interrupted
interrupted = True
def long_running_function():
signal.signal(signal.SIGINT, on_interrupt)
while not interrupted:
time.sleep(1) # do your work here
signal.signal(signal.SIGINT, signal.SIG_DFL)
long_running_function()
The key advantage is that you have control over the point at which the function is interrupted. You can add checks for if interrupted at any place you like. This helps with being in a consistent, resumable state when the function is being interrupted.
(With python3, this could be solved nicer using nonlocal; this is left as an excercise for the reader as the Asker did not specify which Python version they are at.)
(This should work on Windows according to the documentation, but I have not tested it. Please report back if it does not so that future readers are warned.)

Not able to timeout using python eventlet library (eventlet.timeout.Timeout)

I am looping over a list and performing some action on each member of the list.
If a member takes too much time (1 sec in this case), I intend to pass it. However the block inside the try statement is always being processed and is never timing out. I don't understand why.
from eventlet import *
for rule in data:
#Timeout block
t=Timeout(1)
try:
f = expr2bdd(expr(rule))
solutions = satisfy_all(f, count=True)
each_rule["solution"]=solutions
except:
pass
finally:
t.cancel()
Eventlet is a concurrent networking library...
It's not clear what expr2bdd and satisfy_all functions do, but most likely they only do some CPU calculations and no disk/network IO. In this case there is no point where Eventlet gets a chance to run and fire timeout exception.
If you have control over expr2bdd and satisfy_all functions and there are any kind of loops, place eventlet.sleep(0) at each iteration. That's Eventlet idiom for "yield control to other coroutines", that's where timeout will be fired.
If you don't have control over said functions, second best option is to run them in a separate process which you can forcefully kill. On POSIX compatible OS (e.g. Linux, *BSD, OSX), you can use os.fork to run a piece of code in separate process. For maximum portability, use subprocess.Popen([sys.executable,...]) or multiprocessing.Process. The latter gives higher level API, mainly around easier data exchange (serialization) at the cost of performance overhead, which may be negligible in your case. In any case, basic pattern is this: (in a thread or eventlet coroutine, you start a second process and then .communicate()/join() on it. Use eventlet.Timeout or Thread.join() with timeout. If timeout fires, use p.terminate() or p.kill() to stop current calculations.

gobject.timeout_add not working in nose tests

I've added a module to mopidy core that uses gobject.timeout_add() for a repeating function. It works fine when running normally, however when running tests it appears that the handler function never gets called.
The module under test has a method that starts a regular process that emits events roughly every half a second. The test calls the method, then call time.sleep(2). The events from the timer function don't occur, and neither does some debug logging in the timer function. Other events and debug logging (outside the timer function) work fine.
What do I need to do to get gobject.timeout_add() to work in nose tests? Or do I need to use something other than time.sleep() in the test in order to allow the other code to run? It's calling gobject.threads_init() in the test setup, is there anything else I need to do?
You need to be running an event loop. As the documentation for g_timeout_add explains, that function (and other similar functions in glib) will create a new timeout source which is then attached to the event loop (well, the GMainContext, but you don't really need to worry about that). It is not the equivalent of spawning a new thread and having it sleep for whatever you specified as the duration of the timeout, which seems to be the behavior you're expecting—using the main loop allows everything to happen in a single thread.

threadable delay in python 2.7

I'm currently using python (2.7) to write a GUI that has some threads going on. I come across a point that I need to do a roughly about a second delay before getting a piece of information, but I can't afford to have the function takes more than a few millisecond to run. With that in mind, I'm trying to create a Threaded timer that will set a flag timer.doneFlag and have the main function to keep poking to see whether it's done or not.
It is working. But not all the time. The problem that I run into is that sometimes I feel like the time.sleep function in run , doesn't wait fully for a second (sometimes it may not even wait). All I need is that I can have a flag that allow me control the start time and raise the flag when it reaches 1 second.
I maybe doing too much just to get a delay that is threadable, if you can suggest something, or help me find a bug in the following code, I'd be very grateful!
I've attached a portion of the code I used:
from main program:
class dataCollection:
def __init__(self):
self.timer=Timer(5)
self.isTimerStarted=0
return
def StateFunction(self): #Try to finish the function within a few milliseconds
if self.isTimerStarted==0:
self.timer=Timer(1.0)
self.timer.start()
self.isTimerStarted=1
if self.timer.doneFlag:
self.timer.doneFlag=0
self.isTimerStarted=0
#and all the other code
import time
import threading
class Timer(threading.Thread):
def __init__(self, seconds):
self.runTime = seconds
self.doneFlag=0
threading.Thread.__init__(self)
def run(self):
time.sleep(self.runTime)
self.doneFlag=1
print "Buzzzz"
x=dataCollection()
while 1:
x.StateFunction()
time.sleep(0.1)
First, you've effectively rebuilt threading.Timer with less flexibility. So I think you're better off using the existing class. (There are some obvious downsides with creating a thread for each timer instance. But if you just want a single one-shot timer, it's fine.)
More importantly, having your main thread repeatedly poll doneFlag is probably a bad idea. This means you have to call your state function as often as possible, burning CPU for no good reason.
Presumably the reason you have to return within a few milliseconds is that you're returning to some kind of event loop, presumably for your GUI (but, e.g., a network reactor has the same issue, with the same solutions, so I'll keep things general).
If so, almost all such event loops have a way to schedule a timed callback within the event loop—Timer in wx, callLater in twisted, etc. So, use that.
If you're using a framework that doesn't have anything like that, it hopefully at least has some way to send an event/fire a signal/post a message/whatever it's called from outside. (If it's a simple file-descriptor-based reactor, it may not have that, but you can add it yourself just by tossing a pipe into the reactor.) So, change your Timer callback to signal the event loop, instead of writing code that polls the Timer.
If for some reason you really do need to poll a variable shared across threads, you really, really, should be protecting it with a Condition or RLock. There is no guarantee in the language that, when thread 0 updates the value, thread 1 will see the new value immediately, or even ever. If you understand enough of the internals of (a specific version of) CPython, you can often prove that the GIL makes a lock unnecessary in specific cases. But otherwise, this is a race.
Finally:
The problem that I run into is that sometimes I feel like the time.sleep function in run , doesn't wait fully for a second (sometimes it may not even wait).
Well, the documentation clearly says this can happen:
The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine.
So, if you need a guarantee that it actually sleeps for at least 1 second, the only way to do this is something like this:
t0 = time.time()
dur = 1.0
while True:
time.sleep(dur)
t1 = time.time()
dur = 1.0 - (t1 - t0)
if dur <= 0:
break

Is there a way to stop time.sleep in windows?

In *nix python signaling allows me to stop sleep before it is ready. Is there any similar mechanism available in windows -- seems that all the methods end up intercepting code only after the sleep?
Code example:
from time import sleep
.. do something that will intercept the sleep
try:
sleep(60)
finally:
print 'how to get here under 60 seconds in windows?'
Similar question that doesn't have an answer for windows:
break/interrupt a time.sleep() in python
The Python documentation for signal says:
• Although Python signal handlers are called asynchronously as far as
the Python user is concerned, they can only occur between the “atomic”
instructions of the Python interpreter. This means that signals
arriving during long calculations implemented purely in C (such as
regular expression matches on large bodies of text) may be delayed for
an arbitrary amount of time.
whereas time says:
The actual suspension time may be less than that requested because any
caught signal will terminate the sleep() following execution of that
signal’s catching routine.
On Windows, apparently time.sleep() is not implemented in accordance with the documentation, because the handler for a signal received during sleep is not acted upon until the full duration of the sleep is finished. The following example:
import signal, time
def handler(signum, frame):
print('Signal handler called with signal', signum)
signal.signal(signal.SIGINT, handler)
print("signal set, sleeping")
time.sleep(10)
print("sleep done")
prints:
signal set, sleeping
Signal handler called with signal 2
sleep done
with the first line occurring immediately and the second two after 10 seconds, regardless of when the interrupt occurs.
As Thomas K suggests, a better strategy would be to use threads and synchronization.
I'm not sure if this will do it for you, but here's a little workaround you could do:
for i in range(60):
if checkIfIShouldInterruptSleep():
break
time.sleep(1)
print "Finished Sleeping!"
It's only accurate to the second, but it might serve your purpose. Happy coding!

Categories

Resources