In *nix python signaling allows me to stop sleep before it is ready. Is there any similar mechanism available in windows -- seems that all the methods end up intercepting code only after the sleep?
Code example:
from time import sleep
.. do something that will intercept the sleep
try:
sleep(60)
finally:
print 'how to get here under 60 seconds in windows?'
Similar question that doesn't have an answer for windows:
break/interrupt a time.sleep() in python
The Python documentation for signal says:
• Although Python signal handlers are called asynchronously as far as
the Python user is concerned, they can only occur between the “atomic”
instructions of the Python interpreter. This means that signals
arriving during long calculations implemented purely in C (such as
regular expression matches on large bodies of text) may be delayed for
an arbitrary amount of time.
whereas time says:
The actual suspension time may be less than that requested because any
caught signal will terminate the sleep() following execution of that
signal’s catching routine.
On Windows, apparently time.sleep() is not implemented in accordance with the documentation, because the handler for a signal received during sleep is not acted upon until the full duration of the sleep is finished. The following example:
import signal, time
def handler(signum, frame):
print('Signal handler called with signal', signum)
signal.signal(signal.SIGINT, handler)
print("signal set, sleeping")
time.sleep(10)
print("sleep done")
prints:
signal set, sleeping
Signal handler called with signal 2
sleep done
with the first line occurring immediately and the second two after 10 seconds, regardless of when the interrupt occurs.
As Thomas K suggests, a better strategy would be to use threads and synchronization.
I'm not sure if this will do it for you, but here's a little workaround you could do:
for i in range(60):
if checkIfIShouldInterruptSleep():
break
time.sleep(1)
print "Finished Sleeping!"
It's only accurate to the second, but it might serve your purpose. Happy coding!
Related
My Python program takes a lot of time to complete all the iterations of a for loop. The moment I hit a particular key/key combination on the keyboard while it is running, I want it to go into another method and save the variables into the disk (using pickle which I know) and exit the program safely.
Any idea how I can do this?
Is the KeyboardInterrupt a safe way to this just be wrapping the for loop inside the KeyboardInterrupt exception, catching it and then saving the variables in the except block?
It is only safe if, at every point in your loop, your variables are in a state which allows you to save them and resume later.
To be safe, you could instead catch the KeyboardInterrupt before it happens and set a flag for which you can test. To make this happen, you need to intercept the signal which causes the KeyboardInterrupt, which is SIGINT. In your signal handler, you can then set a flag which you test for in your calculation function. Example:
import signal
import time
interrupted = False
def on_interrupt(signum, stack):
global interrupted
interrupted = True
def long_running_function():
signal.signal(signal.SIGINT, on_interrupt)
while not interrupted:
time.sleep(1) # do your work here
signal.signal(signal.SIGINT, signal.SIG_DFL)
long_running_function()
The key advantage is that you have control over the point at which the function is interrupted. You can add checks for if interrupted at any place you like. This helps with being in a consistent, resumable state when the function is being interrupted.
(With python3, this could be solved nicer using nonlocal; this is left as an excercise for the reader as the Asker did not specify which Python version they are at.)
(This should work on Windows according to the documentation, but I have not tested it. Please report back if it does not so that future readers are warned.)
I'm trying to timeout a function if it runs for more than 3 seconds (for example). I'm using signals and alarms but the alarm never fires. I'd like a timeout mechanism that works for any function. As an example of the problem I'm facing:
import signal
def foobar():
x = 42
while x >= 20:
if x >= 40:
x = 23
return x
def handle_alarm(*args):
print("Alarm raised")
raise TimeoutException("timeout reached")
signal.signal(signal.SIGALRM, handle_alarm)
signal.alarm(3)
try:
print(foobar())
except:
print("Exception Caught")
When run, my program just runs forever and my handler never runs. Any idea why this is the case?
As an aside, if I delete the if statement from foobar, then the alarm does trigger.
On my system, Mac OS X with MacPorts, I tested your code with many version of Python. The only version that exhibits the "bug" you found is 2.7. The timeout works in 2.4, 2.5, 2.6, 3.3, and 3.4.
Now, why is this happening, and what can be done about it?
I think it happens because your foobar() is a tight loop which never "yields" control back to Python's main loop. It just runs as fast as it can, doing no useful work, yet preventing Python from processing the signal.
It will help to understand how signals in *nix are usually handled. Since few library functions are "async signal safe," not much can be done within a C signal handler directly. Python needs to invoke your signal handler which is written in Python, but it can't do that directly in the signal handler that it registers using C. So a typical thing that programs do in their signal handlers is to set some flag to indicate that a signal has been received, and then return. In the main loop, then, that flag is checked (either directly or by using a "pipe" which can be written to in the signal handler and poll()ed or select()ed on).
So I would suppose that the Python main loop is happily executing your foobar() function, and a signal comes in, it sets some internal state to know it needs to handle that signal, and then it waits for for foobar() to end, or failing that, at least for foobar() to invoke some interruptible function, such as sleep() or print().
And indeed, if you add either a sleep (for any amount of time), or a print statement to foobar()'s loop, you will get the timeout you desire in Python 2.7 (as well as the other versions).
It is in general a good idea to put a short sleep in busy loops anyway, to "relax" them, thereby helping scheduling of other work which may need doing. You don't have to sleep on every iteration either--just a tiny sleep every 1000 times through the loop would work fine in this case.
How can I modify this code (that uses twisted) so that CTRL+C will cause it to exit? I expect the problem is that doWork does not yield control back to the reactor, so the reactor is not able to terminate its execution.
def loop_forever():
i = 0
while True:
yield i
i += 1
time.sleep(5)
def doWork():
for i in loop_forever():
print i
def main():
threads.deferToThread(doWork)
reactor.run()
Note that this code:
def main():
try:
threads.deferToThread(doWork)
reactor.run()
except KeyboardInterrupt:
print "user interrupted task"
does catch the exception on windows, but not on ubuntu
Twisted uses Python's threading library to implement deferToThread. All of the rules that apply to Python threads apply to the threads you get with deferToThread. One rule is that signals and threads are a bad combination (Ctrl-C sends SIGINT on Linux).
The basic idea for solving this problem is to put some logic into doWork so that it will stop. Perhaps this means setting a global flag that it checks once per iteration. You can probably find lots of information elsewhere regarding strategies for getting a long-running thread to cooperate with shutdown.
You may also want to not use deferToThread for this. If you expect your job to run for most of the lifetime of the process then you may just want to use the stdlib threading module directly. Consider that a job like this is using up one of the thread pool slots. If you have enough of these then your thread pool will be full and other work will not be able to proceed.
You may also want to take doWork out of the thread. It doesn't look like it does a lot of blocking. Instead, run doWork in the reactor thread and only run iterations of loop_forever with deferToThread. Now you no longer have a long-running operation in a thread and several of your problems will probably go away.
This question already has answers here:
Add timeout argument to python's Queue.join()
(4 answers)
Closed 7 years ago.
In python, I have multiple threads running and I need the main process to wait until they are done, so I did this with Queue class .join() method. However, I wanted to implement SIGINT but the handler for it wouldn't execute because join() was blocking it(the threading processes run for at least 5 minutes for what I have them doing). So I modified Queue's .join() and placed a time out in wait():
class CustomQueue(Queue.Queue):
#Can not use .join() because it would block any processing
#for SIGINT untill threads are done. To counter this,
# wait() is given a time out along with while not kill_received
#to be checked
def join(self):
self.all_tasks_done.acquire()
try:
while not kill_received and self.unfinished_tasks:
self.all_tasks_done.wait(10.0)
finally:
self.all_tasks_done.release()
This works beautifully and perfect for me.
But what I don't understand is the time out in wait(). For instance, I should not be able to send a SIGINT and have it process for at least 10 seconds. But, I am able to in less than 10 seconds. It doesn't matter what the seconds are, the SIGINT handler function is able to process without being blocked. Why is this? I should have to wait at least 10 seconds for the wait to time out and self.all_tasks_done.release() to run so the SIGINT function will process...rather than the SIGINT function processing before the wait() time out.
We're missing information here that may be important:
Which version of Python?
Which OS?
It may be important because mixing threads with signals is a cross-platform mess. CPython tries to make some sense of it all, but has had various degrees of success across various Python versions and OSes.
Anyway, the answer to your question may be simple ;-) Queue.all_tasks_done is a threading.Condition, and Condition implements .wait(timeout) (eventually, drilling down) using time.sleep(). As documented,
time.sleep(secs)
Suspend execution for the given number of seconds. ... The actual
suspension time may be less than that requested because any caught
signal will terminate the sleep() following execution of that
signal’s catching routine. ...
By default SIGINT raises KeyboardInterrupt. So if you don't have a handler installed for SIGINT, SIGINT terminates the sleep() early and raises KeyboardInterrupt. If you do have a SIGINT handler installed, SIGINT will still terminate the sleep() early, but what happens after that depends on what your handler does.
Why is the sleep method (for sleeping a thread for some time) stored in the time-module and not in the threading-module (like in java).
I mean, is it not much more related to thread?
Sleep method in time module is derived from *nix sleep function (in unistd.h). It's not more thread related, it's a general function for stopping the execution of the program (or script)
time.sleep pauses the execution of the program/script. As a thread can also be regarded as a second program that can interact with the main thread/program, it also affects threads. Regarding the fact that sleep pauses the thread/program/whatever for a certain time, it is surely not wrong to put it into the time module.