Develop Reference

Is there a way to break out of the time.sleep() method? [duplicate]

I need to break from time.sleep() using ctrl c.
While 1:
time.sleep(60)
In the above code when the control enters time.sleep function an entire 60 seconds needs to elapsed for python to handled the CTRL C
Is there any elegant way to do it. such that I can interrupt even when the control is in time.sleep function
edit
I was testing it on a legacy implementation which uses python 2.2 on windows 2000 which caused all the trouble . If I had used a higher version of python CTRL C would have interrupted the sleep() . I did a quick hack by calling sleep(1) inside a for loop . which temporarily fixed my issue

The correct answer is to use python stdlib's threading.Event
Sure you can tune down your sleep interval so you sleep for very short periods, but what if you actually want to run your loop once every 60s? Then you need to do more work to determine if it's time to run or just keep sleeping. Furthermore, you're still technically blocking but for only a short period of time. Contrast to threading.Event:
from threading import Event
exit = Event()
def main():
while not exit.is_set():
do_my_thing()
exit.wait(60)
print("All done!")
# perform any cleanup here
def quit(signo, _frame):
print("Interrupted by %d, shutting down" % signo)
exit.set()
if __name__ == '__main__':
import signal
for sig in ('TERM', 'HUP', 'INT'):
signal.signal(getattr(signal, 'SIG'+sig), quit);
main()
When the signal handler calls exit.set(), the main thread's wait() call will immediately be interrupted.
Now, you could use an Event to signal that there's more work to do, etc. But in this case it does double duty as a convenient indicator that we want to quit (e.g. the while not exit.is_set() part.)
You also have the option to put any cleanup code after your while loop.

Not sure what the sense of this code is - but if necessary use a shorter sleep() interval and put a for loop around it:
for i in range(60):
sleep(1)
Catching the KeyboardInterrupt exception using try..except is straight-forward

The KeyboardInterrupt exception is raised when a user hits the interrupt key, Ctrl-C. In python this is translated from a SIGINT signal. That means, you can get handle it however you want using the signal module:
import signal
def handler(signum, frame):
print("do whatever, like call thread.interrupt_main()")
signal.signal(signal.SIGINT, handler)
print("Waiting for SIGINT...")
signal.pause()
That way, you can do whatever you want at the receipt of a keyboard interrupt.

The most elegant solution is certainly threading.Event, though if you only need a quick hack, this code works well :
import time
def main():
print("It’s time !")
if __name__ == "__main__":
print("press ctrl-c to stop")
loop_forever = True
while loop_forever:
main()
try:
time.sleep(60)
except KeyboardInterrupt:
loop_forever = False

I tried your code with python versions 2.5, 2.6, 3 under Linux and all throw "KeyboardInterrupt" exception when hitting CTRL-C.
Maybe some exception handling catches the Interrupt or your problem is like this:
Why is KeyboardInterrupt not working in python?

Based on #Andreas Jung answer
for i in range(360):
try:
sleep(1)
except KeyboardInterrupt:
sys.exit(0)

Figured I'd throw this in.
import time
def sleep(seconds):
for i in range(seconds):
try:
time.sleep(1)
except KeyboardInterrupt:
print("Oh! You have sent a Keyboard Interrupt to me.\nBye, Bye")
break
sleep(60)

Rerunning python script after a timeout or an error

I am trying to find a way in which I can re run my code if there is a timeout or an error.
This is something that happens if the internet connection drops - So I'd need to delay for a few seconds whilst it comes back online and try again..
Would there be a way in which I could run my code in another python script and tell it to re run if it times out or disconnects?
Thanks in advance

If you are talking about http request connection error and if you are using the requests library you can use this urllib retry
FYI https://docs.python-requests.org/en/master/api/#requests.adapters.HTTPAdapter
Of course, other libraries will have their own retry function.
If you just want simple retry code, use the below code
retries_count = 3 # hard value
delay = 3 # hard value
while True:
try:
... run some code
return or break
except {Your Custom Error}:
if retries_count <= 0:
raise
retries_count -= 1
time.sleep(delay)
A Google search for "python throttling" will give you a lot of reference.

You can use a try-catch in an infinite loop like this:
while True:
try:
# your code
# call another python script
except:
time.sleep(10)
also, you can check if the error and run whatever you need to run, depend on the error type. for example:
while True:
try:
# your code
# call another python script
# break
except Exception as e:
if e == 'error type':
time.sleep(10)
else:
pass

You actually can do it with try except block.
You can find all about it here: https://docs.python.org/3/tutorial/errors.html
You just make one script, that has something like that:
while True:
try:
another_script_name.main()
except:
time.sleep(20)
Of course, you need to import both time and the other script you made.
What these lines are doing is just an infinite loop, that always tries to run the main function on the other script you made, and if some sort of error occurs, the system will sleep for 20 seconds and then will try again, because it is in the infinite loop.

How do I loop through keyboard interrupt whenever the user presses ctrl+c multiple times in a row

I am using python3 on android 6 (termux linux emulator) and python works great, but I want to create a login everytime termux starts up by adding this program to ~/.bashrc, the problem is when I ask the user for the password, and they enter ctrl-c, the program will be alerted that they tried to bypass security, and force them to wait for 5 minutes before trying again. So my question is how can I loop the KeyboardInterrupt so while they are waiting 5 minutes and they press ctrl-c again it will make them wait 10 minutes, 15 minutes (up by 5) etc and so forth, so they cant bypass the waiting time?

I agree with the user 9000, but if you really want to do this, my solution would be the following:
import time
password_check_passed = False
while not password_check_passed:
try:
password_check_passed = check_password()
except KeyboardInterrupt:
print("You are required to authenticate yourself. You've received a cooldown of 5 minutes.")
end_of_punishment = time.time() + 5*60 # now + 5 minutes
while time.time() < end_of_punishment:
try:
# attempt to sleep until the end of the punitive timeout
time.sleep(end_of_punishment - time.time())
except KeyboardInterrupt:
print("Still not getting it? 5 more minutes!")
end_of_punishment += 5*60
The check_password function would return a boolean indicating that it succeeded and print out any instructions for authentication.
While there is technically a possibility to send two interrupts fast enough for the second one to occur before the inner try clause, it is very small and I wouldn't worry about it.
What you should be more worried about is that this makes termination of your program much more difficult.

Python Threaded Timer Returning Random Errors

I have a python thread that runs every 20 seconds. The code is:
import threading
def work():
Try:
#code here
except (SystemExit, KeyboardInterrupt):
raise
except Exception, e:
logger.error('error somewhere',exc_info=True)
threading.Timer(20, work).start ();
It usually runs completely fine. Once in a while, it'll return an error that doesnt make much sense. The errors are the same two errors. The first one might be legitimate, but the errors after that definitely aren't. Then after that, it returns that same error every time it runs the thread. If I kill the process and start over, then it runs cleanly. I have absolutely no idea what going on here. Help please.

As currently defined in your question, you are most likely exceeding your maximum recursion depth. I can't be certain because you have omitted any opportunities for flow control that may be evident in your try block. Furthermore, everytime your code fails to execute, the general catch for exceptions will log the exception and then bump you into a new timer with a new logger (assume you are declaring that in the try block). I think you probably meant to do the following:
import threading
import time
def work():
try:
#code here
pass
except (SystemExit, KeyboardInterrupt):
raise
except Exception, e:
logger.error('error somewhere',exc_info=True)
t = threading.Timer(20, work)
t.start()
i = 0
while True:
time.sleep(1)
i+=1
if i >1000:
break
t.cancel()
If this is in fact the case, the reason your code was not working is that when you call your work function the first time, it processes and then right at the end, starts another work function in a new timer. This happens add infinitum until the stack fills up, python coughs, and gets angry that you have recursed (called a function from within itself) too many times.
My code fix pulls the timer outside of the function so we create a single timer, which calls the work function once every 20 seconds.
Because threading.timers run in separate threads, we also need to wait around in the main thread. To do this, I added a simple while loop that will run for 1000 seconds and then close the timer and exit. If we didn't wait around in the main loop, it would call your timer and then close out immediately causing python to clean up the timer before it executed even once.

Scrapy freeze on connection timeout

I wrote a scrapy crawler that uses an Internet connection that is pretty unreliable. This is something that is a given. I cannot easily or cheaply change it - once in a while the Internet connection will be lost and after a few seconds or so it will be restored.
I observe behaviour where a Scrapy 18.4 crawler would freeze indefinitely without printing any error messages. It stops reacting to Ctrl+C, which makes me think this happens somewhere pretty deep in the reactor stack, though I cannot be sure.
There are absolutely no error messages which makes things rather hopeless to debug.
Question: Would anyone have any clues as to how to debug this problem? I don't really have any meaningful logs to attach for the reasons laid out above.

You can set up a class for timeout and then run your code in a try except. Something like this:
import signal
class timeout:
def __init__(self, seconds=1, error_message='Timeout'):
self.seconds = seconds
self.error_message = error_message
def handle_timeout(self, signum, frame):
raise TimeoutError(self.error_message)
def __enter__(self):
signal.signal(signal.SIGALRM, self.handle_timeout)
signal.alarm(self.seconds)
def __exit__(self, type, value, traceback):
signal.alarm(0)
def run_crawl():
while True:
print("This runs")
try:
with timeout(seconds=3):
run_crawl()
except Exception as e:
print(e)
Note: Since this uses signal it will only work on UNIX-based systems.
If you want it to go back to running the crawler (auto-restart), then you can just put it all in an infinite loop.
while True:
print("Restarting spider")
try:
with timeout(seconds=3):
run_crawl()
except Exception as e:
print(e)
This is all assuming that you can just keep restarting the bot after x seconds without major negative results. IE if you are constantly scraping the same page(s) over and over again, then this would work pretty seamlessly.
However, if you are scraping a very long list [once] and just want it to finish without an error, then it would work less well, but could still be used by setting x to a number that represents an amount of time greater than the duration of the entire process when it executes successfully (this is true no matter the length of the execution of the program - don't set x to 3 seconds if you are scraping one site that takes 7 seconds to complete, and the same is true if you are doing 5 that take 30 seconds or 500 that take 5 minutes, you will need to set x to an amount greater than that duration.
The reason that I specifically separated your bot having a quick completion time vs. not is that if it fails during a loop with a timeout of 30 seconds then you lose, on average, 15 seconds if it fails, but if you have a 30 minute execution time then you would lose, on average, 15 minutes when it fails, and if your internet is going out every 15 minutes, on average, then you would have it failing a vast majority of the time and you'll need to look more into debugging the problem and actually solving it instead of working around it, as this "solution" should definitely be considered a workaround.