In my complex python program, when it's running, I have a piece of code that executes every 3 seconds that prints the program's progress as the percentage of the execution that's finished like so:
while len(dequeueingFinishedList)!=10:
print(str(len(masterListCSV_RowsListFinished)/float(len(masterListCSV_RowsList))*100) + "% done.")
time.sleep(3)
Is the time.sleep() function going to slow down my program? I read the that sleep function suspends execution. If it is slowing down my program, is there a more correct way of printing the progress to me every 3 seconds?
Yes, time.sleep will halt your program.
Use time.time in your loop and check when three seconds have passed.
time.sleep(seconds) will stop execution on the current thread. Therefore, it will completely stop your program on that thread: nothing else will happen until those seconds pass.
You don't have to worry about this. If the program uses threading, then the other threads shouldn't halt.
from time import time
prev = time()
while True:
now = time()
if now - prev > 3:
print 'report'
prev = now
else:
pass
# runs
The proper way to do this is with signal
import signal
def handler(signum, frame):
print i
if i>100000000:
raise Exception("the end")
else:
signal.alarm(3)
signal.signal(signal.SIGALRM, handler)
signal.alarm(3)
i=0
while True:
i+=1
Related
I have a code which may run into long loops. These are not infinite loops but certain inputs may cause the loop to continue for extended periods. I wish to use an interrupt in case a loop goes too long. With this interrupt, the loop will re-evaluate its inputs, so a keyboard interrupt is not what I am looking for. I am using python 2.7 on Windows.
One possible approach would be polling using time.time(), yet it would be extremely inefficient. The loop itself is not long, however even in normal runs it is iterated 100k times, so I can't poll the time after each iteration, it would decrease efficiency a lot. Alternatively, I could introduce on more variable,
i=0
while i<some_very_large_number:
function(foo, foo1, foo2)
i=i+1
but this again, would be a mediocre solution.
In short, what I am looking for is some python equivalent to the hardware interrupt of microprocessors. I have no experience in multi-threading, so if the answer lies in multi-threading, please elaborate a little bit.
I have checked here and here, yet I do not think they answer my question. The second link could actually help yet apparently signal package is not available in Windows.
The code is long but straightforward. It basically has such a structure.
def function(foo, foo1, foo2, N):
for i in range(N):
performance = performance_evaluator(foo, foo1, foo2)
if performance_takes_too_long:
interrupt ##this if clause is the main objective of this question##
record performance
new_foo, new_foo1, new_foo2 = evolve(foo, foo1, foo2)
One possible approach is to modify your script so that it takes its inputs via commandline arguments, then use subprocess module to run it with a timeout:
# manager.py
import subprocess
try:
code = subprocess.call('python work.py 5', timeout=2)
print('Ended with code:', code)
except subprocess.TimeoutExpired:
print('Ended with timeout')
# work.py
import sys
from time import sleep
try:
wait = int(sys.argv[1])
except:
wait = 10
sleep(wait)
print(f'Waited for {wait} seconds')
output:
Ended with timeout
You can also execute a long-running code in a separate worker process and try to terminate it when a timeout is exceeded and the worker has not been finished yet:
import time
from multiprocessing import Process
def process_data():
while True:
print("processing...")
time.sleep(1)
def main():
worker = Process(target=process_data)
worker.start()
timeout = 5
time.sleep(timeout)
if worker.is_alive():
print("exceeded timeout", timeout, "sec")
print("terminate worker", worker)
worker.terminate()
worker.join()
print("is worker", worker, "alive:", worker.is_alive())
if __name__ == "__main__":
main()
Here is the output:
processing...
processing...
processing...
processing...
processing...
exceeded timeout 5 sec
terminate worker <Process(Process-1, started)>
is worker <Process(Process-1, stopped[SIGTERM])> alive: False
I'm trying to place a call 'on hold' in asterisk using a python AGI script, the function will check if the person is available, when he is asterisk will dial the person if he is not the script should wait 10 seconds before checking availabilty again and dial when the person is available. However I ran into a small problem, using the time.sleep(10) function hangs up the call when a person is not available, I expect this is because the thread the script runs on will sleep and asterisk thinks the script is done running and hangs up the call. Removing the time.sleep() gives me what I want without the time interval.
agi = AGI()
agi.verbose("python agi started")
place_call_on_hold("SIP/6001")
def place_call_on_hold(s):
agi.verbose("entered place_call_on_hold function")
while True:
status = agi.get_variable('DEVICE_STATE(%(redirect)s)'%{'redirect':s})
agi.verbose(status)
if status == 'NOT_INUSE':
agi.verbose("Info: place_call_on_hold: calling the number")
agi.appexec("Dial",s)
else:
agi.verbose("Info: place_call_on_hold: sleeping for 10 sec")
time.sleep(10)
Is there a way to wait 10 seconds without using the sleep() function or how can I make sure the call won't end before the time.sleep wakes back up?
why not try time difference before condition and after condition something like this
import time
start = time.time()
end = time.time()
while (end-start) < 10:
end = time.time()
print(start, end, end-start)
I fixed my problem by just calling the Asterisk Wait(10) function instead of time.sleep(10).
I have a command in a line (Fit.perform() from import xspec, but never mind because the question is general and can be applicated also for other python commands) that takes a while to finish.
I simple want to know the time of execution while the command is running, so when it has not finished its execution yet.
This is necessary if I want to stop the command during its execution, for example because it is taking too much time to end.
So, I need something like this:
if **you_are_taking_so_much_time**:
do_something_else
It is not possible to use methods like time or timeit because they calculate the time only at the end of execution of a command and not while it is running.
Is it possible?
I'm using python 2.7 on MacOS.
You will have to use a monitor thread:
import threading
import time
done = False
def longfun():
global done
print("This will take some time.")
time.sleep(60)
done = True
def monitor():
global done
timeout = 10
print("Wait until timeout.")
while not done and timeout > 0:
time.sleep(1)
timeout -= 1
lt = threading.Thread(target=longfun)
lt.start()
mt = threading.Thread(target=monitor)
mt.start()
mt.join()
if done == False:
print("Long thread not done yet. Do something else.")
lt.join()
Note that this does wait until the 'long' thread is finished. You do not mention you want to stop the long running operation. If you do, you will have to correctly implement it in a thread, including start/stop/progress functionality (usually this works with a while loop that uses a running bit to see if it should continue.
like this:
import time,thread
def test_me(hargs):
func,args,timeout = hargs
start_time = time.time()
thread.start_newthread(func,args)
while True :
if My_expected_value:#where store this ?
print "well done !"
break
elif time.time() > (timeout + start_time) :
print "oh! to late, sorry !"
break
time.sleep(timeout/100)
thread.start_newthread(test_me,((func,args,timeout),))
important warnings : need use thread for Non-freezing application, got 3 thread for this: 1-main app, 2-test_me, 3- Your function(func)
Don't forget adding external variable to your function (for killing your function thread)
Forgive me, I am a newbie. I've surveyed some solution. But it is so hard for me to understand and to modify that. (Or maybe there is no solution in line with my imagination?). And I hope it can work on Ubuntu & Win7.
There is an example like this.
import random,time
def example():
while random.randint(0,10) != 1:
time.sleep(1)
print "down"
example()
And my imagination is...
If, the example() run over 10s, then rerun the example() again. (And maybe there is a place I can code anything else. Like I want to record the timeout event on TXT, and I can code the code at that place.)
Else, do nothing.
Is it possible to do that?
You can run a watch-dog in a separate thread that interrupts the main thread (that runs example) when it exceeds the time limit. Here is a possible implementation, with timeout lowered to 3s for ease of debugging:
import time, threading, thread
def watchdog_timer(state):
time.sleep(3)
if not state['completed']:
thread.interrupt_main()
def run_example():
while True:
state = {'completed': False}
watchdog = threading.Thread(target=watchdog_timer, args=(state,))
watchdog.daemon = True
watchdog.start()
try:
example()
state['completed'] = True
except KeyboardInterrupt:
# this would be the place to log the timeout event
pass
else:
break
I'm not sure if I fully understood what you want to achieve, but as you're constantly looping and only have one short and predictable blocking command, you could simply store the time when the loop started and then compare it to the current time once per loop iteration. If the difference exceeds your limit, do whatever you want:
import random,time
time_limit=10
def example():
time_start = time.time() # store current time (seconds since 1970)
while random.randint(0,10) != 1:
time.sleep(1)
if (time.time() >= time_start + time_limit): # compare with current time
print "canceled!"
break # break the while-loop
print "down"
example()
I have a "do..., until..." structure in Python as follows:
while True:
if foo() == bar():
break
It works fine (jumps out in the end) in most of the cases. However, in some of the cases where the condition is never met, it will get stuck there.
Figuring out what are these cases is kind of difficult, since it is essentially a random process behind. So I wish to set a "timeout" thing for the while loop.
Say, if the loop has been running for 1s, but still not yet stops, I wish the loop to terminate itself.
How may I do this?
Update: Here is the actual code:
while True:
possibleJunctions = junctionReachability[junctions.index(currentJunction)]
nextJunction = random.choice(filter(lambda (jx, jy): (jx - currentJunction[0]) * (endJunction[0] - currentJunction[0]) > 0 or (jy - currentJunction[1]) * (endJunction[1] - currentJunction[1]) > 0, possibleJunctions) or possibleJunctions)
if previousJunction != nextJunction: # never go back
junctionSequence.append(nextJunction)
previousJunction = currentJunction
currentJunction = nextJunction
if currentJunction == endJunction:
break
import time
loop_start = time.time()
while time.time() - loop_start <= 1:
if foo() == bar():
break
EDIT
Dan Doe's solution is simplest and best if your code is synchronous (just runs in a single thread) and you know that the foo and bar functions always terminate within some period of time.
If you have asynchronous code (like a GUI), or if the foo and bar functions you use to test for termination conditions can themselves take too long to complete, then read on.
Run the loop inside a separate thread/process. Run a timer in another process. Once the timer expires, set a flag that would cause the loop to terminate.
Something like this (warning: untested code):
import multiprocessing
import time
SECONDS = 10
event = multiprocessing.Event()
def worker():
"""Does stuff until work is complete, or until signaled to terminate by timer."""
while not event.is_set():
if foo() == bar():
break
def timer():
"""Signals the worker to terminate immediately."""
time.sleep(SECONDS)
event.set()
def main():
"""Kicks off subprocesses and waits for both of them to terminate."""
worker_process = multiprocessing.Process(target=worker)
timer_process = multiprocessing.Process(target=timer)
timer_process.start()
worker_process.start()
timer_process.join()
worker_process.join()
if __name__ == "__main__":
main()
If you were worried about the foo and bar functions taking too long to complete, you could explicitly terminate the worker process from within the timer process.
I recommend using a counter. This is a common trick to detect non-convergence.
maxiter = 10000
while True:
if stopCondition(): break
maxiter = maxiter - 1
if maxiter <= 0:
print >>sys.stderr, "Did not converge."
break
this requires the least overhead and usually adapts best to different CPUs: even on a faster CPU, you want the same termination behavior; instead of a time-based timeout.
However, it would be even better if you would detect being stuck e.g. with some criterion function that no longer improves.