Develop Reference

How to make a loop that calculates running time?

I'm making a simple client/server program in Python 3 and in the client I would like a clock or printout of the running time. I'm trying to make it in a loop that starts at the beginning of the program, but in a thread so the rest of the code keeps going.
class time_thread():
def run(self):
loop = 0
while (zetime > -1):
print(zetime);
zetime = zetime + 1;
time_thread.start()
zetime = 0
This is what I have so far, but it doesn't work. It says:
time_thread has no attribute start()
I'm new to this and haven't used threads before, so I'm not sure how to go about this. Is there a better way?

I think this is what you're looking for:
import time, sys
zetime = 0
while (zetime > -1):
sys.stdout.write("\r" + str(zetime))
sys.stdout.flush()
time.sleep(1)
zetime = zetime + 1;

First of all , to use Thread module, you have to inherit the class Thread on your class, so you can use their methods like start.
To calculate time, you can use datetime.
from datetime import datetime
from time import sleep
start_time = datetime.now()
sleep(5) # code that you want to calculate.
end_time = datetime.now()
print(end_time - start_time)
Just place this

So let's say you define a function elapsed_time such as:
import time, sys
def elapsed_time(time_start):
''' time_start: a time.time()
Goal: print the elapsed time since time_start '''
# Allow to stop the counting once bool_time = False in the main program
global bool_elapsed_time
# loop while the condition
while bool_elapsed_time == True:
# Note: erase and write below does not work in spyder but in a shell yes
print ("Elapsed time: {} seconds".format(int(time.time() - time_start))),
sys.stdout.flush()
print ("\r"),
time.sleep(1)
# erase the line with elapsed time to clean the shell at the end
sys.stdout.flush()
print ("\r"),
Then in your program:
import threading
bool_elapsed_time = True
t = threading.Thread(name='child procs', target=elapsed_time, args=(time.time(),))
t.start()
## Here add the job you want to do as an example:
time.sleep(10)
bool_elapsed_time = False #to stop the elapsed time printing
Should do the job you want to do.
Note: I used python 2.7 so it might be slightly different with 3.x

Making a timer: timeout inaccuracy of threading.Event.wait - Python 3.6

First of all, I am new to Python and not familiar with its functionalities. I've been mainly using MATLAB.
PC brief spec.: Windows 10, Intel i7
I am trying to make a timer class for periodic execution of a function such as MATLAB has, which is obviously borrowed from Java timer. The MATLAB timer has an about 1 ms resolution and I've never seen it exceeds 2 ms in any situation. In fact, it is accurate enough for my project.
Recently, I planned to move to Python because of the poor parallel computing and web access features of MATLAB. However, unfortunately, the standard packages of Python offer somewhat low-level of timer (threading.Timer) compared to MATLAB that I had to make my own timer class. First, I referred to the QnA Executing periodic actions in Python [duplicate]. The solution suggested by Michael Anderson gives a simple idea of drift correction. He used time.sleep() to keep the period. The approach is highly accurate and sometimes showed better accuracy over the MATLAB timer. approx. 0.5 ms resolution. However, the timer cannot be interrupted (pause or resume) during being captured in time.sleep(). But I sometimes have to stop immediately regardless of whether it is in sleep() or not.
A solution to the problem I found is to utilize the Event class in threading package. Refer to Python threading.timer - repeat function every 'n' seconds
. Using the timeout feature of Event.wait(), I could make a time gap between executions and it is used to keep the period. That is, the event is usually cleared so that wait(timeout) can act like time.sleep(interval) and I could exit from wait() immediately, when needed, by setting event.
Everything seemed fine then but there is a critical problem in Event.wait(). The time delay varies too largely from 1 ~ 15 ms. I think it comes from the overhead of Event.wait().
I made an example code that shows accuracy comparison between time.sleep() and Event.wait(). This sums total of 1000 iterations of 1 ms sleep() and wait() to see the accumulated time error. The expected result is about 1.000.
import time
from threading import Event
time.sleep(3) # to relax
# time.sleep()
tspan = 1
N = 1000
t1 = time.perf_counter()
for _ in range(N):
time.sleep(tspan/N)
t2 = time.perf_counter()
print(t2-t1)
time.sleep(3) # to relax
# Event.wait()
tspan = 1
event = Event()
t1 = time.perf_counter()
for _ in range(N):
event.wait(tspan/N)
t2 = time.perf_counter()
print(t2-t1)
Result:
1.1379848184879964
15.614547161211096
The result shows that time.sleep() is much better in accuracy. But I cannot purely rely on time.sleep() as previously mentioned.
In summary,
time.sleep(): accurate but not interruptible
threading.Event.wait(): inaccurate but interruptible
I am currently thinking of a compromise: just as in the example, make a loop of tiny time.sleep() (of 0.5 ms interval) and exit the loop using if-statement and break when needed. As far as I know, the method is used in Python 2.x Python time.sleep() vs event.wait().
It was a verbose introduction, but my question can be summarized as follows.
Can I force thread process to break from time.sleep() by an external signal or event? (This seems to be most efficient.???)
To make Event.wait() more accurate or to reduce overhead time.
Are there any better approaches aside of sleep() and Event.wait() approach to improve timing precision.
Thank you very much.

I ran into the same timing issue with Event.wait(). The solution I came up with was to create a class which mimics threading.Event. Internally, it uses a combination of a time.sleep() loop and a busy loop for greatly increased precision. The sleep loop runs in a separate thread so that the blocking wait() call in the main thread can still be immediately interrupted. When the set() method is called, the sleep thread should terminate shortly afterwards. Also, in order to minimize CPU utilization, I made sure that the busy loop will never run for more than 3 milliseconds.
Here is my custom Event class along with a timing demo at the end (the printed execution times from the demo will be in nanoseconds):
import time
import _thread
import datetime
class Event:
__slots__ = (
"_flag", "_lock", "_nl",
"_pc", "_waiters"
)
_lock_type = _thread.LockType
_timedelta = datetime.timedelta
_perf_counter = time.perf_counter
_new_lock = _thread.allocate_lock
class _switch:
__slots__ = ("_on",)
def __call__(self, on: bool = None):
if on is None:
return self._on
self._on = on
def __bool__(self):
return self._on
def __init__(self):
self._on = False
def clear(self):
with self._lock:
self._flag(False)
def is_set(self) -> bool:
return self._flag()
def set(self):
with self._lock:
self._flag(True)
waiters = self._waiters
for waiter in waiters:
waiter.release()
waiters.clear()
def wait(
self,
timeout: float = None
) -> bool:
with self._lock:
return self._wait(self._pc(), timeout)
def _new_waiter(self) -> _lock_type:
waiter = self._nl()
waiter.acquire()
self._waiters.append(waiter)
return waiter
def _wait(
self,
start: float,
timeout: float,
td=_timedelta,
pc=_perf_counter,
end: _timedelta = None,
waiter: _lock_type = None,
new_thread=_thread.start_new_thread,
thread_delay=_timedelta(milliseconds=3)
) -> bool:
flag = self._flag
if flag:
return True
elif timeout is None:
waiter = self._new_waiter()
elif timeout <= 0:
return False
else:
delay = td(seconds=timeout)
end = td(seconds=start) + delay
if delay > thread_delay:
mark = end - thread_delay
waiter = self._new_waiter()
new_thread(
self._wait_thread,
(flag, mark, waiter)
)
lock = self._lock
lock.release()
try:
if waiter:
waiter.acquire()
if end:
while (
not flag and
td(seconds=pc()) < end
):
pass
finally:
lock.acquire()
if waiter and not flag:
self._waiters.remove(waiter)
return flag()
#staticmethod
def _wait_thread(
flag: _switch,
mark: _timedelta,
waiter: _lock_type,
td=_timedelta,
pc=_perf_counter,
sleep=time.sleep
):
while not flag and td(seconds=pc()) < mark:
sleep(0.001)
if waiter.locked():
waiter.release()
def __new__(cls):
_new_lock = cls._new_lock
_self = object.__new__(cls)
_self._waiters = []
_self._nl = _new_lock
_self._lock = _new_lock()
_self._flag = cls._switch()
_self._pc = cls._perf_counter
return _self
if __name__ == "__main__":
def test_wait_time():
wait_time = datetime.timedelta(microseconds=1)
wait_time = wait_time.total_seconds()
def test(
event=Event(),
delay=wait_time,
pc=time.perf_counter
):
pc1 = pc()
event.wait(delay)
pc2 = pc()
pc1, pc2 = [
int(nbr * 1000000000)
for nbr in (pc1, pc2)
]
return pc2 - pc1
lst = [
f"{i}.\t\t{test()}"
for i in range(1, 11)
]
print("\n".join(lst))
test_wait_time()
del test_wait_time

Chris D's custom Event class works impressively well! For practical purposes, I have included it into an installable package (https://github.com/ovinc/oclock, install with pip install oclock) that also includes other timing tools. From version 1.3.0 of oclock and onwards, one can use the custom Event class discussed in Chris D's answer, e.g.
from oclock import Event
event = Event()
event.wait(1)
with the usual set(), clear(), is_set(), wait() methods of the Event class.
The timing accuracy is much better than with threading.Event, in Windows in particular. For example on a Windows machine with 1000 repeated loops, I get a standard deviation in the duration of the loop of 7ms for threading.Event and less than 0.01 ms for oclock.Event. Props to Chris D!
Note: The oclock package is under the GPLv3 license for compatibility with StackOverflow's CC BY-SA 4.0.

Thank you for this topic and all answers. I also experienced some troubles with inaccurate timing (Windows 10 + Python 3.9 + Threading).
The solution is to use oclock package and also change (temporarily) resolution of Windows system timer by wres package. This package utilizes undocumented Windows API function NtSetTimerResolution (warning: resolution is changed system-wide).
Application of oclock package only does not solve the problem.
With both python packages applied, the code below schedules periodic event correctly and precisely enough. If terminated, original timer resolution is restored.
import threading
import datetime
import time
import oclock
import wres
class Job(threading.Thread):
def __init__(self, interval, *args, **kwargs):
threading.Thread.__init__(self)
# use oclock.Event() instead of threading.Event()
self.stopped = oclock.Event()
self.interval = interval.total_seconds()
self.args = args
self.kwargs = kwargs
def stop(self):
self.stopped.set()
self.join()
def run(self):
prevTime = time.time()
while not self.stopped.wait(self.interval):
now = time.time()
print(now - prevTime)
prevTime = now
# Set system timer resolution to 1 ms
# Automatically restore previous resolution when exit with statement
with wres.set_resolution(10000):
# Create thread with periodic task called every 10 ms
job = Job(interval=datetime.timedelta(seconds=0.010))
job.start()
try:
while True:
time.sleep(1)
# Hit Ctrl+C to terminate main loop and spawned thread
except KeyboardInterrupt:
job.stop()

Python 3.4 Multiprocessing, code does not progress past loop (containing Queue)

I have written a bit of code to test out multiprocessing before I implement in other code. The main code sends a number to the other program (running on another core). That program increments and returns the number. main code increments and returns the value again. All works well, but the loop in the main program, which is a while time < Timeout form, is never exited. Seems simple enough to me, but it never exists the loop. I wondered if it was hanging when no value is returned (.get() ), but I have tried 'try:', and making the timeout very short and the loop huge. Any suggestions what is going on?
The code is running on Windows7, and will eventually run on a Raspberry Pi 2.
Main program
import multiprocessing as mp
import multi_processing_slave as MPS
from time import perf_counter as TimeIs
if __name__ == "__main__":
print("Hello World")
mp.set_start_method("spawn")
q = mp.Queue()
r = mp.Queue()
p = mp.Process(target = MPS.foo, args = (q, r))
p.start()
ThisVar = 0
Timer = TimeIs() + 2
while TimeIs() < Timer - 1: pass
print("time remaining is", Timer - TimeIs())
while TimeIs() < Timer:
#try:
r.put(ThisVar)
#except: pass
#try:
ThisVar = int(q.get()) + 1
#except:
#r.put(ThisVar)
print("master ThisVar", ThisVar, "and time remaining is", round(Timer - TimeIs(), 4))
#p.join()
#p.close()
p.terminate()
print("at end, ThisVar is", ThisVar, "and", Timer - TimeIs(), "seconds remaining")
Slave program named multi_processing_slave
def foo(q, r):
for i in range(100):
ThisVar2 = r.get() + 1
q.put(ThisVar2)
print("foo value", ThisVar2)
print("foo has finished")
return

After the slave process exits, your master process does one more r.put() and then keeps waiting for q.get() to return. You can solve the problem by providing a timeout value (in seconds) to q.get():
ThisVar = int(q.get(timeout=1)) + 1
Note that timing out on q.get() will raise the Empty exception.
How you could find this problem yourself:
Add debugging print statements that show the progress and state of your program.
Learn to use a debugger.

Is it possible to execute function every x seconds in python, when it is performing pool.map?

I am running pool.map on big data array and i want to print report in console every minute.
Is it possible? As i understand, python is synchronous language, it can't do this like nodejs.
Perhaps it can be done by threading.. or how?
finished = 0
def make_job():
sleep(1)
global finished
finished += 1
# I want to call this function every minute
def display_status():
print 'finished: ' + finished
def main():
data = [...]
pool = ThreadPool(45)
results = pool.map(make_job, data)
pool.close()
pool.join()

You can use a permanent threaded timer, like those from this question: Python threading.timer - repeat function every 'n' seconds
from threading import Timer,Event
class perpetualTimer(object):
# give it a cycle time (t) and a callback (hFunction)
def __init__(self,t,hFunction):
self.t=t
self.stop = Event()
self.hFunction = hFunction
self.thread = Timer(self.t,self.handle_function)
def handle_function(self):
self.hFunction()
self.thread = Timer(self.t,self.handle_function)
if not self.stop.is_set():
self.thread.start()
def start(self):
self.stop.clear()
self.thread.start()
def cancel(self):
self.stop.set()
self.thread.cancel()
Basically this is just a wrapper for a Timer object that creates a new Timer object every time your desired function is called. Don't expect millisecond accuracy (or even close) from this, but for your purposes it should be ideal.
Using this your example would become:
finished = 0
def make_job():
sleep(1)
global finished
finished += 1
def display_status():
print 'finished: ' + finished
def main():
data = [...]
pool = ThreadPool(45)
# set up the monitor to make run the function every minute
monitor = PerpetualTimer(60,display_status)
monitor.start()
results = pool.map(make_job, data)
pool.close()
pool.join()
monitor.cancel()
EDIT:
A cleaner solution may be (thanks to comments below):
from threading import Event,Thread
class RepeatTimer(Thread):
def __init__(self, t, callback, event):
Thread.__init__(self)
self.stop = event
self.wait_time = t
self.callback = callback
self.daemon = True
def run(self):
while not self.stop.wait(self.wait_time):
self.callback()
Then in your code:
def main():
data = [...]
pool = ThreadPool(45)
stop_flag = Event()
RepeatTimer(60,display_status,stop_flag).start()
results = pool.map(make_job, data)
pool.close()
pool.join()
stop_flag.set()

One way to do this, is to use main thread as the monitoring one. Something like below should work:
def main():
data = [...]
results = []
step = 0
pool = ThreadPool(16)
pool.map_async(make_job, data, callback=results.extend)
pool.close()
while True:
if results:
break
step += 1
sleep(1)
if step % 60 == 0:
print "status update" + ...
I've used .map() instead of .map_async() as the former is synchronous one. Also you probably will need to replace results.extend with something more efficient. And finally, due to GIL, speed improvement may be much smaller than expected.
BTW, it is little bit funny that you wrote that Python is synchronous in a question that asks about ThreadPool ;).

Consider using the time module. The time.time() function returns the current UNIX time.
For example, calling time.time() right now returns 1410384038.967499. One second later, it will return 1410384039.967499.
The way I would do this would be to use a while loop in the place of results = pool(...), and on every iteration to run a check like this:
last_time = time.time()
while (...):
new_time = time.time()
if new_time > last_time+60:
print "status update" + ...
last_time = new_time
(your computation here)
So that will check if (at least) a minute has elapsed since your last status update. It should print a status update approximately every sixty seconds.
Sorry that this is an incomplete answer, but I hope this helps or gives you some useful ideas.

python add time to a countdown already running

I want to have an app where if I click a button I add X amount of time to my running countdown timer.
I'm guessing I have to use threads for this but am not sure how to implement it..
Here is the code I have so far:
def countdown_controller(add_time):
end_it = False
def timer(time_this):
start = time.time()
lastprinted = 0
finish = start + time_this
while time.time() < finish:
now = int(time.time())
if now != lastprinted:
time_left = int(finish - now)
print time_left
lastprinted = now
if end_it == True:
now = finish
time.sleep(0.1)
# Check if the counter is running otherwise just add time.
try:
time_left
except NameError:
timer(add_time)
else:
if time_left == 0:
timer(add_time)
else:
add_this = time_left
end_it = True
while now != finish:
time.sleep(0.1)
timer(add_time + add_this)
Obviously this will not work, because every time I call countdown_controller(15) fx, it will start counting down for 15 seconds and if I click my button nothing happens until the timer is ended.
Help would be greatly appreciated.

I would say that there is a flaw in the design of the code, because your screen output blocks down the entire program doing nothing (time.sleep(0.1)).
Typically what you want to to do in these cases is having a main loop in your program that cycles through the various operations that make your program run. This guarantees a sensible distribution of system resources between the various tasks.
In your specific case, what you would like to have in your main loop is:
Check user input (has extra time been added?)
Update output of the countdown
Example implementation:
import time
import curses
# The timer class
class Timer():
def __init__(self):
self.target = time.time() + 5
def add_five(self):
self.target += 5
def get_left(self):
return int(self.target-time.time())
# The main program
t = Timer()
stdscr = curses.initscr()
stdscr.nodelay(True)
curses.noecho()
# This is the main loop done in curses, but you can implement it with
# a GUI toolkit or any other method you wish.
while True:
left = t.get_left()
if left <= 0:
break
stdscr.addstr(0, 0, 'Seconds left: %s ' % str(left).zfill(3))
c = stdscr.getch()
if c == ord('x') :
t.add_five()
# Final operations start here
stdscr.keypad(0)
curses.echo()
curses.endwin()
print '\nTime is up!\n'
The above program will increase the counter of 5 seconds if you press the x key (lowercase). Most of the code is boilerplate to use the curses module, but of course if you use PyGTK, PySide or any other graphical toolkit, it will be different.
EDIT: As a rule of thumb, in python you want to avoid threading as much as you can, both because it often (but not always) slows down programs (see "Global Interpreter Lock") and because it makes software harder to debug/maintain.
HTH!

I would probably have a Timer object with a finish attribute that I could simply add an int to. Have that timer running in another thread that you can then query for the current time remaining from your GUI.
class Timer(object):
def __init__(self, length):
self.finish = time.time() + length
def get_time(self):
return time.time() >= self.finish