Develop Reference

How can I get python to print out the current elapsed time while a function is running?

So I currently have python printing how long it took for a function to run after its done running with something like:
import time
t = time.time()
# do something in here
print "\n Time Taken: %.3f sec" % (time.time()-t)
but I want to show the live time that has passed since the function has started, and I cant quite figure out a way to get that to happen.
for example in a terminal I want it to say something like:
Working on xFunction. Time Elapsed 72.485 sec... (live updated time)
xFunction Has finished.
Time Taken: 1152.546 sec
Any help would be appreciated.

Here's an example with a thread that will print how much time has elapsed since it started and can be stopped from the main loop.
import time
import threading
class ElapsedTimeThread(threading.Thread):
""""Stoppable thread that prints the time elapsed"""
def __init__(self):
super(ElapsedTimeThread, self).__init__()
self._stop_event = threading.Event()
def stop(self):
self._stop_event.set()
def stopped(self):
return self._stop_event.is_set()
def run(self):
thread_start = time.time()
while not self.stopped():
print("\rElapsed Time {:.3f} seconds".format(time.time()-thread_start), end="")
#include a delay here so the thread doesn't uselessly thrash the CPU
time.sleep(0.01)
if __name__ == "__main__":
start = time.time()
thread = ElapsedTimeThread()
thread.start()
# do something
time.sleep(5)
# something is finished so stop the thread
thread.stop()
thread.join()
print() # empty print() to output a newline
print("Finished in {:.3f} seconds".format(time.time()-start))
This gives the following output, with the Elapsed Time counting up from zero and being overwritten:
J:\>python thr_time.py
Elapsed Time 5.000 seconds
Finished in 5.001 seconds
Note that this code is in Python 3. More info on stopping threads here & here.
Let me know if you'd like clarification on any portions.

I've modified #import_random 's code to enable the ability to probe elapsed time at any time during the execution of code, by wrapping 2 functions for initialization and finalization of ETC:
import time
import threading
class ElapsedTimeThread(threading.Thread):
""""Stoppable thread that prints the time elapsed"""
def __init__(self):
super(ElapsedTimeThread, self).__init__()
self._stop_event = threading.Event()
self.thread_start = time.time()
def stop(self):
self._stop_event.set()
def stopped(self):
return self._stop_event.is_set()
def getStart(self):
return self.thread_start
def getCurrentTime(self):
print("\rElapsed Time {:.3f} s. ".format(time.time()-self.thread_start), end="", flush=True )
def run(self):
self.thread_start = time.time()
while not self.stopped():
print("\rElapsed Time {:.3f} s. ".format(time.time()-self.thread_start), end="", flush=True)
#include a delay here so the thread doesn't uselessly thrash the CPU
time.sleep(0.01)
def startTimeCounter():
threadTimer = ElapsedTimeThread()
threadTimer.start()
return threadTimer
def stopTimeCounter(threadTimeCounter):
print() # empty print() to output a newline
print("Finished in {:.3f} s. ".format(time.time()-threadTimeCounter.getStart()))
threadTimeCounter.stop()
threadTimeCounter.join()

Python, threads starting in random order when using a thread lock

really new to Python here and trying to get my head around threading. I have the code:
import threading, time
class myThread(threading.Thread):
def __init__(self, threadID, name, counter):
super(myThread, self).__init__()
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
print "Starting ", self.name
threadLock.acquire()
print_time(self.name , self.counter, 3)
#Free the lock to release the next thread
print "%s released, ready for the next thread"%self.name
threadLock.release()
class sillyThread(threading.Thread):
def run(self):
threadLock.acquire()
print "silly silly!"
time.sleep(2)
print "Silly silly!"
threadLock.release()
def print_time(threadName, delay, counter):
while counter:
time.sleep(delay)
print "%s: %s"%(threadName, time.ctime())
counter -= 1
threadLock = threading.Lock()
threads = []
# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)
thread3 = sillyThread()
thread1.start()
thread2.start()
thread3.start()
threads.append(thread1)
threads.append(thread2)
threads.append(thread3)
# Wait for all threads to complete
for t in threads:
t.join()
print "%s is finished"%t.name
print "Exiting Main Thread"
I would expect Thread 1 to first start and Thread 2 and Thread 3 be put on block until Thread 1 finishes. So the order of execution would be Thread 1 , Thread 2 and Thread 3. However it varies and is different each time I run the code. For example sometimes Thread-3 will run first which doesn't make sense as I call the line thread1.start() first and it should be locked after. Could someone give me some pointers to why this might be happening please?

Thread.start simply schedules a thread to start. It doesn't actually start it then and there. Rather, from that point on, the OS will take over and start the thread as it sees fit asynchronous to your main thread.
If you truly want to synchronize running of the threads, you'll need to do so yourself using a mutex or other synchronization primitive.

Multi threading in python using parallel threads

I created two threads each running different functions.
What i tryed to achieve is if first thread ends then the second should also end ( i tryed achieving it using global variable)
Once both the threads end the same procedure should continue.
The script is not working as expected.
I am using Linux - Centos and python 2.7
#!/usr/bin/python
import threading
import time
import subprocess
import datetime
import os
import thread
command= "strace -o /root/Desktop/a.txt -c ./server"
final_dir = "/root/Desktop/"
exitflag = 0
# Define a function for the thread
def print_time(*args):
os.chdir(final_dir)
print "IN first thread"
proc = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
proc.wait(70)
exitflag=1
def print_time1(*args):
print "In second thread"
global exitflag
while exitflag:
thread.exit()
#proc = subprocess.Popen(command1,shell=True,stdout=subprocess.PIPE, sterr=subprocess.PIPE)
# Create two threads as follows
while (1):
t1=threading.Thread(target=print_time)
t1.start()
t2=threading.Thread(target=print_time1)
t2=start()
time.sleep(80)
z = t1.isAlive()
z1 = t2.isAlive()
if z:
z.exit()
if z1:
z1.exit()
threading.Thread(target=print_time1).start()
threading.Thread(target=print_time1).start()
print "In try"
Where am i going wrong?

You could create an object to share state, and have the dependent thread check that state. Something like:
import threading
import time
import datetime
class Worker1( threading.Thread ):
def __init__(self, state):
super(Worker1, self).__init__()
self.state = state
def run(self):
print_time_helper("Worker1 Start")
time.sleep(4)
print_time_helper("Worker1 End")
self.state.keepOnRunning = False
class Worker2( threading.Thread ):
def __init__(self, state):
super(Worker2, self).__init__()
self.state = state
def run(self):
while self.state.keepOnRunning:
print_time_helper("Worker2")
time.sleep(1)
class State( object ):
def __init__(self):
self.keepOnRunning = True
def main():
state = State()
thread1 = Worker1(state)
thread2 = Worker2(state)
thread1.start()
thread2.start()
thread1.join()
thread2.join()
def print_time_helper(name):
print "{0}: {1}".format(name, datetime.datetime.now().time().strftime("%S"))
which will output something like this (numbers show current time seconds):
Worker1 Start: 39
Worker2: 39
Worker2: 40
Worker2: 41
Worker2: 42
Worker1 End: 43
However, this is a bit simplistic for most situations. You might be better off using message queues - this is a good intro.

Use a threading.Event instead of an int and wait for it to be set.
Also your logic appears to be wrong in print_time1 because your while loop will never run since exitflag is initially 0, but even if it was 1 it would still just exit immediately. It's not actually waiting on anything.

End python code after 60 seconds

Below there is some fully functioning code.
I am planning to execute this code through command line, however I would like it to end after 60 seconds.
Does anyone know the best way of going about this?
Thanks in advance.
import time
class listener(StreamListener):
def on_data(self, data):
try:
print data
saveFile = open('twitDB.csv','a')
saveFile.write(data)
saveFile.write('\n')
saveFile.close()
return True
except BaseException, e:
print 'failed ondata,' ,str(e)
time.sleep(5)
def on_error(self, status):
print status

Try this out:
import os
import time
from datetime import datetime
from threading import Timer
def exitfunc():
print "Exit Time", datetime.now()
os._exit(0)
Timer(5, exitfunc).start() # exit in 5 seconds
while True: # infinite loop, replace it with your code that you want to interrupt
print "Current Time", datetime.now()
time.sleep(1)
There are some more examples in this StackOverflow question: Executing periodic actions in Python
I think the use of os._exit(0) is discouraged, but I'm not sure. Something about this doesn't feel kosher. It works, though.

You could move your code into a daemon thread and exit the main thread after 60 seconds:
#!/usr/bin/env python
import time
import threading
def listen():
print("put your code here")
t = threading.Thread(target=listen)
t.daemon = True
t.start()
time.sleep(60)
# main thread exits here. Daemon threads do not survive.

Use signal.ALARM to get notified after a specified time.
import signal, os
def handler(signum, frame):
print '60 seconds passed, exiting'
cleanup_and_exit_your_code()
# Set the signal handler and a 60-second alarm
signal.signal(signal.SIGALRM, handler)
signal.alarm(60)
run_your_code()
From your example it is not obvious what the code will exactly do, how it will run and what kind of loop it will iterate. But you can easily implement the ALARM signal to get notified after the timeout has expired.

This is my favorite way of doing timeout.
def timeout(func, args=None, kwargs=None, TIMEOUT=10, default=None, err=.05):
if args is None:
args = []
elif hasattr(args, "__iter__") and not isinstance(args, basestring):
args = args
else:
args = [args]
kwargs = {} if kwargs is None else kwargs
import threading
class InterruptableThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.result = None
def run(self):
try:
self.result = func(*args, **kwargs)
except:
self.result = default
it = InterruptableThread()
it.start()
it.join(TIMEOUT* (1 + err))
if it.isAlive():
return default
else:
return it.result

I hope this is an easy way to execute a function periodically and end after 60 seconds:
import time
import os
i = 0
def executeSomething():
global i
print(i)
i += 1
time.sleep(1)
if i == 10:
print('End')
os._exit(0)
while True:
executeSomething()

python: how to send packets in multi thread and then the thread kill itself

I have a question. I'd like to send a continuous streams of byte to some host for certain amount of time (let's say 1 minute) using python.
Here is my code so far:
#! /usr/bin/env python
import socket
import thread
import time
IP = "192.168.0.2"
PADDING = "a" * 1000 #assume the MTU is slighly above 1000
DATA = PADDING + "this is sentence number = "
PORT = 14444
killed = False
test_time = 60 #60 seconds of testing
def send_data():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((IP, PORT))
count = 1
starttime = time.clock()
while elapsed < test_time:
sent = s.send(DATA + str(count) + "\n")
if sent == 0: break # assume that if nothing is sent -> connection died
count = count+1
elapsed = time.clock() - starttime
if killed:
break
s.close()
print str(count) + " has been sent"
print "to quit type quit"
thread.start_new_thread(send_data, ())
while True:
var = raw_input("Enter something: ")
if var == "quit":
killed = True
Few question, is there a better way to let a thread die after 60 seconds other than polling the time.clock every time?
When I run this program, it sends the bytes correctly but when I typed quit the other thread won't die, even though I set the var killed = True. I wonder why is that? the scope of var Killed should reach the other thread right?
Thanks

I recommned using threading module. Even more benefit is to use InterruptableThread for terminating the thread. You do not have to use flag for terminating your thread but exception will occur if you call terminate() on this thread from parent. You can handle exception or not.
import threading, ctypes
class InterruptableThread(threading.Thread):
#classmethod
def _async_raise(cls, tid, excobj):
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(excobj))
if res == 0:
raise ValueError("nonexistent thread id")
elif res > 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, 0)
raise SystemError("PyThreadState_SetAsyncExc failed")
def raise_exc(self, excobj):
assert self.isAlive(), "thread must be started"
for tid, tobj in threading._active.items():
if tobj is self:
self._async_raise(tid, excobj)
return
def terminate(self):
self.raise_exc(SystemExit)
EDIT:
You can rewrite your code like this using another thread that is waiting 1 minute and then killing your other thread
def send_data:
IP = ...
# other vars
...
s = socket.socket(.....)
# no killed checking
# no time checking
# just do your work here
...
s.close()
my_thread = InterruptableThread(target=send_data)
my_thread.start()
def one_minute_kill(who):
time.sleep(60)
who.terminate()
killer_thread = InterruptableThread(target=one_minute_kill, args=[my_thread])
killer.start()
print "to quit type quit"
while my_thread.isAlive():
if raw_input("Enter something: ") == "quit":
my_thread.terminate()

I don't know how to do this with the "thread" module, but I can do it with the "threading" module. I think this code accomplishes what you want.
For documentation on the threading module:
http://docs.python.org/library/threading.html
#!/usr/bin/python
import time
from threading import Thread
import threading
import sys
test_time = 10
killed = False
class SillyThread( threading.Thread ):
def run(self):
global killed
starttime = time.time()
counter = 0
while (time.time() - starttime) < test_time:
if killed:
break
counter = counter + 1
time.sleep(0.1)
print "I did %d loops" % counter
class ManageThread( threading.Thread ):
def run(self):
global killed
while True:
var = raw_input("Enter something: ")
if var == "quit":
killed = True
break
print "Got var [%s]" % var
silly = SillyThread()
silly.start()
ManageThread().start()
Thread.join(silly)
print "bye bye"
sys.exit(0)
Note that I use time.time() instead of time.clock(). time.clock() gives elapsed processor time on Unix (see http://docs.python.org/library/time.html). I think time.clock() should work everywhere. I set my test_time to 10 seconds because I don't have the patience for a minute.
Here's what happens if I let it run the full 10 seconds:
leif#peacock:~/tmp$ ./test.py
Enter something: I did 100 loops
bye bye
Here's what happens if I type 'quit':
leif#peacock:~/tmp$ ./test.py
Enter something: quit
Got var [quit]
I did 10 loops
bye bye
Hope this helps.

As mentioned above, use the threading module, it is much easier to use and provides several synchronization primitives. It also provides a Timer class that runs after a specified amount of time.
If you just want the program to exit, you can simply make the sending thread a daemon. You do this by calling setDaemon(True) before calling start() (2.6 might use a daemon attribute instead). Python won't exit so long as a non-daemon thread is running.

You can do this pretty easily without threads. For example, using Twisted, you just set up a timed call and a producer:
from twisted.internet.protocol import ClientFactory, Protocol
from twisted.internet import reactor
class Noisy(Protocol):
def __init__(self, delay, data):
self.delay = delay
self.data = data
def stop(self):
self.transport.unregisterProducer()
self.transport.loseConnection()
reactor.stop()
def resumeProducing(self):
self.transport.write(self.data)
def connectionMade(self):
self.transport.registerProducer(self, False)
reactor.callLater(self.delay, self.stop)
factory = ClientFactory()
factory.protocol = lambda: Noisy(60, "hello server")
reactor.connectTCP(host, port, factory)
reactor.run()
This has various advantages over the threaded approach. It doesn't rely on daemon threads, so you can actually clean up the network connection (eg, to send a close message if necessary) rather than relying on the platform to destroy it. It handles all the actual low level networking code for you (your original example is doing the wrong thing in the case of socket.send returning 0; this code will handle that case properly). You also don't have to rely on ctypes or the obscure CPython API for raising an exception in another thread (so it's portable to more versions of Python and can actually interrupt a blocked send immediately, unlike some of the other suggested approaches).

Ensure that the "quit" is working correctly and add a small print to test that the input is working.
if var == "quit":
print "Hey we got quit"

The variable elapsed is not initialized. Set it to zero above the while loop.

It's easy to test the scope of killed:
>>> import thread
>>> killed = False
>>> import time
>>> def test():
... while True:
... time.sleep(1)
... if killed:
... print 'Dead.'
... break
...
>>> thread.start_new_thread(test, ())
25479680
>>> time.sleep(3)
>>> killed = True
>>> Dead.