I am new to threads and multiprocessing. I have some code in which I start a process and I wanted the output to show an active waiting state of something like this
wating....
The code is similiar to this below:
import threading
import time
class ThreadExample(object):
def __init__(self):
self.pause = True
threading.Thread(target=self.second_thread).start()
# Some other processes
print("Waiting", end="")
while self.pause:
time.sleep(1)
print(".", end="")
print("Hooray!")
def second_thread(self):
print("timer started")
time.sleep(3)
self.pause = False
print("timer finished")
if __name__ == "__main__":
ThreadExample()
When I run the code above, I receive the output:
timer started
Do something else..timer finished
.
Hooray!
not a big surprise, except that only the 'timer started' appears at the beginning and the rest of the text appears in an instant at the end.
If I change the line print(".", end="") to print("."), I receive the following output:
timer started
Do something else.
.
timer finished
.
Hooray
where the dots appear in 1 second increments, which was my intention.
Is there a way to get the 'Waiting...' on one line without the end=""?
And secondly I am guessing this is something to do with the internals of the print() function, and if not, should I perform the threading in another manner? I do not think the problem is the GIL as I have tried multiprocess.Process and got the same result.
This is probably due to print buffering. It is flushed on \n and on some other occasions (like buffer overflow or program exit). Instead of print try this:
import sys
def unbuffered_print(msg):
sys.stdout.write(msg)
sys.stdout.flush()
...
unbuffered_print('.')
everywhere.
Related
I have a command in a line (Fit.perform() from import xspec, but never mind because the question is general and can be applicated also for other python commands) that takes a while to finish.
I simple want to know the time of execution while the command is running, so when it has not finished its execution yet.
This is necessary if I want to stop the command during its execution, for example because it is taking too much time to end.
So, I need something like this:
if **you_are_taking_so_much_time**:
do_something_else
It is not possible to use methods like time or timeit because they calculate the time only at the end of execution of a command and not while it is running.
Is it possible?
I'm using python 2.7 on MacOS.
You will have to use a monitor thread:
import threading
import time
done = False
def longfun():
global done
print("This will take some time.")
time.sleep(60)
done = True
def monitor():
global done
timeout = 10
print("Wait until timeout.")
while not done and timeout > 0:
time.sleep(1)
timeout -= 1
lt = threading.Thread(target=longfun)
lt.start()
mt = threading.Thread(target=monitor)
mt.start()
mt.join()
if done == False:
print("Long thread not done yet. Do something else.")
lt.join()
Note that this does wait until the 'long' thread is finished. You do not mention you want to stop the long running operation. If you do, you will have to correctly implement it in a thread, including start/stop/progress functionality (usually this works with a while loop that uses a running bit to see if it should continue.
like this:
import time,thread
def test_me(hargs):
func,args,timeout = hargs
start_time = time.time()
thread.start_newthread(func,args)
while True :
if My_expected_value:#where store this ?
print "well done !"
break
elif time.time() > (timeout + start_time) :
print "oh! to late, sorry !"
break
time.sleep(timeout/100)
thread.start_newthread(test_me,((func,args,timeout),))
important warnings : need use thread for Non-freezing application, got 3 thread for this: 1-main app, 2-test_me, 3- Your function(func)
Don't forget adding external variable to your function (for killing your function thread)
I have not programmed in over a year now so sorry if this is a stupid question.
I looked up lots of examples on this site for threading but I seem to be getting a different outcome to other people.
From my understanding of threading, using this simple code, it should be printing YO and LO together, something like
YO
LO
YO
LO
but instead I just get
YO
YO
YO
...
from threading import Thread
import time
def printYo():
while(3>1):
print ("YO")
time.sleep(1)
def printLo():
while(3>1):
print ("LO")
time.sleep(1)
t2 = Thread(target=printLo())
t = Thread(target=printYo())
t2.start()
t.start()
You are calling the function instead of just giving it as target for your thread.
t2 = Thread(target=printLo)
t = Thread(target=printYo)
There are two problems, first of all you should pass the thread function to the Thread constructor (ie Thread(target=printLo)). When the thread is started it will call that function in a separate thread.
Second you may (probably) want to keep the main thread running which could be done by having an idle loop. If you don't you will not be able to stop it with ^C, but you still has to handle the termination of the process.
The complete code will consequently be:
from threading import Thread
import time
def printYo():
while(3>1):
print ("YO")
time.sleep(1)
def printLo():
while(3>1):
print ("LO")
time.sleep(1)
t2 = Thread(target=printLo)
t = Thread(target=printYo)
t2.start()
t.start()
try:
while True:
time.sleep(1)
except:
os._exit(0)
A minor note is that the prints will print the output on separate lines, an important note is that there is no guarantee which order the YOs and LOs will be printed - at first they are likely to alternate, but eventually some of the threads may get to print twice before the other gets to print.
I have a piece of code below that creates a few threads to perform a task, which works perfectly well on its own. However I'm struggling to understand why the print statements I call in my function do not execute until all threads complete and the print 'finished' statement is called. I would expect them to be called as the thread executes. Is there any simple way to accomplish this, and why does this work this way in the first place?
def func(param):
time.sleep(.25)
print param*2
if __name__ == '__main__':
print 'starting execution'
launchTime = time.clock()
params = range(10)
pool=multiprocessing.Pool(processes=100) #use N processes to download the data
_=pool.map(func,params)
print 'finished'
For python 3 you can now use the flush param like that:
print('Your text', flush=True)
This happens due to stdout buffering. You still can flush the buffers:
import sys
print 'starting'
sys.stdout.flush()
You can find more info on this issue here and here.
Having run into plenty of issues around this and garbled outputs (especially under Windows when adding colours to the output..), my solution has been to have an exclusive printing thread which consumes a queue
If this still doesn't work, also add flush=True to your print statement(s) as suggested by #Or Duan
Further, you may find the "most correct", but a heavy-handed approach to displaying messages with threading is to use the logging library which can wrap a queue (and write to many places asynchronously, including stdout) or write to a system-level queue (outside Python; availability depends greatly on OS support)
import threading
from queue import Queue
def display_worker(display_queue):
while True:
line = display_queue.get()
if line is None: # simple termination logic, other sentinels can be used
break
print(line, flush=True) # remove flush if slow or using Python2
def some_other_worker(display_queue, other_args):
# NOTE accepts queue reference as an argument, though it could be a global
display_queue.put("something which should be printed from this thread")
def main():
display_queue = Queue() # synchronizes console output
screen_printing_thread = threading.Thread(
target=display_worker,
args=(display_queue,),
)
screen_printing_thread.start()
### other logic ###
display_queue.put(None) # end screen_printing_thread
screen_printing_thread.stop()
I got simple program as below:
import threading
import time
import signal
WITH_DEADLOCK = 0
lock = threading.Lock()
def interruptHandler(signo, frame):
print str(frame), 'received', signo
lock.acquire()
try:
time.sleep(3)
finally:
if WITH_DEADLOCK:
print str(frame), 'release'
lock.release()
signal.signal(signal.SIGINT, interruptHandler)
for x in xrange(60):
print time.strftime("%H:%M:%S"), 'main thread is working'
time.sleep(1)
So, if you start that program and even Ctrl+C is pressed twice within 3 seconds, there is no deadlock. Each time you press Ctrl + C proper line is displayed.
If you change WITH_DEADLOCK=1 and you would press Ctrl+C twice (withing 3 seconds) then program will be hung.
Does anybody may explain why print operation make such difference?
(My python version is 2.6.5)
To be honest I think J.F. Sebastian's comment is the most appropriate answer here - you need to make your signal handler reentrant, which it currently isn't, and it is mostly just surprising that it works anyway without the print statement.
Alright I've been using the time module for time.sleep(x) function for awhile... but I need something that won't pause the shell and so the user can continue using the program while it's counting.
To be more "specific" let's suppose I had a program that needed to wait 5 seconds before executing a function. In this time using the time.sleep() function the user can't type anything into the shell because it's sleeping. However, I need Python to "count the 5 seconds" in the background while the user is able to use the shell. Is this possible?
threading ? You should handle piece of your work in one worker and another separate worker where you would count or sleep with time.sleep
Here is an example that might help you understand and use threading with time.sleep
import threading
import time
def sleeper():
print 'Starting to sleep'
time.sleep(10)
print 'Just waking up..'
print 'snooze'
print 'oh no. I have to get up.'
def worker():
print 'Starting to work'
time.sleep(1) # this also a work. :)
print 'Done with Work'
t = threading.Thread(name='sleeper', target=sleeper)
w = threading.Thread(name='worker', target=worker)
w.start()
t.start()