This question already has an answer here:
Python: How to NOT wait for a thread to finish to carry on? [duplicate]
(1 answer)
Closed 7 months ago.
I am using win10 and python 3.7.3 32-bit
I am trying to achieve the following: every second a data read-out is performed an printed, while a control loop waits for user input to control a device. see my code:
import device
import threading
from threading import Lock
def print_data(t_start):
while True:
data=getData() # generic example
print(f'data: {data} at time {time.time()-t_start}')
def control_func():
while True:
in_ = input(' Press ENTER to switch ON, Press x to quit')
if in_ == '':
device.on()
print('device ON')
elif in_ == 'x' or in_ == 'X':
device.off()
sys.exit()
else: continue
in_ = input(' Press ENTER to switch OFF, Press x to quit')
if in_ == '':
device.off()
print('device OFF')
elif in_ == 'x' or in_ == 'X':
device.off()
sys.exit()
else: continue
t_start = time.time()
device=device()
trd1 = threading.Thread(target=control_func())
trd2 = threading.Thread(target=print_data(t_start))
trd1.start() # starting the thread 1
trd2.start() # starting the thread 2
trd1.join()
trd2.join()
This only gives me the input statements from control_func() or the prints from print_data()
Same wih using mutliprocessing.
I didn't manage to make the two functions run simultanously.
replacing print() with
s_print_lock = Lock()
# Define a function to call print with the Lock
def s_print(*a, **b):
"""Thread safe print function"""
with s_print_lock:
print(*a, **b)
also didn't do the trick.
Since I am noob, please help. Or should I do a different approach all together?
You called the functions in the process of creating the Thread, you didn't pass the functions to the Thread for it to execute, so no actual work occurred in the threads. Change the Thread creation to:
trd1 = threading.Thread(target=control_func) # Remove call parens
trd2 = threading.Thread(target=print_data, args=(t_start,)) # Remove call parens and pass args tuple separately
so the functions themselves are passed, and the Thread actually runs them in the separate logical thread of execution. As written, you ran control_func to completion, then ran print_data to completion, then launched two Threads with target=None (the returned value from both functions) which do nothing, then joined the Threads doing nothing.
Additional notes:
If you're using multiprocessing, make sure to use multiprocessing.Lock, not threading.Lock (the latter is only guaranteed to work within a single process)
While the threaded cases likely doesn't require a lock (at least on CPython where the GIL protects against issues) if you're only printing a single atomic thing at a time, for multiprocessing you should definitely use the lock and add flush=True to all prints; without the flush=True, the actual output might be delayed indefinitely.
You need to provide some means to communicate that the loop is done to print_data; as written, control_func will sys.exit(), but that only exits that thread, not the program. Unless getData will somehow throw an exception as a result of control_func exiting, print_data will never exit, and therefore neither will the main thread.
Solutions for #3 include:
Using a threading.Event(); make a global should_stop = threading.Event(), change the print_data loop to while not should_stop.is_set():, and after trd1.join() returns, have the main thread call should_stop.set()
Make trd2 a daemon thread, and don't bother to join it (assumes it can die immediately when trd1 and the main thread have ended; might not hold for your scenario); it will die forcefully when all non-daemon threads have exited.
Related
This seems like a particularly confusing question based on the other similar answers I found on SO. I have code similar to the following:
def parentFunction():
# Other code
while True:
var1, var2 = anotherFunction1() # Getting client details after listening on open port
threading.Thread(target = anotherFunction2, args=(var1, var2)).start()
childFunction(var1,var2)
print("PRINT #1: Running in Parent Function") # This only prints once for some reason
def childFunction(var1, var2):
threading.Timer(10, childFunction, args=(var1,var2)).start()
print("PRINT #2: Running in child function") # Prints every 10 seconds
# Other code
if (someConditionIsMet):
print("PRINT #3: Exiting")
end_process_and_exit_here()
So basically, when I ran the parentFunction(), I would go into a neverending loop where ever 10 seconds, my console would print "PRINT #2: Running in child function". When the someConditionIsMet was true, my console would print "PRINT #3: Exiting" but then it wouldn't exit. Hence, my loop would carry on forever. I am not sure if it's relevant, but parts of the code has a Threading.Lock as well.
Where I have written end_process_and_exit_here() above, I tried using several methods to kill a thread such as
Raising exceptions and setting flags - These assume that I have started my thread outside of my loop so it's not comparable.
Even this qn about looping threads assumes the thread isnt being looped
Killing using join or stop - stop() was not an option I could access. join() was available but it didn't work i.e. after it was called, the next thread (PRINT #2) continued printing.
Other answers suggesting the use of signals (1) (2), also didn't work.
Using sys.exit() or break in different parts of my code also did not result in the threads stopping.
Is there any method for me to easily exit from such a looping thread?
Note: I need to use threading and not multiprocessing.
You could use python-worker, simply add #worker above you function
pip install python-worker
from worker import worker
#worker
def anotherFunction2(var1,var2):
# your code here
pass
#worker
def parentFunction():
# Other code
while True:
var1, var2 = anotherFunction1() # Getting client details after listening on open port
function2Worker = anotherFunction2(var1,var2) # this will automatically run as thread since you put #worker above your function
childFunction(var1,var2)
print("PRINT #1: Running in Parent Function") # This only prints once for some reason
def childFunction(var1, var2):
parentWorker = parentFunction(var1, var2)
# Other code
if (someConditionIsMet):
parentWorker.abort()
So as an update, I have managed to resolve this issue. The problem with the other answer stated by me (shown below) is that just .cancel() by itself only seemed to work for one timer thread. But as can be seen in the problem, childFunction() itself calls childFunction() and can also be called by the parentFunction, meaning that there may be multiple timer threads.
What worked for my specific case was naming my threads as below:
t1 = threading.Timer(10, childFunction, args=(var1,var2,number))
t1.name = t1.name + "_timer" + str(number)
t1.start()
Thereafter, I could cancel all timer threads that were created from this process by:
for timerthread in threading.enumerate():
if timerthread.name.endswith('timer' + str(number)):
timerthread.cancel()
Below is the ORIGINAL METHOD I USED WHICH CAUSED MANY ISSUES:
I'm not certain if this is a bad practice (in fact I feel it may be based on the answers linked in the question saying that we should never 'kill a thread'). I'm sure there are reasons why this is not good and I'd appreciate anyone telling me why. However, the solution that ultimately worked for me was to use .cancel().
So first change would be to assign your thread Timer to a variable instead of calling it directly. So instead of threading.Timer(10, childFunction, args=(var1,var2)).start(), it should be
t = threading.Timer(10, childFunction, args=(var1,var2))
t.start()
Following that, instead of end_process_and_exit_here(), you should use t.cancel(). This seems to work and stops all threads mid-process. However, the bad thing is that it doesn't seem to carry on with other parts of the program.
I have created a function of squares which will start squaring from 1,2... when i will type "start" and end up squaring when i will type "stop".
When i typed "start",i started getting squares of numbers after every 2 sec but when i typed "stop",the thread didnt stop.I used the variable 'flag' to stop the function.
Here is the code.
import threading
import time
flag=False
def square():
i=1;
global flag
while(True):
print(i*i,"\n")
time.sleep(2)
i=i+1
if(flag):
break
def main():
while(True):
x=input("Enter start/stop")
if(x=="start"):
flag=False
p = threading.Thread(target=square)
p.start()
if(x=="stop"):
flag=True
p.join()
main()
The problem was that the flag variable defined and used in main was local, and had nothing to do with the one used by the thread, so the thread was never notified by its change (and therefore newer knew when it was time to stop). The fix is simple, make the variable global in main (same way as already done in square):
global flag
square function could be also simplified, instead of:
while True:
# The other instructions
if(flag):
break
you could do:
while not flag:
# The other instructions
As a note, there are other ways of synchronizing between threads, e.g.:
threading.Event
make your thread a daemon (p = threading.Thread(target=square, daemon=True), even if generally not recommended), meaning that it will abruptly stop when main will reach its end
For more info, check [Python 3.docs]: threading - Thread-based parallelism.
I have a function I'm calling every 5 seconds like such:
def check_buzz(super_buzz_words):
print 'Checking buzz'
t = Timer(5.0, check_buzz, args=(super_buzz_words,))
t.dameon = True
t.start()
buzz_word = get_buzz_word()
if buzz_word is not 'fail':
super_buzz_words.put(buzz_word)
main()
check_buzz()
I'm exiting the script by either catching a KeyboardInterrupt or by catching a System exit and calling this:
sys.exit('\nShutting Down\n')
I'm also restarting the program every so often by calling:
execv(sys.executable, [sys.executable] + sys.argv)
My question is, how do I get that timer thread to shut off? If I keyboard interrupt, the timer keeps going.
I think you just spelled daemon wrong, it should have been:
t.daemon = True
Then sys.exit() should work
Expanding on the answer from notorious.no, and the comment asking:
How can I call t.cancel() if I have no access to t oustide the
function?
Give the Timer thread a distinct name when you first create it:
import threading
def check_buzz(super_buzz_words):
print 'Checking buzz'
t = Timer(5.0, check_buzz, args=(super_buzz_words,))
t.daemon = True
t.name = "check_buzz_daemon"
t.start()
Although the local variable t soon goes out of scope, the Timer thread that t pointed to still exists and still retains the name assigned to it.
Your atexit-registered method can then identify this thread by its name and cancel it:
from atexit import register
def all_done():
for thr in threading._enumerate():
if thr.name == "check_buzz_daemon":
if thr.is_alive():
thr.cancel()
thr.join()
register(all_done)
Calling join() after calling cancel()is based on a StackOverflow answer by Cédric Julien.
HOWEVER, your thread is set to be a Daemon. According to this StackOverflow post, daemon threads do not need to be explicitly terminated.
from atexit import register
def all_done():
if t.is_alive():
# do something that will close your thread gracefully
register(all_done)
Basically when your code is about to exit, it will fire one last function and this is where you will check if your thread is still running. If it is, do something that will either cancel the transaction or otherwise exit gracefully. In general, it's best to let threads finish by themselves, but if it's not doing anything important (please note the emphasis) than you can just do t.cancel(). Design your code so that threads will finish on their own if possible.
Another way would be to use the Queue() module to send and recieve info from a thread using the .put() outside the thread and the .get() inside the thread.
What you can also do is create a txt file and make program write to it when you exit And put an if statement in the thread function to check it after each iteration (this is not a really good solution but it also works)
I would have put a code exemple but i am writing from mobile sorry
It seems that asynchronous signals in multithreaded programs are not correctly handled by Python. But, I thought I would check here to see if anyone can spot a place where I am violating some principle, or misunderstanding some concept.
There are similar threads I've found here on SO, but none that seem to be quite the same.
The scenario is: I have two threads, reader thread and writer thread (main thread). The writer thread writes to a pipe that the reader thread polls. The two threads are coordinated using a threading.Event() primitive (which I assume is implemented using pthread_cond_wait). The main thread waits on the Event while the reader thread eventually sets it.
But, if I want to interrupt my program while the main thread is waiting on the Event, the KeyboardInterrupt is not handled asynchronously.
Here is a small program to illustrate my point:
#!/usr/bin/python
import os
import sys
import select
import time
import threading
pfd_r = -1
pfd_w = -1
reader_ready = threading.Event()
class Reader(threading.Thread):
"""Read data from pipe and echo to stdout."""
def run(self):
global pfd_r
while True:
if select.select([pfd_r], [], [], 1)[0] == [pfd_r]:
output = os.read(pfd_r, 1000)
sys.stdout.write("R> '%s'\n" % output)
sys.stdout.flush()
# Suppose there is some long-running processing happening:
time.sleep(10)
reader_ready.set()
# Set up pipe.
(pfd_r, pfd_w) = os.pipe()
rt = Reader()
rt.daemon = True
rt.start()
while True:
reader_ready.clear()
user_input = raw_input("> ").strip()
written = os.write(pfd_w, user_input)
assert written == len(user_input)
# Wait for reply -- Try to ^C here and it won't work immediately.
reader_ready.wait()
Start the program with './bug.py' and enter some input at the prompt. Once you see the reader reply with the prefix 'R>', try to interrupt using ^C.
What I see (Ubuntu Linux 10.10, Python 2.6.6) is that the ^C is not handled until after the blocking reader_ready.wait() returns. What I expected to see is that the ^C is raised asynchronously, resulting in the program terminating (because I do not catch KeyboardInterrupt).
This may seem like a contrived example, but I'm running into this in a real-world program where the time.sleep(10) is replaced by actual computation.
Am I doing something obviously wrong, like misunderstanding what the expected result would be?
Edit: I've also just tested with Python 3.1.1 and the same problem exists.
The wait() method of a threading._Event object actually relies on a thread.lock's acquire() method. However, the thread documentation states that a lock's acquire() method cannot be interrupted, and that any KeyboardInterrupt exception will be handled after the lock is released.
So basically, this is working as intended. Threading objects that implement this behavior rely on a lock at some point (including queues), so you might want to choose another path.
Alternatively, you could also use the pause() function of the signal module instead of reader_ready.wait(). signal.pause() is a blocking function and gets unblocked when a signal is received by the process. In your case, when ^C is pressed, SIGINT signal unblocks the function.
According to the documentation, the function is not available for Windows. I've tested it on Linux and it works. I think this is better than using wait() with a timeout.
I have two functions, draw_ascii_spinner and findCluster(companyid).
I would like to:
Run findCluster(companyid) in the backround and while its processing....
Run draw_ascii_spinner until findCluster(companyid) finishes
How do I begin to try to solve for this (Python 2.7)?
Use threads:
import threading, time
def wrapper(func, args, res):
res.append(func(*args))
res = []
t = threading.Thread(target=wrapper, args=(findcluster, (companyid,), res))
t.start()
while t.is_alive():
# print next iteration of ASCII spinner
t.join(0.2)
print res[0]
You can use multiprocessing. Or, if findCluster(companyid) has sensible stopping points, you can turn it into a generator along with draw_ascii_spinner, to do something like this:
for tick in findCluster(companyid):
ascii_spinner.next()
Generally, you will use Threads. Here is a simplistic approach which assumes, that there are only two threads: 1) the main thread executing a task, 2) the spinner thread:
#!/usr/bin/env python
import time
import thread
def spinner():
while True:
print '.'
time.sleep(1)
def task():
time.sleep(5)
if __name__ == '__main__':
thread.start_new_thread(spinner, ())
# as soon as task finishes (and so the program)
# spinner will be gone as well
task()
This can be done with threads. FindCluster runs in a separate thread and when done, it can simply signal another thread that is polling for a reply.
You'll want to do some research on threading, the general form is going to be this
Create a new thread for findCluster and create some way for the program to know the method is running - simplest in Python is just a global boolean
Run draw_ascii_spinner in a while loop conditioned on whether it is still running, you'll probably want to have this thread sleep for a short period of time between iterations
Here's a short tutorial in Python - http://linuxgazette.net/107/pai.html
Run findCluster() in a thread (the Threading module makes this very easy), and then draw_ascii_spinner until some condition is met.
Instead of using sleep() to set the pace of the spinner, you can wait on the thread's wait() with a timeout.
It is possible to have a working example? I am new in Python. I have 6 tasks to run in one python program. These 6 tasks should work in coordinations, meaning that one should start when another finishes. I saw the answers , but I couldn't adopted the codes you shared to my program.
I used "time.sleep" but I know that it is not good because I cannot know how much time it takes each time.
# Sending commands
for i in range(0,len(cmdList)): # port Sending commands
cmd = cmdList[i]
cmdFull = convert(cmd)
port.write(cmd.encode('ascii'))
# s = port.read(10)
print(cmd)
# Terminate the command + close serial port
port.write(cmdFull.encode('ascii'))
print('Termination')
port.close()
# time.sleep(1*60)