I have a thread, where I am using an thread event to control the thread from outside the target function.
flag = threading.event()
In my target function, I have something like this:
def functionname(arguments):
flag.clear()
while not flag.wait(timeout = 0.5):
# do something.
whenever I want to return the thread, from my main function I say, flag.set(). Then flag is set to true, my target function completes execution and the thread is completed.
Now, if I want to use flag.wait(timeout = 5) in my main function, I am expecting to block and wait for five seconds to execute the "do something" part of code. However I am seeing that the "do something" part of the code is executing every 0.5 seconds as usual and my main function is blocked for five seconds.
The wait method has to block until the flag is true or the optional time out ends. It is blocking my main function rather than the target function. Can any one know why this might be?
PS: I defined the flag event in my main function and passed it as an argument to the target function
Many threads can wait on an event at the same time and how long one waits before timeout is independent of how long the others do. If you want your thread's wait time to change from .5 to 5 seconds, then you need some way to tell the thread to change the value it passes in the timeout parameter. This seems like a good job for a shared variable - the thread reads timeout from the variable and main can change that variable.
But where do you put it? Well, class instances exist to hold associated data so a class holding the event and the current timeout value is reasonable. In fact, you can just use inheritance to do the job. Notice in this example, the timeout is only changed for the next interval - the thread waits the current wait time before using the updated value. Getting the change immediately is significantly more difficult and I didn't want to confuse the matter.
import threading
import time
class VariablyTimedEvent(threading.Event):
def __init__(self, initial_timeout=None):
super().__init__()
self.timeout = initial_timeout
def wait(self):
return super().wait(timeout=self.timeout)
# ------------- test -----------------
def functionname(event):
event.clear()
while not event.wait():
do_something()
print('event set, exiting')
_do_something_start = time.time()
def do_something():
# a mock object that shows time since program start
print('%03.1f: do something' % (time.time() - _do_something_start))
print("start timing at .5 second")
event = VariablyTimedEvent(.5)
thread = threading.Thread(target=functionname, args=(event,))
thread.start()
time.sleep(3.1)
print("change timeout to 2 seconds")
event.timeout = 2
time.sleep(7)
print("exit the thread")
event.set()
thread.join()
Related
I have a function in thread A which needs to wait until a function in thread B is called.
The function in thread B is called periodically, so it just needs to wait until the next time it is called. This allows me to sync up with it.
How would I do this?
(Sorry if this is trivial.)
It may be a principle of computer science that no multithreading question is trivial.
There are various ways to do this, but one of the simplest involves the use of a threading.Event object. Events are the simplest of the so-called synchronization primitives. See the manual section on the threading module for more ideas. Here is a working example:
#! python3.8
import threading
import time
t0 = time.time()
def elapsed_time():
return time.time() - t0
class StopMe:
def __init__(self):
self.running = True
def main():
ev1 = threading.Event()
stop = StopMe()
th1 = threading.Thread(target=thread1, args=(ev1, stop))
th1.start()
for _ in range(10):
ev1.wait()
print("The function was just called", elapsed_time())
ev1.clear()
stop.running = False
th1.join()
print("Exit", elapsed_time())
def thread1(event, stop):
def a_function():
event.set()
print("I am the function", elapsed_time())
while stop.running:
time.sleep(1.0)
a_function()
main()
Output:
I am the function 1.0116908550262451
The function was just called 1.0116908550262451
I am the function 2.0219264030456543
The function was just called 2.0219264030456543
I am the function 3.0322916507720947
The function was just called 3.0322916507720947
I am the function 4.033170938491821
The function was just called 4.033170938491821
I am the function 5.043376445770264
The function was just called 5.043376445770264
I am the function 6.043909788131714
The function was just called 6.043909788131714
I am the function 7.054021596908569
The function was just called 7.054021596908569
I am the function 8.06399941444397
The function was just called 8.06399941444397
I am the function 9.064924716949463
The function was just called 9.064924716949463
I am the function 10.066757678985596
The function was just called 10.066757678985596
I am the function 11.076870918273926
Exit 11.076870918273926
Some things to note here:
Once you put a synchronization primitive into your code, you need to give some thought about how to terminate the thread gracefully, and how to terminate the application as a whole. In this example, the threads communicate through the little "StopMe" object, and through the Event object. Note that the main thread may have to wait one second until the secondary thread finishes its sleep function. That occurs if thread1 begins its time delay before the main thread calls the join function. That didn't happen in my test run but it might, depending on how CPU time slices are given to the different threads. If that's not acceptable to you, you have to write more code to get around it.
Also note that the function call ev1.wait() will block the main thread until the event is set from the secondary thread. In a GUI application that is not what you want.
I ran this with Python3.8 but the program doesn't use any version-specific features, so it should work the same with any reasonably recent version of Python.
My code runs N number of threads. I want to stop specific threads on some condition but the remaining threads should continue running. I am doing some operation once each thread finishes its job. Is there a way to stop running thread in Python 3.
My current code is implemented in Python2 which does this by "_Thread__stop()". Is there any identical thing in Python3?
The practice is to "signal" the thread that it is time to finish and then the thread needs to exit. This is not killing like you kill a process but a regular state machine behavior of your thread function.
For example, suppose your thread is lopping. You should insert an if statement inside the loop that instructing the thread function to break or return if stop is True. The stop variable should be a shared variable with the main thread (or the thread who need to stop out thread) that will change it to True. usually after this, the stopper thread will want to wait for the thread completion by join()
It's a bad habit to kill a thread, better is to create a "flag" which will tell you when your thread made its work done.
Consider the following example:
import threading
import random
class CheckSomething(threading.Thread):
def __init__(self, variable):
super(CheckSomething, self).__init__()
self.start_flag = threading.Event()
self.variable = variable
def check_position(self, variable):
x = random.randint(100)
if variable == x:
self.stop_checking()
def run(self):
while True:
self.check_position(self.variable)
def stop_checking():
self.start_flag.set()
def stopped():
return self.start_flag.is_set()
The set() method of Event() set its status to True. More you can read in docs: https://docs.python.org/3.5/library/threading.html
So you need to call stop_checking() when you meet a condition where you want exit.
I'm having a hard hard time with Timer function from threading.
Basically, when my program starts, I want to log stats every x second.
So I thought I could do it with the Timer function (launch function every 5 second).
For now, I did :
from threading import Timer
def startLogger():
while True:
t = Timer(5, function)
t.start()
def function():
print("hey")
But it launch error, so I think it's not the good way to do it.
RuntimeError: can't start new thread
If someone can give me a clue, it would be appreciated!
Instead of starting a new thread every five seconds, you can create just one thread and run all your code from there.
from time import sleep
from threading import Thread
def startLogger():
while True:
function()
sleep(5)
def function():
print("hey")
Thread(target=startLogger).start()
startLogger will continually run. It'll call function, then pause for 5 seconds, then start again, calling function and so on.
It goes in its own thread so that the sleep(5) doesn't also stop your main thread for 5 seconds.
You can try the following. The idea is, that you are scheduling the next function call just at the end of this function's body:
import threading
def mylog():
print "hey"
` threading.Timer(5.0, mylog)`.start()
mylog()
I need to wake up a sleeping process ?
The time (t) for which it sleeps is calculated as t = D/S . Now since s is varying, can increase or decrease, I need to increase/decrease the sleeping time as well. The speed is received over a UDP procotol. So, how do I change the sleeping time of a process, keeping in mind the following:-
If as per the previous speed `S1`, the time to sleep is `(D/S1)` .
Now the speed is changed, it should now sleep for the new time,ie (D/S2).
Since, it has already slept for D/S1 time, now it should sleep for D/S2 - D/S1.
How would I do it?
As of right now, I'm just assuming that the speed will remain constant all throughout the program, hence not notifying the process. But how would I do that according to the above condition?
def process2():
p = multiprocessing.current_process()
time.sleep(secs1)
# send some packet1 via UDP
time.sleep(secs2)
# send some packet2 via UDP
time.sleep(secs3)
# send some packet3 via UDP
Also, as in threads,
1) threading.activeCount(): Returns the number of thread objects that are active.
2) threading.currentThread(): Returns the number of thread objects in the caller's thread control.
3) threading.enumerate(): Returns a list of all thread objects that are currently active.
What are the similar functions for getting activecount, enumerate in multiprocessing?
Not yet tested but, i think this could work :
Instead of using sleep, create a condition object and use it's wait() method.
Create a Timer object, which call the notify() method of the condition object when timed out.
If you want to change the sleep time, just discard the old Timer (with cancel() method), and create a new Timer.
* UPDATE *
I just tested this and it works.
This is the wait() in the process, don't forge to acquire it first.
def process(condition):
condition.acquire()
condition.wait()
condition.release()
and this is wake_up function, called from main process :
def wake_up(condition):
condition.acquire()
condition.notify()
condition.release()
and create and pass a condition object when creating a process (in your main, or other functions) :
condition=multiprocessing.Condition(multiprocessing.Lock())
p=multiprocessing.Process(target=process, args=(condition,))
p.start()
create a Timer (this timer thread will be created on main process) :
timer=threading.Timer(wake_up_time, wake_up, args(condition,))
start_time=time.time()
timer.start()
if you want to change the time, just stop it and make a new Timer :
timer.cancel()
elapsed_time=time.time-start_time
timer=threading.Timer(new_wake_up_time-elapsed_time, wake_up, args(condition,))
timer.start()
I want to execute a function every 60 seconds on Python but I don't want to be blocked meanwhile.
How can I do it asynchronously?
import threading
import time
def f():
print("hello world")
threading.Timer(3, f).start()
if __name__ == '__main__':
f()
time.sleep(20)
With this code, the function f is executed every 3 seconds within the 20 seconds time.time.
At the end it gives an error and I think that it is because the threading.timer has not been canceled.
How can I cancel it?
You could try the threading.Timer class: http://docs.python.org/library/threading.html#timer-objects.
import threading
def f(f_stop):
# do something here ...
if not f_stop.is_set():
# call f() again in 60 seconds
threading.Timer(60, f, [f_stop]).start()
f_stop = threading.Event()
# start calling f now and every 60 sec thereafter
f(f_stop)
# stop the thread when needed
#f_stop.set()
The simplest way is to create a background thread that runs something every 60 seconds. A trivial implementation is:
import time
from threading import Thread
class BackgroundTimer(Thread):
def run(self):
while 1:
time.sleep(60)
# do something
# ... SNIP ...
# Inside your main thread
# ... SNIP ...
timer = BackgroundTimer()
timer.start()
Obviously, if the "do something" takes a long time, then you'll need to accommodate for it in your sleep statement. But, 60 seconds serves as a good approximation.
I googled around and found the Python circuits Framework, which makes it possible to wait
for a particular event.
The .callEvent(self, event, *channels) method of circuits contains a fire and suspend-until-response functionality, the documentation says:
Fire the given event to the specified channels and suspend execution
until it has been dispatched. This method may only be invoked as
argument to a yield on the top execution level of a handler (e.g.
"yield self.callEvent(event)"). It effectively creates and returns
a generator that will be invoked by the main loop until the event has
been dispatched (see :func:circuits.core.handlers.handler).
I hope you find it as useful as I do :)
./regards
It depends on what you actually want to do in the mean time. Threads are the most general and least preferred way of doing it; you should be aware of the issues with threading when you use it: not all (non-Python) code allows access from multiple threads simultaneously, communication between threads should be done using thread-safe datastructures like Queue.Queue, you won't be able to interrupt the thread from outside it, and terminating the program while the thread is still running can lead to a hung interpreter or spurious tracebacks.
Often there's an easier way. If you're doing this in a GUI program, use the GUI library's timer or event functionality. All GUIs have this. Likewise, if you're using another event system, like Twisted or another server-process model, you should be able to hook into the main event loop to cause it to call your function regularly. The non-threading approaches do cause your program to be blocked while the function is pending, but not between functioncalls.
Why dont you create a dedicated thread, in which you put a simple sleeping loop:
#!/usr/bin/env python
import time
while True:
# Your code here
time.sleep(60)
I think the right way to run a thread repeatedly is the next:
import threading
import time
def f():
print("hello world") # your code here
myThread.run()
if __name__ == '__main__':
myThread = threading.Timer(3, f) # timer is set to 3 seconds
myThread.start()
time.sleep(10) # it can be loop or other time consuming code here
if myThread.is_alive():
myThread.cancel()
With this code, the function f is executed every 3 seconds within the 10 seconds time.sleep(10). At the end running of thread is canceled.
If you want to invoke the method "on the clock" (e.g. every hour on the hour), you can integrate the following idea with whichever threading mechanism you choose:
import time
def wait(n):
'''Wait until the next increment of n seconds'''
x = time.time()
time.sleep(n-(x%n))
print(time.asctime())
[snip. removed non async version]
To use asyncing you would use trio. I recommend trio to everyone who asks about async python. It is much easier to work with especially sockets. With sockets I have a nursery with 1 read and 1 write function and the write function writes data from an deque where it is placed by the read function; and waiting to be sent. The following app works by using trio.run(function,parameters) and then opening an nursery where the program functions in loops with an await trio.sleep(60) between each loop to give the rest of the app a chance to run. This will run the program in a single processes but your machine can handle 1500 TCP connections insead of just 255 with the non async method.
I have not yet mastered the cancellation statements but I put at move_on_after(70) which is means the code will wait 10 seconds longer than to execute a 60 second sleep before moving on to the next loop.
import trio
async def execTimer():
'''This function gets executed in a nursery simultaneously with the rest of the program'''
while True:
trio.move_on_after(70):
await trio.sleep(60)
print('60 Second Loop')
async def OneTime_OneMinute():
'''This functions gets run by trio.run to start the entire program'''
with trio.open_nursery() as nursery:
nursery.start_soon(execTimer)
nursery.start_soon(print,'do the rest of the program simultaneously')
def start():
'''You many have only one trio.run in the entire application'''
trio.run(OneTime_OneMinute)
if __name__ == '__main__':
start()
This will run any number of functions simultaneously in the nursery. You can use any of the cancellable statements for checkpoints where the rest of the program gets to continue running. All trio statements are checkpoints so use them a lot. I did not test this app; so if there are any questions just ask.
As you can see trio is the champion of easy-to-use functionality. It is based on using functions instead of objects but you can use objects if you wish.
Read more at:
[1]: https://trio.readthedocs.io/en/stable/reference-core.html