I am trying to implement a timer for calling the function for printing the queue after specific time .I am also able to cancel the timer if the queue gets filled before that specified time and print the queue.But after that my timer object behaves abruptly causing timers to overlap for example if the queue gets filled in 2 sec then it prints the queue in 2,8,2,8... time interval instead of 2,10 time interval
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1'))
channel = connection.channel()
channel.queue_declare(queue='final', durable=True)
global msg_queue
global t
msg_queue=queue.Queue(maxsize=6)
def empty_queue():
print(time.time())
l=[]
i=int(msg_queue.qsize())
while i!=0:
l.append(msg_queue.get())
i-=1
t=threading.Timer(10,empty_queue)
print(l)
t.start()
t=threading.Timer(10,empty_queue)
print(time.time())
t.start()
while True:
if int(msg_queue.qsize())<6:
consume_generator = channel.consume(queue='final', no_ack=True)
result=next(consume_generator)
msg_queue.put(json.loads(result[2].decode("utf-8")))
else:
print("more",time.time())
t.cancel()
empty_queue()
I have solved the issue by cancelling the timer to prevent its own duplicacy
def empty_queue():
global t
print(time.time())
l=[]
i=int(msg_queue.qsize())
while i!=0:
l.append(msg_queue.get())
i-=1
if t.isAlive():
t.cancel()
t=threading.Timer(10,empty_queue)
print(l)
t.start()
You could nest a if statement inside the timer that says if queue is full then disable this if statement then let the timer continue until its done without affecting the queue. I don't think it will cause conflict with your program because the timer is probably a closure.
Related
I have a loop which makes a get request to a webservice to fetch data and do some stuff, but I want to 'manually' terminate the thread/event, which I achieved with the following example:
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
exit.wait(10) #wait 10 seconds
With that, the only thing that terminates it's the sleep time between loops. How can I also kill the loop so it doesn't keep running until it gets to the last iteration?
nvm i've solved it like this
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
if exit.wait(10):
break
the condition returns true when killed and also sleeps the 10 seconds, so it works
you have 2 options ,
kill the thread or process entirely
or making the loop's boolean false. going that way
you could use a global variable in this way: [Python 3.7] , run it to see
from threading import Thread
from time import sleep
global glob
glob=True
def threaded_function():
while glob:
print("\n [Thread] this thread is running until main function halts this")
sleep(0.8)
if __name__ == "__main__":
thread = Thread(target = threaded_function, args = ())
thread.start()
for i in range(4,0,-1):
print("\n [Main] thread will be terminated in "+str(i)+" seconds")
sleep(1)
glob=False
while True:
print("[Main] program is over")
sleep(1)
I want to check if the timer exists and simply stop it by running the following code a second time, but instead it waits for the timer to stop
and runs again.
import time
import threading
def run():
print("foobar")
t = threading.Timer(3, run)
def stop_t():
time.sleep(1)
t.cancel()
print("canceled")
if t.is_alive():
t.cancel()
else:
t.start()
The problem is that if you run this code again, you will redefine the t variable to a new timer that has not started yet, you can try something like this:
import time
import threading
def run():
print("foobar")
def toggle_timer(t):
if t.is_alive():
print('timer is alive, killing it')
t.cancel()
else:
t.start()
print('timer is dead, starting it')
t = threading.Timer(3, run)
toggle_timer(t)
toggle_timer(t)
Output:
timer is dead, starting it
timer is alive, killing it
I want to run a function every few seconds in Python. The function execution takes some time, and I want to include that in the waiting time as well.
I don't want to do this, because it is not strictly executed every 2 seconds and will break the periodicity (my_function also takes time to execute.)
while True:
time.sleep(2)
my_function()
I don't want to do this neither, because it uses too much CPU on the while loop of Thread-2.
# Thread-1
While True:
time.sleep(2)
event.set()
# Thread-2
While True:
if event.is_set():
my_function()
else:
pass
Can anyone please help me?
You can consider ischedule. It takes care of the function execution time right out the box, and doesn't waste CPU time for busy waiting. You can use:
from ischedule import schedule, run_loop
schedule(my_function, interval=2)
run_loop()
I believe the schedule module is your friend
I found this code works pretty well, if I understood your question correctly.
Code broken down:
runs func1
runs func2
waits 2s
does something else after that
waits 1s
does it all again
import threading
import time
def func1():
print("function 1 has been called")
def func2():
print("function 2 has been called")
def loop():
print("loop print 1")
thread = threading.Thread(target=func1, name="thread")
thread.start()
while thread.is_alive():
continue
if not thread.is_alive():
thread2 = threading.Thread(target=func2, name="thread2")
thread2.start()
while thread2.is_alive():
continue
time.sleep(2)
while True:
loop()
print("Some other thing")
time.sleep(1)
The following code:
import threading
import time
from functools import partial
from itertools import count
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.is_set():
break
time.sleep(sleep_interval)
print('Slept %s' % sleep_interval)
print('Prod terminating')
if __name__ == '__main__':
stop_event = threading.Event() #https://stackoverflow.com/a/41139707/281545
target = partial(daemon_loop, sleep_interval=2, stop_event=stop_event)
prod_thread = threading.Thread(target=target,
# daemon=True
)
try:
prod_thread.start()
while True:
time.sleep(10)
except KeyboardInterrupt:
print('Terminating...')
stop_event.set()
prints on a keyboard interrupt:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
Slept 2
2
Prod terminating
Uncommenting the # daemon=True line results in the prod_thread being ended immediately:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
My question is what is the preferred/more pythonic way to deal with thread termination - should I drop the Event machinery and just mark the thread as daemon or is there some edge case I miss?
See:
Daemon Threads Explanation
How to stop daemon thread?
I haven't done enough Python to give you a "Pythonic" answer, but I can answer in more general programming terms.
Firstly, I'm not a fan of terminating threads. There are cases where it is safe and OK, such as your example here - but terminating in the middle of print writing its output would feel a little dirty.
Secondly, if you want to continue using sleep (which I'm also not a fan of) you could repeat your if stop_event.is_set(): and break after the sleep. (Don't move the code, copy it.) The main problem with sleep in this case is that it will wait the full sleep_interval even if the event is set during that time.
Thirdly - and my preference - instead of using sleep, do a wait on the event with a timeout. If the event is not set during the wait, wait returns false after waiting the timeout period. If the event is set before or during the wait, wait returns true immediately (that is, it aborts the timeout, giving you fast, clean shutdown of the thread.)
So your code would look something like this:
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.wait(sleep_interval):
break
print('Slept %s' % sleep_interval)
print('Prod terminating')
I have thrown together a quick threading test:
import threading
def test():
print "it don't work"
while True:
threading.Timer(1, test).start()
It runs test, but it doesn't wait. What's wrong?
In each loop iteration, you start a new thread. Therefore you will reach the limit of allowed thread and you will get an exception : can't start new thread.
while True:
threading.Timer(1, test).start()
You can add global flag and wait until the function is executed - You should use time.sleep to avoid busy waiting.
a = False
def test():
global a
print("hallo")
a = True
threading.Timer(10, test).start()
while not a:
time.sleep(1)
print('done')