I am learning about threading using Python 3.8.2. I have one function with an infinite loop, and then two other functions that use the threading.Timer class.
def x:
while True:
dosomething()
def f1():
dosomething2()
threading.Timer(60,f1).start()
def f2():
dosomething3()
threading.Timer(100,f2).start()
Then I start three threads:
t1 = threading.Thread(target=x)
t2 = threading.Thread(target=f1)
t3 = threading.Thread(target=f2)
When it comes time to execute the f1 or f2, I don't want the x() to be executing at the same time, (they might be using the same resource) and pause, let f2 or f1 finish, and then resume the infinite loop in x(). How can I do this?
I've looked at join() but it seems to me it will wait forever for f1() and f2() because it creates a new thread every time and won't terminate.
Here's a photo to explain the flow:
Here's a possible solution, I added the functions dosomething() dosomething2() dosomething3() into the code to have a working example. I've also changed the timer on the threads to 6 and 10 seconds each instead of 60 and 100 so we don't have to wait that long to see their functionality.
dosomething()
prints 'dosomething is running' every second if no other function is running
dosomething2()
sets dosomething2.status = 'run'
prints 'dosomething2 is running 1st second'
waits one second
prints 'dosomething2 is running 2nd second'
sets dosomething2.status = 'sleep'
dosomething3()
sets dosomething3.status = 'run'
prints 'dosomething3 is running 1st second'
waits one second
prints 'dosomething3 is running 2nd second'
waits one second
prints 'dosomething3 is running 3rd second'
sets dosomething3.status = 'sleep'
The first and last lines in dosomething2() and dosomething3() will be triggers to let our x() function know to run only if both functions are in the state of 'sleep'.
You could use global variables instead of dosomething2.status and dosomething3.status but some people recommend not using them.
Code
import time
import threading
def dosomething():
print('dosomething is running')
time.sleep(1)
def dosomething2():
dosomething2.status = 'run'
print('\tdosomething2 is running 1st second')
time.sleep(1)
print('\tdosomething2 is running 2nd second')
dosomething2.status = 'sleep'
def dosomething3():
dosomething3.status = 'run'
print('\tdosomething3 is running 1st second')
time.sleep(1)
print('\tdosomething3 is running 2nd second')
time.sleep(1)
print('\tdosomething3 is running 3rd second')
dosomething3.status = 'sleep'
dosomething2.status = ''
dosomething3.status = ''
def x():
while True:
if dosomething2.status == 'sleep' and dosomething3.status == 'sleep':
dosomething()
def f1():
dosomething2()
threading.Timer(6,f1).start()
def f2():
dosomething3()
threading.Timer(10,f2).start()
t1 = threading.Thread(target=x)
t2 = threading.Thread(target=f1)
t3 = threading.Thread(target=f2)
t1.start()
t2.start()
t3.start()
Related
I recently started studying threads in python, and I ran into this problem: I need the "two" function to finish executing after executing the function one in the thread, but the join method does not work, apparently because of the while true loop in the third function. I tried using queue, but it didn't work either.
the code itself:
from threading import Thread,Event
def one():
event.set()
thr.join()
for i in range(3):
print('some print')
time.sleep(1)
def two():
t = Thread(target=one)
t.start()
#with t.join() here the program does not work at all, same thing with event.set()
print('func two finished')
def three(callback, event):
c = 0
while True:
c += 1
time.sleep(1)
print('func 3 is working')
if c == 5:
two()
if event.is_set():
callback(c)
print('func 3 is stopped')
break
def callback(t):
print('callback ',t)
def thread(callback):
global event, thr
event = Event()
thr = Thread(target=three, args=(callback, event,))
thr.start()
thr.join()
thread(callback)
current output:
func 3 is working
func 3 is working
func 3 is working
func 3 is working
func 3 is working
func two finished
callback 5
func 3 is stopped
some print
some print
some print
expected:
func 3 is working
func 3 is working
func 3 is working
func 3 is working
func 3 is working
callback 5
func 3 is stopped
some print
some print
some print
func two finished
After running the code I understand by "not working" you mean the program finished before all prints are printed.
The reason is that you join the thr thread twice, one of them by the main thread.
The sequence of return of join is not guaranteed.
When the main thread finished, all threads created by the program also finish, so they terminated no matter what.
Same thing when setting the event, it makes the main thread exit and kill the remaining threads.
To do what you intend, you should wait for the one thread in the main thread.
from threading import Thread,Event
def one():
event.set()
thr.join()
for i in range(3):
print('some print')
time.sleep(1)
def two():
t = Thread(target=one)
t.start()
#with t.join() here the program does not work at all, same thing with event.set()
print('func two finished')
def three(callback, event):
c = 0
while True:
c += 1
time.sleep(1)
print('func 3 is working')
if c == 5:
two()
if event.is_set():
callback(c)
print('func 3 is stopped')
break
def callback(t):
print('callback ',t)
def thread(callback):
global event, thr
event = Event()
thr = Thread(target=three, args=(callback, event,))
thr.start()
thr.join()
thread(callback)
Note that as other said, this might be nice for learning purpesses but has to be modified if you want to actually use it in real life code.
Your program creates a deadlock if you un-comment that t.join() call in function two;
The thr thread cannot finish until after the t thread has finished because the thr thread calls t.join() in function two.
The t thread cannot finish until after the thr thread has finished because the t thread calls thr.join() in function one.
Neither thread can finish until after the other thread finishes. Therefore, neither thread can ever finish.
Why does one join the thr thread?
def one():
event.set()
thr.join() # What were you trying to do here?
for i in range(3):
print('some print')
time.sleep(1)
Your program will give the output you wanted if you comment out that join call, and uncomment the t.join() call in function two.
The sequence you need is obtained by a small permutation of commands. But it is not clear why you need threads if everything is done sequentially.
from threading import Thread, Event
import time
def one(event):
event.set()
for i in range(3):
print('some print')
time.sleep(1)
def two(event):
t = Thread(target=one, args=(event,))
t.start()
t.join()
print('func two finished')
def three(event):
c = 0
while True:
c += 1
time.sleep(1)
print('func 3 is working')
if c == 5:
callback(c)
print('func 3 is stopped')
two(event)
break
def callback(t):
print('callback ', t)
def thread():
event = Event()
thr = Thread(target=three, args=(event,))
thr.start()
thread()
--------------------------------
func 3 is working
func 3 is working
func 3 is working
func 3 is working
func 3 is working
callback 5
func 3 is stopped
some print
some print
some print
func two finished
This is a comment, not an answer.
This makes no sense:
t = Thread(target=one, args=(event,))
t.start()
t.join()
There's no point in starting a new thread if you aren't going to do something concurrently with the thread. Either do this,
t = Thread(target=one, args=(event,))
t.start()
do_something_else_concurrently_with_thread_t(...)
t.join()
Or just just call the function instead of creating a new thread to call it:
one(event)
If you don't want concurrency, then you don't need threads.
I want to run two threads in parallel (on python3.6), which works for following code example:
import threading
from time import sleep
# use Thread to run def in background
# Example:
def func1():
while True:
sleep(1)
print("Working")
def func2():
while True:
sleep(2)
print("Working2")
Thread(target = func1).start()
Thread(target = func2).start()
but it does not work for threading.Thread:
import threading
from time import sleep
# use Thread to run def in background
# Example:
def func1():
while True:
sleep(1)
print("Working")
def func2():
while True:
sleep(2)
print("Working2")
x = threading.Thread(target=func1())
y = threading.Thread(target=func2())
x.start()
y.start()
I would like to use the latter option to check if x or y are still alive.
There's a difference between Thread(target = func1) (first code) and Thread(target=func1()) (second code):
the first one passes the function object to Thread
the second one executes the function (because you called it with func1()) and passes its return value to Thread
Since you want the threads to call your functions, don't call them:
x = threading.Thread(target=func1)
y = threading.Thread(target=func2)
x.start()
y.start()
I am trying to use while loops inside threads for a bigger project. For simplicity I created an easier example to test it, but it doesn`t work.
My goal is to control the thread for the main function and when the variable go_thread_one is switched to False the thread should end. At the moment the second thread is not being used and only the first thread print its text.
How can I fix this error?
Below is the simplified version of my code:
import time
from threading import Thread
go_thread_one = True
def first_thread():
while go_thread_one:
print('Thread 1')
time.sleep(0.5)
def second_thread():
print('Thread 2')
if __name__ == "__main__":
t1 = Thread(target=first_thread())
t2 = Thread(target=second_thread())
t1.daemon = True
t2.daemon = True
t1.start()
t2.start()
time.sleep(2)
go_thread_one = False
print("end main Thread")
First of all, there is a problem in these lines:
t1 = Thread(target=first_thread())
t2 = Thread(target=second_thread())
You should pass a callable object to the Thread, but instead you call a function and pass its result. So you don't even create a t1, but go inside first_thread function and loop there forever.
To fix this, change Thread creation to:
t1 = Thread(target=first_thread)
t2 = Thread(target=second_thread)
Next, the
go_thread_one = False
will not give a desired effect – main thread will finish after time.sleep(2) even without this line.
To deal with it, you can add
t1.join()
t2.join()
I am trying to make a python program that runs multiple processes each in an infinite loop at the same time, but only one process will execute at a time, the first one in code, then the rest of the program will not run. What do i need to do to make both procceses and the main one execute at the same time?
from multiprocessing import *
import time
def test1(q):
while True:
q.put("Banana")
time.sleep(2)
def test2(q):
while True:
q.put("internet")
time.sleep(3)
if __name__ == "__main__":
q = Queue()
t1 = Process(target=test1(q))
t2 = Process(target=test2(q))
t1.start()
t2.start()
q.put("rice")
while True:
print(q.get())
The reason for your problem is with the lines:
t1 = Process(target=test1(q))
t2 = Process(target=test2(q))
There you will actually call test1 and test2, respectively (even though you will never reach the test2 call). After running the functions it will then use the return result a target. What you want is:
t1 = Process(target=test1, args=(q,))
t2 = Process(target=test2, args=(q,))
Thus, you do not want to actually run the test1 and test2functions, but use their references (addresses) as target and then you have to provide their input arguments in a separate parameter args.
I have a code:
function_1()
function_2()
Normally, function_1() takes 10 hours to end.
But I want function_1() to run for 2 hours, and after 2 hours, function_1 must return and program must continue with function_2(). It shouldn't wait for function_1() to be completed. Is there a way to do this in python?
What makes functions in Python able to interrupt their execution and resuming is the use of the "yield" statement -- your function then will work as a generator object. You call the "next" method on this object to have it start or continue after the last yield
import time
def function_1():
start_time = time.time()
while True:
# do long stuff
running_time = time.time() -start_time
if running_time > 2 * 60 * 60: # 2 hours
yield #<partial results can be yield here, if you want>
start_time = time.time()
runner = function_1()
while True:
try:
runner.next()
except StopIteration:
# function_1 had got to the end
break
# do other stuff
If you don't mind leaving function_1 running:
from threading import Thread
import time
Thread(target=function_1).start()
time.sleep(60*60*2)
Thread(target=function_2).start()
You can try to use module Gevent: start function in thread and kill that thread after some time.
Here is example:
import gevent
# function which you can't modify
def func1(some_arg)
# do something
pass
def func2()
# do something
pass
if __name__ == '__main__':
g = gevent.Greenlet(func1, 'Some Argument in func1')
g.start()
gevent.sleep(60*60*2)
g.kill()
# call the rest of functions
func2()
from multiprocessing import Process
p1 = Process(target=function_1)
p1.start()
p1.join(60*60*2)
if p1.is_alive():p1.terminate()
function_2()
I hope this helps
I just tested this using the following code
import time
from multiprocessing import Process
def f1():
print 0
time.sleep(10000)
print 1
def f2():
print 2
p1 = Process(target=f1)
p1.start()
p1.join(6)
if p1.is_alive():p1.terminate()
f2()
Output is as expected:
0
2
You can time the execution using the datetime module. Probably your optimizer function has a loop somewhere. Inside the loop you can test how much time has passed since you started the function.
def function_1():
t_end = datetime.time.now() + datetime.timedelta(hours=2)
while not converged:
# do your thing
if datetime.time.now() > t_end:
return