Develop Reference

How can I make one thread sleep while the others run in python

I'm new to multithreading, and have been trying to make a timer in the background for one of my programs while several other threads execute their code. I currently have my code set up like this:
def timer(length):
global kill
kill = False
print('Timer starting!')
for i in range(length):
time.sleep(1)
kill = True
def r(interval, id):
print(id, "running")
while True:
if kill:
break
time.sleep(interval)
print(f'{id}: {time.localtime()}')
timerThread = threading.Thread(target = timer(15))
runThread1 = threading.Thread(target = r(2, "thread1"))
runThread2 = threading.Thread(target = r(5, "thread2"))
threads = [timerThread, runThread1, runThread2]
timerThread.start()
runThread1.start()
runThread2.start()
for t in threads:
t.join()
Obviously, it's not a useful program (only meant to help me learn the basics of the threading module), but I still need to know how to fix this. I'm 95% sure the issue is caused by using "time.sleep(1)" in the timer function, but I don't know an alternative to use that would only affect one thread. I'd appreciate any suggestions.

You're calling your functions before you even create the thread:
timerThread = threading.Thread(target = timer(15))
runThread1 = threading.Thread(target = r(2, "thread1"))
runThread2 = threading.Thread(target = r(5, "thread2"))
You need to pass a callable target rather than the result of having already called it; the args can be passed separately:
timerThread = threading.Thread(target=timer, args=(15,))
runThread1 = threading.Thread(target=r, args=(2, "thread1"))
runThread2 = threading.Thread(target=r, args=(5, "thread2"))

Blocking other Threads while one Thread is running

Let's say I have two types of threads,
single thread that run every x min. let's call it A thread
multi threads run all the time. B threads when A thread do_something() i want All B threads to wait till A finish then resume them. i can't figure it out what to use.
I try to use threading.Condition, wait()/notifyAll() but it did not work as I want. once i put Condition in, it process 1 by 1 like synco threads or something. I want them to run freely.
This is the sample code I try to put them wait(), then notify them but it do 1 by 1 like join(). No idea what to us.
class ...
check = True
def xxx(self,g,con):
for i in range(3):
with con:
if self.check:
con.wait()
self.check = False
time.sleep(3)
print(g)
con = threading.Condition()
threading.Thread(target=xxx,args=('a',con,)).start()
threading.Thread(target=xxx,args=('b',con,)).start()
threading.Thread(target=xxx,args=('c',con,)).start()
time.sleep(2)
con.notifyAll()

Question: Blocking other Threads while one Thread is running
Instead of using threading.Condition(), this example uses threading.Barrier(...).
Used modules from docs.python.org:
module-threading
event-objects
barrier-objects
import time, threading
from threading import BrokenBarrierError
def worker_A(g, terminate, barrier):
# Counter to simulate conditional workload
do_something = 3
while not terminate.is_set():
if do_something == 0:
# Reset the barrier and wait until n_waiting == 2
barrier.reset()
while not terminate.is_set() and barrier.n_waiting < 2:
time.sleep(0.5)
# Now the other Threads waiting at the barrier
# Simulate worklaod ...
print('worker_A barrier.broken={} n_waiting={}'
.format(barrier.broken, barrier.n_waiting))
time.sleep(3)
# Call the third barrier.wait to release the barrier
try:
barrier.wait()
except BrokenBarrierError:
pass
# Reset counter to restart simulate conditional workload
do_something = 3
else:
# Count down and give the other threads a timeslice
do_something -= 1
time.sleep(0.5)
def worker_B(g, terminate, barrier):
while not terminate.is_set():
# Simulate workload ...
print('worker_B({})'.format(g))
time.sleep(1)
# Block at barrier.wait() if the barrier is NOT in the broken state
try:
barrier.wait()
except BrokenBarrierError:
pass
if __name__ == "__main__":
# Event to terminate all Threads save
terminate = threading.Event()
# Barrier to block worker_B Threads
# We use 3 Threads, therefore init with parties=3
barrier = threading.Barrier(3)
barrier.abort()
# Create and start the Threads
threads = []
for t in [(worker_A, 'a'), (worker_B, 'b'), (worker_B, 'c'), ]:
threads.append(threading.Thread(target=t[0], args=(t[1], terminate, barrier,)))
threads[-1].start()
time.sleep(0.2)
# Simulating MAIN Thread
time.sleep(20)
# Set the `terminate` Event to True,
# and abort the barrier to force all Threads to terminate
print('Terminate...')
terminate.set()
barrier.abort()
# Wait until all Threads terminated
for t in threads:
t.join()
print('EXIT MAIN')
Tested with Python: 3.5

How to use a thread pool to do infinite loop function?

I want to do a infinite loop function.
Here is my code
def do_request():
# my code here
print(result)
while True:
do_request()
When use while True to do this, it's a little slow, so I want to use a thread pool to concurrently execute the function do_request(). How to do this ?
Just like use ab (Apache Bench) to test HTTP server.

Finally, I've solved this problem. I use a variable to limit the thread number.
Here is my final code, solved my problem.
import threading
import time
thread_num = 0
lock = threading.Lock()
def do_request():
global thread_num
# -------------
# my code here
# -------------
with lock:
thread_num -= 1
while True:
if thread_num <= 50:
with lock:
thread_num += 1
t = threading.Thread(target=do_request)
t.start()
else:
time.sleep(0.01)
Thanks for all replies.

You can use threading in Python to implement this.
Can be something similar to this (when using two extra threads only):
import threading
# define threads
task1 = threading.Thread(target = do_request)
task2 = threading.Thread(target = do_request)
# start both threads
task1.start()
task2.start()
# wait for threads to complete
task1.join()
task2.join()
Basically, you start as many threads as you need (make sure you don't get too many, so your system can handle it), then you .join() them to wait for tasks to complete.
Or you can get fancier with multiprocessing Python module.

Try the following code:
import multiprocessing as mp
import time
def do_request():
while(True):
print('I\'m making requests')
time.sleep(0.5)
p = mp.Process(target=do_request)
p.start()
for ii in range(10):
print 'I\'m also doing other things though'
time.sleep(0.7)
print 'Now it is time to kill the service thread'
p.terminate()
The main thread stars a service thread that does the request and goes on until it has to, and then it finishes up the service thread.

Maybe you can use the concurrent.futures.ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor
import time
def wait_on_b(hello):
time.sleep(1)
print(hello) # b will never complete because it is waiting on a.
return 5
def wait_on_a():
time.sleep(1)
print(a.result()) # a will never complete because it is waiting on b.
return 6
executor = ThreadPoolExecutor(max_workers=2)
a = executor.submit(wait_on_b, 3)
b = executor.submit(wait_on_a)

How about this?
from threading import Thread, Event
class WorkerThread(Thread):
def __init__(self, logger, func):
Thread.__init__(self)
self.stop_event = Event()
self.logger = logger
self.func = func
def run(self):
self.logger("Going to start the infinite loop...")
#Your code
self.func()
concur_task = WorkerThread(logger, func = do_request)
concur_task.start()
To end this thread...
concur_task.stop_event.set()
concur_task.join(10) #or any value you like

How properly close thread in Python

I'm having trouble understanding threads in Python. I have this program:
import _thread, time
def print_loop():
num = 0
while 1:
num = num + 1
print(num)
time.sleep(1)
_thread.start_new_thread(print_loop, ())
time.sleep(10)
And my question is if I need to close the thread print_loop, because it looks to me that both threads end when the main thread ends. Is this proper way to handle threads?

First, avoid using the low-level API unless you absolutely have to. The threading module is preferred over _thread. In general in Python, avoid anything starting with an underscore.
Now, the method you are looking for is called join. I.e.
import time
from threading import Thread
stop = False
def print_loop():
num = 0
while not stop:
num = num + 1
print(num)
time.sleep(1)
thread = Thread(target=print_loop)
thread.start()
time.sleep(10)
stop = True
thread.join()

Printing Stdout In Command Line App Without Overwriting Pending User Input

In a basic Unix-shell app, how would you print to stdout without disturbing any pending user input.
e.g. Below is a simple Python app that echos user input. A thread running in the background prints a counter every 1 second.
import threading, time
class MyThread( threading.Thread ):
running = False
def run(self):
self.running = True
i = 0
while self.running:
i += 1
time.sleep(1)
print i
t = MyThread()
t.daemon = True
t.start()
try:
while 1:
inp = raw_input('command> ')
print inp
finally:
t.running = False
Note how the thread mangles the displayed user input as they type it (e.g. hell1o wo2rld3). How would you work around that, so that the shell writes a new line while preserving the line the user's currently typing on?

You have to port your code to some way of controlling the terminal as slightly better than a teletype -- e.g. with the curses module in Python's standard library, or other ways to move the cursor away before emitting output, then move it back to where the user's busy inputting stuff.

You could defer writing output until just after you receive some input. For anything more advanced you'll have to use Alex's answer
import threading, time
output=[]
class MyThread( threading.Thread ):
running = False
def run(self):
self.running = True
i = 0
while self.running:
i += 1
time.sleep(1)
output.append(str(i))
t = MyThread()
t.daemon = True
t.start()
try:
while 1:
inp = raw_input('command> ')
while output:
print output.pop(0)
finally:
t.running = False