my tool stops randomly and it seems like all threads are 'ghosts'.
How does it work:
The tool loops until the max number of allowed threads at the same time are running, in this case 20. When a thread finishes it starts the next one.
Problem:
After like an hour of doing this, the tool is stuck at 20 Threads running but nothing happens anymore.
Thanks in advance everyone!
maxthreadcount = 20
while True:
if threading.active_count() < maxthreadcount:
threading.Thread(target=Dealer).start()
Dealer:
def Dealer():
print("thread started")
return
You need to terminate previously created threads after their job (print command in this case) is done.
Take a look at this example from this article:
class CountdownTask:
def __init__(self):
self._running = True
def terminate(self):
self._running = False
def run(self, n):
while self._running and n > 0:
print('T-minus', n)
n -= 1
time.sleep(5)
c = CountdownTask()
t = Thread(target = c.run, args =(10, ))
t.start()
...
# Signal termination
c.terminate()
# Wait for actual termination (if needed)
t.join()
I think you should call self.terminate() after doing the print. Something like below:
class Dealer():
def __init__(self):
self._running = True
def run(self):
print("thread started")
return self.terminate()
def terminate(self):
self._running = False
Edit
I also believe you can make use of python's ThreadPool to this extent. Instead of spawning threads yourself, you might be able to reuse threads after their assigned task is over, for the new tasks.
I'm trying to light a 5mm LED while a function is running. When this function (more details about this below) is finished and has returned a value I would like to break the while loop.
Current code for while loop:
pins = [3,5,8,15,16]
def piBoard():
finished = 0
while finished!=10:
for pin in pins
GPIO.output(
pin, GPIO.HIGH
)
time.sleep(0.1)
GPIO.output(
pin, GPIO.LOW
)
finished+=1
Now in the above example I just run the while loop until the count is equal to 10, not best practice. I would like the while loop to break if my next function has returned a value.
Function I want to break my while loop when returned its value
def myFunction():
Thread(target = piBoard().start()
// Trying to recognize the song
return the song which is recognized
Thanks, - K.
It sounds to me like you want to write a class that extends Thread and implements __enter__ and __exit__ methods to make it work in the with statement. Simple to implement, simple syntax, works pretty well. The class will look like this:
import threading
class Blinky(threading.Thread):
def __init__(self):
super().__init__()
self.daemon = True
self._finished = False
def __enter__(self):
self.start()
def __exit__(self, exc_type, exc_val, exc_tb):
self.stop()
def run(self):
# turn light on
while not self._finished:
time.sleep(.5)
# turn light off
def stop(self):
self._finished = True
Then, to run your function, you simply put:
with Blinky():
my_function()
The light should turn on once the with statement is reached and turn off up to a half second after the context of the with is exited.
In while condition put true and in while loop put if statement which will check if your function return any value if return write break
You need some kind of inter-thread communication. threading.Event is about as simple as you can get.
import threading
song_recognized_event = threading.event()
in your song recognizer, call set() once the song is recognized.
In your LED loop, check isSet() occasionally while toggling LEDs.
while not song_recognized_event.isSet():
# toggle LEDs
Run clear() to reset it.
if you are open to using threads.
you can achieve this by using threads.
here's the example code
from concurrent.futures._base import as_completed
from concurrent.futures.thread import ThreadPoolExecutor
WORK_FINISHED = False
def piBoard():
while not WORK_FINISHED:
# Do some stuff
# Drink some coffee
def myFunction():
time.sleep(5)
global WORK_FINISHED
WORK_FINISHED = True #update gobal status flag
return something
if __name__ == '__main__':
futures = []
MAX_WORKERS = 5 #max number of threads you want to create
with ThreadPoolExecutor(MAX_WORKERS) as executor:
executor.submit(piBoard)
# submit your function to worker thread
futures.append(executor.submit(myFunction))
# if you need to get return value from `myFunction`
for fut in as_completed(futures):
res = fut.result()
Hope this helps.
Using decorator and asyncio, inspired by #Eric Ed Lohmar:
import asyncio
def Blink():
from functools import wraps
async def _blink():
while True:
print("OFF")
await asyncio.sleep(.5)
print("ON")
await asyncio.sleep(.5)
def Blink_decorator(func):
#wraps(func)
async def wrapper(*args,**kwargs):
asyncio.ensure_future(_blink())
await func(*args,**kwargs)
return wrapper
return Blink_decorator
#Blink()
async def longTask():
print("Mission Start")
await asyncio.sleep(3)
print("Mission End")
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(longTask())
I want two objects of the same class to operate concurrently. The class "MyClass" has a function that connects an instance to another instance of the class. I also need to keep track of the objects that have been created (oList). what I am trying is:
main.py:
from MyClass import MyClass
import time
oList = []
class oCreator1(Thread):
def __init__(self):
Thread.__init__(self)
self.o1 = MyClass()
def run(self):
while 1:
time.sleep(1)
print "Hi"
def getO1(self):
return self.o1
class oCreator2(Thread):
def __init__(self):
Thread.__init__(self)
self.o2 = MyClass()
def run(self):
while 1:
time.sleep(1)
print "Bye!"
def getO2(self):
return self.o2
main():
threadList = []
global oList
oc1 = oCreator1()
threadList.append(oc1)
o1 = oc1.getO1()
oList.append(o1)
oc2 = oCreator2()
threadList.append(oc2)
o2 = oc2.getO2()
oList.append(o2)
o1.connToAnotherO(o2)
print oList
for t in threadList:
t.start()
t.join()
if __name__ == '__main__':
main()
But the only thing that is printed is "Hi". I really want to know the things I'm doing wrong and the right way to do it. Thank you in advance.
for t in threadList:
t.start()
t.join()
The t.join() call waits for the thread t to finish. That means when you start the first thread, you wait for it to finish before starting the second, but the first thread is just going to keep printing Hi forever. It'll never finish.
Don't join, or don't start joining until all threads have started.
I have a simple example script constructed that defines three separate processes using multiprocessing in python. My objective is to have one parent thread that spawns two smaller threads that will collect and process data.
Currently, my implementation looks like this:
from Queue import Queue,Empty
from multiprocessing import Process
import time
import hashlib
class FillQueue(Process):
def __init__(self,q):
Process.__init__(self)
self.q = q
def run(self):
i = 0
while i is not 5:
print 'putting'
self.q.put('foo')
i+=1
self.q.put('|STOP|')
class ConsumeQueue(Process):
def __init__(self,q):
Process.__init__(self)
self.q = q
def run(self):
print 'Consume'
while True:
try:
value = self.q.get(False)
print value
if value == '|STOP|':
print 'done'
break;
except Empty:
print 'Nothing to process atm'
class Ripper(Process):
q = Queue()
def __init__(self):
self.fq = FillQueue(self.q)
self.cq = ConsumeQueue(self.q)
self.fq.daemon = True
self.cq.daemon = True
def run(self):
try:
self.fq.start()
self.cq.start()
except KeyboardInterrupt:
print 'exit'
if __name__ == '__main__':
r = Ripper()
r.start()
As it runs presently, the output from the script on CLI looks like this:
putting
putting
putting
putting
putting
Consume
foo
foo
foo
foo
foo
|STOP|
done
Obviously, the way I am starting my two threads is blocking, since the consumer doesn't even begin to process the items in the queue until the filler finishes adding items.
How should I rewrite this to make both threads begin immediately and not block, so the consumer will simply pass to the Empty except block while there is no work to process, but will exit completely when it receives the stop message?
EDIT: typo, had the start and run methods mixed up
You seem to be starting multiple processes using multiprocessing.Process.
However, you are using Queue.Queue which is only threadsafe, and not designed to be used by multiple processes.
shevek's answer is valid as well, but as a start, you should replace Queue.Queue with multiprocessing.Queue.
try this:
from Queue import Empty
from multiprocessing import Process, Queue
import time
import hashlib
class FillQueue(object):
def __init__(self, q):
self.q = q
def run(self):
i = 0
while i < 5:
print 'putting'
self.q.put('foo %d' % i )
i+=1
time.sleep(.5)
self.q.put('|STOP|')
class ConsumeQueue(object):
def __init__(self, q):
self.q = q
def run(self):
while True:
try:
value = self.q.get(False)
print value
if value == '|STOP|':
print 'done'
break;
except Empty:
print 'Nothing to process atm'
time.sleep(.2)
if __name__ == '__main__':
q = Queue()
f = FillQueue(q)
c = ConsumeQueue(q)
p1 = Process(target=f.run)
p1.start()
p2 = Process(target=c.run)
p2.start()
p1.join()
p2.join()
I think your program works fine. The CPU processes only one thing at a time, for a short time. However, the time required to put all your stuff in the queue is very short. So there is no reason that the filler cannot do this in one time slice.
If you add some delays in the filler, I think you should see that it actually works as you expect.
I need to wait in a script until a certain number of conditions become true?
I know I can roll my own eventing using condition variables and friends, but I don't want to go through all the trouble of implementing it, since some object property changes come from external thread in a wrapped C++ library (Boost.Python), so I can't just hijack __setattr__ in a class and put a condition variable there, which leaves me with either trying to create and signal a Python condition variable from C++, or wrap a native one and wait on it in Python, both of which sound fiddly, needlessly complicated and boring.
Is there an easier way to do it, barring continuous polling of the condition?
Ideally it would be along the lines of
res = wait_until(lambda: some_predicate, timeout)
if (not res):
print 'timed out'
Unfortunately the only possibility to meet your constraints is to periodically poll, e.g....:
import time
def wait_until(somepredicate, timeout, period=0.25, *args, **kwargs):
mustend = time.time() + timeout
while time.time() < mustend:
if somepredicate(*args, **kwargs): return True
time.sleep(period)
return False
or the like. This can be optimized in several ways if somepredicate can be decomposed (e.g. if it's known to be an and of several clauses, especially if some of the clauses are in turn subject to optimization by being detectable via threading.Events or whatever, etc, etc), but in the general terms you ask for, this inefficient approach is the only way out.
Another nice package is waiting - https://pypi.org/project/waiting/
install:
pip install waiting
Usage:
You pass a function that will be called every time as a condition, a timeout, and (this is useful) you can pass a description for the waiting, which will be displayed if you get TimeoutError.
using function:
from waiting import wait
def is_something_ready(something):
if something.ready():
return True
return False
# wait for something to be ready
something = # whatever
wait(lambda: is_something_ready(something), timeout_seconds=120, waiting_for="something to be ready")
# this code will only execute after "something" is ready
print("Done")
Note: the function must return a boolean - True when the wait is over, False otherwise
Here is another solution. The goal was to make threads to wait on each other before doing some work in a very precise order. The work can take unknown amount of time. Constant polling is not good for two reasons: it eats CPU time and action does not start immediately after condition is met.
class Waiter():
def __init__(self, init_value):
self.var = init_value
self.var_mutex = threading.Lock()
self.var_event = threading.Event()
def WaitUntil(self, v):
while True:
self.var_mutex.acquire()
if self.var == v:
self.var_mutex.release()
return # Done waiting
self.var_mutex.release()
self.var_event.wait(1) # Wait 1 sec
def Set(self, v):
self.var_mutex.acquire()
self.var = v
self.var_mutex.release()
self.var_event.set() # In case someone is waiting
self.var_event.clear()
And the way to test it
class TestWaiter():
def __init__(self):
self.waiter = Waiter(0)
threading.Thread(name='Thread0', target=self.Thread0).start()
threading.Thread(name='Thread1', target=self.Thread1).start()
threading.Thread(name='Thread2', target=self.Thread2).start()
def Thread0(self):
while True:
self.waiter.WaitUntil(0)
# Do some work
time.sleep(np.random.rand()*2)
self.waiter.Set(1)
def Thread1(self):
while True:
self.waiter.WaitUntil(1)
# Do some work
time.sleep(np.random.rand())
self.waiter.Set(2)
def Thread2(self):
while True:
self.waiter.WaitUntil(2)
# Do some work
time.sleep(np.random.rand()/10)
self.waiter.Set(0)
Waiter for multiprocessing:
import multiprocessing as mp
import ctypes
class WaiterMP():
def __init__(self, init_value, stop_value=-1):
self.var = mp.Value(ctypes.c_int, init_value)
self.stop_value = stop_value
self.event = mp.Event()
def Terminate(self):
self.Set(self.stop_value)
def Restart(self):
self.var.value = self.init_value
def WaitUntil(self, v):
while True:
if self.var.value == v or self.var.value == self.stop_value:
return
# Wait 1 sec and check aiagn (in case event was missed)
self.event.wait(1)
def Set(self, v):
exit = self.var.value == self.stop_value
if not exit: # Do not set var if threads are exiting
self.var.value = v
self.event.set() # In case someone is waiting
self.event.clear()
Please comment if this is still not the best solution.
You've basically answered your own question: no.
Since you're dealing with external libraries in boost.python, which may change objects at their leisure, you need to either have those routines call an event handler refresh, or work with a condition.
Here is the threading extention to Alex's solution:
import time
import threading
# based on https://stackoverflow.com/a/2785908/1056345
def wait_until(somepredicate, timeout, period=0.25, *args, **kwargs):
must_end = time.time() + timeout
while time.time() < must_end:
if somepredicate(*args, **kwargs):
return True
time.sleep(period)
return False
def wait_until_par(*args, **kwargs):
t = threading.Thread(target=wait_until, args=args, kwargs=kwargs)
t.start()
print ('wait_until_par exits, thread runs in background')
def test():
print('test')
wait_until_par(test, 5)
From the computational perspective there must be a check for all conditions somewhere, sometime. If you have two parts of code, one that generates conditions changes and the other one that should be executed when some are true, you can do the following:
Have the code that changes conditions in, say, main thread, and the code that should be launched when some conditions are true, in a worker thread.
from threading import Thread,Event
locker = Event()
def WhenSomeTrue(locker):
locker.clear() # To prevent looping, see manual, link below
locker.wait(2.0) # Suspend the thread until woken up, or 2s timeout is reached
if not locker.is_set(): # when is_set() false, means timeout was reached
print('TIMEOUT')
else:
#
# Code when some conditions are true
#
worker_thread = Thread(target=WhenSomeTrue, args=(locker,))
worker_thread.start()
cond1 = False
cond2 = False
cond3 = False
def evaluate():
true_conditions = 0
for i in range(1,4):
if globals()["cond"+str(i)]: #access a global condition variable one by one
true_conditions += 1 #increment at each true value
if true_conditions > 1:
locker.set() # Resume the worker thread executing the else branch
#Or just if true_conditions > 1: locker.set();
#true_conditions would need be incremented when 'True' is written to any of those variables
#
# some condition change code
#
evaluate()
For more information concerning this method, visit: https://docs.python.org/3/library/threading.html#event-objects
Proposed solution:
def wait_until(delegate, timeout: int):
end = time.time() + timeout
while time.time() < end:
if delegate():
return True
else:
time.sleep(0.1)
return False
Usage:
wait_until(lambda: True, 2)
I once used this in my code:
while not condition:
pass
Hope this helps
In 2022 now you could use https://trio-util.readthedocs.io/en/latest/#trio_util.AsyncValue
I think this comes closest to what you want in its "smoothest" form
This worked for me
direction = ''
t = 0
while direction == '' and t <= 1:
sleep(0.1)
t += 0.1
This is for waiting for a signal while making sure time limit of 1 second
here's how:
import time
i = false
while i == false:
if (condition):
i = true
break
Here's my Code I used during one of my Projects :
import time
def no() :
if (Condition !!!) :
it got true
oh()
else:
time.sleep(1) /Don't remove or don't blame me if ur system gets ""DEAD""
no()
def oh() : /Ur main program
while True:
if(bla) :
.......
no()
else :
time.sleep(1)
oh()
oh()
Hope it Helps