really new to Python here and trying to get my head around threading. I have the code:
import threading, time
class myThread(threading.Thread):
def __init__(self, threadID, name, counter):
super(myThread, self).__init__()
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
print "Starting ", self.name
threadLock.acquire()
print_time(self.name , self.counter, 3)
#Free the lock to release the next thread
print "%s released, ready for the next thread"%self.name
threadLock.release()
class sillyThread(threading.Thread):
def run(self):
threadLock.acquire()
print "silly silly!"
time.sleep(2)
print "Silly silly!"
threadLock.release()
def print_time(threadName, delay, counter):
while counter:
time.sleep(delay)
print "%s: %s"%(threadName, time.ctime())
counter -= 1
threadLock = threading.Lock()
threads = []
# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)
thread3 = sillyThread()
thread1.start()
thread2.start()
thread3.start()
threads.append(thread1)
threads.append(thread2)
threads.append(thread3)
# Wait for all threads to complete
for t in threads:
t.join()
print "%s is finished"%t.name
print "Exiting Main Thread"
I would expect Thread 1 to first start and Thread 2 and Thread 3 be put on block until Thread 1 finishes. So the order of execution would be Thread 1 , Thread 2 and Thread 3. However it varies and is different each time I run the code. For example sometimes Thread-3 will run first which doesn't make sense as I call the line thread1.start() first and it should be locked after. Could someone give me some pointers to why this might be happening please?
Thread.start simply schedules a thread to start. It doesn't actually start it then and there. Rather, from that point on, the OS will take over and start the thread as it sees fit asynchronous to your main thread.
If you truly want to synchronize running of the threads, you'll need to do so yourself using a mutex or other synchronization primitive.
Related
This is a Producer Consumer Problem. I need a single producer and multiple consumers to access the shared data cell and each consumer needs to access the produced data before the producer makes additional data. The code works fine when there is a single consumer. I have attempted to make a list of the Producer and Consumers in order to .join() and .start() them. The program works so far as the first consumer, but hangs up when it gets to the second consumer. I have tried to change the locking mechanisms from "notify" to "notifyAll" in the getData and setData, I am a beginner in python and this stuff is pretty foreign to me but I have been trying stuff for 10 hours and would really appreciate some help.
import time, random
from threading import Thread, currentThread, Condition
class SharedCell(object):
def __init__(self):
self.data = -1
self.writeable = True
self.condition = Condition()
def setData(self, data):
self.condition.acquire()
while not self.writeable:
self.condition.wait()
print("%s setting data to %d" % \
(currentThread().getName(), data))
self.data = data
self.writeable = False
self.condition.notifyAll()
self.condition.release()
def getData(self):
self.condition.acquire()
while self.writeable:
self.condition.wait()
print(f'accessing data {currentThread().getName()} {self.data}')
self.writeable = True
self.condition.notifyAll()
self.condition.release()
return self.data
class Producer(Thread):
def __init__(self, cell, accessCount, sleepMax):
Thread.__init__(self, name = "Producer")
self.accessCount = accessCount
self.cell = cell
self.sleepMax = sleepMax
def run(self):
print("%s starting up" % self.getName())
for count in range(self.accessCount):
time.sleep(random.randint(1, self.sleepMax))
self.cell.setData(count + 1)
print("%s is done producing\n" % self.getName())
class Consumer(Thread):
def __init__(self, cell, accessCount, sleepMax):
Thread.__init__(self)
self.accessCount = accessCount
self.cell = cell
self.sleepMax = sleepMax
def run(self):
print("%s starting up" % self.getName())
for count in range(self.accessCount):
time.sleep(random.randint(1, self.sleepMax))
value = self.cell.getData()
print("%s is done consuming\n" % self.getName())
def main():
accessCount = int(input("Enter the number of accesses: "))
sleepMax = 4
cell = SharedCell()
producer = Producer(cell, accessCount, sleepMax)
consumer = Consumer(cell, accessCount, sleepMax)
consumerTwo = Consumer(cell, accessCount, sleepMax)
threads = []
threads.append(producer)
threads.append(consumer)
threads.append(consumerTwo)
print("Starting the threads")
for thread in threads:
thread.start()
thread.join()
main()
The join function blocks the current thread and waits until the indicated thread terminates. In your loop at the end of your main function, why do you join each thread immediately after starting it? That would result in starting thread 1, and then waiting for it to terminate before starting thread 2, and then waiting that it to terminate before starting thread 3, and so on.
Perhaps you meant something like this:
for thread in threads:
thread.start()
for thread in threads:
thread.join()
so that every thread is started before you wait for them to terminate.
I'm trying to create loop of threads and so far code is good. But I have problem when thread exits because of some exception.
Now I'm trying to figure out how to start additional thread after one thread exits because of exception. I did browse around but I didn't find any example that would work for this complex code. Any help would be great!
If thread stopped and queue is not empty restart stopped thread and continue with rest of the list.
This is my code:
some_list = [1,2,3,4,5,6,7,8]
exitFlag = 0
class threads():
#staticmethod
def process_data(threadName, q,queueLock):
workQueue = q
while not exitFlag:
queueLock.acquire()
if not workQueue.empty():
data = q.get()
queueLock.release()
print "%s processing %s" % (threadName, data)
else:
queueLock.release()
sleep(1)
def run_threads(self):
threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = some_list
queueLock = threading.Lock()
workQueue = Queue.Queue(1000000)
threads = []
threadID = 1
# Create new threads
for tName in threadList:
thread = myThread(threadID, tName, workQueue,queueLock)
thread.start()
threads.append(thread)
threadID += 1
# Fill the queue
queueLock.acquire()
for word in nameList:
workQueue.put(word)
queueLock.release()
# Wait for queue to empty
while not workQueue.empty():
pass
# Notify threads it's time to exit
global exitFlag
exitFlag = 1
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"
class myThread (threading.Thread,threads):
def __init__(self, threadID, name, q,queueLock):
self.thread = threading.Thread(target=self.run)
threading.Thread.__init__(self,target=self.run)
self.threadID = threadID
self.queueLock = queueLock
self.name = name
self.q = q
def run(self):
print "Starting " + self.name
threads.process_data(self.name, self.q,self.queueLock)
print "Exiting " + self.name
threads().run_threads()
Something like this should work:
...
# Wait for queue to empty
while not workQueue.empty():
for (i, t) in enumerate(threads):
if not t.is_alive():
print("Recreating thread " + t.name)
thread = myThread(threadID, threadList[i], workQueue,queueLock)
thread.start()
threads[i] = thread
threadID += 1
...
I would advice putting the thread-starting code into some method, as it will now be duplicated and hard to maintain.
The problem here is that you might "loose" the data that was popped from queue by the fatal thread.
I'm working with a toy multiprocessing problem, and event signalling is not working as expected. The multiprocessing documentation refers detail description of Event() to the multithreading documentation, and the description of the methods there are precisely what I'm trying to do. I want worker processes (subclassed from multiprocessing.Process) spawned by a parent class, then wait for a start signal from the parent class, do their thing, then terminate. What seems to be happening, however, is that the first process, once running, blocks any others. What's going on here, and how do I fix?
class Worker(Process):
def __init__(self, my_id, caller):
Process.__init__(self)
self.caller = caller
self.my_id = my_id
def run(self):
print("%i started"%self.my_id)
self.caller.start_flag.wait()
print("%i sleeping"%self.my_id)
sleep(2000)
class ParentProcess(object):
def __init__(self, num_procs):
self.procs = []
self.start_flag = Event()
for i in range(num_procs):
self.procs.append(Worker(i, self))
def run(self):
for proc in self.procs:
proc.run()
self.start_flag.set()
for proc in self.procs:
proc.join()
print("%i done"%proc.my_id)
if __name__ == '__main__':
cpus = cpu_count()
world = ParentProcess(cpus)
start = time()
world.run()
end = time()
runtime = end - start
print("Runtime: %3.6f"%runtime)
This is only outputting "0 started", then hanging. It seems the Event.wait() is blocking all other threads, even the caller. The documentation implies this should not happen.
He is a working version of the code. When you subclass process you implement the run method to define what should run in that process. When you actually want the process to start you should call the start method on it (proc.start()).
from multiprocessing import Process, Event
from time import time, sleep
class Worker(Process):
def __init__(self, my_id, caller):
Process.__init__(self)
self.caller = caller
self.my_id = my_id
def run(self):
print("%i started"%self.my_id)
self.caller.start_flag.wait()
print("%i sleeping"%self.my_id)
sleep(5)
class ParentProcess(object):
def __init__(self, num_procs):
self.procs = []
self.start_flag = Event()
for i in range(num_procs):
self.procs.append(Worker(i, self))
def run(self):
for proc in self.procs:
proc.start()
self.start_flag.set()
for proc in self.procs:
proc.join()
print("%i done"%proc.my_id)
if __name__ == '__main__':
cpus = 4
world = ParentProcess(cpus)
start = time()
world.run()
end = time()
runtime = end - start
print(runtime)
Outputs:
0 started
1 started
2 started
2 sleeping
0 sleeping
1 sleeping
3 started
3 sleeping
0 done
1 done
2 done
3 done
5.01037812233
Let's say I have the following python3 program:
from threading import Timer
import time
thread = Timer(600, print, args=['I am running'])
thread.start()
while threading.activeCount() > 1:
{{Function to calculate time until thread will start running}}
print ("Thread will start in %d seconds" % {get above value})
time.sleep(10)
What I'm looking at is a little more complex, with multiple threads, but essentially, for a given Timer thread, is there any way to inspect it to see when it is scheduled to run?
I'm not sure I'm getting what you say right, but you may want something like that:
from threading import Thread
import time
thread = Timer(600, print, args=['I am running'])
thread.start()
class ThreadTimeoutLauncher(Thread):
def __init__(self, timeout, cb, *args, **kwarg):
super(Thread, self).__init__()
self.timeout = timeout
self._cb = cb
self._args = args
self._kwarg = kwarg
def run():
print ("Thread will start in %d seconds" % self.timeout)
while self.timeout > 0:
time.sleep(1)
self.timeout -= 1
self.cb(*self._args, **self._kwarg)
the idea here, is to recreate a Timer thread that will count down until the time is out, and updates the "timeout value" while doing so. When it's over it launches the Thread event. So when you do:
def foo():
print "Thread launched!"
t = ThreadTimeoutLauncher(600, foo)
t.start()
while True:
time.sleep(0.5)
print "thread will be launched in: %d sec." % t.timeout
It may also be possible to inherit from Timer and change the run() method of Timer, but it'd mean to UTSL ;-)
Why doesn't this code "act" threaded? (Please see the output.)
import time
from threading import Thread
def main():
for nums in [range(0,5), range(5,10)]:
t = Spider(nums)
t.start()
print 'started a thread'
t.join()
print "done"
class Spider(Thread):
def __init__(self, nums):
Thread.__init__(self)
self.nums = nums
def run(self): # this is an override
for num in self.nums:
time.sleep(3) # or do something that takes a while
print 'finished %s' % (num, )
if __name__ == '__main__':
main()
Output:
started a thread
finished 0
finished 1
finished 2
finished 3
finished 4
started a thread
finished 5
finished 6
finished 7
finished 8
finished 9
done
When you say t.join(), you're telling it to wait for the thread to end.
This means, you're asking it to make a thread, start it, then wait for the thread to end before making a new one.
If you want it to act multithreaded, you'll need to move the join()s outside of the loop.
def main():
# We will store the running threads in this
threads = []
# Start the threads
for nums in [range(0,5), range(5,10)]:
t = Spider(nums)
t.start()
print 'started a thread'
threads.append(t)
# All the threads have been started
# Now we wait for them to finish
for t in threads:
t.join()
print "done"
See also:
Documentation of Thread.join()
Your Thread join t.join blocks the main thread until the thread completes execution ( http://docs.python.org/library/threading.html#threading.Thread.join ). Change your code to look something like this:
def main():
threads = []
for nums in [range(0,5), range(5,10)]:
t = Spider(nums)
t.start()
print 'started a thread'
threads.append(t)
for t in threads: t.join()
print "done"
You need to start both the threads first, and then join with them once they are both running.