Community!
I have a little Problem with the Python-DBus-API. I create a signal-receiver and it does its job. However if I try to remove the signal, it won't be removed and the signalhandler (sigHandler) is happily called every time the signal matches.
class A(threading.Thread)
bus = None
mainloop = None
systemBusMainLoop = None
signalReceiver = None
def __init__(self,dbusMainLoop):
log("Hello.")
super(A, self).__init__()
gobject.threads_init()
self.mainloop = gobject.MainLoop()
self.systemBusMainLoop = dbusMainLoop
self.bus = dbus.SystemBus(mainloop=dbusMainLoop)
self.signalReceiver = self.bus.add_signal_receiver(self.sigHandler,
bus_name="org.bluez",
dbus_interface="org.freedesktop.DBus.Properties",
signal_name="PropertiesChanged",
path_keyword="path")
def run(self):
self.mainloop.run()
log("Running.")
def end(self):
log("Shutting down...")
self.bus.remove_signal_receiver(self.sigHandler,
self.signalReceiver,
dbus_interface="org.freedesktop.DBus.Properties")
#self.signalReceiver.remove() #tried this also
if (self.mainloop):
self.mainloop.quit()
del self.signalReceiver
log("Bye.")
def sigHandler(self, interface, changed, invalidated, path)
print interface
print changed
print invalidated
print path
Called:
dbusA = A(dbusMainLoop=dbus.mainloop.glib.DBusGMainLoop())
dbusA.run()
#doing something unrelated
dbusA.end() #remove the Signal
del dbusA
Do i miss something? Why ist my sigHandler not removed (or why is my match not removed).
Thanks in advance!
The line you commented out (self.signalReceiver.remove()) works.
add_signal_receiver() returns a SignalMatch instance on which you can call remove() method to unregister your callback.
The problem in your code snippet is that dbusA.run() is a blocking call because it calls self.mainloop.run() which itself is a blocking call.
This said, program execution will never reach dbusA.end(), thus signal unregistering and quitting the run loop will not happen. You can however call this method from a different thread, or from any event handler of your dbus connection.
Related
I want to have a main program that works like a console from where I can call other processes (infinite loops) and kill them selectively whenever certain commands are entered.
For that I created this class:
class RunInThread(threading.Thread):
def __init__(self, function):
self.function = function
self.kill_pill = threading.Event()
threading.Thread.__init__(self)
def start(self): # This is controversial.
self.__init__(self.function)
threading.Thread.start(self)
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
The documentation for thread.Thread says that only the __init__() and run() methods should be overridden.
Is there any clear issue with my code? It works the way I intended but since it's going to be running for long periods of time I need to make sure I'm not creating any memory problems.
EDIT:
What about this solution?:
class StoppableThread(threading.Thread):
# threading.Thread class but can be stopped with the stop() method.
def __init__(self, function):
threading.Thread.__init__(self)
self.function = function
self.kill_pill = threading.Event()
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
class RunInThread():
def __init__(self, function, prnt=False):
self.function = function
self.running = False
self.prnt = prnt
def start(self):
if not self.running:
self.thread = StoppableThread(self.function)
self.thread.start()
self.running = True
else:
if self.prnt:
print('Thread already running.')
def stop(self):
self.thread.stop()
self.running = False
If you want to find out what things that could break, I'd suggest looking into the implementation of Thread class.
Among other things, Thread.__init__() initialises an Event() object to detect thread startup and shutdown, manages cleanup hooks/callbacks, some internal lock objects, and registers the thread to a list so you can introspect running threads. By calling Thread.__init__(), these variables gets reinitialised and screws up the internal mechanisms of many of these functionalities.
What could go wrong? I didn't test any of these, but from skimming through threading.py, these are likely some of the things that I expect could go wrong:
your python process now will be running a OS threads that doesn't show up in enumerate_thread()
multiple OS thread will now return the same Thread object when it calls current_thread(), which will likely also break threadlocal and anything that depends on threadlocal
Thread.join() depends on some internal locks, which likely would now become thread unsafe to call
Unhandled reception can go to the wrong exception hook handler
register_at_fork and shutdown handler likely will get confused
In other words, don't try to be sneaky. Create a new Thread object for each thread you want to start.
There's a good reason that the Thread class spent efforts trying to prevent you from accidentally calling start() twice. Don't try to subvert this.
I want to create a PyZMQ eventloop in a background thread, and have it work correctly with both standalone Python scripts and IPython scripts. (IPython uses PyZMQ eventloops located in the main thread, so this is causing me problems and why I want to start a private ioloop in a background thread.)
I want to run code in Thread A while having the PyZMQ eventloop handle received data from a socket in Thread B. There are times in Thread A where I will need to wait for an event set in Thread B.
How can I get this to work? There seems to be something wrong if I try in IPython:
from zmq.eventloop import ioloop
import threading
class IOBackgroundLoop(object):
def __init__(self):
self._loop = None
self._thread = threading.Thread(target=self.run)
self._thread.daemon = True
self._started = threading.Event()
#property
def loop(self):
return self._loop
def run(self):
self._loop = ioloop.IOLoop()
self._loop.initialize()
self._loop.make_current()
self._started.set()
self._loop.start()
def start(self):
self._thread.start()
self._started.wait()
bkloop = IOBackgroundLoop()
bkloop.start()
for loop in [bkloop.loop, ioloop.IOLoop.instance()]:
print "%s running: %s" % (loop, loop._running)
This prints out two separate instances of IOLoop, but if I go to use it, it doesn't seem to work. I can't think of a small example program to demonstrate this; I've tried with a timeout function:
import time
def print_timestamp(key):
print "%s: %s" % (time.time(), key)
for loop in [bkloop.loop, ioloop.IOLoop.instance()]:
loop.add_timeout(bkloop.loop.time() + 1.0, lambda: print_timestamp("hi from %s" % loop))
print_timestamp("here")
time.sleep(2.0)
print_timestamp("there")
and I get this as a result (no "hi":
1412889057.68: here
1412889059.68: there
1412889059.68: here
1412889061.68: there
Then when I hit another shift+Enter, I get
1412889061.68: hi from <zmq.eventloop.ioloop.ZMQIOLoop object at 0x000000000467E4E0>
which is the IOLoop object from the main thread, but my private instance IOLoop never prints hi.
What could I be doing wrong?
Argh, I just noticed this in the tornado docs:
Note that it is not safe to call add_timeout from other threads.
Instead, you must use add_callback to transfer control to the
IOLoop's thread, and then call add_timeout from there.
It also appears as though the zmq.eventloop.zmqstream needs to be setup in the same thread as the ioloop for it to work properly.
Quick question on the use of QThread in PyQt4 and Python 2.7. I am creating a process inherited from QObject, and assigning this to a Qthread I have created in a separate class (also inherited from QObject).
Is it safe to pass the QThread object to the process object, so that I can call thread.msleep(mseconds) from within the process itself?
I want to be able to make the thread wait or sleep, but I have read that time.sleep(seconds) is dodgy when used with PyQt multi-threading.
I did try to send a signal from the process object to a slot in the main thread (attached to thread.msleep(mseconds) for that process object), but I found that this failed to work; the process object continued executing until complete, with the slot only being executed after this time. Even after adjusting priorities, this continued to happen. This is unacceptable since I want the process loop to run continuously.
Any other recommendations?
I eventually managed to alter my code to achieve the functionality that I required in my question: namely the ability to make a thread wait or sleep for a specified amount of time.
Firstly, my research seems to show that one of the main reasons subclassing QThread became ill-advised in Qt was that a thread should not be able to manage itself. Though there is no official documentation on my question, I can only surmise that passing the thread object to the process object running on it would also be ill-advised, because the thread would again be able to control itself directly.
The solution I have found is to dispense with msleep() altogether. Qt documentation on QThread recommends that sleep() and wait() functions are avoided because they do not fit well with the event driven nature of Qt. They recommend that QTimer() is used to call a function via a signal after it times out, in place of msleep(). By default QTimer() is used to send a repeating signal every time interval, but can also send a signal once using QTimer.singleShot(). It is also stated in the documentation that it is safe to call QSleep() from within a thread.
I only use a repeating QTimer to call a single slot foo() multiple times, but to add a delay within foo(), QTimer.singleShot() could be used to call a second function moo() after a set number of milliseconds.
EDIT: I have decided to include my threading code, which subclasses QObject and QThread to perform a task on a thread in a continual loop every given time interval. It is, as far as I can tell, fully functional, though could do with a little work.
# -*- coding: utf-8 -*-
import sys
from PyQt4 import QtCore, QtGui
# Class to be assigned to a thread.
# This should be subclassed to provide new functionality.
class GenericLoop(QtCore.QObject):
def __init__(self):
super(GenericLoop, self).__init__()
# We use this signal to tell the main thread
# when this thread is finished.
finished_Sig = QtCore.pyqtSignal()
# Default timeout is 0, i.e. do work on thread after
# other events have been dealt with
__timeout = 0
__processTimer = None
__args = None
__kwargs = None
# We use this function to set the arguments used by run(),
# if we want to change them mid-execution
#QtCore.pyqtSlot(tuple, dict)
def changeArgs(self, args, kwargs):
self.__args = args
self.__kwargs = kwargs
# We can change the timeout used to make the thread run
# at given intervals. Note that the timing is not exact,
# since this is impossible with a real time operating system
#QtCore.pyqtSlot(int)
def setTimeout(self, mseconds):
self.__timeout = int(mseconds)
# Call either a singleShot QTimer (one execution),
# or a normal QTimer (repeated), to start the loop
#QtCore.pyqtSlot(bool, tuple, dict)
def startTimer(self, singleShot, args, kwargs):
self.__processTimer = QtCore.QTimer()
# We can't pass args and kwargs directly because QTimer.timeout
# emits a signal with allowing no contained variables
# so we copy args and kwargs to local variables instead
self.changeArgs(args, kwargs)
if singleShot:
self.__processTimer.singleShot(self.__timeout, self.callRun)
else:
self.__processTimer.timeout.connect(self.callRun)
self.__processTimer.start(self.__timeout)
# Call finish from within subclass using self.finish(), or
# from another thread using signals. finish() will stop the
# QTimer causing execution of the loop. The loop can be started again
# by calling startTimer() or stopTimer() from another thread
#QtCore.pyqtSlot()
def stopTimer(self):
if self.__processTimer.isActive():
self.__processTimer.stop()
else:
print "ERROR: stopTimer() has been called but no timer is running!"
# We call this to delete the thread.
#QtCore.pyqtSlot()
def deleteThread(self):
self.finished_Sig.emit()
# This calls run(), in order to enable the passing of
# command line arguments to the loop
#QtCore.pyqtSlot()
def callRun(self):
self.run(self.__args, self.__kwargs)
# run() can be called directly from another thread if required
#QtCore.pyqtSlot(tuple, dict)
def run(self, args, kwargs):
print "ERROR: run() has not been defined! Stopping thread..."
self.stopTimer()
# Class for creating threads
class GenericThread(QtCore.QObject):
# Private variables include the thread.
__sendArguments_Sig = QtCore.pyqtSignal(tuple, dict)
__startTimer_Sig = QtCore.pyqtSignal(int, tuple, dict)
__setTimeout_Sig = QtCore.pyqtSignal(int)
__obj = None
__finished_Sig = None
__thread = QtCore.QThread()
# Object to be threaded must be specified when
# creating a GenericThread object
def __init__(self, obj):
super(GenericThread, self).__init__()
self.__obj = obj
self.moreInit()
# Set up object on thread
def moreInit(self):
self.__thread = QtCore.QThread()
self.__obj.moveToThread(self.__thread)
# Allows thread to delete itself when done
self.__obj.finished_Sig.connect(self.__thread.deleteLater)
self.__sendArguments_Sig.connect(self.__obj.changeArgs)
self.__startTimer_Sig.connect(self.__obj.startTimer)
self.__setTimeout_Sig.connect(self.__obj.setTimeout)
self.__thread.start()
# Sets the QTimer timeout and does some checking
# to make sure that types are as they should be
def setTimeout(self, mseconds):
if mseconds >= 0 and type(mseconds) is type(int()):
self.__setTimeout_Sig.emit(mseconds)
elif mseconds < 0 and type(mseconds) is type(int()):
print "Error: timeout of below 0 ms specified."
else:
print "Error: timeout period is specified with a type other than int."
# Starts a function in the thread via signals, and can pass
# it arguments if required. Function executes until QTimer is stopped
def startLoop(self, *args, **kwargs):
if (self.__thread == None):
print "ERROR: Thread has been deleted!"
else:
self.__startTimer_Sig.emit(False, args, kwargs)
# Starts a function in the thread via signals, once
def startOnce(self, *args, **kwargs):
if (self.__thread == None):
print "ERROR: Thread has been deleted!"
else:
self.__startTimer_Sig.emit(True, args, kwargs)
# Calls a very simple GUI just to show that the program is responsive
class GUIBox(QtGui.QWidget):
def __init__(self):
super(GUIBox, self).__init__()
self.initUI()
def initUI(self):
self.resize(250, 150)
self.setWindowTitle('Threading!')
self.show()
# Subclass GenericLoop to reimplement run and such.
class SubClassedLoop(GenericLoop):
def __init__(self):
super(SubClassedLoop, self).__init__()
__i = 0
#QtCore.pyqtSlot(tuple, dict)
def run(self, args, kwargs):
if self.__i>=50:
self.stopTimer()
return
print self.__i, args
self.__i += 1
app = QtGui.QApplication(sys.argv)
ex = GUIBox()
# Create 3 worker objects to do the actual calculation
worker1 = SubClassedLoop()
worker2 = SubClassedLoop()
worker3 = SubClassedLoop()
# Create 3 thread managing objects to do the thread control
thread1 = GenericThread(worker1)
thread2 = GenericThread(worker2)
thread3 = GenericThread(worker3)
# Set the threads to execute as soon as there is no work to do
thread1.setTimeout(125)
thread2.setTimeout(125)
thread3.setTimeout(125)
# Start threads
thread1.startLoop(1)
thread2.startLoop(2)
thread3.startLoop(3)
# Quit the program when the GUI window is closed
sys.exit( app.exec_() )
I have python TCP client and need to send media(.mpg) file in a loop to a 'C' TCP server.
I have following code, where in separate thread I am reading the 10K blocks of file and sending it and doing it all over again in loop, I think it is because of my implementation of thread module, or tcp send. I am using Queues to print the logs on my GUI ( Tkinter ) but after some times it goes out of memory..
UPDATE 1 - Added more code as requested
Thread class "Sendmpgthread" used to create thread to send data
.
.
def __init__ ( self, otherparams,MainGUI):
.
.
self.MainGUI = MainGUI
self.lock = threading.Lock()
Thread.__init__(self)
#This is the one causing leak, this is called inside loop
def pushlog(self,msg):
self.MainGUI.queuelog.put(msg)
def send(self, mysocket, block):
size = len(block)
pos = 0;
while size > 0:
try:
curpos = mysocket.send(block[pos:])
except socket.timeout, msg:
if self.over:
self.pushlog(Exit Send)
return False
except socket.error, msg:
print 'Exception'
return False
pos = pos + curpos
size = size - curpos
return True
def run(self):
media_file = None
mysocket = None
try:
mysocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysocket.connect((self.ip, string.atoi(self.port)))
media_file = open(self.file, 'rb')
while not self.over:
chunk = media_file.read(10000)
if not chunk: # EOF Reset it
print 'resetting stream'
media_file.seek(0, 0)
continue
if not self.send(mysocket, chunk): # If some error or thread is killed
break;
#disabling this solves the issue
self.pushlog('print how much data sent')
except socket.error, msg:
print 'print exception'
except Exception, msg:
print 'print exception'
try:
if media_file is not None:
media_file.close()
media_file = None
if mysocket is not None:
mysocket.close()
mysocket = None
finally:
print 'some cleaning'
def kill(self):
self.over = True
I figured out that it is because of wrong implementation of Queue as commenting that piece resolves the issue
UPDATE 2 - MainGUI class which is called from above Thread class
class MainGUI(Frame):
def __init__(self, other args):
#some code
.
.
#from the above thread class used to send data
self.send_mpg_status = Sendmpgthread(params)
self.send_mpg_status.start()
self.after(100, self.updatelog)
self.queuelog = Queue.Queue()
def updatelog(self):
try:
msg = self.queuelog.get_nowait()
while msg is not None:
self.printlog(msg)
msg = self.queuelog.get_nowait()
except Queue.Empty:
pass
if self.send_mpg_status: # only continue when sending
self.after(100, self.updatelog)
def printlog(self,msg):
#print in GUI
Since printlog is adding to a tkinter text control, the memory occupied by that control will grow with each message (it has to store all the log messages in order to display them).
Unless storing all the logs is critical, a common solution is to limit the maximum number of log lines displayed.
A naive implementation is to eliminate extra lines from the begining after the control reaches a maximum number of messages. Add a function to get the number of lines in the control and then, in printlog something similar to:
while getnumlines(self.edit) > self.maxloglines:
self.edit.delete('1.0', '1.end')
(above code not tested)
update: some general guidelines
Keep in mind that what might look like a memory leak does not always mean that a function is wrong, or that the memory is no longer accessible. Many times there is missing cleanup code for a container that is accumulating elements.
A basic general approach for this kind of problems:
form an opinion on what part of the code might be causing the problem
check it by commenting that code out (or keep commenting code until you find a candidate)
look for containers in the responsible code, add code to print their size
decide what elements can be safely removed from that container, and when to do it
test the result
I can't see anything obviously wrong with your code snippet.
To reduce memory usage a bit under Python 2.7, I'd use buffer(block, pos) instead of block[pos:]. Also I'd use mysocket.sendall(block) instead of your send method.
If the ideas above don't solve your problem, then the bug is most probably elsewhere in your code. Could you please post the shortest possible version of the full Python script which still grows out-of-memory (http://sscce.org/)? That increases your change of getting useful help.
Out of memory errors are indicative of data being generated but not consumed or released. Looking through your code I would guess these two areas:
Messages are being pushed onto a Queue.Queue() instance in the pushlog method. Are they being consumed?
The MainGui printlog method may be writing text somewhere. eg. Is it continually writing to some kind of GUI widget without any pruning of messages?
From the code you've posted, here's what I would try:
Put a print statement in updatelog. If this is not being continually called for some reason such as a failed after() call, then the queuelog will continue to grow without bound.
If updatelog is continually being called, then turn your focus to printlog. Comment the contents of this function to see if out of memory errors still occur. If they don't, then something in printlog may be holding on to the logged data, you'll need to dig deeper to find out what.
Apart from this, the code could be cleaned up a bit. self.queuelog is not created until after the thread is started which gives rise to a race condition where the thread may try to write into the queue before it has been created. Creation of queuelog should be moved to somewhere before the thread is started.
updatelog could also be refactored to remove redundancy:
def updatelog(self):
try:
while True:
msg = self.queuelog.get_nowait()
self.printlog(msg)
except Queue.Empty:
pass
And I assume the the kill function is called from the GUI thread. To avoid thread race conditions, the self.over should be a thread safe variable such as a threading.Event object.
def __init__(...):
self.over = threading.Event()
def kill(self):
self.over.set()
There is no data piling up in your TCP sending loop.
Memory error is probably caused by logging queue, as you have not posted complete code try using following class for logging:
from threading import Thread, Event, Lock
from time import sleep, time as now
class LogRecord(object):
__slots__ = ["txt", "params"]
def __init__(self, txt, params):
self.txt, self.params = txt, params
class AsyncLog(Thread):
DEBUGGING_EMULATE_SLOW_IO = True
def __init__(self, queue_max_size=15, queue_min_size=5):
Thread.__init__(self)
self.queue_max_size, self.queue_min_size = queue_max_size, queue_min_size
self._queuelock = Lock()
self._queue = [] # protected by _queuelock
self._discarded_count = 0 # protected by _queuelock
self._pushed_event = Event()
self.setDaemon(True)
self.start()
def log(self, message, **params):
with self._queuelock:
self._queue.append(LogRecord(message, params))
if len(self._queue) > self.queue_max_size:
# empty the queue:
self._discarded_count += len(self._queue) - self.queue_min_size
del self._queue[self.queue_min_size:] # empty the queue instead of creating new list (= [])
self._pushed_event.set()
def run(self):
while 1: # no reason for exit condition here
logs, discarded_count = None, 0
with self._queuelock:
if len(self._queue) > 0:
# select buffered messages for printing, releasing lock ASAP
logs = self._queue[:]
del self._queue[:]
self._pushed_event.clear()
discarded_count = self._discarded_count
self._discarded_count = 0
if not logs:
self._pushed_event.wait()
self._pushed_event.clear()
continue
else:
# print logs
if discarded_count:
print ".. {0} log records missing ..".format(discarded_count)
for log_record in logs:
self.write_line(log_record)
if self.DEBUGGING_EMULATE_SLOW_IO:
sleep(0.5)
def write_line(self, log_record):
print log_record.txt, " ".join(["{0}={1}".format(name, value) for name, value in log_record.params.items()])
if __name__ == "__main__":
class MainGUI:
def __init__(self):
self._async_log = AsyncLog()
self.log = self._async_log.log # stored as bound method
def do_this_test(self):
print "I am about to log 100 times per sec, while text output frequency is 2Hz (twice per second)"
def log_100_records_in_one_second(itteration_index):
for i in xrange(100):
self.log("something happened", timestamp=now(), session=3.1415, itteration=itteration_index)
sleep(0.01)
for iter_index in range(3):
log_100_records_in_one_second(iter_index)
test = MainGUI()
test.do_this_test()
I have noticed that you do not sleep() anywhere in the sending loop, this means data is read as fast as it can and is sent as fast as it can. Note that this is not desirable behavior when playing media files - container time-stamps are there to dictate data-rate.
I have a threaded python application with a long-running mainloop in the background thread. This background mainloop is actually a call to pyglet.app.run(), which drives a GUI window and also can be configured to call other code periodically. I need a do_stuff(duration) function to be called at will from the main thread to trigger an animation in the GUI, wait for the animation to stop, and then return. The actual animation must be done in the background thread because the GUI library can't handle being driven by separate threads.
I believe I need to do something like this:
import threading
class StuffDoer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.max_n_times = 0
self.total_n_times = 0
self.paused_ev = threading.Event()
def run(self):
# this part is outside of my control
while True:
self._do_stuff()
# do other stuff
def _do_stuff(self):
# this part is under my control
if self.paused_ev.is_set():
if self.max_n_times > self.total_n_times:
self.paused_ev.clear()
else:
if self.total_n_times >= self.max_n_times:
self.paused_ev.set()
if not self.paused_ev.is_set():
# do stuff that must execute in the background thread
self.total_n_times += 1
sd = StuffDoer()
sd.start()
def do_stuff(n_times):
sd.max_n_times += n_times
sd.paused_ev.wait_for_clear() # wait_for_clear() does not exist
sd.paused_ev.wait()
assert (sd.total_n_times == sd.max_n_times)
EDIT: use max_n_times instead of stop_time to clarify why Thread.join(duration) won't do the trick.
From the documentation for threading.Event:
wait([timeout])
Block until the internal flag is true.
If the internal flag is true on entry,
return immediately. Otherwise, block
until another thread calls set() to
set the flag to true, or until the
optional timeout occurs.
I've found I can get the behavior I'm looking for if I have a pair of events, paused_ev and not_paused_ev, and use not_paused_ev.wait(). I could almost just use Thread.join(duration), except it needs to only return precisely when the background thread actually registers that the time is up. Is there some other synchronization object or other strategy I should be using instead?
I'd also be open to arguments that I'm approaching this whole thing the wrong way, provided they're good arguments.
Hoping I get some revision or additional info from my comment, but I'm kind of wondering if you're not overworking things by subclassing Thread. You can do things like this:
class MyWorker(object):
def __init__(self):
t = Thread(target = self._do_work, name "Worker Owned Thread")
t.daemon = True
t.start()
def _do_work(self):
While True:
# Something going on here, forever if necessary. This thread
# will go away if the other non-daemon threads terminate, possibly
# raising an exception depending this function's body.
I find this makes more sense when the method you want to run is something that is more appropriately a member function of some other class than it would be to as the run method on the thread. Additionally, this saves you from having to encapsulate a bunch of business logic inside of a Thread. All IMO, of course.
It appears that your GUI animation thread is using a spin-lock in its while True loop. This can be prevented using thread-safe queues. Based on my reading of your question, this approach would be functionally equivalent and efficient.
I'm omitting some details of your code above which would not change. I'm also assuming here that the run() method which you do not control uses the self.stop_time value to do its work; otherwise there is no need for a threadsafe queue.
from Queue import Queue
from threading import Event
class StuffDoer:
def __init__(self, inq, ready):
self.inq = inq
self.ready = ready
def _do_stuff(self):
self.ready.set()
self.stop_time = self.inq.get()
GUIqueue = Queue()
control = Event()
sd = StuffDoer(GUIqueue, control)
def do_stuff(duration):
control.clear()
GUIqueue.put(time.time() + duration)
control.wait()
I ended up using a Queue similar to what #wberry suggested, and making use of Queue.task_done and Queue.wait:
import Queue
import threading
class StuffDoer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.setDaemon(True)
self.max_n_times = 0
self.total_n_times = 0
self.do_queue = Queue.Queue()
def run(self):
# this part is outside of my control
while True:
self._do_stuff()
# do other stuff
def _do_stuff(self):
# this part is under my control
if self.total_n_times >= self.max_n_times:
try:
self.max_n_times += self.do_queue.get(block=False)
except Queue.Empty, e:
pass
if self.max_n_times > self.total_n_times:
# do stuff that must execute in the background thread
self.total_n_times += 1
if self.total_n_times >= self.max_n_times:
self.do_queue.task_done()
sd = StuffDoer()
sd.start()
def do_stuff(n_times):
sd.do_queue.put(n_times)
sd.do_queue.join()
assert (sd.total_n_times == sd.max_n_times)
I made solution based on #g.d.d.c advice for this question. There is my code:
threads = []
# initializing aux thread(s) in the main thread ...
t = threading.Thread(target=ThreadF, args=(...))
#t.setDaemon(True) # I'm not sure does it really needed
t.start()
threads.append(t.ident)
# Block main thread
while filter(lambda thread: thread.ident in threads, threading.enumerate()):
time.sleep(10)
Also, you can use Thread.join to block the main thread - it is better way.