Develop Reference

Python lock class attribute while main thread is executing

My main script launches an API and another thread executes a daily task to update data.
Currently, the daily update can change data during an API call and it will cause an error.
I think my problem can be summarized as the following code :
import schedule
from threading import Thread, Lock
import time
def periodically_run_pending_tasks(interval=0.5):
while True:
schedule.run_pending()
time.sleep(interval)
def change_state(item):
item.a = "New state"
class MyClass:
def __init__(self):
self.a = "Initial state"
def print_state_twice(self, lk):
lk.acquire()
print(self.a)
time.sleep(2)
print(self.a)
lk.release()
my_class = MyClass()
lock = Lock()
thread = Thread(target=periodically_run_pending_tasks)
schedule.every().second.do(change_state, my_class)
thread.start()
my_class.print_state_twice(lock)
I want this script to output either twice Initial state or twice New state but it returns Initial State New state.
I've also tried :
import schedule
from threading import Thread, Lock
import time
def periodically_run_pending_tasks(interval=0.5):
while True:
schedule.run_pending()
time.sleep(interval)
def change_state(item):
item.a = "New state"
class MyClass:
def __init__(self):
self.a = "Initial state"
def print_state_twice(self, lk):
lk.acquire()
print(self.a)
time.sleep(2)
print(self.a)
lk.release()
my_class = MyClass()
lock = Lock()
thread_2 = Thread(target=my_class.print_state_twice, args=(lock,))
thread_2.start()
thread = Thread(target=periodically_run_pending_tasks)
schedule.every().second.do(change_state, my_class)
thread.start()

Multi threading in python using parallel threads

I created two threads each running different functions.
What i tryed to achieve is if first thread ends then the second should also end ( i tryed achieving it using global variable)
Once both the threads end the same procedure should continue.
The script is not working as expected.
I am using Linux - Centos and python 2.7
#!/usr/bin/python
import threading
import time
import subprocess
import datetime
import os
import thread
command= "strace -o /root/Desktop/a.txt -c ./server"
final_dir = "/root/Desktop/"
exitflag = 0
# Define a function for the thread
def print_time(*args):
os.chdir(final_dir)
print "IN first thread"
proc = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
proc.wait(70)
exitflag=1
def print_time1(*args):
print "In second thread"
global exitflag
while exitflag:
thread.exit()
#proc = subprocess.Popen(command1,shell=True,stdout=subprocess.PIPE, sterr=subprocess.PIPE)
# Create two threads as follows
while (1):
t1=threading.Thread(target=print_time)
t1.start()
t2=threading.Thread(target=print_time1)
t2=start()
time.sleep(80)
z = t1.isAlive()
z1 = t2.isAlive()
if z:
z.exit()
if z1:
z1.exit()
threading.Thread(target=print_time1).start()
threading.Thread(target=print_time1).start()
print "In try"
Where am i going wrong?

You could create an object to share state, and have the dependent thread check that state. Something like:
import threading
import time
import datetime
class Worker1( threading.Thread ):
def __init__(self, state):
super(Worker1, self).__init__()
self.state = state
def run(self):
print_time_helper("Worker1 Start")
time.sleep(4)
print_time_helper("Worker1 End")
self.state.keepOnRunning = False
class Worker2( threading.Thread ):
def __init__(self, state):
super(Worker2, self).__init__()
self.state = state
def run(self):
while self.state.keepOnRunning:
print_time_helper("Worker2")
time.sleep(1)
class State( object ):
def __init__(self):
self.keepOnRunning = True
def main():
state = State()
thread1 = Worker1(state)
thread2 = Worker2(state)
thread1.start()
thread2.start()
thread1.join()
thread2.join()
def print_time_helper(name):
print "{0}: {1}".format(name, datetime.datetime.now().time().strftime("%S"))
which will output something like this (numbers show current time seconds):
Worker1 Start: 39
Worker2: 39
Worker2: 40
Worker2: 41
Worker2: 42
Worker1 End: 43
However, this is a bit simplistic for most situations. You might be better off using message queues - this is a good intro.

Use a threading.Event instead of an int and wait for it to be set.
Also your logic appears to be wrong in print_time1 because your while loop will never run since exitflag is initially 0, but even if it was 1 it would still just exit immediately. It's not actually waiting on anything.

How to return value from function running by QThread and Queue

Please explain how do we send/receive data from Thread managed by Queue....
First I subclass 'QThread' defining its run() method which is started when QThread's.start() is called:
class SimpleThread(QtCore.QThread):
def __init__(self, queue, parent=None):
QtCore.QThread.__init__(self, parent)
self.queue=queue
def run(self):
while True:
arg=self.queue.get()
self.fun(arg)
self.queue.task_done()
def fun(self, arg):
for i in range (3):
print 'fun: %s'%i
self.sleep(1)
return arg+1
Then I declare two Thread instances (so only two CPU cores are taken) sending self.queue instance as an argument.
self.queue=queue.Queue()
for i in range(2):
thread=SimpleThread(self.queue)
thread.start()
Now if I understand it correctly thread.start() is not starting anything. The real "start" happens only when I call queue.put():
for arg in [1,2,3]: self.queue.put(arg)
This last line is what makes a "real" call. Aside from creating and starting Queue item put() allows to save any arbitrary value to each Queue item. .put() does several things at once: it creates, it starts, it moves the processing through the Queue and it allows to place a variable "inside" of the queue item (which later can be retrieved from inside of the function-processor: using Queue item's '.get()` method).
But how do I return the value from fun() function. A "regular" fun()'s return resultValue doesn't work. And I can't use self.queue.put() method since this method aside from storing a data "creates" a new queue item...
EDITED LATER:
Here is slightly tweaked code (copy/pasted from another post) showing an approach on how to return a value from completed Thread. I am not sure if the the approach used here would work with QThread... please correct me if I am wrong:
import os, sys
import threading
import Queue
def callMe(incomingFun, daemon=False):
def execute(_queue, *args, **kwargs):
result=incomingFun(*args, **kwargs)
_queue.put(result)
def wrap(*args, **kwargs):
_queue=Queue.Queue()
_thread=threading.Thread(target=execute, args=(_queue,)+args, kwargs=kwargs)
_thread.daemon=daemon
_thread.start()
_thread.result_queue=_queue
return _thread
return wrap
#callMe
def localFunc(x):
import time
x = x + 5
time.sleep(5)
return x
thread=localFunc(10)
# this blocks, waiting for the result
result = thread.result_queue.get()
print result

In normal circumstances you'd use a result queue to send results back, and then have some other thread running that waits for the results:
class SimpleThread(QtCore.QThread):
def __init__(self, queue, result_queue, parent=None):
QtCore.QThread.__init__(self, parent)
self.queue=queue
self.result_queue = result_queue
def run(self):
while True:
arg=self.queue.get()
self.fun(arg)
self.queue.task_done()
def fun(self, arg):
for i in range (3):
print 'fun: %s'%i
self.sleep(1)
self.result_queue.put(arg+1)
def handle_results(result_queue):
while True:
result = result_queue.get()
print("Got result {}".format(result))
Main thread:
self.queue=queue.Queue()
self.result_queue = queue.Queue()
result_handler = threading.Thread(target=handle_results, self.result_queue)
for i in range(2):
thread=SimpleThread(self.queue, self.result_queue)
thread.start()
Doing it this way will keep you from blocking the GUI's event loop while you wait for the results. Here's what the equivalent would look like with multiprocessing.pool.ThreadPool:
from multiprocessing.pool import ThreadPool
import time
def fun(arg):
for i in range (3):
print 'fun: %s'%i
time.sleep(1)
return arg+1
def handle_result(result):
print("got result {}".format(result))
pool = ThreadPool(2)
pool.map_async(fun, [1,2,3], callback=handle_result)
Which is a lot simpler. It internally creates a result handling thread, which will automatically call handle_result for you when fun completes.
That said, you're using QThread, and you want the results to update GUI widgets, so you really want your results to be sent back to the main thread, not to a result handling thread. In that case, it makes sense to use Qt's signaling system, so that you can safely update the GUI when you receive the result:
from PyQt4 import QtCore, QtGui
import sys
import Queue as queue
class ResultObj(QtCore.QObject):
def __init__(self, val):
self.val = val
class SimpleThread(QtCore.QThread):
finished = QtCore.pyqtSignal(object)
def __init__(self, queue, callback, parent=None):
QtCore.QThread.__init__(self, parent)
self.queue = queue
self.finished.connect(callback)
def run(self):
while True:
arg = self.queue.get()
if arg is None: # None means exit
print("Shutting down")
return
self.fun(arg)
def fun(self, arg):
for i in range(3):
print 'fun: %s' % i
self.sleep(1)
self.finished.emit(ResultObj(arg+1))
class AppWindow(QtGui.QMainWindow):
def __init__(self):
super(AppWindow, self).__init__()
mainWidget = QtGui.QWidget()
self.setCentralWidget(mainWidget)
mainLayout = QtGui.QVBoxLayout()
mainWidget.setLayout(mainLayout)
button = QtGui.QPushButton('Process')
button.clicked.connect(self.process)
mainLayout.addWidget(button)
def handle_result(self, result):
val = result.val
print("got val {}".format(val))
# You can update the UI from here.
def process(self):
MAX_CORES=2
self.queue = queue.Queue()
self.threads = []
for i in range(MAX_CORES):
thread = SimpleThread(self.queue, self.handle_result)
self.threads.append(thread)
thread.start()
for arg in [1,2,3]:
self.queue.put(arg)
for _ in range(MAX_CORES): # Tell the workers to shut down
self.queue.put(None)
app = QtGui.QApplication([])
window = AppWindow()
window.show()
sys.exit(app.exec_())
Output when the button is pushed:
fun: 0
fun: 0
fun: 1
fun: 1
fun: 2
fun: 2
fun: 0
got val 2
got val 3
Shutting down
fun: 1
fun: 2
Shutting down
got val 4

How to make this code work well with Queue?

I have a little doubt if one could solve my issue, and create successful communication between threads.
First example and this is how it should be working, but does not work well:
import Queue,threading,time
class th(threading.Thread):
def __init__(self,q):
threading.Thread.__init__(self)
self.q = q
self.t = time
def run(self):
for i in range(5):
self.q.put(i)
self.t.sleep(0.5) # <----------
self.q.put('end')
class main(object):
def __init__(self):
self.q = Queue.Queue()
self.thread = th(self.q)
self.thread.setDaemon(True)
self.thread.run()
self.call()
def call(self):
while True:
recv = self.q.get();
if recv == 'end':
break
else:
print recv
if __name__ == '__main__':
root = main()
root.call()
In this example, all printed at the same time:
0,1,2,3,4
Second example:
import Queue,threading,time
class th(threading.Thread):
def __init__(self,q):
threading.Thread.__init__(self);
self.q = q
self.t = time
def run(self):
for i in range(5):
self.q.put(i) # <------ no sleep()
self.q.put('end')
class main(object):
def __init__(self):
self.q = Queue.Queue()
self.thread = th(self.q)
self.thread.setDaemon(True)
self.thread.run()
self.call()
def call(self):
while True:
recv = self.q.get()
if recv == 'end':
break
else:
print recv
if __name__ == '__main__':
root = main()
root.call()
the code is printed as it has to
0,
1
2
3
4
one to one
is there any way that the sleep function in the same way?

You don't want to call the run method on a thread directly. Call start instead, which will kick off the child thread, which will in turn run the run method.
Your current code is essentially single threaded, since the run call does the work of the child thread in the parent instead. The child thread is never actually started! (You're also calling your main.call method twice, which I'd expect to block or raise an exception, but that's a separate issue.)

sorry, it was something very simple, really simple, just had to replace
self.thread.run()
by
self.threat.start()

Python Thread Update Arg

Searched for this but could not find anything, so I suspect it can't be done.
I need to update arguments to a running thread:
def doLeds(*leds):
onoff = 0
while 1:
onoff=not onoff
for led in leds:
wpi.digitalWrite(led,onoff)
time.sleep(0.025)
def thread():
from threading import Thread
leds = Thread(target=doLeds, args=([17,22,10,9,11]))
leds.start()
print(leds)
time.sleep(5)
leds['args']=[10,9,11]
Is it possible to update thread vars / arguments after the thread has been started ?

Sure. It's easy if you subclass Thread. Then, you can directly assign to its attributes.
class LedThread(Thread):
def __init__(self, args):
Thread.__init__(self)
self.args = args
def run(self):
self.doLeds()
def doLeds(self):
onoff = 0
while 1:
onoff=not onoff
for led in self.args:
wpi.digitalWrite(led,onoff)
time.sleep(0.025)
t = LedThread([17,22,10,9,11])
t.start()
time.sleep(10)
t.args = [10,9,11]
time.sleep(10)

If you use the basic threads with a callable, you can't change the arguments. That's because after the thread starts, it passes the arguments to your target, then forgets about them.
What you can do instead is create a class that extends threading.Thread. Then, you can have a field with what you used to pass as an argument. For example:
from threading import Thread
class LedThread(Thread):
def __init__(self, leds):
Thread.__init__(self)
self.leds = leds
def run(self):
onoff = 0
while 1:
onoff=not onoff
for led in leds:
wpi.digitalWrite(led,onoff)
time.sleep(0.025)
def thread():
leds = LedThread([17,22,10,9,11])
leds.start()
print(leds)
time.sleep(5)
leds.leds=[10,9,11]