Here's the code I have created to load animations in background using pyglet.resource.animation() function, while the app does some other things.
import pyglet
import threading
animations = dict()
class LoadingThread(object):
def __init__(self):
thread = threading.Thread(target=self.run, args=())
thread.start() # Start the execution
def run(self):
""" Method that runs forever """
loadAnimations()
print("Loaded all animations.")
def loadAnimations():
global animations
print("In loadAnimations")
for animation in os.listdir(os.getcwd()):
if animation.endswith(".gif"):
print(animation)
#Gives segmentation fault here
animations[animation] = pyglet.resource.animation(animation)
print("Loaded animations")
thread = LoadingThread()
Runs well when called normally without a thread.
If there is any other way to deal with loading animations in background in pyglet, please suggest.
Thanks.
As suggested by #Frank Merrow. The problem here was that I was using the function
pyglet.resource.animation("filename.gif") function in my main thread too. So it was creating a segmentation fault.
I found another function that can also load animation.
pyglet.image.load_animation("filename.gif")
Using this solved my problem.
Also this problem can be solved by starting two threads synchronously running main flow and the background work.
Related
I have two python programs which one of them connects to a bluetooth device(socket package), it receives and saves data from device, and another one read the stored data and draw a real time plot. I should make one application from these two programs.
I tried to mix these two python programs, but since bluetooth should wait to receive data (through a while loop), the other parts of program does not work. I tried to solve this problem using Clock.schedule_interval, but the program will hang after a period of time. So I decided to run these two programs simultaneously. I read, we can run some python programs at a same time using a python script. Is there any trick to join these two programs and build one application?
Any help would be greatly appreciated.
Install threaded:
pip install threaded
Create a new python file:
from threading import Thread
def runFile1(): import file1
def runFile2(): import file2
Thread(target=runFile1).start()
runFile2()
Run the new python file.
It can be done with threading. To do communication between the threaded function and your main function, use objects such as queue.Queue and threading.Event.
the bluetooth functions can be placed into a function that is the target of the thread
import time
from threading import Thread
from queue import Queue
class BlueToothFunctions(Thread):
def __init__(self, my_queue):
super().__init__()
self.my_queue = my_queue
# optional: causes this thread to end immediately if the main program is terminated
self.daemon = True
def run(self) -> None:
while True:
# do all the bluetooth stuff foreverer
g = self.my_queue.get()
if g == (None, None):
break
print(g)
time.sleep(1.0)
print("bluetooth closed")
if __name__ == '__main__':
_queue = Queue() # just one way to communicate to a thread
# pass an object reference so both main and thread have a way to communicate on this common queue
my_bluetooth = BlueToothFunctions(_queue)
my_bluetooth.start() # creates the thread and executes run() method
for i in range(5):
# communicate to the threaded functions
_queue.put(i)
_queue.put((None, None)) # optional, a way to cause the thread to end
my_bluetooth.join(timeout=5.0) # optional, pause here until thread ends
print('program complete')
I'm currently using a daemon thread in Python to constantly detect parts of the screen in the background while other more important functions are running. testImToStr is in the same file as the rest of the code I am using.
def testImToStr():
while True:
pospro.imagetoString();
doImageToString = threading.Thread(target=testImToStr, args=(), daemon=True)
doImageToString.start()
while True:
#other stuff i was too lazy to copy over
This version is working as it does both the image processing and the other stuff in the while loop.
However, then the target thread is in another module:
doImageToString = threading.Thread(target=pospro.loopedImToStr, args=(), daemon=True)
doImageToString.start()
while True:
#other stuff i was too lazy to copy over
Other module:
def loopedImToStr():
while True:
imagetoString()
def imagetoString():
#stuff here
It only loops the target thread and does not run the while loop in the file that originally made the thread. How is it that both loops are run when the thread is in the same file as the loop but only the thread is run when it they are in different files?
I think all your problem makes the most common mistake - target has to be function's name without () - so called callback
Thread(target=pospro.loopedImToStr, daemon=True)
and later Thread.start() will use () to run it.
In your code you run testImToStr() at once like
result = testImToStr()
doImageToString = threading.Thread(target=result, ...)
so testImToStr() blocks all code and it can't run other loop.
I have functioning code that displays data in a GUI which is periodically updated with new information downloaded from the web. (The base code for the threaded approach was sourced from https://www.oreilly.com/library/view/python-cookbook/0596001673/ch09s07.html) I am using a threaded solution so as to improve blocking IO issues (IO code not included in the simplified code example below, as the IO does not appear to be the problem). The code runs fine if I run it as a single instance. However, it would be most convenient if I could use multiprocessing to run several instances of the code in parallel, using a different input list for each instance. When I try to implement the multiprocessing version, each separate process hangs during the attempt to create the root window: "window = tk.Tk()". Here is the working single instance version:
import threading
import random
import tkinter as tk
import random
import queue #Queue
import multiprocessing
import psutil
class GuiPartBase:
def __init__(self, master, queue, myList, endCommand):
self.queue = queue
# Set up the GUI
a = Label(master, text="Test Tkinter Display!")
a.pack()
## etc
def processIncoming(self):
"""Handle all messages currently in the queue, if any."""
while self.queue.qsize():
try:
result = (self.queue.get(0))
## do stuff with incoming data...
print('result =', result)
except queue.Empty:
# just on general principles...
pass
class ThreadedClientBase:
"""
Launch the main part of the GUI and the worker thread. periodicCall and
endApplication could reside in the GUI part, but putting them here
means that you have all the thread controls in a single place.
"""
def __init__(self, master, mylist):
"""
Start the GUI and the asynchronous threads. We are in the main
(original) thread of the application, which will later be used by
the GUI as well. We spawn a new thread for the worker (I/O).
"""
self.master = master
self.mylist = mylist
# Create the queue
self.queue = queue.Queue()
# Set up the GUI part
self.gui = GuiPartBase(self.master, self.queue, mylist, self.endApplication)
# Set up the thread to do asynchronous I/O
# More threads can also be created and used, if necessary
self.running = 1
self.thread1 = threading.Thread(target=self.workerThread1)
self.thread1.start()
# Start the periodic call in the GUI to check if the queue contains
# anything
self.periodicCall()
def periodicCall(self):
"""
Check every 200 ms if there is something new in the queue.
"""
self.gui.processIncoming()
if not self.running:
# This is the brutal stop of the system. You may want to do
# some cleanup before actually shutting it down.
import sys
sys.exit(1)
self.master.after(200, self.periodicCall)
def workerThread1(self):
"""
This is where we handle the asynchronous I/O. For example, it may be
a 'select( )'. One important thing to remember is that the thread has
to yield control pretty regularly, by select or otherwise.
"""
while self.running:
# simulate asynchronous I/O,
time.sleep(rand.random() * 1.5)
msg = rand.random()
self.queue.put(msg)
def endApplication(self):
self.running = 0
def runGUIthread(threadedList2Get):
print('entering runGUIthread...')
print('list2Get = ', threadedList2Get)
window = tk.Tk()
print('type of window = ', type(window))
print('window = ', window)
client = ThreadedClientBase(window, threadedList2Get)
print('type of client = ', type(client))
print('client = ', client)
window.mainloop()
if __name__ == '__main__':
rand = random.Random()
testList2a = ['abc','def','geh']
testList2b = ['xyz', 'lmn', 'opq']
allLists = [testList2a,testList2b]
runGUIthread(testList2a)
So, like I said, the above works - a single tkinter GUI is displayed appropriately without errors. However, if I attempt to implement multiprocessing with the following code below, the code spawns two processes as expected, and as documented by the printout of pid. However, each process prints the 'list2Get' (in runGUIthread), and then there is nothing else. There is no error message and the python code seems to have exited as there is no persistent process listed in the system activity monitor. Presumably the code is "hanging"/ exiting at the line "window = tk.TK()", as the line "print('type of window=',type(window))" is never executed:
if __name__ == '__main__':
rand = random.Random()
testList2a = ['abc','def','geh']
testList2b = ['xyz', 'lmn', 'opq']
allLists = [testList2a,testList2b]
#runGUIthread(testList2a)
for list in allLists:
p = multiprocessing.Process(target=runGUIthread, args=(list,))
p.start()
ps = psutil.Process(p.pid)
print('pid = ', ps)
#with multiprocessing.Pool(processes=2) as pool:
# pool.map(runGUIthread, allLists)
I am not experienced with multiprocessing, so perhaps I have implemented it incorrectly. I tried using multiprocessing.Pool(), with the same results.
I have not been able to find info indicating that tkinter can't spawn multiple GUI displays in the same program. In fact I found an instance of somebody accidentally spawning multiple GUI's, although this appears to be with Python 3.8 using concurrent.futures.ProcessPoolExecutor (Concurrent.futures opens new windows in tkinter instead of running the function). I am currently on Python 3.7, and was hoping not to have to reinstall a new enviroment to make this multiprocessing code work, although perhaps that is necessary...?
Other info: using python 3.7.6, tkinter 8.6.8, Eclipse 4.11.0, macOS10.13.6.
Any help appreciated.
You cannot use tkinter code across multiple processes. At least, you can't run the same tkinter code. It simply isn't designed to be used that way. When you create a root window, underneath the covers a tcl interpreter is created, and this interpreter can't be pickled or shared between processes, and doesn't use python's global interpreter lock.
In short, all of your GUI code needs to be in a single thread in a single process.
The following answer is a slightly better explanation, written by one of the developers on the Tcl core team: https://stackoverflow.com/a/38767665/7432. Here is the opening paragraph of that answer:
Each Tcl interpreter object (i.e., the context that knows how to run a Tcl procedure) can only be safely used from the OS thread that creates it. This is because Tcl doesn't use a global interpreter lock like Python, and instead makes extensive use of thread-specific data to reduce the number of locks required internally. (Well-written Tcl code can take advantage of this to scale up very large on suitable hardware.)
I found that this has been reported as part of a bug related to tkinter, python, and macOSX: https://bugs.python.org/issue33111.
(The bug was reported for python 2.7 and 3.6.4 and OSX 10.11.6, but apparently is still a problem with python 3.7.6 and OSX 10.13.6.)
However, there is a partial workaround (also reported at the same site), which for my case seems to work perfectly:
import multiprocessing
multiprocessing.set_start_method("spawn", force=True)
...
... other code same as initial ...
...
if __name__ == '__main__':
testList2a = ['abc','def','geh']
testList2b = ['xyz', 'lmn', 'opq']
allLists = [testList2a,testList2b]
with multiprocessing.Pool(processes=2) as pool:
pool.map(runGUIthread, allLists)
The result is the spawning of multiple GUIs, one for each process.
Per the bug report, in python 3.8, the default start method for multiprocessing when run on MacOS has been changed and is now "spawn" instead of "fork", so the problem will not reveal itself (unless you change the start method to be "fork", in which case the code will fail).
I am having some trouble with a GUI that is freezing during a file save operation that is taking some time, and I'd love to understand why that is.
I've followed the instructions of Schollii's wonderful answer on a similar question, but there must be something I'm missing because I cannot get the GUI behaving as I expect.
The below example is not runnable, since it shows only the relevant parts, but hopefully it's enough to get a discussion going. Basically I have a main application class that generates some large data, and I need to save it to HDF5 format, but this takes some time. To leave the GUI responsive, the main class creates an object of the Saver class and a QThread to do the actual data saving (using moveToThread).
The output of this code is pretty much what I would expect (i.e. I see a message that the "saving thread" has a different thread id than the "main" thread) so I know that another thread is being created. The data is successfully saved, too, so that part is working correctly.
During the actual data saving however (which can take some minutes), the GUI freezes up and goes "Not responding" on Windows. Any clues as to what is going wrong?
Stdout during running:
outer thread "main" (#15108)
<__main__.Saver object at 0x0000027BEEFF3678> running SaveThread
Saving data from thread "saving_thread" (#13624)
Code sample:
from PyQt5 import QtCore, QtGui, QtWidgets
from PyQt5.QtCore import QThread, pyqtSignal, pyqtSlot, QObject
class MyApp(QtWidgets.QMainWindow, MyAppDesign.Ui_MainWindow):
def save_file(self):
self.save_name, _ = QtWidgets.\
QFileDialog.getSaveFileName(self)
QThread.currentThread().setObjectName('main')
outer_thread_name = QThread.currentThread().objectName()
outer_thread_id = int(QThread.currentThreadId())
# print debug info about main app thread:
print('outer thread "{}" (#{})'.format(outer_thread_name,
outer_thread_id))
# Create worker and thread to save the data
self.saver = Saver(self.data,
self.save_name,
self.compressionSlider.value())
self.save_thread = QThread()
self.save_thread.setObjectName('saving_thread')
self.saver.moveToThread(self.save_thread)
# Connect signals
self.saver.sig_done.connect(self.on_saver_done)
self.saver.sig_msg.connect(print)
self.save_thread.started.connect(self.saver.save_data)
self.save_thread.start())
#pyqtSlot(str)
def on_saver_done(self, filename):
print('Finished saving {}'.format(filename))
''' End Class '''
class Saver(QObject):
sig_done = pyqtSignal(str) # worker id: emitted at end of work()
sig_msg = pyqtSignal(str) # message to be shown to user
def __init__(self, data_to_save, filename, compression_level):
super().__init__()
self.data = data_to_save
self.filename = filename
self.compression_level = compression_level
#pyqtSlot()
def save_data(self):
thread_name = QThread.currentThread().objectName()
thread_id = int(QThread.currentThreadId())
self.sig_msg.emit('Saving data '
'from thread "{}" (#{})'.format(thread_name,
thread_id))
print(self, "running SaveThread")
h5f = h5py.File(self.filename, 'w')
h5f.create_dataset('data',
data=self.data,
compression='gzip',
compression_opts=self.compression_level)
h5f.close()
self.sig_done.emit(self.filename)
''' End Class '''
There are actually two issues here: (1) Qt's signals and slots mechanisms, and (2) h5py.
First, the signals/slots. These actually work by copying arguments passed to the signal, to avoid any race conditions. (This is just one of the reasons you see so many signals with pointer arguments in the Qt C++ code: copying a pointer is cheap.) Because you're generating the data in the main thread, it must be copied in the main thread's event loop. The data is obviously big enough for this to take some time, blocking the event loop from handling GUI events. If you instead (for testing purposes) generate the data inside the Saver.save_data() slot, the GUI remains responsive.
However, you'll now notice a small lag after the first "Saving data from thread..." message is printed, indicating that the main event loop is blocked during the actual save. This is where h5py comes in.
You're presumably importing h5py at the top of your file, which is the "correct" thing to do. I noticed that if you instead import h5py directly before you create the file, this goes away. My best guess is that the global interpreter lock is involved, as the h5py code is visible from both the main and saving threads. I would have expected that the main thread would be entirely inside Qt module code at this point, however, over which the GIL has no control. So, like I said, I'm not sure what causes the blocking here.
As far as solutions, to the extent you can do what I described here, that will alleviate the problem. Generating the data outside the main thread, if possible, is advisable. It may also be possible to pass some memoryview object, or a numpy.view object, to the saving thread, though you'll then have to deal with thread-synchronization yourself. Also, importing h5py inside the Saver.save_data() slot will help, but isn't feasible if you need the module elsewhere in the code.
Hope this helps!
I would like to ask how to read a big file from disk and maintain the PyQt4 UI responsive (not blocked). I had moved the load of the file to a QThread subclass but my GUI thread get freezed. Any suggestions? I think it must be something with the GIL but I don't know how to sort it?
EDIT:
I am using vtkGDCMImageReader from the GDCM project to read a multiframe DICOM image and display it with vtk and pyqt4. I do this load in a different thread (QThread) but my app freeze until the image is loaded. here is an example code:
class ReadThread(QThread):
def __init__(self, file_name):
super(ReadThread, self).__init__(self)
self.file_name = file_name
self.reader.vtkgdcm.vtkGDCMImageReader()
def run(self):
self.reader.SetFileName(self.file_name)
self.reader.Update()
self.emit(QtCore.SIGNAL('image_loaded'), self.reader.GetOutput())
I'm guessing you're directly calling the run to begin the thread. That would make the GUI freeze because you're not activating the thread.
So you'd be missing the start there, which would indirectly and properly call the run:
thread = ReadThread()
thread.begin()
class ReadThread(QThread):
def __init__(self, file_name):
super(ReadThread, self).__init__(self)
self.file_name = file_name
self.reader.vtkgdcm.vtkGDCMImageReader()
def run(self):
self.reader.SetFileName(self.file_name)
self.reader.Update()
self.emit(QtCore.SIGNAL('image_loaded'), self.reader.GetOutput())
def begin(self):
self.start()
A bit late, but I think I can pinpoint the problem you face.
The image is large, and the unpacking is probably a CPU intensive task. This means that your GUI thread goes to sleep and the loading thread is CPU bound. At that point the loading thread has the GIL, and the GUI cannot start.
Even if you can get into the loading thread, and introduce a sleep(0), to alow the GUI to continue, this will not help on a mulit-core or multi-processor machine. What happens is the O/S has two threads and thinks it can run both. The loading thread is set up on (say) core 1 and the GUI can be loaded and run on (say) core 2. So after initiating the load and start of the GUI thread on core 2, the O/S resumes the loading thread on core 1 - which promtply grabs the GIL. Moments later the GUI thread is ready to start, and attempts to aquire the GIL, which fails. All it can do without the GIL is go back to sleep!
One solution is to insert a short (greater than zero) sleep in the background thread at strategic intervals, so that the GUI can run. This is not always possible.
Take a look at threading-in-a-pyqt-application-use-qt-threads-or-python-threads
Maybe create your reader object in the tread:
class ReadThread(QThread):
def __init__(self, file_name):
super(ReadThread, self).__init__(self)
self.file_name = file_name
- self.reader = vtkgdcm.vtkGDCMImageReader()
def run(self):
+ self.reader = vtkgdcm.vtkGDCMImageReader()
self.reader.SetFileName(self.file_name)
self.reader.Update()
self.emit(QtCore.SIGNAL('image_loaded'), self.reader.GetOutput())