I have a simulation code and I would like to add diagnostic plotting to it with pyqtgraph.
The simulation is implemented in a Python class. In pseudo-code,
class Simulation:
[...]
'''
Defines self.data, the stuff that needs plotting
and self.done, which is False and becomese True once the simulation
is over.
'''
def evolve(self):
''' takes care of updating self.data'''
[...]
self.done = True
def get_data(self):
return self.data
and I would like to keep it independent from the plotting infrastructure.
What I'm trying to do is to create a plotting class, which so far I tried to implement with python threading, that reads
import pyqtgraph as pg
import threading
from time import sleep
from pyqtgraph.Qt import QtGui
class Plot(threading.Thread):
def __init__(self, source, target_fps=10):
threading.Thread.__init__(self, daemon=True)
self.source = source
self.wait_time = 1. / target_fps
self.win = pg.GraphicsWindow()
self.plot = self.win.addPlot()
self.curve = self.plot.plot()
self.curve.setData(self.source.get_data())
def run(self):
while not self.source.done:
self.curve.setData(self.source.get_data())
sleep(self.wait_time)
whose job is to poll the source class once in a while and update the plot. Then my __main__ would be something along the lines of
if __name__ == "__main__":
sim = Simulation() # initialise simulation code
plotter = Plot(sim) # <- By commenting these three lines
plotter.start() # <- I fall back to a working simulation
sim.evolve() #
plotter.join() # <- without plotting diagnostics
Of course the previous __main__ doesn't work because it is missing the Qt event loop required by pyqtgraph, something of the like of pg.QtGui.QApplication.exec_(), which carries the inconvenient side effect of blocking the execution of the rest of the code after it. If I try to put the exec_() inside Plot.run() I get a warning that the event loop must be run from within the main thread, and plots don't show up.
Is there a workaround that I can use to make the plotting class work and not block the execution of the simulation? I would like not to touch the Simulation class.
Related
this is a bit longer, the first part is just a description of the problem, the second one the question if my "fix" is correct.
I started with python programming. I created a program that communicates with an Arduino that reads the temperature of a furnace of our melting lab. The temperature is then used in a PID algorithm and an output is set to the Arduino. The communication is done via pyserial. So far, everthing works, including live plotting of the temperature signals, PID-variables and so on. The script includes a the main loop and 3 threads (serial communication, a datashifter that reads from serialport, the set temperature from the QWidget and the output of the PID algorithm. This values are used to create an array for displaying within pyqtgraph. Finally, the third thread shifts the data from the datashifter to the QWidget.
When using my Linux-Notebook, everything works fine, and the GUI never stops updating. In contrast, when using any Windows-Host, i have the problem that some pyqtgraphs stop to refresh. The behavior is strange, because i set all data at more or less the same time, with the same numpy array (just different columns) - some plots refresh longer (hours), some stop earlier (minutes). After searching more or less the hole internet ;-) I think that I found the problem: Its the passing of data from from a thread to the GUI. Some dummy code to explain what's going on:
DataUpdaterToGUI(QThread):
#sets the QWidget from main loop
def setGUI(self, gui):
self.gui = gui
def run()
while True:
with lock(): # RLock() Instance
copyArray = self.dataArray[:] # copy the array from the shifter
self.gui.plot1.copyArray(dataArray[:, 0], copyArray[:, 1])
self.gui.plot2.copyArray(dataArray[:, 0], copyArray[:, 2])
# self.gui.update()
# QApplication.instance().processEvents()
Neither calling self.gui.update() nor processEvents() has any influence on the outcome: The plots stop redrawing after a while (on windows).
Now i have a very simple example, and just want to make sure if I'm using the threading-stuff correctly. It works fine, but I have some questions:
Does the signal-slot approach copy the passed data?
Why is it not necessary to call the update() method of the QWidget?
Do I have to use any kind of locks when using signals?
class Main(QWidget):
def __init__(self):
super().__init__()
self.layout = QGridLayout(self)
self.graph = pg.PlotWidget()
self.graph.setYRange(0,1000)
self.plot = self.graph.plot()
self.layout.addWidget(self.graph,0,0)
self.show()
def make_connection(self, data_object):
data_object.signal.connect(self.grab_data)
#pyqtSlot(object)
def grab_data(self, data):
print(data)
self.plot.setData(data)
class Worker(QThread):
signal = pyqtSignal(object)
def __init__(self):
super().__init__()
def run(self):
self.data = [0, 1]
i = 2
while True:
self.data[1] = i
self.signal.emit(self.data)
time.sleep(0.01)
i += 1
if __name__ == "__main__":
app = QApplication(sys.argv)
widget = Main()
worker = Worker()
widget.make_connection(worker)
worker.start()
sys.exit(app.exec_())
Does the signal-slot approach copy the passed data? The signals are thread-safe and when transferring data they make a copy so the thread that precedes the data and the thread that consumes it (GUI Thread) will not have conflicts
Why is it not necessary to call the update() method of the QWidget? Actually pyqtgraph calls the update method, plot is a PlotDataItem, so if we check the source code of setData() method, it calls the updateItems() method, in that method the setData() method of the curve or scatter attribute is called (according to the type of graphics), in the case of curve its setData() method calls updateData(), and the updateData() method calls update, and in the case of the scatter its setData() method calls addpoint(), and addPoints() calls invalidate(), and this invalidate() method calls update().
Do I have to use any kind of locks when using signals? No, as the signals are thread-safe so Qt already has the guards set to avoid the collision.
I designed a program to solve Rubik's cubes, and now I'm building a GUI for it using PySide. My program generates a list of moves required to solve the cube, and then executes them one by one. I would like my program to show the status of the cube for a short period of time between each move.
Currently, I'm trying to use the time module to make the program wait between executing moves. Basically like this:
for move in algorithm:
executeMove()
updateDisplay()
time.sleep(0.1)
I figured this method would work just fine. However, when I run the application, it looks like it's sleeping for the sum time of each of the sleep calls, then showing the end result of the algorithm. I would ideally like to make it show a move, sleep 0.1, show a move, sleep 0.1, etc.
Is the sleep function ideal for the type of behavior I'm trying to get? Should I be using something different entirely? Thank you for your suggestions.
It'd be good to see a little more code, but it looks like you're probably blocking the main Qt thread. To accomplish what you want to do, you will need to be multi-threaded and use pyQtSignals to update the UI. Here's a (maybe buggy) template
class MainWindow(QtWidgets.QMainWindow):
updateSignal = QtCore.pyqtSignal()
def __init__(self, algorithm):
super(MainWindow, self).__init__()
self.algorithm = algorithm
self.updateSignal.connect(self.updateDisplay)
self.loopThread = None
self.startMonitoring()
def startMonitoring(self):
self.loopThread = MonitorLoop(self.updateSignal.emit, self.algorithm)
self.loopThread.start()
def updateDisplay(self):
""" Update the UI here"""
pass
class MonitorLoop(QtCore.QThread):
def __init__(self, signal, algorithm):
super(MonitorLoop, self).__init__()
self.signal = signal # needs to be a pyQtSignal if it will update the UI
self.algorithm = algorithm
def run(self):
for move in self.algorithm:
executeMove()
self.signal()
time.sleep(0.1)
If you are using Qt 4, you will need to substitute QtGui for QWidgets. Of course, I don't actually know what algorithm is, so your implementation will need to accomodate that.
So I'm trying to create a timer or sorts, the start button starts the timer, i need to stop the timer with the stop button and then record the time... I can't seem to figure out how to stop the timer function once its started. Ive tried if statements, disconnect(), and many others...
I realize that there is a timer in qt already but I'm trying to figure it out this way. thanks.
import sys
import time
from PyQt4 import QtCore, QtGui, uic
form_class = uic.loadUiType("/Users/Home/Desktop/Timer/timer.ui")[0]
class MyWindowClass(QtGui.QMainWindow, form_class):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.setupUi(self)
self.startButton.clicked.connect(self.timer)
def timer(self):
timeL = 0
self.stopButton.clicked.connect(False)
while True:
self.timeView.display(timeL)
timeL = timeL + 1
time.sleep(1)
app.processEvents()
app = QtGui.QApplication(sys.argv)
myWindow = MyWindowClass(None)
myWindow.show()
app.exec_()
TL;DR: You're after QElapsedTimer, not QTimer.
The operating system is already measuring the passage of time for you. You won't do any better job at it yourself.
"I'm trying to figure it out this way" - it will be less accurate than using QElapsedTimer because you're assuming that exactly as much time has passed as you wished to sleep for. This is almost never true: the actual amount of time that passed is different than the argument to sleep. Worse yet, the errors are usually of a systematic kind too, so your time accumulation will have a bias and will get less accurate as time passes. So, don't do it. It makes no sense. Perhaps you're not telling us exactly what you're trying to do: if you're asking about a particular solution that doesn't work, it helps to say what problem the solution is supposedly to. Why are you trying to figure it out this (wrong) way?
There are conceptually three different kinds of timers in Qt:
QElapsedTimer is like a stopwatch: it is an interface to the operating system's way of measuring the passage of time. That's the class you should be using to measure how much time has passed between the button clicks.
QTime is like a wall clock: you can ask it what time it is through currentTime(), and take difference between two readings of time to obtain elapsed time. Use this class only if you need to know the absolute time, otherwise QElapsedTimer will offer better resolution for elapsed time measurements.
QTimer is a source of timeouts: it is a way to periodically call your code. This is not meant to be used in measuring time, but merely to let your code run periodically, e.g. when you wish to refresh screen display, or implement a metronome that beeps periodically. There are no guarantees that your code will be called on time, and no guarantees that some ticks won't be missed. You want it guaranteed, you need to write a kernel driver, no way around that.
Below is a complete example using PyQt4, for Python 3.5. It uses QElapsedTimer to measure the passage of time between button presses, and QTimer to keep the time display refreshed.
#!/usr/bin/env python
#https://github.com/KubaO/stackoverflown/tree/master/questions/interval-timer-38036583
# -*- coding: utf-8 -*-
import sys
from PyQt4 import QtCore, QtGui
if __name__ == "__main__":
running = False
app = QtGui.QApplication(sys.argv)
w = QtGui.QWidget()
layout = QtGui.QVBoxLayout(w)
label = QtGui.QLabel()
button = QtGui.QPushButton('Start')
timer = QtCore.QElapsedTimer()
updateTimer = QtCore.QTimer()
layout.addWidget(label)
layout.addWidget(button)
def onTimeout():
label.setText('Elapsed: {0}ms'.format(timer.elapsed()))
def onClicked():
global running
if running:
onTimeout()
updateTimer.stop()
button.setText('Start')
else:
timer.start()
updateTimer.start()
button.setText('Stop')
running = not running
updateTimer.setInterval(1000/25) # ~25fps update rate
updateTimer.timeout.connect(onTimeout)
button.clicked.connect(onClicked)
w.show()
sys.exit(app.exec_())
Use a flag to control the loop. Then reset the flag in the slot connected to the stop button:
self.startButton.clicked.connect(self.timer)
self.stopButton.clicked.connect(self.stop)
def stop(self):
self._timer_flag = False
def timer(self):
timeL = 0
self._timer_flag = True
while self._timer_flag:
self.timeView.display(timeL)
timeL = timeL + 1
time.sleep(1)
app.processEvents()
A QTimer is better, because there is no lag in updating the ui. But you can improve your example by using an inner loop to call processEvents more frequently:
def timer(self):
timeL = 0
self._timer_flag = True
while self._timer_flag:
self.timeView.display(timeL)
timeL = timeL + 1
for _ in range(10):
# process events for up to 50 milliseconds
app.processEvents(QtCore.QEventLoop.AllEvents, 50)
time.sleep(0.1)
Sounds like you want to use QTimer on a QThread.
This answer should give you everything you need.
https://stackoverflow.com/a/18960953/5757280
How can I show a Chaco plot that is created in a running thread? I think an example will make my idea a bit clearer:
Have a look at my example code that creates a plot with Chaco.
from traits.api import HasTraits, Instance
from traitsui.api import View, Item
from chaco.api import ArrayPlotData, Plot
from enable.component_editor import ComponentEditor
class LinePlot(HasTraits):
plot = Instance(Plot)
traits_view = View(
Item('plot', editor=ComponentEditor(),
show_label=False
),
kind='live'
)
def __init__(self):
super(LinePlot, self).__init__()
x = range(10)
plotdata = ArrayPlotData(x=x, y=x)
self.plot = Plot(plotdata)
self.plot.plot(('x','y'))
def run():
l = LinePlot()
l.edit_traits()
do_something()
def do_something():
import time;time.sleep(10)
if I just call the run function via
run()
the plot will show. However if I do something like
import threading
t = threading.Thread(target=run)
t.start()
the plot is unresponsive during the execution of do_something() and then it is closed. I am asking for an explanation and even more for a workaround.
First, the problem is not limited or caused by chaco. It comes from the underlying gui toolkit, properly PyQt or wx. With calling sleep, you also forbid your gui to process events. As a general rule, never do gui changes is a thread.
I have been working on a problem for a while (I am a chemical engineer, so it takes me forever to understand how to code something) of how to have several tabs operating in their own process, but with each tab having it's own data to show in a matplotlib plot.
I have been running into many pickling errors and I wondered if anyone had any somewhat simple solutions. I believe the main reasons for the pickling errors is due to the object I am trying to pass as a property into the tab object. This object holds some data as well as many other objects which help fit the data it holds.
I feel like these objects are very nice and rather necessary, but I also realize that they are causing the problems with pickling.
Here is a very simplified version of my code:
(This will still compile if you want to copy/paste to test it out.)
import multiprocessing as mp
from PyQt4 import QtGui, QtCore
import numpy as np
import matplotlib
matplotlib.use('QtAgg')
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib import figure
import sys
import lmfit
# This object will just hold certain objects which will help create data objects ato be shown in matplotlib plots
# this could be a type of species with properties that could be quantized to a location on an axis (like number of teeth)
#, which special_object would hold another quantization of that property (like length of teeth)
class object_within_special_object:
def __init__(self, n, m):
self.n = n
self.m = m
def location(self, i):
location = i*self.m/self.n
return location
def NM(self):
return str(self.n) + str(self.m)
# This is what will hold a number of species and all of their properties,
# as well as some data to try and fit using the species and their properties
class special_object:
def __init__(self, name, X, Y):
self.name = name
self.X = X
self.Y = Y
self.params = lmfit.Parameters()
self.things = self.make_a_whole_bunch_of_things()
for thing in self.things:
self.params.add('something' + str(thing.NM()) + 's', value = 3)
def make_a_whole_bunch_of_things(self):
things = []
for n in range(0,20):
m=1
things.append(object_within_special_object(n,m))
return things
# a special type of tab which holds a (or a couple of) matplotlib plots and a special_object ( which holds the data to display in those plots)
class Special_Tab(QtGui.QTabWidget):
def __init__(self, parent, special_object):
QtGui.QTabWidget.__init__(self, parent)
self.special_object = special_object
self.grid = QtGui.QGridLayout(self)
# matplotlib figure put into tab
self.fig = figure.Figure()
self.plot = self.fig.add_subplot(111)
self.line, = self.plot.plot(self.special_object.X, self.special_object.Y, 'r-')
self.canvas = FigureCanvas(self.fig)
self.grid.addWidget(self.canvas)
self.canvas.show()
self.canvas.draw()
self.canvas_BBox = self.plot.figure.canvas.copy_from_bbox(self.plot.bbox)
ax1 = self.plot.figure.axes[0]
def process_on_special_object(self):
# do a long fitting process involving the properties of the special_object
return
def update_GUI(self):
# change the GUI to reflect changes made to special_object
self.line.set_data(special_object.X, special_object.Y)
self.plot.draw_artist(self.line)
self.plot.figure.canvas.blit(self.plot.bbox)
return
# This window just has a button to make all of the tabs in separate processes
class MainWindow(QtGui.QMainWindow):
def __init__(self, parent = None):
# This GUI stuff shouldn't be too important
QtGui.QMainWindow.__init__(self)
self.resize(int(app.desktop().screenGeometry().width()*.6), int(app.desktop().screenGeometry().height()*.6))
self.tabs_list = []
central_widget = QtGui.QWidget(self)
self.main_tab_widget = QtGui.QTabWidget()
self.layout = QtGui.QHBoxLayout(central_widget)
button = QtGui.QPushButton('Open Tabs')
self.layout.addWidget(button)
self.layout.addWidget(self.main_tab_widget)
QtCore.QObject.connect(button, QtCore.SIGNAL("clicked()"), self.open_tabs)
self.setCentralWidget(central_widget)
central_widget.setLayout(self.layout)
# Here we open several tabs and put them in different processes
def open_tabs(self):
for i in range(0, 10):
# this is just some random data for the objects
X = np.arange(1240.0/1350.0, 1240./200., 0.01)
Y = np.array(np.e**.2*X + np.sin(10*X)+np.cos(4*X))
# Here the special tab is created
new_tab = Special_Tab(self.main_tab_widget, special_object(str(i), X, Y))
self.main_tab_widget.addTab(new_tab, str(i))
# this part works fine without the .start() function
self.tabs_list.append(mp.Process(target=new_tab))
# this is where pickling errors occur
self.tabs_list[-1].start()
return
if __name__ == "__main__":
app = QtGui.QApplication([])
win = MainWindow()
win.show()
sys.exit(app.exec_())
I have noticed that the errors are coming from matplotlib axes (I'm not sure how?) and gives the error pickle.PicklingError: Can't pickle <class 'matplotlib.axes.AxesSubplot'>: it's not found as matplotlib.axes.AxesSubplot. In addition, I have noticed that commenting out the matplotlib plots will also give the pickling error pickle.PicklingError: Can't pickle <function <lambda> at 0x012A2B30>: it's not found as lmfit.parameter.<lambda>. I think this is because lambda functions cannot be pickled and I guess lmfit has a lambda somewhere in it's depths... but I don't really know what to do without some way around these errors.
Oddly enough, the error I see form the original code (not the simplified version shown here) is slightly different, but still basically the same in sentiment. The error I get in my other code is pickle.PicklingError: Can't pickle 'BufferRegion' object: <BufferRegion object at 0x037EBA04>
Does anyone have a better solution to this problem by moving the objects around in regards to where I pass them or any other ideas?
I very much appreciate your time and effort and any help on this problem.
EDIT: I tried unutbu's idea in a way, but with some alterations to the position of the process funciton.
The only problem with the proposed solution is that the do_long_fitting_process() function calls another function which iteratively updates the lines in the matplotlib plots. So the do_long_fitting_process() needs to have some access to the Special_Tab properties to change them and show the updates to the GUI.
I have tried doing this by pushing the do_long_fitting_process() function to just a global function and calling this:
[code]
def open_tabs(self):
for i in range(0, 10):
...
self.tabs_list.append(new_tab)
consumer, producer = mp.Pipe()
process = mp.Process(target=process_on_special_object, args=(producer,))
process.start()
while(True):
message = consumer.recv()
if message == 'done':
break
tab.update_GUI(message[0], message[1])
process_list[-1].join()
[/code]
Where I am passing the data to update_GUI() via a mp.Pipe(), but the window just goes to "Not Responding" as soon as I start the processes.
Separate the GUI code from the computation code. The GUI must run in a single process (though it may spawn multiple threads). Let the computation code be contained within in the special_object.
Let the Special_Tab call mp.Process when you want to perform a long-running computation:
class special_object:
def do_long_fitting_process(self):
pass
class Special_Tab(QtGui.QTabWidget):
def process_on_special_object(self):
# do a long fitting process involving the properties of the
# special_object
proc = mp.Process(target = self.special_object.do_long_fitting_process)
proc.start()
class MainWindow(QtGui.QMainWindow):
def open_tabs(self):
for i in range(0, 10):
...
self.tabs_list.append(new_tab)
new_tab.process_on_special_object()
The problem is that not all parts of an Axes object can be serialized, which is necessary to move data between the processes. I would suggest a slight re-organization of your code to push computation off to separate processes (that is any thing that will take more than a fraction of a second), but keep all of your plotting on the main process and only pass data back and forth.
Another option is to us QThread See time.sleep() required to keep QThread responsive? for two different ways of implementing it (one in the question, one in my answer).