I have been developing a GUI for reading continuous data from a serial port. After reading the data, some calculations are made and the results will be plotted and refreshed (aka dynamic plotting). I use the wx backend provided in the matplotlib for this purposes. To do this, I basically use an array to store my results, in which I keep appending it to, after each calculation, and replot the whole graph. To make it "dynamic", I just set the x-axis lower and upper limits for each iteration. Something like found in:
http://eli.thegreenplace.net/2008/08/01/matplotlib-with-wxpython-guis/
The problem, however, is that since the data is continuous, and if I keep plotting it, eventually the system memory will run out and system will crash. Is there any other way I can plot my result continuously?
To do this, I basically use an array
to store my results, in which I keep
appending it to
Try limiting the size of this array, either by deleting old data or by deleting every n-th entry (the screen resolution will prevent all entries to be displayed anyway). I assume you write all the data to disk so you won't lose anything.
Also, analise your code for memory leaks. Stuff you use and don't need anymore but that doesn't get garbage-collected because you still have a reference to it.
I have created such a component with pythons Tkinter. The source is here.
Basically, you have to keep the plotted data somewhere. You cannot keep an infinite amount of data points in memory, so you either have to save it to disk or you have to overwrite old data points.
Data and representation of data are two different things. You might want to store your data to disk if it's important data to be analyzed later, but only keep a fixed period of time or the last N points for display purposes. You could even let the user pick the time frame to be displayed.
I actually ran into this problem (more of a mental block, actually...).
First of all I copy-pasted some wx Plot code from wx Demo Code.
What I do is keep a live log of a value, and compare it to two markers (min and max, shown as red and green dotted lines) (but I will make these 2 markers optional - hence the optional parameters).
In order to implement the live log, I first wanted to use the deque class, but since the data is in tuple mode (x,y coordinates) I gave up and just tried to rewrite the entire parameter list of tuples: see _update_coordinates.
It works just fine for keeping track of the last 100-10,000 plots. Would have also included a printscreen, but I'm too much of a noob at stackoverflow to be allowed :))
My live parameter is updated every 0.25 seconds over a 115kbps UART.
The trick is at the end, in the custom refresh method!
Here is most of the code:
class DefaultPlotFrame(wx.Frame):
def __init__(self, ymin=0, ymax=MAXIMUM_PLOTS, minThreshold=None,
maxThreshold=None, plotColour='blue',
title="Default Plot Frame",
position=(10,10),
backgroundColour="yellow", frameSize=(400,300)):
self.minThreshold = minThreshold
self.maxThreshold = maxThreshold
self.frame1 = wx.Frame(None, title="wx.lib.plot", id=-1, size=(410, 340), pos=position)
self.panel1 = wx.Panel(self.frame1)
self.panel1.SetBackgroundColour(backgroundColour)
self.ymin = ymin
self.ymax = ymax
self.title = title
self.plotColour = plotColour
self.lines = [None, None, None]
# mild difference between wxPython26 and wxPython28
if wx.VERSION[1] < 7:
self.plotter = plot.PlotCanvas(self.panel1, size=frameSize)
else:
self.plotter = plot.PlotCanvas(self.panel1)
self.plotter.SetInitialSize(size=frameSize)
# enable the zoom feature (drag a box around area of interest)
self.plotter.SetEnableZoom(False)
# list of (x,y) data point tuples
self.coordinates = []
for x_item in range(MAXIMUM_PLOTS):
self.coordinates.append((x_item, (ymin+ymax)/2))
self.queue = deque(self.coordinates)
if self.maxThreshold!=None:
self._update_max_threshold()
#endif
if self.lockThreshold!=None:
self._update_min_threshold()
#endif
self.line = plot.PolyLine(self.coordinates, colour=plotColour, width=1)
self.lines[0] = (self.line)
self.gc = plot.PlotGraphics(self.lines, title, 'Time', 'Value')
self.plotter.Draw(self.gc, xAxis=(0, MAXIMUM_PLOTS), yAxis=(ymin, ymax))
self.frame1.Show(True)
def _update_max_threshold(self):
if self.maxThreshold!=None:
self.maxCoordinates = []
for x_item in range(MAXIMUM_PLOTS):
self.maxCoordinates.append((x_item, self.maxThreshold))
#endfor
self.maxLine = plot.PolyLine(self.maxCoordinates, colour="green", width=1)
self.maxMarker = plot.PolyMarker(self.maxCoordinates, colour="green", marker='dot')
self.lines[1] = self.maxMarker
#endif
def _update_live_param(self, liveParam, minParam, maxParam):
if minParam!=None:
self.minThreshold = int(minParam)
self._update_min_threshold()
#endif
if maxParam!=None:
self.maxThreshold = int(maxParam)
self._update_max_threshold()
#endif
if liveParam!=None:
self._update_coordinates(int(liveParam))
#endif
def _update_coordinates(self, newValue):
newList = []
for x,y in self.coordinates[1:]:
newList.append((x-1, y))
#endfor
newList.append((x, newValue))
print "New list", newList
self.line = (plot.PolyLine(newList, colour=self.plotColour, width=1))
self.lines[0] = self.line
self.coordinates = newList
def _MyLIVE_MAGIC_refresh__(self, liveParam=None, minParam=None, maxParam=None):
self._update_live_param(liveParam, minParam, maxParam)
self.gc = plot.PlotGraphics(self.lines, self.title, 'Time', 'Value')
self.plotter.Draw(self.gc, xAxis=(0, MAXIMUM_PLOTS), yAxis=(self.ymin, self.ymax))
self.plotter.Refresh()
self.frame1.Refresh()
Related
Exactly like the title says, I've used some code that I collected from multiple places to build up this Chord diagram, but unfortunately one last thing that's kind of going to hinder that 100% perfection is this popup that shows up whenever I hover over the label. The pop up remains there even when the mouse is moving elsewhere...
By the way, I'm using Python and Holoviews for plotting the chord.
I'm pretty sure this is due to a bug or something... however, I'd love to find a way to bypass it...
Example here : bug example
Code :
%%opts Chord [height=600 width=600 title="Transactions from 2000 to 2021" ]
#plot
#edit links = moves[moves['Transactions'] > x] // x refers to the number of transactions minimum for the diagram display
moves = moves[moves['Transactions']>5]
links = moves
chord = hv.Chord(links)
chord
nodesl = []
c = []
for i, row in moves.iterrows():
c.append(row[0])
c.append(row[1])
c = pd.DataFrame(c)
c.drop_duplicates(inplace=True)
c.reset_index(inplace=True)
c.drop(columns='index', inplace=True)
c
nodes = []
for i, row in c.iterrows():
nodes.append({'name':row[0]})
nodes = pd.DataFrame(nodes)
# nodes
nodes = hv.Dataset(nodes, 'name')
nodes.data.head()
%%opts Chord [height=800 width=800 bgcolor="black"]
%%opts Chord [title="Transactions from 2000 to 2021 (Countries with over 5 moves)\nTip: Please do not hover over the label as it might produce a bug, else refresh the page" ]
chord = hv.Chord((links, nodes)).select(value=(5, None))
#this function allows text to fit perfectly on the screen
def rotate_label(plot, element):
text_cds = plot.handles['text_1_source']
length = len(text_cds.data['angle'])
text_cds.data['angle'] = [0]*length
xs = text_cds.data['x']
text = np.array(text_cds.data['text'])
xs[xs<0] -= np.array([len(t)*0.019 for t in text[xs<0]])
chord.opts(
opts.Chord(cmap='Category10',
edge_color=dim('Target').str(),
node_color=dim('name').str(),
labels='name',
label_text_color="white",
hooks=[rotate_label]
))
chord
The ??? usually indicates that some field it's trying to display isn't available; not sure why that would be in this case. You can always override the default tools HoloViews uses for Bokeh by setting default_tools=[] and then specify whatever tools you do want, without 'hover', e.g. tools=['save', 'pan', 'wheel_zoom', 'box_zoom', 'reset'].
So I'm comparing NBA betting lines between different sportsbooks over time
Procedure:
Open pickle file of scraped data
Plot the scraped data
The pickle file is a dictionary of NBA betting lines over time. Each of the two teams are their own nested dictionary. Each key in these team-specific dictionaries represents a different sportsbook. The values for these sportsbook keys are lists of tuples, representing timeseries data. It looks roughly like this:
dicto = {
'Time': <time that the game starts>,
'Team1': {
Market1: [ (time1, value1), (time2, value2), etc...],
Market2: [ (time1, value1), (time2, value2), etc...],
etc...
}
'Team2': {
<SAME FORM AS TEAM1>
}
}
There are no issues with scraping or manipulating this data. The issue comes when I plot it. Here is the code for the script that unpickles and plots these dictionaries:
import matplotlib.pyplot as plt
import pickle, datetime, os, time, re
IMAGEPATH = 'Images'
reg = re.compile(r'[A-Z]+#[A-Z]+[0-9|-]+')
noDate = re.compile(r'[A-Z]+#[A-Z]+')
# Turn 1 into '01'
def zeroPad(num):
if num < 10:
return '0' + str(num)
else:
return num
# Turn list of time-series tuples into an x list and y list
def unzip(lst):
x = []
y = []
for i in lst:
x.append(f'{i[0].hour}:{zeroPad(i[0].minute)}')
y.append(i[1])
return x, y
# Make exactly 5, evenly spaced xticks
def prune(xticks):
last = len(xticks)
first = 0
mid = int(len(xticks) / 2) - 1
upMid = int( mid + (last - mid) / 2)
downMid = int( (mid - first) / 2)
out = []
count = 0
for i in xticks:
if count in [last, first, mid, upMid, downMid]:
out.append(i)
else:
out.append('')
count += 1
return out
def plot(filename, choice):
IMAGEPATH = 'Images'
IMAGEPATH = os.path.join(IMAGEPATH, choice)
with open(filename, 'rb') as pik:
dicto = pickle.load(pik)
fig, axs = plt.subplots(2)
gameID = noDate.search(filename).group(0)
tm = dicto['Time']
fig.suptitle(gameID + '\n' + str(tm))
i = 0
for team in dicto.keys():
axs[i].set_title(team)
if team == 'Time':
continue
for market in dicto[team].keys():
lst = dicto[team][market]
x, y = unzip(lst)
axs[i].plot(x, y, label= market)
axs[i].set_xticks(prune(x))
axs[i].set_xticklabels(rotation=45, labels = x)
i += 1
plt.tight_layout()
#Finish
outputFile = reg.search(filename).group(0)
date = (datetime.datetime.today() - datetime.timedelta(hours = 6)).date()
fig.savefig(os.path.join(IMAGEPATH, str(date), f'{outputFile}.png'))
plt.close()
Here is the image that results from calling the plot function on one of the dictionaries that I described above. It is pretty much exactly as I intended it, except for one very strange and bothersome problem.
You will notice that the bottom right tick looks haunted, demonic, jpeggy, whatever you want to call it. I am highly suspicious that this problem occurs in the prune function, which I use to set the xtick values of the plot.
The reason that I prune the values with a function like this is because these dictionaries are continuously updated, so setting a static number of xticks would not work. And if I don't prune the xticks, they end up becoming unreadable due to overlapping one another.
I am quite confused as to what could cause an xtick to look like this. It happens consistently, for every dictionary, every time. Before I added the prune function (when the xticks unbound, overlapping one another), this issue did not occur. So when I say I'm suspicious that the prune function is the cause, I am really quite certain.
I will be happy to share an instance of one of these dictionaries, but they are saved as .pickle files, and I'm pretty sure it's bad practice to share pickle files over the internet. I have been warned about potential malware, so I'll just stay away from that. But if you need to see the dictionary, I can take the time to prettily print one and share a screenshot. Any help is greatly appreciated!
Matplotlib does this when there are many xticks or yticks which are plotted on the same value. It is normal. If you can limit the number of times the specific value is plotted - you can make it appear indistinguishable from the rest of the xticks.
Plot a simple example to test this out and you will see for yourself.
I've encountered something very strange when having a function which generates an NdOverlay of Points to a DynamicMap, where the function is tied to panel widgets (I don't think the panel widgets are important).
The below code is a working example which produces the expected behavior. Whenever you change the widget values a new plot is generated with two sets of Points overlaid, with different colors and respective legend entries. Image shown below code.
a_widget = pn.widgets.Select(name='A', options=[1,2,3,4])
b_widget = pn.widgets.IntSlider(name='B', start=10, end=20, value=10)
widget_box = pn.WidgetBox(a_widget, b_widget, align='center')
#pn.depends(a=a_widget.param.value, b=b_widget.param.value)
def get_points(a, b):
return hv.NdOverlay({x: hv.Points(np.random.rand(10,10)) for x in range(1,3)})
points = hv.DynamicMap(get_points)
pn.Row(widget_box, points)
The second example shown below, is meant to demonstrate that in certain situations you might want to just simply return an empty plot and the way that I've done it in this example is done in the same way as in this example: http://holoviews.org/gallery/demos/bokeh/box_draw_roi_editor.html#bokeh-gallery-box-draw-roi-editor
The result of this code is an empty plot as expected when a == 1, but when a has values other than 1, the result is quite strange as illustrated in the image below the code.
The points all have the same color
When changing the slider for instance, some points are frozen and never changes, which is not the case in the above working example.
a_widget = pn.widgets.Select(name='A', options=[1,2,3,4])
b_widget = pn.widgets.IntSlider(name='B', start=10, end=20, value=10)
widget_box = pn.WidgetBox(a_widget, b_widget, align='center')
#pn.depends(a=a_widget.param.value, b=b_widget.param.value)
def get_points(a, b):
if a == 1:
return hv.NdOverlay({None: hv.Points([])})
else:
return hv.NdOverlay({x: hv.Points(np.random.rand(10,10)) for x in range(1,3)})
points = hv.DynamicMap(get_points)
pn.Row(widget_box, points)
While I can not help the observed issue with NdOverlay, creating plots with or without content can be done with the help of Overlay.
As b_widget is never used in your code, I removed it for simplicity.
a_widget = pn.widgets.Select(name='A', options=[1,2,3,4])
widget_box = pn.WidgetBox(a_widget, align='center')
#pn.depends(a=a_widget.param.value)
def get_points(a):
images = []
if a == 3:
images.append(hv.Points(np.random.rand(10,10), label='None'))
else:
for x in range(1,3):
images.append(hv.Points(np.random.rand(10,10), label=str(x)))
return hv.Overlay(images)
points = hv.DynamicMap(get_points)
pn.Row(widget_box, points)
The way how to use NdOverlay that is described in the documentation for NdOverlay is different to your approach, this might be a reason for the observed problems.
Anyway, to narrow down which part of the code is responsible for the observed issue, I removed all code that is not necessary to reproduce it.
For clarity, I renamed the values of a, and I also made sure, that a start value for a is provided.
It turned out while testing the code, that the if-else-statement is neither important, so I removed that too.
And just to make sure, that variables behave like expected, I added some print-statements.
This gives the following minimal reproducable example:
a_widget = pn.widgets.Select(name='A', value='Test', options=['Test','Test1', 'Test2'])
#pn.depends(a=a_widget.param.value)
def get_points(a):
dict_ = {}
dict_[str(a)] = hv.Points(np.random.rand(10,10))
print(dict_)
overlay = hv.NdOverlay(dict_)
print(overlay)
return overlay
points = hv.DynamicMap(get_points)
# using the server approach here to see the outpout of the
# print-statements
app = pn.Row(a_widget, points)
app.app()
When running this code, and choosing the different options in the select widget, it turns out that option Test is not updated, once one of the options Test1 and Test3 have been choosen.
When we change the default value in the first line like this
a_widget = pn.widgets.Select(name='A', value='Test2', options=['Test','Test1', 'Test2'])
now Test2 is not updated correctly.
So it looks like this is an issue of DynamicMap using NdOverlay.
So I suggest you report this issue to the developers (if not already done), either wait for new release or use a different approach (e.g. as shown above).
I am currently working on a little script that creates a crane-like rig automatically in Autodesk Maya, the user gets to choose the amount of joints by a UI.
My question is how do I take the integer input of the user and use it as the variable value for my "jointAmount"?
I am also wondering how I would be able to call my function(AutoCraneRig) to actually run the script from the UI. I have a "apply"-button but I am unsure how to connect it to my function.
I have seen similar posts like mine but I feel that the solutions shown are somewhat hard for me to understand and/or I can't really relate what is shown to my own problem.
If anything is unclear or more information is needed from me please don't hesitate to call me out.
Here is what my current UI look like
import maya.cmds as cmds
import pymel.core as pm
def jntctrl():
number = pm.intField(jnt, q=1, v=1)
print(number)
if pm.window("stuff", exists = True):
pm.deleteUI("stuff")
pm.window("stuff", t = "Crane Rig Generator", w=400, h=200)
pm.columnLayout(adj = True)
pm.text(label="Joint Amount:")
jnt = pm.intField(changeCommand = 'jntctrl()')
pm.button(label="Create Crane")
pm.showWindow()
#Defining how many joints the user want to have for their crane rig
jointAmmount = 5
#Defining how many controllers the user want to have to orient the crane.
#May not exceed the joint amount
controllerAmount = 5
def autoCraneRig():
#Creating the joints
for i in range(jointAmmount):
pm.joint()
pm.move(0, i, 0)
#Creating the controllers
for i in range(controllerAmount):
pm.circle()
pm.rotate (0,90,0)
pm.makeIdentity (apply= True)
#Creating the groups
for i in range(controllerAmount):
pm.group()
#Somehow one of the nurbs get parented to a group when running the script, here i select both the groups and then unparent them.
pm.select("group*", "nurbsCircle*")
pm.parent(world = True)
#Creating lists/dictionaries for the groups
#Since I wanted to parent my objects by their number I had to put all objects in lists/dictionries to get access.
groups = pm.ls('group*')
nbs = [int(n.split('group')[-1]) for n in groups]
groupDic = dict(zip(nbs, groups))
#Create a list/dictionary for the joints
joint = pm.ls('joint*', type='joint')
nbs = [int(n.split('joint')[-1]) for n in joint]
jointDic = dict(zip(nbs, joint))
common = list(set(groupDic.keys())&set(jointDic.keys()))
#Parenting the groups to the joints
for i in common:
pm.parent(groupDic[i], jointDic[i])
#Reseting the transformations of the groups and then unparenting them to still have the transformation data of the joints
pm.select("group*")
pm.makeIdentity()
pm.parent(world = True)
#Creating a list/dictionary for the nurbs aswell that will be parented to the groups in numeric order
nurbs_sh = pm.ls('nurbsCircle*', type='nurbsCurve')
#I had to get the transformation information from the nurbs before parenting them with anything would work(took a long time to get it right).
nurbs_tr = pm.listRelatives(nurbs_sh, p=1)
nbs = [int(n.split('nurbsCircle')[-1]) for n in nurbs_tr]
curveDic = dict(zip(nbs, nurbs_tr))
common = list(set(groupDic.keys())&set(curveDic.keys()))
#Parent the nurbs to the groups
for i in common:
pm.parent(curveDic[i], groupDic[i])
#Select the nurbs and reset transformations and then freeze transform
pm.select("nurbsCircle*")
pm.makeIdentity()
#Orient constrain the controllers/nurbs to the joints
for i in common:
pm.orientConstraint(curveDic[i], jointDic[i])
#Parent the 2nd group with the first controller. Do this for the whole hierarchy.
for i in common:
pm.parent(groupDic[i+1], curveDic[i])
#I'm getting keyError after I put the "+1" in my groupDic and I don't know why, although it still works, I guess.
autoCraneRig()
Here's an example for how to call a specific function/command when a button is clicked, and how to get the value of an int field. The key is in naming the fields, so you can reference the UI control later.
import pymel.core as pm
def ui():
if (pm.window("myWindow", exists=True)):
pm.deleteUI("myWindow")
window = pm.window("myWindow", t="My Window", w=400, h=200)
pm.columnLayout(adj=True)
pm.intField("myIntField")
pm.button("Button", aop=True, command="action()")
pm.showWindow(window)
def action():
print("Button clicked!")
value = pm.intField("myIntField", q=True, v=True)
print(value)
ui()
If you want to get more into making UI's, I would recommend you watch these two videos:
PySide UI Creation in Maya: Video One
PySide UI Creation in Maya: Video Two
i have this code for creating a series of image from a series of matrixes, and in each image i want to add a specific text. this is my typical code :
ax = axes([0,0,1,1])
for i in range(0,5):
text(1,1,str(i))
ax.imshow(a[:,:,i],origin='lower')
savefig(str("%04d" % int(i))+'.png',format="png")
del ax.texts[-1]
but the problem is that as the number of iteration increases, the speed decease and it becomes so so slow. It seems that there is something wrong with opening a lot of windows in background.
Any suggestion?
Instead of creating a new image and text objects every loop reuse the objects.
ax = axes([0,0,1,1])
t = text(1,1,str(0))
img = ax.imshow(a[:,:,0],origin='lower')
for i in range(0,5):
t.set_text(str(i)
img.set_data(a[:,:,i])
savefig(str("%04d" % int(i))+'.png',format="png")
also see
Visualization of 3D-numpy-array frame by frame
I just added this single line at the end of the loop and it works fine now. It was simply the problem of accumulating previuosly opened figures in the memory.
ax = axes([0,0,1,1])
for i in range(0,5):
text(1,1,str(i))
ax.imshow(a[:,:,i],origin='lower')
savefig(str("%04d" % int(i))+'.png',format="png")
del ax.texts[-1]
close(gcf())