My goal is to create a bokeh script that shows the processor and memory usage of several machines in my network. The first script pulls the cpu and memory usage, and the bokeh script shows a time series plot of these stats over the last few seconds.
I have copy pasted some bokeh code (thanks internet) that updates two plots in a line based on random numbers every 500 seconds:
import numpy as np
from bokeh.plotting import figure, curdoc
from bokeh.driving import linear
from bokeh.layouts import layout
import random
tools = 'pan'
#linear()
def update(step):
# Instead of random numbers, fetch stats from another script
ds1.data['x'].append(step)
ds1.data['y'].append(random.randint(0,100))
ds2.data['x'].append(step)
ds2.data['y'].append(random.randint(0,100))
ds1.trigger('data', ds1.data, ds1.data)
ds2.trigger('data', ds2.data, ds2.data)
# don't get this yet, but #linear is a decorator. Instead of having #linear(), we could also have
# update = linear(update)
p = figure(plot_width=400, plot_height=400)
r1 = p.line([], [], color="firebrick", line_width=2, legend='line1')
r2 = p.line([], [], color="navy", line_width=2, legend='line2')
ds1 = r1.data_source
ds2 = r2.data_source
curdoc().add_root( p )
# Add a periodic callback to be run every 500 milliseconds
curdoc().add_periodic_callback(update, 500)
Let's assume the first script pulls tuples of memory/cpu usage every 500 ms, how would I get the bokeh script to fetch that information? Once I have that information, I'll be set.
The only idea I have right now is to write the output of the first script to a dict/json/h5 file, and have the bokeh script read it and append that data to the plot. I'm wondering if there is a better way.
Since the update script is just grabbing some numbers, your best bet is probably wrapping that thing inside an update function, processing those numbers so they fit a ColumnDataSource, then update the main CDS, which will then update the plot. Something like:
def update():
# Do stuff to get new data
numbers = get_usage()
# Process new data into a CDS friendly format
data1, data2 = process_numbers(numbers)
# Update plot_source, this being whatever source is
r1_source.data = data1
r2_source.data = data2
curdoc().add_periodic_callback(update, 500)
Related
Making a script that grabs the latest COVID-19 England figures and then displays it as a matplotlib line graph. Early stages, but just thought I'd give a bit of background.
def plot(cases,deaths):
plt.show(block=False)
cases.plot(x='Specimen date',y='Cumulative lab-confirmed cases',color='green')
deaths.plot(x='Reporting date',y='Cumulative deaths',color='red')
plt.show()
This is one of my .py scripts (make_graph.py) that is meant to draw the graph after the .csv files are downloaded and parsed.
# Then, parse the data: we only need the rows with 'England', and the date and cumulative values
imported = csv_parser()
cases_filtered_import = imported[0]
deaths_filtered_import = imported[1]
# Finally, make the graph
plot(cases_filtered_import,deaths_filtered_import)
# !! This is the end of the main.py file
My only issue is that I cannot get the plot windows to be separated from main.py. I want it so that the graphs are displayed and then the code finishes after, instead of having to close the windows before the code execution finishes.
Any help would be appreciated. I am aware that this is probably a duplicated question, but I cannot seem to find a solution that works.
Is it perhaps the way that I executing the program (making separate .py files for different functions, and then importing and executing them from the main.py script)?
This is indeed an old question. The thing is that matplotlib plots are interactive, not static (you can change the scale), so they do require a thread to run. Hence the only way to detach a plot is to arrange a separate thread for it. Then you can call plots there, and proceed with your code. Here is an example:
import numpy as np
import matplotlib.pyplot as plt
from multiprocessing import Process
from time import sleep
x = np.random.rand(10)
y = np.random.rand(10)
def my_plot(x, y):
plt.scatter(y, x)
plt.show()
p = Process(target=my_plot, args=(x, y))
p.start()
sleep(1)
print('keep going')
# At the end
p.join()
I am looking for a way to plot realtime data line plot or scatter plots from python.
With the plots I want to monitor long-running loops when experimenting with algorithms with scientific computing. I.e. to help me answer the question: Are my results still improving with each iteration or can I cancel the loop?
I am looking for a quick and dirty method. I saw that with Bokeh and Dash one can program dashboards with realtime updates, but an awful lot of boilerplate code seems to be required just to get an updating plot.
Here is a simple "live streaming" example for Bokeh v1.3.0. You can run it with bokeh serve --show app.py
app.py:
from bokeh.plotting import figure, curdoc
from datetime import datetime
import random
plot = figure(plot_width = 1200, x_axis_type = 'datetime', tools = 'pan,box_select,crosshair,reset,save,wheel_zoom')
line = plot.line(x = 'time', y = 'value', line_color = 'black', source = dict(time = [datetime.now()], value = [random.randint(5, 10)]))
def update():
line.data_source.stream(dict(time = [datetime.now()], value = [random.randint(5, 10)]))
curdoc().add_root(plot)
curdoc().add_periodic_callback(update, 1000)
From the last days, I have been trying to use Bokeh to plot real-time data and display on a .html in order to be embeed in a webpage. I have sucessuly adapted one of the bokeh examples to my needs. I am using a buffer of 50 elements on the plot and I am noting the following behaviour:
1) In case I run the script and go to the browser the x_range fully adapts to incomming data and everything works correctly
2) If I click on "Refresh" on the browser the x_range stops to adapt to incoming data and freezes to the last value.
I tried to force the x_axis to initial and end values but the visualization behaves poorly.
I think I am not correctly understanding what does the "Refresh" hit impacts my code and how I can workaround this issue.
""" To view this example, first start a Bokeh server:
bokeh serve --allow-websocket-origin=localhost:8000
And then load the example into the Bokeh server by
running the script:
python animated.py
in this directory. Finally, start a simple web server
by running:
python -m SimpleHTTPServer (python 2)
or
python -m http.server (python 3)
in this directory. Navigate to
http://localhost:8000/animated.html
"""
from __future__ import print_function
import io
from numpy import pi, cos, sin, linspace, roll
from bokeh.client import push_session
from bokeh.embed import server_session
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource
fa = open('Accelerometer.txt', 'r')
source = ColumnDataSource(data=dict(x=[], y=[]))
fg = figure(width=250, plot_height=250, title="RT-Test")
fg.line(x='x', y='y', color="olive", source=source)
fg.x_range.follow = "end"
# Visualization scale and aesthetics
fg.xgrid.grid_line_color = None
fg.ygrid.grid_line_color = None
fg.background_fill_color = "snow"
# add the plot to curdoc
curdoc().add_root(fg)
# open a session which will keep our local doc in sync with server
session = push_session(curdoc())
html = """
<html>
<head></head>
<body>
%s
</body>
</html>
""" % server_session(fg, session_id=session.id, relative_urls=False)
with io.open("animated.html", mode='w+', encoding='utf-8') as f:
f.write(html)
print(__doc__)
def update():
line = fa.readline().split(',')
x = float(line[0])
y = float(line[1])
print(x, y)
# construct the new values for all columns, and pass to stream
new_data = dict(x=[x], y=[y])
source.stream(new_data, rollover=50)
curdoc().add_periodic_callback(update, 100)
session.loop_until_closed() # run forever
This kind of usage of the Bokeh server, with the actual code running in a separate process and calling session.loop_until_closed, is discouraged in the strongest terms. In the next release, all of the examples of this sort will be deleted, and mentions of this approach removed from the docs. This usage is inherently inferior in many ways, as outlined here, and I would say that demonstrating it so prominently for so long was a mistake on our part. It is occasionally useful for testing, but nothing else.
So what is the good/intended way to use the Bokeh server? The answer is to have a Bokeh app run in the Bokeh server itself, unlike the code above. This can be done in a variety of ways, but one common way os to wirte a simple script, then execute that script with
bokeh serve -show myapp.py
I don't have access to your "Accelerate.py" dataset, but a rough pass at updating your code would look like:
# myapp.py
from numpy import pi, cos, sin, linspace, roll
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource
fa = open('Accelerometer.txt', 'r')
source = ColumnDataSource(data=dict(x=[], y=[]))
fg = figure(width=250, plot_height=250, title="RT-Test")
fg.line(x='x', y='y', color="olive", source=source)
fg.x_range.follow = "end"
fg.xgrid.grid_line_color = None
fg.ygrid.grid_line_color = None
fg.background_fill_color = "snow"
curdoc().add_root(fg)
def update():
line = fa.readline().split(',')
x = float(line[0])
y = float(line[1])
# construct the new values for all columns, and pass to stream
new_data = dict(x=[x], y=[y])
source.stream(new_data, rollover=50)
curdoc().add_periodic_callback(update, 100)
Now if you run this script with the bokeh serve command, then any refresh will get you a brand new session of this app. It's also worth nothing that code written in this way is considerably simpler and shorter.
These kinds of apps can be embedded in Jupyter notebooks, Flask and other web-apps, or made into "regular" python scripts run with python instead of bokeh serve. For more information see Running a Bokeh Server in the User's Guide.
I'm new to Bokeh and Python, and this is my first Stack Overflow question as well.
I'm using Bokeh to plot trajectory profiles of particles diffusing in the brain, but have it be animated. I have been able to successfully create a program that plots the points, but once all the points are plotted, it stops. I want to be able to loop the animation so that once all the points are plotted, it clears itself and starts over.
I am still very unfamiliar with coding terms, and I wasn't able to find something that could do this. I thought I was on the right track with importing using the reset function inside an if statement, but it doesn't seem to work. I have looked at the following as well for reference:
How to animate a circle using bokeh
Here is my code so far plotting a random trajectory:
import numpy as np
from bokeh.plotting import figure, show, gridplot, vplot, hplot, curdoc
from bokeh.io import output_notebook
from bokeh.client import push_session
from bokeh.core.state import State as new
# This is where the actual coding begins.
b = np.random.rand(300, 3)
xlist = b[:, 1]
ylist = b[:, 2]
# create a plot and style its properties. Change chart title here.
p = figure(title='PEG_PLGA15k_F68_R2_P81', title_text_font_size='13pt',
x_range=(min(xlist), max(xlist)), y_range=(min(ylist), max(ylist)),)
# add a text renderer to out plot (no data yet)
r = p.line(x=[], y=[], line_width=3, color='navy')
session = push_session(curdoc())
i = 0
ds = r.data_source
# create a callback that will add a number in a random location
def callback():
global i
ds.data['x'].append(xlist[i])
ds.data['y'].append(ylist[i])
ds.trigger('data', ds.data, ds.data)
if i < xlist.shape[0] - 1:
i = i + 1
else:
new.reset()
# Adds a new data point every 67 ms. Change at user's discretion.
curdoc().add_periodic_callback(callback, 67)
session.show()
session.loop_until_closed()
If all you want is to restart the animation once you reach some condition (like "all points have been plotted") you can just reset the DataSource. So, for instance, on your example you should have:
else:
i = 0
ds.data['x'] = []
ds.data['y'] = []
instead of:
else:
new.reset()
and that should do the trick. Just use your datasource... State is a more general component that should be used on different level and not to manage plot glyphs and datasources.
A couple of quick notes here:
On your question you've mentioned a link to the 0.10 version documentation but from your code I can tell you are not using a newer version (0.11.x). Always be sure to use the right docs for the version of Bokeh you are using since there might be a few changes between one version and another before the project reach 1.0.
You don't need to call ds.trigger('data', ds.data, ds.data) since bokeh property system will automatically detect your changes to the datasource fields inside your callback
You are designing/running your script as a bokeh script that uses a client session to the server (so you'll have a running instance of bokeh server somewhere and your script communicates with it). I'd suggest you to consider running your code as a Bokeh App instead, so your session and your code run inside the bokeh server instance. You can see more details about the difference at the bokeh server section on the official docs.
I'm trying to get real-time spectrum analyzer type plot in matplotlib. I've got some code working (with help from other posts on StackOverflow) as follows:
import time
import numpy as np
import matplotlib.pyplot as plt
plt.axis([0, 1000, 0, 1])
plt.ion()
plt.show()
i=0
np.zeros([1,500],'float')
lines=plt.plot(y[0])
while 1:
i=i+1
lines.pop(0).remove()
y = np.random.rand(1,100)
lines=plt.plot(y[0])
plt.draw()
The code works and I'm getting what I want, but there is a serious problem. The plot window would freeze after some time. I know the program is still running by inspecting the i variable (I'm running the code in Anaconda/Spyder so I can see the variables). However the plot window would show "Non responding" and if I terminate the python program in Spyder by ctrl+c, the plot window comes back to life and show the latest plot.
I'm out of wits here as how to further debug the issue. Anyone to help?
Thanks
I am not sure that adding plt.pause will entirely solve your issue. It may just take longer before the application crash. The memory used by your application seems to constantly increase over time (even after adding plt.pause). Below are two suggestions that may help you with your current issue:
Instead of removing/recreating the lines artists with each iteration with remove and plot, I would use the same artist throughout the whole animation and simply update its ydata.
I'll use explicit handlers for the axe and figure and call show and draw explicitly on the figure manager and canvas instead of going with implicit calls through pyplot, following the advices given in a post by tcaswell.
Following the above, the code would look something like this:
import numpy as np
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([0, 100, 0, 1])
y = np.random.rand(100)
lines = ax.plot(y)
fig.canvas.manager.show()
i=0
while 1:
i=i+1
y = np.random.rand(100)
lines[0].set_ydata(y)
fig.canvas.draw()
fig.canvas.flush_events()
I've run the above code for a good 10 minutes and the memory used by the application remained stable the whole time, while the memory used by your current code (without plt.pause) increased by about 30MiB over the same period.
To answer myself, I solved the issue by adding
plt.pause(0.01)
after the
plt.draw()
This probably allows the GUI to finish the drawing and clear the buffer somewhere (my guess) before the new data comes in.
I know I'm late to answer this question, but for your issue you could look into the "joystick" package. It is based on the line.set_data() and canvas.draw() methods, with optional axes re-scaling, hence most probably faster than removing a line and adding a new one. It also allows for interactive text logging or image plotting (in addition to graph plotting).
No need to do your own loops in a separate thread, the package takes care of it, just give the update frequency you wish. Plus the terminal remains available for more monitoring commands while live plotting, which is not possible with a "while True" loop.
See http://www.github.com/ceyzeriat/joystick/ or https://pypi.python.org/pypi/joystick (use pip install joystick to install)
try:
import joystick as jk
import numpy as np
import time
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the doc
"""
self._t0 = time.time() # initialize time
self.xdata = np.array([self._t0]) # time x-axis
self.ydata = np.array([0.0]) # fake data y-axis
# create a graph frame
self.mygraph = self.add_frame(jk.Graph(name="test", size=(500, 500), pos=(50, 50), fmt="go-", xnpts=100, xnptsmax=1000, xylim=(None, None, 0, 1)))
#_infinite_loop(wait_time=0.2)
def _generate_data(self): # function looped every 0.2 second to read or produce data
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.2 seconds
"""
# concatenate data on the time x-axis
self.xdata = jk.core.add_datapoint(self.xdata, time.time(), xnptsmax=self.mygraph.xnptsmax)
# concatenate data on the fake data y-axis
self.ydata = jk.core.add_datapoint(self.ydata, np.random.random(), xnptsmax=self.mygraph.xnptsmax)
self.mygraph.set_xydata(t, self.ydata)
t = test()
t.start()
t.stop()