I am looking for a way to plot realtime data line plot or scatter plots from python.
With the plots I want to monitor long-running loops when experimenting with algorithms with scientific computing. I.e. to help me answer the question: Are my results still improving with each iteration or can I cancel the loop?
I am looking for a quick and dirty method. I saw that with Bokeh and Dash one can program dashboards with realtime updates, but an awful lot of boilerplate code seems to be required just to get an updating plot.
Here is a simple "live streaming" example for Bokeh v1.3.0. You can run it with bokeh serve --show app.py
app.py:
from bokeh.plotting import figure, curdoc
from datetime import datetime
import random
plot = figure(plot_width = 1200, x_axis_type = 'datetime', tools = 'pan,box_select,crosshair,reset,save,wheel_zoom')
line = plot.line(x = 'time', y = 'value', line_color = 'black', source = dict(time = [datetime.now()], value = [random.randint(5, 10)]))
def update():
line.data_source.stream(dict(time = [datetime.now()], value = [random.randint(5, 10)]))
curdoc().add_root(plot)
curdoc().add_periodic_callback(update, 1000)
Related
I've been looking into using the Bokeh library to create animated data visualisations for some small projects I am working on. I feel that the gif format would be the best file format to export these visualisations in. It is widely used and they should be easy to share.
Is it possible (and advisable) to export bokeh animated plots in the gif format?
If so, will I need to make use of any additional tools to do this?
If not, is there a different file format that would be better suited to this?
I found this thread about potential options for creating gifs in Python, but I'm not sure if any of them are relevant to the Bokeh library. Programmatically generate video or animated GIF in Python?
Any help would be much appreciated. Many thanks.
Bokeh plot has a SaveTool which allows you to save the plot canvas manually in PNG format but this would be a lot of work for you to do. Alternatively you could automate this process by implementing Bokeh server app with update() function that updates the data_source property of your plot e.g. each second and saves a screenshot using export_png() function. Then you could use those images to build an animation e.g. using the Python lib you mentioned above.
This is an example script to run with bokeh serve --show app.py:
The content of app.py:
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure, curdoc
from datetime import datetime
from bokeh.io import export_png
import random
source = ColumnDataSource(dict(time = [datetime.now()], value = [random.randint(5, 10)]))
plot = figure(plot_width = 1200, x_axis_type = 'datetime', tools = 'pan,box_select,crosshair,reset,save,wheel_zoom')
plot.line(x = 'time', y = 'value', line_color = 'black', source = source)
counter = 0
def update():
global counter
new_source_data = dict(time = [datetime.now()], value = [random.randint(5, 10)])
source.stream(new_source_data)
counter = counter + 1
export_png(plot, filename = "plot_" + str(counter) + ".png")
curdoc().add_root(plot)
curdoc().add_periodic_callback(update, 1000)
To make this script work you need to have phantomjs installed on your system. So first you need to install nodejs and npm, then install phantomjs like this:
sudo npm install -g phantomjs-prebuilt
If you are using Mac then another option is to use QuickTime Player screen recording to make a movie and then convert it into an animated gif using FFMPEG like explained in this post
I recently picked up learning bokeh and I am completely lost making callbacks work.
What I would like to do is update the source using the PointDrawTool. It does update the plot and the table, but apparently it does not update the renderer or the source. This has me seriously confused and I'd appreciate some help.
What I have working is as follows:
from bokeh.models.glyphs import Circle
from bokeh.plotting import figure, show, output_notebook, Column, Row
from bokeh import events
from bokeh.models import DataTable, TableColumn, PointDrawTool, ColumnDataSource, CustomJS
output_notebook()
p = figure(width = 400, height = 600)
source = ColumnDataSource({
'x': [38], 'y': [-12], 'color': ['red']
})
renderer = p.circle(x='x', y='y',
source=source,
color='color',
size=10)
columns = [TableColumn(field="x", title="x"),
TableColumn(field="y", title="y"),
TableColumn(field='color', title='color')]
table = DataTable(source=source, columns=columns, editable=True, height=200)
draw_tool = PointDrawTool(renderers=[renderer],
empty_value='red')
p.add_tools(draw_tool)
p.toolbar.active_tap = draw_tool
show(Row(p,table))
Using your method of rendering the chart (show) it isn't possible to update the source for the chart (unless you write custom JavaScript to do so). In order to achieve this you would need to use the Bokeh server, as described here.
Basically, put all your code in a file called 'main.py', and then save this in a folder with your project name. Then in the terminal run
bokeh serve --show project_name
I haven't used the PointDrawTool before, but if it's a widget, you'll also need to write functions to program how the source data should be updated, using the on_click or on_change methods described here.
My goal is to create a bokeh script that shows the processor and memory usage of several machines in my network. The first script pulls the cpu and memory usage, and the bokeh script shows a time series plot of these stats over the last few seconds.
I have copy pasted some bokeh code (thanks internet) that updates two plots in a line based on random numbers every 500 seconds:
import numpy as np
from bokeh.plotting import figure, curdoc
from bokeh.driving import linear
from bokeh.layouts import layout
import random
tools = 'pan'
#linear()
def update(step):
# Instead of random numbers, fetch stats from another script
ds1.data['x'].append(step)
ds1.data['y'].append(random.randint(0,100))
ds2.data['x'].append(step)
ds2.data['y'].append(random.randint(0,100))
ds1.trigger('data', ds1.data, ds1.data)
ds2.trigger('data', ds2.data, ds2.data)
# don't get this yet, but #linear is a decorator. Instead of having #linear(), we could also have
# update = linear(update)
p = figure(plot_width=400, plot_height=400)
r1 = p.line([], [], color="firebrick", line_width=2, legend='line1')
r2 = p.line([], [], color="navy", line_width=2, legend='line2')
ds1 = r1.data_source
ds2 = r2.data_source
curdoc().add_root( p )
# Add a periodic callback to be run every 500 milliseconds
curdoc().add_periodic_callback(update, 500)
Let's assume the first script pulls tuples of memory/cpu usage every 500 ms, how would I get the bokeh script to fetch that information? Once I have that information, I'll be set.
The only idea I have right now is to write the output of the first script to a dict/json/h5 file, and have the bokeh script read it and append that data to the plot. I'm wondering if there is a better way.
Since the update script is just grabbing some numbers, your best bet is probably wrapping that thing inside an update function, processing those numbers so they fit a ColumnDataSource, then update the main CDS, which will then update the plot. Something like:
def update():
# Do stuff to get new data
numbers = get_usage()
# Process new data into a CDS friendly format
data1, data2 = process_numbers(numbers)
# Update plot_source, this being whatever source is
r1_source.data = data1
r2_source.data = data2
curdoc().add_periodic_callback(update, 500)
From the last days, I have been trying to use Bokeh to plot real-time data and display on a .html in order to be embeed in a webpage. I have sucessuly adapted one of the bokeh examples to my needs. I am using a buffer of 50 elements on the plot and I am noting the following behaviour:
1) In case I run the script and go to the browser the x_range fully adapts to incomming data and everything works correctly
2) If I click on "Refresh" on the browser the x_range stops to adapt to incoming data and freezes to the last value.
I tried to force the x_axis to initial and end values but the visualization behaves poorly.
I think I am not correctly understanding what does the "Refresh" hit impacts my code and how I can workaround this issue.
""" To view this example, first start a Bokeh server:
bokeh serve --allow-websocket-origin=localhost:8000
And then load the example into the Bokeh server by
running the script:
python animated.py
in this directory. Finally, start a simple web server
by running:
python -m SimpleHTTPServer (python 2)
or
python -m http.server (python 3)
in this directory. Navigate to
http://localhost:8000/animated.html
"""
from __future__ import print_function
import io
from numpy import pi, cos, sin, linspace, roll
from bokeh.client import push_session
from bokeh.embed import server_session
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource
fa = open('Accelerometer.txt', 'r')
source = ColumnDataSource(data=dict(x=[], y=[]))
fg = figure(width=250, plot_height=250, title="RT-Test")
fg.line(x='x', y='y', color="olive", source=source)
fg.x_range.follow = "end"
# Visualization scale and aesthetics
fg.xgrid.grid_line_color = None
fg.ygrid.grid_line_color = None
fg.background_fill_color = "snow"
# add the plot to curdoc
curdoc().add_root(fg)
# open a session which will keep our local doc in sync with server
session = push_session(curdoc())
html = """
<html>
<head></head>
<body>
%s
</body>
</html>
""" % server_session(fg, session_id=session.id, relative_urls=False)
with io.open("animated.html", mode='w+', encoding='utf-8') as f:
f.write(html)
print(__doc__)
def update():
line = fa.readline().split(',')
x = float(line[0])
y = float(line[1])
print(x, y)
# construct the new values for all columns, and pass to stream
new_data = dict(x=[x], y=[y])
source.stream(new_data, rollover=50)
curdoc().add_periodic_callback(update, 100)
session.loop_until_closed() # run forever
This kind of usage of the Bokeh server, with the actual code running in a separate process and calling session.loop_until_closed, is discouraged in the strongest terms. In the next release, all of the examples of this sort will be deleted, and mentions of this approach removed from the docs. This usage is inherently inferior in many ways, as outlined here, and I would say that demonstrating it so prominently for so long was a mistake on our part. It is occasionally useful for testing, but nothing else.
So what is the good/intended way to use the Bokeh server? The answer is to have a Bokeh app run in the Bokeh server itself, unlike the code above. This can be done in a variety of ways, but one common way os to wirte a simple script, then execute that script with
bokeh serve -show myapp.py
I don't have access to your "Accelerate.py" dataset, but a rough pass at updating your code would look like:
# myapp.py
from numpy import pi, cos, sin, linspace, roll
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource
fa = open('Accelerometer.txt', 'r')
source = ColumnDataSource(data=dict(x=[], y=[]))
fg = figure(width=250, plot_height=250, title="RT-Test")
fg.line(x='x', y='y', color="olive", source=source)
fg.x_range.follow = "end"
fg.xgrid.grid_line_color = None
fg.ygrid.grid_line_color = None
fg.background_fill_color = "snow"
curdoc().add_root(fg)
def update():
line = fa.readline().split(',')
x = float(line[0])
y = float(line[1])
# construct the new values for all columns, and pass to stream
new_data = dict(x=[x], y=[y])
source.stream(new_data, rollover=50)
curdoc().add_periodic_callback(update, 100)
Now if you run this script with the bokeh serve command, then any refresh will get you a brand new session of this app. It's also worth nothing that code written in this way is considerably simpler and shorter.
These kinds of apps can be embedded in Jupyter notebooks, Flask and other web-apps, or made into "regular" python scripts run with python instead of bokeh serve. For more information see Running a Bokeh Server in the User's Guide.
Is it possible to show and update Pandas plots in Bokeh without using show()? Are there any examples of this online? I can't seem to find any. For example, something like:
def bar_plot(fig, source):
p = pd.DataFrame()
p = p.from_dict(source.data)
fig = p.plot.bar()
return fig
def update_data():
data = source.data
data['y'] = random.sample(range(0,100),len(data['y']))
source.data = data
button.on_click(update_data)
source = ColumnDataSource(data)
fig = bar_plot(fig, source)
layout = layout([[button,fig]])
curdoc().add_root(layout)
Pandas' built-in .plot method uses Matplotlib to generate images. The Bokeh server has no way of synchronizing or updating MPL plots across the Python/JS boundary. The Bokeh server can only show and update plots created using native Bokeh APIs (i.e. you can create a bar plot from your data frame using Figure.vbar or similar Bokeh functions).