Develop Reference

How to remove points from a dataframe based on a selected area on a plot

I have some experimental data that is often flawed with artifacts exemplified with something like this:
I need a quick way to manually select these random spikes and remove them from datasets.
I figured that any plotting library with a focus on interactive plots should have an easy way to do this but so far I keep struggling with finding a simple way to do what I want.
I'm a Matplotlib/Seaborn guy and this calls for interactive solution. I briefly checked Plotly, Bokeh and Altair and decided to go with the first one. My first attempt looks like this:
import pandas as pd
import plotly.graph_objects as go
from ipywidgets import interactive, HBox, VBox, Button
url='https://drive.google.com/file/d/1hCX8Bn_y30aXVN_TyHTTx015u44pO9yB/view?usp=sharing'
url='https://drive.google.com/uc?id=' + url.split('/')[-2]
df = pd.read_csv(url, index_col=0)
f = go.FigureWidget()
for col in df.columns[-1:]:
f.add_scatter(x = df.index, y=df[col], mode='markers+lines',
selected_marker=dict(size=5, color='red'),
marker=dict(size=1, color='lightgrey', line=dict(width=1, color='lightgrey')))
t = go.FigureWidget([go.Table(
header=dict(values=['selector range'],
fill = dict(color='#C2D4FF'),
align = ['left'] * 5),
cells=dict(values=['None selected' for col in ['ID']],
fill = dict(color='#F5F8FF'),
align = ['left'] * 5)
)])
def selection_fn(trace,points,selector):
t.data[0].cells.values = [selector.xrange]
def update_axes(dataset):
scatter = f.data[0]
scatter.x = df.index
scatter.y = df[dataset]
f.data[0].on_selection(selection_fn)
axis_dropdowns = interactive(update_axes, dataset = df.columns)
button1 = Button(description="Remove points")
button2 = Button(description="Reset")
button3 = Button(description="Fit data")
VBox((HBox((axis_dropdowns.children)), HBox((button1, button2, button3)), f,t))
Which gives:
So I managed to get Selector Box x coordinates (and temporarily print them inside the table widget). But what I couldn't figure out is how to easily bind a function to button1 that would take as an argument Box Selector coordinates and remove selected points from a dataframe and replot the data. So something like this:
def on_button_click_remove(scatter.selector.xrange):
mask = (df.index >= scatter.selector.xrange[0]) & (df.index <= scatter.selector.xrange[1])
clean_df = df.drop(df.index[mask])
scatter(data = clean_df...) #update scatter plot
button1 = Button(description="Remove points", on_click = on_button_click_remove)
I checked https://plotly.com/python/custom-buttons/ but I am still not sure how to use it for my purpose.

I suggest to use Holoviews and Panel.
They are high level visualization tools that facilitate the creation and control of low level bokeh, matplotlib or plotly figures.
Here are an example:
import panel as pn
import holoviews as hv
import pandas as pd
from bokeh.models import ColumnDataSource
# This example use bokeh as backend.
# You can try plotly or matplotlib with minor modification on the codes below.
# For example you can use on_selection callback from Plotly
# https://plotly.com/python/v3/selection-events/
hv.extension('bokeh')
display( pn.extension( ) ) # activate panel
df=pd.read_csv('spiked_data.csv',index_col=0).reset_index()
pt = hv.Points(
data=df, kdims=['index', 'A' ]
).options( marker='x', size=2,
tools=['hover', 'box_select', 'lasso_select', 'reset'],
height=250, width=600
)
fig = hv.render(pt)
source = fig.select({'type':ColumnDataSource})
bt = pn.widgets.Button(name='remove selected')
def rm_sel(evt):
i = df.iloc[source.selected.indices].index # get index to delete
df.drop(i, inplace=True, errors='ignore') # modify dataframe
source.data = df # update data source
source.selected.indices=[] # clear selection
pn.io.push_notebook(app) # update figure
bt.on_click(rm_sel)
app=pn.Column(fig,'Click to delete the selected points', bt)
display(app)
A related example can be found in this SO answer

Hvplot : How to delete default tools

I'm using HvPlot and it works perfectly but I don't know how to delete the default tools 'pan', 'wheel_zoom' and 'box_zoom'.
My code for the HvPlot is :
points = df.hvplot.line(x='x', y='y',
grid=True,
tools=['xpan', # move along x
'xwheel_pan', # move along x with wheel
'xwheel_zoom', # zoom on x with wheel
'xzoom_in', # zoom in on x
'xzoom_out', # zoom out on x
'crosshair', # show where the mouse is on axis
'xbox_zoom', # zoom on selection along x
'undo', # undo action
'redo'], # redo action
width=1200, height=550,
aggregator='any',
datashade=True)
I have this figure :

You can use .opts(default_tools=[]) to get rid of the default tools:
Example code:
import numpy as np
import pandas as pd
import hvplot.pandas
df = pd.DataFrame({
'x': np.random.normal(size=50),
'y': np.random.normal(size=50),
})
scatter_plot = df.hvplot.scatter(
x='x', y='y',
tools=['tap','box_select'])
# manually specifying the default tools gets rid of any preset default tools
# you also just use an empty list here
scatter_plot.opts(default_tools=['wheel_zoom'])
See also this question + answers:
How to control (active) tools in holoviews with bokeh backend
Resulting plot with only the specified tools and default tools:

Bokeh: disable Auto-ranging while using Edit Tools

I've included the PolyDrawTool in my Bokeh plot to let users circle points. When a user draws a line near the edge of the plot the tool expands the axes which often messes up the shape. Is there a way to freeze the axes while a user is drawing on the plot?
I'm using bokeh 1.3.4
MRE:
import numpy as np
import pandas as pd
import string
from bokeh.io import show
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource, LabelSet
from bokeh.models import PolyDrawTool, MultiLine
def prepare_plot():
embedding_df = pd.DataFrame(np.random.random((100, 2)), columns=['x', 'y'])
embedding_df['word'] = embedding_df.apply(lambda x: ''.join(np.random.choice(list(string.ascii_lowercase), (8,))), axis=1)
# Plot preparation configuration Data source
source = ColumnDataSource(ColumnDataSource.from_df(embedding_df))
labels = LabelSet(x="x", y="y", text="word", y_offset=-10,x_offset = 5,
text_font_size="10pt", text_color="#555555",
source=source, text_align='center')
plot = figure(plot_width=1000, plot_height=500, active_scroll="wheel_zoom",
tools='pan, box_select, wheel_zoom, save, reset')
# Configure free-hand draw
draw_source = ColumnDataSource(data={'xs': [], 'ys': [], 'color': []})
renderer = plot.multi_line('xs', 'ys', line_width=5, alpha=0.4, color='color', source=draw_source)
renderer.selection_glyph = MultiLine(line_color='color', line_width=5, line_alpha=0.8)
draw_tool = PolyDrawTool(renderers=[renderer], empty_value='red')
plot.add_tools(draw_tool)
# Add the data and labels to plot
plot.circle("x", "y", size=0, source=source, line_color="black", fill_alpha=0.8)
plot.add_layout(labels)
return plot
if __name__ == '__main__':
plot = prepare_plot()
show(plot)

The PolyDrawTool actually updates a ColumnDataSource to drive a glyph that draws what the users indicates. The behavior you are seeing is a natural consequence of that fact, combined with Bokeh's default auto-ranging DataRange1d (which by default also consider every glyph when computing the auto-bounds). So, you have two options:
Don't use DataRange1d at all, e.g. you can provide fixed axis bounds when you call figure:
p = figure(..., x_range=(0,10), y_range=(-20, 20)
or you can set them after the fact:
p.x_range = Range1d(0, 10)
p.y_range = Range1d(-20, 20)
Of course, with this approach you will no longer get any auto-ranging at all; you will need to set the axis ranges to exactly the start/end that you want.
Make DataRange1d be more selective by explicitly setting its renderers property:
r = p.circle(...)
p.x_range.renderers = [r]
p.y_range.renderers = [r]
Now the DataRange models will only consider the circle renderer when computing the auto-ranged start/end.

Change legend position using holoviews / hvplot

Hvplot has default the position of the legend on the right outside of the plot.
How can I change this default legend position?
import numpy as np
import pandas as pd
import hvplot
import hvplot.pandas
import holoviews as hv
data = np.random.normal(size=[50, 2])
df = pd.DataFrame(data, columns=['a', 'b'])
df.hvplot.line()

You can change the legend position to top left by adding .opts(legend_position='top_left') to your code.
df.hvplot.line().opts(legend_position='top_left')
If you would like to position your legend inside your plot, you can choose from the following options:
['top_right', 'top_left', 'bottom_left', 'bottom_right']
If you want to place your legend outside your plot, you can choose from these options:
['right', 'left', 'top', 'bottom']

Holoviews AdjointLayout with Bokeh Widgets

I am trying to append an AdjointLayout of a Scatter plot with two supporting histograms to a Bokeh dashboard. However, whenever trying to incorporate the two in a single row, the Bokeh widgets encounter display issues and the AdjointLayout never scales. Is this the current expected behavior or is here a different approach I need to take to currently accomplish this?
Minimal Example of the problem:
import numpy as np
import pandas as pd
import holoviews as hv
from bokeh.layouts import layout
from bokeh.models import Select
from bokeh.io import curdoc
renderer = hv.renderer('bokeh').instance(mode='server')
np.random.seed(10)
data = np.random.rand(100,4)
opts = {}
opts['color_index'] = 2
opts['size_index'] = 3
opts['scaling_factor'] = 50
points = hv.Points(data, vdims=['z', 'size']).opts(plot=opts)
fields = ['berry', 'cherry', 'dairy']
x = Select(title='X-Axis:', value=fields[0], options=fields)
y = Select(title='Y-Axis:', value=fields[1], options=fields)
dashboard = points + points[0.3:0.7, 0.3:0.7].hist()
app = renderer.get_plot(dashboard).state
dashboard = layout([
[[x, y], app],
])
curdoc().add_root(dashboard)
Using Bokeh 0.13.0 and Holoviews 1.10.5