Develop Reference

Plotly Weekly & Monthly Range selector buttons not working on time series data

I am having last 1 year time series financial data. I removed weekends as no data is present.
I created plotly viz, 2w,1m range selector buttons are not showing correct time range.
While YTD,6m,all showing correct data.
2w button showing 2 week forward days but there is no data, i want to see last two week and 1m data.
Reproducable Code-
# !pip install investpy
import pandas as pd
import investpy
import datetime
import plotly.graph_objects as go
import plotly.figure_factory as ff
import plotly.express as px
today = datetime.now() #Today Datetime
today_fut = today.strftime("%Y,%m,%d") #Converting today date to string format "%Y,%m,%d"
today_fut = datetime. strptime(today_fut, '%Y,%m,%d').date() #Converting today's date to "date" format for NSE Tools library
today = today.strftime("%d/%m/%Y") #Converting today date to string format "%d/%m/%Y"
one_year= datetime.today() - timedelta(days=370) #Subrtracting todays date with 370 Days
one_year = one_year.strftime("%d/%m/%Y") #Converting into string format "%d/%m/%Y"
df = investpy.get_index_historical_data(index="Nifty 50",country="India",from_date=str(one_year),to_date= str(today))
print("Investpy NF Dataframe",df.tail())
bnf = investpy.get_index_historical_data(index="Nifty Bank",country="India",from_date=str(one_year),to_date= str(today))
print("Investpy BNF Dataframe",bnf.tail())
fig = go.Figure(data = [ go.Scatter(x = df.index,y = df['Close'],line=dict(color = 'Steelblue',width=2),mode='lines+markers',name = 'NIFTY'),
go.Scatter(x = bnf.index,y = bnf['Close'],line=dict(color = 'yellowgreen',width=2),mode='lines+markers',name = 'BANK NIFTY'),
])
fig.update_layout(
title='NF and BNF',template = 'plotly_dark',xaxis_tickformat = ' %d %B (%a)<br> %Y',
yaxis_title='NF & BNF',yaxis_tickformat= "000",yaxis_side = 'right',xaxis_title='Date',legend = dict(bgcolor = 'rgba(0,0,0,0)'))
layout = go.Layout(showlegend=True)
##https://plotly.com/python/legend/
fig.update_layout(legend=dict(
yanchor="top",
y=0.99,
xanchor="left",
x=0.01 ))
#hide weekends
fig.update_xaxes( rangeslider_visible=True, rangebreaks=[
dict(bounds=["sat", "mon"]) ])
##https://plotly.com/python/legend/
fig.update_layout(legend=dict(
orientation="h",
yanchor="bottom",
y=1.02,
xanchor="right",
x=1
))
config = dict({'scrollZoom': False})
fig.update_layout(
xaxis=dict(rangeselector=dict(buttons=list([
dict(count=14,label="2w",step="day",stepmode="todate"),
dict(count=1,label="1m",step="month",stepmode="backward"),
dict(count=3,label="3m",step="month",stepmode="backward"),
dict(count=6,label="6m",step="month",stepmode="backward"),
dict(count=1,label="YTD",step="year",stepmode="todate"),
dict(step="all") ])),rangeslider=dict(visible=True),type="date"))
fig.update_layout(
xaxis_rangeselector_font_color='white',
xaxis_rangeselector_activecolor='red',
xaxis_rangeselector_bgcolor='green',
)
fig.show()
pio.write_html(fig, file='wrong_range_selecor_for-2w&1m_button.html', auto_open=True)
Pls let me know , if there is any bug in plotly range selector or I am doing something wrong in xaxis=dict(rangeselector=dict(buttons=list([xaxis=dict(rangeselector=dict(buttons=list([ code.
Snap- When I click on 2w but it show future dates, where no data is present in dataframe.
1m button snap-

Yes it seems like a bug with the range selector (type='date'). It occurs when setting the scatter plot mode to 'markers' or 'lines+markers' : in fact, using markers leads to the addition of an extra range added to the xaxis, extending the default date range from the input data.
So when you click on 2w, the slider goes from that out-of-range date (in the future) back to 2w before that date, the same happens with the other ranges actually, not only 1m, but it is less noticeable for bigger ones.
You could set a fixed range to circumvent the problem using rangeslider_range and range :
fig.update_xaxes(
rangeslider_visible=True,
rangebreaks=[dict(bounds=["sat", "mon"])],
range=[df.index[0], df.index[-1]],
rangeslider_range=[df.index[0], df.index[-1]]
)
...until you click on "all". The problem is that the range selector button step='all' will always reset the layout to use the extended range.
This does not happen when you set the scatter mode to just mode='lines'. In this case the default/'all' slider range fits the data.
So I guess the quick fix is to remove markers, the long one is to use Dash callbacks to have a full control of what the buttons do.

Plotly: How to style a plotly figure so that it doesn't display gaps for missing dates?

I have a plotly graph of the EUR/JPY exchange rate across a few months in 15 minute time intervals, so as a result, there is no data from friday evenings to sunday evenings.
Here is a portion of the data, note the skip in the index (type: DatetimeIndex) over the weekend:
Plotting this data in plotly results in a gap over the missing dates Using the dataframe above:
import plotly.graph_objs as go
candlesticks = go.Candlestick(x=data.index, open=data['Open'], high=data['High'],
low=data['Low'], close=data['Close'])
fig = go.Figure(layout=cf_layout)
fig.add_trace(trace=candlesticks)
fig.show()
Ouput:
As you can see, there are gaps where the missing dates are. One solution I've found online is to change the index to text using:
data.index = data.index.strftime("%d-%m-%Y %H:%M:%S")
and plotting it again, which admittedly does work, but has it's own problem. The x-axis labels look atrocious:
I would like to produce a graph that plots a graph like in the second plot where there are no gaps, but the x-axis is displayed like as it is on the first graph. Or at least displayed in a much more concise and responsive format, as close to the first graph as possible.
Thank you in advance for any help!

Even if some dates are missing in your dataset, plotly interprets your dates as date values, and shows even missing dates on your timeline. One solution is to grab the first and last dates, build a complete timeline, find out which dates are missing in your original dataset, and include those dates in:
fig.update_xaxes(rangebreaks=[dict(values=dt_breaks)])
This will turn this figure:
Into this:
Complete code:
import plotly.graph_objects as go
from datetime import datetime
import pandas as pd
import numpy as np
# sample data
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/finance-charts-apple.csv')
# remove some dates to build a similar case as in the question
df = df.drop(df.index[75:110])
df = df.drop(df.index[210:250])
df = df.drop(df.index[460:480])
# build complete timepline from start date to end date
dt_all = pd.date_range(start=df['Date'].iloc[0],end=df['Date'].iloc[-1])
# retrieve the dates that ARE in the original datset
dt_obs = [d.strftime("%Y-%m-%d") for d in pd.to_datetime(df['Date'])]
# define dates with missing values
dt_breaks = [d for d in dt_all.strftime("%Y-%m-%d").tolist() if not d in dt_obs]
# make fiuge
fig = go.Figure(data=[go.Candlestick(x=df['Date'],
open=df['AAPL.Open'], high=df['AAPL.High'],
low=df['AAPL.Low'], close=df['AAPL.Close'])
])
# hide dates with no values
fig.update_xaxes(rangebreaks=[dict(values=dt_breaks)])
fig.update_layout(yaxis_title='AAPL Stock')
fig.show()

Just in case someone here wants to remove gaps for outside trading hours and weekends,
As shown below, using rangebreaks is the way to do it.
fig = go.Figure(data=[go.Candlestick(x=df['date'], open=df['Open'], high=df['High'], low=df['Low'], close=df['Close'])])
fig.update_xaxes(
rangeslider_visible=True,
rangebreaks=[
# NOTE: Below values are bound (not single values), ie. hide x to y
dict(bounds=["sat", "mon"]), # hide weekends, eg. hide sat to before mon
dict(bounds=[16, 9.5], pattern="hour"), # hide hours outside of 9.30am-4pm
# dict(values=["2020-12-25", "2021-01-01"]) # hide holidays (Christmas and New Year's, etc)
]
)
fig.update_layout(
title='Stock Analysis',
yaxis_title=f'{symbol} Stock'
)
fig.show()
here's Plotly's doc.

thanks for the amazing sample! works on daily data but with intraday / 5min data rangebreaks only leave one day on chart
# build complete timepline
dt_all = pd.date_range(start=df.index[0],end=df.index[-1], freq="5T")
# retrieve the dates that ARE in the original datset
dt_obs = [d.strftime("%Y-%m-%d %H:%M:%S") for d in pd.to_datetime(df.index, format="%Y-%m-%d %H:%M:%S")]
# define dates with missing values
dt_breaks = [d for d in dt_all.strftime("%Y-%m-%d %H:%M:%S").tolist() if not d in dt_obs]

To fix problem with intraday data, you can use the dvalue parameter of rangebreak with the right ms value.
For example, 1 hour = 3.6e6 ms, so use dvalue with this value.
Documentation here : https://plotly.com/python/reference/layout/xaxis/
fig.update_xaxes(rangebreaks=[dict(values=dt_breaks, dvalue=3.6e6)])

Why is my plotly graph plotting datetime on x axis as exponents?

I am trying to plot a graph with dates (pandas datetime) on the x axis. However, they are plotting in numerical format instead (showing up as exponents).
Example of dates:
0 2014-05-01
1 2014-05-02
2 2014-05-03
3 2014-05-04
4 2014-05-05
Name: date, dtype: datetime64[ns]
Code for plotly:
trace1 = go.Scatter(x = df_iso_h.date,
y=del18_f_hum,
mode = 'markers')
data = [trace1]
py.iplot(data)
My x-axis:
Not sure how to fix this??

You need to add layout and specify parameter xaxis in it. Such as here.
So try this:
# Create trace
trace1 = go.Scatter(x = df_iso_h.date,
y=del18_f_hum,
mode = 'markers')
# Add trace in data
data = [trace1]
# Create layout. With layout you can customize plotly plot
layout = dict(title = 'Scatter',
# Add what you want to see at xaxis
xaxis = df_iso_h.date
)
#Do not forget added layout to fig!
fig = dict(data=data, layout=layout)
# Plot scatter
py.iplot(data, filename="scatterplot")
This should help you.
Update: Try to convert datetime column with strftime (new column should be in object format!):
df_iso_h["date"] = df_iso_h["date"].dt.strftime("%d-%m-%Y")
If not worked, add this column in xaxis. Maybe plotly do not support datetime format yyyy-mm-dd... Notice, you xaxis will be looks like 01-05-2014

Figured it out... Plotly does not take pandas datetime, so I had to convert my pandas datetime to python datetime.datetime or datetime.date.

It seems that this was a regression introduced in plotly.py Version 3.2.0 and has been fixed in Version 3.2.1
You can now simply pass the pandas datetime column to plotly and it will handle the proper conversion for you like in the past.
See https://github.com/plotly/plotly.py/issues/1160

Datetime on an X Axis in Bokeh

I'm having trouble understanding how datetime works in Bokeh.
My code is outputting milliseconds on the x axis, which seems to be the default if the axis type is 'datetime', but I would like to have it separated by every 5 years.
How can I accomplish this in Bokeh? The dates are before 1950 so I can't use a timestamp. When I exclude the datetime axis type, it displays every single value as a tick on the x axis. When I try to change the inputted data as a datetime object, Bokeh rejects it.
I've read other posts on the subject, as well as Bokeh's documentation on
DatetimeTickFormatter, which doesn't seem to be working for me either.
Datetime axis in Bokeh
Bokeh FixedTicker with Custom Datetime/Timestamp values
https://docs.bokeh.org/en/latest/docs/reference/models/formatters.html?highlight=formatters#module-bokeh.models.formatters
All my code for the plot is below:
r = requests.get('https://raw.githubusercontent.com/FreeCodeCamp/ProjectReferenceData/master/GDP-data.json')
data = json.loads(r.text)
chart_data = {
'date': [a[0] for a in data['data']], #this is the problem data
'gdp': [a[1] for a in data['data']],
}
source = ColumnDataSource(chart_data)
hover = HoverTool(
tooltips=[
('GDP', '#gdp{$0,0.00}'),
('Date', '#date{%Y - %B}'),
],
formatters={
'date' : 'datetime',
}
)
bar = figure(x_range=chart_data['date'],
x_axis_type='datetime',
plot_height=750,
plot_width=1000,
tools='wheel_zoom, reset, save',
title=data['source_name'])
bar.vbar(x='date', top='gdp', width=0.9, source=source)
bar.add_tools(hover)
#visual settings
bar.xaxis.formatter = DatetimeTickFormatter(years = ['%Y'])
bar.xaxis.major_label_orientation = 'vertical'
bar.xaxis.minor_tick_line_color = None
bar.xgrid.grid_line_color = None
bar.yaxis.formatter = NumeralTickFormatter(format='$0,0.00')
bar.ygrid.grid_line_color = None
show(bar)
Thank you for the help, this has been driving me crazy!

Thanks to help from bigreddot. My issue was that my plot was acting as a Categorical range due to me setting the x_range property, instead of defaulting to datetime.

mpld3 does not display dates on x axis correctly

I am plotting a large dataset from a database using matplotlib and I use mpld3 to pass the figure to the browser. On the x-axis there are dates. The issue here is that while plotting without the mpld3 works perfect, when I use it, the dates don't appear correctly.
Here is my code:
date1 = '2015-04-22 20:28:50'
date2 = '2015-04-23 19:42:09'
db = Base('monitor').open()
result_set = db.select(['MeanVoltage','time'],"time>=start and time<=stop", start=date1, stop=date2)
V = [float(record.MeanVoltage) for record in result_set if record != 0]
Date = [str(record.time) for record in result_set]
dates = [datetime.datetime.strptime(record, '%Y-%m-%d %H:%M:%S') for record in Date]
dates = matplotlib.dates.date2num(dates)
fig, ax = plt.subplots()
plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%m/%d/%Y %H:%M:%S' ))
plt.gcf().autofmt_xdate()
ax.plot(dates,V)
#mpld3.fig_to_html(fig)
#mpld3.show(fig)
plt.show()
that shows the plot perfectly like this:
.
Now, if I comment out this line only:
plt.show()
and uncomment these two:
mpld3.fig_to_html(fig)
mpld3.show(fig)
the figure appears in the browser like this:
As you can see, the only issue is how the dates appear in the x-axis.
Is there any way to overcome it?

Before creating the HTML figure, add the following line to specify that it is a date axis:
ax.xaxis_date()

The answer above is correct.
If you are exclusively passing through dates, for example
df["Date"][0] = "2018-11-23"
Then you can also pass that through in the format native mpl format below, without making an ordinal value by using date2num.
df["Date"] = [dt.datetime.strptime(d, '%Y-%m-%d') for d in df["Date"]]
ax.plot(df["Dates"].tolist(), some_y_value_list)