result
year Month Min_days Avg_days Median_days Count MonthName-Year
2015 1 9 12.56 10 4 2015-Jan
2015 2 10 13.67 9 3 2015-Feb
........................................................
2016 12 12 15.788 19 2 2016-Dec
and so on...
I wish to create a line plot plotting min_days, avg_days, median_days, count according to month and year say. Code used for that(which works perfectly):
import matplotlib.pyplot as plt
result=freq_start_month_year_to_date_1(df,'Jan','2015','Dec','2019')
#Visualisations
fig, ax = plt.subplots()
for col in ["Min_days", "Median_days", "Count",'Target_days_before_customer_dead']:
ax.plot(result["Month Name-Year"], result[col], label=col)
ax.legend(loc="best")
ax.tick_params(axis="x", rotation=30)
I am getting a plot . The only issue is that the x axis is too crowded and all the values 2015-Jan, 2015-Feb etc are overlapping so nothing is readable in the x axis, it looks like black scrabbling...I am unable to increase the size of the plot.
I tried below code but that too did not work
fig, ax = plt.subplots(2,2, figsize=(20,20))
Using the above code I got 4 empty sub plots
The problem is you preformatted your x-axis as string and thus robbed matplotlib of the chance to apply its own formatter. matplotlib tried to cram all the strings into the axis so you can never make it wide enough to hold all the labels.
Create a new date column and use it to form your x axis:
from matplotlib import dates as mdates
# The new column to be used as x axis
result['Date'] = pd.to_datetime(result[['Year', 'Month']].assign(Day=1))
# Plot the data
fig, ax = plt.subplots(figsize=(10, 2))
for col in ['Min_days', 'Median_days', 'Count', 'Target_days_before_customer_dead']:
ax.plot(result['Date'], result[col], label=col)
years = mdates.YearLocator() # only print label for the years
months = mdates.MonthLocator() # mark months as ticks
years_fmt = mdates.DateFormatter('%Y')
ax.xaxis.set_major_locator(years)
ax.xaxis.set_minor_locator(months)
ax.xaxis.set_major_formatter(years_fmt)
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))
Result (with random data):
Related
I'm attempting to plot a pandas stacked bar plot with the x axis showing Months on the major ticks, or years on Jan 1, ideally with small ticks identifying the weeks but with no label.
I have a dataset with a datetime index that was then grouped by week and then I plot that dataset. If I don't attempt to control the settings the dates show up but are vertical and don't fit. So I used the set formatter to fix that but then the axes changed to 1970 as if following an index number instead of date. If I replace the pandas plotting with a regular bar chart, the "ConciseDateFormatter" works as desired/expected. But I wanted to use stacked with pandas as creating a regular stacked bar chart is a pain. I don't understand why I can't control pandas axes like I can a regular plot.
One thing I notice is that the index is shown as an object. If I convert it to to_datetime() it then adds 00:00 for times that I don't want on the axes or my data.
My data is a simple set of weekly random data:
date A B C D
3/20/2022 1.540765154 0.504616419 1.543679189 2.952934623
3/27/2022 1.781135128 4.594966635 4.799026389 3.499803401
4/3/2022 0.254059207 0.69835265 0.323039575 1.628138491
4/10/2022 3.112760301 0.287056897 4.372938373 0.130817579
4/17/2022 0.497273044 0.913246096 1.296612207 1.250610278
4/24/2022 1.370087689 3.124985109 4.322253295 4.49571603
5/1/2022 3.952629538 3.976896924 1.679311114 1.265443147
5/8/2022 3.470328161 1.266161308 3.990502436 1.364929959
5/15/2022 2.296588269 4.639761391 0.04685036 1.438471692
5/22/2022 3.443458637 2.66592719 0.968656871 2.349325343
5/29/2022 1.820278464 4.794211675 2.435710815 2.156110694
6/5/2022 4.328825266 0.049132356 1.842839099 3.665701299
6/12/2022 0.184631564 0.412976815 4.787477069 4.80052839
6/19/2022 4.846734385 3.471474741 1.808871854 2.440013553
6/26/2022 1.612870444 0.70191857 3.55713114 1.438699834
7/3/2022 2.896859156 4.025996887 0.209608767 4.174881655
Code:
import datetime
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import numpy as np
import pandas as pd
maxval = 200
values = ['A','B','C','D']
cum = [v + '_CUM' for v in values]
df = pd.read_csv('test_data.csv', index_col='date', parse_dates=True,
infer_datetime_format=True)
#df.index = pd.to_datetime(df.index.date).strftime("%b %d")
df = df.join(df.cumsum(), rsuffix="_CUM")
df = df.join(df[cum]/maxval * 100, rsuffix="_LIFE")
fig, axs = plt.subplots(nrows=2, ncols=1, sharex=False, squeeze=False,
facecolor='white')
axs = axs.flatten()
ax = axs[0]
df[values].plot.bar(ax=ax, grid=True, stacked=True, legend=True)
ax.xaxis.set_major_locator(mdates.MonthLocator())
ax.xaxis.set_major_formatter(mdates.ConciseDateFormatter
(ax.xaxis.get_major_locator()))
# ax.xaxis.set_tick_params(rotation = 0)
plt.show(block=False)
I have one-year data and I want to plot their seasonal patterns. SO I just created sub data for each season. but my winter data plot has a gap. It cannot plot three months in sequence.
Here is my data:
winter = pd.concat([countData19_gdf.loc['2019-12-01':'2019-12-31'], countData19_gdf.loc['2019-01-01':'2019-02-28']])
winter= winter.sort_index()
min_count = countData19_gdf['volume'].min()
max_count = countData19_gdf['volume'].max() + 20
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(16,10))
line_width = 2
ax[0,0].plot(winter.resample('d').mean()['volume'].index, winter.resample('d').mean()['volume'], c='blue', lw=line_width);
ax[0,1].plot(countData19_gdf.loc['2019-03-01': '2019-05-31'].resample('d').mean()['volume'].index, countData19_gdf.loc['2019-03-01': '2019-05-31'].resample('d').mean()['volume'] ,c='orange',lw=line_width);
ax[1,0].plot(countData19_gdf.loc['2019-06-01': '2019-08-31'].resample('d').mean()['volume'].index, countData19_gdf.loc['2019-06-01': '2019-08-31'].resample('d').mean()['volume'], c='green', lw=line_width);
ax[1,1].plot(countData19_gdf.loc['2019-09-01': '2019-11-30'].resample('d').mean()['volume'].index, countData19_gdf.loc['2019-09-01': '2019-11-30'].resample('d').mean()['volume'], c='brown', lw=line_width);
ax[0,0].title.set_text('Winter')
ax[0,1].title.set_text('Spring')
ax[1,0].title.set_text('Summer')
ax[1,1].title.set_text('Fall')
for ax in [ax[0,1], ax[1,0], ax[1,1]]:
# Set minor ticks with day numbers
ax.xaxis.set_minor_locator(dates.DayLocator(interval=10))
ax.xaxis.set_minor_formatter(dates.DateFormatter('%d'))
# Set major ticks with month names
ax.xaxis.set_major_locator(dates.MonthLocator())
ax.xaxis.set_major_formatter(dates.DateFormatter('\n%b'))
plt.savefig('seasonal_global.png')
plt.show()
The gap in your plot occurs because you are displaying the winter months of two different winters, one that started in 2018 and ended in 2019, and another that started in 2019 and ended in 2020.
You need to subset your data so that it gathers the appropriate months, as the following code does:
import numpy as np
import pandas as pd
np.random.seed(42)
datetime_index = pd.date_range(start='2018-01-01', end='2020-12-31')
volume = np.random.randint(low=30, high=60, size=datetime_index.shape[0])
data = pd.DataFrame({'volume': volume},
index=datetime_index)
winter = data['2019-12':'2020-02']
winter.plot()
It plots this:
If you don't have more than one year's worth of data, then you can fill the gap with the other seasons in light grey, such as the graph below:
fig, ax = plt.subplots(1, 1, figsize=(16,10))
line_width = 2
ax.plot(data['volume'], c='grey', lw=line_width, label='All year')
ax.plot(data[:'2019-02'], c='blue', lw=line_width, label='Winter')
ax.plot(data['2019-12':], c='blue', lw=line_width)
plt.legend()
plt.title('Volume across 2019')
plt.xlabel('Month')
plt.ylabel('Volume')
plt.show()
The synthetic data that I've used is more volatile than the real data. You could smooth the time series with a moving average using rolling(), to improve the readability of the changes over time.
I have a time series data on which I would like to build a overlayed scatterplot and boxplot. The data is as so:
TokenUsed date
0 8 2020-01-05
1 8 2020-01-05
2 8 2020-01-05
3 8 2020-01-05
4 8 2020-01-05
... ... ...
51040 7 2020-02-23
51041 7 2020-02-23
51042 7 2020-02-23
51043 7 2020-02-23
51044 7 2020-02-23
This time series can be neatly shown as a boxplot (I've had trouble with the x-axis being a date, but solved it converting it to string). Now I would like to show only the data on which sum is superior to a threshold (>81) in my case. The code and the resulting image are below:
fig, ax = plt.subplots(figsize = (12,6))
ax = sns.boxplot(x="date", y="TokenUsed", data=df, ax= ax, whis=[0,100])
ax.axhline(81)
plt.locator_params(axis='x', nbins=10)
plt.show()
When I add a scatter plot, I get image (2) and by filtering only those >81 I get image(3). What I don't understand is why it can't seem to match the x-axis between the two graphs!
Code:
fig, ax = plt.subplots(figsize = (12,6))
ax = sns.boxplot(x="date", y="TokenUsed", data=df, ax= ax, whis=[0,100])
# Without filter
ax = sns.scatterplot(x="date", y="TokenUsed", data=df, ax= ax,color=".25")
# Filter
ax = sns.scatterplot(x="date", y="TokenUsed", data=df[df["TokenUsed"]>81], ax= ax,color=".25")
ax.axhline(81)
plt.locator_params(axis='x', nbins=10)
plt.show()
Answer:
Try editing your filtering such that no rows of df are actually removed. That is, apply a mask specifically on the TokenUsed column, such that values are replaced with NaN (rather than the whole row being removed). Here's how I would implement this:
#make a new copy df, use that to plot
df2['TokenUsed'] = df2['TokenUsed'].mask(df2['TokenUsed'] < 81)
ax = sns.scatterplot(x="date", y="TokenUsed", data=df2, ax= ax,color=".25")
Explanation
Caveat: this is really my understanding of what is going on from my own observations; I am not actually aware of the implementation behind the scenes
seaborn is less aware of the dates then you are expecting. When creating the boxplot and using the date column for the x-axis, seaborn groups the data by each unique value in the date column. It orders these strings and then creates an integer position for each of them (starting from 0). The y-data are then plotted against these integer values, and the x-tick-labels are replaced with the corresponding string value. So in your case, there are 8 unique date strings, and they are plotted at x-positions from 0 to 7. Also, it doesn't actually matter that they look like dates. You could add more string values to the date column; their position relative to prior data would depend on their alphabetical order (e.g. I would guess the string '00-00-0000' would appear first and the string '999' would appear last).
The filter df[df["TokenUsed"]>81] removes any rows where the TokenUsed value is below 81. This means that the filtered DataFrame will not have as many string date values as the original data. This then creates the unexpected result when plotting. In your filtered data, the first date with values above 81 is 2020-02-09. So in the scatterplot call, those values get plotted at x=0, which is confusing because the values from 2020-01-05 were plotted at x=0 in the call to boxplot.
The fix is to make sure all the original dates are still present in the filtered data, but to replace the filtered out values with NaN so nothing gets plotted.
Here is the example I used to test this:
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
# fake data, only one date has values over 80
dr = ['01-05-2020'] * 100 + ['01-12-2020'] * 100 + ['01-19-2020'] * 100
data = list(np.random.randint(0,80,200)) + list(np.random.randint(50,150,100))
df = pd.DataFrame({'date':dr, 'TokenUsed':data})
fig, ax = plt.subplots(figsize = (12,6))
ax = sns.boxplot(x="date", y="TokenUsed", data=df, ax=ax, whis=[0,100])
df2 = df.copy()
df2['TokenUsed'] = df2['TokenUsed'].mask(df2['TokenUsed'] < 81)
# the fix
df2 = df.copy()
df2['TokenUsed'] = df2['TokenUsed'].mask(df2['TokenUsed'] < 81)
ax = sns.scatterplot(x="date", y="TokenUsed", data=df2, ax= ax,color=".25")
ax.axhline(81)
plt.locator_params(axis='x', nbins=10)
plt.show()
If I use the same filter that you applied, I get the same issue.
i have x-axis which is in terms of days (366 days Feb was taken as 29 days) but instead I want to convert it in terms of months (Jan - Dec). What should i do...
def plotGraph():
line, point = getXY()
plt.plot(line['xlMax'], c='orangered', alpha=0.5, label = 'Minimum Temperature (2005-14)')
plt.plot(line['xlMin'], c='dodgerblue', alpha=0.5, label = 'Minimum Temperature (2005-14)')
plt.scatter(point['xsMax'].index, point['xsMax'], s = 10, c = 'maroon', label = 'Record Break Minimum (2015)')
plt.scatter(point['xsMin'].index, point['xsMin'], s = 10, c = 'midnightblue', label = 'Record Break Maximum (2015)')
ax1 = plt.gca() # Primary axes
ax1.fill_between(line['xlMax'].index , line['xlMax'], line['xlMin'], facecolor='lightgray', alpha=0.25)
ax1.grid(True, alpha = 1)
for spine in ax1.spines:
ax1.spines[spine].set_visible(False)
ax1.spines['bottom'].set_visible(True)
ax1.spines['bottom'].set_alpha(0.3)
# Removing Ticks
ax1.tick_params(axis=u'both', which=u'both',length=0)
plt.show()
I think the quickest change might be to just set new ticks and tick labels at the starts of months; I found the conversion from day-of-the-year to month here, the first table:
import numpy as np
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
x = range(1,367)
y = np.random.rand(len(range(1,367)))
ax.plot(x,y)
month_starts = [1,32,61,92,122,153,183,214,245,275,306,336]
month_names = ['Jan','Feb','Mar','Apr','May','Jun',
'Jul','Aug','Sep','Oct','Nov','Dec']
ax.set_xticks(month_starts)
ax.set_xticklabels(month_names)
Note I assumed your days were numbered 1 to 366; if they are 0 to 365 you may have to change the range.
But I think usually a better approach is to get your days into some sort of datetime; this is more flexible and usually pretty smart. If say, your days were not confined to one year, it would be more complicated to associate day numbers with months.
This example uses datetime instead of integers. The dates are plotted on the x-axis directly, and then the DateFormatter and MonthLocator from matplotlib.dates are used to format the axis appropriately:
import datetime as dt
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import numpy as np
start = dt.datetime(2016,1,1) #there has to be a year given, even if it isn't plotted
new_dates = [start + dt.timedelta(days=i) for i in range(366)]
fig, ax = plt.subplots()
x = new_dates
y = np.random.rand(len(range(1,367)))
xfmt = mdates.DateFormatter('%b')
months = mdates.MonthLocator()
ax.xaxis.set_major_locator(months)
ax.xaxis.set_major_formatter(xfmt)
ax.plot(x,y)
I want to display my third plot x-axis data in the datetime like my other two plots (see linked figure). I have used similar approaches to each graph, but resampled the third dataset to plot precipitation in a bar graph for every hour in my time period. When I originally attempted to format the date for the third plot as I did in the previous two, the x-axis labels either disappeared or the data doesn't plot correctly. In the link below, the data is displayed the way I intended.
Three subplots of rainfall
My timeseries data appears like this, where I'm only concerned about 'Reading' and 'Value':
Reading,Receive,Value,Unit,Quality
2018-04-07 13:09:28,2018-04-07 13:09:35,0.00,in,A
2018-04-07 06:01:25,2018-04-07 06:01:35,0.04,in,A
2018-04-07 04:38:15,2018-04-07 04:38:35,0.04,in,A
Here is how I achieved the correct scheme in the second plot:
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.patches as patches
import matplotlib.dates as mdates
import datetime as dt
#read data from csv
data2 = pd.read_csv('Arroyo_Corte_Madera_del_Presidio_38021_Precipitation_Accumulation_0.txt', usecols=['Reading','Value'], parse_dates=['Reading'])
#set date as index
data2.set_index('Reading',inplace=True)
#plot data
ax2 = plt.subplot(3, 1, 2)
data2.plot(ax=ax2)
#set ticks every 12 hours
ax2.xaxis.set_major_locator(mdates.HourLocator(byhour=range(0,24,12)))
plt.xticks(rotation=0, ha='center')
#format date
ax2.xaxis.set_major_formatter(mdates.DateFormatter('%b %d\n%H:%M:%S'))
ax2.legend().set_visible(False)
ax2.set_title('Accumulated Rainfall\nApril 5-7, 2018')
ax2.set_xlabel('')
ax2.set_ylabel('Inches Since Oct 1 2017')
ax2.set_ylim(17.5, 22)
arrow_date2 = mdates.datestr2num('04/07/2018 04:30:00')
start_date2 = mdates.datestr2num('04/07/2018 03:00:00')
end_date2 = mdates.datestr2num('04/07/2018 06:00:00')
text_date2 = mdates.datestr2num('04/07/2018 03:00:00')
ax2.axvspan(start_date2, end_date2, 0.86, 0.97, color='green', alpha=0.35)
ax2.annotate("Approximate time of\nSlope Failure", xy=(arrow_date2, 21.5), xycoords='data', xytext=(text_date2, 19), textcoords='data', arrowprops=dict(arrowstyle="->", connectionstyle="arc3"))
My code so far for the third subplot:
#read data from csv
data =pd.read_csv('Arroyo_Corte_Madera_del_Presidio_38021_Precipitation_Increment_0.txt', usecols=['Reading','Value'], parse_dates=['Reading'])
#set date as index
data.set_index('Reading',inplace=True)
resamp = data.resample('1H').sum().reset_index()
#plot data
ax3 = plt.subplot(3, 1, 3)
resamp.plot(kind='bar',ax=ax3, x='Reading', y='Value', width=0.9)
#set ticks every other hour
plt.xticks(ha='center')
for label in ax3.xaxis.get_ticklabels()[::2]:
label.set_visible(False)
ax3.legend().set_visible(False)
ax3.set_title('Rainfall in Hours\nApril 6-7, 2018')
ax3.set_xlabel('')
ax3.set_ylabel('Precipitation Increment (in)')
plt.show()
How do I fix my code to make the axis labels plot in the way I want them to plot?
My code was wrong, obviously. When I resampled the data, I reset the index. This created a new index column that was messing with my desired x values ('Reading'). Additionally, I shouldn't have been plotting 'x' in resamp.plot. This solution helped: Plotting with Pandas. Here is the corrected code:
#read data from csv
data = pd.read_csv('Arroyo_Corte_Madera_del_Presidio_38021_Precipitation_Increment_0.txt', usecols=['Reading','Value'], parse_dates=['Reading'])
#set date as index
data.set_index('Reading',inplace=True)
resamp = data.resample('1H').sum() # changed here
#plot data
ax3 = plt.subplot(3, 1, 3)
resamp.plot(ax=ax3, y='Value', kind='bar', width=0.9) # changed here
ax3.set_xticklabels([dt.strftime('%b %d\n%H:%M:%S') for dt in resamp.index])
plt.xticks(rotation=0, ha='center')
for i, tick in enumerate(ax3.xaxis.get_major_ticks()):
if (i % (4) != 0): # 4 hours
tick.set_visible(False)
ax3.legend().set_visible(False)
ax3.set_title('Rainfall in Hours\nApril 6-7, 2018')
ax3.set_xlabel('')
ax3.set_ylabel('Precipitation Increment (in)')
ax3.set_ylim(0.00, 0.40)
plt.show()