I have some data in a pandas series and when I type
mydata.head()
I get:
BPM
timestamp
2015-04-07 02:24:00 96.0
2015-04-07 02:24:00 96.0
2015-04-07 02:24:00 95.0
2015-04-07 02:24:00 95.0
2015-04-07 02:24:00 95.0
Also, when using
mydata.info()
I get:
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 33596 entries, 2015-04-07 02:24:00 to 2015-07-15 14:23:50
Data columns (total 1 columns):
BPM 33596 non-null float64
dtypes: float64(1)
memory usage: 524.9 KB
When I go to plot using
import matplotlib.pyplot as pyplot
fig, ax = pyplot.subplots()
ax.plot(mydata)
I just get a complete mess, it's like it's joining lots of points together that should not be joined together.
How can I sort this out to display as a proper time series plot?
Just use
mydata.plot(style="o")
and you should get a Pandas plot without lines connecting the points.
Just tell matplotlib to plot markers instead of lines. For example,
import matplotlib.pyplot as pyplot
fig, ax = pyplot.subplots()
ax.plot(my data, '+')
If you prefer another marker, you can change it (see this link).
You can also plot directly from pandas:
mydata.plot('+')
If you really want the lines, you need to sort your data before plotting it.
Related
I have a pandas data series to display that contains basically a temporal series where each observation is contained in df['date'] (x-axis) and df['value'] (y-axis):
df['date']
0 2004-01-02
1 2004-01-05
2 2004-01-06
3 2004-01-07
4 2004-01-08
...
4527 2021-12-27
4528 2021-12-28
4529 2021-12-29
4530 2021-12-30
4531 2021-12-31
Name: session, Length: 4532, dtype: object
Notice how the Series contains str types:
print(type(df['date'].values[0]))
<class 'str'>
df['values'] are just integers.
If I plot the series using matplotlib and try to use df['date'] I obtain a too dense chart where the xtick labels can not be read:
ax.plot(df['date'], df['value']);
If I want to display xticks on every month change (so 2004/01, 2004/02, .... 2021/11, 2021/12) and labels just when the year changes (2004, 2005, ... 2021), which would be the best way to accomplish that either via numpy or pandas to get the arrays that .set_xticks require?
Just to further elaborate on tdy's comment (all credit on the answer to tdy), this is the code excerpt to implement to_datetime and use the locators:
ax.plot(pd.to_datetime(df['date']), df['value'])
ax.xaxis.set_major_locator(mdates.YearLocator())
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y'))
.In existing thread (Annotate Time Series plot in Matplotlib), they annotate a single line graph. I am after annotation of multiple line graphs that share the same -axis: I have two data frames which look like as follow:
df:
Value
Week
2020-04-05 0.330967
2020-04-12 1.307075
2020-04-19 2.406805
2020-04-26 2.562565
2020-05-03 2.868995
2020-05-10 5.174968
2020-05-17 5.734933
2020-05-24 6.903961
2020-05-31 7.205925
2020-06-07 9.960470
2020-06-14 11.106135
2020-06-21 12.356842
2020-06-28 13.247175
2020-07-05 13.600287
2020-07-12 15.098775
2020-07-19 16.754835
2020-07-26 18.596575
2020-08-02 20.118878
2020-08-09 21.168825
2020-08-16 21.201978
2020-08-23 21.784821
2020-08-30 22.329772
2020-09-06 23.981835
2020-09-13 23.981835
2020-09-20 23.981835
df2:
Value
Date
2020-09-27 29.003255
2020-10-04 29.642155
2020-10-11 30.872583
2020-10-18 32.492713
2020-10-25 33.436226
2020-11-01 35.187827
2020-11-08 35.589155
2020-11-15 37.185094
2020-11-22 37.575597
2020-11-29 39.273018
2020-12-06 40.047140
2020-12-13 41.621320
2020-12-20 42.563794
2020-12-27 43.750932
2021-01-03 44.823089
2021-01-10 45.797449
2021-01-17 47.109407
2021-01-24 48.045107
2021-01-31 49.472744
2021-02-07 50.355325
2021-02-14 51.717578
2021-02-21 52.602765
2021-02-28 53.886987
2021-03-07 54.888933
2021-03-14 56.108036
2021-03-21 57.226216
2021-03-28 58.345462
I plot these two data frames as a line graph using the following code:
I want to plot these data frames and want to show the data labels on the graph. For this purpose, I was following this article (https://queirozf.com/entries/add-labels-and-text-to-matplotlib-plots-annotation-examples) to plot labels on the line graph. As I have two different data frames so I tried a slightly different method to get the value of xs and ys. Here is my code:
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
ys = np.array([df.index,df2.index])
xs = np.array([df.Value,df2.Value])
fig, ax = plt.subplots(figsize=(12,6))
ax.plot(df.index,df['Value'],'-',color='c')
ax.plot(df2.index,df2['Value'],'-',color='g')
for x,y in zip(xs,ys):
label = "{:.2f}".format(y)
plt.annotate(label, (x,y), textcoords="offset points", ha='center')
plt.show()
When I ran the above code, it gave me the following error:
TypeError: unsupported format string passed to DatetimeIndex.__format__
Could anyone guide me where am I making the mistake?
The problems could be solved by keeping things more clear. Specifically, you make an array of appended data from the two data frames and then you sometimes use that, and sometimes use the unappended data frames, and things are getting confused.
Instead, I'd suggest just keep the data frames separate throughout, since you are clearly interpreting them as distinct because you plot them in different colors, and loop over through the dataframes so you don't duplicate code. So something like this:
df0 = pd.read_csv("data5001.csv", sep="\s+") # uninteresting, my reading in the data, but do what you have here
df1 = pd.read_csv("data5002.csv", sep="\s+")
fig, ax = plt.subplots(figsize=(16,8)) # basically what you have
ax.plot(df0['Date'], df0['Value'],'-',color='c')
ax.plot(df1['Date'], df1['Value'],'-',color='g')
plt.setp(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")
for df in (df0, df1): # loop through the dataframes
for index, v in df.iterrows(): # loop through the data in each frame
label = "{:.2f}".format(v['Value']) # I assume you want the value and not the date, but, whatever, it should be clear now
plt.annotate(label, (v['Date'], v['Value']), ha='center')
I won't address the over-crowding problems since that's an entirely separate question.
I've got two pandas series, one with a 7 day rolling mean for the entire year and another with monthly averages. I'm trying to plot them both on the same matplotlib figure, with the averages as a bar graph and the 7 day rolling mean as a line graph. Ideally, the line would be graph on top of the bar graph.
The issue I'm having is that, with my current code, the bar graph is showing up without the line graph, but when I try plotting the line graph first I get a ValueError: ordinal must be >= 1.
Here's what the series' look like:
These are first 15 values of the 7 day rolling mean series, it has a date and a value for the entire year:
date
2016-01-01 NaN
2016-01-03 NaN
2016-01-04 NaN
2016-01-05 NaN
2016-01-06 NaN
2016-01-07 NaN
2016-01-08 0.088473
2016-01-09 0.099122
2016-01-10 0.086265
2016-01-11 0.084836
2016-01-12 0.076741
2016-01-13 0.070670
2016-01-14 0.079731
2016-01-15 0.079187
2016-01-16 0.076395
This is the entire monthly average series:
dt_month
2016-01-01 0.498323
2016-02-01 0.497795
2016-03-01 0.726562
2016-04-01 1.000000
2016-05-01 0.986411
2016-06-01 0.899849
2016-07-01 0.219171
2016-08-01 0.511247
2016-09-01 0.371673
2016-10-01 0.000000
2016-11-01 0.972478
2016-12-01 0.326921
Here's the code I'm using to try and plot them:
ax = series_one.plot(kind="bar", figsize=(20,2))
series_two.plot(ax=ax)
plt.show()
Here's the graph that generates:
Any help is hugely appreciated! Also, advice on formatting this question and creating code to make two series for a minimum working example would be awesome.
Thanks!!
The problem is that pandas bar plots are categorical (Bars are at subsequent integer positions). Since in your case the two series have a different number of elements, plotting the line graph in categorical coordinates is not really an option. What remains is to plot the bar graph in numerical coordinates as well. This is not possible with pandas, but is the default behaviour with matplotlib.
Below I shift the monthly dates by 15 days to the middle of the month to have nicely centered bars.
import matplotlib.pyplot as plt
import numpy as np; np.random.seed(42)
import pandas as pd
t1 = pd.date_range("2018-01-01", "2018-12-31", freq="D")
s1 = pd.Series(np.cumsum(np.random.randn(len(t1)))+14, index=t1)
s1[:6] = np.nan
t2 = pd.date_range("2018-01-01", "2018-12-31", freq="MS")
s2 = pd.Series(np.random.rand(len(t2))*15+5, index=t2)
# shift monthly data to middle of month
s2.index += pd.Timedelta('15 days')
fig, ax = plt.subplots()
ax.bar(s2.index, s2.values, width=14, alpha=0.3)
ax.plot(s1.index, s1.values)
plt.show()
The problem might be the two series' indices are of very different scales. You can use ax.twiny to plot them:
ax = series_one.plot(kind="bar", figsize=(20,2))
ax_tw = ax.twiny()
series_two.plot(ax=ax_tw)
plt.show()
Output:
I have a dataframe which is structured as:
Date ticker adj_close
0 2016-11-21 AAPL 111.730
1 2016-11-22 AAPL 111.800
2 2016-11-23 AAPL 111.230
3 2016-11-25 AAPL 111.790
4 2016-11-28 AAPL 111.570
...
8 2016-11-21 ACN 119.680
9 2016-11-22 ACN 119.480
10 2016-11-23 ACN 119.820
11 2016-11-25 ACN 120.740
...
How can I plot based on the ticker the adj_close versus Date?
Simple plot,
you can use:
df.plot(x='Date',y='adj_close')
Or you can set the index to be Date beforehand, then it's easy to plot the column you want:
df.set_index('Date', inplace=True)
df['adj_close'].plot()
If you want a chart with one series by ticker on it
You need to groupby before:
df.set_index('Date', inplace=True)
df.groupby('ticker')['adj_close'].plot(legend=True)
If you want a chart with individual subplots:
grouped = df.groupby('ticker')
ncols=2
nrows = int(np.ceil(grouped.ngroups/ncols))
fig, axes = plt.subplots(nrows=nrows, ncols=ncols, figsize=(12,4), sharey=True)
for (key, ax) in zip(grouped.groups.keys(), axes.flatten()):
grouped.get_group(key).plot(ax=ax)
ax.legend()
plt.show()
Similar to Julien's answer above, I had success with the following:
fig, ax = plt.subplots(figsize=(10,4))
for key, grp in df.groupby(['ticker']):
ax.plot(grp['Date'], grp['adj_close'], label=key)
ax.legend()
plt.show()
This solution might be more relevant if you want more control in matlab.
Solution inspired by: https://stackoverflow.com/a/52526454/10521959
The question is How can I plot based on the ticker the adj_close versus Date?
This can be accomplished by reshaping the dataframe to a wide format with .pivot or .groupby, or by plotting the existing long form dataframe directly with seaborn.
In the following sample data, the 'Date' column has a datetime64[ns] Dtype.
Convert the Dtype with pandas.to_datetime if needed.
Tested in python 3.10, pandas 1.4.2, matplotlib 3.5.1, seaborn 0.11.2
Imports and Sample Data
import pandas as pd
import pandas_datareader as web # for sample data; this can be installed with conda if using Anaconda, otherwise pip
import seaborn as sns
import matplotlib.pyplot as plt
# sample stock data, where .iloc[:, [5, 6]] selects only the 'Adj Close' and 'tkr' column
tickers = ['aapl', 'acn']
df = pd.concat((web.DataReader(ticker, data_source='yahoo', start='2020-01-01', end='2022-06-21')
.assign(ticker=ticker) for ticker in tickers)).iloc[:, [5, 6]]
# display(df.head())
Date Adj Close ticker
0 2020-01-02 73.785904 aapl
1 2020-01-03 73.068573 aapl
2 2020-01-06 73.650795 aapl
3 2020-01-07 73.304420 aapl
4 2020-01-08 74.483604 aapl
# display(df.tail())
Date Adj Close ticker
1239 2022-06-14 275.119995 acn
1240 2022-06-15 281.190002 acn
1241 2022-06-16 270.899994 acn
1242 2022-06-17 275.380005 acn
1243 2022-06-21 282.730011 acn
pandas.DataFrame.pivot & pandas.DataFrame.plot
pandas plots with matplotlib as the default backend.
Reshaping the dataframe with pandas.DataFrame.pivot converts from long to wide form, and puts the dataframe into the correct format to plot.
.pivot does not aggregate data, so if there is more than 1 observation per index, per ticker, then use .pivot_table
Adding subplots=True will produce a figure with two subplots.
# reshape the long form data into a wide form
dfp = df.pivot(index='Date', columns='ticker', values='Adj Close')
# display(dfp.head())
ticker aapl acn
Date
2020-01-02 73.785904 203.171112
2020-01-03 73.068573 202.832764
2020-01-06 73.650795 201.508224
2020-01-07 73.304420 197.157654
2020-01-08 74.483604 197.544434
# plot
ax = dfp.plot(figsize=(11, 6))
Use seaborn, which accepts long form data, so reshaping the dataframe to a wide form isn't necessary.
seaborn is a high-level api for matplotlib
sns.lineplot: axes-level plot
fig, ax = plt.subplots(figsize=(11, 6))
sns.lineplot(data=df, x='Date', y='Adj Close', hue='ticker', ax=ax)
sns.relplot: figure-level plot
Adding row='ticker', or col='ticker', will generate a figure with two subplots.
g = sns.relplot(kind='line', data=df, x='Date', y='Adj Close', hue='ticker', aspect=1.75)
I'm experimenting with pandas and non-matplotlib plotting. Good suggestions are here. This question regards yhat's ggplot and I am running into two issues.
Plotting a series in pandas is easy.
frequ.plot()
I don't see how to do this in the ggplot docs. Instead I end up creating a dataframe:
cheese = DataFrame({'time': frequ.index, 'count' : frequ.values})
ggplot(cheese, aes(x='time', y='count')) + geom_line()
I would expect ggplot -- a project that has "tight integration with pandas" -- to have a way to plot a simple series.
Second issue is I can't get stat_smooth() to display when the x axis is time of day. Seems like it could be related to this post, but I don't have the rep to post there. My code is:
frequ = values.sampler.resample("1Min", how="count")
cheese = DataFrame({'time': frequ.index, 'count' : frequ.values})
ggplot(cheese, aes(x='time', y='count')) + geom_line() + stat_smooth()
Any help regarding non-matplotlib plotting would be appreciated. Thanks!
(I'm using ggplot 0.5.8)
I run into this problem frequently in Python's ggplot when working with multiple stock prices and economic timeseries. The key to remember with ggplot is that data is best organized in long format to avoid any issues. I use a quick two step process as a workaround. First let's grab some stock data:
import pandas.io.data as web
import pandas as pd
import time
from ggplot import *
stocks = [ 'GOOG', 'MSFT', 'LNKD', 'YHOO', 'FB', 'GOOGL','HPQ','AMZN'] # stock list
# get stock price function #
def get_px(stock, start, end):
return web.get_data_yahoo(stock, start, end)['Adj Close']
# dataframe of equity prices
px = pd.DataFrame({n: get_px(n, '1/1/2014', date_today) for n in stocks})
px.head()
AMZN FB GOOG GOOGL HPQ LNKD MSFT YHOO
Date
2014-01-02 397.97 54.71 NaN 557.12 27.40 207.64 36.63 39.59
2014-01-03 396.44 54.56 NaN 553.05 28.07 207.42 36.38 40.12
2014-01-06 393.63 57.20 NaN 559.22 28.02 203.92 35.61 39.93
2014-01-07 398.03 57.92 NaN 570.00 27.91 209.64 35.89 40.92
2014-01-08 401.92 58.23 NaN 571.19 27.19 209.06 35.25 41.02
First understand that ggplot needs the datetime index to be a column in the pandas dataframe in order to plot correctly when switching from wide to long format. I wrote a function to address this particular point. It simply creates a 'Date' column of type=datetime from the pandas series index.
def dateConvert(df):
df['Date'] = df.index
df.reset_index(drop=True)
return df
From there run the function on the df. Use the result as the object in pandas pd.melt using the 'Date' as the id_vars. The returned df is now ready to be plotted using the standard ggplot() format.
px_returns = px.pct_change() # common stock transformation
cumRet = (1+px_returns).cumprod() - 1 # transform daily returns to cumulative
cumRet_dateConverted = dateConvert(cumRet) # run the function here see the result below#
cumRet_dateConverted.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 118 entries, 2014-01-02 00:00:00 to 2014-06-20 00:00:00
Data columns (total 9 columns):
AMZN 117 non-null float64
FB 117 non-null float64
GOOG 59 non-null float64
GOOGL 117 non-null float64
HPQ 117 non-null float64
LNKD 117 non-null float64
MSFT 117 non-null float64
YHOO 117 non-null float64
Date 118 non-null datetime64[ns]
dtypes: datetime64[ns](1), float64(8)
data = pd.melt(cumRet_dateConverted, id_vars='Date').dropna() # Here is the method I use to format the data in the long format. Please note the use of 'Date' as the id_vars.
data = data.rename(columns = {'Date':'Date','variable':'Stocks','value':'Returns'}) # common to rename these columns
From here you can now plot your data however you want. A common plot I use is the following:
retPlot_YTD = ggplot(data, aes('Date','Returns',color='Stocks')) \
+ geom_line(size=2.) \
+ geom_hline(yintercept=0, color='black', size=1.7, linetype='-.') \
+ scale_y_continuous(labels='percent') \
+ scale_x_date(labels='%b %d %y',breaks=date_breaks('week') ) \
+ theme_seaborn(style='whitegrid') \
+ ggtitle(('%s Cumulative Daily Return vs Peers_YTD') % key_Stock)
fig = retPlot_YTD.draw()
ax = fig.axes[0]
offbox = ax.artists[0]
offbox.set_bbox_to_anchor((1, 0.5), ax.transAxes)
fig.show()
This is more of a workaround but you can use qplot for quick, shorthand plots using series.
from ggplot import *
qplot(meat.beef)