Develop Reference

pandas/matplotlib graph on frequency of appearance

I am a pandas newbie and I want to make a graph from a CSV I have. On this csv, there's some date written to it, and I want to make a graph of how frequent those date appears.
This is how it looks :
2022-01-12
2022-01-12
2022-01-12
2022-01-13
2022-01-13
2022-01-14
Here, we can see that I have three records on the 12th of january, 2 records the 13th and only one records the 14th. So we should see a decrease on the graph.
So, I tried converting my csv like this :
date,records
2022-01-12,3
2022-01-13,2
2022-01-14,1
And then make a graph with the date as the x axis and the records amount as the y axis.
But is there a way panda (or matplotlib I never understand which one to use) can make a graph based on the frequency of appearance, so that I don't have to convert the csv before ?

There is a function of PANDAS which allows you to count the number of values.
First off, you'd need to read your csv file into a dataframe. Do this by using:
import pandas as pd
df = pd.read_csv("~csv file name~")
Using the unique() function in the pandas library, you can display all of the unique values. The syntax should look like:
uniqueVals = df("~column name~").unique()
That should return a list of all the unique values. Then what you'll do is use the function value_counts() with whatever value you are trying to count in square brackets after the normal brackets. The syntax should look something like this:
totalOfVals = []
for date in uniqueVals:
numDate = df[date].valuecounts("~Whatever date you're looking for~")
totalOfVals.append(numDate)
Then you can use the two arrays you have for the unique dates and the amount of dates there are to then use matplotlib to create a graph.
You'll want to use the syntax:
import matplotlib.pyplot as mpl
mpl.plot(uniqueVals, totalOfVals, color = "~whatever colour you want the line to be~", marker = "~whatever you want the marker to look like~")
mpl.xlabel('Date')
mpl.ylabel('Number of occurrences')
mpl.title('Number of occurrences of dates')
mpl.grid(True)
mpl.show()
And that should display a graph with all the dates and number of occurrences with a grid behind it. Of course if you don't want the grid just either set mpl.grid to False or just get rid of it.

The matplotlib chart changes when I change the index in python pandas dataframe

I have a dataset of S&P500 historical prices with the date, the price and other data that i don't need now to solve my problem.
Date Price
0 1981.01 6.19
1 1981.02 6.17
2 1981.03 6.24
3 1981.04 6.25
. . .
and so on till 2020
The date is a float with the year, a dot and the month.
I tried to plot all historical prices with matplotlib.pyplot as plt.
plt.plot(df["Price"].tail(100))
plt.title("S&P500 Composite Historical Data")
plt.xlabel("Date")
plt.ylabel("Price")
This is the result. I used df["Price"].tail(100) so you can see better the difference between the first and the second graph(You are going to see in a sec).
But then I tried to set the index from the one before(0, 1, 2 etc..) to the df["Date"] column in the DataFrame in order to see the date in the x axis.
df = df.set_index("Date")
plt.plot(df["Price"].tail(100))
plt.title("S&P500 Composite Historical Data")
plt.xlabel("Date")
plt.ylabel("Price")
This is the result, and it's quite disappointing.
I have the Date where it should be in the x axis but the problem is that the graph is different from the one before which is the right one.
If you need the dataset to try out the problem here you can find it.
It is called U.S. Stock Markets 1871-Present and CAPE Ratio.
Hope you've understood everything.
Thanks in advance
UPDATE
I found something that could cause the problem. If you look in depth at the date you can see that in month #10 each is written as a float(in the original dataset) like this: example Year:1884 1884.1. The problem occur when you use pd.to_datetime() to transform the Date float series to a Datetime. So the problem could be that the date in the month #10, when converted into a Datetime, become: (example from before) 1884-01-01 which is the first month in the year and it has an effect on the final plot.
SOLUTION
Finally, I solved my problem!
Yes, the error was the one I explain in the UPDATE paragraph, so I decided to add a 0 as a String where the lenght of the Date (as a string) is 6 in order to change, for example: 1884.1 ==> 1884.10
df["len"] = df["Date"].apply(len)
df["Date"] = df["Date"].where(df["len"] == 7, df["Date"] + "0")
Then i drop the len column i've just created.
df.drop(columns="len", inplace=True)
At the end I changed the "Date" to a Datetime with pd.to_datetime
df["Date"] = pd.to_datetime(df["Date"], format='%Y.%m')
df = df.set_index("Date")
And then I plot
df["Price"].tail(100).plot()
plt.title("S&P500 Composite Historical Data")
plt.xlabel("Date")
plt.ylabel("Price")
plt.show()

The easiest way would be to transform the date into an actual datetime index. This way matplotlib will automatically pick it up and plot it accordingly. For example, given your date format, you could do:
df["Date"] = pd.to_datetime(df["Date"].astype(str), format='%Y.%m')
df = df.set_index("Date")
plt.plot(df["Price"].tail(100))
Currently, the first plot you showed is actually plotting the Price column against the index, which seems to be a regular range index from 0 - 1800 and something. You suggested your data starts in 1981, so although each observation is evenly spaced on the x axis (it's spaced at an interval of 1, which is the jump from one index value to the next). That's why the chart looks reasonable. Yet the x-axis values don't.
Now when you set the Date (as float) to be the index, note that you're not evenly covering the interval between, for example, 1981 and 1982. You have evenly spaced values from 1981.1 - 1981.12, but nothing from 1981.12 - 1982. That's why the second chart is also plotted as expected. Setting the index to a DatetimeIndex as described above should remove this issue, as Matplotlib will know how to evenly space the dates along the x-axis.

I think your problem is that your Date is of float type and taking it as an x-axis does exactly what is expected for taking an array of the kind ([2012.01, 2012.02, ..., 2012.12, 2013.01....]) as x-axis. You might convert the Date column to a DateTimeIndex first and then use the built-in pandas plot method:
df["Price"].tail(100).plot()

It is not a good idea to treat df['Date'] as float. It should be converted into pandas datetime64[ns]. This can be achieved using pandas pd.to_datetime method.
Try this:
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv('ie_data.csv')
df=df[['Date','Price']]
df.dropna(inplace=True)
#converting to pandas datetime format
df['Date'] = df['Date'].astype(str).map(lambda x : x.split('.')[0] + x.split('.')[1])
df['Date'] = pd.to_datetime(df['Date'], format='%Y%m')
df.set_index(['Date'],inplace=True)
#plotting
df.plot() #full data plot
df.tail(100).plot() #plotting just the tail
plt.title("S&P500 Composite Historical Data")
plt.xlabel("Date")
plt.ylabel("Price")
plt.show()
Output:

String to Date Conversion in Python

In the following piece of code:
df['Year']=pd.DatetimeIndex(df['Date']).year
df['Month']=pd.DatetimeIndex(df['Date']).month
df['Day']=pd.DatetimeIndex(df['Date']).day
df['MM_DD_str']=df['Month'].astype(str).str.zfill(2)+'-'+df['Day'].astype(str).str.zfill(2)
Since I want only MM-DD i did this way and it is a string now. But later on the program I want them in the date format. Especially I need month in order to plot a graph. Can i extract a date by extracting month from it.
Edited:
I want to plot a graph in which the Xtick should have the months like Jan, Feb, Mar upto Dec. I have to extract month from the dataframe df['MM_DD_str'] and make them as tick labels for the graph.
This is the final code i have written for plotting graph:
md_str = df['MM_DD_str']
get_month =md_str.apply(lambda d: pd.to_datetime(d, format='%m-%d').month)
#print(get_month)
plt.xticks(get_month,('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'))
plt.show()enter code here
I am not getting neither output nor error

If I understand correctly, you currently have a date string, like "06-23" for example, and you later want to extract the month from it as a datetime object:
md_col = df['MM_DD_str']
get_month = lambda d: pd.to_datetime(d, format='%m-%d').month
md_col.apply(get_month)
get_month is a lambda function that takes a string, converts it to a datetime object, and then extracts the month.
.apply() takes a dataframe column and applies a function to all the rows in the column
Note that if your column contains NaNs or strings that cannot be converted to dates, you could include the errors argument in the .to_datetime function:
get_month = lambda d: pd.to_datetime(d, errors='ignore', format='%m-%d').month

I did not understand the question properly but
the df['date'] column could be used to plot the graph since it is already in date-time format
pd.to_datetime() can be used
so lets say
date='2019-05'
date=pd.to_datetime(date)
date.month
EDIT:
Matplotlib needs numeric values to plot on the x axis
when you say plt.xticks() as some string values you cant plot the graph however you can change the labels . so this is an example adjust to your labels
import matplotlib.pyplot as plt
figure=plt.figure()
ax=plt.axes()
df=pd.DataFrame()
months=['june','july','august','september']
dates=['2019-06','2019-07','2019-08','2019-09']
df['dates']=dates
df['values']=[1,4,7,10]
df['dates']=pd.to_datetime(df['dates']) #pd is for pandas
df['values'].plot(ax=ax)
ax.set_xticks([0,1,2,3,4]) #numerical values that get plotted
ax.set_xticklabels(months) #actual labels for those numerical values

Factorplot with multiindex dataframe

This is the dataframe I am working with:
(only the first two years don't have data for country 69 I will fix this). nkill being the number of killed for that year summed from the original long form dataframe.
I am trying to do something similar to this plot:
However, with the country code as a hue. I know there are similar posts but none have helped me solve this, thank you in advance.
By Hue I mean that in the seaborn syntactical use As pictured in this third picture. See in this example Hue creates a plot for every type of variable in that column. So if I had two country codes in the country column, for every year it would plot two bars (one for each country) side by side.

Just looking at the data it should be possible to directly use the hue argument.
But first you would need to create actual columns from the dataframe
df.reset_index(inplace=True)
Then something like
sns.barplot(x = "year", y="nkill", hue="country", data=df)
should give you the desired plot.

Complex dataframe plotting with Pandas / Matplotlib

I'd like to create a single time-series graph from a pandas dataframe that looks like the following:
*sample of a simplified version of my dataframe:*
index to_network count
201401 net_1 100
201401 net_2 200
201401 net_3 150
201402 net_1 300
201402 net_2 250
201403 net_1 175
Ultimately, the final graph should be a time-series line graph (x-axis being the index and the y-axis being 'count') with multiple lines, and each line being a network in the to_network column (e.g., one line should be net_1).
I've been reading the 'python for data analysis' book, but they don't appear to be this complex.

Does it work?
df.groupby('to_network').count.plot()
If you want to show the date correctly, you can try:
df.index=pd.to_datetime(df.index,format='%Y%m')

The default behavior of plot in pandas is to use the index as an x-axis and plot one line per column. So you want to reshape your data frame to mirror that structure. You can do the following:
df.pivot_table(index='index', columns = 'to_network', values = 'count', aggfunc = 'sum').plot()
This will pivot your df (which is in the long format ala ggplot style) into a frame from which pandas default plot behavior will produce your desired result of one line per network type with index as the x-axis and count as the value.

To answer your question, I have checked in a notebook here: http://nbviewer.ipython.org/github/ericmjl/Stack-Overflow-Answers/blob/master/20141020%20Complex%20Pandas%20Plotting/Untitled0.ipynb
The core idea is to do a groupby, and then plot only the column that you're interested in.
Code is also pasted below here:
df = pd.read_csv("data.csv")
df.groupby("to_network")['count'].plot()
Also, be sure to add in Daniele's contribution, where you format the index correctly:
df.index=pd.to_datetime(df.index,format='%Y%m')
For attribution, I have up-voted her answer in addition to citing it here.
I hope this answers the question; if it did, please accept the answer!