I am importing a .csv file that contains 2 columns (date, permits). When plotting the dataframe, I am converting the 'date' column to be the index and to be formatted as Datetime, but as soon as i do the latest, problems start with the plot.
I have tried two ways without luck:
Option 1) Importing with column 0 (date) as the index, and then converting to datetime format
Option 2) Importing the whole dataframe, converting 'date' into datetime and then adding it as index of the dataframe but all of them have issues
code option 1:
import pandas as pd
import matplotlib.pyplot as plt
BP = pd.read_csv('/Users/josedelgado/PycharmProjects/Fundamentals/Building_Permits.csv',index_col=0)
BP.index = pd.to_datetime(BP.index)
BP = BP.iloc[:(12*50)] #chopping the data since before 1970, datetime turn 1969 into 2069
print(BP)
BP.plot()
plt.ylabel(' Permits')
plt.title('New Building Permits - last 50 years')
plt.show()
results of code #1
code option 2
import pandas as pd
import matplotlib.pyplot as plt
BP = pd.read_csv('/Users/josedelgado/PycharmProjects/Fundamentals/Building_Permits.csv')
BP = BP.iloc[:(12*50)] #chopping the data since before 1970, datetime turn 1969 into 2069
BP['Date'] = pd.to_datetime(BP['Date'])
BP.set_index(BP['Date'], inplace = True)
plt.plot(BP['Permits'])
plt.ylabel(' Permits')
plt.title('New Building Permits - last 50 years')
plt.show()
results on code #2
if i don't convert the date values to date time format, then plot is correct:
right date sampling with wrong xlabel
the sampling of the data looks correct here
I want to reduce the xlim label because i'm using datetime information and that take long space of the xlim. The problem it's when i want to read that
So i need some like to scale that, i think
dates = pd.read_csv("EURUSDtest.csv")
dates = dates["Date"]+" " + dates["Time"]
plt.title("EUR/USD")
plt.plot(dates, data_pred)
plt.xticks(rotation="vertical")
plt.tick_params(labelsize=10)
plt.plot(forecasting)
The problem...
IIUC: You need to convert the dates column to pandas datetime type by calling pd.to_datetime.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# To reproduce the issue you have lets create a date column as string
df = pd.DataFrame({"Dates":pd.date_range(start='2018-1-1', end='2019-1-1', freq='15MIN').strftime("%m-%d-%Y %H-%M-%S")})
# Convert the date string to date type
df["Dates"] = pd.to_datetime(df["Dates"])
# Add column to assign some dummy values
df = df.assign(VAL=np.linspace(10, 110, len(df)))
# Plot the graph
# Now the graph automatically adjusts the XLIM based on the size of the graph
plt.title("eur/usd")
plt.plot(df["Dates"], df["VAL"])
plt.xticks(rotation="vertical")
plt.show()
However if you need to further control xlim to your needs you need to go through matplotlib tutorials.
I'm working with a DataFrame. My data is using for a Candlestick.
The problem is I can't remove the weekend dates. I mean, my code shows this:
enter image description here
And I'm looking for this:
enter image description here
Here is my code:
import matplotlib.ticker as ticker
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from mpl_finance import candlestick_ohlc
df = pd.read_csv('AAPL.csv')
df['Date'] = pd.to_datetime(df['Date'])
df["Date"] = df["Date"].apply(mdates.date2num)
dates = df['Date'].tolist()
ohlc = df[['Date', 'Open', 'High', 'Low','Close']]
f1, ax = plt.subplots(figsize = (12,6))
candlestick_ohlc(ax, ohlc.values, width=.5, colorup='green', colordown='red')
ax.xaxis.set_major_locator(ticker.MultipleLocator(1.0))
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
plt.setp(ax.get_xticklabels(), rotation=70, fontsize=7)
close = df['Close'].values
plt.plot(dates,close, marker='o')
plt.show()
Dataframe:
Date,Open,High,Low,Close,Adj Close,Volume
2019-02-04,167.410004,171.660004,167.279999,171.250000,170.518677,31495500
2019-02-05,172.860001,175.080002,172.350006,174.179993,173.436157,36101600
2019-02-06,174.649994,175.570007,172.850006,174.240005,173.495911,28239600
2019-02-07,172.399994,173.940002,170.339996,170.940002,170.210007,31741700
2019-02-08,168.990005,170.660004,168.419998,170.410004,170.410004,23820000
2019-02-11,171.050003,171.210007,169.250000,169.429993,169.429993,20993400
2019-02-12,170.100006,171.000000,169.699997,170.889999,170.889999,22283500
2019-02-13,171.389999,172.479996,169.919998,170.179993,170.179993,22490200
2019-02-14,169.710007,171.259995,169.380005,170.800003,170.800003,21835700
2019-02-15,171.250000,171.699997,169.750000,170.419998,170.419998,24626800
2019-02-19,169.710007,171.440002,169.490005,170.929993,170.929993,18972800
2019-02-20,171.190002,173.320007,170.990005,172.029999,172.029999,26114400
2019-02-21,171.800003,172.369995,170.300003,171.059998,171.059998,17249700
2019-02-22,171.580002,173.000000,171.380005,172.970001,172.970001,18913200
2019-02-25,174.160004,175.869995,173.949997,174.229996,174.229996,21873400
2019-02-26,173.710007,175.300003,173.169998,174.330002,174.330002,17070200
2019-02-27,173.210007,175.000000,172.729996,174.869995,174.869995,27835400
2019-02-28,174.320007,174.910004,172.919998,173.149994,173.149994,28215400
This is "NOT" enough solution, but I can suggest something for u.
Just use
import mplfinance as mpf
mpf.plot(df, type='candle')
This ignores non-trading days automatically in the plot and make me happier little bit, though I couldn't be fully-satisfied with. I hope this would help u.
Check this out.
https://github.com/matplotlib/mplfinance#basic-usage
You can slice it from the dataframe before processing
please check this link Remove non-business days rows from pandas dataframe
Do not use date/time as your index but use a candle number as index.
then your data becomes continuously and you have no interruption of the time series.
So use candle number as Index , for plotting the data you need to plot it not with a date/time
If you want plot with a date/time you need to use a column where you have put the timestamp of the candle and put that into a plot .. but then you will have gaps again.
Try to filter your dataframe.
df = df[df.Open.notnull()]
Add this to your plot.
show_nontrading=False
I'm attempting to put a MongoDB database that I've imported with PyMongo into a pandas dataframe and then plot it by time with a "date" column of type datetime64 with matplotlib. However, I'm getting randomly connected dates. Does anyone know how I might fix this problem?
The date column seems to be unsorted. To reproduce consider e.g.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame(np.random.rand(15,5), columns=list("ABCDE"))
a = np.arange("2018-05-05", "2018-05-20", dtype="datetime64[D]")
np.random.shuffle(a)
df["date"] = a
plt.plot("date", "C", data=df)
plt.show()
If we sort the dataframe by the date column now,
df.sort_values(by="date", inplace=True)
the result looks much nicer.
A tangential remark here: I would recommend deciding for one style, either
plt.plot("date", "C", data=df)
or
plt.plot(df["date"], df["C"])
and not mix the two by supplying the x argument as Series and the y as string.
I have a csv file with data in the following format -
Issue_Type DateTime
Issue1 03/07/2011 11:20:44
Issue2 01/05/2011 12:30:34
Issue3 01/01/2011 09:44:21
... ...
I'm able to read this csv file, but what I'm unable to achieve is to plot a graph or rather trend based on the data.
For instance - I'm trying to plot a graph with X-axis as Datetime(only Month) and Y-axis as #of Issues. So I would show the trend in line-graphy with 3 lines indicating the pattern of issue under each category for the month.
I really don't have a code for plotting the graph and hence can't share any, but so far I'm only reading the csv file. I'm not sure how to proceed further to plot a graph
PS: I'm not bent on using python - Since I've parsed csv using python earlier I though of using the language, but if there is an easier approach using some other language - I would be open explore that as well.
A way to do this is to use dataframes with pandas.
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
df = pd.read_csv("D:\Programmes Python\Data\Data_csv.txt",sep=";") #Reads the csv
df.index = pd.to_datetime(df["DateTime"]) #Set the index of the dataframe to the DateTime column
del df["DateTime"] #The DateTime column is now useless
fig, ax = plt.subplots()
ax.plot(df.index,df["Issue_Type"])
ax.xaxis.set_major_formatter(mdates.DateFormatter('%m')) #This will only show the month number on the graph
This assumes that Issue1/2/3 are integers, I assumed they were as I didn't really understand what they were supposed to be.
Edit: This should do the trick then, it's not pretty and can probably be optimised, but it works well:
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
df = pd.read_csv("D:\Programmes Python\Data\Data_csv.txt",sep=";")
df.index = pd.to_datetime(df["DateTime"])
del df["DateTime"]
list=[]
for Issue in df["Issue_Type"]:
list.append(int(Issue[5:]))
df["Issue_number"]=list
fig, ax = plt.subplots()
ax.plot(df.index,df["Issue_number"])
ax.xaxis.set_major_formatter(mdates.DateFormatter('%m'))
plt.show()
The first thing you need to do is to parse the datetime fields as dates/times. Try using dateutil.parser for that.
Next, you will need to count the number of issues of each type in each month. The naive way to do that would be to maintain lists of lists for each issue type, and just iterate through each column, see which month and which issue type it is, and then increment the appropriate counter.
When you have such a frequency count of issues, sorted by issue types, you can simply plot them against dates like this:
import matplotlib.pyplot as plt
import datetime as dt
dates = []
for year in range(starting_year, ending_year):
for month in range(1, 12):
dates.append(dt.datetime(year=year, month=month, day=1))
formatted_dates = dates.DateFormatter('%b') # Format dates to only show month names
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(issues[0], dates) # To plot just issues of type 1
ax.plot(issues[1], dates) # To plot just issues of type 2
ax.plot(issues[2], dates) # To plot just issues of type 3
ax.xaxis.set_major_formatter(formatted_dates) # Format X tick labels
plt.show()
plt.close()
honestly, I would just use R. check this link out on downloading / setting up R & RStudio.
data <- read.csv(file="c:/yourdatafile.csv", header=TRUE, sep=",")
attach(data)
data$Month <- format(as.Date(data$DateTime), "%m")
plot(DateTime, Issue_Type)