I have a function that reads csv and outputs me 12 graphs. But it displays the time with a very small interval.
Here's the function!
def Gr():
df = pd.read_csv('DataSet.csv',)
'''start = df['Time'].iloc[0]
start = str(start)
start1 = start.replace(':', '-')
end = df['Time'].iloc[-1]
end = str(end)
end1 = end.replace(':', '-')
index = pd.date_range(start = start1, end = end1, freq = "S")
index = [pd.to_datetime(date, format='%H:%M:%S').date() for date in index] '''
names = ['P', 'Filter', 'Answers', 'step','step2','Comulative', 'Delta_ema','ComulativePOC', 'Delta_P', 'Sum','SpeedUp', 'M' ]
features = df[names]
features.index = df['Time']
axs = features.plot(subplots=True)
cursor = MultiCursor(axs[1].get_figure().canvas, axs)
plt.subplots_adjust(wspace=0.19, hspace=0.05, top=0.99, right=0.988, bottom=0.052, left=0.055)
plt.show()
Here is a screenshot of the result of the functions. I circled the time at the bottom. I would like to increase the interval to at least once every 5 seconds or even 1 second.
Is it possible to do it this way ? Without a figure?
you should uses something like this before your plt.show() line
import matplotlib.dates as m_dates
ax = plt.gca() # get the current axis
ax.xaxis.set_major_locator(m_dates.SecondLocator(interval=5)) # every five seconds
info about SecondLocator:
https://matplotlib.org/stable/api/dates_api.html#matplotlib.dates.SecondLocator
info dates in general:https://matplotlib.org/stable/api/dates_api.html
info ticks locating: https://matplotlib.org/stable/api/ticker_api.html?highlight=ticks%20locator
Related
I'm working on a graph that ilustrates computer usage each day. I want to have a button that will group dates monthly for last year and set y as AVERAGE (mean) and draw avg line.
My code:
import datetime
import numpy as np
import pandas as pd
import plotly.graph_objects as go
example_data = {"date": ["29/07/2022", "30/07/2022", "31/07/2022", "01/08/2022", "02/08/2022"],
"time_spent" : [15840, 21720, 40020, 1200, 4200]}
df = pd.DataFrame(example_data)
df["date"] = pd.to_datetime(df["date"], dayfirst=True)
df['Time spent'] = df['time_spent'].apply(lambda x:str(datetime.timedelta(seconds=x)))
df['Time spent'] = pd.to_datetime(df['Time spent'])
df = df.drop("time_spent", axis=1)
dfall = df.resample("M", on="date").mean().copy()
dfyearly = dfall.tail(12).copy()
dfweekly = df.tail(7).copy()
dfmonthly = df.tail(30).copy()
del df
dfs = {'Week':dfweekly, 'Month': dfmonthly, 'Year' : dfyearly, "All" : dfall}
for dframe in list(dfs.values()):
dframe['StfTime'] = dframe['Time spent'].apply(lambda x: x.strftime("%H:%M"))
frames = len(dfs) # number of dataframes organized in dict
columns = len(dfs['Week'].columns) - 1 # number of columns i df, minus 1 for Date
scenarios = [list(s) for s in [e==1 for e in np.eye(frames)]]
visibility = [list(np.repeat(e, columns)) for e in scenarios]
lowest_value = datetime.datetime.combine(datetime.date.today(), datetime.datetime.min.time())
highest_value = dfweekly["Time spent"].max().ceil("H")
buttons = []
fig = go.Figure()
for i, (period, df) in enumerate(dfs.items()):
print(i)
for column in df.columns[1:]:
fig.add_bar(
name = column,
x = df['date'],
y = df[column],
customdata=df[['StfTime']],
text=df['StfTime'],
visible=True if period=='Week' else False # 'Week' values are shown from the start
)
#Change display data to more friendly format
fig.update_traces(textfont=dict(size=20), hovertemplate='<b>Time ON</b>: %{customdata[0]}</br>')
#Change range for better scalling
this_value =df["Time spent"].max().ceil("H")
if highest_value <= this_value:
highest_value = this_value
fig.update_yaxes(range=[lowest_value, highest_value])
#Add average value indicator
average_value = df["Time spent"].mean()
fig.add_hline(y=average_value, line_width=3, line_dash="dash",
line_color="green")
# one button per dataframe to trigger the visibility
# of all columns / traces for each dataframe
button = dict(label=period,
method = 'restyle',
args = ['visible',visibility[i]])
buttons.append(button)
fig.update_yaxes(dtick=60*60*1000, tickformat='%H:%M')
fig.update_xaxes(type='date', dtick='D1')
fig.update_layout(updatemenus=[dict(type="dropdown",
direction="down",
buttons = buttons)])
fig.show()
EDIT 1.
Thanks to vestland I managed to get semi-working dropdown.
The problem is that the line added with add_hline affect all bar charts. I want it to display only on the chart that it had been added for. Also after passing in custom data for nicer display, the space between bars is doubled. Any way to fix above issues?
I've been trying to calculate the 14 RSI of stocks and I managed to get it to work, somewhat, it gives me inaccurate numbers
import pandas as pd
import datetime as dt
import pandas_datareader as web
ticker = 'TSLA'
start = dt.datetime(2018, 1, 1)
end = dt.datetime.now()
data = web.DataReader(ticker, 'yahoo', start, end)
delta = data['Adj Close'].diff(1)
delta.dropna(inplace=True)
positive = delta.copy()
negative = delta.copy()
positive[positive < 0] = 0
negative[negative > 0] = 0
days = 14
average_gain = positive.rolling(window=days).mean()
average_loss = abs(negative.rolling(window=days).mean())
relative_strenght = average_gain / average_loss
rsi = 100.0 - (100.0 / (1.0 + relative_strenght))
print(ticker + str(rsi))
It ends up giving me 77.991564 (14 days RSI) when I should be getting 70.13 (14 days RSI), does any know what I'm doing wrong?
also yes I've read Calculating RSI in Python but it doesn't help me with what I need
Here is one way to calculate by yourself RSI. The code could be optimized, but I prefer to make it easy to understand, and the let you optimize.
For the example, we assume that you've got a DataFrame called df, with a column called 'Close', for the close prices. By the way, notice that if you compare results of the RSI with a station, for example, you should be sure that you compare the same values. For example, if in the station, you've got the bid close, and that you calculate by your own on the mid or the ask, it will not be the same result.
Let's see the code :
def rsi(df,_window=14,_plot=0,_start=None,_end=None):
"""[RSI function]
Args:
df ([DataFrame]): [DataFrame with a column 'Close' for the close price]
_window ([int]): [The lookback window.](default : {14})
_plot ([int]): [1 if you want to see the plot](default : {0})
_start ([Date]):[if _plot=1, start of plot](default : {None})
_end ([Date]):[if _plot=1, end of plot](default : {None})
"""
##### Diff for the différences between last close and now
df['Diff'] = df['Close'].transform(lambda x: x.diff())
##### In 'Up', just keep the positive values
df['Up'] = df['Diff']
df.loc[(df['Up']<0), 'Up'] = 0
##### Diff for the différences between last close and now
df['Down'] = df['Diff']
##### In 'Down', just keep the negative values
df.loc[(df['Down']>0), 'Down'] = 0
df['Down'] = abs(df['Down'])
##### Moving average on Up & Down
df['avg_up'+str(_window)] = df['Up'].transform(lambda x: x.rolling(window=_window).mean())
df['avg_down'+str(_window)] = df['Down'].transform(lambda x: x.rolling(window=_window).mean())
##### RS is the ratio of the means of Up & Down
df['RS_'+str(_window)] = df['avg_up'+str(_window)] / df['avg_down'+str(_window)]
##### RSI Calculation
##### 100 - (100/(1 + RS))
df['RSI_'+str(_window)] = 100 - (100/(1+df['RS_'+str(_fast)]))
##### Drop useless columns
df = df.drop(['Diff','Up','Down','avg_up'+str(_window),'avg_down'+str(_window),'RS_'+str(_window)],axis=1)
##### If asked, plot it!
if _plot == 1:
sns.set()
fig = plt.figure(facecolor = 'white', figsize = (30,5))
ax0 = plt.subplot2grid((6,4), (1,0), rowspan=4, colspan=4)
ax0.plot(df[(df.index<=end)&(df.index>=start)&(df.Symbol==_ticker.replace('/',''))]['Close'])
ax0.set_facecolor('ghostwhite')
ax0.legend(['Close'],ncol=3, loc = 'upper left', fontsize = 15)
plt.title(_ticker+" Close from "+str(start)+' to '+str(end), fontsize = 20)
ax1 = plt.subplot2grid((6,4), (5,0), rowspan=1, colspan=4, sharex = ax0)
ax1.plot(df[(df.index<=end)&(df.index>=start)&(df.Symbol==_ticker.replace('/',''))]['RSI_'+str(_window)], color = 'blue')
ax1.legend(['RSI_'+str(_window)],ncol=3, loc = 'upper left', fontsize = 12)
ax1.set_facecolor('silver')
plt.subplots_adjust(left=.09, bottom=.09, right=1, top=.95, wspace=.20, hspace=0)
plt.show()
return(df)
To call the function, you just have to type
df = rsi(df)
if you keep it with default values, or to change _window and/or _plot for the arg.
Notice that if you input _plot=1, you'll need to feed starting and ending of the plot, with a string or a date time.
Here I have one csv file with different values and date_time. I wrote the code and run it. Then it gave me an error time data '8/6/2018 6:45' does not match format ' %d/%m/%Y %H:%M:%S'
I want to plot this different three values in one graph. Can anyone help me to solve this problem?
Here is my code:
condition = ""
date_time = []
x1 = []
x2 = []
x3 = []
def convertTime(s):
tm = time.strptime(s, " %d/%m/%Y %H:%M:%S")
return datetime.datetime(date.tm_year,date.tm_mon, date.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec)
with open('data43.csv','r') as csv_file:
csv_data = csv.reader(csv_file, delimiter=',')
row_num = 0
for row in csv_data:
if(row_num == 0):
condition = row[0]
elif(row_num > 2): #Data starts here
if(row[0] != ''):
date_time.append(convertTime(row[0]))
if(row[1] != ''):
x1.append(int(row[1]))
if(row[2] != ''):
x2.append(int(row[2]))
if(row[3] != ''):
x3.append(int(row[3]))
row_num = row_num + 1
fig1 = plt.figure(1)
ax = fig1.add_subplot(2,1,1)
ax.plot(date_time,x1)
ax.stem(date_time,x2,'C1--','C1o',linefmt=None, markerfmt=None, basefmt=None)
ax.stem(date_time,x3,'C2--','C2o',linefmt=None, markerfmt=None, basefmt=None)
ax.legend()
ax.xaxis_date()
ax.get_xaxis().set_major_formatter(DateFormatter('%d/%m/%Y %H:%M:%S'))
plt.xlabel('t')
plt.ylabel('k')
leg = plt.legend( loc = 'upper right')
plt.draw() # Draw the figure so you can find the positon of the legend.
bb = leg.get_bbox_to_anchor().inverse_transformed(ax.transAxes)
xOffset = 0.3
bb.x0 += xOffset
bb.x1 += xOffset
leg.set_bbox_to_anchor(bb, transform = ax.transAxes)
plt.rcParams["figure.figsize"] = [20,20]
ax.plot(style='.-')
plt.show()
error :
I put datetime as :
so I include seconds as :00 after that I read this csv file.
If I changed time into 12 hours reading then what will happened ?
like this
I think you might have an error with the data, not the python file.
8/6/2018 6:45 Does not match the format %d/%m/%Y %H:%M:%S because there are no seconds.
I would test with a :00 added to your time data and test again.
I guess this is supposed to be simple.. But I cant seem to make it work.
I have some stock data
import pandas as pd
import numpy as np
df = pd.DataFrame(index=pd.date_range(start = "06/01/2018", end = "08/01/2018"),
data = np.random.rand(62)*100)
I am doing some analysis on it, this results of my drawing some lines on the graph.
And I want to plot a 45 line somewhere on the graph as a reference for lines I drew on the graph.
What I have tried is
x = df.tail(len(df)/20).index
x = x.reset_index()
x_first_val = df.loc[x.loc[0].date].adj_close
In order to get some point and then use slope = 1 and calculate y values.. but this sounds all wrong.
Any ideas?
Here is a possibility:
import pandas as pd
import numpy as np
df = pd.DataFrame(index=pd.date_range(start = "06/01/2018", end = "08/01/2018"),
data=np.random.rand(62)*100,
columns=['data'])
# Get values for the time:
index_range = df.index[('2018-06-18' < df.index) & (df.index < '2018-07-21')]
# get the timestamps in nanoseconds (since epoch)
timestamps_ns = index_range.astype(np.int64)
# convert it to a relative number of days (for example, could be seconds)
time_day = (timestamps_ns - timestamps_ns[0]) / 1e9 / 60 / 60 / 24
# Define y-data for a line:
slope = 3 # unit: "something" per day
something = time_day * slope
trendline = pd.Series(something, index=index_range)
# Graph:
df.plot(label='data', alpha=0.8)
trendline.plot(label='some trend')
plt.legend(); plt.ylabel('something');
which gives:
edit - first answer, using dayofyear instead of the timestamps:
import pandas as pd
import numpy as np
df = pd.DataFrame(index=pd.date_range(start = "06/01/2018", end = "08/01/2018"),
data=np.random.rand(62)*100,
columns=['data'])
# Define data for a line:
slope = 3 # unit: "something" per day
index_range = df.index[('2018-06-18' < df.index) & (df.index < '2018-07-21')]
dayofyear = index_range.dayofyear # it will not work around the new year...
dayofyear = dayofyear - dayofyear[0]
something = dayofyear * slope
trendline = pd.Series(something, index=index_range)
# Graph:
df.plot(label='data', alpha=0.8)
trendline.plot(label='some trend')
plt.legend(); plt.ylabel('something');
I have a data-set of sleep-time information and would like to produce a visualisation of it using Python.
The .csv data-set I have looks like the this:
SleepStartDate,SleepStartTime,SleepStopTime
17/03/2017,23:45,07:25
19/03/2017,01:05,09:10
19/03/2017,23:50,08:25
The visualisation I want to produce should be similar to the following:
Image source: http://quantifiedself.com/wp-content/uploads/2015/08/qs2.png
I know this is a really simple visualisation, and imagine that it's built into some already existing library, but my best Googling efforts have been unable to locate it. I'd much appreciate it if someone could point me in the right direction.
Thanks in advance for your time and wisdom.
Wasn't able to find a library with functionality to do what I wanted, so ended up writing a script to do it for myself:
Script:
import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
import matplotlib.patches as patches
import datetime as dt
import csv
import sys
MINUTES_IN_DAY = 1440.0
COLUMN_COLOUR = 'b'
# Graph data using matplotlib visualization
def plotData(data,columnColour,maxDate,minDate):
# Set up an invisible background scatterplot give graph the correct size
# Make a series of events that are one day apart
x = mpl.dates.drange(minDate,maxDate,dt.timedelta(days=1))
# Offset first event to top of graph to give correct height
x[0] += 0.85
# Extract the time using a modulo 1, and adding an arbitrary base date
# int used so that y-axis starts at midnight
times = x % 1 + int(x[0])
fig = plt.figure()
fig.suptitle('Daily Sleep Patterns', fontsize=14, fontweight='bold')
ax = fig.add_subplot(111)
# Set background scatterplot to invisible
ax.plot_date(x, times, 'ro', color='w', visible=False)
ax.yaxis_date()
fig.autofmt_xdate()
start, end = ax.get_ylim()
# Fix division sizes and labels to show hours on y-axis
hourDivision = 1.0 / 24.0
ax.yaxis.set_ticks(np.arange(start,end,hourDivision))
ax.set_yticklabels(['Midnight','1am','2am','3am','4am','5am','6am','7am','8am','9am','10am','11am','Midday','1pm','2pm','3pm','4pm','5pm','6pm','7pm','8pm','9pm','10pm','11pm','Midnight'])
# Iterate through data
for i in range(0,len(data)):
# If period starts and finishes on different days, slit and add to both days
if data[i].startTime > data[i].stopTime:
currentDataItem = data[i]
currentDate = dt.datetime(currentDataItem.year,currentDataItem.month,currentDataItem.day)
currentDate -= dt.timedelta(days=0.5)
tomorrow = currentDate + dt.timedelta(days=1)
plt.axvspan(xmin=currentDate, xmax=tomorrow, ymin=currentDataItem.startTime, ymax=1, facecolor=columnColour, alpha=0.5)
theDayAfterTomorrow = tomorrow + dt.timedelta(days=1)
plt.axvspan(xmin=tomorrow, xmax=theDayAfterTomorrow, ymin=0, ymax=currentDataItem.stopTime, facecolor=columnColour, alpha=0.5)
# Else, add to given day
else:
currentDataItem = data[i]
currentDate = dt.datetime(currentDataItem.year,currentDataItem.month,currentDataItem.day)
currentDate -= dt.timedelta(days=0.5)
tomorrow = currentDate + dt.timedelta(days=1)
plt.axvspan(xmin=currentDate, xmax=tomorrow, ymin=currentDataItem.startTime, ymax=currentDataItem.stopTime, facecolor=columnColour, alpha=0.5)
ax.set_ylabel('Hours',fontweight='bold')
#ax.legend()
ax.grid(True)
plt.show()
# Read data from csv file
def readDataFromFile(dataFile):
f = open(dataFile,'rt')
listOfInputLists = []
try:
reader = csv.reader(f)
for row in reader:
listOfInputLists.append(row)
finally:
f.close()
return listOfInputLists
# Class to store time and date data read from file
class sleepInstance(object):
def __init__(self,listOfInputLists):
self.day = 0
self.month = 0
self.year = 0
self.formatDate(listOfInputLists[0])
self.startTime = self.formatTime(listOfInputLists[1])
self.stopTime = self.formatTime(listOfInputLists[2])
# Extracts date information variables
def formatDate(self,unformattedDate):
date = dt.datetime.strptime(unformattedDate,"%d/%m/%y")
self.day = int(date.strftime("%d"))
self.month = int(date.strftime("%m"))
self.year = int(date.strftime("%Y"))
# Formats time as a decimal fraction of day, for use in graph
def formatTime(self,unformattedTime):
timeSinceMidnight = dt.datetime.strptime(unformattedTime,'%H:%M:%S')
midnight = dt.datetime(1900,1,1)
minutesSinceMidnight = ((timeSinceMidnight - midnight).total_seconds() / 60.0)
fractionOfDay = minutesSinceMidnight / MINUTES_IN_DAY
return fractionOfDay
# Formats data read from file as a list of sleepInstance objects
def formatDataForPlot(listOfInputLists):
sleeps = []
for i in range(1,len(listOfInputLists)):
sleeps.append(sleepInstance(listOfInputLists[i]))
return sleeps
# Extracts earliest (min) and latest (max) dates from data, for use in setting graph limits
def getMaxAndMinDates(plotDataList):
dateTimeList = []
for item in plotDataList:
nextDate = dt.datetime(item.year,item.month,item.day)
dateTimeList.append(nextDate)
maxDate = max(dateTimeList)
minDate = min(dateTimeList)
return maxDate, minDate
dataFile = 'sleepData.csv'
listOfInputLists = readDataFromFile(dataFile)
plotDataList = formatDataForPlot(listOfInputLists)
maxDate, minDate = getMaxAndMinDates(plotDataList)
plotData(plotDataList,COLUMN_COLOUR,maxDate,minDate)
Input:
Date,Start,Finish
17/03/17,03:15:00,03:55:00
17/03/17,06:20:00,06:35:00
17/03/17,09:00:00,09:40:00
17/03/17,13:10:00,13:35:00
17/03/17,15:45:00,16:30:00
17/03/17,18:45:00,19:25:00
17/03/17,21:15:00,21:35:00
18/03/17,00:30:00,02:00:00
18/03/17,04:50:00,05:05:00
18/03/17,08:20:00,08:40:00
18/03/17,12:30:00,13:10:00
18/03/17,16:30:00,17:00:00
18/03/17,18:45:00,19:00:00
18/03/17,20:30:00,21:00:00
19/03/17,00:00:00,12:00:00
19/03/17,18:00:00,23:59:00
19/03/17,13:00:00,14:00:00
20/03/17,12:00:00,11:00:00
Output:
May fancy it up a bit when I have time: https://github.com/ambidextrous/timeLogGrapher
The classic Python choice would be to use the matplotlib package. Looking at your sample graph it looks like a vertical bar graph.