I have this dataframe and I want to line plot it. As I have plotted it.
Graph is
Code to generate is
fig, ax = plt.subplots(figsize=(15, 5))
date_time = pd.to_datetime(df.Date)
df = df.set_index(date_time)
plt.xticks(rotation=90)
pd.DataFrame(df, columns=df.columns).plot.line( ax=ax,
xticks=pd.to_datetime(frame.Date))
I want a marker of innovationScore with value(where innovationScore is not 0) on open, close line. I want to show that that is the change when InnovationScore changes.
You have to address two problems - marking the corresponding spots on your curves and using the dates on the x-axis. The first problem can be solved by identifying the dates, where the score is not zero, then plotting markers on top of the curve at these dates. The second problem is more of a structural nature - pandas often interferes with matplotlib when it comes to date time objects. Using pandas standard plotting functions is good because it addresses common problems. But mixing pandas with matplotlib plotting (and to set the markers, you have to revert to matplotlib afaik) is usually a bad idea because they do not necessarily present the date time in the same format.
import pandas as pd
from matplotlib import pyplot as plt
#fake data generation, the following code block is just for illustration
import numpy as np
np.random.seed(1234)
n = 50
date_range = pd.date_range("20180101", periods=n, freq="D")
choice = np.zeros(10)
choice[0] = 3
df = pd.DataFrame({"Date": date_range,
"Open": np.random.randint(100, 150, n),
"Close": np.random.randint(100, 150, n),
"Innovation Score": np.random.choice(choice, n)})
fig, ax = plt.subplots()
#plot the three curves
l = ax.plot(df["Date"], df[["Open", "Close", "Innovation Score"]])
ax.legend(iter(l), ["Open", "Close", "Innovation Score"])
#filter dataset for score not zero
IS = df[df["Innovation Score"] > 0]
#plot markers on these positions
ax.plot(IS["Date"], IS[["Open", "Close"]], "ro")
#and/or set vertical lines to indicate the position
ax.vlines(IS["Date"], 0, max(df[["Open", "Close"]].max()), ls="--")
#label x-axis score not zero
ax.set_xticks(IS["Date"])
#beautify the output
ax.set_xlabel("Month")
ax.set_ylabel("Artifical score people take seriously")
fig.autofmt_xdate()
plt.show()
Sample output:
You can achieve it like this:
import matplotlib.pyplot as plt
plt.plot([1, 2, 3], "ro-") # r is red, o is for larger marker, - is for line
plt.plot([3, 2, 1], "b.-") # b is blue, . is for small marker, - is for line
plt.show()
Check out also example here for another approach:
https://matplotlib.org/3.3.3/gallery/lines_bars_and_markers/markevery_prop_cycle.html
I very often get inspiration from this list of examples:
https://matplotlib.org/3.3.3/gallery/index.html
Related
Simple data as below and I want to put them in a scatter plot.
It goes well if there's not outliers (i.e. extremely big numbers).
import pandas as pd
import matplotlib.pyplot as plt
from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()
dates = ["2021-01-01",
"2021-01-01", "2021-01-06",
"2021-01-08", "2021-01-12",
"2021-02-01", "2021-02-11",
"2021-02-12", "2021-02-15",
"2021-02-16", "2021-03-11",
"2021-03-21", "2021-03-22",
"2021-03-23", "2021-03-24",
"2021-04-02", "2021-04-12",
"2021-04-22", "2021-04-26",
"2021-04-30"]
numbers= [6400,
5100,5000,
4000,3686,
9000,8050,
8000,6050,
6000,9000,
8500,7800,
7000,6000,
10000,9600,
8000,7883,
6686]
dates = [pd.to_datetime(d) for d in dates]
plt.scatter(dates, numbers, s =100, c = 'red')
plt.show()
But when there are one or more extreme numbers, for example the last number 6686 became 66860. The new plot shows most the scatters insignificant (because of the the new y-axis).
What's the good solution to have a scatter plot as before (keeping the y-axis as it was), and still visualizing the extreme numbers?
The purpose of the chart is show and focus the distribution of the scatters under 10000, and also note there are extreme numbers.
If you don't want to use a log scale, you can break the plot in two (or more) and plot the values below/above a threshold:
df = pd.DataFrame({'num': numbers}, index=dates)
thresh = 12000
f, (ax1, ax2) = plt.subplots(nrows=2, sharex=True,
gridspec_kw={'height_ratios': (1,3)},
figsize=(10,4)
)
lows = df.mask(df['num'].ge(thresh))
highs = df.mask(df['num'].lt(thresh))
ax2.scatter(df.index, lows)
ax1.scatter(df.index, highs)
output:
I would like to write scout report on some football players and for that I need visualizations. One type of which is pie charts. Now I need some pie charts that looks like below, with different size of slices ( proportionate to the number of the thing the slice indicates) . Can anyone suggest how to do it or have any link to websites where I can learn this?
What you are looking for is called a "Radar Pie Chart". It's analogous to the more commonly used "Radar Chart", but I think it looks better as it highlights the values, rather than focus on meaningless shapes.
The challenge you face with your football dataset is that each category is on a different scale, so you want to plot each value as a percentage of some max. My code will accomplish that, but you'll want to annotate the original values to finish off these charts.
The plot itself can be done with just the standard matplotlib library using polar axes. I borrowed code from here (https://raphaelletseng.medium.com/getting-to-know-matplotlib-and-python-docx-5ee67bad38d2).
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from math import pi
from random import random, seed
seed(12345)
# Generate dataset with 10 rows, different maxes
maxes = [5, 5, 5, 2, 2, 10, 10, 10, 10, 10]
df = pd.DataFrame(
data = {
'categories': ['category_{}'.format(x) for x, _ in enumerate(maxes)],
'scores': [random()*max for max in maxes],
'max_values': maxes,
},
)
df['pct'] = df['scores'] / df['max_values']
df = df.set_index('categories')
# Plot pie radar chart
N = df.shape[0]
theta = np.linspace(0.0, 2*np.pi, N, endpoint=False)
categories = df.index
df['radar_angles'] = theta
ax = plt.subplot(polar=True)
ax.bar(df['radar_angles'], df['pct'], width=2*pi/N, linewidth=2, edgecolor='k', alpha=0.5)
ax.set_xticks(theta)
ax.set_xticklabels(categories)
_ = ax.set_yticklabels([])
I had previously work with rose or polar bar chart. Here is the example.
import plotly.express as px
df = px.data.wind()
fig = px.bar_polar(df, r="frequency", theta="direction",
color="strength", template="plotly_dark",
color_discrete_sequence= px.colors.sequential.Plasma_r)
fig.show()
I've got a df with messages from a WhatsApp chat, the sender and the corresponding time in datetime format.
Time
Sender
Message
2020-12-21 22:23:00
Sender 1
"..."
2020-12-21 22:26:00
Sender 2
"..."
2020-12-21 22:35:00
Sender 1
"..."
I can plot the histogram with sns.histplot(df["Time"], bins=48)
But now the ticks on the x-axis don't make much sense. I end up with 30 ticks even though it should be 24 and also the ticks all contain the whole date plus the time where I would want only the time in "%H:%M"
Where is the issue with the wrong ticks coming from?
Thanks!
Both seaborn and pandas use matplotlib for plotting functions. Let's see who returns the bin values, we would need to adapt the x-ticks:
import numpy as np
import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(15, 5))
#fake data generation
np.random.seed(1234)
n=20
start = pd.to_datetime("2020-11-15")
df = pd.DataFrame({"Time": pd.to_timedelta(np.random.rand(n), unit="D") + start, "A": np.random.randint(1, 100, n)})
#print(df)
#pandas histogram plotting function, left
pd_g = df["Time"].hist(bins=5, xrot=90, ax=ax1)
#no bin information
print(pd_g)
ax1.set_title("Pandas")
#seaborn histogram plotting, middle
sns_g = sns.histplot(df["Time"], bins=5, ax=ax2)
ax2.tick_params(axis="x", labelrotation=90)
#no bin information
print(sns_g)
ax2.set_title("Seaborn")
#matplotlib histogram, right
mpl_g = ax3.hist(df["Time"], bins=5, edgecolor="white")
ax3.tick_params(axis="x", labelrotation=90)
#hooray, bin information, alas in floats representing dates
print(mpl_g)
ax3.set_title("Matplotlib")
plt.tight_layout()
plt.show()
Sample output:
From this exercise we can conclude that all three refer to the same routine. So, we can directly use matplotlib which provides us with the bin values:
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
from matplotlib.dates import num2date
fig, ax = plt.subplots(figsize=(8, 5))
#fake data generation
np.random.seed(1234)
n=20
start = pd.to_datetime("2020-11-15")
df = pd.DataFrame({"Time": pd.to_timedelta(np.random.rand(n), unit="D") + start, "A": np.random.randint(1, 100, n)})
#plots histogram, returns counts, bin border values, and the bars themselves
h_vals, h_bins, h_bars = ax.hist(df["Time"], bins=5, edgecolor="white")
#plot x ticks at the place where the bin borders are
ax.set_xticks(h_bins)
#label them with dates in HH:MM format after conversion of the float values that matplotlib uses internally
ax.set_xticklabels([num2date(curr_bin).strftime("%H:%M") for curr_bin in h_bins])
plt.show()
Sample output:
Seaborn and pandas make life easier because they provide convenience wrappers and some additional functionality for commonly used plotting functions. However, if they do not suffice in the parameters they provide, one has often to revert to matplotlib which is more flexible in what it can do. Obviously, there might be an easier way in pandas or seaborn, I am not aware of. I will happily upvote any better suggestion within these libraries.
I am trying to generate a smooth line using a dataset that contains time (measured as number of days) and a set of numbers that represent a socioeconomic variable.
Here is a sample of my data:
date, data
726,1.2414
727,1.2414
728,1.2414
729,1.2414
730,1.2414
731,1.2414
732,1.2414
733,1.2414
734,1.2414
735,1.2414
736,1.2414
737,1.804597701
738,1.804597701
739,1.804597701
740,1.804597701
741,1.804597701
742,1.804597701
743,1.804597701
744,1.804597701
745,1.804597701
746,1.804597701
747,1.804597701
748,1.804597701
749,1.804597701
750,1.804597701
751,1.804597701
752,1.793103448
753,1.793103448
754,1.793103448
755,1.793103448
756,1.793103448
757,1.793103448
758,1.793103448
759,1.793103448
760,1.793103448
761,1.793103448
762,1.793103448
763,1.793103448
764,1
765,1
This is my code so far:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
%matplotlib inline
out_file = "path_to_file/file.csv"
df = pd.read_csv(out_file)
time = df['date']
data = df['data']
ax1 = plt.subplot2grid((4,3),(0,0), colspan = 2, rowspan = 2) # Will be adding other plots
plt.plot(time, data)
plt.yticks(np.arange(1,5,1)) # Include classes 1-4 showing only 1 step changes
plt.gca().invert_yaxis() # Reverse y axis
plt.ylabel('Trend', fontsize = 8, labelpad = 10)
This generates the following plot:
Test plot
I have seen posts that answer similar questions (like the ones below), but can't seem to get my code to work. Can anyone suggest an elegant solution?
Generating smooth line graph using matplotlib
Python Matplotlib - Smooth plot line for x-axis with date values
Im trying to smooth a graph line out but since the x-axis values are dates im having great trouble doing this. Say we have a dataframe as follows
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
%matplotlib inline
startDate = '2015-05-15'
endDate = '2015-12-5'
index = pd.date_range(startDate, endDate)
data = np.random.normal(0, 1, size=len(index))
cols = ['value']
df = pd.DataFrame(data, index=index, columns=cols)
Then we plot the data
fig, axs = plt.subplots(1,1, figsize=(18,5))
x = df.index
y = df.value
axs.plot(x, y)
fig.show()
we get
Now to smooth this line there are some usefull staekoverflow questions allready like:
Generating smooth line graph using matplotlib,
Plot smooth line with PyPlot
Creating numpy linspace out of datetime
But I just cant seem to get some code working to do this for my example, any suggestions?
You can use interpolation functionality that is shipped with pandas. Because your dataframe has a value for every index already, you can populate it with an index that is more sparse, and fill every previously non-existent indices with NaN values. Then, after choosing one of many interpolation methods available, interpolate and plot your data:
index_hourly = pd.date_range(startDate, endDate, freq='1H')
df_smooth = df.reindex(index=index_hourly).interpolate('cubic')
df_smooth = df_smooth.rename(columns={'value':'smooth'})
df_smooth.plot(ax=axs, alpha=0.7)
df.plot(ax=axs, alpha=0.7)
fig.show()
There is one workaround, we will create two plots - 1) non smoothed /interploted with date labels 2) smoothed without date labels.
Plot the 1) using argument linestyle=" " and convert the dates to be plotted on x-axis to string type.
Plot the 2) using the argument linestyle="-" and interpolating the x-axis and y-axis using np.linespace and make_interp_spline respectively.
Following is the use of the discussed workaround for your code.
# your initial code
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy.interpolate import make_interp_spline
%matplotlib inline
startDate = "2015-05-15"
endDate = "2015-07-5" #reduced the end date so smoothness is clearly seen
index = pd.date_range(startDate, endDate)
data = np.random.normal(0, 1, size=len(index))
cols = ["value"]
df = pd.DataFrame(data, index=index, columns=cols)
fig, axs = plt.subplots(1, 1, figsize=(40, 12))
x = df.index
y = df.value
# workaround by creating linespace for length of your x axis
x_new = np.linspace(0, len(df.index), 300)
a_BSpline = make_interp_spline(
[i for i in range(0, len(df.index))],
df.value,
k=5,
)
y_new = a_BSpline(x_new)
# plot this new plot with linestyle = "-"
axs.plot(
x_new[:-5], # removing last 5 entries to remove noise, because interpolation outputs large values at the end.
y_new[:-5],
"-",
label="interpolated"
)
# to get the date on x axis we will keep our previous plot but linestyle will be None so it won't be visible
x = list(x.astype(str))
axs.plot(x, y, linestyle=" ", alpha=0.75, label="initial")
xt = [x[i] for i in range(0,len(x),5)]
plt.xticks(xt,rotation="vertical")
plt.legend()
fig.show()
Resulting Plot
Overalpped plot to see the smoothing.
Depending on what exactly you mean by "smoothing," the easiest way can be the use of savgol_filter or something similar. Unlike with interpolated splines, this method means that the smoothed line does not pass through the measured points, effectively filtering out higher-frequency noise.
from scipy.signal import savgol_filter
...
windowSize = 21
polyOrder = 1
smoothed = savgol_filter(values, windowSize, polyOrder)
axes.plot(datetimes, smoothed, color=chart.color)
The higher the polynomial order value, the closer the smoothed line is to the raw data.
Here is an example.