I am using matplotlib's hist2d function to make a 2d histogram of data that I have, however I am having trouble interpreting the result.
Here is the plot I have:
This was created using the line:
hist = plt.hist2d(X, Y, (160,160), norm=mpl.colors.LogNorm(vmin=1, vmax=20))
This returns a 2d array of (160, 160), as well as the bin edges etc.
In the plot there are bins which have a high frequency of values (yellow bins). I would like to be able to get the results of this histogram and filter out the bins that have low values, preserving the high bins. But I would expect there to be 160*160 values, but I can only find 160 X and 160 Y values.
What I would like to do is essentially filter out the more dense data from the less dense data. If this means representing the data as a single value (a bin), then that is ok.
Am I misinterpreting the function or am I not accessing the data results correctly? I have tried with spicy also but the results seem to be in the same or similar format.
Not sure if this is what you wanted.
The hist2d docs specify that the function returns a tuple of size 4, where the first item h is a heatmap.
This h will have the same shape as bins.
You can capture the output (it will still plot), and use argwhere to find coordinates where values exceed, say, the 90th percentile:
h, xedges, yedges, img = hist = plt.hist2d(X, Y, bins=(160,160), norm=mpl.colors.LogNorm(vmin=1, vmax=20))
print(list(np.argwhere(h > np.percentile(h, 90))))
You need Seaborn package.
You mentioned
I would like to be able to get the results of this histogram and filter out the bins that have low values, preserving the high bins.
You should definitely be using one of those:
seaborn.joinplot(...,kind='hex') : it shows the counts of observations that fall within hexagonal bins. This plot works best with relatively large dataset.
seaborn.joinplot(...,kind='kde') : use the kernel density estimation to visualize a bivariate distribution. I recommed it better.
Example 'kde'
Use number of levels n_levels and shade_lowest=False to ignore low values.
import seaborn as sns
import numpy as np
import matplotlib.pylab as plt
x, y = np.random.randn(2, 300)
plt.figure(figsize=(6,5))
sns.kdeplot(x, y, zorder=0, n_levels=6, shade=True, cbar=True,
shade_lowest=False, cmap='viridis')
I am attempting to show the results of a tensorflow model onto a timeplot. I have two arrays, one for the predictions of the model and one for the actual values. Both arrays are of the size (3500,) with values ranging from 0-16. I want to be able to show a timeplot that displays the predicted value and the actual value at a certain point in time. I do not know how to setup a timeplot that has the values on the y-axis and time on the x-axis.
I am currently using the Matplotlib library to make this work but I am not too familiar with it
plt.scatter(max_test,max_predictions)
plt.show()
This gives me a scatter plot with the values (0-16) on both the y and x-axis.
I want to replace the values on the x-axis with the time that each value occurred at (preferably every 5 seconds) and only show one value for both the arrays. Ideally it would look something like this:
If you have the start time and stop time, you can import numpy and use linspace for the x axis:
x_axis_val = np.linspace(start, stop, len(max_test))
To plot both sets of y-values on the same graph do:
plots = [max_predictions, max_test]
for plot in plots:
plt.scatter(x_axis_val,plot)
plt.show()
I have a CDF plot with data of wifi usage in MB. For better understanding I would like to present the usage starting in KB and finishing in TB. I would like to know how to set a specific range for x axis to replace the produce by plt.plot() and show the axis x, per example, as [1KB 10KB 1MB 10MB 1TB 10TB], even the space between bins not representing the real values.
My code for now:
wifi = np.sort(matrix[matrix['wifi_total_mb']>0]['wifi_total_mb'].values)
g = sns.distplot(wifi, kde_kws=dict(cumulative=True))
plt.show()
Thanks
EDIT 1
I know that I can use plt.xticks and i already tried it: plt.xticks([0.00098, 0.00977, 1, 10, 1024, 10240, 1048576, 10485760, 24117248]). These are values in MB that represents the sample range I specified before. But the plot is still wrong.
The result expected
In excel it is pretty easy makes what I want to. Look the image, with the same range I get the plot I wanted.
Thanks
It may be better to calculate the data to plot manually, instead of relying on some seaborn helper function like distplot. This also makes it easier to understand the underlying issue of histogramming with very unequal bin sizes.
Calculating histogram
The histogram of the data can be calculated by using np.histogram(). It can take the desired bins as argument.
In order to get the cummulative histogram, np.cumsum does the job.
Now there are two options here: (a) plotting the real data or (b) plotting the data enumerated by bin.
(a) Plotting the real data:
Because the bin sizes are pretty unequal, a logarithmic scaling seems adequate, which can be done by semilogx(x,y). The bin edges can be shown as xticks using set_xticks (and since the semilogx plot will not automatically set the labels correctly, we also need to set them to the bin edges' values).
(b) Plotting data enumerated by bin:
The second option is to plot the histogram values bin by bin, independent of the actual bin size. Is is very close to the Excel solution from the question. In this case the x values of the plot are simply values from 0 to number of bins and the xticklabels are the bin edges.
Here is the complete code:
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
#use the bin from the question
bins = [0, 0.00098, 0.00977, 1, 10, 1024, 10240, 1048576, 10485760, 24117248]
# invent some data
data = np.random.lognormal(2,4,10000)
# calculate histogram of the data into the given bins
hist, _bins = np.histogram(data, bins=bins)
# make histogram cumulative
cum_hist=np.cumsum(hist)
# normalize data to 1
norm_cum_hist = cum_hist/float(cum_hist.max())
fig, (ax, ax2) = plt.subplots(nrows=2)
plt.subplots_adjust(hspace=0.5, bottom=0.17)
# First option plots the actual data, i.e. the bin width is reflected
# by the spacing between values on x-axis.
ax.set_title("Plotting actual data")
ax.semilogx(bins[1:],norm_cum_hist, marker="s")
ax.set_xticks(bins[1:])
ax.set_xticklabels(bins[1:] ,rotation=45, horizontalalignment="right")
# Second option plots the data bin by bin, i.e. every bin has the same width,
# independent of it's actual value.
ax2.set_title("Plotting bin by bin")
ax2.plot(range(len(bins[1:])),norm_cum_hist, marker="s")
ax2.set_xticks(range(len(bins[1:])))
ax2.set_xticklabels(bins[1:] ,rotation=45, horizontalalignment="right")
for axes in [ax, ax2]:
axes.set_ylim([0,1.05])
plt.show()
How do I plot a histogram using matplotlib.pyplot.hist?
I have a list of y-values that correspond to bar height, and a list of x-value strings.
Related: matplotlib.pyplot.bar.
If you want a histogram, you don't need to attach any 'names' to x-values because:
on x-axis you will have data bins
on y-axis counts (by default) or frequencies (density=True)
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
np.random.seed(42)
x = np.random.normal(size=1000)
plt.hist(x, density=True, bins=30) # density=False would make counts
plt.ylabel('Probability')
plt.xlabel('Data');
Note, the number of bins=30 was chosen arbitrarily, and there is Freedman–Diaconis rule to be more scientific in choosing the "right" bin width:
, where IQR is Interquartile range and n is total number of datapoints to plot
So, according to this rule one may calculate number of bins as:
q25, q75 = np.percentile(x, [25, 75])
bin_width = 2 * (q75 - q25) * len(x) ** (-1/3)
bins = round((x.max() - x.min()) / bin_width)
print("Freedman–Diaconis number of bins:", bins)
plt.hist(x, bins=bins);
Freedman–Diaconis number of bins: 82
And finally you can make your histogram a bit fancier with PDF line, titles, and legend:
import scipy.stats as st
plt.hist(x, density=True, bins=82, label="Data")
mn, mx = plt.xlim()
plt.xlim(mn, mx)
kde_xs = np.linspace(mn, mx, 300)
kde = st.gaussian_kde(x)
plt.plot(kde_xs, kde.pdf(kde_xs), label="PDF")
plt.legend(loc="upper left")
plt.ylabel("Probability")
plt.xlabel("Data")
plt.title("Histogram");
If you're willing to explore other opportunities, there is a shortcut with seaborn:
# !pip install seaborn
import seaborn as sns
sns.displot(x, bins=82, kde=True);
Now back to the OP.
If you have limited number of data points, a bar plot would make more sense to represent your data. Then you may attach labels to x-axis:
x = np.arange(3)
plt.bar(x, height=[1,2,3])
plt.xticks(x, ['a','b','c']);
If you haven't installed matplotlib yet just try the command.
> pip install matplotlib
Library import
import matplotlib.pyplot as plot
The histogram data:
plot.hist(weightList,density=1, bins=20)
plot.axis([50, 110, 0, 0.06])
#axis([xmin,xmax,ymin,ymax])
plot.xlabel('Weight')
plot.ylabel('Probability')
Display histogram
plot.show()
And the output is like :
This is an old question but none of the previous answers has addressed the real issue, i.e. that fact that the problem is with the question itself.
First, if the probabilities have been already calculated, i.e. the histogram aggregated data is available in a normalized way then the probabilities should add up to 1. They obviously do not and that means that something is wrong here, either with terminology or with the data or in the way the question is asked.
Second, the fact that the labels are provided (and not intervals) would normally mean that the probabilities are of categorical response variable - and a use of a bar plot for plotting the histogram is best (or some hacking of the pyplot's hist method), Shayan Shafiq's answer provides the code.
However, see issue 1, those probabilities are not correct and using bar plot in this case as "histogram" would be wrong because it does not tell the story of univariate distribution, for some reason (perhaps the classes are overlapping and observations are counted multiple times?) and such plot should not be called a histogram in this case.
Histogram is by definition a graphical representation of the distribution of univariate variable (see Histogram | NIST/SEMATECH e-Handbook of Statistical Methods & Histogram | Wikipedia) and is created by drawing bars of sizes representing counts or frequencies of observations in selected classes of the variable of interest. If the variable is measured on a continuous scale those classes are bins (intervals). Important part of histogram creation procedure is making a choice of how to group (or keep without grouping) the categories of responses for a categorical variable, or how to split the domain of possible values into intervals (where to put the bin boundaries) for continuous type variable. All observations should be represented, and each one only once in the plot. That means that the sum of the bar sizes should be equal to the total count of observation (or their areas in case of the variable widths, which is a less common approach). Or, if the histogram is normalised then all probabilities must add up to 1.
If the data itself is a list of "probabilities" as a response, i.e. the observations are probability values (of something) for each object of study then the best answer is simply plt.hist(probability) with maybe binning option, and use of x-labels already available is suspicious.
Then bar plot should not be used as histogram but rather simply
import matplotlib.pyplot as plt
probability = [0.3602150537634409, 0.42028985507246375,
0.373117033603708, 0.36813186813186816, 0.32517482517482516,
0.4175257731958763, 0.41025641025641024, 0.39408866995073893,
0.4143222506393862, 0.34, 0.391025641025641, 0.3130841121495327,
0.35398230088495575]
plt.hist(probability)
plt.show()
with the results
matplotlib in such case arrives by default with the following histogram values
(array([1., 1., 1., 1., 1., 2., 0., 2., 0., 4.]),
array([0.31308411, 0.32380469, 0.33452526, 0.34524584, 0.35596641,
0.36668698, 0.37740756, 0.38812813, 0.39884871, 0.40956928,
0.42028986]),
<a list of 10 Patch objects>)
the result is a tuple of arrays, the first array contains observation counts, i.e. what will be shown against the y-axis of the plot (they add up to 13, total number of observations) and the second array are the interval boundaries for x-axis.
One can check they they are equally spaced,
x = plt.hist(probability)[1]
for left, right in zip(x[:-1], x[1:]):
print(left, right, right-left)
Or, for example for 3 bins (my judgment call for 13 observations) one would get this histogram
plt.hist(probability, bins=3)
with the plot data "behind the bars" being
The author of the question needs to clarify what is the meaning of the "probability" list of values - is the "probability" just a name of the response variable (then why are there x-labels ready for the histogram, it makes no sense), or are the list values the probabilities calculated from the data (then the fact they do not add up to 1 makes no sense).
This is a very round-about way of doing it but if you want to make a histogram where you already know the bin values but dont have the source data, you can use the np.random.randint function to generate the correct number of values within the range of each bin for the hist function to graph, for example:
import numpy as np
import matplotlib.pyplot as plt
data = [np.random.randint(0, 9, *desired y value*), np.random.randint(10, 19, *desired y value*), etc..]
plt.hist(data, histtype='stepfilled', bins=[0, 10, etc..])
as for labels you can align x ticks with bins to get something like this:
#The following will align labels to the center of each bar with bin intervals of 10
plt.xticks([5, 15, etc.. ], ['Label 1', 'Label 2', etc.. ])
Though the question appears to be demanding plotting a histogram using matplotlib.hist() function, it can arguably be not done using the same as the latter part of the question demands to use the given probabilities as the y-values of bars and given names(strings) as the x-values.
I'm assuming a sample list of names corresponding to given probabilities to draw the plot. A simple bar plot serves the purpose here for the given problem. The following code can be used:
import matplotlib.pyplot as plt
probability = [0.3602150537634409, 0.42028985507246375,
0.373117033603708, 0.36813186813186816, 0.32517482517482516,
0.4175257731958763, 0.41025641025641024, 0.39408866995073893,
0.4143222506393862, 0.34, 0.391025641025641, 0.3130841121495327,
0.35398230088495575]
names = ['name1', 'name2', 'name3', 'name4', 'name5', 'name6', 'name7', 'name8', 'name9',
'name10', 'name11', 'name12', 'name13'] #sample names
plt.bar(names, probability)
plt.xticks(names)
plt.yticks(probability) #This may be included or excluded as per need
plt.xlabel('Names')
plt.ylabel('Probability')
i try to plot data in a histogram or bar in python. The data size (array size) is between 0-10000. The data itself (each entry of the array) depends on the input and has a range between 0 and e+20 (mostly the data is in th same range). So i want to do a hist plot with matplotlib. I want to plot how often a data is in some intervall (to illustrate the mean and deviation). Sometimes it works like this:
hist1.
But sometimes there is a problem with the intevall size like this:
hist2.
In this plot i need more bars at point 0-100 etc.
Can anyone help me with this?
The plots are just made with:
from numpy.linalg import *
import matplotlib.pyplot as plt
plt.hist(numbers,bins=100)
plt.show()
By default, hist produces a plot with an x range that covers the full range of your data.
If you have one outsider at very high x in comparison with the other values, then you will see this image with a 'compressed' figure.
I you want to have always the same view you can fix the limits with xlim.
Alternatively, if you want to see your distribution always centered and as nicer as possible, you can calculate the mean and the standard deviation of your data and fix the x range accordingly (p.e. for mean +/- 5 stdev)