Develop Reference

How do I use matplotlib to create a bar chart of a very large dataset?

The data I am working with is an array 27,000 elements long which is a histogram of a few million data points but what I have is the histogram and I need to plot it in my program, preferably with vertical bars.
I've tried using the 'bar' function in matplotlib but this takes a minute or two to plot whereas using just regular plot (with just points on the chart) is almost immediate but obviously does not achieve the effect I want (i.e. bars). I'm not sure why the bar function is so much slower so I was wondering if there was a more effective way to plot a histogram with vertical bars using matplotlib?
I've looked at the hist function with matplotlib but it's purpose to my understanding is to take data, make a histogram, and then plot it but I already have a histogram so I don't believe it works for my case. I greatly appreciate any help!
Here's a reference to the hist function documentation, maybe I missed something.
https://matplotlib.org/3.2.0/api/_as_gen/matplotlib.pyplot.hist.html
Thanks in advance! Let me know if you would like an example of the code I am working with but it is just your most generic my_axes.plot(my_data) or my_axes.bar(my_data) so I'm not sure how helpful it would be.
I've taken a look at this as well now: https://gist.github.com/pierdom/d639a1d3b8934ee31db8b2ab9997ae92.
This also works but has the same time issue as using bar so I suppose this is just an issue with rendering a lot of vertical bars? (though I still wonder why rendering 27000 points happens so quickly)

Apparently, this is a known and discussed limitation of the bar graph as it is currently implemented. See this issue and this discussion. Though there are questions about it's usefulness, in my particular case I have a toolbar across the top that allows the user to zoom in and move around the data set (which is very practical method for my use case).
However, a great alternative does exist in the form of stairs. Simply use fill and you have an effective bar graph, that is much more performant.
import matplotlib.pyplot as plt
import random
bins = range(27001) # Note that bins needs to be one greater then heights
heights = [random.randint(0, i) for i in range(27000)]
ax = plt.gca()
ax.stairs(heights, bins, fill=True)
plt.show()

matplotlib's bar should be pretty fast to execute so I'm guessing you're passing all the data points to it (although you mention you have "histogram data", so if you can provide more details on the format, that'd help).
bar takes the x positions for the bars and the heights, so if you want the bar function to produce a histogram you need to bin and count.
This will produce something similar to matplotlib's hist:
import matplotlib.pyplot as plt
bins = [0, 1, 2, 3]
heights = [1, 2, 3, 4]
ax = plt.gca()
ax.bar(bins, heights, align='center', width=1)

Data visualization in python (matplotlib) [duplicate]

I'm not really new to matplotlib and I'm deeply ashamed to admit I have always used it as a tool for getting a solution as quick and easy as possible. So I know how to get basic plots, subplots and stuff and have quite a few code which gets reused from time to time...but I have no "deep(er) knowledge" of matplotlib.
Recently I thought I should change this and work myself through some tutorials. However, I am still confused about matplotlibs plt, fig(ure) and ax(arr). What is really the difference?
In most cases, for some "quick'n'dirty' plotting I see people using just pyplot as plt and directly plot with plt.plot. Since I am having multiple stuff to plot quite often, I frequently use f, axarr = plt.subplots()...but most times you see only code putting data into the axarr and ignoring the figure f.
So, my question is: what is a clean way to work with matplotlib? When to use plt only, what is or what should a figure be used for? Should subplots just containing data? Or is it valid and good practice to everything like styling, clearing a plot, ..., inside of subplots?
I hope this is not to wide-ranging. Basically I am asking for some advice for the true purposes of plt <-> fig <-> ax(arr) (and when/how to use them properly).
Tutorials would also be welcome. The matplotlib documentation is rather confusing to me. When one searches something really specific, like rescaling a legend, different plot markers and colors and so on the official documentation is really precise but rather general information is not that good in my opinion. Too much different examples, no real explanations of the purposes...looks more or less like a big listing of all possible API methods and arguments.

pyplot is the 'scripting' level API in matplotlib (its highest level API to do a lot with matplotlib). It allows you to use matplotlib using a procedural interface in a similar way as you can do it with Matlab. pyplot has a notion of 'current figure' and 'current axes' that all the functions delegate to (#tacaswell dixit). So, when you use the functions available on the module pyplot you are plotting to the 'current figure' and 'current axes'.
If you want 'fine-grain' control of where/what your are plotting then you should use an object oriented API using instances of Figure and Axes.
Functions available in pyplot have an equivalent method in the Axes.
From the repo anatomy of matplotlib:
The Figure is the top-level container in this hierarchy. It is the overall window/page that everything is drawn on. You can have multiple independent figures and Figures can contain multiple Axes.
But...
Most plotting occurs on an Axes. The axes is effectively the area that we plot data on and any ticks/labels/etc associated with it. Usually we'll set up an Axes with a call to subplot (which places Axes on a regular grid), so in most cases, Axes and Subplot are synonymous.
Each Axes has an XAxis and a YAxis. These contain the ticks, tick locations, labels, etc.
If you want to know the anatomy of a plot you can visit this link.

I think that this tutorial explains well the basic notions of the object hierarchy of matplotlib like Figure and Axes, as well as the notion of current figure and current Axes.
If you want a quick answer: There is the Figure object which is the container that wraps multiple Axes(which is different from axis) which also contains smaller objects like legends, line, tick marks ... as shown in this image taken from matplotlib documentation
So when we do
>>> import matplotlib.pyplot as plt
>>> fig, ax = plt.subplots()
>>> type(fig)
<class 'matplotlib.figure.Figure'>
>>> type(ax)
<class 'matplotlib.axes._subplots.AxesSubplot'>
We have created a Figure object and an Axes object that is contained in that figure.

pyplot is matlab like API for those who are familiar with matlab and want to make quick and dirty plots
figure is object-oriented API for those who doesn't care about matlab style plotting
So you can use either one but perhaps not both together.

Python Heatmaps (Basic and Complex)

What's the best way to do a heatmap in python (2.7)? I've found the heatmap.py module, and I was wondering if people have any advice on using it, or if there are other packages that do a good job.
I'm dealing with pretty basic data, like xy = np.random.rand(1000,2) superimposed on an image.
Although there's another thing I want to try, which is doing a heatmap that's scaled to a different heatmap. E.g., I have
attempts = np.random.rand(5000,2)
successes = np.random.rand(500,2)
And I want a heatmap of the successes relative to the density of the attempts. Is this possible?

Seaborn is a pretty widely-used library for making nice-looking plots, and has a heatmap function. Seaborn uses matplotlib under the hood.
import numpy as np
import seaborn as sns
xy = np.random.rand(1000,2)
sns.heatmap(xy, yticklabels=100)
Regarding your second question, I'm not sure what you mean. But my advice would be to create a numpy array or pandas dataframe of "successes [scaled] relative to the density of the attempts", however you mean that, and then pass that scaled array or dataframe to sns.heatmap

You can plot very complex heatmap using python package PyComplexHeatmap: https://github.com/DingWB/PyComplexHeatmap
https://github.com/DingWB/PyComplexHeatmap/blob/main/examples.ipynb

The most basic heatmap you can get is an image plot:
import matplotlib.pyplot as plt
import numpy as np
xy = np.random.rand(100,2)
plt.imshow(xy, aspect="auto")
plt.colorbar()
plt.show()
Note that using more points than you have pixels to show the heatmap might not make too much sense.
There are of course also different methods to draw a heatmaps and you may go through the matplotlib example gallery and see which plot appeals most to you.

matplot and seaborn figure parameters/customizations

I'm so confused between the two. Every time I make a chart on either pyplot or seaborn, I have to guess what syntax to use. For example, for seaborn doesn't have a title setter so I have to remember to use plt.title. Or, for seaborn charts, plt.xlabel doesn't work, so I have to use sns.axlable(x,y).
And also, randomly I run into the following problem. I'm simply trying to make my seaborn jointplot bigger but I have no success trying both the plt nor the seaborn methods (any tips as to a good documentation showing all the chart parameters??? I find them scattered on the web and it seems like each solution on stack overflow is unique...which adds to the overall confusion).
Here's my code:
a = plt.figure(figsize=(30,30))
a.set_size_inches(30,30)
sns.jointplot(x='COAST',y='NORTH',data = data_df, kind = 'kde')
Notice I used the plt method and the sns.set_size_inches methods. Both gave me a small chart.
So frustrated with the random overlaps of the two libraries. Any pro tips to lessen the confusion will be greatly appreciated!
edit: This is also true for seaborn's pairplot. I have no success in changing the pairplot's size.

sns.jointplot creates its own figure instance (as #tcaswell suspected). It doesn't appear that you can tell jointplot to use an existing figure. I think you have two options:
You can give sns.jointplot the size option. e.g.:
sns.jointplot(x='COAST', y='NORTH', data=data_df, kind='kde', size=30)
You can alter the JointGrid figure size after creating it, using:
g=sns.jointplot(x='COAST', y='NORTH', data=data_df, kind='kde')
g.fig.set_size_inches(30,30)
I presume option 1 is the better option, as it is a built-in seaborn option

Barchart (o plot) 3D in Python

I need to plot some data in various forms. Currently I'm using Matplotlib and I'm fairly happy with the plots I'm able to produce.
This question is on how to plot the last one. The data is similar to the "distance table", like this (just bigger, my table is 128x128 and still have 3 or more number per element).
Now, my data is much better "structured" than a distance table (my data doesn't varies "randomly" like in a alphabetically sorted distance table), thus a 3D barchart, or maybe 3 of them, would be perfect. My understanding is that such a chart is missing in Matplotlib.
I could use a (colored) Countor3d like these or something in 2D like imshow, but it isn't really well representative of what the data is (the data has meaning just in my 128 points, there isn't anything between two points). And the height of bars is more readable than color, IMO.
Thus the questions:
is it possible to create 3D barchart in Matplotlib? It should be clear that I mean with a 2D domain, not just a 2D barchart with a "fake" 3D rendering for aesthetics purposes
if the answer to the previous question is no, then is there some other library able to do that? I strongly prefer something Python-based, but I'm OK with other Linux-friendly possibilities
if the answer to the previous question is no, then do you have any suggestions on how to show that data? E.g. create a table with the values, superimposed to the imshow or other colored way?

For some time now, matplotlib had no 3D support, but it has been added back recently. You will need to use the svn version, since no release has been made since, and the documentation is a little sparse (see examples/mplot3d/demo.py). I don't know if mplot3d supports real 3D bar charts, but one of the demos looks a little like it could be extended to something like that.
Edit: The source code for the demo is in the examples but for some reason the result is not. I mean the test_polys function, and here's how it looks like:
example figure http://www.iki.fi/jks/tmp/poly3d.png
The test_bar2D function would be even better, but it's commented out in the demo as it causes an error with the current svn version. Might be some trivial problem, or something that's harder to fix.

MyavaVi2 can make 3D barcharts (scroll down a bit). Once you have MayaVi/VTK/ETS/etc. installed it all works beautifully, but it can be some work getting it all installed. Ubuntu has all of it packaged, but they're the only Linux distribution I know that does.

One more possibility is Gnuplot, which can draw some kind of pseudo 3D bar charts, and gnuplot.py allows interfacing to Gnuplot from Python. I have not tried it myself, though.

This is my code for a simple Bar-3d using matplotlib.
import mpl_toolkits
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
%matplotlib inline
## The value you want to plot
zval=[0.020752244,0.078514652,0.170302899,0.29543857,0.45358061,0.021255922,0.079022499,\
0.171294169,0.29749654,0.457114286,0.020009631,0.073154019,0.158043498,0.273889264,0.419618287]
fig = plt.figure(figsize=(12,9))
ax = fig.add_subplot(111,projection='3d')
col=["#ccebc5","#b3cde3","#fbb4ae"]*5
xpos=[1,2,3]*5
ypos=range(1,6,1)*5
zpos=[0]*15
dx=[0.4]*15
dy=[0.5]*15
dz=zval
for i in range(0,15,1):
ax.bar3d(ypos[i], xpos[i], zpos[i], dx[i], dy[i], dz[i],
color=col[i],alpha=0.75)
ax.view_init(azim=120)
plt.show()
http://i8.tietuku.com/ea79b55837914ab2.png

You might check out Chart Director:
http://www.advsofteng.com
It has a pretty wide variety of charts and graphs and has a nice Python (and several other languages) API.
There are two editions: The free version puts a blurb on the generated image, and the
pay version is pretty reasonably priced.
Here's one of the more interesting looking 3d stacked bar charts:
(source: advsofteng.com)