Develop Reference

Make all data points of a matplotlib plot homogeneously transparent

I'd like to plot two scatter plots into the same Axes and turn the upper one's data points transparent such that the other plot shines through. However, I want the whole upper plot to have a homogeneous transparency level, such that superimposed markers of the upper plot do not add up their opacity as they would do if I simply set alpha=0.5.
In other words, I'd like both scatter plots to be rendered first and being set to one constant transparency level. Technically this should be possible for both raster and vector graphics (as SVG supports layer transparency, afaik), but either would be fine for me.
Here is some example code that displays what I do not want to achieve. ;)
import numpy as np
import matplotlib.pyplot as plt
fig = plt.figure(1, figsize=(6,4), dpi=160)
ax = fig.gca()
s1 = ax.scatter(np.random.randn(1000), np.random.randn(1000), color="b", edgecolors="none")
s2 = ax.scatter(np.random.randn(1000), np.random.randn(1000), color="g", edgecolors="none")
s2.set_alpha(0.5) # sadly the same as setting `alpha=0.5`
fig.show() # or display(fig)
I'd like the green markers around (2,2) to not be darker where they superimpose, for example. Is this possible with matplotlib?
Thanks for your time! :)

After searching some more, I found related questions and two solutions, of which at least one kind of works for me:
As I hoped one can render one layer and then afterwards blend them together like this:
import numpy as np
import matplotlib.pyplot as plt
fig = plt.figure(1, figsize=(6,4), dpi=160)
ax1 = fig.gca()
s1 = ax1.scatter(np.random.randn(1000), np.random.randn(1000), color="#3355ff", edgecolors="none")
ax1.set_xlim(-3.5,3.5)
ax1.set_ylim(-3.5,3.5)
ax1.patch.set_facecolor("none")
ax1.patch.set_edgecolor("none")
fig.canvas.draw()
w, h = fig.canvas.get_width_height()
img1 = np.frombuffer(fig.canvas.buffer_rgba(), np.uint8).reshape(h, w, -1).copy()
ax1.clear()
s2 = ax1.scatter(np.random.randn(1000), np.random.randn(1000), color="#11aa44", edgecolors="none")
ax1.set_xlim(-3.5,3.5)
ax1.set_ylim(-3.5,3.5)
ax1.patch.set_facecolor("none")
ax1.patch.set_edgecolor("none")
fig.canvas.draw()
img2 = np.frombuffer(fig.canvas.buffer_rgba(), np.uint8).reshape(h, w, -1).copy()
fig.clf()
plt.imshow(np.minimum(img1,img2))
plt.subplots_adjust(0, 0, 1, 1)
plt.axis("off")
plt.show()
I may have to come up with better methods than just taking the np.minimum of both layers to keep more color options (and probably save the axes and labels to a third layer).
I did not try mplcairo as suggested in the other linked answer as it has too many dependencies for my use case (my solution should be portable).
I am still happy for additional input. :)

Adding PatchCollection with Affine Transformations

I have some patches on which I apply different Affine2D transformations in matplotlib.
Is there a possibility to add them as a collections.PatchCollection? Somehow I am only able to draw them if I call ax.add_patch() separately for each of them.
from matplotlib import pyplot as plt, patches, collections, transforms
fig, ax = plt.subplots()
trafo1 = transforms.Affine2D().translate(+0.3, -0.3).rotate_deg_around(0, 0, 45) + ax.transData
trafo2 = transforms.Affine2D().translate(+0.3, -0.3).rotate_deg_around(0, 0, 65) + ax.transData
rec1 = patches.Rectangle(xy=(0.1, 0.1), width=0.2, height=0.3, transform=trafo1, color='blue')
rec2 = patches.Rectangle(xy=(0.2, 0.2), width=0.3, height=0.2, transform=trafo2, color='green')
ax.add_collection(collections.PatchCollection([rec1, rec2], color='red', zorder=10))
# ax.add_patch(rec1)
# ax.add_patch(rec2)

It looks like PatchCollection does not support individually transformed elements. From the Matplotlib documentation, we can read a Collection is a
class for the efficient drawing of large collections of objects that share most properties, e.g., a large number of line segments or polygons.
You can understand this with creating the collection without any individually transformed patches:
rec1 = patches.Rectangle(xy=(0.1, 0.1), width=0.2, height=0.3, color='blue')
rec2 = patches.Rectangle(xy=(0.2, 0.2), width=0.3, height=0.2, color='green')
col = collections.PatchCollection([rec1, rec2], color='red', zorder=10)
print(col.get_transform())
That prints IdentityTransform() for the last statement, and correctly displays the (non-transformed) patches. These patches can be transformed all-at-once from the PatchCollection, without individual specification.
On the contrary, when you apply the individual transform for each patch (like in your case), the .get_transform() method returns an empty list. This is probably due to the fact that PatchCollection classes are made to gather patches with a lot of common attributes in order to accelerate the drawing efficiency (as mentioned above), including the transform attribute.
Note: on this answer, you can find a workaround with a patch to path conversion, then to a PathCollection with an increase drawing efficiency compared to individual patch draw.

Is there a way to plot a Line2D in points coordinates in Matplotlib in Python?

It is very straight forward to plot a line between two points (x1, y1) and (x2, y2) in Matplotlib using Line2D:
Line2D(xdata=(x1, x2), ydata=(y1, y2))
But in my particular case I have to draw Line2D instances using Points coordinates on top of the regular plots that are all using Data coordinates. Is that possible?

As #tom mentioned, the key is the transform kwarg. If you want an artist's data to be interpreted as being in "pixel" coordinates, specify transform=IdentityTransform().
Using Transforms
Transforms are a key concept in matplotlib. A transform takes coordinates that the artist's data is in and converts them to display coordinates -- in other words, pixels on the screen.
If you haven't already seen it, give the matplotlib transforms tutorial a quick read. I'm going to assume a passing familiarity with the first few paragraphs of that tutorial, so if you're
For example, if we want to draw a line across the entire figure, we'd use something like:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
# The "clip_on" here specifies that we _don't_ want to clip the line
# to the extent of the axes
ax.plot([0, 1], [0, 1], lw=3, color='salmon', clip_on=False,
transform=fig.transFigure)
plt.show()
This line will always extend from the lower-left corner of the figure to the upper right corner, no matter how we interactively resize/zoom/pan the plot.
Drawing in pixels
The most common transforms you'll use are ax.transData, ax.transAxes, and fig.transFigure. However, to draw in points/pixels, you actually want no transform at all. In that case, you'll make a new transform instance that does nothing: the IdentityTransform. This specifies that the data for the artist is in "raw" pixels.
Any time you'd like to plot in "raw" pixels, specify transform=IdentityTransform() to the artist.
If you'd like to work in points, recall that there are 72 points to an inch, and that for matplotlib, fig.dpi controls the number of pixels in an "inch" (it's actually independent of the physical display). Therefore, we can convert points to pixels with a simple formula.
As an example, let's place a marker 30 points from the bottom-left edge of the figure:
import matplotlib.pyplot as plt
from matplotlib.transforms import IdentityTransform
fig, ax = plt.subplots()
points = 30
pixels = fig.dpi * points / 72.0
ax.plot([pixels], [pixels], marker='o', color='lightblue', ms=20,
transform=IdentityTransform(), clip_on=False)
plt.show()
Composing Transforms
One of the more useful things about matplotlib's transforms is that they can be added to create a new transform. This makes it easy to create shifts.
For example, let's plot a line, then add another line shifted by 15 pixels in the x-direction:
import matplotlib.pyplot as plt
from matplotlib.transforms import Affine2D
fig, ax = plt.subplots()
ax.plot(range(10), color='lightblue', lw=4)
ax.plot(range(10), color='gray', lw=4,
transform=ax.transData + Affine2D().translate(15, 0))
plt.show()
A key thing to keep in mind is that the order of the additions matters. If we did Affine2D().translate(15, 0) + ax.transData instead, we'd shift things by 15 data units instead of 15 pixels. The added transforms are "chained" (composed would be a more accurate term) in order.
This also makes it easy to define things like "20 pixels from the right hand side of the figure". For example:
import matplotlib.pyplot as plt
from matplotlib.transforms import Affine2D
fig, ax = plt.subplots()
ax.plot([1, 1], [0, 1], lw=3, clip_on=False, color='salmon',
transform=fig.transFigure + Affine2D().translate(-20, 0))
plt.show()

You can use the transform keyword to change between data coordinate (the default) and axes coordinates. For example:
import matplotlib.pyplot as plt
import matplotlib.lines as lines
plt.plot(range(10),range(10),'ro-')
myline = lines.Line2D((0,0.5,1),(0.5,0.5,0),color='b') # data coords
plt.gca().add_artist(myline)
mynewline = lines.Line2D((0,0.5,1),(0.5,0.5,0),color='g',transform=plt.gca().transAxes) # Axes coordinates
plt.gca().add_artist(mynewline)
plt.show()

How to zoomed a portion of image and insert in the same plot in matplotlib

I would like to zoom a portion of data/image and plot it inside the same figure. It looks something like this figure.
Is it possible to insert a portion of zoomed image inside the same plot. I think it is possible to draw another figure with subplot but it draws two different figures. I also read to add patch to insert rectangle/circle but not sure if it is useful to insert a portion of image into the figure. I basically load data from the text file and plot it using a simple plot commands shown below.
I found one related example from matplotlib image gallery here but not sure how it works. Your help is much appreciated.
from numpy import *
import os
import matplotlib.pyplot as plt
data = loadtxt(os.getcwd()+txtfl[0], skiprows=1)
fig1 = plt.figure()
ax1 = fig1.add_subplot(111)
ax1.semilogx(data[:,1],data[:,2])
plt.show()

Playing with runnable code is one of the
fastest ways to learn Python.
So let's start with the code from the matplotlib example gallery.
Given the comments in the code, it appears the code is broken up into 4 main stanzas.
The first stanza generates some data, the second stanza generates the main plot,
the third and fourth stanzas create the inset axes.
We know how to generate data and plot the main plot, so let's focus on the third stanza:
a = axes([.65, .6, .2, .2], axisbg='y')
n, bins, patches = hist(s, 400, normed=1)
title('Probability')
setp(a, xticks=[], yticks=[])
Copy the example code into a new file, called, say, test.py.
What happens if we change the .65 to .3?
a = axes([.35, .6, .2, .2], axisbg='y')
Run the script:
python test.py
You'll find the "Probability" inset moved to the left.
So the axes function controls the placement of the inset.
If you play some more with the numbers you'll figure out that (.35, .6) is the
location of the lower left corner of the inset, and (.2, .2) is the width and
height of the inset. The numbers go from 0 to 1 and (0,0) is the located at the
lower left corner of the figure.
Okay, now we're cooking. On to the next line we have:
n, bins, patches = hist(s, 400, normed=1)
You might recognize this as the matplotlib command for drawing a histogram, but
if not, changing the number 400 to, say, 10, will produce an image with a much
chunkier histogram, so again by playing with the numbers you'll soon figure out
that this line has something to do with the image inside the inset.
You'll want to call semilogx(data[3:8,1],data[3:8,2]) here.
The line title('Probability')
obviously generates the text above the inset.
Finally we come to setp(a, xticks=[], yticks=[]). There are no numbers to play with,
so what happens if we just comment out the whole line by placing a # at the beginning of the line:
# setp(a, xticks=[], yticks=[])
Rerun the script. Oh! now there are lots of tick marks and tick labels on the inset axes.
Fine. So now we know that setp(a, xticks=[], yticks=[]) removes the tick marks and labels from the axes a.
Now, in theory you have enough information to apply this code to your problem.
But there is one more potential stumbling block: The matplotlib example uses
from pylab import *
whereas you use import matplotlib.pyplot as plt.
The matplotlib FAQ says import matplotlib.pyplot as plt
is the recommended way to use matplotlib when writing scripts, while
from pylab import * is for use in interactive sessions. So you are doing it the right way, (though I would recommend using import numpy as np instead of from numpy import * too).
So how do we convert the matplotlib example to run with import matplotlib.pyplot as plt?
Doing the conversion takes some experience with matplotlib. Generally, you just
add plt. in front of bare names like axes and setp, but sometimes the
function come from numpy, and sometimes the call should come from an axes
object, not from the module plt. It takes experience to know where all these
functions come from. Googling the names of functions along with "matplotlib" can help.
Reading example code can builds experience, but there is no easy shortcut.
So, the converted code becomes
ax2 = plt.axes([.65, .6, .2, .2], axisbg='y')
ax2.semilogx(t[3:8],s[3:8])
plt.setp(ax2, xticks=[], yticks=[])
And you could use it in your code like this:
from numpy import *
import os
import matplotlib.pyplot as plt
data = loadtxt(os.getcwd()+txtfl[0], skiprows=1)
fig1 = plt.figure()
ax1 = fig1.add_subplot(111)
ax1.semilogx(data[:,1],data[:,2])
ax2 = plt.axes([.65, .6, .2, .2], axisbg='y')
ax2.semilogx(data[3:8,1],data[3:8,2])
plt.setp(ax2, xticks=[], yticks=[])
plt.show()

The simplest way is to combine "zoomed_inset_axes" and "mark_inset", whose description and
related examples could be found here:
Overview of AxesGrid toolkit
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1.inset_locator import zoomed_inset_axes
from mpl_toolkits.axes_grid1.inset_locator import mark_inset
import numpy as np
def get_demo_image():
from matplotlib.cbook import get_sample_data
import numpy as np
f = get_sample_data("axes_grid/bivariate_normal.npy", asfileobj=False)
z = np.load(f)
# z is a numpy array of 15x15
return z, (-3,4,-4,3)
fig, ax = plt.subplots(figsize=[5,4])
# prepare the demo image
Z, extent = get_demo_image()
Z2 = np.zeros([150, 150], dtype="d")
ny, nx = Z.shape
Z2[30:30+ny, 30:30+nx] = Z
# extent = [-3, 4, -4, 3]
ax.imshow(Z2, extent=extent, interpolation="nearest",
origin="lower")
axins = zoomed_inset_axes(ax, 6, loc=1) # zoom = 6
axins.imshow(Z2, extent=extent, interpolation="nearest",
origin="lower")
# sub region of the original image
x1, x2, y1, y2 = -1.5, -0.9, -2.5, -1.9
axins.set_xlim(x1, x2)
axins.set_ylim(y1, y2)
plt.xticks(visible=False)
plt.yticks(visible=False)
# draw a bbox of the region of the inset axes in the parent axes and
# connecting lines between the bbox and the inset axes area
mark_inset(ax, axins, loc1=2, loc2=4, fc="none", ec="0.5")
plt.draw()
plt.show()

The nicest way I know of to do this is to use mpl_toolkits.axes_grid1.inset_locator (part of matplotlib).
There is a great example with source code here: https://github.com/NelleV/jhepc/tree/master/2013/entry10

The basic steps to zoom up a portion of a figure with matplotlib
import numpy as np
from matplotlib import pyplot as plt
# Generate the main data
X = np.linspace(-6, 6, 1024)
Y = np.sinc(X)
# Generate data for the zoomed portion
X_detail = np.linspace(-3, 3, 1024)
Y_detail = np.sinc(X_detail)
# plot the main figure
plt.plot(X, Y, c = 'k')
# location for the zoomed portion
sub_axes = plt.axes([.6, .6, .25, .25])
# plot the zoomed portion
sub_axes.plot(X_detail, Y_detail, c = 'k')
# insert the zoomed figure
# plt.setp(sub_axes)
plt.show()

Moving matplotlib legend outside of the axis makes it cutoff by the figure box

I'm familiar with the following questions:
Matplotlib savefig with a legend outside the plot
How to put the legend out of the plot
It seems that the answers in these questions have the luxury of being able to fiddle with the exact shrinking of the axis so that the legend fits.
Shrinking the axes, however, is not an ideal solution because it makes the data smaller making it actually more difficult to interpret; particularly when its complex and there are lots of things going on ... hence needing a large legend
The example of a complex legend in the documentation demonstrates the need for this because the legend in their plot actually completely obscures multiple data points.
http://matplotlib.sourceforge.net/users/legend_guide.html#legend-of-complex-plots
What I would like to be able to do is dynamically expand the size of the figure box to accommodate the expanding figure legend.
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(-2*np.pi, 2*np.pi, 0.1)
fig = plt.figure(1)
ax = fig.add_subplot(111)
ax.plot(x, np.sin(x), label='Sine')
ax.plot(x, np.cos(x), label='Cosine')
ax.plot(x, np.arctan(x), label='Inverse tan')
lgd = ax.legend(loc=9, bbox_to_anchor=(0.5,0))
ax.grid('on')
Notice how the final label 'Inverse tan' is actually outside the figure box (and looks badly cutoff - not publication quality!)
Finally, I've been told that this is normal behaviour in R and LaTeX, so I'm a little confused why this is so difficult in python... Is there a historical reason? Is Matlab equally poor on this matter?
I have the (only slightly) longer version of this code on pastebin http://pastebin.com/grVjc007

Sorry EMS, but I actually just got another response from the matplotlib mailling list (Thanks goes out to Benjamin Root).
The code I am looking for is adjusting the savefig call to:
fig.savefig('samplefigure', bbox_extra_artists=(lgd,), bbox_inches='tight')
#Note that the bbox_extra_artists must be an iterable
This is apparently similar to calling tight_layout, but instead you allow savefig to consider extra artists in the calculation. This did in fact resize the figure box as desired.
import matplotlib.pyplot as plt
import numpy as np
plt.gcf().clear()
x = np.arange(-2*np.pi, 2*np.pi, 0.1)
fig = plt.figure(1)
ax = fig.add_subplot(111)
ax.plot(x, np.sin(x), label='Sine')
ax.plot(x, np.cos(x), label='Cosine')
ax.plot(x, np.arctan(x), label='Inverse tan')
handles, labels = ax.get_legend_handles_labels()
lgd = ax.legend(handles, labels, loc='upper center', bbox_to_anchor=(0.5,-0.1))
text = ax.text(-0.2,1.05, "Aribitrary text", transform=ax.transAxes)
ax.set_title("Trigonometry")
ax.grid('on')
fig.savefig('samplefigure', bbox_extra_artists=(lgd,text), bbox_inches='tight')
This produces:
[edit] The intent of this question was to completely avoid the use of arbitrary coordinate placements of arbitrary text as was the traditional solution to these problems. Despite this, numerous edits recently have insisted on putting these in, often in ways that led to the code raising an error. I have now fixed the issues and tidied the arbitrary text to show how these are also considered within the bbox_extra_artists algorithm.

Added: I found something that should do the trick right away, but the rest of the code below also offers an alternative.
Use the subplots_adjust() function to move the bottom of the subplot up:
fig.subplots_adjust(bottom=0.2) # <-- Change the 0.02 to work for your plot.
Then play with the offset in the legend bbox_to_anchor part of the legend command, to get the legend box where you want it. Some combination of setting the figsize and using the subplots_adjust(bottom=...) should produce a quality plot for you.
Alternative:
I simply changed the line:
fig = plt.figure(1)
to:
fig = plt.figure(num=1, figsize=(13, 13), dpi=80, facecolor='w', edgecolor='k')
and changed
lgd = ax.legend(loc=9, bbox_to_anchor=(0.5,0))
to
lgd = ax.legend(loc=9, bbox_to_anchor=(0.5,-0.02))
and it shows up fine on my screen (a 24-inch CRT monitor).
Here figsize=(M,N) sets the figure window to be M inches by N inches. Just play with this until it looks right for you. Convert it to a more scalable image format and use GIMP to edit if necessary, or just crop with the LaTeX viewport option when including graphics.

Here is another, very manual solution. You can define the size of the axis and paddings are considered accordingly (including legend and tickmarks). Hope it is of use to somebody.
Example (axes size are the same!):
Code:
#==================================================
# Plot table
colmap = [(0,0,1) #blue
,(1,0,0) #red
,(0,1,0) #green
,(1,1,0) #yellow
,(1,0,1) #magenta
,(1,0.5,0.5) #pink
,(0.5,0.5,0.5) #gray
,(0.5,0,0) #brown
,(1,0.5,0) #orange
]
import matplotlib.pyplot as plt
import numpy as np
import collections
df = collections.OrderedDict()
df['labels'] = ['GWP100a\n[kgCO2eq]\n\nasedf\nasdf\nadfs','human\n[pts]','ressource\n[pts]']
df['all-petroleum long name'] = [3,5,2]
df['all-electric'] = [5.5, 1, 3]
df['HEV'] = [3.5, 2, 1]
df['PHEV'] = [3.5, 2, 1]
numLabels = len(df.values()[0])
numItems = len(df)-1
posX = np.arange(numLabels)+1
width = 1.0/(numItems+1)
fig = plt.figure(figsize=(2,2))
ax = fig.add_subplot(111)
for iiItem in range(1,numItems+1):
ax.bar(posX+(iiItem-1)*width, df.values()[iiItem], width, color=colmap[iiItem-1], label=df.keys()[iiItem])
ax.set(xticks=posX+width*(0.5*numItems), xticklabels=df['labels'])
#--------------------------------------------------
# Change padding and margins, insert legend
fig.tight_layout() #tight margins
leg = ax.legend(loc='upper left', bbox_to_anchor=(1.02, 1), borderaxespad=0)
plt.draw() #to know size of legend
padLeft = ax.get_position().x0 * fig.get_size_inches()[0]
padBottom = ax.get_position().y0 * fig.get_size_inches()[1]
padTop = ( 1 - ax.get_position().y0 - ax.get_position().height ) * fig.get_size_inches()[1]
padRight = ( 1 - ax.get_position().x0 - ax.get_position().width ) * fig.get_size_inches()[0]
dpi = fig.get_dpi()
padLegend = ax.get_legend().get_frame().get_width() / dpi
widthAx = 3 #inches
heightAx = 3 #inches
widthTot = widthAx+padLeft+padRight+padLegend
heightTot = heightAx+padTop+padBottom
# resize ipython window (optional)
posScreenX = 1366/2-10 #pixel
posScreenY = 0 #pixel
canvasPadding = 6 #pixel
canvasBottom = 40 #pixel
ipythonWindowSize = '{0}x{1}+{2}+{3}'.format(int(round(widthTot*dpi))+2*canvasPadding
,int(round(heightTot*dpi))+2*canvasPadding+canvasBottom
,posScreenX,posScreenY)
fig.canvas._tkcanvas.master.geometry(ipythonWindowSize)
plt.draw() #to resize ipython window. Has to be done BEFORE figure resizing!
# set figure size and ax position
fig.set_size_inches(widthTot,heightTot)
ax.set_position([padLeft/widthTot, padBottom/heightTot, widthAx/widthTot, heightAx/heightTot])
plt.draw()
plt.show()
#--------------------------------------------------
#==================================================

I tried a very simple way, just make the figure a bit wider:
fig, ax = plt.subplots(1, 1, figsize=(a, b))
adjust a and b to a proper value such that the legend is included in the figure