I am using Jupyter (with IPython) to analyze research data, as well as export figures. I really like the notebook approach offered by Jupyter: when I revisit an experiment after a long time, I can easily see how the figures correspond to the data. This is of course using the inline backend.
However, when I want to explore new data, I prefer to use the QT backend. It is faster than the inline one, and allows to easily scale, zoom in and out, and nicely displays the X and Y coordinates in the bottom left corner. Moreover, I can use the QT backend to determine good x and y limits to use in the inline backend.
I have tried using the %matplotlib notebook magic, but it is simply too slow. For some experiments I am plotting ~500 spectra (each consists of ~1000 data points), which is already slow in the inline backend. Even with less data points, the notebook backend is just too slow to use.
Therefore, I would like to use both the QT backend, and the inline backend whenever I plot something. (So, whenever I execute a cell which plots data, it should both display the inline image, and pop up a QT backend window). This way, I still have a nice overview of plots in my notebook, while also allowing me to easily explore my data. Is there a way to achieve this?
As far as I know it is not possible to use more than one backend at once.
What I do is to switch backend and replot. I have done it with matplotlib.use() or %matplotlib <backend>, according to which version or environment I was working in.
I had the impression that this sometimes doesn't work or maybe it is not responsive, so I need to restart the environment. I imagine it is not easy to maintain it, so I am even surprised it works so smoothly almost always.
Available backends can be listed with:
matplotlib.rcsetup.interactive_bk
matplotlib.rcsetup.non_interactive_bk
matplotlib.rcsetup.all_backends
Hopefully here there are other useful info:
https://matplotlib.org/stable/tutorials/introductory/usage.html
This allows you to run QtConsole, plotting with the plotSin function, both inline and through the QtConsole.
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
..
def plotChirp(Type, Exp, Rand):
# Orignal Chirp Funciton From:
# http://stackoverflow.com/questions/19410042/how-to-make-ipython-notebook-matplotlib-plot-inline
x = np.linspace(0, 3*np.pi, Rand)
plt.plot(x, np.sin(x**int(Exp)))
plt.title('A simple chirp ' + Type)
plt.show()
..
plotChirp("A", 5, 200) # Plots inline if you choose
%connect_info # For your own connection
%qtconsole
QtConsole opens and now you can call your function to plot externally..
Using %matplotlib qt allows for printing in a loop, but unfortunately it seems to overlap the plots. Looking into subplots as a possible solution.
%matplotlib qt
for i in range(0,2):
if i == 0:
plotChirp("B",1, 400)
else:
plotChirp("c",6, 1000)
Related
Have a look at the code in the following image:
Question:
Since I am not resetting the plt variable, how did I get different limits of X and Y dimensions in the 2 different code cells of Google Colab?
matplotlib.pyplot is not a variable, it's a module, that encloses functions from inside matplotlib for ease of use.
the behavior depends on the backend used by matplotlib, when running in an ipy notebook, matplotlib uses an inline backend.
the inline backend automatically calls plt.show() on every code block, thus releasing the last figure, and a new figure is created in each code block.
Introduction and examples.
It is a widely known
best practice to
close matplotlib figures after opening them
to avoid consuming too much memory.
In a standalone Python script, for example,
I might do something like this:
fig, ax = plt.subplots()
ax.plot(x, y1);
plt.show() # stop here and wait for user to finish
plt.close(fig)
Similarly, for a Jupyter notebook
it's a common pattern to create a figure in one cell
and then close it in the next cell.
A minimal example might look something like this,
where each blank line is a new cell:
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 100)
y1 = np.sin(x)
fig, ax = plt.subplots()
ax.plot(x, y1);
plt.close(fig)
However, this has a distinct disadvantage:
the figure is blank when run with "Kernel" -> "Restart & Run All"
or "Cell" -> "Run All".
That is, the figure doesn't display until all cells finish evaluating,
but the figure doesn't have time to render
since we are using plt.close right afterward.
Here's an example screenshot based on the example above:
This is different from running each cell individually:
as long as I run each cell slowly enough,
I can get each figure to display:
Full Jupyter notebook .ipynb files are available here:
https://github.com/nbeaver/jupyter-figure-rendering-tests
Inadequate workarounds.
The simplest workaround is start from the top
and manually step through each cell,
wait for it to render, and then move onto the next cell.
I don't consider this an acceptable workaround,
as it becomes impractically laborious for large notebooks
and is generally contrary to the purpose of an executable notebook environment.
Another workaround is to simply never call plt.close(fig).
This is not a good option for several reasons:
It results in excess memory usage,
as evidenced by warnings about opening too many figures.
On some machines or resource-constrained environments
this may prevent the notebook execution from completing at all.
Even in environments with abundant memory,
cursor tracking and interactive functionality like zoom or pan
becomes very slow when many figures are open at once.
As mentioned above, in general none of the figures will
display until the final cell has finished executing,
which is undesirable for notebooks
where execution time may be long for certain cells further down.
Another workaround is to use %matplotlib inline
instead of %matplotlib notebook.
This is unsatisfactory also:
The inline mode does not permit interactive inspection of the figure
such as cursor position values or pan and zoom.
This functionality may be desirable or essential for analysis.
The inline and notebook settings cannot in general be toggled on a per-cell basis,
so effectively this is an all-or-nothing setting.
Even if it were possible to toggle between inline and notebook,
I would prefer to only use notebook and then
close the figure in a subsequent cell,
so that I can return to the cell later
and re-run it to get the interactive controls
without needing to edit the cell.
An analogous workaround is to call savefig for each cell with a figure
and then browse the generated images with an external image viewing program.
While this allows limited zooming and panning,
it doesn't give cursor positions
and it's really not comparable to the interactive notebook plots.
Criteria and current workaround.
Here are my requirements:
The effect of "Restart & Run All" must render all figures eventually;
no figures can be left blank.
Use %matplotlib notebook for all cells
so that I can re-run the cell later and inspect the figure interactively.
Allow plt.close(fig) after each cell so that notebook
resources can be conserved.
Essentially, I would like a way to force the kernel
to render the current figure
before proceeding on to plt.close(fig).
My hope is that this is a well-known behavior
and that I've simply missed something.
Here's what I have tried so far that didn't help at all:
plt.show() at the end of a cell or between cells.
fig.show() at the end of a cell or between cells.
plt.ioff() at the end of a cell or between cells.
time.sleep(1) at the end of a cell or between cells.
plt.pause(1) at the end of a cell or between cells.
fig.canvas.draw_idle() at the end of a cell or between cells.
Doing from IPython.display import display and then display(fig)
at the end of a cell or between cells.
calling plt.close('all') at end of notebook instead of between each cell.
Currently, the best I've been able to do
is call fig.canvas.draw() in a separate cell
between the figure and the cell with plt.close(fig).
Using the example above, here's what this looks like:
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 100)
y1 = np.sin(x)
fig, ax = plt.subplots()
ax.plot(x, y1);
fig.canvas.draw()
plt.close(fig)
This works reliably in smaller notebooks,
and for a while I thought this had solved the problem.
However, this doesn't always work;
particularly in large notebooks with many figures,
some of them still come out blank,
suggesting that fig.canvas.draw() adds some delay
but is not always sufficient,
perhaps due to a race condition.
Since fig.canvas.draw() is not a documented method
for making Jupyter notebooks render the current figure,
I would hesitate to describe this as a bug,
although it seems to be the closely related to this matplotlib issue,
which ultimately seems to be a Jupyter bug:
The simplest work around may be to put the input() call in the next cell or to add plt.gcf().canvas.draw() above the input call. This will still result in "dead" figures (which may not have caught up to the hi-dpi settings of your browser), but they will at least show.
I've observed this behavior in many combinations of matplotlib and Jupyter,
including matplotlib version 2.1.1 and 3.5.1
and Jupyter version 4.4.0 and 6.4.8.
I've also observed it in both Google Chrome 99.0.4844.51 and Firefox 97.0.2
and on both Windows 10 and Ubuntu 18.04.6.
Related questions (not duplicates):
Programmatically Stop Interaction for specific Figure in Jupyter notebook
Specify where in output matplotlib figures are rendered in jupyter using figure handles
Get Jupyter notebook to display matplotlib figures in real-time
Force matplotlib to fully render plots in an IPython/Jupyter notebook cell before advancing to next cell
I am having a really weird issue with using the %matplotlib inline code in my jupyter notebook for plotting graphs using both pyplot and the pandas plotting function.
The problem is they show up without any axes, and basically just show the graph area without anything aside from data points.
I found adding:
import matplotlib as mpl
mpl.rcParams.update(mpl.rcParamsDefault)
reverse it, but I find it odd that should do that every time as the effect disappears as soon as I run %matplotlib inlinecommand.
an example could be
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.scatter(A,A)
plt.tight_layout()
plt.xlabel('here')
plt.show()
This would generate the graph below:
Weird enough if I uses the savefig it get plotted with the axis, if I uses the right-click -> new output -> save as figure, I also get the graph with the figures !!
like this:
Can anyone help me understand what is wrong, which global setting did I mess up, and how do I revert it?
(I don't remember messing around with any settings aside from some settings for pandas, but don't think they should have had an impact)
as mentioned running mpl.rcParams.update(mpl.rcParamsDefault) command does bring it back to normal until I run %matplotlib inline` again !!
Any help would be much appreciated.
Okay I am sorry I think I can answer the question myself now.
With the helpfull #Mr. T asking for the imgur link made me realize what was going on. I had starting using the dark jupyter lab theme, and the graph would generate plots with transparent background, ie. the text and lines where there, but I just couldn't see them.
The trick is to change the background color preferably globally, but that will be a task for tomorrow.
I'm using getdist to plot some simulation results.
In jupyter writing just this line
g = plots.getSubplotPlotter()
g.triangle_plot([samples, samples2], filled=True)
Python will show plots as we can see here.
Now if we want to write it in Python shell and run it with IDLE, this does not produce any plot. plt.show() does not work here.
How to instruct python or matplotlib to show the plots and save them?
The problem is that getdist sets the backend to Agg (in this line), which is a non-interactive backend and hence cannot produce an interactive figure via plt.show().
This is pretty bad style, because the user should select the backend, not the package. You might want to inform the developpers about this design flaw.
Anyways it would be possible to switch the backend after importing getdist, via plt.switch_backend(..). As backend you would need to use any interactive backend you have available, e.g. "Qt5Agg" or "TkAgg".
import numpy as np
from getdist import plots, MCSamples
import matplotlib.pyplot as plt
plt.switch_backend("Qt5Agg")
# .. some code ..
g = plots.getSubplotPlotter()
g.triangle_plot([samples, samples2], filled=True)
plt.show()
I love using Jupyter notebooks, but can't seem to find the correct backend for visualizing plots: %matplotlib inline generates really low-resolution, bitmap images, but fast, and %matplotlib nbagg or %matplotlib notebook are slow, but high-resolution vector graphics.
Could the latter be slow because it has to set up the interaction interface? I generally want all my figures to be reproducible with the click of a button so I don't want to manually manipulate them -- maybe I can disable interaction for every plot?
Or could the former be adjusted to show figures in higher-resolution, or to load up vector graphics instead of the bitmap images?
As often happens, I found the answer right after posting the question. Just use
%config InlineBackend.figure_format = 'retina'
%matplotlib inline
for high-resolution bitmap, or for vector graphics,
%config InlineBackend.figure_format = 'svg'
%matplotlib inline
and that does the trick!
Also figured out how to make the backend preserve the printed-figure bounding box even if Artists go outside -- just use
%config InlineBackend.print_figure_kwargs = {}
because by default, its value is {'bbox_inches': 'tight'}. Could not find any documentation on this, but jupyter displays the different options if you enter %config InlineBackend.