Plotting figures using matplotlib, over ssh using X11 and OS X - python

Summary: How do I plot figures, over SSH, to a remote computer when the local computer is running OS X?
I have computer A and I am trying to use matplotlib to plot on computer B. The problem I am having is that when I use matplotlib the plots only display on Computer A. I VNC in and watch them pop up. I can ssh -X/-Y into Computer A and run xcalc and it will display on computer B. I can connect Computer B to a third computer, running Red Hat, and plots will display on Computer B. I am convinced it is not Computer B that is the problem. I believe my problem is the same as this problem: none of the package installers support X11 backends for matplotlib. I cannot comment so I'm stuck putting what I've tried in a new question. This is another description of the same problem with multiple solution attempts that do not work.
As mentioned, I have tried a lot of solutions in terms of installing backends for matplotlib on Computer A. I've tried all manor of macports and homebrew and pip combinations. I'm pretty sure it is a bad idea to mix so many package handlers, but so many solutions seem to be "sudo ***** install package-name".
To test whether matplotlib is doing what I want I use the following python snippet:
import matplotlib
matplotlib.use('gtk') # gtk is an example, I change it
import matplotlib.pyplot as plt
plt.ion()
plt.figure(1)
plt.plot([1]*10)
I can't recall all the things I have tried. Some things I have tried:
I tried using GTK and GTKCairo, which did not solve my problem because I cannot get GTK to work. Homebrew GTK does not support X11 anymore anyway, so even if it did install properly I don't think it will solve my problem. I have not yet tried to install GTK some other way. Would I have to install it from source? Has anyone got this working?
The GTK error:
ImportError: No module named _backend_gdk
Backends MacOS, TkAgg, qt5agg all work but figures display on Computer A. I had to install pyqt5. If I am not connected via VNC, then python will thrown an error about no displays. All three give the same error:
Feb 22 13:00:22 Python[57649] <Error>: Set a breakpoint at CGSLogError to catch errors as they are logged.
Feb 22 13:00:22 Python[57649] <Error>: This user is not allowed access to the window system right now.
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
This does not solve my problem. This doesn't either.
Details:
Computer A runs OS X 10.11
Computer B runs OS X 10.12
Edit:
I installed PyQt4 from source (along with QT) and it didn't help. I explicitly downloaded and installed the X11 version. I set matplotlib.use('qt4agg') and figures still appear on Computer A. Perhaps it was my install of QT for "macosx"? I dunno

This ended up leading me to the right answer. The problem was I couldn't get a backend installed that used X11. It turns out macports has a tk version that uses x11. I think this is actually the default setting when installing python using macports.
The steps I took to get matplotlib plotting over ssh/X11 from an OS X server (Computer A) were:
1) Uninstall the previous macports install of matplotlib and tk:
sudo port uninstall py27-matplotlib
sudo port uninstall py27-tkinter
sudo port uninstall tk
2) Re-install matplotlib to use X11
sudo port install tk
sudo port install py27-tkinter
sudo port install py27-matplotlib -cairo +tkinter
The "-cairo" may not be necessary. tk has no flags because x11 is default.
3) Test python+matplotlib:
import matplotlib
matplotlib.use('tkagg') # set the backend to tk, using agg renderer
import matplotlib.pyplot as plt
plt.ion()
plt.figure(1)
plt.plot([1]*10)
This opens a figure on Computer B.
Some notes:
1) Make sure there isn't anything in your $PYTHONPATH environment variable that points to a python installation performed by another installer, like homebrew or the system python. It might cause the wrong install of matplotlib to load.
2) You can use a matplotlibrc configuration file to avoid having to re-specify the backend every time you open matplotlib.
3) As noted in my original post, Homebrew does not support X11 anymore for GTK. There is, apparently a way to get homebrew to use tk with X11, but I haven't figured out how to make it work yet.

Related

Python3 plot not showing in WSL

I'm pretty new to Python. I have a simple script that shows a simple plot on the console. It works perfectly when I execute the script in VSCode, and the plots shows up as expected. However, when running the same script in Python in WSL, the plot doesn't show up, and I also don't get any error messages.
I'm on Windows 10.
Here is my code snippet that I typed directly in Python under WSL:
import matplotlib.pyplot as plt
x = [1,2,3,4,5]
y = [x**2 for x in x]
plt.plot(x,y)
[<matplotlib.lines.Line2D object at 0x7f57445be700>] # I get this message and I don't know what this means.
plt.show() # No error message, but the plot doesn't show up in my console.
Again, the plot shows up when I execute the script in VSCode.
How can I display the plot under WSL?
Since you say that your MRE works in Visual Studio Code, I'm guessing:
You are using the Windows version of Python in VSCode
Or you are using Jupyter Notebooks there.
Edit: I just noticed your comment:
It's interesting because when I run the same code in regular command prompt (not bash), the plot shows up
So yes, it appears that you are also running the Windows version of Python. You can use WSL inside of VSCode, but that's not what you doing in this case, it seems.
Under WSL, there may be several things you need to do, depending on your Windows version:
First, Windows 11 is recommended here, since it includes built-in support for displaying Linux GUI applications.
Older versions of WSL on Windows 10 do not support this directly. On Windows 10, you'll need to either install a third-party X server or use XRDP. The easiest way to do this (other than Windows 11) is with Xfce4 + XRDP, but it's definitely the slowest method as well. Also see my answer on What's the easiest way to run GUI apps on Windows Subsystem for Linux?.
Since you are on Windows 10, you'll have to get GUI support working first on WSL. Test with something like xterm to make sure you have it working) and then come back here.
[<matplotlib.lines.Line2D object at 0x7f57445be700>]
I get this message and I don't know what this means.
That's an easy one. When you are entering code directly into the Python REPL, it will always show the value/return-result for each line. All of the other lines that you typed up to that point either had no return result or you stored the return value in a variable. In the doc, you'll notice that the return value from plot is a "list of Line2D" objects.
You could suppress the message by simply storing the result in a variable, but there's no need for this in the REPL.
plt.show()
No error message, but the plot doesn't show up in my console.
Hmmm. When I try that in a base Ubuntu 22.04 distribution under WSL, I do get an error message:
<stdin>:1: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
Since you aren't receiving an error, it appears (from the doc) that Matplotlib is detecting some GUI environment and auto-selecting a higher-precedent backend because of this:
Without a backend explicitly set, Matplotlib automatically detects a usable backend based on what is available on your system and on whether a GUI event loop is already running. The first usable backend in the following list is selected: MacOSX, QtAgg, GTK4Agg, Gtk3Agg, TkAgg, WxAgg, Agg. The last, Agg, is a non-interactive backend that can only write to files. It is used on Linux, if Matplotlib cannot connect to either an X display or a Wayland display.
But I don't know why you aren't getting an error (but also no display) on that backend. It might be useful to run plt.get_backend() to see which backend Matplotlib has auto-selected. For me, on a freshly installed Ubuntu, that's agg, but it sounds like that's going to be different for you.
Regardless, that brings us to the next likely step -- Make sure that all dependencies for a Matplotlib GUI backend are enabled. This includes:
The Linux libraries needed for the selected backend.
The Python modules needed for the selected backend.
On a typical Ubuntu installation, the Linux libraries would already be installed, since Linux desktop installations are running GUI's. However, the Ubuntu/WSL distributions are based on Ubuntu Server, and don't include any graphical apps out-of-the-box.
Assuming you want to use the Qt5 backend, which is the highest priority other than Mac in the Matplotlib auto-select list, we'll need to install those libraries:
sudo apt install libqt5gui5
Then, as per this answer, you'll need the Python support for Qt5:
# Don't do this yet. Read next step.
pip install pyqt5
Note: It's recommended that you do this in a virtual environment (venv) so that you don't "pollute" your system Python installation with unnecessary libraries. Since you are new to Python, I'm going to recommend that you read-up on Python virtual environments, but here's the summary to get started:
sudo apt install python3-venv
cd <project_dir>
python3 -m venv .venv
source .venv/bin/activate
Your prompt should change to include (.venv) to show that the environment is active.
Then, anything you pip install will be in that virtual environment rather than your system environment:
# Now we can do this "safely"
pip install pyqt5
python3
With this in place, try your code again. Hopefully, you should now see that plt.get_backend() returns qtAgg. If not, then there's additional troubleshooting to do.
And with that in place, plt.plot() should (it does for me) display the plot directly on the screen.

Which backend for matplotlib using MacOS?

The question of using matplotlib with MacOS is a tricky one which has already been thoroughly reviewed by a number of discussions (see below). The problem is the following:
using MacOS Mojave 10.14.3
using python 3.7.2 in a conda environment
using matplotlib 3.0.3
Here is the simplest code snippet I came up with which allows reproducing the issue:
from matplotlib import pyplot as plt
x = [1, 2, 3]
y = [1, 2, 3]
plt.plot(x, y)
plt.show()
This throws the following error:
2019-03-22 12:25:43.429 python3.7[22209:554135] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7f85866b9de0
2019-03-22 12:25:43.431 python3.7[22209:554135] \*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7f85866b9de0'
*** First throw call stack:([...])
libc++abi.dylib: terminating with uncaught exception of type NSException
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
The issue is documented here. One solution is to install the PyQt5 package to your Python installation and to add the following lines at the beginning of your script:
import matplotlib
matplotlib.use("Qt5Agg")
While this works perfectly well, I am wondering why other backends fail to provide similar behavior.
Indeed I tried using MacOSX backend :
import matplotlib
matplotlib.use('MACOSX')
Which yields to the error:
from matplotlib.backends import _macosx
ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.
The issue is documented here, there and in plenty of other threads.
Two solutions came out :
install python.app(conda install python.app) and launch your script with pythonw instead of python
use TKAggbackend
Using the first one works well but I wonder:
why do we need to call pythonw instead of python ?
what exactly is the python.app package ?
how can we make this solution work using an IDE (let say PyCharm for instance) ?
As for the second one, it does "work" up to a certain point: when running matplotlib using TkAgg, the plot window is really buggy. Indeed, it often needs several clicks on the "zoom", "pan" or "home" buttons to get them to actually work. It really is a great pain to use it. I asked several colleagues or friends using matplotlib with TkAgg and they all have the same issue.
Does anyone know the reason for this state of fact? Or if there is any workaround to avoid this issue (apart from installing pyqt5)?
Using the first option is your best bet since you are already working with a virtual environment.
According to matplotlib, there are two variants of python:
A Framework build - Quite important for GUI stuffs in MacOXs
A regular build.
Matplotlib in this case would want to interact natively with OSX and for this, it needs the Framework build this is the reason why installing the python.app type of python is important.
More information can be gotten from Matplotlib FAQ.
Check this link for more about the need for a framework build python.
I'm going to make some assumptions. If they're wrong I apologize.
You installed Python with Anaconda.
Personally, I've never had any problems on mac with matplotlib. My setup is: Mojave, Python3.7.3 in a venv using the python built in module (python3 -m venv), and matplotlib 3.0.3.
I can't answer your question on how to fix your problem, but I'm kind of trying to answer your "is there any workaround" question. Personally, I've always had issues with using Anaconda/Spyder/Conda for Python. I've always felt installing it as its own binary/app on the system leads to the fewest errors.
Now, I'm not saying you have to download and install by hand though. I use homebrew and it saves me headaches everyday I assume (such as upgrading applications and packages). That's the "work around" I'd suggest. Because isn't installing via Anaconda/Spyder already a workaround to installing Python properly? I've always felt performing one work around requires more workarounds for full functionality. Such as having to specify the matplotlib backend when it should be detected by default.
Obviously, I'm a little biased against that tool and that may be reflected in this answer, so take it with a grain of salt. Even though Conda is a legitimate tool that I think is useful, I find it annoying having to use both pip and conda when conda doesn't contain the packages I want.
There are two things you can try.
You can read Matplotlib info page on that, https://matplotlib.org/3.1.0/faq/osx_framework.html,
The default python provided in (Ana)conda is not a framework build. However, a framework build can easily be installed, both in the main environment and in conda envs: install python.app (conda install python.app) and use pythonw rather than python.
And follow the instructions.
Or simply follow the error message you get when you try %matplotlib inline,
(...)
UnknownBackend: No event loop integration for 'inline'. Supported event loops are: qt, qt4, qt5, gtk, gtk2, gtk3, tk, wx, pyglet, glut, osx
I did %matplotlib osx and have plt.imshow(myimg) working just fine afterwards.

matplotlib does not import PyQt4, PyQt5 or PySide

I'm using Canopy 2.1.3 distribution (on Ubuntu 16.04) which as you may know contains the matplotlib package. Simply running in the Canopy's shell import matplotlib.pyplot as plt gives me
ImportError: Matplotlib qt-based backends require an external PyQt4, PyQt5,
or PySide package to be installed, but it was not found.
I have installed all of the three packages with Synaptic but the problem persists.
Could this be related to environment variables? Or, could it be because Canopy's Python 3.5 and Ubuntu's Python 2.7 are conflicting?
EDIT:
I have finally used the PyQt by installing it from the Canopy's package manager.
It is likely that the default backend for matplotlib.pyplot is set to qt.
I have very little experience with canopy, but it should be possible to change the default to another back end; maybe someone else will be able to tell you how, or you can find it by yourself.
In the meantime, you can try to add this above all other matplotlib imports:
import matplotlib
matplotlib.use('TkAgg')
From the user guide:
Known Issue: In Canopy running Python 3, GUI backend must be explicitly set
If you ask matplotlib to display a plot, or otherwise run Python code that creates a GUI window, by default you will get an error traceback ending with: ImportError: No module named 'PyQt4'
The solutions are to install PyQt (which is GPL-licensed) in the Canopy Package Manager, or to use the Canopy Preference menu’s Python tab to specify the TK PyLab backend for creating GUIs. For details, see “Python 3 in Canopy 2 - ‘No module named PyQt4’ error”

PyQt5: How to make pyqtgraph work on Debian OS?

I have already installed PyQt5 on my respberry pi 3, and I need pyqtgraph to do live plotting. I have tried to install pyqtgraph using either pip install, or download debian version of pyqtgraph from its official websites, but it just doesn't work. Actually, it is successfully installed, but when I run my program, it just automatically quit without any error message. Also when I comment out the lines related to pyqtgraph, everything goes well. Can someone explain what is wrong with debian/linux version of pyqtgraph?

plot window doesn't show - Tkinter present but header files are not found.

I have an application in python2.7 which runs fine.
Because i wanted to run it on a different laptop,i installed the matplotlib1.1 .
The problem is that the application finishes but no plot window exists.
Problem with Gtk or Tk? I tried to do in the matplotlibrc file "backend:TkAgg" or "GtkAgg" but still the same.
I forgot to mention that the laptop runs linux.
----------EDIT-------------------
I tried the installation again and it shows a message:
You may need to define display for tk to work so that setup can
determine where your libraries are located.Tkinter present but header
files are not found.
Some part of code:
import matplotlib.pyplot as plt
import scipy as sc
from matplotlib.animation import ArtistAnimation
...........
fig=plt.gcf()
ani=ArtistAnimation(fig,result,interval=10,repeat=False)
plt.show()
Even with TkAgg, matplotlib does not necessarily show the plot until you explicitly tell it to.
If you use pylab, you can explicitly cause it to show with something like this:
import pylab
pylab.plot([1,2,1,3])
pylab.show()
The call to show() will return when you close the window.
You can make it automatically show when you do a plot by writing
interactive: true
in your matplolibrc file. There is a system-wide config file in mpl-data/matplotlibrc in the installed packate. You have a personal config in ~/.matplotlib/matplotlibrc
Some systems seem to have interactive=true by default, others do not.
I found the answer here.
Finally!
(I have done manual install because it wanted the version 1.1)
I tried this :
sudo apt-get install python-pip
sudo pip install matplotlib

Categories

Resources