I would like to create some graphics with Python that matplotlib seems to be unable to do well, in terms of drawing closed curves on 3D surfaces. A bit of online research led me to mayavi.mlab for that task (Matplotlib: Plot 3D curve on top of 3D surface). I am currently running Python 3.7.11, but I can upgrade to 3.9 or higher if necessary. I have tried multiple ways to install mayavi and all of them fail in one way or another, including the answers to some of the related questions here. Here are my symptoms:
With Anaconda
I tried both conda install mayavi and conda install -c conda-forge mayavi from the command line. Both of them have the same problem, which is being unable to "Solve the environment" with "frozen solve". Then it tries a flexible solve and still fails. Then it tries a different repo and again is unable to solve the environment with frozen solve.
I also tried Anaconda Navigator for the same thing, but it hangs during a step called "Solving package dependencies" for about six hours, at which point I stopped it.
With pip
Another website told me that pip wheels are more reliable for installing mayavi, but that you should pip install vtk first before mayavi. So I did that: pip install vtk and then pip install mayavi. Both seemed to work during install, in the sense that there were no warnings or errors that I could tell. But as soon as I try to import mayavi in a jupyter notebook I get this error:
This looks to my novice eyes like a problem with a Qt interface of some kind, but beyond that I don't know what is going on.
Summary
I'm obviously not an expert at Python but I don't know what else to try. Usually conda seems to be able to figure out package dependencies and install what is needed but not this time. I would be happy for any advice, and I apologize in advance if I have left out some important information. Is the problem due to using the anaconda3 version of python 3.7 but installing mayavi with pip? I would prefer to install mayavi with anaconda anyway so I would appreciate any pointers about how to do that.
Installing VTK itself (not just python pip package of vtk) is also a prerequisite for Mayavi.
(MacOS only:)
The following steps install both VTK and Mayavi:
Prerequisites:
brew install llvm. which requires manual installation steps in brew info llvm.
pip install wheel # optional. I did it before other steps.
Steps:
brew install vtk # very heavy
pip install vtk
pip install mayavi (not needed, but for some reason I needed it)
brew install qt5
brew info qt5 # slow
# Follow instructions of above. (Set PATH plus LDFLAGS and CPPFLAGS for compiler) and close and open the terminal.
qmake # Test qmake works on terminal (to test the PATH of qt5)
pip install pyqt5 # Slow: builds using clang.
export ETS_TOOLKIT=
export QT_API=pyqt5
brew install pyside # (I installed pyside too, at earlier steps. It might be unnecessary)
I did many other things as well but this is the steps to reproduce the effective steps. I hope I haven't missed something that I have done (or if there are unnecessary steps). Since it changes the configs, it is difficult to revert back and test from scratch.
This installs the combination of following versions: vtk#9.1, qt5.
Also clang 13.1.6 .
Which installs both VTK and Mayavi.
Note, I use venv, so the exact pip command may differ for you if you use conda.
Tested on:
Python: 3.9.13, MacOS: 12.4 Monterey M1
Can you shed some light on the interaction between the Python interpreter distributed with OS X and the one that can be installed through MacPorts?
While installing networkx and matplotlib I am having difficulties with the interaction of /usr/bin/python and /opt/local/bin/python2.7. (The latter is itself a soft pointer to /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7)
How can I be certain which Python, pip, and Python libraries I am using at any one time?
More importantly, it appears that installing matplotlib is not possible on Lion. It fails with Requires numpy 1.6 or later to build. (Found 1.5.1). If I upgrade by running sudo pip install --upgrade numpy, it does not help. Subsequently attempting to install matplotlib (sudo /usr/local/bin/pip install matplotlib) still fails with the same (Requires numpy 1.6...) message. How can I install matplotlib?
Points to keep in mind about Python
If a script foobar.py starts with #!/usr/bin/env python, then you will always get the OS X Python. That's the case even though MacPorts puts /opt/local/bin ahead of /usr/bin in your path. The reason is that MacPorts uses the name python2.7. If you want to use env and yet use MacPorts Python, you have to write #!/usr/bin/env python2.7.
If a script foobar.py starts explicitly with #!/usr/bin/python or with #!/opt/local/bin/python2.7, then the corresponding Python interpreter will be used.
What to keep in mind about pip
To install pip for /usr/bin/python, you need to run sudo /usr/bin/easy_install pip. You then call pip (which will not be installed by easy_install in /usr/bin/pip, but rather in /usr/local/bin/pip)
To install pip for /opt/local/bin/python2.7, you need to run sudo port install py27-pip. You would then call pip-2.7. You will get the pip in /opt/local/bin. Be careful, because if you type pip2.7 you will get /usr/local/bin/pip2.7 (the OS X pip).
Installing networkx and matplotlib
To install networkx for the OS X Python you would run sudo /usr/local/bin/pip install networkx. I don't know how to install matplotlib on OS X Lion. It may be that OS X has to stick to numpy 1.5.1 because it uses it internally.
To install networkx and matplotlib for MacPorts-Python, call sudo pip-2.7 install networkx and sudo pip-2.7 install matplotlib. matplotlib installs with a lot of warnings, but it passes.
May I also suggest using Continuum Analytics "anaconda" distribution. One benefit in doing so would be that you won't then need to modify he standard OS X python environment.
I'm trying to install the basic scipy stack (numpy, scipy, matplotlib, pandas, sympy, ipython, nose) into a virtualenv; currently, I'm using OSX Mountain Lion. From the installation pages for these packages, as well as various threads here and around the web, it seems that pandas, sympy, and nose can be installed easily with just pip (although some list numpy/scipy/etc. as dependencies).
However, there seems to be conflicting and kind of convoluted instructions for properly installing numpy, scipy, matplotlib, and, to a certain extent, ipython*. Installing solely with pip doesn't seem to be the proper way to install these packages; it seems that some dependencies need to be installed with homebrew, but various places list different packages to brew install before pip installing numpy/scipy/etc. Is there a comprehensive and current list of package dependencies that need to be installed with homebrew before pip installing numpy, scipy, and matplotlib?
Just as a note, I've looked at tapping homebrew/python for installing numpy, scipy, and matplotlib properly with homebrew, but I want to install into a virtualenv and I don't think I can use homebrew to do that.
Any help would be greatly appreciated; thanks in advance!
*Also, for ipython, the installation page says that pip install ipython[all] should be sufficient, but some other sources (http://www.coderstart.com/setup/python-setup.html; http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/) seem to suggest that the qt, pyqt, and zmq packages need to be brew installed first before pip installing ipython; are the homebrew installation necessary or is it fine to just pip install according to the ipython installation page?
Answering my own question here, but hopefully this is helpful; feel free to correct if there are any mistakes. The original version is a gigantic wall of text, so I've added a tl;dr to the top with just the steps to hopefully make the process more clear.
tl;dr: In terminal/bash, go into a virtualenv (if you want to install into one) and enter these commands in order. This is tested for OSX Mountain Lion.
pip install numpy
brew install gcc
pip install scipy
brew install freetype
pip install matplotlib
pip install nose
pip install pandas
pip install sympy
pip install ipython[all]
brew install pyqt
brew install qt
brew install sip
echo "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH" >> ~/.bash_profile
source ~/.bash_profile
*Notes: brew installing pyqt might already install qt and sip; if it does, no need to install qt and sip after installing pyqt. For the second to last line, might be be more reliable to directly c/p that line into ~/.bash_profile, since it might need to be at the top of the ~/.bash_profile contents. Also, brew install pandoc is optional, but necessary for ipython notebook's nbconvert command work properly.
edit 10/13/14 [see edit at bottom]: editing PYTHONPATH in ~/.bash_profile forces virtual environments to inherit global packages; if you want to be able to make isolated environments, don't do the last two steps. Instead, assuming virtualenvwrapper is installed, edit the local postactivate and predeactivate scripts in the bin directory under the virtualenv that contains the scipy stuff.
In postactivate, enter:
export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
In predeactivate, enter:
unset PYTHONPATH
This should edit PYTHONPATH when when the virtualenv with scipy stuff is activated so that ipython qtconsole works, but should reset PYTHONPATH when the virtualenv is deactivated, so that other virtualenvs are not affected by PYTHONPATH changes.
Below is the long narrative version.
Anyway, from some trial and error after I originally posted the question, I found that the following steps work; I used the two sets of instructions that I linked above as general guides, and this was tested on OSX Mountain Lion.
After activating the virtualenv in which the packages are going to be installed, first pip install numpy; this should work as expected and numpy should be installed (note that numpy should be installed first since a lot of the other packages in the scipy stack depend on numpy).
Now, before installing scipy, several sources note that gfortran (this seems to be the most common, but I think any fortran compiler should work) needs to be installed; brew install gfortran returns an error that says that gcc now includes gfortran so the gfortran formula is deprecated. Hence, we brew install gcc (note that even though xcode command line tools, which is generally installed before homebrew, already includes gcc, its version of gcc somehow doesn't work or doesn't include gfortran). After installing gcc, pip install scipy works as expected, and scipy should be installed. Quick aside: brew install gcc installs gcc as well as a bunch of dependencies, namely cloog, gcc, gmp, isl, libmpc, mpfr. [These should all be installed into /usr/local/Cellar, which is the default install location for homebrew.]
For installing matplotlib, freetype needs to be installed first, so we brew install freetype; this should install freetype and libpng, which seems to be a freetype dependency. Afterwards, pip install matplotlib works as expected, installing matplotlib successfully. Note that matplotlib installs with mock (needed to run matplotlib test suite), pyparsing (required for mathtext support), python-dateutil (required for date axis support), six (no reason given) along with it. [These should all be installed, along with any other pip install within a virtualenv, into the site-packages directory within the virtualenv.]
Installing nose, sympy, and pandas simply require pip installing, as they don't have any dependencies that need to be brew installed. However, of these, note that at least pandas depends on numpy, so installing pandas (not sure about the others) after installing numpy is probably preferred. Also, note that pandas installs with pytz (for time zone calculations).
Installing ipython is pretty simple, but setting it up is a little more convoluted. As a heads up, ipython can be used with both a qt console and something called ipython notebook, which have various benefits. You can choose to just install ipython with pip install ipython and install optional dependencies later on as needed, but I installed all the main optional dependencies with pip install ipython[all]. This installs ipython, along with a lot of other packages dependencies (installs with backports.ssl-match-hostname (from tornado), certifi (from tornado), docutils (from sphinx), gnureadline, ipython, jinja2, markupsafe (from jinja2), numpydoc (from ipython[all]), pygments, pyzmq, sphinx, tornado). This should be a good foundation for ipython to use both the standard ipython shell, the qt console, and the ipython notebook. However, it's not fully set up if you want to use either the qt console or the notebook.
To use the qt console, the pyqt, qt, and sip packages must be brew installed, as these are dependencies that cannot be installed with pip; from experience, brew install pyqt seems to install all three packages, but individually installing the three might be a safer bet. After doing this, go into ~/.bash_profile and add the line 'export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH' to it; then 'source ~/.bash_profile' in terminal to reload the shell. This should successfully allow the qt console to be launched. [I'm not totally sure why this line needs to be added, since I had already edited the PATH variable to put /usr/local/bin before /usr/bin, but perhaps qt/pyqt/sip were still trying to link themselves with the system default python instead of homebrew installed python.]
The notebook seems to work fine out of the box as far as I've seen, but there's one thing of note: in order to use nbconvert (converting the notebook to different file formats), the pandoc package must be installed, presumably with homebrew. Like qt/pyqt/sip, it can't be installed with pip which is why it wasn't installed with pip install ipython[all].
edit 10/13/14: So apparently, editing PYTHONPATH will nullify empty virtual environments, making global packages also available in a virtualenv (how to isolate virtualenv from local dist-packages?); this, for the most part, defeats the purpose of a virtualenv, assuming that you want to a fresh environment, but is necessary for ipython qtconsole to work correctly.
The fix is to edit (assuming virtualenvwrapper is installed) the local postactivate and predeactivate scripts in the bin file of your virtualenv. In postactivate, enter the line "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH; in predeactivate, enter the line "reset PYTHONPATH". Don't do the last two steps of the original sequence, or delete the line that was added to ~/bash_profile. This should make it so that the changes to PYTHONPATH are made only when the virtualenv with our installed packages is activated, so that qtconsole works, but are reset before the virtualenv deactivates, so that other environments are not affected.
I'm on OSX 10.9, but I had good luck following these instructions and get roughly the same environment that you outlined above
http://hackercodex.com/guide/python-development-environment-on-mac-osx/
Hope this helps
I am trying to install the Scipy python stack, but having issues.
I have a new mac os with mavericks (10.9).
I have installed homebrew and am using brew to manage installations.
I used
% brew install python
then (I verified pip is from my homebrew installation)
% pip install numpy [this works]
% pip install scipy [this works]
% pip install matplotlib OR pip install http://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.3.1/matplotlib-1.3.1.tar.gz
Both fail with the same error. After checking the backend, I get
Fatal Python error: PyThreadState_Get: no current thread
The error message from mac os X (the pop up error window) shows errors very similar to those in the previous thread
Homebrew + Python on mac os x 10.8: Fatal Python error: PyThreadState_Get: no current thread importing mapnik
It seems I need to fix the python bindings, as some of them linked against the /System/Library python that ships with the mac. What is not clear to me from the above post is
which of the Binary Images need to be "re-linked"? all of them? some seem quite general an not related to matplotlib
how do I actually use install_name_tool to fix the link? As I mentioned, some seem quite general - do I really want to change the path? Do I cp the dylib from old to new?
I have also tried
brew uninstall python
and re-brewing, all to get the same error. I am trying not to use EPD because EPD installs its own libxml2 and others, which I need for other software I will install, and this has caused me problems on an old machine. I am open to considering a virtualenv, but was hoping to learn to just fix my current problem if possible.
Thanks,
Andre
PS, when pip install matplotlib`, the following warning appears, which seems odd because matplotlib is such a staple piece of software
You are installing an externally hosted file. Future versions of pip will default to disallowing externally hosted files.
You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.
This is an issue in pygtk. You can test it with brew (it should throw the same error):
brew test pygtk
I ran into the same problem, and got past that error with the following steps from https://github.com/mxcl/homebrew/issues/13654
brew rm py2cairo
brew install py2cairo
You can test that pygtk is working using brew test pygtk again.
I'm still having other problems getting pip install matplotlib to work, so you may also run into more problems, but that should get you past that particular issue.
I had the same problem, and after some experimentation it seems that installing under Python 3 works more reliably. i.e. this installed without error:
pip3 install matplotlib
I then had to re-install everything else using pip3, and make sure I was running under python3 at execution time.
brew install freetype
it works on mac os sierra!
I'm using Python 2.7.3 in 64-bit. I installed pandas as well as matplotlib 1.1.1, both for 64-bit. Right now, none of my plots are showing. After attempting to plot from several different dataframes, I gave up in frustration and tried the following first example from http://pandas.pydata.org/pandas-docs/dev/visualization.html:
INPUT:
import matplotlib.pyplot as plt
ts = Series(randn(1000), index=date_range ('1/1/2000', periods=1000))
ts = ts.cumsum()
ts.plot()
pylab.show()
OUTPUT:
Axes(0.125,0.1;0.775x0.8)
And no plot window appeared. Other StackOverflow threads I've read suggested I might be missing DLLs. Any suggestions?
I'm not convinced this is a pandas issue at all.
Does
import matplotlib.pyplot as plt
plt.plot(range(10))
plt.show()
bring up a plot?
If not:
How did you install matplotlib? Was it from source or did you install it from a package manager/pre-built binary?
I suspect that if you run:
import matplotlib
print matplotlib.rcParams['backend']
The result will be a non-GUI backend (almost certainly "Agg"). This suggests you don't have a suitable GUI toolkit available (I personally use Tkinter which means my backend is reported as "TkAgg").
The solution to this depends on your operating system, but if you can install a GUI library (one of Tkinter, GTK, QT4, PySide, Wx) then pyplot.show() should hopefully pop up a window for you.
HTH,
I had this problem when working from within a virtualenv.
Cause
The cause of the problem is that when you pip install matplotlib, it fails to find any backends (even if they are installed on your machine), so it uses the "agg" backend, which does not make any plots, just writes files. To confirm that this is the case, go: python -c "import matplotlib; print matplotlib.get_backend()", you probably see agg.
I could however, successfully use matplotlib on the system (outside the virtualenv). I also failed to install PySide, PyQt, or get it to work for TkAgg, for various different reasons.
Solution
I eventually just made a link to my system version of matplotlib (starting from outside the venv):
...$ pip install matplotlib
...$ cd /to/my/venv/directory
...$ source venv/bin/activate
(venv) .... $ pip uninstall matplotlib
(venv) .... $ ln -s /usr/lib/pymodules/python2.7/matplotlib $VIRTUAL_ENV/lib.python*/site-packages
After that, I could use matplotlib and the plots would show. Your local version of matplotlib may be in a different place. To see where it is, go (outside of the venv, in python)
...$ python -c 'import matplotlib; matplotlib.__file__'
Try installing these libraries, it works for me:
$ sudo apt-get install tcl-dev tk-dev python-tk python3-tk